Build Your Own Baloney Detector

A tool-kit for avoiding being fooled

Tuesday, November 23, 2010

Context Free Stats in the Wild

Here’s an example of context-free statistics that I came across a few days after my last post.

Without commenting on the underlying statistics or political conclusions (really, even if I wanted to, there’s not enough information for me to fairly do so, I think), consider what we’re told here:

For example, Marissa Mayer, known as “the face of Google,” gave $30,400 to the Democratic Congressional Campaign Committee in 2009.

OK… so what? Do we know who else she donated to? In the case of political donations, it’s certainly not uncommon for the wealthy to donate to more than one party, so knowing only that she donated to this campaign doesn’t tell us much even about this one individual. And that last word is also important: it’s just one individual. What about others?

In fact, of the top 10 contributions made by Google in 2009, only one — by CEO Eric Schmidt — was to the Republican National Committee.

On the surface, this seems damning. Maybe it is. But I can’t tell! To whom did the other nine contributions go? All Democrats? The DNC? Or was this only contribution to the RNC and the others were distributed to various candidates (possibly form both parties)? It’s entirely possible that the implied meaning of this statistic really represents the underlying data set, but I can’t tell from this stat. (It definitely has an implied meaning, though.)

It gets weird from here.

Steve Ballmer donated $5,000 to the Democratic Congressional Campaign Committee as well, but other Microsoft donations show an even split between Democrats and Republicans.

So… why single out that one contribution? (It could have been the largest or just the one from the most prominent Microsoft exec, but there’s no explanation.)

And then,

However, according to data collected by Consumer Watchdog, a consumer and advocacy group, Google has made more financial contributions to Republicans lately than to Democrats. The company has contributed 55 percent to Republicans and 45 percent to Democrats.

Wait, what? OK, just for the record, this article is contradicting is major claim by suggesting the opposite of what the headline proclaims. But I can’t even tell that, to be honest: has Google made more contributions in terms of number or dollar amounts? And are talking Google or Google employees/execs. (There’s a difference, although not necessarily from a perspective of how the money influences policy makers.)

From here, the article mostly devolves into a lot of interpretation from pundits. (Few of the claims are supported and there are a few more context-free stats, like how much Steve Jobs has given to Democrats, sans info about whether/how much he has given to Republicans.)

The sad thing is that if you follow the link to Adam Bonica’s site, he seems to have a good analysis going on. (I’m not an expert in political science, so I can’t judge terribly well. Some of the aspects of the graphs are worrisome, but we’re in very different fields with very different statistical norms.) But you can’t really tell from this story because the statistics aren’t given context.

posted by John Weiss at 19:15  

Powered by WordPress