“Green guilt”, a study in statistical significance

by

An article dated May 7 2008 in the on-line edition of USA Today describes the results of the second annual “green guilt” survey, conducted at the behest of the Rechargeable Battery Recycling Commission. (“Green guilt” is a reaction to the belief that one ought to live more ecologically than one does or attempts to do.) As the USA Today article puts it, people’s guilt levels are rising! And we’re recycling more! Unless you’re male, that is.

To its credit, the article does give some of the statistical details, but those details fail to support the general conclusions argued for in the article. From 2007 to 2008, the percentage of Americans feeling guilty over their non-carbon-neutrality “jumped” from 20% to 22%, although for the men in the survey, that percentage actually declined from 18% to 17%, while among women surveyed the jump was from 22% to 26%. And we don’t just feel guilt – we act on it! 89% of those surveyed participate in at least some recycling, up from 87% the previous year.

Before we get too excited about changing attitudes and growing self-awareness among the American populace, we should note that the 2008 survey “was based on phone interviews with 525 women and 481 men. The margin of error was plus or minus 3 percentage points.” (A press release on the Rechargeable Battery Recycling Commission’s website stated that the study was done at a 95% confidence level.)

What does this mean? For the overall guilt levels, the 2008 survey estimated them to be at 22%; the 3% margin of error at a 95% confidence level means that there is a 95% likelihood that the actual level of green guilt in the general US population lies between 19% and 25%. (The comparable interval for 2007 would be 17% to 23%.)

For men, the 2007 interval would have been 15% to 21%, the 2008 interval 14% to 20%. For women, the 2007 interval becomes 19% to 25%, the 2008 interval 23% to 29%.

Even in the case of female survey participants, where the jump from 2007 to 2008 was largest, it is entirely consistent with the data given that no change whatsoever in green guilt attitudes has taken place, and that the apparent change in guilt levels has been entirely driven by the inherent (predictable) variability that comes from random sampling from a population.

I have assumed the 2007 survey was based on a similar sample size and comparable margin of error (it is hard to imagine a nonprofit investing in a survey methodology that would yield a margin of error much smaller than 3%). Given that, it is entirely consistent with the data that nothing at all has changed from 2007 to 2008, or even that fewer people are recycling now than were a year ago.

As Benjamin Disraeli is famously quoted as saying, “there are three types of lies – lies, damn lies, and statistics.” One of the more egregious misuses of statistical data is to draw hasty generalizations without due regard to statistical significance.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: