Why a Study is Like a Painting

by DavalosMcCormack on August 24, 2009

Because the way it’s framed is so important.

You may have seen or heard about that study with the screaming headlines “Organic food no healthier than conventional”. It cited what researchers were calling the world’s biggest research project into the issue, saying there are no significant nutritional differences between conventional produce and organic fruits and vegetables.

Now, for anyone who has been touting the health advantages of eating organic the news was something of a bummer. After all, why pay substantially more for an organic peach or cabbage when it’s no better for you than one sprayed with pesticides.

But, if you take a closer look at how the study was done you may not be so quick to chuck away the organics.

The study was done by researchers at the London School of Hygiene and Tropical Medicine and was paid for by the British Food Standards agency, which wanted to determine if the UK’s organic industry could claim higher health benefits for its produce.

All well and good so far, they’re all fine upstanding folks.

The researchers took 162 scientific papers that compared organic and non-organic produce, they threw out 107 of those studies that they said contained flaws, and focused on 55 papers that they said were of the highest quality.

The researchers then checked those papers to see if they found any differences inĀ  mineral and vitamin content for organic and non-organic fruits and veg. The overall conclusion was that in 20 of 23 nutritional categories there was no substantial difference between the two.

So, debate over, case decided. Thank you very much and goodnight. Right? Well, no. Not quite.

Other experts, who were not part of the study, say the way the research was framed helped predict its findings.

They say many of the 107 studies the researchers threw out were perfectly fine. When those were examined to compare organic and non-organic the difference was apparent. For example organic produce had, on average, 53 percent more beta carotenes and 38 percent more flavanoids than non-organic produce.

Another leading expert, Carlo Leifert, a professor of ecology at Newcastle University, said the study is ignoring lots of other evidence. Leifert is spearheading a four year study funded by the European Union into organic milk. Some initial results published last year found that organic milk contained 60 percent more antioxidants and beneficial fatty acids than regular milk.

Leifert says the way the researchers selected the papers, and the way they decided which ones to exclude, heavily influenced the findings they got.

One thing the FSA study did not take into account was pesticides. The researchers didn’t look at whether the cocktail of pesticides you consume when you eat non-organic food has any long-term health impacts. The researchers said a study like that was beyond their ability.

So, just as a terrific frame can make an ordinary painting look like fine art, the way you frame a study can also heavily shape the way people see it. But that doesn’t make it good science, any more than a frame can make a bad painting good art.

{ 2 comments… read them below or add one }

Linda Vear L.Ac. July 31, 2009 at 6:58 am

Thank you for exposing this. Keep up the good work!

Reply

Davalos/McCormack July 31, 2009 at 9:38 am

We got another comment from:
Daniel D. Woo
Who funded the research? Were researchers also paid consultants for other entities? What would happen if grocery stores offered little jars of pesticides so that we can sprinkle a bit on everything we eat, sort of like a flavor enhancer?

Reply

Leave a Comment

Previous post:

Next post: