Lots of chatter, anger over Stanford organic food study

What to do if you don’t like/disagree with the findings of a scientific study? For some, it appears that the answer is to start a petition to have the study retracted, and to accuse the researchers of bias and being in the pay of nefarious industry concerns.

After days of heated reaction to a study published last week about organic foods, north of 2,900 people have signed the petition, at change.org, calling for the paper to be withdrawn.

Here are the nuts and bolts of the report by Stanford University scientists, which was published in the Annals of Internal Medicine:

The researchers pooled together studies addressing the health benefits of organic and conventionally grown foods. Nutritionwise, they didn’t find many differences in the more than 200 reports they looked at. But they did find some evidence of higher blood levels of pesticide residues among children who ate conventionally grown food, and they noted that while organic and conventionally grown food put people at equal risk for food-borne illnesses, antibiotic-resistant microbes are more commonly found among conventionally reared chicken and pork.

Insidious stuff!

Now to the change.org petition, which states in part:

“The fatally flawed Stanford study claiming that organic food is the same as conventional … failed to examine key food issues such as the use of GMOs, high-fructose corn syrup, mercury in the food supply, and countless other factors. Stanford University has also been found to have deep financial ties to Cargill, a powerful proponent of genetically engineered foods and an enemy of GMO labeling Proposition 37.”

And elsewhere: “It is essential that we make enough waves within the media to force Stanford and the mainstream media to issue a retraction.”

In the moments left before our appointment to be tarred and feathered and run out of town on a rail, a little review:

The article focused specifically on health aspects of organic food versus conventional food; in an interview, the first author said that she and the senior coauthor, both doctors, often get asked by their patients if eating organic food is healthier, so they decided to look at it.

The scientists weren’t studying genetically modified foods (though if GMO foods were in the conventional data, one might think that GMO-caused health factors would have revealed themselves in the results). And they weren’t studying high-fructose corn syrup -- they were only reviewing fruits, vegetables, eggs, grains, dairy, poultry and meat. Not processed foods.

The article, in other words, wasn’t about the entirety of everything that people think is wrong about the way our food is grown and produced today. It wasn’t even about every type of difference between organic and conventionally grown food.

And did we miss something -- or didn’t the authors actually report differences that come down in favor of organic food? Didn’t they write that antibiotic-resistant strains of microbes were more common in food from conventionally raised animals? Didn’t they say that pesticide residues were higher in children eating conventional foods? When they said that they didn’t know the health significance of the detected pesticide levels -- might that be because they didn’t know the health significance of the detected pesticide levels?

Or…. is something more sinister behind it all? The change.org petition (and various blogs have said similar things) calls one of the scientists on the research paper “a known statistical ‘liar’ for Big Tobacco companies” and says that he:

“worked with Stanford University to develop a ‘multivariate’ statistical algorithm, which is essentially a way to lie with statistics. This research ultimately became known as the ‘Dr. Ingram Olkin multivariate Logistic Risk Function’ and it was a key component in Big Tobacco’s use of anti-science to attack whistleblowers and attempt to claim cigarettes are perfectly safe.”

We asked Jan de Leeuw, distinguished professor and chair of the UCLA department of statistics, to explain this mutivariate stuff to us.

De Leeuw first said the paper was nothing remarkable, no worse than others using meta-analysis -- though that he’s not a big fan of meta-analysis because he thinks there are too many methodological problems with the technique.

He was puzzled by the uproar. He said he himself sees reasons other than nutrition to opt for organic foods, but even on the health side, “I think the conclusions are pretty good from the organic point of view,” he said. “I don’t understand why people get so worked up about it.”

Olkin, he said, “is a famous, well-known, well-respected, honorable man. The fact that some statistician did some study in 1976 for tobacco companies -- in the first place, it’s 35 or something years ago, and people can change. In the second place, it’s not that remarkable because almost every famous statistician did a study for the tobacco companies in the 1970s. It doesn’t mean they are corrupt .... tobacco companies at that particular time had a lot of money.”

We also asked De Leeuw, who is editor-in-chief of the Journal of Multivariate Analysis, whether the method was a way to “lie with statistics.”

“There are a lot of ways to lie with statistics, but it doesn’t have anything to do with multivariate algorithms,” de Leeuw said. “Multivariate algorithms are basically designed to take a large number of variables into account when doing statistical analysis. They are used everywhere -- in physical sciences, social sciences, life sciences.”

Say, for example, you’re interested in knowing whether living near a freeway increases the risk for cancer. You’d use multivariate analysis to crunch all the data about the people you’re studying -- their age, gender, socioeconomic status, ethnicity, how near to the freeway they live, whether they smoke, and many more factors besides.

Multivariate analyses are imperfect, de Leeuw said, and can come out with different answers depending on how the data are processed before and during analysis. This is why you can end up with statisticians -- both using multivariate analysis -- concluding “A” on the one hand and “not A” on the other, as is often seen in legal battles over an issue (diesel fuel emissions, say), with each side claiming through an expert that the facts are on their side. Though statisticians could deliberately massage the data to get a desired result -- i.e, lie -- results could turn out differently for all kinds of other, innocent reasons. And everyone is using some form of multivariate analysis.

The authors of the Annals paper are not commenting on the controversy, but Stanford University had this to say: “This paper was published in a reputable, peer-reviewed journal, and the researchers received no funding for the study from any outside company. We stand by the work and the study authors.”

And, they add, “Stanford Center for Health Policy [where the study was conducted] has never received research money from Cargill.”

Here are some comments about the study from some who are concerned about nutrition, food safety and sustainable food production. All managed to express any criticisms without demanding retractions or calling people liars and shills.

Marion Nestle, NYU professor and author of the Food Politics blog:

“Organics is about production methods free of certain chemical pesticides, herbicides, irradiation, GMOs, and sewage sludge in plant crops, and antibiotics and hormones in animals. This meta-analysis confirms that organic foods have much lower levels of these things. I’d call that doing exactly what it is supposed to.

But what about nutrients? I can’t think of a single reason why organics should have fewer nutrients than conventional crops, and plenty of reasons why they might have a bit more if the soils are rich enough. Plants make their own vitamins. The vitamin levels should not be expected to differ significantly. The mineral content might.

But even if organics do have higher levels of nutrients, so what? Will people eating them be healthier as a result? Just as with supplements, additional nutrients do not make healthy people healthier.”

In an email, Nestle said she thought that the study was “naïve” because of its focus on nutrition. “They didn’t seem to know much about the history or philosophy of organics and focused only on studies that compared nutrients or contaminants -- a standard nutritionism approach, as if nutrients were all that mattered. Hence: the conclusion.”

Still, she wrote, “I’m surprised at the vehemence of the response. The paper really touched a nerve.”

Sonya Lunder, senior analyst at the Environmental Working Group:

“The study confirms the message that EWG and scores of public health experts have been sending for years, that consumers who eat organic fruits and vegetables can significantly reduce pesticide concentrations in their bodies.”

Michael Pollan, author of “The Omnivore’s Dilemma” and other books, in an interview with KQED:

“This is not new research, it’s a meta-study [a review of previously conducted research], and I’ve seen the exact same data analyzed in a very different direction. A lot of it depends on how you manage your assumptions and statistical method.

“I think we’re kind of erecting a straw man and then knocking it down, the straw man being that the whole point of organic food is that it’s more nutritious. The whole point of organic food is that it’s more environmentally sustainable. That’s the stronger and easier case to make.

“It’s true the body of research around nutrition is really equivocal, and we need to do more studies on that. But the success of organic doesn’t stand or fall on that question.... [it] has a lot more to do with how the soil is managed and the exposure to pesticides, not just in the eater’s diet but to the farmworker.”

Urvashi Rangan, director, consumer safety and sustainability for Consumer Reports:

“Organic was meant as a healthier way of farming that is good for the environment -- and that has been proven true. Fewer pesticides, antibiotics, 100% organic animal feed (which cannot have poultry litter and other animal byproducts), hygiene management on the farm -- are all healthier practices for the environment and in some cases, humans too. We also see that organic farming, and sustainable agriculture practices can have health benefits for the consumer—and that, too, is demonstrated by many findings in this study. In fact, we are learning more and more about those benefits.

“However, the study asks inappropriate scientific questions about health benefits such as whether organic foods are less allergenic. That isn’t part of what organic food production even is and it isn’t surprising to learn there may not be any difference—and it is interesting that the authors found one study suggesting it might be. They also concur that while children who ate organic diets do have less pesticides in their bodies, the authors could not find a clinical benefit among the studies published—but we are unaware of studies attempting to answer this question.”

David Schardt, senior nutritionist with the Center for Science in the Public Interest:

Schardt said an interview that he was “astonished” by the outpouring of anger about the study. He assumes it’s because the media attention focused mostly on the first part of the analysis -- the nutrient content. But this is not the first such report, he said: There have been ones in the past, some that come down nutritionally in favor of organic foods and others that do not. One problem he sees with the Annals study is that about half the studies of nutrient comparisons were not done side-by-side on an experimental farm where potentially confounding factors such as soil and rainfall were controlled.

But in any case, he added, “I don’t know if many people are buying organic foods because they think it has more nutrients.” And, he said, the study “corroborated what we know from other studies” about pesticide levels and antibiotic-resistant bacteria. ”And of course, we haven’t even talked about the environmental impact of the different systems.”

“It’s a conventional scientific paper,” he said. “There is no reason to doubt the sincerity of the authors.”

And the furor? “Maybe it’s a sign that more people are aware of organic foods and are more sensitive to criticism.”