As any parent knows, if you offer children a cookie or a carrot, the vast majority will choose the cookie. So when it comes to improving school meals, health advocates logically want to remove the items that win out over healthier fare: nixing sugary chocolate milk, for example, so kids have to take white milk, or making sure pizza isn’t offered on the same day as a healthier entree.
But what if you could keep the treats around and instead use subtle environmental cues to nudge kids toward the right choices – of their own accord and without their conscious awareness? That’s the enticing promise of the Smarter Lunchroom Initiative, the brainchild of Dr. Brian Wansink, director of the Cornell Food and Brand Lab.
You may not know of Wansink or his lab, but you’ve likely heard about at least one of his studies in the media. As a professor of consumer behavior, Wansink has a knack for producing the kind of catchy research that’s catnip to news producers. There was the “bottomless” bowl of soup study, for example, in which subjects at far more soup when their bowl was surreptitiously refilled, or the one in which subjects happily consumed five-day-old stale popcorn just because it was offered in a bigger container. Wansink even once looked at the food depicted in Da Vinci’s The Last Supper to supposedly prove how portion sizes have increased over time.
In 2010, Wansink, along with fellow researchers David Just and Joe McKendry, shared more attention-grabbing study results with the public. In a New York Times opinion piece that included a nifty interactive graphic (shown above), Wansink and his team claimed that very subtle tweaks to a cafeteria’s design could have a huge impact on students’ choices. Just by placing a bowl of fresh fruit near the end of the line, for example, fruit sales could increase by a whopping 70 percent. By giving the vegetables appealing names (“Creamy Corn” instead of “Corn”), vegetable sales could increase by more than 25 percent, and salad sales could increase by a third just by having a cafeteria worker ask children if they wanted salad. Best of all, most of these proposed cafeteria tweaks were very low-cost – or even free – to implement.
Wansink’s data happened to appear in the Times just three months before President Obama signed the Healthy, Hunger-Free Kids Act, a new law that would greatly improve the nutritional standards for school meals around the country. But for kids who were long used to burgers and fries, these healthier school meals were likely to be a hard sell. A low-cost way to boost student acceptance of healthy food couldn’t have come at a better time, and just one month after the Times piece appeared, the U.S. Department of Agriculture (USDA) announced that it was giving Wansink and his team $1 million to “lead, coordinate and disseminate” more of this highly promising cafeteria design research.
When I interviewed Wansink here on The Lunch Tray soon after, that “dissemination” was taking the form of regular newsletters sent to members of the School Nutrition Association, each explaining a different technique to get kids to eat better in the cafeteria. Since then, the USDA has invested millions more in the program, including offering individual grants to schools to help them implement Wansink’s findings. The USDA holds Wansink’s ideas in such high esteem that his Smarter Lunchroom Initiative has even been woven into federal regulations: the new federal rule on local wellness policies specifically informs school districts that
at a minimum, FNS [the USDA’s Food and Nutrition Service] expects [districts] to review “Smarter Lunchroom” tools and strategies, which are evidence-based, simple, low-cost or no-cost changes that are shown to improve student participation in the school meals program while encouraging consumption of more whole grains, fruits, vegetables, and legumes, and decreasing plate waste.
Similarly, the USDA’s HealthierUS Schools Challenge, launched in 2010 to encourage healthier school environments, is now called “HealthierUS School Challenge: Smarter Lunchrooms,” and to qualify for bronze, silver or gold certification, schools must now demonstrate that they’re using Wansink’s techniques in at least six areas in their cafeterias.
But just how solid is the scientific support behind the Smarter Lunchroom Initiative? That question has come to the fore after a recent investigation of Wansink’s data (not specifically in the area of Smarter Lunchrooms) by three researchers — Jordan Anaya, Nick Brown and Tim van der Zee — all experts in statistics and data analysis.
What they’ve discovered raises serious red flags.
As reported in New York magazine last week (“A Popular Diet-Science Lab Has Been Publishing Really Shoddy Research“), the trio’s curiosity was first piqued by a blog post written by Wansink himself, in which he proudly described how he’d managed to churn out five published papers in a very short time, all derived from from the data obtained in a single (failed) experiment.
According to New York magazine, the team’s investigation “led to the subsequent discovery of 150 errors in just four of [Wansink’s] lab’s papers, strong signs of major problems in the lab’s other research, and a spate of questions about the quality of the work that goes on there.” In a subsequent article published yesterday on Medium, “Cornell’s Alternative Statistics,” Anaya adds that he has since “found errors in 6 more of their papers, Andrew Gelman detailed problems with their methodology, we learned the lab has been using incorrect statistics, making errors, and sweeping them under the rug for years.”
While the researchers were not focused on the Smarter Lunchroom Initiative per se, Anaya happens to choose in his Medium piece one of the studies underlying the initiative (titled “Attractive names sustain increased vegetable intake in schools”) to demonstrate the serious flaws in Wansink’s methodology and analysis. Drilling down into the data, Anaya found everything from simple math errors to reported standard deviations that were off by a factor of 100. Anaya concludes his devastating critique by saying that:
an entire post could be written just on the inappropriate methodology and statistical tests used in this paper, such as assuming independent observations when in fact the same students are having their choices recorded each day, or employing a high school student to carry out Study 2, who presumably was not blinded to the expected outcome of the study. (I can’t help but wonder if this lab also gets high school volunteers to do their stats for them.) . . . .
How many more papers with problems do we have to find before something is done? If this is the type of work Cornell endorses at a minimum I suggest we take any work that comes out of Cornell with a grain of salt.
To date, Wansink has refused to hand over his raw data for any of the critiqued studies to allow further investigation.
I’m hardly an expert in statistical analysis and will freely admit that most of Anaya’s Medium post sailed right over my head. But I’ve actually had my own serious concerns about Wansink’s study methodology ever since 2014, when I published a blog post titled, “Moms, ‘Food Fears’ and the Power of the Internet.” In that post, I described how Wansink garnered a great deal of national media attention, including an appearance on the Today Show, with a study about moms who use social media to share and stoke “irrational food fears” about various ingredients like MSG, sodium benzoate and high-fructose corn syrup. (The latter ingredient was the one featured in the study, which happened to have been funded by the Corn Refiners Association, the trade group that represents makers of high-fructose corn syrup.)
In his own YouTube video promoting the study’s findings, Wansink managed to paint a very unflattering portrait of these Nervous-Nelly, attention-seeking moms (an image bolstered by his lab’s accompanying infographic, seen to the right). He told viewers that these women “hate” junk food, that they get their news from unreliable sources and that they’re “much more likely to need social approval” than other people. But as I systematically demonstrated in that 2014 post, not one of these assertions was backed up by Wansink’s data. Indeed, his study design was so fatally flawed that even a non-scientist like myself could easily see the gaping holes.
So what’s the upshot of all this? Unless Wansink is willing to turn over his data to independent researchers for analysis, there’s just no way to know whether the lofty promises of the Smarter Lunchroom Initiative really hold up to scrutiny.
If it does turn out that his data is flawed, no harm has been done to school children by calling their carrots “X-ray Vision Carrots” or by moving fruit bowls closer to cash registers. The real harm, if any, would be the unnecessary expenditure of millions of dollars by the USDA on this effort – funds that might have been better spent elsewhere to improve school food.
I’ll of course keep you updated here of any further developments.
Do you love The Lunch Tray? ♥♥♥ Follow TLT on Facebook and Twitter! You can also subscribe to Lunch Tray posts, and be sure to download my FREE 40-page guide, “How to Get Junk Food Out of Your Child’s Classroom.”