Was it unethical to hoax the world about chocolate as a weight loss 'accelerator'?

Yes, of course.

But.

Is it somehow more ethical to conduct and publicize a bad study because you honestly don't know any better? Journals and news outlets are full of small-scale, p-hacked, badly reported studies. This one is a drop in a vast ocean of sloppy science.

The full story is here: A journalist, a doctor, a statistician, and a couple of filmmakers ran and publicized a trial of chocolate on health outcomes, resulting in international headlines about how chocolate is scientifically proven to help you lose weight.

The genius of the plan is that it was "real" science, badly done. Most of the key flaws are ones that, as a journalist, I see crossing my inbox every day:

Only a handful of subjects. The paper didn't actually say how many, which would have been a major red flag to me; the expose reveals that there were fifteen. With small numbers of subjects, studies don't have enough statistical power to tell the difference between signal and noise.

P-hacking, or the green jelly bean problem: If you ask 20 questions, and accept answers that would seem to be rarer than a 1-in-20 chance, you're bound to turn up a bogus result or two. This is a Stats 101 mistake, but it happens far too often. Take this study about ACL injuries in soccer players: "Soccer players are at greatest risk for ACL injury when defending, especially when tackling the opponent in an attempt to win possession of the ball. Females are more likely to injure their ACLs when defending and are at greater risk for noncontact injuries in their left lower extremity." As far as I can tell, the authors tested 96 different combinations of conditions. (I'm not totally clear on that number, because the section describing their analysis was only two sentences long.) This was probably done naively, but it's every bit as wrong here as in the chocolate hoax.

Chocolate. Why, oh why, do people keep falling for studies that are about chocolate? And studies that are sort of ostensibly about chocolate but in fact feature a supplement or toothpaste with an extract that shares an ingredient with chocolate, but then they get published alongside stock photo of women posing seductively with chocolate bars? I wrote about the minefield of chocolate-and-health studies here on Lifehacker. Bottom line: a very little science fuels a lot of marketing and wishful thinking.

But Was It Ethical?

Technically no, but I have a hard time believing any substantial harm was done on the journalism/publication side. The millions of people who read a blurb in Shape or the Daily Mail have moved on: the latest on chocolate is that it helps alertness. (Thanks, Hershey.) And even without the recent hoax/expose study, scientists were already feeding magazines headlines about chocolate helping weight loss, like in this Women's Health article from 2013.

What about the fifteen (originally sixteen) research subjects? Author John Bohannon told Retraction Watch that there was no ethics board involved, which probably violates a variety of principles, laws, and treaties like the Declaration of Helsinki—but the subjects were told they were being filmed for a documentary, which was true. Ethical or not, if approval was required every time people were being manipulated for entertainment, reality TV would never have gotten off the ground.

Could it have been approved? Ethically, there's always a small risk to participants as a result of participating in a study—breach of privacy, for example. Ethics boards usually want the benefit from the study to outweigh the harms or risks that participants take on. Is that the case here? What's the benefit of a hoax? It's hard to say.

The Bright Side

The most hopeful part of the story is the part that hasn't been told: How many publications didn't publish news about the fake study? Bohannon says that none of the articles quoted an outside scientist, but that's not quite how a smart journalist, or editor, works. If a story has a promising headline but is based on sketchy science, you either figure this out yourself by reading the study, or (if you're still not sure), you contact the outside expert first, before reaching out to the study author. If the outsider trashes the paper, you either report it with that information, or kill the story.

This has happened to me. It's an awkward conversation when you say to your editor: "Hey, remember that story I convinced you to assign me? Three experts say it's garbage and I don't think we should publish it." (Your contract should ensure that you get a "kill fee" in this case—partial payment to account for the research you did.)

So we don't know the names of the journalists and publications that didn't bite on this story. Who knows, I may have been one. I scroll past a lot of overhyped health stories every day.

In the end, although I don't officially approve, I appreciate this hoax. Because a lot of bad science gets published, and usually the news cycle passes it by, sweeping it into the archives with all the other garbage that managed to glide to publication unquestioned. I wish every bad study came with a "Gotcha!' post afterward.