Good Intentions Can't Hide Bad Science

Editor’s Note: Neuwrite is a national organization with a mission to develop novel approaches for communicating science to the public. Synapse is pleased to introduce the newly formed San Francisco chapter of Neuwrite. In this recurring column, we will feature science communication articles. For the first Neuwrite piece Alex Clemente criticizes some poor, and poorly communicated science. This piece also appeared on the NeuwriteSF blog.

It's bad when a philanthropic organization uses bad science to justify their means, but it's dangerous when bad science misdirects policy. 

I’m sure many good people work in the Bill and Melinda Gates Foundation. People with good intentions and scientific reasoning. I’m confident these groups are overlapping and good-willed scientists carrying out the mission of the Gates Foundation “to help all people lead healthy, productive lives.” I’m sure there was good will, but good reasoning was nowhere to be found when the image above was posted on their Facebook page.

This figure is trying to show that brains of malnourished children become enlarged after they start a “therapy.”

At least, that’s what I took from it. The caption on this figure is “Good nutrition matters,” leaving the message open to interpretation. I chose this interpretation because there seems to be more bright neon blue on the brain to the right.

Many who come across this picture will look at the brains, agree that malnutrition is bad, like or share the picture, and consider their quota of social media activism fulfilled. Unfortunately, at first glance, the message underlying this image is vague and problematic.

While it makes sense that if you feed malnourished children, their brains will grow, this image does not demonstrate that point. First of all, there is no scale bar. Without a scale bar to define the true size of the figure, there is no way to know whether the apparent change in brain size is real, or an illusion, due to someone manipulating the size of the picture in Powerpoint.

A more convincing way to claim that brain size has changed would be to quantify the change in brain size, instead of simply providing a qualitative picture. A claim that one is bigger than the other, solely based on a picture where the difference isn’t that obvious, is a dubious claim.

The people at the Gates Foundation would like us to conclude that the "treated" brain on the right looks more like the brain of a healthy child than does the "malnourished" brain on the left.

In any experiment, you need to prove a claim like this with direct comparisons to "controls," as scientists call them, which represent "normalcy."

For this particular experiment, the controls should be children of the same age and sex as the malnourished and treated children studied. Using these controls, scientists can assess whether the treatment is actually returning the subject "back to normal."

Since there are no appropriate control images to indicate if the nutrition therapy is actually returning the child's brain to "normal," we cannot conclude anything about the efficacy of malnutrition treatment on children's brains. For all we know, the "treated" brain on the right still might not look closer to a healthy brain than the "malnourished" brain on the left.

The more I thought about these blue brains of purportedly different sizes, the more I realized that these images weren’t even comparable because they weren’t taken at the same horizontal plane.

The holes inside the brain, the ventricles, change in size depending on what plane you’re looking at.

Since the ventricles look different in this image, I can tell that they weren’t taken at the same plane. Because the head is round, a cross section taken close to the top of the head will look smaller than one taken lower down, regardless of the size of the brain, just like the way the very top of a tennis ball will look smaller than the equator of a golf ball.

If that is the only information you have, you will wrongly conclude that the tennis ball is smaller than the golf ball. The same thing is happening with these brains.

You can’t compare the size of brain images at different planes. If these two images taken from different planes are the only available data, the conclusion they are trying to make is entirely unfounded.

Regardless, I decided to keep my cool and not battle it out in the comments sections because:

1) Saying that the Gates Foundation was spreading bogus science made me feel uncomfortable. I didn’t want to negatively impact an organization trying to do good work.

2) I didn’t want to unleash notifications from strangers in faraway lands calling me their favorite curse words.

3) The figure had a citation, so maybe it wasn’t that bogus after all.

In order for an article to be published in a respected scientific journal, a group of peers well-versed in the topic of the article must review it.

Scientist must have done this experiment and sent it to a journal to be reviewed by their peers, and the reviewers concurred that the research was believable enough to publish. Right? Nope!

Another embattled scientist in the comments section pointed out that this research was published in a "non-peer reviewed" journal.

I have come into this experience with years of training in the field of empirical research. I go to journal clubs where we take amazing work that was approved by the peer review process and sit down to find its flaws.

As scientists, we strive to leave our results up to as little interpretation as possible by using the appropriate control experiments to diminish the possible interpretations of our results. We take no fact for granted.

Most people are not trained scientists. I’m afraid some will look at this figure and just think, “Cool! These blue brains convince me!” share it on their Facebook and have more people believing in the claims advanced by these two blue brains.

Using bad science to support claims about the way malnutrition damages the brain only hurts a very worthy cause.

While sharing this particular image on Facebook won't cause a world calamity, what if this figure were instead detailing facts about the unfounded links of vaccines and autism? Or bogus research claiming that global warming doesn’t exist? And what if these fake figures weren’t on Facebook but were instead presented to decision makers?

Scientists and journalists sharing these figures have a social responsibility to share sound science and guide the public in their interpretation. We have to understand that people shouldn’t have to have PhDs to understand the research we do. But just because we tailor the information for a general public doesn’t mean we should mislead them.

Science communication has to be a thoughtful process where we explain the necessary details without overburdening the public with the specifics.

Of course, there will still be people who want to use the general public's misinformation as a means to mislead them. To avoid this, both the general public and scientists must be able to have an open dialogue.

To converse about science, the general public has to expand their scientific vocabulary and knowledge through education.

If scientists and the public engage in discussion, fewer people out there will take a CNN headline as scripture and decide not to vaccinate their children.

I hope that in the future, the majority of the population will be able to look at that figure and say “Gates Foundation, you haven’t convinced me that children need good nutrition to feed their brains; I need more.”

It will take thoughtful, dedicated work from scientists and non-scientists, teachers and researchers, to get to that point. We deserve a society where we can have that level of discourse, even in the comments section of Facebook.