Can You Trust the Research You’re Reading?
Written on November 5, 2014 at 7:34 pm, by Eric Cressey
Today's guest post comes from the bright minds at Examine.com, who just released their new continuing education resource, Examine Research Digest. I love their stuff, and I'm sure you will, too. -EC
The internet is one of the last true democracies.
It’s a place where anybody with the necessary tools (a computer and an internet connection) can actively shape the perception of information...even if they have no qualification to do so.
Though the democratization of information is a good thing, one would assume that certain topics like scientific research would remain steeped in their foundations, because...well...that's how they remain reliable.
Unfortunately, in efforts to keep up with the demands for new, sexy content, many writers have taken to regurgitating information with little to no understanding of its context or how it affects you: the end reader. This is one of the many ways information gets skewed.
It’s often said that misinformation is a symptom of misinterpretation. The very definition of words can mean different things to different people.
One example of this is during research when a conclusion is reported as "significant." When scientists use this term, it implies "statistical significance." What this means is that the probability of the observations being due to the intervention is much greater than simply by chance.
This is very different than the general understanding of “significant.” Think of it this way: if your deadlift goes up from 405 to 410, that could be considered statistically significant in science. Would you say "my deadlift went up significantly," though? Probably not!
Now imagine how this simple misunderstanding of a term can impact the interpretation of a study. Something that may mean very little to a researcher is taken out of context by a well meaning blogger, eventually ending up as a eye-catching headline in your Facebook timeline.
A second way that information becomes misinformation is through the process of simplification.
When scientific studies are written, they are done so to most effectively relay their findings to other scientists, facilitating future studies and discoveries on the topic in question. If you’ve ever read a research study, you know that this approach to writing hinges on the use of precise terminology and complex verbiage so that nothing gets misinterpreted.
Unfortunately, this approach is less than ideal for relaying important findings to the people who can apply it. This leaves a few options:
1. "Dumb down" the content, hoping nothing gets lost in translation.
2. Keep as-is, with the understanding that it won't be able to reach as many people as intended.
3. In the most egregious option, data gets turned into "sound bites" that are easily transmitted by traditional media outlets.
Once one or more of these things happen, all traces of relevance to the original source get lost and misinformation starts to get spread. Moreover, another equally insidious way misinformation gets spread is by shifting focus onto one study (cherry-picking) rather than the entire body of evidence.
The internet has rapidly increased the speed of the news cycle. Information that once had time to be verified has taken a backseat to "as-it-happens" tidbits on Twitter. For the media to keep up, more factually inaccurate information gets disseminated in far less time.
Now, appreciate the fact that a news organization only has so much air time or so many words to talk about a new publication, and you can see how there isn't enough time to allow an adequate in-depth analysis of past studies or how the new study fits into the overall body of evidence.
Remember the media screaming “a high-protein diet is as bad as smoking?” Or that “fish oil caused prostate cancer?” These are perfect examples of two well-intentioned studies blown way out of proportion.
This leads to the fourth and final way misinformation gets spread: the reliance on controversy to gain an audience.
Earlier this year a blog post theorizing the connection between creatine consumption and cancer took social media by storm. The writers were savvy enough to understand that a title proclaiming creatine to be harmful had far more appeal than yet another post confirming its athletic performance benefits.
This sort of thing isn’t a new occurrence, but for some strange reason, audiences never tire of it. Once an controversial article starts getting shared, a case of broken telephone comes into play, transforming once-quality research into misinformation. As an industry, this is a problem we need to address.
"Epilogue" from EC
In spite of all this misinformation, there are people still fighting the good fight - and that's why I’m a big fan of Examine.com. They wrote our most popular guest post ever (on the science of sleep). And, whenever people ask me about supplementation, I refer them to Examine.com.
To that end, for those who want to be on the cutting edge of research, and want something that counters the overwhelming amount of misinformation, I'd recommend Examine.com's fantastic new resource, the Examine Research Digest (ERD).
Before a study is presented in ERD, it's analyzed and reviewed by the researchers, then all references and claims are double-checked by a panel of editors. Subsequently, a final pass is done by a review panel of industry and academic leaders with decades of experience. Because you have a panel from different backgrounds, you know that you’re getting the complete picture, not the analysis of a single person.
Needless to say, I'm excited to take advantage of this resource personally to stay on up-to-date on some of the latest nutrition and supplementation research - and its practical applications for my clients and readers. I'd strongly encourage you to do the same, especially since it's available at a 20% off introductory price this week only. You can learn more HERE.
Sign-up Today for our FREE Newsletter and receive a four-part video series on how to deadlift!