Introduction

It can be tricky to read and understand science, technology, health, and the environment. Some science sources may unintentionally or intentionally twist or obscure facts. When you're fact-checking, make sure you understand technical sources, from scientists to scientific papers to datasets.

"Too often, authors overstate the significance of their work... This is probably the most common problem I saw in submitted manuscripts."

The Scholarly Kitchen

Here are some of the more common errors made in reading and interpreting science reporting:

• Correlation isn't Causation: Sometimes a dataset A seems to follow the same trend as a dataset B. This doesn't mean A is causing B or vice versa. In fact, you can graph all sorts of datasets in ways that make it look like they are connected, even though they are not.
• Absolute Risk isn't Relative Risk: Studies often report study results in terms of relative risk, which compares two test groups. The author may confuse the relative risk as absolute, or the likelihood of that event happening in any scenario.
• Statistical Significance: Statistical significance is supposed to show how likely a result is due to chance. So, the greater the significance, the more likely a result is showing a real phenomenon; the lower, the more likely you're just seeing noise. But beware, significance does not translate directly to importance.
• Sample Size: If a sample size is too small it won't reflect the larger population. If a story cites a study with a small sample size, the story should give context: What the sample size actually was and why it may not mean much more broadly.
• False Balance: Journalism ethics dictate that reporting should be balanced. In a sense: Get comments from both sides (or many sides, as appropriate). But in some cases, seeking the other side of a story actually introduces a false sense of balance - particularly in many science stories, where the majority of the evidence falls to one side.
• Mice aren't Humans: Check if study results were based on human trials. Claims on drug efficacy may be promising but remain speculative if based on animal trials only.
• Source: Just because someone is a scientist doesn't mean they're always right. It also doesn't guarantee they are trustworthy. Same goes for a scientific journal: Some are legitimate, some are not. Be sure to look at a source's potential conflicts of interest, such as funding or affiliations. Scientific studies should be peer-reviewed.

For current science news, several websites are available to assist your fact-checking. These include SciCheck located at FactCheck.org and Science Feedback at ScienceFeedback.org.

External References