Friday, November 22, 2013

20 Tips for Analyzing Claims of a Scientific Study

Alex B. Berezow November 20, 2013

One of the big problems in science journalism is the tendency to hype scientific research. You're familiar with the routine: A new study comes out on, say, how coffee might lead to a slight increase in a particular disease. Then, plastered all over the front pages of websites and newspapers are headlines like, "Too Much Coffee Will Kill You!"…….. Poor science journalism misleads the public and policymakers. Is there a way to prevent such hype?  Yes, say three researchers in the latest issue of the journal Nature. They give 20 tips and concepts that readers should keep in mind when trying to properly analyze the claims made in a scientific paper:

1. Variation happens.
2. Measurements aren't perfect.
3. Research is often biased.
4. When it comes to sample size, bigger is better. Less is more? Please. More is more.
5. Correlation does not mean causation.
6. Beware regression to the mean.
7. Beware data extrapolation.
8. Mind the predictive value of tests.
9. Control groups are essential.
10. Experimental subjects should be randomized.."
11. Look for replications.
12. Scientists aren't perfect.
13. Mind your statistics.
14. There's a difference between "no effect" and "not significant."
15. There's a difference between "significant" and "important."
16. Results may not be generalizable.
17. People are terrible at risk perception.
18. Don't always assume independent events.
19. Beware of cherry-picked data.
20. Beware of extreme data.

No comments: