In 1990, Canadian medical doctor Gordon Guyatt coined the term “evidence-based medicine”. It means doctors should care for patients using the best scientific evidence.
There are 3 levels of evidence:
Randomized trials reduce bias by randomly assigning patients to treatments or placebos.
A meta-analysis further reduces bias by analyzing results from many trials and reporting a relative risk or effect size. For example, a meta-analysis of 44 studies with 1.5 million people found that exercise reduced risk of cardiac death by 23% relative to not exercising. In another example, a meta-analysis of 70 studies with 4,200 people found that creativity training improved problem solving (effect size = 0.84).
Effect sizes are categorized as small (0.20), medium (0.50), or large (0.80). An effect size of 0.20 is equivalent to going from the 50th percentile to the 58th, whereas an effect size of 0.80 is going to the 79th.
About 50% of the time, journalists report results from randomized or non-randomized trials that are later disproven by meta-analyses. That’s why the scientific references at the back of this book consist of meta-analyses.
In your life, what is the quality of evidence for your actions?
99% of the public are not qualified to do their own research and arrive at reasonable conclusions. But most of us think we can.
Evidence-based medicine. (2021, March 5). Wikipedia. https://en.wikipedia.org/wiki/Evidence-based_medicine
Murad MH et al. (2016). New evidence pyramid. Evid Based Med. 21(4): 125–127.
Evidence pyramids have basic science and case series at the bottom, followed by case-control and cohort studies in the middle, then randomized controlled trials (RCTs), and finally systematic reviews and meta-analyses at the top
Ioannidis JPA. (2005). Why most published research findings are false. PLoS Med. 2(8): e124.
Meta-analysis. (2021, March 5). Wikipedia. https://en.wikipedia.org/wiki/Meta-analysis
Cheng W et al. (2018). Associations of leisure-time physical activity with cardiovascular mortality: A systematic review and meta-analysis of 44 prospective cohort studies. Eur J Prev Cardiol. 25(17): 1864–1872.
Scott G, Leritz LE, Mumford MD. (2004). The effectiveness of creativity training: A quantitative review. Creativity Research Journal. 16(4): 361–388.
McLeod SA. (2019, July 10). What does effect size tell you? Simply Psychology. https://www.simplypsychology.org/effect-size.html
Dumas-Mallet E et al. (2017). Poor replication validity of biomedical association studies reported by newspapers. PLoS One. 12(2): e0172650.
156 studies were covered by 1,561 newspaper articles and only 48.7% were confirmed by subsequent meta-analyses
For psychiatric research, 234 newspaper articles covered 35 initial studies that were later disconfirmed, but only 4 articles covered the refutation
Did you enjoy this chapter? Read the full book. Download your free copy: EPUB MOBI PDF
Copyright © by Raven Dojo Inc.
Limit of Liability/Disclaimer of Warranty