The How to Search? team — content verified on the basis of Cochrane guidelines, the PRISMA standard, and documented cases from The Lancet, Vaccine, and PNAS (full list of sources in the text), current as of September 2026.
Why it matters at all
In May 2015, journalist John Bohannon published a “study” that supposedly proved that eating chocolate speeds up weight loss. The experiment was real — but deliberately designed to produce a statistically significant yet worthless result: 16 people, a dozen or so measured variables, and no correction for multiple comparisons. The story made the front pages of newspapers in more than a dozen countries as “studies show chocolate helps you lose weight.” Bohannon himself later described the whole hoax to show how easily a single, weak study turns into a media fact.
This is not an isolated case. The most notorious is the story of Andrew Wakefield, who in 1998 published an article in the prestigious The Lancet suggesting a link between the MMR vaccine and autism — on the basis of just 12 children. The article was retracted only in 2010, when the falsified data came to light and the author lost his medical license. Meanwhile, as early as 2014, a meta-analysis by Taylor and co-authors in the journal Vaccine, covering data on more than 1.2 million children from many cohort and case-control studies, found no link at all. One study and a systematic review of dozens of studies can lead to completely different conclusions — and a media headline rarely tells you which of the two you’re dealing with.
What the difference is
A single study answers a question once, under one set of conditions, in one group of participants. A systematic review doesn’t conduct a new experiment — it gathers all the available studies on a given topic, evaluates their quality according to clear criteria, and synthesizes the results. A meta-analysis goes a step further: it combines numerical data from many studies into a single, statistically weighted result. That’s why, in the hierarchy of the credibility of scientific evidence, systematic reviews and meta-analyses usually rank highest, above individual randomized trials, observational studies, and case reports.
A single study
One team, one group of participants, one moment in time. The result may be accidental, dependent on the specific sample, or on a methodological error that no one has verified yet.
A systematic review
Dozens or hundreds of studies, searched in several databases according to a pre-registered protocol, assessed for risk of bias, and synthesized into a single conclusion.
This doesn’t mean a single study is worthless — after all, every systematic review is made up of single studies. The problem arises when one preliminary, often small study is presented in the media as if it were final, established scientific evidence.
15 ways to check it
The list below contains concrete, technical tricks — from one click in PubMed to analyzing a chart in a publication — that let you determine in a few minutes what kind of scientific evidence you’re really dealing with.
Find and read the original publication, not just the press article
Media coverage almost always links to (or at least mentions) the source publication or the university/institute that issued it. Enter the authors’ names and the study title in Google Scholar or PubMed — this is the first and most important step, because all the other tips require access to the original.
Check the “Publication type” right next to the title in PubMed
Every record in PubMed has a publication type label: Randomized Controlled Trial, Observational Study, Case Reports, Review, Systematic Review, or Meta-Analysis. It’s one line of text that settles the dispute immediately — no need to read the whole content.
Use the “Article Type” filter in the search results panel
On the left side of the PubMed results (and similarly in Europe PMC), you can tick the “Systematic Review” or “Meta-Analysis” filter to immediately narrow the list to secondary research on a given topic — useful when you want to check whether such a review exists at all.
Count the number of participants (n=)
A single clinical trial usually involves anywhere from a dozen to a few hundred people. A systematic review adds up data from many studies — count in thousands or millions, as in the aforementioned meta-analysis on vaccines and autism (more than 1.2 million children). A small sample doesn’t disqualify a study, but it limits how broad the conclusions drawn from it can be.
Look for a PRISMA diagram
Reliable systematic reviews are reported according to the PRISMA standard and contain a characteristic flowchart: how many publications were found, how many were rejected at each stage, and how many were ultimately included in the analysis. The absence of such a diagram in an article that calls itself a “review” is a warning sign.
Check the protocol registration in PROSPERO
Credible systematic reviews (especially medical ones) are registered in advance in the international PROSPERO database before the authors start collecting data — the registration number usually appears in the abstract or in the methods section. This protects against cherry-picking results to fit a predetermined thesis.
Read the “Methods” section — how many databases were searched
A solid review describes a search of at least several independent databases (e.g., PubMed, Embase, Cochrane Library, Web of Science, Scopus) over a specific period. If the methods section describes only one experiment on one group of subjects, then regardless of the title, it’s not a systematic review.
Find the forest plot
The characteristic chart with a row of horizontal lines and a diamond-shaped symbol at the bottom is a “forest plot” — the visual signature of a meta-analysis. Each line is one source study, and their combined result is shown at the very bottom. If a publication has such a chart, it almost certainly combines data from many studies.
Check the I² heterogeneity statistic
In meta-analyses, it’s worth glancing at the I² statistic, which shows how much the results of the individual studies differ. Values of 0–40% usually indicate low heterogeneity, while 75–100% calls for great caution with the combined conclusion — even if it is formally a “systematic review.”
Note whether it’s a study on humans, animals, or cells
Headlines like “studies show ingredient X cures cancer” very often describe results in mice or in cell cultures in a laboratory (in vitro), not in humans. That’s still valuable science, but a completely different level of evidence than a clinical trial — check this in the methods section, not in the headline.
Distinguish a “case report” from a population study
A “case report” is a description of one or a few patients — the lowest level of evidence in the evidence-based medicine hierarchy, yet it is sometimes cited in the media as a full-fledged “study.” The words “case report” or “case series” in the title immediately set the right expectations.
Check whether it’s a preprint or a peer-reviewed article
Preprints published on servers such as medRxiv or bioRxiv haven’t yet gone through peer review — they may change or be rejected. These platforms explicitly label texts as preprints, so it’s worth checking this before you treat a result as final.
Check the journal and the study in the Retraction Watch Database
Retraction Watch maintains a publicly available database of retracted scientific publications. If a famous “study” from years ago is still circulating on social media, it’s worth checking in this database whether it has already been officially retracted — as happened with Wakefield’s article in 2010.
Check whether there’s already a Cochrane review on the topic
The Cochrane Library collects some of the most rigorous systematic reviews in medicine, which are regularly updated. Before you consider a single new press report a breakthrough, check in the Cochrane search engine whether the topic hasn’t already been studied repeatedly and systematically — the answer is often more measured than the headline.
Compare the language: “suggests” or “proves”
The authors of single, preliminary studies usually write cautiously: “the results suggest,” “may indicate an association,” “further research is needed.” When a popular article turns this into a firm “scientists have proven” or “studies clearly show,” it’s most often journalistic overinterpretation, not the conclusion of the authors themselves — it’s always worth comparing the language of the headline with the language of the original abstract.
Frequently asked questions
Not always — a well-designed, large randomized trial can be very strong evidence, and a systematic review made up of a few weak studies may give an unreliable result. The key is to check the quality of the methodology, not just the label “review” or “study.”
A narrative review describes a topic freely, without a predetermined protocol for searching and selecting studies. A systematic review uses an explicit, reproducible methodology — with a registered protocol, inclusion and exclusion criteria, and a risk-of-bias assessment — which is why it’s less susceptible to biased selection of sources.
Not quite. A systematic review is a method of collecting and evaluating studies; a meta-analysis additionally combines their numerical results statistically into a single measure. Most meta-analyses are part of a systematic review, but not every systematic review includes a meta-analysis — sometimes the studies are too different to be combined mathematically.
The fastest is PubMed — under the title of each record there is a “Publication type” label (e.g., Randomized Controlled Trial, Systematic Review, Meta-Analysis, Case Reports). It’s a single line that settles the matter in a few seconds.
Yes, it’s a valuable stage of science — many therapies start with animal models. The problem arises when the media leave out this detail and write “studies show that X cures Y in humans,” while the original study concerned only laboratory mice.
Want to get better at telling solid science from a one-day news story?
Buy the “How to Search?” guide
Concrete methods for searching and verifying information — including scientific information — step by step.
See the guide → ☕Support us on Suppi
We create these materials in our free time. Every bit of support helps us write more and more often.
Support → ▶️Watch us on YouTube
The same topics in a shorter video format — subscribe to the channel.
See the channel →More from our blog
Sources and further reading: J. Bohannon, “I Fooled Millions Into Thinking Chocolate Helps Weight Loss” (io9/Gizmodo, 2015); Wikipedia, “Fraudulent Lancet MMR vaccine–autism study”; L. Taylor et al., “Vaccines are not associated with autism: An evidence-based meta-analysis of case-control and cohort studies,” Vaccine, 2014; Cochrane Polska, “How to interpret systematic reviews?”; PRISMA guidelines (prisma-statement.org); the PROSPERO register (University of York); the Retraction Watch database.