Common Mistakes in Systematic Reviews and Meta-Analyses
Systematic reviews are often undermined by avoidable mistakes in planning, searching, selection, appraisal, synthesis and reporting. Knowing them in advance, and checking your project against a reporting guideline, makes your review more trustworthy.
Frequent mistakes and fixes
| Stage | Mistake | Fix |
|---|---|---|
| Planning | No protocol or registration | Write and register a protocol first |
| Question | Too broad or vague | Use PICO and clear eligibility |
| Search | One database; no strategy record | Search several sources and document |
| Screening | One reviewer, no calibration | Two independent reviewers with piloting |
| Extraction | Unpiloted form; no verification | Pilot and check |
| Bias | Not assessed or not used | Use RoB 2 or ROBINS-I and use the results |
| Synthesis | Pooling very different studies | Judge clinical similarity first |
| Heterogeneity | Ignored | Report, explore, and interpret |
| Reporting bias | Not considered | Search registries; assess small-study effects |
| Certainty | Not graded | Use GRADE or explain limits |
| Reporting | Incomplete methods | Follow PRISMA 2020; share data |
| Updates | Old search dates | Update before submission |
Illustrative example
Invented for teaching. A team pools 12 studies of an intervention including children, adults and older adults, with I² = 85%, and concludes that it "works". A better approach is to explore the sources of heterogeneity, present subgroups carefully, and describe the evidence as uncertain.
Checking your own review before submission
Print the PRISMA 2020 checklist and mark each item with the section where you addressed it. Ask a colleague who did not work on the review to read the methods and say whether they could repeat the search and selection. Verify that the numbers in the abstract, text, tables and flow diagram agree, and that every deviation from the protocol is described. These simple checks catch many of the problems editors and peer reviewers raise.
Common pitfalls
- Treating the review as a formality to reach a pre-set conclusion.
- Not reporting deviations from the protocol.
Key takeaways
- Most mistakes are preventable with a protocol, duplicate work and transparent reporting.
- Use PRISMA 2020 as a guide while planning, not only while writing.
References and further reading
- Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi:10.1136/bmj.n71
- Higgins JPT, Thomas J, Chandler J, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. Cochrane. Link
- PROSPERO: International prospective register of systematic reviews (Centre for Reviews and Dissemination, University of York). Link
- Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924-926. doi:10.1136/bmj.39489.470347.AD
Related reading
This page is general educational information about research methods. It is not medical, legal or regulatory advice; follow your institution's ethics board and your target journal's instructions.