Skip to Main Content

Systematic Reviews

What is a Systematic Review?

A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a research question.1 Systematic Reviews are research projects that provide new insight on a topic and are designed to minimize bias. The project creates accessible research that examines relevant literature aiding decision makers by aggregating information in a systematic way. Methodological transparency and its systematic approach and project reproducibility are key to the purposes of a systematic review.

1. Taken from Lasserson TJ, Thomas J, Higgins JPT. Chapter 1: Starting a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane, 2020. Available from www.training.cochrane.org/handbook

Standards and Terminology

All searching projects should be systematic but not all meet the standards of systematic reviews. The TMC Library provides instruction on systematic reviews according to PRISMA and Cochrane standards. This is to provide the best services and resources to researchers to increase the chances of publication, whether the researcher has access to the Basic Service or the Full Service

The TMC Library works with many students/researchers on assignments that utilize adapted systematic review methodology. The liaison librarian will distinguish between feedback on class assignments and feedback that they may have for publication, when the instructions may differ between the assignment and what is considered best practices for publishing scientific systematic reviews. 

Components of a Systematic Review

Key elements of a systematic review include:

  • A specific and well-formulated question
  • A reproducible methodology intended to avoid bias 
  • Multiple databases searched for the review's data
  • Specified and predefined inclusion and exclusion criteria 
  • Multiple reviewers of the literature 
  • Study assessments conducted in a standardized way with definitive methodology
  • Adherence to a standardized reporting guideline such as PRISMA

Systematic reviews can have an impact on the development of public health policies and on resource allocation decisions. They can inform clinical practices and implement evidence-based interventions for diseases and illnesses. Moreover, systematic reviews can compare benefits and harms of treatment options.

The systematic review process has been developed to minimize bias and ensure transparency. Methods should be adequately documented so that they can be replicated. The integrity of a systematic review is based on its transparency and reproducibility of the methods used for the review. 

There are many resources for how to conduct, organize, and publish a systematic review. This guide is by no means exhaustive, and it can provide a starting place for understanding the core of what a systematic review is and how to conduct one.

What does it take to do a systematic review?

Time: On average, systematic reviews can require 18 months of preparation

A team: A systematic review can't be done alone! Systematic Review Methodology requires a team. You need to work with subject experts to clarify issues related to the topic; librarians to develop comprehensive search strategies and identify appropriate databases; reviewers to screen abstracts and read the full text; a statistician who can assist with data analysis; and a project leader to coordinate the team and movement of data.

A clearly defined question: Clarify the key question of you systematic review and the rationale for the review. Use a framework (e.g. the PICO framework) to identify key concepts of the question.

Inclusion/exclusion criteria: Define the terms of the investigation. These include the pre-defined question; study types; study-analysis criteria (i.e. criteria for reporting bias within studies); and quantitative methods to be used for any statistical analysis. 

A written protocol: The protocol should outline the study methodology. The protocol should include the rationale for the systematic review; the research question broken into PICO components; explicit inclusion/exclusion criteria; relevant known literature on the research question; preliminary search terms and databases to be used; intended data abstraction/data management tools; and other components that may be unique to register the protocol. 

A registered protocol A few recommendations are PROSPERO, an International Prospective Register of Systematic Reviews; Cochrane; and the Agency for Healthcare Research and Quality. Registering a protocol is important because it reduces duplication of effort and promotes transparency. 

Comprehensive literature searches: Identify appropriate databases and conduct comprehensive and detailed literature searches that can be documented and duplicated. Cochrane recommends that 3+ different databases be used to conduct the searches. A strategy must be developed and then translated across the multiple databases that were preselected, preferably by an information specialist.  

Citation management: You should have working knowledge of EndNote or another citation management system that will be accessible to the research team to help manage citations retrieved from literature searches.

Follow reporting guidelines: Use appropriate guidelines for reporting your review for publication.

A timeline

Time requires about 18 months of preparation.

 The suggested timeline for a Cochrane review is: 

  • Preparation of protocol 1-2 months
  • Searches for published and unpublished studies 3-8 months
  • Pilot test of eligibility criteria 2-3 months
  • Inclusion assessments 3-8 months
  • Pilot test of ‘Risk of bias’ assessment 3 months
  • Validity assessments 3-10 months
  • Pilot test of data collection 3 months
  • Data collection 3-10 months
  • Data entry 3-10 months
  • Follow up of missing information 5-11 months
  • Analysis 8-10 months
  • Preparation of review report 1-11 months
  • Keeping the review up-to-date 12 months