Skip to Main Content

Systematic Reviews

Study Analysis

Evaluation is the stage where researchers will synthesize the data and summarize it. Researchers at this stage may do a meta analysis. 

After screening and for evaluating the eligible studies, researchers assess the quality of the literature. In terms of quality, the major concern is the risk of bias found in the individual studies. At the beginning of the review, bias assessment criteria should be identified within the inclusion and exclusion criteria for study evaluation.

Data Management and Software

Information to collect

There is a lot of data generated during the conduction of a systematic review. The research team must have a data management plan and software to properly coordinate research efforts. Data to collect* and synthesize includes:

  • Exhaustive search strategy
    • databases searched; subject headings and keywords used in each; search dates; and other search strategy data
  • Grey literature searches 
    • names of resources; access or retrieved dates; locations of resources consulted
  • Supplemental search strategies
    • may be unique to each systematic review
  • Experts and organizations contacted
    • Names, titles, and contact information for experts consulted; information on organizations and individuals contacted from those institutions with their information 
  • PRISMA flowchart information
    • number of records retrieved; number of duplicates found; and number of records and studies eligible and ineligible 
  • Complete inclusion and exclusion criteria 
    • criteria for study inclusion such as target population identified; precise dosage/administration of treatment; and explicit identification of the problem being studied for study inclusion
    • criteria for study exclusion such as those outside of target population; identifiable characteristics for study ineligibility; ineligible study types; and the bias criteria to be used for exclusion
  • Data extracted from studies
    • the qualitative and quantitative criteria used to extract data from studies; the statistical methods implemented to extract data; data extraction tools; and various models or algorithms used according to review needs.

For Cochrane reviews, access to their software RevMan is recommended and made available. RevMan can also be used for free for purely academic purposes, and for commercial use researchers must purchase a license. The RevMan software can be downloaded here.  

*adapted from Assembling the pieces of a systematic review: a guide for librarians edited by Margaret J. Foster and Sarah T. Jewell 2017.

 

Statistical Software recommendations

During the evaluating stage, the process of synthesizing these studies is initiated. For data management, choices on software can vary depending on the needs of the systematic review. Moreover, institutions may already have statistical software being licensed and used within the organization. Resources ready to use at the research team's disposal and which do not require extra time for training, are recommended. The research team will be more comfortable utilizing software they have used previously. A few recommendations for statistical software include: SAS, SPSS, and STATA. 

 

Reporting the Review

Each journal has different requirements for publishing a systematic review. Most adhere to the rigorous guidelines set out by Cochrane, and others may have additional requirements. Systematic reviews should be written, so they can be easily understood. The target audience should not be solely geared towards health care experts/professionals who understand technical vocabulary; but, policy makers and those who make healthcare decisions (non medical experts) will also use these reviews to decide the allocation of health resources. 

Journal Requirements

The University of Toledo has created a database of over 6,000 journals in the health and life sciences. This database provides links to publishers' resources for journal publication criteria.