Systematic Reviews for the Social Sciences: Preparation

Systematic Reviews Consultation

Interested in consulting with a Social Sciences Librarian regarding your systematic review? Fill out our online consultation request form!

Software

Preparing a Systematic Review

This guide provides an overview of the steps in preparaing a systematic review (SR). For more detailed information please consult the sources used to create this guide at the bottom of the page.

1. Assemble a Research Team

The review team is responsible for managing and conducting the SR. In order to minimize bias and produce a high quality SR, multidisciplinary teams of two or more qualified individuals should be organized.

First time reviewers should work with others who are experienced in the process.

The team should consist of the following:  

  • clinical content experts
  • experts in SR methodology, with knowledge in reducing risk of bias, study design, data collection, management & analysis, and statistical analys
  • librarians or information specialists for database searching
  • other relevant users and stakeholders (as feasible) - i.e. consumers, individuals skilled in project management, writing, editing

Potential conflicts of interest (COI) should be disclosed for each team member, including financial, professional or intellectual bias. Consideration for exclusion should be given for individuals whose COI may diminish credibility of the review.   

2. Create a Focused and Answerable Research Question

It is important to develop a focused, answerable, well-formulated question as this will determine many other components of the review, including the search for studies, data extraction, synthesis, and presentation of findings.  

The following factors should be considered when formulating a research question:

  • is this a new or novel topic
  • does the question focus on uncertainties that underlie disagreement in practice
  • are the outcomes and interventions of interest to patients and clinicians

Formulating a precise question can be difficult.  The PICO mnemonic is a structured format used to improve the scientific rigor of a SR.

Image Source:  http://books.nap.edu/openbook.php?record_id=13059&page=68

Click here to learn more about PICO

3. Has a systematic review been published or registered?

Conducting a SR is an expensive and time consuming process.  Before embarking in the process the team should ensure that another SR does not already exist.  It is useful to search the following databases for completed SRs or protocols for those in progress.

4. Develop a Research Protocol

Once it has been determined that a SR should be conducted the next step is to prepare a protocol. A protocol is a detailed description of the objectives and methods of the review.  It is important for several reasons:

  • allows for careful planning and anticipation of potential problems
  • explicit documentation allows for comparison and replication of the protocol and completed review
  • the validity of planned methods can be judged
  • prevents arbitrary decision making regarding inclusion criteria and extraction of data
  • may reduce duplication of efforts

Source: Shamseer (2015)

It is common to amend protocols after the review has started.  Common reasons include extending the period of search to include older and newer studies, broadening eligibility criteria, and adding new analyses suggested by the primary analysis. Researchers should not modify the protocol based on knowledge of the results of analyses.  IOM - standards for SR paper

The PRISMA-P checklist (below) can be used to develop protocols:

5. Set a Realistic Timeline

Conducting a SR takes time - anywhere from 6 to 12 months!

A SR has a fixed component and a variable component that increases with the number of citations that must be reviewed. An analysis of time spent showed that it took on average 1,139 hours to conduct a SR. Hours were broken down into the following:

  • 588 hours - protocol development, searching and retrieval
  • 144 hours - statistical analysis
  • 206 hours - report writing
  • 201 hours - administration

Source: Glasziou et. al. (2001) Systematic reviews in health care: A practical guide

The Cochrane Group provides an example of a timeline for a 12 month review:

Image Source:  Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0