Systematic Reviews (in the Health Sciences)

Whether you are conducting a systematic review, meta-analysis, scoping review, or other evidence synthesis project,  success starts with gathering a team with the appropriate knowledge and time; finding a research gap; and selecting a question where there appears to be abundant evidence that can be combined. There is not a specific order in which to do these steps, and most teams do all three simultaneously.

An evidence synthesis project team needs to include at least 3 individuals who will be a part of the team for the entire process of developing the research methods, searching, screening abstracts and full text, analyzing results, and writing the manuscript. There may be additional team members needed if special types of reviews are conducted: it's very important to have a statistician involved from the start in planning and conducting a meta-analysis.  There may also be additional team members who join for parts of the project (for example, adding more members during abstract screening can speed up this time-consuming process). Successful teams include 3 permanent members:

• 1 content expert/ lead investigator who has expertise and relevant experience in the subject area of the review and knowledge of current research being done in this subject.
• 1 methodology expert who has knowledge and/or experience with conducting evidence syntheses projects. They may or may not have deep knowledge of the subject area of the review.
• At least 1 co-investigator who has some knowledge and/or experience with the subject area of the review and general knowledge of evidence syntheses.

A team can also include these members, who participate in one or more portions of the project:

• Statistician, to plan study selection and data extraction, and assist with data analysis and presentation. It is helpful to have the statistician involved from the start of the project, to ensure the question and selected publications can provide the needed data for statistical analyses.
• Librarian, to design search methodology or advise on search methods, document search, and manage citations.

If another recent evidence synthesis project answering your exact question already exists, your project will be duplicative and difficult to get published. Search databases of biomedical research articles for systematic reviews covering your topic, or specialized databases only including systematic reviews and other syntheses, and read these reviews to determine what questions remain unanswered.

Systematic reviews can take several years to plan, conduct, and get published. Many teams will publish a protocol for a systematic review near the start of the project as a notification to the research community that they are working on this question and intend to publish a review. It can be helpful to review published protocols and avoid selecting questions that are currently in process.

The methods of evidence synthesis require that there be abundant amount of published basic evidence (individual studies). The Methods for conducting systematic reviews and meta-analyses require that this evidence is similar and can be combined. There is no minimum number of studies necessary to do a systematic review or meta-analysis. The only way to really determine if there is enough evidence to do a systematic review is to do the process and find out. The Cochrane Collaboration will frequently publish reviews where the result is "there is not enough similar evidence to answer this question at this time."

To help yourself determine if there may be enough evidence, start with the basic research evidence and determine if there is enough volume and similarity to combine the results. Use a literature database that includes publications covering your topic, search for your topic a few times using some different keywords, and read the full text of several (10 to 20) recent research articles on your topic. Compare the approaches taken by each article:

• What kind of participants are included? Consider age, gender, disease stage or severity, and other factors.
• What kind of research is being done? Is it prospective or retrospective? Chart reviews? Randomized controlled trials? Observational studies?
• What is being measured and how is it being measured?
• Over what time period did data collection take place?
• What outcomes are being measured, and how?

If after following this process, you:

-  find at least 3 articles that appear to have similar populations, similar exposures, similar measurements, similar outcomes, and similar methods, this is a sign that a systematic review may be possible to conduct. Keep reading this guide to learn how to develop your idea into a complete systematic review-formatted question.

- are not able to find at least 10 research articles-- all you find are editorials, commentaries, reviews, and other forms of non-research. This is a sign that it may not be a question that can be answered using the systematic review research methodology, which relies on finding abundant original research evidence.You may need to choose a new topic area that has more abundant, similar, research evidence, or consider doing a literature review which allows you to summarize information from non-research sources.

-  can find 20 research articles, but all of the articles are all very dissimilar--  every study enrolled people with different conditions, different co-existing conditions, or at different stages of severity, used different treatment methods, used different scales to measure outcomes, were conducted for disparate periods of time--  your topic may be too broad to answer as formulated.  Consider focusing your topic to one or two conditions, disease stages, treatments, scales, etc.

It is very common that questions researchers want to answer using a systematic review approach are not able to be answered because of the paucity of research evidence, lack of reported outcomes, or the lack of similarity in the research population studied. This is not a reflection of research skills, but a reality of evidence synthesis as a research method.