This guide provides advice for conducting the steps of appraising results for inclusion in a systematic review:
Processes that have been used by published reviews include:
These are not the only ways to review abstracts, and additional methods are also acceptable. No matter what your author team selects, be sure to keep track of your methods so you can appropriately write the methods section of your systematic review manuscript.
Searches in databases result in citations and abstracts. These most commonly describe journal articles, but other publication types-- conference papers and posters, grey literature, clinical trials data, etc.-- can also be included in systematic reviews. Most systematic review preparation information (Prisma Flow Diagram, Cochrane Handbook for Systematic Reviews of Interventions) suggest that authors begin by appraising abstracts only, not the full text of the article or other item, as it can be time-consuming and costly to obtain the full text of every item retrieved in a search. Authors of systematic reviews have varied approaches for appraisal and there is no gold standard.
Now that you've appraised your abstracts, it's time to move to the next step: Obtaining Full Text.
After obtaining the full items, two authors should independently read each item to determine if they meet the inclusion/exclusion criteria. The reviewer opinions should be examined by the entire team, and disagreements in inclusion/exclusion should be discussed by the entire group of authors. You can use similar processes as you did for examining abstracts to record the reviewers' opinions of the full-text items.
During reading, you may realize that some items do not include all the needed information to determine if it meets inclusion/exclusion criteria, or do not provide complete results. It is entirely appropriate to contact the corresponding authors of items and inquire about specific methodologies or data that was not presented in the published item, and to consider the authors' responses along with the published item.
In addition to using the inclusion/exclusion criteria previously developed to assess items, you can also examine the quality of the study and the quality of the presented data analysis. The kinds of quality measures to consider vary between study types. A randomized controlled trial and a cohort study are created using different methodologies and the data arising from these two experiment types will be analyzed using different methods.
To get ideas about appropriate methodological quality issues to consider including, the following sites provide worksheets for critical appraisal of different study types:
After you have completed this step, you will have identified all the acceptable articles and other items that meet your criteria. These items should now be moved onto the next steps: data extraction and synthesis.
Free Appraisal Software
Appraisal Software Available to Purchase