Evidence Synthesis: Systematic & Scoping Reviews for the University Park Campus: Screening

Processes that have been used by published reviews include:

• Placing all abstracts into a folder in a citation management program (Zotero, EndNote, RefWorks, etc; see the Citation Management Guide).
• Upload Library to Covidence for screening abstracts and Full text phases

These are not the only ways to review abstracts, and additional methods are also acceptable. No matter what your author team selects, be sure to keep track of your methods so you can appropriately write the methods section of your systematic review manuscript.

Searches in databases result in citations and abstracts. These most commonly describe journal articles, but other publication types-- conference papers and posters, grey literature, clinical trials data, etc.-- can also be included in systematic reviews. Most systematic review preparation information (Prisma Flow Diagram, Cochrane Handbook for Systematic Reviews of Interventions) suggest that authors begin by appraising abstracts only, not the full text of the article or other item, as it can be time-consuming and costly to obtain the full text of every item retrieved in a search. Authors of systematic reviews have varied approaches for appraisal and there is no gold standard.

The Covidence software allows for automatic de-duplicating, managing the abstract screening process, the full text screening process, and creates PRISMA chart automatically.

• Remove duplicate abstracts from your search results. You may find the same citation in two or more databases. You do not need more than one copy of each abstract, so removing duplicates first can save you time.
• The reviewers should have access to the inclusion and exclusion criteria created earlier in the process.
• Each abstract should be reviewed by at least 2 reviewers, independently.
• Each abstract should be read in full by the reviewer. If it meets the inclusion/exclusion criteria and should be included, mark it as INCLUDED. If it does not meet the criteria, mark it as EXCLUDED. If you are not sure, mark it as MAYBE.
• Not all abstracts contain all the needed information to make a decision. For example, perhaps your criteria requires double-blinded randomized controlled trials. An abstract states that it describes a randomized controlled trial. You cannot determine if the study was double-blinded from this abstract. If you cannot determine if the abstract meets the criteria, mark it as MAYBE.
• Likewise, any citations that do not include an abstract should be marked as MAYBE.
• Be over inclusive at this stage, rather than less inclusive.
• After each abstract has been reviewed by 2 people, the whole group of authors should examine the reviewers' marks. 

• Abstracts where both reviewers agreed to include the abstract should be included in the next step.
• Abstracts where both reviewers agreed to exclude the abstract should be excluded from the next step.
• Abstracts that both reviewers marked as Maybe should be included in the next step.
• If there is disagreement-- one reviewer marked an abstract as "include" and one marked as "exclude"-- the abstract should be read by a third reviewer. All three reviewers should then discuss the abstract as it relates to the inclusion/exclusion criteria and make final decisions to include or exclude from the next step. Again, be over-inclusive at this stage. 

Now that you've appraised your abstracts, it's time to move to the next step: Obtaining Full Text.

After obtaining the full items, two authors should independently read each item to determine if they meet the inclusion/exclusion criteria. The reviewer opinions should be examined by the entire team, and disagreements in inclusion/exclusion should be discussed by the entire group of authors. You can use similar processes as you did for examining abstracts to record the reviewers' opinions of the full-text items. 

During reading, you may realize that some items do not include all the needed information to determine if it meets inclusion/exclusion criteria, or do not provide complete results. It is entirely appropriate to contact the corresponding authors of items and inquire about specific methodologies or data that was not presented in the published item, and to consider the authors' responses along with the published item.

In addition to using the inclusion/exclusion criteria previously developed to assess items, you can also examine the quality of the study and the quality of the presented data analysis. The kinds of quality measures to consider vary between study types. A randomized controlled trial and a cohort study are created using different methodologies and the data arising from these two experiment types will be analyzed using different methods. 

To get ideas about appropriate methodological quality issues to consider including, the following sites provide worksheets for critical appraisal of different study types:

• JAMAEvidence
• Centre for Evidence-Based Medicine (Oxford)
• Evidence Based Medicine Toolkit, University of Alberta

After you have completed this step, you will have identified all the acceptable articles and other items that meet your criteria. These items should now be moved onto the next steps: data extraction and synthesis.

Free Appraisal Software

Appraisal Software Available to Purchase