Systematic reviews are often advocated as a mechanism for evaluating evidence for clinical practice, but how do you assess the quality of a systematic review?
When appraising a systematic review, you should ask the following questions:
- What type of review question is being asked?
- Is the review question relevant?
- Did the review address a clearly defined issue?
- Does the review have a clearly stated set of objectives and a pre-defined protocol that includes eligibility criteria?
- Was the scope of the review appropriate for the review question?
- Was there enough information given on the population studied, the intervention applied, any comparators used, and the outcomes considered?
- Does the review have an explicit, reproducible methodology?
- Was the review performed according to the original protocol?
- Were all of the important and relevant studies included?
- Which bibliographic databases were included?
- Was the search strategy appropriate and comprehensive?
- Were reference lists followed up?
- Were grey literature and unpublished studies included?
- Were studies included regardless of their date of publication?
- Were studies included regardless of positive or negative outcomes?
- Were non-English language studies included?
- Were adequate bibliographic details provided for each of the included studies?
- Were appropriate steps taken to minimise bias and errors in the selection of included studies?
- Were the exclusion criteria appropriate?
- Did the review authors rigorously assess the quality of included studies and were appropriate measures taken to minimise bias and errors in the quality assessment process?
- Are the results of all included studies clearly displayed?
- Were appropriate measures taken to minimise bias and errors in the data extraction process?
- Were appropriate methods used for statistical analyses and were they performed correctly?
- What is the overall result of the review and how are the results expressed?
- If the results of the review have been combined, was it appropriate to do so?
- How precise are the results and are they presented with confidence intervals?
- Were differences between the studies assessed?
- Can the results be applied to the local population?
- Were all important outcomes considered?
- Do the review conclusions accurately reflect the evidence that was reviewed?
- Does the review report all of the relevant findings according to the PRISMA checklist?
- Do the benefits outweigh any potential harms and costs?