LEARN | Module 9
Systematic Reviews
This module will provide you with an opportunity to understand how to interpret a systematic review and appraise their usefulness in patient treatment. As always, please post any questions, comments or suggestions in the Disqus comment feed at the bottom of the module!
A systematic review is an overview of primary studies that contains an explicit statement of objectives, materials and methods and has been conducted according to explicit and reproducible methodology. It uses an explicit search strategy and inclusion/exclusion criteria, making it thorough and reproducible, reducing the bias associated with narrative reviews and providing an up-to-date summary on the current state of knowledge.
Systematic reviews can contain meta-analyses (quantitative synthesis of data from different studies of varying size, location, result, etc.), which can provide additional information (e.g. the actual quantitative effect of an intervention rather than a qualitative result), as well as increased statistical power than any individual study (by increasing “sample size”).
Video
Steps Undertaken in a Systematic Review
There are several steps involved in completing a systematic review/meta-analysis:
Formulate a structured question specifying the qualities of primary studies, including populations, interventions, comparisons and outcomes to be considered for the review
Conduct an extensive search of the literature documenting the search strategy
Select studies for inclusion in the review, justifying any exclusions
Assessment of the methodological quality of included studies using pre-specified criteria
Abstraction of relevant data
Data synthesis (meta-analysis if appropriate)
Discussion of findings
Sources of Bias in Systematic Reviews
Like any study, a systematic review also has avenues for bias. Here are a few sources of bias:
Inadequate literature searches (e.g. only searching MEDLINE) omit relevant studies. This bias can be minimised by using a thorough search strategy.
Publication bias, where statistically significant, ‘positive’ results are more likely to be published (publication bias), published rapidly (time lag bias), published in English (language bias), published more than once (multiple publication bias) and then cited by others (citation bias). This makes them much easier to find. Publication bias can be detected using a funnel plot of results from the trials to be used in the review and looking to see that an inverted funnel is present (see video for details).
Inclusion of poor quality primary studies means that poor quality study results could skew the results of a systematic review. Inclusion and exclusion criteria should be determined before proceeding with a systematic review.
Selective reporting of outcomes is when studies that are negative or not statistically significant are not published by the assessor. This means that "positive" studies are more likely to be published and therefore reviewed, despite the existence of "negative" studies.
Measuring Outcomes
Outcomes can be measured using either dichotomous measures (and reported as odds ratios, relative risks, absolute risks and numbers needed to treat) or continuous measures (and reported as weighted mean difference or standardised mean difference).
Data Synthesis
Meta-analyses collate data from multiple studies using an algorithm that calculates a weighted average, meaning the studies are combined based on their statistical power and validity. Data is then plotted on a forrest plot with dots that correspond to the power of studies, lines indicating confidence intervals and a weighted summation of results at the bottom. The horizontal axis presents a scale from favouring the intervention to favouring control, with no effect in the middle.
Heterogeneity
Heterogeneity refers to the differences in the various studies that constitute a systematic review (e.g. different initial patient populations). There are two types of heterogeneity: statistical (involving using I squared values and degrees of freedom to determine the level of heterogeneity) and clinical (where a straight line should be able to be drawn that connects all the confidence intervals of the studies on a forrest plot where there is clinically insignificant heterogeneity).
Sensitivity Analysis
Sensitivity analysis involves changing assumptions in which studies are included. For example, using only the high quality studies may show a different result to the combination of poor and high quality studies.
Subgroup Analysis
Subgroup analysis involves dividing studies into subgroups, say by age, and analysing individual subgroups.