Session 17 & 18
Evidence-based Medicine Recap: Systematic Review and Meta-analysis

Session Objectives:

At the end of these sessions, students will be expected to:

  1. Define and distinguish between systematic review and meta-analysis.
  2. Explain how meta-analysis can be used to quantitatively combine results of several studies, and when meta-analysis should be attempted as an analytical technique.
  3. Identify standard methodological criteria to use as guidelines when doing a literature review, or when critically reading a review and attempting to apply it to patient care.
  4. Describe ways in which systematic review and meta-analysis may be useful in doing an ISMS project.

DEFINITIONS:

Systematic Review

"Systematic Review = Overview = the application of scientific strategies that limit bias to the systematic assembly, critical appraisal, and synthesis of all relevant studies on a specific topic." Potsdam Conference on Meta-analysis (Cook, Sackett & Spitzer 1994)

Meta-analysis

"A quantitative method of combining the results of independent studies (usually drawn from the published literature) and synthesizing summaries and conclusions that may be used to evaluate therapeutic effectiveness, plan new studies, etc., with application chiefly in the areas of research and medicine." The National Library of Medicine (1989)

"Meta-analysis = Quantitative Overview = a systematic review that employs statistical methods to combine and summarize the results of several studies." Potsdam Conference on Meta-analysis (Cook, Sackett & Spitzer 1994)

ELEMENTS OF A HIGH QUALITY SYSTEMATIC REVIEW:
  1. State objectives of the review, and outline eligibility (inclusion/exclusion) criteria for studies
  2. Exhaustively search for studies that seem to meet eligibility criteria
  3. Tabulate characteristics of each study identified and assess its methodological quality
  4. Apply eligibility criteria and justify any exclusions
  5. Assemble the most complete dataset feasible, with involvement of investigators
  6. Analyze results of eligible studies. Use statistical synthesis of data (meta-analysis) if appropriate & possible
  7. Perform sensitivity analyses, if appropriate and possible (including subgroup analyses)
  8. Prepare a structured report of the review, stating aims, describing materials and methods, and reporting results

 

Cochrane Collaboration

Mission Statement: The Cochrane Collaboration is an international organisation that aims to help people make well-informed decisions about healthcare by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of healthcare interventions.

Principles: The Cochrane Collaboration's work is based on nine key principles:

  1. Collaboration, by internally and externally fostering good communications, open decision-making and teamwork.
  2. Building on the enthusiasm of individuals, by involving and supporting people of different skills and backgrounds.
  3. Avoiding duplication, by good management and co-ordination to maximise economy of effort.
  4. Minimising bias, through a variety of approaches such as scientific rigour, ensuring broad participation, and avoiding conflicts of interest.
  5. Keeping up to date, by a commitment to ensure that Cochrane Reviews are maintained through identification and incorporation of new evidence.
  6. Striving for relevance, by promoting the assessment of healthcare interventions using outcomes that matter to people making choices in health care.
  7. Promoting access, by wide dissemination of the outputs of the Collaboration, taking advantage of strategic alliances, and by promoting appropriate prices, content and media to meet the needs of users worldwide.
  8. Ensuring quality, by being open and responsive to criticism, applying advances in methodology, and developing systems for quality improvement.
  9. Continuity, by ensuring that responsibility for reviews, editorial processes and key functions is maintained and renewed.

 

The Cochrane Library

This is the main output of the Collaboration, updated quarterly and distributed on an annual subscription basis on disk, CD-ROM and via the Internet. It currently includes several different databases:

  • The Cochrane Database of Systematic Reviews contains protocols and reviews prepared and maintained by Collaborative Review Groups. It includes a Comments and Criticisms System to enable users to help improve the quality of Cochrane Reviews.
  • The Database of Abstracts of Reviews of Effectiveness assembled and maintained by the NHS Centre for Reviews and Dissemination in York, England, contains critical assessments and structured abstracts of other systematic reviews, conforming to explicit quality criteria.
  • The Cochrane Controlled Trials Register contains bibliographic information on tens of thousands of controlled trials, including reports published in conference proceedings and many other sources not currently listed in other bibliographic databases.
  • The Cochrane Review Methodology Database contains references to articles and books on the science of reviewing research. The Cochrane Library also contains a handbook on how to conduct a systematic review, and a glossary of terms.
  • The Cochrane Collaboration section in The Cochrane Library contains contact details and other information about Collaborative Review Groups and other contributing entities within the Cochrane Collaboration.

Required reading for this topic:

Oxman AD, Cook DJ, Guyatt GH. Users’ guide to the medical literature. VI. How to use an overview. Evidence-Based Medicine Working Group. JAMA 1994; 272:1367-1371. (This is one of the McMaster series articles on clinical epidemiology in your syllabus.)

Tom Fahey, Nigel Stocks, and Toby Thomas. Quantitative systematic review of randomised controlled trials comparing antibiotic with placebo for acute cough in adults. BMJ 1998; 316: 906-910. [Abstract] [Full text] [extra]

Suggested reading for this topic:

Bero L, Rennie D. The Cochrane Collaboration. Preparing, maintaining, and disseminating systematic reviews of the effects of health care. JAMA. 1995;274:1935-8.

Chalmers I, Altman DG (Eds). Systematic Reviews. London, BMJ Publishing, 1995.

Cook DJ, Mulrow CD, Haynes RB.. Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions. Ann Intern Med 1997;126:376-380. http://www.acponline.org/journals/annals/01mar97/bestevid.htm

Cook DJ, Sackett DL, Spitzer WO. Methodological guidelines for systematic reveiws of randomized control trials in health care for the Potsdam Consultation on Meta-analysis. J Clin Epidemiol 1994;48:167-171.

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med 1997;127:380-7.

Gray JAM. Evidence-based Healthcare: How to make health policy & management decisions. London/New York, Churchill Livingston, 1997.

Greenhalgh T. How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses). BMJ 1997; 315: 672-675.

Hunt DL, McKibbon KA. Locating and appraising systematic reviews. Ann Intern Med. 1997;126:532-8. http://www.acponline.org/journals/annals/01apr97/systemat.htm

Meade MO, Richardson WS. Selecting and appraising studies for a systematic review. Ann Intern Med 1997;127:531-7.

Lau J, Ioannidis JP, Schmid CH. Quantitative synthesis in systematic reviews. Ann Intern Med 1997;127:820-6. http://www.acponline.org/journals/annals/01nov97/quantsyn.htm

Sackett DL, Richardson WS, Rosenberg W, Haynes, RB. Evidence-based medicine: How to practice teach EBM. London/New York, Churchill Livingston, 1997.

Internet Web Sites
©1999, Fredric M. Wolf, Ph. D.