<< Back to Agenda

Slide 1: Evidence-Based Medicine: Problems and Methods

A Short History

NIH, January 11, 2005

Slide 2: Problems of Clinical Research

• Human variation
• Therapeutic relationship: physician-patient roles and expectations
• Ethics of research: subject selection, subject compliance, continuance
• Multiple stakeholders: physicians, patients, policymakers, manufacturers

Slide 3: Methods

• Comparative clinical trials
• Randomization => statistical analysis
• Blinding
• "Intention to treat"
• Meta-analysis
• Systematic review of evidence

Slide 4: Comparative Trials: Precursors

• Work not accepted at the time
• James Lind on scurvy (1753)
• Pierre Louis on bloodletting (1835)
• Ignaz Semmelweis on puerperal fever (1847)
• Work accepted at the time
• John Snow on cholera (1854)
• Walter Reed on yellow fever (1901)

Slide 5: 1900: Calls for reform of patent medicine market

• 1905: the AMA's Council on Chemistry and Pharmacy - the need for " a rational therapeutics"
• 1906: the Pure Food and Drug Act - the need for a scientific basis for regulation
• 1931: the Therapeutic Trials Committee of the Medical Research Council

Slide 6: Borrowed Ideas: Randomization

• R.A. Fisher's work at Rothamsted Agricultural Station
• The Design of Experiments (1935)
• Randomization of assignment to experimental and control groups allows the investigator to calculate the statistical significance and likelihood of error of observed differences in effect.

Slide 7: Borrowed Ideas: Blinding

• Blindfold Tests: widely used by advertisers and consumer groups in the 1930s and 1940s
• Torald Sollmann suggested a placebo control and blinded observer as a solution to investigator bias as early as 1930

Slide 8: The Streptomycin TB Trials

• Feldman and Hinshaw's principles of design 1944: "some procedure of chance"
• The VA 1946: problems of physician compliance
• The MRC 1947-48: drug scarcity ensures compliance to protocol
• The US Public Health Service 1947: replacing physician judgment with standardized criteria

Slide 9: Working Guideline to Regulatory Law

• Explosion of new drugs in 1950s raised concerns over pricing and exploitation
• Thalidomide disaster of 1962 pushed passage of Kefauver-Harris Amendments
• FDA empowered to review efficacy of new and existing drugs; relied initially on expert clinical judgment - the Drug Efficacy Study panels 1966-69

Slide 10: RCT As Evidentiary Standard

• FDA moved to withdraw drugs from market on DESI recommendations
• Pharmaceutical companies went to court, forcing FDA to define "substantial evidence."
• May 1970 regulations specified:
• Criteria for patient selection
• Exclusion of bias
• Comparability of variables
• Use of control group
• Statistical analysis of data

Slide 11: 1970-1990: More Players, More Evidence

• Investigators produced much trial evidence of varying quality
• Limited resources for major definitive trials
• Who has the expertise to assess the evidence?
• How do we select the "best" evidence/determine where more is needed?
• What is the "answer" we are looking for?
• Best fit to theoretical concepts
• Comparative efficacy
• User efficacy

Slide 12: A Statistical Solution: Meta-Analysis

• First used in astronomy 1860s
• Karl Pearson's work on typhoid vaccine 1904
• Introduced into social sciences 1970s and then into medicine
• Do results correlate with those of large RCTs? (Not always - LeLorier et al, 1997)
• Does meta-analysis resolve problems of small, poor-quality trials? (No - Jüni et al, 1999)

Slide 13: Evidence Review: Art => Science

• Cochrane 1972: collective ignorance of much of the available evidence
• Mulrow 1987: Selection bias, haphazard analysis found in literature reviews
• Ox
• man and Guyatt 1988: Guidelines for reading reviews
• McMaster method => Evidence-based Medicine 1990-95
• The U.K. Cochrane Centre 1992

Slide 14: But Systematic Review Projects Need Evidence to Review

Factors of bias, subject recruitment, ethics, career incentives, data measurability lead to problems of experimental and publication selection:

Only certain trials are likely to be done and not all the results will be published.

<< Back to Agenda