1	Evidence-Based Medicine: Problems and Methods A Short History NIH, January 11, 2005
2	Problems of Clinical Research Human variation Therapeutic relationship: physician-patient roles and expectations Ethics of research: subject selection, subject compliance, continuance Multiple stakeholders: physicians, patients, policymakers, manufacturers
3	Methods Comparative clinical trials Randomization => statistical analysis Blinding “Intention to treat” Meta-analysis Systematic review of evidence
4	Comparative Trials: Precursors Work not accepted at the time James Lind on scurvy (1753) Pierre Louis on bloodletting (1835) Ignaz Semmelweis on puerperal fever (1847) Work accepted at the time John Snow on cholera (1854) Walter Reed on yellow fever (1901)
5	1900: Calls for reform of patent medicine market 1905: the AMA’s Council on Chemistry and Pharmacy – the need for “ a rational therapeutics” 1906: the Pure Food and Drug Act – the need for a scientific basis for regulation 1931: the Therapeutic Trials Committee of the Medical Research Council
6	Borrowed Ideas: Randomization R.A. Fisher’s work at Rothamsted Agricultural Station The Design of Experiments (1935) Randomization of assignment to experimental and control groups allows the investigator to calculate the statistical significance and likelihood of error of observed differences in effect.
7	Borrowed Ideas: Blinding Blindfold Tests: widely used by advertisers and consumer groups in the 1930s and 1940s Torald Sollmann suggested a placebo control and blinded observer as a solution to investigator bias as early as 1930
8	The Streptomycin TB Trials Feldman and Hinshaw’s principles of design 1944: “some procedure of chance” The VA 1946: problems of physician compliance The MRC 1947-48: drug scarcity ensures compliance to protocol The US Public Health Service 1947: replacing physician judgment with standardized criteria
9	Working Guideline to Regulatory Law Explosion of new drugs in 1950s raised concerns over pricing and exploitation Thalidomide disaster of 1962 pushed passage of Kefauver-Harris Amendments FDA empowered to review efficacy of new and existing drugs; relied initially on expert clinical judgment – the Drug Efficacy Study panels 1966-69
10	RCT As Evidentiary Standard FDA moved to withdraw drugs from market on DESI recommendations Pharmaceutical companies went to court, forcing FDA to define “substantial evidence.” May 1970 regulations specified: Criteria for patient selection Exclusion of bias Comparability of variables Use of control group Statistical analysis of data
11	1970-1990: More Players, More Evidence Investigators produced much trial evidence of varying quality Limited resources for major definitive trials Who has the expertise to assess the evidence? How do we select the “best” evidence/determine where more is needed? What is the “answer” we are looking for? Best fit to theoretical concepts Comparative efficacy User efficacy
12	A Statistical Solution: Meta-Analysis First used in astronomy 1860s Karl Pearson’s work on typhoid vaccine 1904 Introduced into social sciences 1970s and then into medicine Do results correlate with those of large RCTs? (Not always – LeLorier et al, 1997) Does meta-analysis resolve problems of small, poor-quality trials? (No – Jüni et al, 1999)
13	Evidence Review: Art => Science Cochrane 1972: collective ignorance of much of the available evidence Mulrow 1987: Selection bias, haphazard analysis found in literature reviews Oxman and Guyatt 1988: Guidelines for reading reviews McMaster method => Evidence-based Medicine 1990-95 The U.K. Cochrane Centre 1992
14	But Systematic Review Projects Need Evidence to Review Factors of bias, subject recruitment, ethics, career incentives, data measurability lead to problems of experimental and publication selection: Only certain trials are likely to be done and not all the results will be published.