1	Clash Between Data, Practice, and Policy
2	The Problem A health concern gets substantial public attention Pressure builds from all quarters to reach a policy decision High quality RCT data are not available to conclusively answer the question What to do? The example of ephedra
3	Ephedra the Herb
4	Ephedrine the Molecule
5	Why Do People Use Ephedra? In China: used as part of a concoction for a variety of symptoms In Europe: approved in Germany to treat asthma and congestion In USA: To lose weight To improve athletic performance
6	"“FDA targets herbal product" “FDA targets herbal product” (Financial Times, London, 8/19/2002) “Petition urges U.S. to ban supplements with ephedra” (NYT, 9/6/2001) “Ephedra is now under intense scrutiny” (LA Times, 8/2/2002) “For young athletes, limited gains--at a high risk (Washington Post, 4/9/02) “Ephedrine: safe or lethal?” (USA Today, 11/8/2001) Public Citizen asks Secretary of HHS to ban ephedra as a threat to public health Ephedra is an “imminent hazard…” Senator Durbin
7	"HHS Secretary Thompson announces that..." HHS Secretary Thompson announces that RAND will conduct comprehensive review of the existing science on ephedrine alkaloids (6/14/2002) “The results of the RAND review will help FDA's scientists to develop future regulatory actions on dietary supplements containing ephedrine alkaloids.” Lester M. Crawford, DVM, PhD. Acting Commissioner, FDA (10/8/2002) AHRQ, with funding from Office of Dietary Supplements and National Center for Complementary and Alternative Medicine, sponsor an Evidence Report by the Southern California Evidence Based Practice Center at RAND.
8	The Key Question
9	Systematic Review & Meta-analysis of the Efficacy & Safety of Ephedra and Ephedrine
10	Results of Weight Loss Comparisons
11	Conclusions: Weight Loss Ephedra alone and in combination is associated with weight loss No long-term studies of effectiveness Few head-to-head comparisons with alternative therapies Ephedra likely to be less effective than weight loss drugs
12	Conclusions: Athletic Performance 8 studies identified None on Ephedra 6 by Bell and colleagues All studied particular interventions and outcomes, and could not be pooled statistically All studied small numbers of young, fit males and assessed immediate performance Studies report improved performance only for ephedrine + caffeine
13	Key Questions: Safety Does ephedra use have adverse effects? Does using ephedra increase the risk of heart disease or other serious and life-threatening events?
14	Adverse Events: Data from Controlled Clinical Trials
15	Adverse Events:Clinical Trials No deaths, heart attacks, or other serious adverse events reported in clinical trials Clinical trials studied only a few thousand people This number is too small to detect a rate of adverse events smaller than 1 per 1,000
16
17
18	Criteria for Classification of Case Reports as “Sentinel Events” Sentinel Event Documentation of the adverse event Documentation that ephedra or ephedrine was consumed within the prior 24 hours (by report or toxicology examination) Documentation that other potential causes of the adverse event were evaluated and excluded Possible Sentinel Event Documentation of the adverse event Documentation that ephedra or ephedrine was consumed within the prior 24 hours (by report or toxicology examination) Other potential causes exist or were not effectively excluded
19	FDA Reports of Adverse Events
20
21	Conclusions of the Evidence Report (2/28/03) Neither side - the critics or the supporters - is completely right or wrong Ephedra does promote short-term weight loss But long-term effects are unknown Ephedrine plus caffeine does have modest short-term effects on athletic performance After use of a single dose Ephedra is associated with some side effects Upper GI symptoms, autonomic hyperactivity, psychiatric symptoms, palpitations
22	But does ephedra cause serious adverse events? Ephedrine acts on receptors in the cardiovascular system and in the brain Chemically related compounds have been proven to cause these same adverse events Phenylpropanolamine, amphetamines The sentinel events we identified are a signal that ephedra may also cause serious adverse events Recent case-control study reported an increased risk for hemorrhagic stroke with ephedra use Adjusted odds ratio of 3.95 (95% CI: 0.7-18.0; p=0.07)
23	Limitations of the Analysis Clinical trials Usual limitations for clinical trial meta-analysis No deaths or other serious adverse events reported but insufficient sample size to detect a rate of adverse events smaller than 1 per 1000 Case report adverse event data Under-reporting? Many reports did not contain sufficient documentation No estimate of the number of people using ephedra Study design considered insufficient to warrant conclusions regarding causality In order to assess safety at the conventional threshold for statistical significance and inference, a definitive hypothesis-testing study is needed.
24	Key Issues to Consider There is no provable certainty; therefore, the question is what are the ramifications of a policy decision or non-decision, and how much uncertainty can we tolerate? In the case of ephedra, the degree of uncertainty regarding a causal relationship between use and serious adverse events did not meet the conventional “p<0.05” threshold.
25	Key Issues to Consider The time needed to conduct a hypothesis-testing study of ephedra would be 2-5 years. If ephedra is causally related to an increased risk of serious adverse events, then during the time period needed to conduct a study some number of ephedra users would succumb. If ephedra is not causally related to an increased risk of serious adverse events, then a decision to withdraw it from the market would deny some potential ephedra users the possibility of weight loss benefits we observed.
26	The Policy Decision Policy decision was made to withdraw ephedra from the market.
27	Key Issues to Consider
28