1	Strengths & Limitations of Common Study Designs [In Defense of the Large Randomized Trial] Sylvan B. Green, M.D. Arizona Cancer Center University of Arizona Tucson AZ
2	Randomized Trials Important in evaluating interventions for the prevention, diagnosis, and treatment of disease Ethical in the presence of uncertainty Robust large trials recommended to increase reliability Applicable to studies of efficacy and of effectiveness Can answer more than one question at a time (factorial trials and other designs) In some situations, can randomize entire groups (e.g., communities, medical practices)
3	Statistical Issues In designing any clinical study, we have to keep in mind two issues related to patient heterogeneity: the effect of chance the effect of bias (whether conscious or unconscious) These are addressed by: having adequate numbers of patients in the study using randomization for treatment assignment
4	Observational (non-randomized) Studies Are they useful? Epidemiologic investigations (etiology) Medical databases may provide information on patterns of care, cost, and both clinician & patient preferences analyses of such data may generate important hypotheses to be tested in future trials Should they be recommended for comparing alternative interventions?
5	Problems with Non-randomized Controls Effect of unmeasured or unknown prognostic factors Differential patient selection due to requirements for consent Bias in treatment assignment (unlike epidemiologic research concerning cause) Defining "time zero" Possible time trends in: patient population & disease characteristics diagnostic methods & supportive care
6
7	Why Randomize? Bias (conscious or unconscious) is avoided Predictive factors (known and unknown) tend to be balanced between intervention & comparison groups Randomization provides a valid basis for statistical tests of significance Having a concurrent comparison group controls for time trends Results are more likely to be convincing
8	Beta Carotene and Cancer - 1 Alpha-Tocopherol Beta-Carotene Cancer Prevention Study [Ref: The ATBC Cancer Prevention Study Group. N Engl J Med 1994; 330: 1029-1035] METHODS. Randomized, double-blind, placebo-controlled primary prevention trial; 29,133 male smokers from southwestern Finland. RESULTS. Unexpectedly, a higher incidence of lung cancer among the men who received beta carotene (change in incidence, 18 percent; 95% confidence interval, 3 to 36 percent). CONCLUSIONS. No reduction in the incidence of lung cancer among male smokers after 5-8 years of alpha-tocopherol or beta carotene. In fact, this trial raises the possibility that these supplements may actually have harmful as well as beneficial effects.
9	Beta Carotene and Cancer - 2 Beta Carotene and Retinol Efficacy Trial [Ref: Omenn GS, Goodman GE, Thornquist MD, et al. N Engl J Med 1996; 334: 1150-1155] METHODS. Multicenter, randomized, double-blind, placebo-controlled primary prevention trial; 18,314 smokers, former smokers, and workers exposed to asbestos. RESULTS. Compared with the placebo group, the treatment group had relative risk of lung cancer 1.28 (95% confidence interval, 1.04 to 1.57; P=0.02) and relative risk of death from lung cancer 1.46 (95% confidence interval, 1.07 to 2.00) The trial was stopped 21 months earlier than planned. CONCLUSIONS. Beta carotene plus vitamin A (4 yrs average) had no benefit and may have had an adverse effect in smokers and workers exposed to asbestos.
10	Beta Carotene and Cancer - 3 Physicians' Health Study [Ref: Hennekens CH, Buring JE, Manson JE, et al. N Engl J Med 1996; 334: 1145-1149] METHODS. Randomized, double-blind, placebo-controlled trial; 22,071 male physicians. CONCLUSIONS. In this trial among healthy men, 12 years of supplementation with beta carotene produced neither benefit nor harm in terms of the incidence of malignant neoplasms, cardiovascular disease, or death from all causes.
11	Hierarchy of Strength of Evidence Concerning Efficacy of Treatment 1. Anecdotal case reports 2. Case series without controls 3. Series with literature controls 4. Analyses using computer databases 5. "Case-Control" observational studies 6. Series based on historical control groups 7. Single randomized controlled clinical trials 8. Confirmed randomized controlled clinical trials [Green SB, Byar DP. Statistics in Medicine 1984; 3: 361-70]
12	Factorial Design Example: Physicians' Health Study (2 x 2 factorial)
13	When to Use Factorial Designs Genuinely interested in more than one intervention The interventions can actually be given together (i.e., they are not known to interfere with each other; toxicity does not add to unacceptable levels) Mechanisms of action of the interventions are different Serious interactions are not expected — OR — Information on interactions is of particular interest (but larger sample size may then be needed)
14	Interactions and Subgroup Analyses Is it expected that the actual treatment effect may differ in a meaningful way between different subgroups? Apparent differences can result by chance alone increased risk of spurious results with greater number of subgroup analyses statistical power for formally testing interactions requires larger sample size
15	Recommendation Large trials (adequate sample size) for reliable inferences when prior reason to suspect important interaction, trial large enough to investigate subgroups (adequate power to test the interaction) otherwise, focus on primary question(s); can explore data for subgroup interactions, but interpret cautiously (may suggest hypothesis for future study)
16	Large Simple Trials More use of randomized trials is needed to address areas of uncertainty in medicine Given patient heterogeneity and the play of chance, large numbers of patients are needed to provide reliable estimates of the effect of treatment Realistic effects are relatively modest in size (but still potentially of great public health importance) Simplicity of trials permits larger numbers of patients with lesser expenditure of resources simplified eligibility criteria focus data collection on important endpoints
17	""There is simply no..." "There is simply no serious scientific alternative to the generation of large-scale randomized evidence. If trials can be vastly simplified, as has already been achieved in a few major diseases, and thereby made vastly larger, then they have a central role to play in the development of rational criteria for the planning of health care throughout the world." Peto R, Collins R, Gray R. J Clin Epidemiol 1995; 48: 23-40
18	Randomization by Group Units of group randomization: Communities Small towns / villages Factories (workplaces) Schools / classrooms Religious institutions Chapters of social organizations Families Clinical practices Less efficient statistically than randomization by individual The design and analysis must account for the correlation of individuals within a group
19	Reasons for Randomizing by Group 1. Feasibility of delivery. 2. Political and administrative considerations. 3. To avoid contamination. 4. Nature of intervention. 5. Ready-made endpoints measured at group level. 6. Exploit existing arrangement to decrease cost. 7. Use site-specific resources to decrease cost. 8. Greater generalizability.
20	Randomized Trials as a Desirable Option Important to obtain unbiased comparisons of interventions Large trials (adequate sample size) for reliable inferences Randomized trials can present to participants the best choice for state-of-the-art intervention (consider current uncertainty about efficacy & toxicity) Increased knowledge of trials can benefit participants and science