<< Back to Agenda

Slide 1: Strengths & Limitations of Common Study Designs [In Defense of the Large Randomized Trial]

Sylvan B. Green, M.D.
Arizona Cancer Center
University of Arizona
Tucson AZ

Slide 2:Randomized Trials

• Important
• in evaluating interventions for the prevention, diagnosis, and treatment of disease
• Ethical
• in the presence of uncertainty
• Robust
• large trials recommended to increase reliability
• Applicable to studies of efficacy and of effectiveness
• Can answer more than one question at a time
• (factorial trials and other designs)
• In some situations, can randomize entire groups
• (e.g., communities, medical practices)

Slide 3: Statistical Issues

• In designing any clinical study, we have to keep in mind two issues related to patient heterogeneity:
• the effect of chance
• the effect of bias (whether conscious or unconscious)
• These are addressed by:
• having adequate numbers of patients in the study
• using randomization for treatment assignment

Slide 4: Observational (non-randomized) Studies

• Are they useful?
• Epidemiologic investigations (etiology)
• Medical databases
• may provide information on patterns of care, cost, and both clinician & patient preferences
• analyses of such data may generate important hypotheses to be tested in future trials
• Should they be recommended for comparing alternative interventions?

Slide 5: Problems with Non-randomized Controls

• Effect of unmeasured or unknown prognostic factors
• Differential patient selection due to requirements for consent
• Bias in treatment assignment (unlike epidemiologic research concerning cause)
• Defining "time zero
• "
• Possible time trends in:
• patient population & disease characteristics
• diagnostic methods & supportive care

Slide 6: Time Trends in Diagnostic Methods (Chart)

Slide 7: Why Randomize?

• Bias (conscious or unconscious) is avoided
• Predictive factors (known and unknown) tend to be balanced between intervention & comparison groups
• Randomization provides a valid basis for statistical tests of significance
• Having a concurrent comparison group controls for time trends
• Results are more likely to be convincing

Slide 8: Beta Carotene and Cancer - 1

Alpha-Tocopherol Beta-Carotene Cancer Prevention Study

[Ref: The ATBC Cancer Prevention Study Group. N Engl J Med 1994; 330: 1029-1035]

METHODS. Randomized, double-blind, placebo-controlled primary prevention trial; 29,133 male smokers from southwestern Finland.

RESULTS. Unexpectedly, a higher incidence of lung cancer among the men who received beta carotene (change in incidence, 18 percent; 95% confidence interval, 3 to 36 percent).

CONCLUSIONS. No reduction in the incidence of lung cancer among male smokers after 5-8 years of alpha-tocopherol or beta carotene. In fact, this trial raises the possibility that these supplements may actually have harmful as well as beneficial effects.

Slide 9: Beta Carotene and Cancer - 2

Beta Carotene and Retinol Efficacy Trial

[Ref: Omenn GS, Goodman GE, Thornquist MD, et al. N Engl J Med 1996; 334: 1150-1155]

METHODS. Multicenter, randomized, double-blind, placebo-controlled primary prevention trial; 18,314 smokers, former smokers, and workers exposed to asbestos.

RESULTS. Compared with the placebo group, the treatment group had relative risk of lung cancer 1.28 (95% confidence interval, 1.04 to 1.57; P=0.02) and relative risk of death from lung cancer 1.46 (95% confidence interval, 1.07 to 2.00)

The trial was stopped 21 months earlier than planned.

CONCLUSIONS. Beta carotene plus vitamin A (4 yrs average) had no benefit and may have had an adverse effect in smokers and workers exposed to asbestos.

Slide 10: Beta Carotene and Cancer - 3

Physicians' Health Study

[Ref: Hennekens CH, Buring JE, Manson JE, et al. N Engl J Med 1996; 334: 1145-1149]

METHODS. Randomized, double-blind, placebo-controlled trial; 22,071 male physicians.

CONCLUSIONS. In this trial among healthy men, 12 years of supplementation with beta carotene produced neither benefit nor harm in terms of the incidence of malignant neoplasms, cardiovascular disease, or death from all causes.

Slide 11: Hierarchy of Strength of Evidence Concerning Efficacy of Treatment

1. 1. Anecdotal case reports
2. Case series without controls
3. Series with literature controls
4. Analyses using computer databases
5. "Case-Control" observational studies
6. Series based on historical control groups
7. Single randomized controlled clinical trials
8. Confirmed randomized controlled clinical trials

[Green SB, Byar DP. Statistics in Medicine 1984; 3: 361-70]

Slide 12: Factorial Design Example: Physicians' Health Study (2 x 2 factorial) (Chart)

Slide 13: When to Use Factorial Designs

• Genuinely interested in more than one intervention
• The interventions can actually be given together (i.e., they are not known to interfere with each other; toxicity does not add to unacceptable levels)
• Mechanisms of action of the interventions are different
• Serious interactions are not expected
• -OR -
• Information on interactions is of particular interest (but larger sample size may then be needed)

Slide 14: Interactions and Subgroup Analyses

• Is it expected that the actual treatment effect may differ in a meaningful way between different subgroups?
• Apparent differences can result by chance alone
• increased risk of spurious results with greater number of subgroup analyses
• statistical power for formally testing interactions requires larger sample size

Slide 15: Recommendation

Large trials (adequate sample size) for reliable inferences

• when prior reason to suspect important interaction, trial large enough to investigate subgroups (adequate power to test the interaction)
• otherwise, focus on primary question(s); can explore data for subgroup interactions, but interpret cautiously (may suggest hypothesis for future study)

Slide 16: Large Simple Trials

• More use of randomized trials is needed to address areas of uncertainty in medicine
• Given patient heterogeneity and the play of chance, large numbers of patients are needed to provide reliable estimates of the effect of treatment
• Realistic effects are relatively modest in size (but still potentially of great public health importance)
• Simplicity of trials permits larger numbers of patients with lesser expenditure of resources
• simplified eligibility criteria
• focus data collection on important endpoints

Slide: 17

ere is simply no serious scientific alternative to the generation of large-scale randomized evidence. If trials can be vastly simplified, as has already been achieved in a few major diseases, and thereby made vastly larger, then they have a central role to play in the development of rational criteria for the planning of health care throughout the world."

Peto R, Collins R, Gray R. J Clin Epidemiol 1995; 48: 23-40

Slide 18: Randomization by Group

• Units of group randomization:
• Communities
• Small towns / villages
• Factories (workplaces)
• Schools / classrooms
• Religious institutions
• Chapters of social organizations
• Families
• Clinical practices
• Less efficient statistically than randomization by individual
• The design and analysis must account for the correlation of individuals within a group

Slide 19: Reasons for Randomizing by Group

1. Feasibility of delivery.
2. Political and administrative considerations.
3. To avoid contamination.
4. Nature of intervention.
5. Ready-made endpoints measured at group level.
6. Exploit existing arrangement to decrease cost.
7. Use site-specific resources to decrease cost.
8. Greater generalizability.

Slide 20: Randomized Trials as a Desirable Option

• Important to obtain unbiased comparisons of interventions
• Large trials (adequate sample size) for reliable inferences
• Randomized trials can present to participants the best choice for state-of-the-art intervention (consider current uncertainty about efficacy & toxicity)
• Increased knowledge of trials can benefit participants and science

<< Back to Agenda