“The Future of Longitudinal Studies:
What we know; What we don’t know; What we need to
know”
Inferring Causality from Longitudinal Studies
Chaired By: Elizabeth Owens, University Of California, Berkeley
Friday, March 21, 2003
Robert E. Larzelere
"The Difficulty of Making Valid Causal Inferences from Passive
Longitudinal Designs"
University of Nebraska Medical Center
Valid causal inferences require: 1) association, 2)
temporal sequence, and 3) isolation. Compared to cross-sectional
studies, passive longitudinal studies (PLSs) only enhance the temporal
sequence criterion (with T1 preceding T2), and then not as well
as single-group, pre-post clinical designs which allow a clearer
temporal demarcation between pre-test, treatment/intervention, and
post-test. “Isolation” of the purported cause from plausible
alternative explanations is the most difficult requirement for valid
causal inferences, a difficulty that is often under-estimated (although
astronomy shows that it is possible). Unless plausible alternative
explanations are ruled out, causal inferences are likely to be based
on the logical fallacy of “affirming the consequence,”
especially in non-experimental (passive) designs.
For example, the intervention selection bias is an alternative explanation
that, if ignored, can lead to incorrect causal conclusions about
corrective interventions. For instance, the longitudinal zero-order
association between mental health (MH) treatment and subsequent
suicides in youth indicates that MH treatment predicts a substantially
increased risk of suicide, 14.3 times as high a risk, based on the
median of 9 prospective studies. In general, corrective interventions
at T1 tend to be positively associated with detrimental outcomes
at T2 due to selection processes that make corrective interventions
more likely for those with a poor prognosis. This applies to corrective
interventions in medicine (e.g., radiation treatment, hospitalization),
education (Head Start), clinical psychology (marital counseling,
suicide treatment), and parenting (power assertive discipline, homework
assistance). If a corrective intervention were perfect, such that
its recipients scored as well at T2 as those who never had the presenting
problems, the longitudinal zero-order correlation would be .00.
Thus positive longitudinal correlations only indicate that the corrective
intervention fell short of perfection, but they do not discriminate
between effective and counterproductive interventions. Most passive
longitudinal studies of parenting have assumed that effective corrective
interventions would be negatively correlated with detrimental outcomes,
but that is achievable only for interventions that exceed perfection,
i.e., that make the recipients better than if they had never had
the presenting problems in the first place.
This bias against corrective interventions can be illustrated with
the least favored of power assertive disciplinary tactics, namely
corporal punishment. In a recent meta-analysis, the strongest evidence
against corporal punishment consisted of zero-order longitudinal
correlations (Gershoff, 2002). The intervention selection bias (i.e.,
child effects) predicts similar longitudinal correlations for all
disciplinary tactics. Accordingly, a meta-analysis of differences
in effect sizes between corporal punishment and alterative disciplinary
tactics found that only overly severe and predominant use of corporal
punishment compared unfavorably with alternative tactics. Customary
corporal punishment averaged the same effect size as alternatives,
and optimal spanking (calm spanking for defiance in 2- to 6-year-olds)
had effect sizes about 1/3 to ½ a standard deviation better
than alternative tactics in reducing antisocial behavior and noncompliance,
respectively (Larzelere, 2003).
Similarly, a replication of Straus’s strongest evidence against
corporal punishment (Straus, Sugarman, & Giles-Sims, 1997, which
controlled for T1 antisocial) found similar apparently detrimental
effects for 4 alternative disciplinary tactics and two other corrective
interventions (Head Start and visiting a psychiatrist), all of which
became non-significant when the measure of T1 antisocial was improved
(Larzelere & Smith, 2000). Most statistical controls reduce,
but do not eliminate confounds such as the selection bias (which
Campbell & Boruch, 1975, termed the “under-adjustment
bias” in their critiques of pessimistic Head Start evaluations).
In conclusion, the key to making valid causal inferences from PLSs
is to generate plausible alternative interpretations of data and
systematically test the differential implications of the competing
explanations. Causal inferences can also be strengthened with empirical
confirmations of specific causal mechanisms. Statistical innovations
may supplement the above types of careful thinking, but they are
not a panacea. There may be advantages to predicting change scores
rather than T2 scores and in matching the presumed causal lag to
the longitudinal interval.
References:
Campbell, D. P., & Boruch, R. F. (1975). Making the case for
randomized assignment to treatments by considering the alternatives:
Six ways in which quasi-experimental evaluations in compensatory
education tend to underestimate effects. In C. A. Bennett &
A. A. Lumsdaine (Eds.), Evaluation and experiment: Some critical
issues in assessing social programs (pp. 195-296). New York: Academic
Press.
Gershoff, E. T. (2002). Corporal punishment by parents and associated
child behaviors and experiences: A meta-analytic and theoretical
review. Psychological Bulletin, 128, 539-579.
Larzelere, R. E. (2003, April). A meta-analysis comparing the effect
sizes and correlates of corporal punishment with alternative disciplinary
tactics. Paper presented at the Society for Research in Child Development,
Tampa, FL.
Larzelere, R. E., & Smith, G. L. (2000, August). Controlled
longitudinal effects of five disciplinary tactics on antisocial
behavior. Paper presented at the annual convention of the American
Psychological Association, Washington, DC.
Straus, M. A., Sugarman, D. B., & Giles-Sims, J. (1997). Spanking
by parents and subsequent antisocial behavior of children. Archives
of Pediatrics and Adolescent Medicine, 151, 761-767.
Robert E. Larzelere's presentation "The
Difficulty of Making Valid Causal Inferences from Passive Longitudinal
Designs" can be viewed in PDF format, using Adobe®
Acrobat® Reader®.