Research report no.14 2005
The efficacy of early childhood interventions
by Sarah Wise, Lisa da Silva, Elizabeth Webster and Ann Sanson
6. Effects of early childhood interventions
Assessing program outcomes
The available information on significant effects was examined to assess program outcomes. While non-significant trends may be part of a larger pattern, in and of themselves they cannot be interpreted reliably. Thus, only effects that were significant at .05 level were included. The use of the .05 cut-off is a commonly used cut-off in statistical analyses and was therefore adopted for consistency with scientific reports.
Long-running interventions have often been researched at different time points. Effect sizes are thus reported in terms of the timing of effects - whether the reported outcome effect was: short term (data collected during or immediately after the intervention); intermediate (data collected up to two years after the intervention period); or long-term (data collected more than two years after the intervention period).5
For ease of reporting and interpretation, the magnitude of effects are grouped into four categories: negligible (Neg) (effect size under 0.20); small (Sm) (effect size 0.20-0.49); medium (Med) (effect size 0.50-0.79); or large (Lg) (effect size 0.80 or greater).
A negative effect size (-) indicates that effects were in the opposite direction to that which was expected. That is, the control group performed better on a measured outcome than the intervention group.
Tables 6-10 summarise the available data on intervention effect sizes. Missing cells indicate that effect sizes were either not calculated, or not reported in the evaluation material reviewed. 6 The number of evaluation design elements present for each intervention is also included in these tables for reference.
Effects of cluster 1 interventions
The outcomes measured in the interventions in cluster 1 were all child outcomes and effect sizes were available for three of the six interventions in this cluster. Most of the effects were negligible to small, and very few effects were large. In addition, for the Perry project (that reported short, intermediate and long-term effect sizes), the positive effects on cognitive outcomes tended to diminish. In terms of specific outcomes, the effects of intervention on child cognitive abilities varied, with the Perry project reporting large short-term gains and PIDI reporting negligible short-term gains. Similarly, children's academic gains reported by the Perry project were larger than those reported by CPC. (See Table 6.)
Effects of cluster 2 interventions
Effects sizes were available for child and family outcomes for three of the interventions in cluster 2. Again, most of the effects were negligible to small, with some effects diminishing over time and others remaining stable in the intermediate term.
In terms of specific outcomes, the effects on child cognitive and academic outcomes were quite varied between, and even within, interventions. The HIPPY program found positive short and intermediate term effects on children's cognitive skills and academic skills that ranged from negligible to medium, while the PCDC found mainly negligible long-term effects, although academic effects were medium for boys. (See Table 7.)
Effects of cluster 3 interventions
Only one of the three evaluations in intervention cluster 3 reported effect sizes. Both child and family outcomes were measured. Outcomes were measured in the intermediate term and most of the effect sizes were small. In terms of specific outcomes, the effects on child behaviour ranged from small to medium, while the effects on parent outcomes (including income, child care use and support) were small. (See Table 8.)
Effects of cluster 4 interventions
Evaluations in intervention cluster 4 measured child and family outcomes. Effect sizes were available for four of the twelve evaluations; most effect sizes were for short-term outcomes only. Again, most of the effect sizes were negligible to small.
In terms of specific outcomes, the effects of interventions in cluster 4 on child cognitive outcomes were varied. Early Head Start found negligible effects, while BBBF found effects that ranged from negligible to medium, and IHDP found large effects. Effects on child emotional and behavioural outcomes were generally negligible, although Incredible Years and BBBF found some small to medium effects. Effects on parenting were contrasting, with Early Head Start reporting negligible effects, Incredible Years reporting small to medium effects and BBBF reporting effects that ranged from medium (but in the opposite direction) to large. (See Table 9.)
Effects of cluster 5 interventions
Triple P was the only evaluation in intervention cluster 5 that reported effect sizes. This evaluation found three large short-term effects on parent and child outcomes (child behaviour, parenting style and parent conflict over child rearing). Two of these large effects diminished at intermediate follow- up to small (child behaviour) and medium (parenting style). Effects on parent-child relationships were negligible, and small program effects were found on parent mental health, both of which were maintained at the same level at intermediate follow-up. (See Table 10.)
Summary of intervention effects
In summary, effect sizes indicated that child cognitive outcomes demonstrated the greatest change in the short-term; however, the size of these effects diminished over time. The more enduring effects were found on acts of delinquency and crime, with lower incidences of crime and delinquency among intervention participants. Most of the available effects on parent and family outcomes were negligible to small. However, it should be noted that few of the evaluations reporting effect sizes measured parent outcomes, and, in contrast to these findings, the Triple P program found large effects on parent outcomes.
Consistent with Benasich, Brooks-Gunn and Clewell (1992) and Brooks-Gunn (2003), who reported that most of the positive effects of interventions on child outcomes are the result of centre-based interventions, as opposed to home-visiting or case management interventions, the largest effects on child outcomes were found for intervention cluster 1 (where all programs were centre-based).
However, it is difficult to make any firm conclusions about the effects of the early intervention programs under review in this report, as many of the evaluations did not provide effect sizes, and the programs included in this review are not intended to be representative. In addition, it is acknowledged that the assignment of the terms 'negligible', 'small', 'medium' and 'large', to the size of effects is following standard definitions, but does not equal actual impact or value, and can ignore the inherent worth of particular outcomes and their value to society (see reviews by Boyle and Hertzman in Russell 2002). A focus only on medium or large effect sizes might miss a valuable outcome with a small effect size, when that is all that is needed to 'tip the balance' of health of a population (see Russell 2002).
Importantly, Brooks-Gunn (2003) notes that even a small effect size at a primary school aged follow- up is impressive, and an effect at adulthood even more so. The long-term follow-up of the Perry Preschool project reported here was conducted when participants were aged 27 years; approximately 22 years after participants had completed the intervention. To retain an effect, albeit negligible, for such a period of time is extremely impressive.
- This definition of long-term effects was adopted given the paucity of longitudinal follow-up in the early childhood intervention literature. However, our interests are really in effects that last throughout childhood and adolescence into adulthood.
- Although a comprehensive search was undertaken, it is possible that some effect sizes are available that are not included in this report.