Research report no.14 2005

The efficacy of early childhood interventions

by Sarah Wise, Lisa da Silva, Elizabeth Webster and Ann Sanson

12. Lessons for Australia

New findings from developmental neuroscience, and growing evidence from longitudinal studies have indicated that children's experiences in early childhood provide an important foundation for subsequent development. There has thus been increased interest in the potential for early childhood interventions to ensure children start life on a positive developmental pathway, particularly those children whose family background might indicate problems in the sensitive formative years.

This report focuses attention on the potential for early childhood interventions to produce returns on public investment in the long run. It reviews selected early childhood interventions to examine the effect of these programs, carefully considering intervention design, implementation and evaluation rigour. It establishes the conceptual framework within which program costs and outcomes can be understood, evaluates cost-benefit methodologies, and reviews published estimates of costs and benefits of applicable early childhood interventions.

This section summarises findings about the efficacy of early childhood interventions for improving outcomes for children and the relative cost-savings potential of different early childhood intervention programs. It concludes with recommendations for conducting cost-benefit analyses of early childhood interventions in Australia.

Are early childhood interventions efficacious?

While this review provides a basis for estimating likely future benefits of early childhood interventions, it is not a comprehensive study. The dearth of evaluation data on interventions generally, and missing data on the restricted and unrepresentative number of interventions in this review, makes it impossible to comment on the usefulness of early childhood interventions as a general strategy to sustain improvements for children in the long-term.

Examination of 108 large-scale, public early childhood interventions from around the world revealed relatively little empirical data on program effectiveness. Indeed, of the 108 interventions identified in the current review, only 32 interventions had a strong evaluation component, including only three interventions developed and currently operating in Australia.

In an attempt to identify the most effective 'type' of early childhood intervention, programs were grouped into five clusters according to the availability of the intervention, the intended effects of the intervention, where the intervention took place, and the focal age of children targeted for the intervention.

On balance, the interventions produced a number of important improvements across a wide range of outcome domains. The greatest improvements were observed in respect to children's cognitive skills, and child outcomes in general, with parent-related outcomes showing the least improvement (studies reporting effect sizes on parent and family outcomes were in the negligible to small range, although the Triple P program was an exception).17

Most of the positive effects on child outcomes were the result of centre-based interventions, as opposed to 'home-visiting' or 'case management' interventions. These interventions were grouped in cluster 1, which included programs like the Perry Preschool Project and Head Start. This is most likely a testament to the fact that cluster 1 interventions were consistently superior in terms of key elements of design and implementation quality such as dosage, intensity, participation rates, 'drop-out' rates and program integrity. By contrast, there was great variability in design and implementation adequacy within cluster 4 (targeted, holistic interventions, such as Sure Start) and very little information was available on interventions in cluster 5 (universally available programs, such as Triple P), which made it difficult to comment definitively on these interventions as a group. It may also be true that more intensive effort is required to achieve substantive change in parentrelated outcomes, such as parenting skills and social support, than what was offered by the current interventions.

Although the review of early childhood interventions reviewed here is not representative, it supports the case for well-designed, well-executed and high quality interventions. Differences in benefits observed across the programs reviewed here may in fact relate to differences in program quality and funding.

The measured effects of early childhood interventions were mostly limited to the immediate and short-term. Reductions in acts of delinquency and crime (which are easily measured) were the most enduring intervention effects reported. However, only 13 of the 32 reviewed interventions (40.6 per cent) followed up participants for more than two years, and the Perry Preschool Project stands out as the only intervention to collect comprehensive evaluation data on participants into adulthood. Impressively, the adult follow-up of participants in the Perry Preschool Project, collected after 22 years when participants were aged 27 years, showed positive effects on aspects of intellectual ability as well as income and employment outcomes in adulthood.

It is also possible that interventions produce different effects at different developmental stages. Effects that disappear after a few short years may in fact re-emerge at a later developmental stage, showing what is known as a 'sleeper effect' (for example, initial gains in cognitive and language performance following experience in centre-based child care may 'fade-out', only to re-appear at entry to school, for example). Interventions that do not conduct lengthy follow-ups could in fact be underestimating intervention effects, or incorrectly reporting diminishing effects over time.

Although it is natural to consider benefits in terms of outcomes that an intervention was designed to produce, gains from early childhood intervention may also occur beyond the domains measured in an evaluation. It is instructive that when Head Start began, for example, it was primarily concerned with enhancing cognitive performance. Later evaluations have seen this intervention as also contributing to positive early moral developmental and language regulation (Emde 2003: 8). Moreover, program effectiveness is often determined in terms of outcomes that are easily measured, such as acts of crime. Less tangible effects - the capacity to sustain functional relationships, as one example - may fall off an evaluator's radar simply because of the complexity (and potentially cost) of measurement.

The need for longitudinal study after an early childhood intervention is clear. This is important to understand what is needed to sustain and enhance intervention effects, how long programs should last, and to appreciate possible influences of program participation on later stages of development.

Do early childhood interventions have long-term payoffs?

Very few sound cost-benefit and cost-savings analyses of early childhood intervention programs with long-term follow-ups have been conducted. Of the 108 interventions that were initially identified, only eight interventions included a cost-benefit study. With the exception of a cost-effectiveness study of Triple P, there have been no cost-benefit analyses undertaken of Australian interventions, making it difficult for government to decide objectively on how much funding to allocate to these interventions vis-à-vis other social and economic expenditures.

There is evidence, however, that early childhood interventions can produce potential returns in public investment. Although it is not possible to generalise these findings, among the early childhood interventions with a cost-benefit analysis reviewed here, programs that involved children as program participants, or that focused on improving parenting skills or levels of parenting support, produced a greater return on investment than interventions that focused on family economic circumstances.

Planning a cost-benefit analysis of an Australian early childhood intervention

Clearly, much more Australian data is needed on interventions in early childhood to determine their effects and benefits in this context. The review of cost-benefit studies provided in section 11, combined with information about the process for undertaking a cost-benefit analysis and estimating program costs and benefits is instructive in this regard. What follows is a summary of the important steps in planning a cost-benefit analysis of an Australian early childhood intervention.

There are a number of general principles that may be used by decision-makers considering costbenefit analysis of an early childhood intervention program, which may need to be tailored to the specific circumstances of a given intervention and its evaluation design.

Ideally, an evaluation should be planned at the same time as the intervention is designed to enable random assignment and the cheapest form of data collection. There are four parts to a cost-benefit analysis that are to some extent conducted separately:

Before the evaluation is designed three things need to be established: first, the intended benefits of the intervention (for example, educational, crime related, employment, social engagement); second, the target population; and third, additional factors that may affect outcomes other than the characteristics of the target population (for example, if the target population is children from low income households, then other correlated but intrinsically different factors may be the parents' refugee status, parents' history of substance abuse, age of parents, relationship status of parents, parents' criminal record, peer group etc.).

Data can be collected from repeat surveys and administrative records on participants. Pecuniary values for non-pecuniary costs and benefits (crime, loss of health, unemployment) are usually derived from secondary literature, which has made these estimates.

Selecting the intervention and comparison groups

Setting up the evaluation ideally requires the inclusion of a comparison group that is similar to the program group demographically and/or on relevant pre-tests. There are many ways to build a comparison group, with random assignment18 of the target population (children from low-income households, parents with substance abuse, parents with long term unemployment etc.) typically viewed as the best way of ensuring that intervention and comparison groups are equivalent initially.

In the case of child participants, parental consent is required for both participation in the intervention and for ethics approval for the collection of administrative data (such as school and government records). This requirement for approval will introduce a bias in the selection of children into the intervention which ideally should be accounted for though the regression analysis.

After random assignment, including parental consent, children and parents should be surveyed to ascertain: first, the background characteristics of the family with respect to different types of disadvantage that may impact on the child's social, educational and psychological development; and second, depending on the age of the child, any developmental assessments of the child before the intervention has begun.

Rather than defining comparison groups through random assignment or some other means, researchers can attempt to estimate the effects of participation in an early childhood intervention through the use of random surveys. Ideally, specific information about program participation should be collected. In the absence of specific program participation information, random surveys can be used to compare how well program participants fared compared to a population group.

Collecting data on intervention costs

Usually only the running costs need to be recorded. Fixed costs associated with the establishment of the intervention are only relevant if they will be incurred every time the intervention is extended. Fixed costs associated with the design of the intervention that are one-off are not relevant.

It is rare for the costs of the intervention to extend beyond the intervention period. If they occur, they will in most cases be revealed as negative benefits and will be monitored through the benefits section.

Collecting data on intervention benefits

Benefits should be measured over time through surveys and the collection of administrative data. Relevant survey measures include educational achievement records, school retention, employment history, incidence of criminal record, social and health problems (substance abuse, social dysfunction). Administrative records can supplement survey data (for example, school achievement records, social security records, and so on).

The frequency of the surveys and administrative data collection depends on funding for the evaluation and the occurrence of critical milestones in the child's development. The latter may include the start of primary, secondary and tertiary education, age 18 or age 21 years.

Conducting PV calculations

Present value calculations can be calculated at anytime after the intervention has ended although the earlier the evaluation the less clear the results and the more we are required to rely upon conjectures about how early lifecycle indicators map into later outcomes.

It is advisable to calculate present value for several rates of discount to make the sensitivity of the result to variation in the rates apparent.

Conclusion

Most of the evaluations summarised in this report are of good quality, although weaknesses were noted across a number of aspects of program design and implementation (notably attrition from the program), thus some interpretation of evaluation findings is required. Nevertheless, evaluation findings suggest that early childhood interventions can produce improvements across a wide range of outcome domains. There is also some limited evidence that early childhood interventions can produce potential returns in public investment.

Unfortunately, however, no evaluation can demonstrate that a program that worked well in one setting will have similar positive results when adopted in a new location. Thus, evaluations that are conducted in the Australian context are essential to understand the potential benefits of early childhood interventions undertaken here. Ideally, an evaluation should be planned at the same time as the intervention is designed to ensure methodologically strong evaluations that will support cost-benefit analyses and other evaluative endeavours.


  1. Evaluation findings need to be interpreted with the consistency and dependability of the measurement in mind. The reliability of evaluation measures is presented in the review of programs in Section 5 of this report.
  2. Random assignment and matching methods do not in themselves ensure that families in the intervention and comparison groups do not differ from one-another in unmeasured ways.

Research Report 14: Contents | References | Previous