28 July 2009
The National Evaluation of the Stronger Families and Communities Strategy
Professor Ilan Katz, Director and Professor, Social Policy Research Centre (SPRC) UNSW
Dr Kristy Muir, Senior Research Fellow and Evaluations Manager, Social Policy Research Centre (SPRC) UNSW
Dr Ben Edwards, Research Fellow, Australian Institute of Family Studies
Dr Matthew Gray, Deputy Director (Research), Australian Institute of Family Studies
Abstract, slides and audio of presentation
Edited transcript
The following audio presentation is brought to you by the Australian Institute of Family Studies as part of our monthly seminar series in which we showcase national and international research related to the family.
The seminars are designed to promote a forum for discussion and debate. They are open to the public and free of charge.
Seminar facilitated & speaker introduced by Professor Alan Hayes.
Professor Ilan Katz:
I would like to reiterate that this evaluation was a collaboration. It was a consortium event or process between SPRC and AIFS and that collaboration worked really well, very complementary strengths that we had. I would like to start off by thanking you for inviting me here this afternoon.
As Alan said, I'm going to start off by talking about the background and some of the methodology. Then we're going to hand over to the AIFS team and they will talk about the impact of communities for children which is one aspect of the program. Kristy will then go into a little bit more detail about why and how the effects worked and then I'll finish it off and then you can ask all the difficult questions you want to.
Talking about the background, this initiative; Stronger Families and Communities Strategy, came out of a realisation that early childhood - nought to five years - are critical for cognitive and social development and long-term outcomes for children. So research was emerging, particularly in the late '90s which really emphasised the importance of the early years for longer term outcomes for children.
The key elements in terms of children's outcomes were, on the one hand, parenting and parental inputs and, secondly, community and the impact on neighbourhoods and communities.
Subsequent to this there has been quite a lot of research in Australia (which Ben has been very active in) which has reaffirmed the importance of community and neighbourhood on children's outcome.
The other realisation, coming from the United States originally, was that disadvantage is often concentrated in certain geographic areas and therefore programs to help disadvantaged children might be best served by focusing on particular areas of disadvantage. Thus, so-called 'area-based initiatives' were established around the world, the biggest one was Sure Start in the UK and then Communities for Children in Australia, and there have been a number of similar programs in America as well.
The other push, was the need for evidence-based interventions and hence rigorous evaluations of these programs have been thought of as increasingly important. This is so that people can learn firstly what works but, secondly, some issues around implementation and process.
Just a little bit about the Stronger Families and Communities Strategy. Technically speaking we're talking about phase two of the Stronger Families and Communities Strategy 2000 to 2004 . There was a previous iteration but it wasn't focused specifically on the early years as this one was. The strategy consisted of three strands. First, the Communities for Children and that's what we are really going to be focusing on this afternoon. It was a four-year program, $100 million over those four years. A range of programs in 45 sites around Australia.
Another strand was Local Answers which provided small or medium grants to time limited projects 616, of them . Invest to Grow which were 26 early intervention programs and resources to help families, those were specific interventions and each one of those were separately evaluated.
Going back to Communities for Children, this is the logic of that program, so Communities for Children involved a facilitating partner which was an NGO which was given the money by FaHCSIA. The facilitating partner then facilitated the program in that particular area. They undertook an asset mapping of the area and looked for gaps in services and, secondly, set up a partnership which consisted of all the various stakeholders, NGOs and state services and federal services as well.
They then brought together these services and tried to plug some of the gaps and facilitate the services to work more effectively as a system. Alongside that, to do community development work so that the community as a whole became more child friendly and increase social capital within the community.
All of that, together with improved and higher quality services, more coordinated services and more child friendly communities were predicted to lead to better outcomes for children and parents. The evidence base for this is based on a range of evidence mainly from the United States but that whole logic had never been evaluated before. Sure Start was looking at something similar but when this was established the Sure Start evaluation had not reported yet.
I will just talk very briefly. I don't want to give you a methodological discourse about evaluation, but just to say what we actually did. Our evaluation had a number of components. I guess the most important one, in a way, was the outcome evaluation. In other words, that was looking at the impact of Communities for Children on children and families.
The main component of that was the Stronger Families in Australia study which is what Ben and Matt are going to talk about, so I won't go into details. But it was a study of 2202 families in 10 of those sites.
The second way we addressed outcomes was to use outcome indicators; ie secondary data such as hospital admissions, child protection data, breastfeeding data and crime data etc. This data is already available from agencies which we tried to pull together from those 45 sites around the country and then track changes in that data over time.
That was an extremely challenging thing to do and only relatively successful but we did make some progress and I'm not going to talk very much about that this afternoon. If you come to the Outcome Indicators Conference later on this year we will have a paper on that.
The process evaluation consisted of demographic profiles where we pulled together a range of data from various different sources in all those 45 sites, mapped as far as we could, all the services for under fives in those sites, surveyed local service providers to see to what extent services were coordinated and working together and the quality of services. We did that twice to look at changes in the coordination of services and then we had a partnership model study which was where we went into the 10 sites that the longitudinal study was done and did some intensive qualitative work to look at issues around service coordination and the quality of services.
We also looked at the progress reports. These were reports submitted by the sites to FaHCSIA about their activities and we did some analysis of that.
Finally, we looked at the costs and effects of Communities for Children and I'll talk a little bit about that further down the line. The overall evaluation, remember, wasn't only Communities for Children, it was of the Strategy as a whole. So the other components were we synthesised some of the evaluations of Invest to Grow. We looked at some of the data relating to Local Answers and then we had two components which were called Cross Strategy Evaluation. The first is the field studies Indigenous, Fathers and Hard to Reach and the reports of those are similarly available on the FaHCSIA website. So if you're interested in Indigenous, Fathers or Hard to Reach the reports are on the FaHCSIA website.
Then the Promising Practice Profiles (PPP) which, again, was AIFS' responsibility and I don't think we've quite produced all of them but there are 59 available now and so they are available on the Communities and Families Clearinghouse (CAFCA) website which is part of the AIFS website. That just gives examples of promising practice based on all those three streams.
Now, Ben and Matt are going to talk about outcomes for Families, Children and Communities.
Dr Matthew Gray:
Thank you Ilan. Can people hear me up the back? I've got the really boring part of this really, but I think pretty important. I've got to kind of talk about the methodology and how you go about evaluating something like this in order to try and demonstrate causation which is a really difficult thing to do.
We know that individual organisations and people are doing really good things. The question of how it works as a whole is really quite challenging to identify causation. I want to say that a lot of people at the institute worked on this report or worked on getting the information that led into this report. Daryl Higgins, Sarah Wise who is pretty instrumental in it all as, you know, Anglicare who couldn't be here today but was pretty involved. Helen Cheney and many others, so it's a good example of how you can get teams to work on this type of thing.
So I figured that I get to talk about the methodology because I'm an economist. What do they say, economists know the price of everything and the value of nothing? So I think that they'd rather talk about the outcomes for children.
I don't really want to talk about this. This has sort of been talked about already by Ulan and covered pretty well. This is really context, so this is work that Ilan referred to that Ben has done. This is data from The Longitudinal Study of Australian Children (LSAC) and it's the children who are four to five years old and we've just picked an outcome - socio-emotional development - and along the bottom we've got from the kids really in the most disadvantaged neighbourhoods according to the socio-economic status of the neighbourhood through to the most advantaged and you've got five groups with 20 per cent each.
What you can see in the age four to five is that the higher score means that the kids are doing better. The children in these areas are doing better in relation to the main and so this really illustrates the very strong socio-economic gradient that's occurring even at age four to five.
I suppose that these sort of area-based interventions are really trying to address this. These figures here take into account differences in parental education and a whole range of socio-economic background. So the independent effect of living in a disadvantaged neighbourhood and this is the same thing except it's learning, again, you see this very clear gradient.
Interestingly, here is not so much different at the bottom and the middle but the sort of higher neighbourhoods - the kids in the higher socio-economic neighbourhoods are going ahead.
As I alluded to in evaluating the impact of the Communities for Children initiative, which is one aspect of the broader strategy, is very challenging. One thing is the key action areas abroad, so they related to healthy young families which really we operationalise in terms of things like injuries, child physical functioning, emotional and behavioural problems, pro-social behaviour, overweight children and there's some outcomes for parents in terms of their physical health and mental health.
A second broad area is supporting families and parents which relates to parents' ability to parent well, you know, hostility or irritability, the parenting efficacies type of - you know, their ability to parent well.
A third area is early learning and care which relates to receptive - how the kids are doing in terms of learning outcomes and their ability to go on and learn through school. So receptive vocabulary, achievement, [unclear] and then child friendly communities which, again, is something which is quite [unclear] but you kind of know what it means when you say it, but how do you think about - you know when you see it but how do you kind of measure?
So there's things like the extent to which parents are supporting in raising children, community social capital, how parents feel about the neighbourhood as a place to raise children, the quality of playgrounds and they don't think it's safe, these type of things.
The second thing is that the communities are targeted with intervention. Often these intervention studies, you provide an intervention to a particular family and then your interested in how that family does compared to how well they would have done if they hadn't had the intervention and you do that in a variety of ways, control groups. In some places they randomise control and so on.
But because the community is the target of the intervention you're trying to measure community level outcomes. It's implemented in 45 areas nationally so very broad and because of the model which I think Ilan will talk about later, or Kristy - I'm not sure - one of the great strengths is that the content target [unclear] specific objectives differ at each CfC site. So there's this idea of this individual tailoring which then kind of makes it challenging to evaluate as a whole because while there are some similarities there are also some differences and we're looking at short run effects. So we were looking at, in terms of this, was it 18 months at maximum after intervention? So Ilan sort of talked about this but I'll just go to this again as it's quite important.
This is the model where you have the facilitating partners, who got funded, they were fairly large and generally sophisticated NGO organisations on the whole. They had CfC committee of local people and so on and they developed plans and managed funds and that led to funding of community partners to actually deliver services.
The logic of it all was to provide new services but importantly to increase service coordination and cooperation and the idea was that not only was this some new services but also that you can make much better use of what was there. So this is really a social inclusion type framework like all these services but how do you wrap that around the person, how do you coordinate service so that people get the services they need and also get them in a more coherent way.
So the idea of the model is - this is what we're trying to estimate really - that the model improved services. The funding and the model improved services at a local level and that does two things. For those who use the services it improves it has a direct impact on better outcomes for children and parents. But it also is designed to improve the general community strength, social capital, so that then has an indirect outcome on children.
So we don't actually try and distinguish the two because we can't identify those who actually access the services. It's kind of hard to even think about because people actually don't really quite know what services they used and who funded them and so on. So we can't identify that and so what we're doing is trying to estimate the combined effect of these things together.
So how do we do it? We know that the 10 sites where you have the intervention and when we want to find - so they're our intervention sites - and then we found five contrast sites which were matched to be sort of similar types of areas which didn't receive intervention. The idea is that in the absence of the intervention the sites where the intervention occurred would have looked like the sites where there was no intervention. So the idea is that you can try and use the experience of what happened in the contrast sites to infer what would have happened in intervention sites if you hadn't had the intervention.
We have identified the families in the areas as families receiving Family Tax Benefit Part A or B. That was chosen because at the time of the survey - that's tightened up a bit now but at the time of the survey there was almost complete coverage of families with children in these areas receiving Family Tax Benefit. In the end we interviewed 42 per cent of families with children within these areas. So it's a pretty high proportion, I would say.
One of the things about what we did is we were able to know which sites were going to have a CfC, but to go to them before the CfC funding came into effect and so that's our base line. So we got pre-data in wave one and then that was June to August 2006. The families and study child was two years old and we went to 2,200, spread across the 10 intervention sites and five contrast sites.
In wave two, March to July 2007, the children on the whole were about three years old, we interviewed 2,026. At this time the process was started in the communities but it's really kind of just at the early intervention stages so you wouldn't really expect to have been much of an impact. So it was either just pre or just post and then in wave three it was clearly post, February to end of May 2008. Kids were four years old and there's 1800 interviewed, so that's our post.
This is trying to illustrate what's going to underline a lot of the numbers. So here we've got a timeline, wave two, baseline or pre-intervention, wave two, wave three, post-intervention. Here we have got some hypothetical outcomes - child emotional development - higher score means better in this case.
So the solid black line you're saying - it's hypothetical - in that this is a contrast so there's no intervention and so in this case we've just drawn in so that kids have improved slowly over time on this particular measure. It could have been flat, going down, it doesn't matter. That's what was happening in the absence of intervention.
What we're assuming in this hypothetical example is the pre-intervention there's no difference between the intervention and the CfC site and the non-CfC site. So in the absence of CfC they would have just tracked along here. When the intervention comes along here and suddenly they start to do better, and so the difference between the two is the impact of intervention.
So the three are the same in the starting point. If we just go post intervention and look at the gap but there's also a concern that they're not. It's possible that there were some differences pre-separation in which case the method you use is called difference indifference and you can kind of take out the starting differences. I can talk a little bit more about that probably don't need to.
The real strength of what we're doing is that we've got longitudinal data, data on the same families over time and then, secondly, we've got the contrast site and the intervention site and, thirdly, we've got pre-data. For example, my understanding of Sure Start is that they have - they don't necessarily do longitudinal but they don't have the pre-data. So they've got contrast intervention sites but they don't have pre-data so they can't actually test whether they were different pre-intervention. It doesn't really matter but we use - I mean, it does matter but I won't go into details - but we use difference-in-difference or OLS regressions.
One of the things is that for a lot of the measures we have pre but some of them you could only really ask when the children got to age four. So you can't ask pre-intervention for them and also there were a couple that we decided later on we really should add. So for those, we've only got the post-data but we can kind of infer because there were no differences on anything pre-intervention. It was unlikely on these variables there were pre-difference intervention and so we can use this gap as the impact of the intervention.
Basically, to cut to the chase, we find that the contrast sites were a valid comparison. Actually there are few differences at wave one on anything and there were a couple that were different but once you take into account differences in parental education between the areas which were small but the differences disappear so we don't think that's a problem. We've got low attrition and we find that it wasn't systematic at longitudinal follow-up. We believe we've got an appropriate match comparison group and the interviewers were blind as to who received intervention. So when they go into interviews they don't know that this is a CfC site versus not, so it's not that they were kind of looking for positive outcomes.
Dr Ben Edwards:
Thanks Matt. I'm going to talk about the results. I presented some preliminary results in Canberra and the projector died so I just had to say, trust me, these are the results. This won't happen here because our battery won't run out.
Essentially, we've got a whole range of different outcomes, as Matt alluded to. All these outcomes have been assessed using valid and reliable tests that have been validated in the main using, through the process of The Longitudinal Study of Australian Children (LSAC) which is a national longitudinal study, Australia's first national longitudinal study of Australian children.
We've got a range of child outcome variables, so we've got child number of injuries requiring medical attention, child physical health, child emotional behavioural problems, pro-social behaviour, whether the child's overweight, parental health. So we've got a general measure of parental health and mental health. Then we've got some parenting, so hostile or irritable parenting or harsh parenting. The extent to which parents feel confident, so self-efficacy and we've got level of conflict within the parental relationship for those families that were couples. Also we've got whether the children are living in a jobless household. The sort of early learning and care things like receptive vocabulary, achievement in verbal ability, language skills, the quality of the home learning environment and then a whole range of community and social capital things like getting support in raising children from relatives, involvement in community service activities, whether the neighbourhood is a good place to raise children, the extent to which there's trust and social cohesion in the neighbourhood, the quality of facilities and whether there's any unmet services and needs.
Up the top of this table we've got DID, which is difference indifference, different types of estimation in wave three. That's either the OLS or logistic regression. Essentially what we find, these are significant effects. We don't find a huge number of positive effects. What we've done here, just to simplify things, is not show the numbers but rather show whether it's positive or negative. These are statistically significant at conventional levels so we see that the lower numbers of kids in CfC sites compared to contrast sites, living in jobless households, using both estimation procedures. At wave three children tend to be - their parents are reporting less harsh parenting or lower levels of hostile parenting and increased levels of parenting self-confidence or self-efficacy.
What we find also is that children tend to have lower levels of physical health. We were surprised by this and we spent much time discussing it and wondering why this might be the case. We have come up with a few hypotheses as to why this might be the case. The first thing is that parents are asked, does your child have a problem with a certain number of types of physical health things - physical health outcomes like walking, running and so on. In coming into contact with health services parents are going to become more aware of the issues with their children so that's our hypothesis as to why that's there. What we do find is that there are some health impacts.
So they're the overall effects. What I want to do now is just to compare them to Sure Start in the UK. If you not aware of Sure Start in the UK, there's been two evaluations. The first was after three years and they refer to that as the early impacts. So we have SS early impacts where children have been in Sure Start areas for three years.
Remember we're talking about 18 months, in our case. In Sure Start later impact, the children are assessed at age five but they've been in Sure Start areas for the whole of their lives. The directly comparable estimation procedure is CfC wave three which is the sort of dark brown bars.
Over on the far left, your far right and my far left, we've got the brown bar which is a little bit bigger. So this is the average of all the positive effects of CfC compared to Sure Start. What we can see is, if you use the directly comparable one is that if we summarise all those positive effects they're slightly better than Sure Start's early impact but not as good as later impact. That suggests that the effects are generally in the positive direction.
If we go over to your far left, looking at child injuries, the little negative sign on top suggests that there's a reduction in child injuries so it's not an increase, it's a reduction. There's obviously less of a reduction in Sure Start but if you go across I think you can see that they're generally in the range of the Sure Start effects. They're not huge but they're in a comparable level.
The child cognitive is their language skills outcome and that's a lot greater. What we would say, in general terms is these are relatively small effects which is what you would expect when you're talking about short-run effects of a community-based intervention where not everyone's receiving the actual intervention.
The general rules of thumb with these sorts of things, you can see in the main, we're talking about small effects.
Now, Cohen who termed these rules of thumb has been criticised because it's essentially just that. It's an arbitrary cut-off and really what you have to do is take into account the factors associated with the intervention so it's a community-based intervention. Not everyone's receiving it and so on.
We also looked at whether or not CfC had positive impacts on disadvantaged groups and we looked at three, in the main, because what the general literature on interventions finds is that children who are the most disadvantaged are at risk of poor outcomes but also they're less likely to benefit from both area-based intervention but also interventions more generally. They're more difficult to move up so, in some instances and Sure Start early impacts this is the case, the inequality was actually increased. So the more disadvantaged families they had worse outcomes relative to the more advantaged.
So we've got three different types of groups that we're studying. We've got the hard to reach which is essentially looking at a whole range of different factors that service providers often find difficult to access. When there's no father present and the mother and father are not employed, when there's low household income, a combination of these types of factors, straight out households with low income or where their mothers are with low education. What we can see if we look at the hard to reach is that we see the general pattern with child physical health, we see a positive effect in terms of reduction in harsh parenting, reduction in the proportion of those children who are living in a jobless household, a slight increase in parental general health.
Across the board in terms of the not hard to reach we see positive effects of CfC but an increase in unmet service needs by the not hard to reach in terms of reporting their services - the services that they need haven't been met.
If we look at another definition of hard to reach, the low education, we can see similar types of findings. We see that there's an increase in parental mental health in CfC for low educated mothers. We had a reduction in jobless households again and we had quite a large positive effect on children's receptive vocabulary and verbal ability. This is really quite high, so if you think about this in terms of effect size terms, we're talking about half a standard deviation. So we're talking about a medium effect size in Cohen's terms which is quite a significant finding.
We also see mothers who have low education also are more involved in community service activities. A similar type of pattern for the grade year 10 group.
We also see a reduction in child emotional and behavioural problems so there's some advantaged outcomes for children's social and emotional outcomes.
In terms of low income, a similar type of pattern, more involvement in community service activity and social cohesion and reduction in living in a jobless household but poorer general health for the parents, and a similar type of finding. So it's summarised that, because that's a lot of information.
We can see generally across the board these are just the significant effects so we're talking about short run effect and statistically significant effects where we're controlling for a range of other background variables. So a lot of socio- demographic variables we're also controlling for. What you can see from this slide is essentially positive effects across the board for parenting. For the not hard to reach there's a greater proportion who are reporting their service needs are not met. Living in a household with a job there's a greater proportion of those. There's more involvement with community service activities for the low education of mothers and in particular there's an increase in child receptive vocabulary for children in these households. There's some positive benefits for the not hard to reach in terms of fewer emotional behavioural problems.
It's a consistent finding with child physical health and some sort of smattering of lower levels of general health and also mental health for the parents' reports.
So this gives you a general sense that we think that on the whole you would say that there's positive effects. We would argue that the negative effect for health would dissipate with time as families are more engaged in the health service system.
What I also want to do is just show you a last slide which is sort of the general trends in the coefficients and these are not necessarily statistically significant effects. What you can see is by far and away the ticks outnumber the crosses in terms of the effects. So we're talking about generally positive effects, small effects which is what you would expect for a short-run evaluation of this kind.
So to summarise, we're talking about positive effects in terms of engagement and involvement for disadvantaged families, lower rates of jobless households, better parenting, the variability of children with mothers with relatively low education is better and fewer children experiencing emotional and behaviour problems for the non disadvantaged groups.
There are some small negative effects for child physical health and parental health particularly for the disadvantaged groups.
Now I will just pass it over to Kristy who will explain why.
Dr Kristy Muir:
Basically, as Ben said, my job is to talk about why we think some of the findings might have occurred; so under what kind of circumstances do we think the outcomes that Ben's talked to us about and that Matt's explained occurred?
Basically we're back to the logic model which both Ilan and Matt were talking about earlier. We can pretty much categorise the reasons into two changes: changes to the services and the Communities for Children model.
In terms of changes to services, there are three broad things that happened as a result of Communities for Children. One, we saw an increase in the number of services. Secondly, we saw a change in the focus and capacity of services and, thirdly, services changed in terms of the way they worked, which was all about service coordination.
In terms of the number of services there was substantial growth in each of the Communities for Children sites. According to reports by the facilitating partners, that growth was estimated to be about an extra 12 per cent in terms of the numbers of services that are available in those communities for children between zero and five years of age and their families.
In terms of a change in focus and capacity, it was largely about services in communities (and not just those that were particularly designed as part of Communities for Children but services that more broadly existed) focusing more on the zero to five age group and their families.
Communities for Children was largely all about having some universal services - services that everybody actually could go to in that community, which was enormously important in terms of decreasing stigma and making it [Communities for Children] open and accessible to everyone.
The other part of the change in focus was that services were specifically developed and tailored for people from particular groups, people who were otherwise missing out on services and families who might be normally considered hard to reach, people from cultural and linguistic diverse backgrounds, indigenous families, those from really low socio-economic families.
Finally, I guess, in terms of changes in regard to focus in capacity there was a real shift in terms of services in CfC sites engaging and recruiting families who previously haven't actually accessed services which is obviously enormously important. They were doing this largely by using soft entry approaches. That was a really common theme that we saw across a lot of the Communities for Children sites and what we mean by soft entry approaches is taking formalised services, like a child health nurse or a physiotherapist or a speech therapist, into really informal friendly safe environments where families just happen to be. So into places like playgrounds or playgroups.
The final change to services that was enormously important and that we saw as both an outcome and an important factor in creating those positive outcomes or the small positive changes for families and kids was a change in service coordination between agencies.
I'm just going to talk about that a little bit more. In regard to service coordination and collaboration we saw changes in two ways. We saw more people working together more often, which was all about collaboration. On top of that we saw that not only were they working together more often, they were working together more effectively.
So the first table shows you the numbers of people who were working together. You can see from the first row that in wave one when we surveyed people in 2006, only 35 per cent of service providers reported working closely together most of the time. By 2008 two-thirds of the sample reported that they were working closely together most of the time.
We know that people weren't just saying, "yes, we're working more closely together" because we asked people a whole range of different types of activities that they might have been doing together, like simple things like referring or shared training and the more complex things like joint planning and joint case management.
We saw improvements across a range of those types of activities and we also saw that people were working together more effectively when they were getting together and working together to support families. We saw this not just in terms of the survey but in terms of the 200 or so interviews we did in the 10 CfC communities where people were talking about this cultural change.
They were talking about a cultural change in the community between service providers in terms of how they thought about working together and talked about increases in trust and respect between service providers.
Obviously a lot of the changes have to be in relation to the services provided so I thought it might be good to just give a couple of examples of the types of services that were actually going on. These are just three that I've picked out and the Promising Practice Profiles will give you lots of other examples and there's also some more examples in the report.
The first picture is actually a legitimate picture of a Communities for Children site and that's a garden in Inala in Queensland. It was a garden that they put together at that particular site to attract families from culturally diverse backgrounds to inform them about nutrition, to try and increase connections between those families and then to try and introduce those families to other services.
The outreach nurse example is actually in Miller and the surrounds in New South Wales, but it was a kind of thing that was done across a lot of the sites as a soft entry approach, as I discussed before. It was basically an outreach nurse going into a playground where families would gather.
Then finally, HIPPY in Tasmania was all about teaching parents to be the first educators of their kids.
You will remember when I started, I was talking about there are really only two big categories of things that change in terms of the circumstances under which outcomes occurred. One was around services, which we've just dealt with, and I'm going to move on now to talk about the second factor - critical components of the model.
Basically there were three important issues in regard the the model – money, the facilitating partner and the community focus.
In terms of funding, Ilan is going to talk a little bit more about the costs in a moment and so I'm not going to talk about that. What I do want to say is what was really important about Communities for Children, which was quite different to other types of services and government initiatives, is that funding was not only available for services and the facilitating partner but also for the coordination of activities. Having funding put directly into service coordination helped make CfC successful.
Interestingly, even though there was some competition and some people were a bit upset when money was originally announced, the facilitating partners and the community partners liked the funding model. They liked it better than direct funding because it was a more flexible and community based.
I will just talk quickly about the Facilitating Partner. The Facilitating Partner was enormously important in terms of their role in developing and implementing Communities for Children. They were responsible for starting out and conducting the asset mapping. They did an enormous amount of work to facilitate the broad program but also the coordination between services. So we know they're a major strength but they're a major strength when certain things happened. Largely, they were really strong when they were well known in the communities in which they were based and when they were agencies with lots of other supporting infrastructure.
So it [CfC] didn't tend to work well when agencies were flown in and weren't well known in that community. As with any program, the skills and qualifications of the staff were really critical.
Facilitating Partners did have some challenges in terms of recruitment difficulties and staff turnover and there were also some challenges around engaging state government departments. I won't explain that now but you can ask me in question time because I think we're going to run out of time otherwise.
Finally, the other critical component of the model is all about the community focus and Matt mentioned that that was an important part of the logic model. It was important because they did go into communities: they looked at what communities needed; they responded to what those community needs and gaps were; and they consulted with people in those communities. Consultation was important across a lot of the communities but particularly so where there was a substantial proportion of Indigenous Australians.
The community focus worked well when there was a diversity in membership of the CfC Committee; where they had regular meetings, where they had effective communication and decision-making was done in a collaborative way. There were particular challenges in regard to short timelines which feeds back into this issue that consultation was really tight in sites with lots of Indigenous families and there were also some problems in trying to engage certain groups of people in a particular community.
Ilan will now talk about the costs and effects of CfC.
Prof Ilan Katz:
Thanks Kristy. So now we know that it worked, number one, and we know a little bit about why it worked and so the third question is, was it cost effective? We couldn't do a classic cost effectiveness analysis which is when you compare a number of different interventions for a particular outcome, nor a cost benefit analysis, where you estimate over a long time, how much is saved in terms of savings on crime etc.
So what we did do is to consider the outcomes that we've got and then compare them to similar programs that Ben was talking about and to assess whether the effects from those programs were similar and then whether the costs were similar.
I won't go into detail but obviously, if you look at it, the two programs we compared to were Early Headstart which is the flagship American program and Sure Start which is the UK program. Early Headstart is not an area-based initiative but it is focused on a similar group of people. You can see that Early Headstart an annual unit cost of the equivalent of $15,000. Sure Start per year is just over $1500 and Communities for Children is $210.
The outcomes were reasonably similar. So obviously we're looking at a very, very cost effective program. There are some caveats though in that these programs are very different. Sure Start, for example, also financed buildings which Communities for Children didn't do and they had a different funding model. Having said all of that, clearly we're talking about 10 or 12 times the cost for these other programs.
So for outcome measures that were the same across all these programs CfC had similar or more positive outcomes than Sure Start, early outcomes and varied outcomes compared to Early Headstart with the exception of health which Ben has discussed .
Moving on swiftly to the broad challenges Staffing, Kristy already mentioned, that was a challenge and it's always a challenge for under fives services but particularly for short-term funding. By the time the middle of the third year funding comes people are already looking for other jobs.
Second, what we call territoriality, which basically means that different agencies were fighting over their own territory and this took energy, in some areas, to bring them all together. The three to four-year timeframe, we've mentioned several times, was a real challenge. If there's one message for bureaucrats or funding agencies about early intervention is that three-year programs just aren't effective. You really need stability of funding.
Geography - some of the sites were pretty weird if you looked at them and there were fairly arbitrary boundaries, based on, for example, postcodes or whatever which didn't really make sense to the community. Another barrier concerned social issues like housing, domestic violence etc which obviously prevented some families from receiving services because it's a voluntary intervention. You couldn't force families to access it. Some of these more severe social problems prevented them from receiving the services.
So, very briefly, the conclusions are that the evaluation has demonstrated that area-based intervention can work in Australia. It's a relatively low cost model, it works if it's under certain conditions as appropriately funded. You've got to have skilled, motivated staff. It has to be community focused and the programs have to address the communities and fill the gaps etc. Then you have to have this mix of universal and targeted programs.
So that's all we have to say. Just one last issue is, I think this presents quite a challenge for manualised so-called evidence-based programs because there was such a diversity of different interventions here and yet even so, there were positive outcomes at a relatively low cost. So it does make you think about what sorts of interventions might be effective for this age group. So that's all, and thanks.
-- end --
IMPORTANT INFORMATION - PLEASE READ
The transcript is provided for information purposes only and is provided on the basis that all persons accessing the transcript undertake responsibility for assessing the relevance and accuracy of its content. Before using the material contained in the transcript, the permission of the relevant presenter should be obtained.
The Commonwealth of Australia, represented by the Australian Institute of Family Studies (AIFS), is not responsible for, and makes no representations in relation to, the accuracy of this transcript. AIFS does not accept any liability to any person for the content (or the use of such content) included in the transcript. The transcript may include or summarise views, standards or recommendations of third parties. The inclusion of such material is not an endorsement by AIFS of that material; nor does it indicate a commitment by AIFS to any particular course of action.