While changes in management approach within the public sector may create a greater role for research, it is proposed that a number of barriers exist which restrict the extent to which research can inform decision making at all levels within organisations and in government decision making generally. Therefore, if research is to have an increased influence then strategies which address the identified barriers should be developed.

In this paper policy will be used in a broad sense to cover decisions made at all levels of an organisation which concern choices in direction or approach. Thus it will include decisions made which influence the overall direction of government or an organisation to the decisions which are made at the practitioner level as to the practice approaches which will be adopted.

In order to determine what the barriers to the more effective use of research may be at the organisational level, unstructured interviews were conducted with staff at various levels and with different responsibilities in the South Australian Department for Family and Community Services. The barriers which exist at the broader government level in Australia were identified using a recently published Senate report on establishing a set of national indicators of well-being.

A FOCUS ON RESULTS

Integral to changes in public sector management has been the move towards performance measurement of government provided and funded services. The Council of Australian Governments (COAG) agreed that in attempting to clarify the roles of Commonwealth, State and Local Governments in regard to service delivery, the overriding goals should be 'to improve outcomes for clients and value for money for taxpayers'. This is in line with international trends towards 'results -oriented service delivery'. The Steering Committee on Commonwealth and State Service Provision (SCCSSP) was established to address this issue. (SCCSSP, 1995:4)

In this committees view, central to achieving these goals is understanding how well existing services perform and the extent to which they meet the needs of consumers. The benefits cited for clients, governments and taxpayers of a better understanding of performance, include: more relevant responsive and effective services, greater cost effectiveness and a greater capacity for tax payers to assess the value for money they are getting.

Additionally the use of performance measures is to increase transparency and therefore strengthen accountability. The Steering Committee argued that this performance information should be concerned with what was actually achieved, that is with actual outcomes, and not just with how much is being spent or how many people actually use a service. By focusing on the actual impact of services, inequities in access and outcomes for disadvantaged clients can be identified and addressed.

The development and monitoring of performance indicators were considered by the committee as a means of clarifying the objectives of service delivery and contributing to a clearer delineation of Commonwealth State and Territory and Local government roles and responsibilities.

A further benefit of a move to performance monitoring is in regard to the ability of this information to assist the improvement of services through yardstick competition by which is meant the ability to compare programs with similar objectives within the same jurisdiction, across jurisdictions, or between different modes of service delivery.

While it is acknowledged that making these comparisons is not straightforward and caution needs to be exercised, yardstick competition was considered as a means of identifying best practice and trends in service delivery.

Efficiency and effectiveness, the components of an assessment of cost effectiveness are included in the framework adopted by the Committee for the development of performance indicators. 'Efficiency is about producing more services and/or better quality services with existing or less resources.' 'Effectiveness is aimed at achieving objectives, which requires measurement of outcomes.' In commenting on outcomes the committee stated 'The lack of available information on outcomes (the impacts of the service on the client group) is a feature of government service delivery in Australia.' (SCCSSP, 1995:10)

It is this lack of information on outcomes, or information about how services actually affect consumers which creates a need for increased research efforts focused on consumers and their views. A further impetus for consumer oriented research derives from the fact that a stated desire to be more consumer focused provides part of the rationale for the introduction of market type mechanisms into public sector management. Therefore, knowing what the consumer thinks about the service, and how it actually impacts on him/her should receive increased research attention.

In the following part of the paper the barriers which exist in regard to using research to inform policy and improve outcomes will be discussed. The issues discussed are not meant to be exhaustive, rather they reflect the issues raised by staff at various levels of the Department for Family and Community Services in South Australia.

At the broadest level, the most significant barrier to using research for policy purposes what Rein has characterised as the incompatibility of the research role with the political role. Politics he sees as trading in values, research as trading in facts. The politician is searching for a course of action which will keep a range of interests happy, the researcher on the other hand is ostensibly concerned about the truth. (Rein M., 1983:216)

In addition to the philosophical incompatibility of politics and research there are practical barriers to the extent politicians are able to employ research findings. The immediacy of response expected to problems raised by the media frequently precludes a response at least in part based on research. The differing time frames which operate in policy making and research is not only problematic to the politician, but also to the public servant faced with short turn around times in providing advice on complex issues. At the same time the resolution of specific issues was often seen to require specific research which was not available.

For those issues which are the almost exclusive domain of the states such as child protection, the fact of each state and territory having a differing legislative framework and consequently different systems of service delivery, presents a range of practical barriers to the efficient production of nationally useful research on this topic. The scattering of existing and potential research capability fragmented between all the States and Territories means that the best value is not necessarily obtained for the research dollar. Policy practitioners also cite the fact that research undertaken in the larger eastern states is not necessarily applicable in the other States.

A substantial amount of data is collected nationwide in regard to child abuse and neglect, however the ability to use the information on other than a state by state basis is compromised by the differences in data collected by each jurisdiction. The Australian Institute for Health and Welfare have substantially improved the situation however the lack of comparability hampers research efforts.

Resource constraints were cited as resulting in reduced resources within the organisation which were specifically committed to research. In the past specific staff positions were allocated to research functions. In the current financial year no funds are specifically allocated to research and funding for this activity is reliant on savings being made in other areas. This may reflect the priority given to research in comparison with direct service delivery at a time of resource constraint.

At the practitioner level, the demands of service delivery were seen to preclude practice oriented research designed to improve outcomes for clients. Given the need to specify outputs and outcomes in performance monitoring, better information on what social workers actually do and what they actually achieve is needed to develop accurate performance measures.

At the same time improving outcomes requires the systematic comparison of different approaches to practice and therefore better research on what is actually happening at this level. As noted by the Committee on Government Service Provision information concerning the actual impact of services on clients is very limited in Australia. Gaining information from clients about their experience of service delivery is one way of contributing to the information available on the impact of services.

A lack of availability of systematic evaluations of programs was considered a barrier to improved decision making. It is interesting to note that McGurk and Tomison state that in regard to the prevention of child abuse and neglect that although evaluation of programs has increased in Australia, the propensity of many funding bodies to only fund pilot projects, or projects of limited duration prevents the use of evaluative information in improving services.(Tomison A and McGurk H, 1996:30)

Therefore at the organisation level a range of barriers exist to using research to inform policy and ultimately lead to better outcomes for clients.

Performance indicators such as those currently being developed by the Committee on Commonwealth State Service Provision should provide much useful information on the efficiency and effectiveness of particular program areas such as child protection, housing, and the operations of our hospitals.

As they are focussed on the performance of services they will improve the information available to us about the impact of these on individuals and target groups, however they will not provide information on how well the community at the local or national level is doing in terms of well-being or their quality of life.

The level of well-being or the quality of life experienced in a particular locality, whether this is a neighbourhood or a nation can not be directly attributed exclusively to government policy, particularly in these days of the globalised economy. Assuming that improving the well being of the people is a goal of government, information concerning outcomes for the community at this level would enhance the information available to those concerned with broad direction in policy at the macro level. The second part of this paper will deal with the barriers that exist to the collection of this information.

Earlier this year the Senate Legal and Constitutional References Committee (SLCRC) reported on its terms of reference which were to inquire into:

The desirable structure and process for a system of national indications and benchmarks capable of

(a) measuring at regular intervals the extent to which legal, economic, social and cultural rights and responsibilities of Australian citizens are implemented

(b) comparing Australian levels with international best practice, and

(c) providing a framework for defining and meeting desirable standards.

Part of the rationale for the committees work was the pre-emanence of economic statistics and indicators as a basis for political and community debate. The Committee stated 'One of the motivations behind this inquiry is that there is presently considerable focus on the economic well-being of Australian citizens with no corresponding level of interest in their legal, social and cultural well-being', a situation which they noted was not unique to Australia.(SLCRC,1996:2.)

While the availability of economic indicators and statistics is obviously necessary the committee considered it problematic for the following reason:

'The preponderance of economic indicators virtually to the exclusion of social indicators has been seen to be destructive on a number of levels. First, economic indicators can become proxies for social indicators, a role that they were not designed to fulfil. A further and more systemic criticism is that excessive reliance on economic data in public policy can lead to a narrow economistic outlook which eliminates alternatives and areas of concern due to the lack of appropriate indicators'.(SLCRC, 1996:3)

The skewed nature of available information thus has the potential to limit public policy debates. It thus becomes a significant barrier to the development of more responsive policy and the achievement of better outcomes for the Australian population.

In considering the desirability of developing a system to measure and monitor national well-being the Committee cited a number of significant barriers to the achievement of this aim. At the international level there appears to be a degree of consensus on the structure of economic indicators which is lacking in regard to social indicators. Given the longer term aim of having a system of information which can be used to develop benchmarks which allow for international comparisons in particular areas a lack of consensus presents a formidable barrier.

The concept of well-being and the complexities associated with the interrelationships between the component parts provided a significant obstacle. The Committee noted that 'there does not appear to be any single comprehensive definition of well-being' and that the choice of particular indicators will ultimately reflect value judgements and therefore be subject to political debate. (SLCRC,1995:21)

Being explicit in regard to the goals or values which are implicit in indicators was seen as necessary if they were not to be easily dismissed as missing the mark. The Australian Institute for Health and Welfare submitted that the meaning of terms such as well-being have a particular context and that 'at no time could the meaning of these terms be said to be fixed; rather it remains a site of contention between different political ideas, social philosophies, disciplinary paradigms and social practices.'(SLCRC,1996:43)

Once again, the lack of nationally consistent data in important areas presented a major problem, although significant work is underway to standardise data collections.

The connection between indicators and benchmarks, or the setting of agreed standards in particular areas was considered particularly difficult, with the setting of benchmarks in contentious areas ultimately the result of the political process. Benchmarking, the Committee suggested should grow out of a process of collecting and analysing indicators in terms of the trends they reveal. Trends will create expectations which will assist in the definition of benchmarks.

While significant barriers exist to the development of a set of national indicators reflecting well-being, the Committee identified even in the event that a systematic collection of indicators can be developed, they are not of much use if decision makers do not use them. Based on research undertaken in the United States in the 1970's about the use decision makers made of a significant collection of indicators, it was concluded that in the absence of an explicit organising principle such as a shared set of clearly defined national goals the indicators are of little value.

They stated '...the non impact...demonstrates that information without specific relatedness to a shared set of goal commitments lacks policy shaping power because the data are not seen as relevant to the needs of policy makers.' In order to be useful the researchers concluded that a number of preconditions were necessary including: the deliberate setting of national goals, the institutionalisation of commitment to those goals throughout government, agreement to use specific social indicators for the purpose of evaluating progress, and the establishment of bureaucratic arrangements with the capacity for legitimising the importance of the indicators as a source of information. (SLCRC,1996:73)

Therefore, unless the indicators were enmeshed with the workings of government and institutionalised as part of the decision making processes, chances are they would not be used.

The barriers discussed above in relation to the development of a national set of indicators regarding well-being and its consequent use in policy are considerable and may appear insurmountable. That change of the magnitude required for the development of an ongoing set of indicators and benchmarks reflecting the well-being of a community can be developed is demonstrated by the Oregon Option. This project was cited by both the Senate Committee and the Committee on Government Service Provision as a systematic approach to the ongoing monitoring of a comprehensive set of benchmarks designed to reflect the outcomes of government activity.

Arising in part from a desire to base its funding relationship with the Federal Government on more equitable grounds the Oregon Option aims to base its funding on the achievement of mutually agreed and measurable outcomes for public service delivery. The benchmarks have been developed in three areas, benchmarks for people which includes a range of benchmarks on family life, benchmarks for quality of life and benchmarks for the economy. The project arose out of a statewide strategic planning process in which three broad goals for the state were identified. These were increased jobs and incomes creating a diversified, productive economy; protect and enhance Oregon's quality of life; and invest in the capability of Oregonians. 272 benchmarks were developed to measure progress towards these goals. (SLCRC,1996:99).

The benchmarks are used to inform policy and budget decisions and are regularly publicly reported on. A critical factor in the apparent success of the Oregon approach would appear to be the public nature of the process used to develop the indicators and the fact that a broad consensus appears to exist between the government, the community and non-government organisations and the business sector in regard to the benchmarks which are used by a wide range of organisations for planning and funding activities. They have been developed with a long term view and with clear publicly supported goals in mind. The Oregon approach demonstrates that the barriers to making more effective use of research can be overcome in the interests of better outcomes for people.

Before concluding I will briefly discuss the South Australian Quality of Life Indicators Project, a modest attempt to improve outcomes for South Australians by monitoring a range of social, environmental and economic data.. Still in its early stages the project has overcome some of the barriers to more effectively using a comprehensive range of information about the state of the South Australian community, economy and the environment to inform policy decisions.

At this stage only statewide data has been collected however in a second stage regional data will be collected to provide information on the differences which exist between regions and hopefully enhance decision making. Ultimately it is hoped that South Australia can be benchmarked against similar Australian and International communities. At this stage it is proposed that the initial set of information collected will be released for public consultation with the aim of achieving agreement on the issues which should be tracked and hopefully on the longer term goals which should be set.

The project can not claim to have overcome the multitude of methodological problems which are inherent in a project of this type. The availability of time series data in the environmental and social areas has limited the range of issues which can be included.. In contrast a substantial amount of economic information which meets the criteria for inclusion is available. Given the need for a medium to long term time frame so that trends can be identified, ongoing political support will be required. This of course carries risks for any government as the information will highlight weaknesses as well as strengths.

In order to ensure that the information generated is of use to decision makers it is planned that the process will be integrated into the Government Management Framework which is currently being developed and aims to institute an outcome based budgeting process for South Australia.

This paper has aimed to demonstrate that a range of barriers exist at both the organisational and whole of government level to effectively using research in policy making. Given the current emphasis on outcomes and a consumer focus in public sector management, attention to addressing these barriers would appear to be necessary if Australians are to get better results from governments.

REFERENCES

Rein M., From Policy to Practice, M. E. Sharp, Inc. Armonk, New York, 1983.

Report by the Senate Legal and Constitutional References Committee, National Well-being: A system of National Citizenship Indicators and Benchmarks, Commonwealth of Australia, Canberra, 1996.

Steering Committee for the Review of Commonwealth/State Service provision, Report on Government Service Provisions, Australian Government Publishing Service, Canberra, 1995.

Tomison A and McGurk H., Preventing Child Abuse, A discussion paper for the South Australian Department of Family and Community Services, unpublished paper, Australian Institute for Family Studies, Melbourne, 1996.