Australian Institute of Family Studies - Staff paper

 



'Blending qualitative and quantitative approaches: case tracking studies in child protection systems'

Adam M. Tomison, AIFS, and Chris R. Goddard, Family Violence Unit, Monash University


Paper presented to the Association for Qualitative Research 'Issues of rigour in Qualitative Research' International Conference, 6-10 July 1999, Melbourne.

Case tracking studies enable the development of a rich picture of the realities of child protection case management via the collection of both qualitative and quantitative in situ data. They are easily adapted for a variety of mixed method designs, methodological and/or data triangulation. The present paper discusses the methodological and theoretical benefits which result from utilising case tracking methods as a means of investigating professionals’ case management of child protection cases in situ. It also identifies and explores some of the problems which have confronted the authors in a number of large-scale case tracking studies which have been conducted in Victoria, Australia. The authors use their experience of conducting such studies to describe some systemic and methodological problems which can significantly affect the design and overall success of such studies.

WHAT IS CASE TRACKING?

Case tracking studies are essentially descriptive research tools which combine elements of interviewing, direct observation and archival analysis. They involve an attempt to follow or ‘track’ cases of interest within an organisation, or between organisations in a particular professional or social network (e.g. within the various departments of a hospital or welfare department; between a hospital, other medical settings and other non-medical agencies). As such, case tracking studies enable researchers to trace the connections between actors within the context of their involvement in the ‘cases’ or phenomenon of interest. Case tracking studies are therefore examples of social network analysis (Scott, 1991).

In the late 1980s, Goddard began a series of classic, mainly hospital-based, studies where children confirmed as having been maltreated, were ‘tracked’ to determine the respective roles of police units and child protection services (e.g. Goddard, 1988; Hiller, Goddard & Diemer, 1991; Goddard & Hiller, 1992; Goddard et al., 1996). The studies typically involved archival analysis of case records in combination with semi-structured interviews with workers and/or participant observation.

In most cases an attempt was made to collect data on the case demographics (i.e. child and family characteristics, definition of the type and severity of the maltreatment), identification of the professionals who had been, or were still, involved with the case, the procedures that had been followed by the workers involved with the case and the worker’s perceptions of the significant aspects of the case management process. Often the resultant data was utilised in qualitative and quantitative analyses: analysed quantitatively, using descriptive statistics (e.g. Chi-Square and measures of association), and qualitatively via case studies or cross-case analysis.

TOMISON’S STUDY

Tomison (1994) extended the application of case tracking by carrying out a six month case tracking study in an attempt to map one complete child protection network and to evaluate the decision making of various professionals involved in the management of suspected child abuse and neglect cases. The specific aims of the research were:

The base sample consisted of 110 professionals who worked in the major health/counselling and investigative agencies in the region. These workers played a significant role in the reporting, assessment and/or alleviation of child maltreatment, and all agreed to participate in the study.

The professions were: police officers (from the regional Community Policing Squad, Victoria Police), child protection workers (Department of Human Services Victoria), paediatricians, social workers (from all major treatment agencies and the regional hospital), school medical and counselling personnel, and infant welfare nurses.

Procedure

Suspected cases of physical, sexual, or emotional abuse, or neglect, identified by any of the participating professionals were tracked during the six month period of study. Each worker was to report any cases with which s/he had had a role in the case management process, regardless of how brief the involvement, the type of involvement, and whether or not the case was deemed to be unsubstantiated, referred on, or subject to an ongoing case management plan.

Case data was collected from workers by structured interview (done in the majority of cases), or self-complete questionnaire. In many cases more than one worker was involved with individual cases simultaneously and/or at different stages of case management. In order to monitor the abused child’s progress through the treatment network, each questionnaire or interview identified the worker involved (using a coding system to protect worker confidentiality), and provided the abused child’s age, sex, date of birth, and postcode of normal residential address.

While a significant proportion of the interview schedule was intended to collect case ‘facts’ and was thus closed-ended, a number of open-ended questions were incorporated into the schedule to provide greater flexibility and to ensure the development of a comprehensive picture of in situ professional decision making. This approach provided informants with a greater opportunity to truly describe the realities of their situation, outside the bounds of set questions.

The development of measures consisting predominantly of closed-ended questions, asked in a predetermined order, has been criticised for failing to adequately allow informants to describe what is relevant to them, or to express different views (O’Connell-Davidson & Layder, 1994; Minichiello et al., 1995). For that reason most of the closed-ended questions also provided an ‘Other’ response category (and an appropriate amount of space) to ensure that a more accurate response could be provided. Finally, information imparted by workers which did not fit into the schedule’s questions, but which was interesting or important data was noted on the schedule and utilised in a qualitative analysis of the cases (cross-case analysis in particular).

Research design

Case tracking studies as employed by the authors, are epidemiological in nature and designed to elicit an understanding of the internal dynamics of a phenomenon (Patton, 1980). The focus is on how something is produced, rather than looking at the outcome itself. Hence the focus in the case tracking studies that have been undertaken has been aspects of professional decision making and case management, defining the strengths and weaknesses of the professional child protection network, and describing how and where children and their families enter the professional system and the path they subsequently take within the network. The process of in situ (real life) case management processes has been of interest, not individual case outcomes. Thus, the case sample that is collected is merely a vehicle to illustrate the decision making process.

The specific design developed by Tomison, utilised a multiple methods approach (Campbell & Fiske, 1959; Patton, 1980, Hammersley & Atkinson, 1983), with the assumption that confidence in findings is increased if diverse forms of data support the same conclusion (Campbell & Fiske, 1959; Patton, 1980; Hammersley & Atkinson, 1983; Huberman & Miles, 1994). Such a design has been described as simultaneous triangulation (Greene, Caracelli & Graham, 1989; Cresswell, 1997), or as an integrated mixed methods approach (Bazeley, 1999), where a set of data are analysed by a variety of qualitative and quantitative methods. The aim is for each analysis to inform the others as a means of developing a clearer, more valid perspective on professional case management processes.

This study incorporated three major elements of data analysis, all of which utilised the results of the six-month case tracking study: quantitative statistical analysis; cross case analysis; and social network analysis, the latter two being qualitative methods involving some use of quantitative techniques. The inclusion of qualitative analyses was regarded as essential ‘to make sense of actual lived experience’ (Marecek, Fine and Kidder, 1997:632) and because of the difficulty of capturing the ‘confidential and politically sensitive data’ with which the study of child protection case management decisions must deal (Fernandez, 1996:75).

Thus, the research involved the use of a broad, quantitative statistical survey as a ‘wide angle lens’ to capture the content of professional decision making. The more intensive qualitative exploration of the cases enabled the researcher to ‘zoom in’ on the specific dynamics of the decision making process.

 

The benefits of triangulation/multiple methods approach

The benefits of adopting a triangulation or multiple methods approach, is that the use of data collected from a variety of sources, and/or a number of different methods of data analysis, enables the researcher to build on the strengths of each approach, while minimising the weaknesses of any particular technique (Patton, 1980; Minichiello et al., 1995).

Some qualitative researchers have criticised the use of triangulation on the grounds that the ontological and epistemological incompatibility of some methods is ignored because the strategy offers a means to overcome problems in bias and validity (Silverman, 1985; Minichiello et al., 1995). Norman Denzin, one of triangulation’s best-known advocates, has been criticised by qualitative researchers for abdicating ‘the interpretivist concern for the primacy of meaning in favour of a positivist concern about validity and bias’ (Blaikie, 1988:11; as cited in Minichiello et al., 1995).

Such concerns become particularly salient when reconciling qualitative and quantitative data in situations where different methods have produced conflicting explanations of observed phenomena. The issue is the use of different methods to choose between competing versions of the same social situation. For qualitative researchers ‘without bias there would be no phenomenon . . . the [researcher’s] role is not to adjudicate between participants’ competing versions but to understand the situated work that they do’ (Silverman, 1985:106).

This need not result in the adoption of a strongly positivist approach (Patton, 1980; O’Connell-Davidson & Layder, 1994). One extreme qualitative approach advocated by Denzin (1992), in his later writings as a postmodernist, was that researchers simply accept the subjective world view of research subjects as the most reliable viewpoint, rather than considering the research issue from multiple vantage points in order to develop a more reliable overview. He suggested that researchers give more weight to the ‘local knowledge’, feelings and emotions of subjects. However, the adoption of such a qualitative approach to, the exclusion of other methods, has also been criticised for providing a quite restrictive perspective (O’Connell-Davidson & Layder, 1994).

Differentiating between conflicting findings produced by different methods is usually not a straightforward exercise (Patton, 1980; O’Connell-Davidson & Layder, 1994). Qualitative and quantitative methods often end up answering different questions that may not produce a simple, well-integrated picture of the situation.

Trend (1978; as cited in Patton, 1990) maintained that the best approach was to reject the tendency to relegate one type of analysis to a secondary role, to give alternative viewpoints the chance to arise, and to postpone the immediate rejection of information that does not fit in with the majority viewpoint. Such an approach was adopted in this research.

Adjudicating between conflicting explanations was based on the careful consideration of all available evidence, an assessment of which analysis was best able to directly answer the research question, and the development of plausible explanations for any methodological disparity. Given that the research represented an explicit attempt to describe the realities of professional decision making in child protection cases, and that two of the three main analyses to be used in this research had a strong qualitative basis, any bias towards quantitative results was effectively reduced.

Quantitative analysis

The utilisation of case tracking data, collected by structured interview or self-complete questionnaire, was used to develop a quantitative statistical profile of the case sample. The profile included a description of the child and family situation: family history of involvement with the child protection system (for the child(ren) currently suspected of being maltreated and any siblings); family stress factors; the child’s presenting problems; the types of suspected and/or confirmed child maltreatment.

A profile of professional involvement with the family was also presented, describing: prior professional involvements (including previous Protection Applications taken out); referrals and consultations that were made; and the occurrence of specific case management issues, (e.g. violence directed towards workers by the family; the number of case conferences held). Much of this data was reported as frequencies or in crosstabulations tables. Additional analyses, focusing on specific factors from the ‘family’ and ‘professional’ systems, which it was hypothesised would affect professional decision making, were carried out on the entire case sample and for a number of professional sub-samples. Finally, Multiple Regression analysis was used to create statistical models of particular decisions (e.g. the decision to lay criminal charges).

Case studies

A danger in analysing data at high levels of inference, that is, using quantitative data in order to determine generalised trends, is that the uniqueness of the cases and the ecological validity of the research may be reduced (Huberman & Miles, 1994; Marecek et al., 1997). A qualitative case study approach is regarded as an essential component of exploratory research, enabling the ‘richness’ of the data to become apparent. The use of case studies enables the formation of an ‘inside view’ of the phenomenon (Young & Mills, 1980; Yegedis & Weinbach, 1991). Drawing on the phenomenological tradition, behaviour is able to be analysed at the microlevel, focusing on individual’s interactions, interpretations and negotiations with others (Luckman, 1978).

Yegedis and Weinbach (1991) note that such methodology is well suited to the strengths of social work practitioners since it uses methods of data collection that rely on developing close relationships with research subjects, interviewing skills and non-judgemental attitudes. In addition, case studies are a valuable research tool when attempting to answer the ‘how’ and ‘why’ of particular phenomena (Yin, 1984a). Such questions are explanatory and deal with operational links which may need to be traced over time, something case studies are well suited to, but which is not achievable by merely assessing frequency or incidence data.

Case studies may be employed qualitatively or quantitatively (Huberman & Miles, 1994; Stake, 1994). Irrespective of the way in which they are utilised, they generally adhere to four underlying tenets: bringing expert knowledge to bear upon the phenomena studied; the collection of all relevant data; the examination of rival interpretations; and giving consideration to the extent to which the findings are generalisable (Yin, 1992; as cited in Stake, 1994).

In the last two decades a number of researchers have conducted investigations based on a number of sets of groups or individuals within an environment, often using multiple methods (Huberman & Miles, 1994). Similarly, although the classic form of case study is the qualitative, single case study (Huberman & Miles, 1994; Stake, 1994), a number of studies have incorporated cross-case analyses in order to identify configurations of subjects, variables, or causal influences, which hold in some settings, but not in others; the result is the development of ‘families’ or ‘clusters’ of cases. Investigating multiple cases across multiple settings makes it possible to test key processes, constructs or explanations in a number of different configurations, thus enhancing the generalisability of findings (Huberman & Miles, 1994; Stake, 1994).

Cross case analysis was incorporated into the current research. Utilising a variable-oriented strategy, (Huberman & Miles, 1994) also known as a collective case study approach (Stake, 1994), themes which cut across cases were identified a posteriori and used to describe in situ case management practice and/or to support evidence from the quantitative analysis and descriptive case mapping (Huberman & Miles, 1994). An embedded case study design (Yin et al., 1983) was adopted, where each case is summarised and assessed in a standardised manner on a series of ‘embedded’ factors (e.g. family demographics, or the actions taken by professionals involved with the cases).

Generalisability, validity and reliability

In purely statistical terms, the generalisability of case study designs is not high. Yet Yin (1984b) argues that generalisability is analytical not only statistical. That is, the goal of such research is often to generalise the results to a specific theory rather than a particular population. Taking this approach, the results of case studies either support or refute theoretical assumptions and are useful in this regard (Yin, 1984b). The use of multiple case studies, or cross case study designs also increases external validity and generalisability (Yin, 1984b).

The internal validity of case studies is enhanced by triangulation (O’Connell-Davidson & Layder, 1994). To further enhance validity, Lincoln and Guba (1985) note the importance of incorporating participants’ input about the accuracy of interpretations, done by follow-up with participants, where possible. In addition, incorporating a period of prolonged engagement with the participants prior to data collection enables the researcher to gain a better understanding of the phenomenon s/he is investigating. In this study the researcher spent over 18 months familiarising himself with the setting, spending time with all participating agencies prior to testing and collecting additional data from agency managers to supplement an overall understanding of agency roles, case loads and work practices.

Experimenter effects, reflexivity and triangulation

Face-to-face interviewing and case study approaches have the benefits of increased response rates and many researchers have noted the value of observing a phenomena as a quasi-participant: the potential for informants to describe the issues that are salient to them, rather than being limited to a researcher’s strict agenda; the benefits of being able to explore issues that arise; and the opportunity the interviewer is presented with to ask for further clarification of particular issues, where required (Yegedis & Weinbach, 1991; O’Connell-Davidson & Layder, 1994; Minichiello et al., 1995).

As a consequence of rapport-building however, interviewing may lead researchers to become over-involved or to get too close to subjects, and thus ‘objectivity’ may suffer and experimenter bias may be introduced (O’Connell-Davidson & Layder, 1994). This may be negated to some extent, when structured interviewing and the use of standardised questions is adopted, as it has in this study, to ensure the comparability of cases and to enhance reliability, replicability and objectivity (O’Connell-Davidson & Layder, 1994; Minichiello et al., 1995).

While some aspire to a truly objective approach to research, as is advocated by the positivist school, others note that researchers are influenced and informed by the phenomenon they investigate and they bring to it, their own social identity, set of experiences, perceptions, biases and theoretical preconceptions (O’Connell-Davidson & Layder, 1994). Inevitably these ‘experimenter effects’ influence the questions they choose to ask, the methods they select and the analyses they ultimately perform (Huberman & Miles, 1994; O’Connell-Davidson & Layder, 1994).

Yegedis and Weinbach (1991) propose that by choosing a ‘participant as observer’ role and refraining from interpretative or evaluative comment, the researcher can attempt to minimise the impact of her/his presence on subjects’ behaviour. Alternatively, rather than imagining that a researcher can neutrally describe ‘facts’ or the reality of social phenomena, many researchers have advocated the adoption of the practice of reflexivity in research. Under such an approach researchers are required to continually reflect upon the ways in which their own social identify and values can affect the data they gather (Huberman & Miles, 1994; O’Connell-Davidson & Layder, 1994). Rather than denying experimenter influences, the values and assumptions a researcher brings to research are identified and their possible effects are described and discussed. O’Connell-Davidson and Layder (1994) encourage researchers to use the human qualities of ‘empathy, reason and self-consciousness’ because they believe that rather than hindering research, better research and a more accurate picture of the social reality emerges if these qualities are used appropriately.

Overall, the purpose of social research is to attempt to produce a better understanding of social reality and to contribute to knowledge. O’Connell-Davidson and Layder (1994) recommend that all researchers should, where possible, adopt a reflexive approach to their research, but utilise triangulation and a multiple methods approach as a means of cross-checking research findings. As O’Connell-Davidson and Layder note, the use of triangulation in conjunction with the practice of reflexivity, provides a possible solution to ‘some of the methodological and philosophical problems which arise from the fact that there are different versions of the truth, and that social researchers do not and cannot observe neutrally’ (1994:55).

While this study involves predominantly the use of structured interviewing, an attempt was made to incorporate some methodological flexibility, (open-ended questions, the option of exploring issues further, if required), to ensure that a clear picture of the realities of child maltreatment case management could emerge. The interpretation required in the social network and cross case analyses suggests that the potential for some experimenter bias. Triangulation has been incorporated into the research design as a means of cross-checking data and ‘as a strategy that adds rigour, breadth, and depth to [the] investigation’ (Denzin & Lincoln 1994:2).

Social Network Analysis

The research was centred around the descriptive ‘mapping’ of a child protection system for a period of six months. A major facet of the case mapping was the determination of interprofessional and interagency communication and collaboration patterns, that is, the collection of relational data.

Relational data can be defined as the ‘contacts, ties and connections, the group attachments and meetings, which relate one agent to another and so cannot be reduced to the properties of individual agents themselves’ (Scott, 1991:3). Thus, relations are not the properties of agents, but of systems of agents; they connect pairs of agents into larger relational systems. Relational data may be used to quantify the cohesiveness of a collection of actors (network density), to identify pockets of cohesion (cliques), the degree to which actors initiate interactions (actor centrality), the degree to which participants initiate contact with particular actors (actor prestige), and

While relational data can be represented quantitatively, social network analysis also enables a qualitative analysis of network structure (Scott, 1991). Defined as a ‘set of methods for the analysis of social structures, methods which are specifically geared towards an investigation of the relational aspects of these structures’ (Scott, 1991:39), social network analysis provides ‘a means to derive a more complete view of a given social environment’ (Koehly & Shivy, 1998:3). Such an approach allows researchers to thoroughly investigate the quality of interactions between participants, the pattern of such interactions and the different perspectives the ‘network actors’ bring to the social environment under examination (Scott, 1991; Koehly & Shivy, 1998).

The network diagram, the ‘sociogram’, is one of the earliest techniques for formalising social network analysis and it forms the basis of the case mapping undertaken in this study (Scott, 1991). Network elements are organised such that sets of elements or ‘points’ and their inter-relationships are identified (Scott, 1991), resulting in the construction of a ‘graph’ of points connected by lines (i.e. a sociogram).

Graph theory is frequently used as a quantitative framework for social network analysis, providing a formal language for describing networks and their features (Scott 1991; Koehly & Shivy, 1998). However, because of the size and complexity of the target network, the inability of the researcher to collect the entire population of cases active during the case tracking, and the decision to conduct a qualitative social network analysis, (thus, mathematical indices would not contribute anything significant to the study), it was decided not to employ graph theory to calculate mathematical measures of the relational links between participants.

In summary

Overall, the research design provides a broad statistical survey of the children, families and professionals involved in the suspected child maltreatment cases and an assessment of the factors influencing professional case management decisions. The more intensive qualitative exploration of the cases via case studies and descriptive mapping enables more detailed analysis of the specific dynamics of the decision making process. The combination of quantitative and qualitative analyses offered the potential to triangulate data, counteracting the limitations of each technique by drawing on multiple perspectives to illuminate aspects of the research.

Goddard’s experiences in conducting case tracking studies, and more recent studies carried out by Tomison, (including Tomison, 1994), are used to inform the remaining sections of this paper.

CASE TRACKING — IMPLEMENTATION ISSUES

Case tracking studies are complex, in situ studies that typically require significant lead time in order to ensure official permissions, ethical clearances and the participation of the various agencies or professional groups under investigation. For example, Tomison (1994) took approximately two years to complete negotiations, develop the tracking materials, and brief the participants, before the main tracking study could finally begin.

A number of systemic and methodological problems may significantly affect the design and overall success of tracking studies, and that must be confronted before studies can commence, have been identified in the ‘pre-testing’ period of a variety of case tracking studies that have been completed by the authors. These issues are discussed in terms of their effect on in situ research, and their relationship to problems often faced by the abused child and her/his family, once they enter a child protection network.

Issues of consent

In order to effectively track suspected child maltreatment cases between the various agencies and professionals in the child protection network, it is necessary to have access to information that adequately identifies both the case and the worker(s) who make the decisions that significantly affect case management. The research requires the identification of the worker(s) making the decisions, the stage of case management at which this occurs and enough identification of the case to be able to correctly match data collected from several sources for the same case, and to be able to develop case samples for each participating agency or professional group.

With regard to the former, when agencies and workers are briefed on the studies, it is made clear that in order to determine which workers made particular decisions for child maltreatment cases and to enable the collection of information from these workers, worker identification is a necessity. It is emphasised that the research team is not interested in evaluating individual workers’ case management, but with the investigation of trends in case management within the region. This explanation and the measures taken to ensure worker anonymity generally allay any worker or agency concerns.

With regard to the identification of cases, the following procedure has been used previously by the authors and his colleagues, using the minimum identifying information required that will allow the case tracking process to be carried out without the need to fully identify the children or the families who constitute the suspected or confirmed child maltreatment cases identified in a case tracking study:

1. name of the worker within each agency/profession assigned to the case;

2. child’s first name ( optional, and rarely provided);

3. child’s age, sex, date of birth, and postcode of normal residence.

Access to this ‘matching data’ is strictly limited to the primary data collection team member; the names of the child and worker(s) are removed from the completed questionnaires/interview schedules and an identifying numerical code is assigned to each. These codes are then used to differentiate between cases and workers during data analysis, and throughout the research. A ‘master file’ linking the case identities with their respective numerical codes is securely stored in a locked cabinet when not in use, and kept separate from the tracking data. At the conclusion of the research the master file and all information which can identify the children, their families, or the workers involved with the cases, is destroyed. Steps must be taken to ensure that no information that could identify children, families or workers is revealed during analysis, or in any discussion or publication of the findings.

Informed consent of the families

The research is primarily focused on the decision making process or actions taken by the professionals involved in the management of suspected or confirmed child maltreatment cases. The cases collected in the case tracking form a database from which to assess and compare the decisions made by the different workers at different stages of case management. All data is collected via archival analysis or worker report; the research does not involve direct intervention or experimentation on the children or families who constitute the case sample. In addition, the tracking process has been developed to ensure that the full identities of the children and families involved in the tracked cases will not be known.

In general, while full informed consent of any and all participants is mandatory in most studies, under the principles of the Australian National Health and Medical Research Council Ethical Guidelines, epidemiological studies such as this which do not involve actual testing of participants, are able to proceed without the informed consent of the clients whose case information is used only as the basis for describing a phenomenon.

Len Doyal (1997), a Professor of Medical Ethics encapsulated the current arguments for and against full informed consent and concluded that full consent should generally be sought in all research, with three possible exceptions. Epidemiological research based on client records was one of the three possible exceptions. Doyal (1997) notes that for such research to proceed, minimal conditions which must be met are:

On this basis, virtually all case tracking studies conducted to date, have proceeded without informed consent of children or their families, with the approval of a number of University and hospital ethics committees. However, in Tomison (1994), two agencies would only participate on the basis of full informed consent. A procedure was therefore developed which would enable workers to brief clients on the project and to obtain their informed consent, and yet would ensure the researcher was not provided with the full identification of the clients. In one of the agencies staff were very motivated to participate and this procedure worked very well, with all clients who were approached providing consent. In the second agency, workload issues and difficulties in accessing staff meant that the researchers’ ability to motivate workers to actively participate in the study, and to approach families to obtain their consent, meant that the agency provided no cases for the tracking and were subsequently excluded.

The authors are reluctant to employ such procedures because of the high probability that the study will be substantially compromised and/or made unworkable. For example, in 1998, Tomison and colleagues (Tomison, Wise & Murray, 1999) attempted to conduct another case tracking study involving five different agencies in each of two regions of Victoria. In this study the majority of agencies required that informed consent procedures be developed. Using a procedure similar to that developed in Tomison (1994), all agencies were requested to obtain informed consent of clients for the study. However, once this protocol was in place, very few families were subsequently approached by workers, despite the research team trialing a number of different approaches aimed at facilitating the consent process, including the provision of a toll free number for potential participants to ring the research team and to be fully briefed prior to providing consent to the host agency. The majority of those that were approached (two thirds) did consent to the agency releasing their data, but overall, the very small number of consents obtained resulted in the study having to be abandoned.

Therefore, while full informed consent in studies is highly desirable, the procedures required to obtain consent in case tracking studies rely heavily on professional participants to identify and approach families and children. Worker motivation, and the researcher’s ability to maintain motivation and to support workers in this task, is therefore crucial to the success of the project.

Unfortunately, the level of contact required makes such a process exceeding difficult to achieve without substantial staff resources, and will often lead to research staff being able to fully identify participants, something that is not appropriate in child protection studies and may breach the conditions under which the research is allowed to proceed.

A number of other issues have been identified by the authors as impacting on the development and success of child protection case studies.

The coordination of diverse services/professions.

It is common for complex health and social issues to be managed by a number of professionals (Jones et al., 1987). Tomison (1994) involved the participation of eleven different agencies or professional groups. A major problem was the coordination of the groups to ensure that each had access to the research submissions; that each group had the research proposal approved by its appropriate management bodies; and that each group’s workers were briefed and ready to begin data collection by a set date.

Such problems may reflect some of the generally identified problems of service coordination in child protection. Having to coordinate the work of a number of services has often been cited in the literature as leading to less than optimal case management (e.g. Dale, Davies, Morrison & Waters, 1986; Jones et al., 1987; Hill, 1990; CSV, 1991a; 1991b; Hallett & Birchall, 1992) and in a research sense, if the level of coordination required is substantial, it may lead to a less than optimal research process and subsequent results.

Worker collaboration - theory versus practice.

The crux of the negotiation process for such projects is in achieving the active support of all levels of the selected professional groups. This typically involves conducting numerous briefings/meeting with personnel from senior management (state-level), down to the individual workers who are asked to collect the data. During this process, the research team will often have to overcome the personal biases of some management staff, and of some of the workers themselves, in order to achieve project approval and a high response rate. The latter may require the modification of data collection procedures, in order to minimise as much as possible the inconvenience to the workers, and thus enhance data collection or ‘response rate’. There will also be a need to allow for the ‘theory versus practice’ of project support. While workers may generally agree to participate in the project when requested to do so, having to actually complete the required work may not produce the same degree of enthusiasm.

For example, in Tomison (1994) it was initially planned to collect data via a self-completed questionnaire. One group with a high caseload was quite motivated to be involved with the project, but it became evident during a pilot study that they were not collecting data because of the tracking materials impinging on the group’s already high amount of paperwork. As the workers remained highly motivated with regard to the project, it became necessary for the authors to offer workers the option of completing structured case interviews. Although researcher workloads increased dramatically, this change enabled a better quality of information to be collected, and enabled the exploration of a number of issues related to casework that had been omitted from the original measure.

Existing Tensions between Practitioners and Researchers.

Finally, workers in highly sensitive areas like child protection, where the responsibilities are enormous, are generally suspicious of people who in effect judge the work they are doing, and who point out deficits in their service provision. Stevenson notes that ‘increasingly, social workers in the field of child care are under hostile scrutiny’ (1992:30). The existence of such a ‘siege-like’ mentality makes it crucial from the researcher’s perspective to convince participants of the research team’s neutral stance regarding worker actions, and to provide sureties that the research findings will not be misused, or corrupted. As Brown & Goddard (1991) note, in order to successfully run large scale in situ research projects, they cannot be imposed, rather they must be developed in a partnership with the agencies and workers ‘on the ground’.

As part of this trust-building process, attempts should be made to ensure the participating groups are briefed on how the research findings may be used positively, to enhance intergroup relationships, and case management decision making. As a function of the establishment of trust and rapport between researchers and participants, researchers are then more likely to achieve a motivated subject population. In Tomison (1994), the briefing process proved to be successful to the extent that some groups, once they were fully briefed, actively promoted the research, and extended their commitment to the project.

The establishment of a good working relationship also benefits the practitioner in a more general sense. Collecting a better quality of information enables the researcher to formulate a more accurate picture of the realities of case management, and the production of ecologically valid representations of casework provides practitioners with more accurate information about the realities of child protection case management.

Finally, with regard to worker-client relationships, developing trust and rapport with clients has long been established as a basic principle in the training of the helping professions. In child protection case management, the worker needs to develop as clear a picture as possible of the child’s situation in order to make the best case management decisions for the child and family. Just as researchers are more likely to have a better response rate and a better quality of data when a good bond is formed with workers, the worker is more likely to get better information from a child and/or family, when she/he has established a good relationship. Therefore, it is contended that relationship-building is another area where the experiences of the child and family in the network can, in part, be reflected by the researcher.

CONCLUSION

In conclusion, the authors acknowledge the inherent difficulties which confront the researcher brave enough to leave the laboratory. However, the opportunity to redress the numerous laboratory-based experimental simulations of child protection decision making, and to attempt to test many of the qualitative hypotheses advanced over the years, has led the authors (and colleagues) to pursue the development of the case tracking methodology as an excellent means of developing an ecologically valid perception of decision making processes, one that combines the best of the qualitative and quantitative methods.

More importantly than the theoretical and methodological interest such studies may generate, the significant contribution they have made to the understanding of child protection case management may in turn lead to enhanced child protection systems for the system users: ‘at risk’ and maltreated children and their families.

REFERENCES

Bazeley, P. (1999). Methodological purity and rigour mortis. Paper presented to the Association for Qualitative Research Conference ‘Issues of Rigour in Qualitative Research’, 6-10 July, 1999, Melbourne.

Brown, T.C. & Goddard, C.R. (1991). Practice to research and research to practice. Paper presented at the First National BAPSCAN Congress ‘Turning Research into Practice’, September, 1991, Leicester.

Campbell, D.T. & Fiske, D.W. (1959). Convergent and discriminant validation by the mutitrait-multimethod matrix. Psychological Bulletin, 56 (2), 297-311.

Community Services Victoria. (1991a). Confidential Report of the Ministerial Panel of Inquiry into the Death of D.V. Melbourne: L.V. North Government Printer.

Community Services Victoria. (1991b). Summary of the Report of the Ministerial Panel of Inquiry into the Death of R.M. Melbourne: L.V. North Government Printer.

Dale, P., Davies, M., Morrison, T. & Waters, J. (1986). Dangerous Families: Assessment and treatment of child abuse. London: Tavistock.

Denzin, N.K. (1992). Symbolic interactionism and cultural studies. Newbury Park, CA: Sage.

Denzin, N.K. & Lincoln, Y.S. (1994). Introduction: Entering the fields of qualitative research. In N.K. Denzin & Y.S. Lincoln (Eds.), Handbook of Qualitative Research, (pp.1-17). Thousand Oaks, CA.: Sage.

Department of Health and Social Security. (1982). Child Abuse: A study of inquiry reports 1973-1981. London: HMSO.

Fernandez, E. (1996). Significant harm: Unravelling child protection decisions and substitute care careers for children. Aldershot: Avebury.

Goddard, C.R. (1988). A child sexual abuse police tracking project: A hospital-based study for the Law Reform Commission of Victoria. In Law Reform Commission of Victoria, Sexual offences against children: Research reports (pp.61-185). Melbourne: Victorian Government Printers.

Goddard, C.R. & Hiller, P.C. (1992). Tracking physical and sexual abuse cases from a hospital setting into Victoria's criminal justice and child protection systems - A report for the Victorian Law Foundation (Vols 1-3). Melbourne: Departments of Social Work & Anthropology & Sociology, Monash University.

Goddard, C.R. & Hiller, P.C. (1993). Child sexual abuse: Assault in a violent context. Australian Journal of Social Issues, 28(1), February, 20-33.

Goddard, C.R., Tucci, J., Liddell, M., Hiller, P. & O’Connell, J. (1996), A provisional analysis of decision-making by statutory child protection workers in a rural context. Melbourne: Child Abuse and Family Violence Research Unit, Monash University.

Greene, J.C., Caracelli, V.J. & Graham, W.E. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation & Policy Analysis, 11, 255-274.

Hallett, C. & Birchall, E. (1992). Coordination and Child Protection: A review of the literature. Edinburgh: HMSO.

Hallett, C. & Stevenson, O. (1980). Child Abuse: Aspects of Interprofessional Co-operation. London: Allen & Unwin.

Hammersley, M. & Atkinson, P. (1983). Ethnography: Principles and practice. New York: Tavistock.

Hill, M. (1990). The manifest and latent lessons of child abuse inquiries. British Journal of Social Work, 20(3), 197-213.

Hiller, P.C., Goddard, C.R. & Diemer, K.M. (1991). 98 cases of child abuse presenting at a major Melbourne hospital: Socio-demographic factors and institutional reactions (Vols. 1-2). Melbourne: Departments of Anthropology & Sociology, and Social Work, Monash University.

Huberman, A.M. & Miles, M.B. (1994). Data management and analysis methods. In N.K. Denzin & Y.S. Lincoln (Eds.), Handbook of qualitative research (pp.428-444). Thousand Oaks, CA.: Sage.

Jones, D.N., Pickett, J., Oates, M.R. & Barbor, P.R.H. (1987). Understanding Child Abuse (2nd ed.). Basingstoke: Macmillan.

Koehly, L.M. & Shivy, V.A. (1998). Social network analysis: A new methodology for counseling research. Journal of Counseling Psychology, 45 (1), 3-17.

Lincoln, Y.S. & Guba, E.G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.

Luckman, T. (Ed.) (1978). Phenomenology and sociology. Harmondsworth: Penguin.

Maracek, J., Fine, M. & Kidder, L. (1997). Working between worlds: Qualitative methods and social psychology. Journal of Social Issues, 53(4), 631-644.

Minichiello, V., Aroni, R., Timewell, E. & Alexander, L. (1995). In-depth interviewing: Principles, techniques, analysis (2nd ed.). Melbourne: Longman Australia.

O’Connell Davidson, J. & Layder, D. (1994). Methods, sex and madness. London: Routledge.

Parton, N. (1989). Child Abuse. In B. Kahan (Ed.), Child Care Research, Policy and Practice (pp. 55-79). London: Hodder and Stoughton.

Patton, M.Q. (1980). Qualitative evaluation methods. Beverley Hills: Sage.

Scott, J. (1991). Social network analysis: A handbook. London: Sage.

Silverman, D. (1985). Qualitative methodology and sociology. Aldershot, Hants: Gower.

Stake, R.E. (1994). Case studies. In N.K. Denzin & Y.S. Lincoln (Eds.), Handbook of qualitative research (pp.236-247). Thousand Oaks, CA.: Sage.

Stevenson, O. (1992). Social work intervention to protect children: Aspects of research and practice. Child Abuse Review,1, 19-32.

Tomison, A.M. (1994). An evaluation of decision making in child abuse services in the Barwon region: A report for the Victorian Health Promotion Foundation (Vols. 1-2). Melbourne: Department of Social Work & Human Services, Monash University.

Tomison, A.M., Wise, S. & Murray, B. (1999). An Evaluation of the Enhanced Client Outcomes (ECO) Project: A report for the Department of Human Services, unpub.

Yegedis, B. & Weinbach, R. (1991). Research methods for social workers. New York: Longman.

Yin, R. (1984a). Casestudy research: Design and methods. London: Sage.

Yin, R. (1984b). Studying phenomenon and context across sites. American Behavioral Scientist, 26, 84-100.

Yin, R., Bateman, P.G. & Moore, G.B. (1983). Case studies and organisational innovation: Strengthening the connection, Washington D.C.: Cosmos Corporation.

Young, K. & Mills, L. (1980). Public policy research: A review of qualitative methods. London: Social Science Research Council.

Return to Papers and presentations

Line
AIFS Home
AIFS
Home
Email queries to webmaster@aifs.gov.au | Copyright, privacy, disclaimer | Site map

Australian Institute of Family Studies, Level 20, 485 La Trobe Street, Melbourne Vic 3000, Australia. Tel: (03) 9214 7888. Fax: (03) 9214 7839. URL: http://www.aifs.gov.au/