Risk assessment in child protective services: consensus and actuarial model reliability.

Abstract

Three widely used child protective service assessment models (two consensus based, one actuarial) were examined to determine their reliability. Although no system approached 100% interrater reliability, raters employing the actuarial model made consistent estimates of for a high percentage of the cases they assessed, and interrater reliability for the actuarial model was much higher than that of the other systems. The decisions that child protective service (CPS) workers make at the conclusion of a child abuse or neglect investigation are critical to the protection of children. At that point, the worker must decide whether to: (1) close the case, (2) open the case for protective service intervention or intensive in-home family preservation services, or (3) remove the child to out-ofhome care. Because this decision has important consequences for children, their families, and protective service agencies, it must be made as consistently and accurately as possible. Frontline CPS workers must base their decisions on the best interest of the child, a determination that is largely dependent on the worker's estimation of the of future harm. If conclusions regarding in a particular case vary widely (depending on who does the rating), then some children will be left in situations with a high potential for continued maltreatment, while others who could have remained at home in relative safety will be placed in out-of-home care. Mistakes can have enormous consequences, ranging from unnecessary expenditures to emotional upheaval and trauma, to serious injury to and even the death of a child. Clearly, decisionmaking in human services is difficult-personal relationships in families served are dynamic and difficult to assess. Those charged with protecting children--CPS investigative staff and managers-represent a wide spectrum of educational backgrounds and personal and professional experiences, and bring different values and perspectives to the job. This mix of conditions-the potentially grave consequences of error, the inherent difficulty of accurately assessing family situations and relationships, and the range of skills evident in the nation's CPS staff-presents a near-perfect equation for widespread disparity in case decisionmaking. CPS Risk-Assessment Systems and the NCCAN Study Risk-assessment systems are simply formalized methods that provide a uniform structure and criteria for determining risk [Keller et al. 1988: 21. CPS risk-assessment systems have developed to help workers accurately estimate the future of abuse/ neglect and thus make better service decisions for families. The expectation is that such systems will increase the reliability and accuracy of CPS worker decisionmaking. The use of risk-assessment systems, although widespread, is relatively recent. In 1996, the American Public Welfare Association (APWA) conducted a survey of 54 states, territories, and large county child welfare agencies to determine their use of and satisfaction with CPS risk-assessment systems. Of the 44 jurisdictions that responded, 38 had some assessment or safety assessment in place. Of these, 26 first implemented their assessment after 1987 [Tatara 19961. Although risk-assessment models continue to be developed, experts have expressed concern that the theoretical and empirical support for these systems is inadequate [Cicchinelli 1991]. As noted by Murphy-Berman [1994], risk-assessment procedures vary on a number of dimensions, and the task of comparing one to another is quite complex. Generally, CPS risk-assessment systems fall into two basic types: 1. Consensus-based systems are those in which workers assess specific client characteristics identified by the consensus judgement of experts and then exercise their own clinical judgment about the of future abuse or neglect. 2. Actuarial systems are based on an empirical study of CPS cases and future abuse/neglect outcomes. …