Research finds offender risk assessment tools in US are promising, but questions remain

The criminal justice system in the United States uses a variety of tools to assess the behavior of criminal offenders, and those risk assessments can have a significant impact on an offender's fate. A new meta-analysis of the research conducted in the U.S. on these tools shows that—while promising—it is still unclear whether these tools reduce bias against offenders based on race or other factors.

At issue are recidivism risk assessment instruments, or RAIs, which are tools that assess the likelihood that someone who has been arrested for a crime will commit another crime in the future. RAIs are used by professionals in correctional settings, such as parole or probation officers and prison psychologists.

In theory, RAIs reduce bias against minority groups in criminal justice decision-making, and are used to inform where prisoners serve their time and when they are released. RAIs may also be used to inform sentencing and recommendations for intervention or rehabilitation programs. While RAIs are primarily used with convicted offenders, they are sometimes used before an individual has had his or her day in court.

"RAIs are important because they are widely used in the United States in an attempt to reduce biased assessments of recidivism risk and are increasingly being mandated at the local, state and federal level," says Sarah Desmarais, an associate professor of psychology at NC State and lead author of a paper on the research.

"We know that this sort of structured tool can improve identification of offenders who are more likely to commit crimes in the future," Desmarais says. "However, a lot of the research has been conducted in other countries and there has not been a comprehensive evaluation of RAIs in the U.S. Which tools are most accurate in identifying recidivism risk in U.S. offenders? And are any of them biased against African Americans or other minority groups? Those are the questions we tried to address in this paper."

There are, at a conservative estimate, more than 60 recidivism RAIs in use in various parts of the U.S. - most of which are used only in a specific state or municipality. The researchers looked at the 19 most commonly used, which are applied in multiple jurisdictions.

For their meta-analysis, the researchers evaluated all of the research conducted in the U.S. on any of those 19 RAIs. Altogether, the meta-analysis incorporated 53 studies, which looked at a total of 362,323 offenders. Specifically, the researchers looked at how accurate each of the RAIs was at predicting recidivism. The meta-analysis also assessed each RAI's ease of use and "inter-rater" reliability." Inter-rater reliability measures how consistent an RAI's results are among users. So if an RAI is used by two different correctional officers to evaluate a single offender, for example, how likely are those officers to get the same result?

The overall findings suggest that RAIs are pretty accurate.

"But the work also shows that there are a lot of things we still don't know," Desmarais says. "There were only a handful of studies for each of the 19 instruments, and very few looked at anything that could give us insight into potential bias against racial or minority groups."

For example, only two of the studies looked at inter-rater reliability—and both of those studies were evaluating the same RAI. "This is important, because without that inter-rater reliability information, we can't tell if an RAI is actually increasing consistency - which is why RAIs were developed in the first place," Desmarais says.

"And we couldn't evaluate the impact of race on RAI accuracy, because only two of the studies reported on race and accuracy," Desmarais says. "Those two studies did find almost identical rates of accuracy across races, but you need more than two studies to draw firm conclusions."

The researchers also found that no single instrument was more effective than the others—it depended on what type of recidivism was being assessed. For example, one tool was the most effective at predicting if offenders would violate the conditions of their parole or probation. But three other tools were more accurate at predicting whether offenders would commit new crimes.

"Our analysis tells us that RAIs can be useful tools in the U.S.," Desmarais says. "But it also tells us that a lot of additional work needs to be done."