The Trouble with Using Risk Assessment Instruments to Quantify the Chance of Future Offending

Abstract

Risk assessments in carceral settings have proliferated in recent decades and are now prominent in numerous states and regions. A ubiquitous variety is actuarial risk assessment instruments that are used on children and adults to predict their future chance for misconduct (e.g., recidivism) in several vital decision points in carceral processing (e.g., pretrial confinement). These instruments rely on information about past behavior (e.g., criminal history) and an understanding of offending (e.g., antisocial personality) that is thought to be neutral, reliable, and enjoys predictive validity. However, it will be argued that when justice system personnel assess the chance of unwanted behavior in the future, several risk domains are differentially prevalent and more frequently experienced by some groups. Much of this disparity is caused by, or due to, forces external to those being assessed, for instance, inequitable social and economic conditions and inequitable decisions by justice personnel to arrest, charge, or sentence people of color. As such, risk assessment instruments inevitably and disproportionately mark some groups of people as a higher risk to violate rules, conditions, orders, or laws. Consequently, risk assessment instruments systematically disfavor disadvantage, and by inference, favor advantage, leading to the need for a radical shift in the taxonomy of classifying risk for future misconduct.

1. Introduction

Risk prediction continues to be a favored armament in the arsenal of crime control and prevention. New methods are being put into action, for instance, geographic-based algorithms. These tools estimate where crime is most likely to occur, and pattern recognition tools help discover crimes that might be related. In addition, several other established methods of risk prediction continue to expand, including facial recognition and risk assessment instruments. For some time, these actuarial risk assessment instruments have been used with children and adults by justice system agencies, for example, probation, courts, prisons, to predict their future chance for misconduct, such as recidivism.

In the US, government management agencies embrace risk assessments as a best practice. For instance, the Office of Juvenile Justice and Delinquency Prevention, the Conference of State Court Administrators, the Federal Bureau of Prisons, the American Bar Association, and the National Association of Pretrial Services Agencies all recommend their use. The practice of assessing risk dates to the 1920s [1], but only became abundant in the US in the 1980s. Having gone through a period of development in the 1990s and early 2000s, today’s risk assessment instruments, often referred to as 4th-generation instruments, have anywhere from 42 to 150 items to estimate a person’s risks and needs as they relate to recidivism. As one researcher recently put it: “The offender risk assessment enterprise has been moving at warp speed since the turn of the millennium and shows no signs of slowing” [2].

Significant to all of this is that justice system agencies use risk assessments at several vital decision points: whether to release someone from custody before their next court date or before trial, the level of supervision, and the category or duration of a sentence. In addition, these instruments influence various other important decisions, including treatment alternatives, probation conditions, reentry eligibility, and parole requirements. Justice system agencies have begun to use risk assessment tools to support efforts to safely reduce the number of people in prison and reduce inmates’ prison sentences [3,4].

Risk-based decisions are consequential to those who experience contact with the justice system. The influence of a risk assessment score can have a pronounced impact on a person’s future educational attainment, employment and employability, income, housing, and eligibility to apply for housing, credit, and more—much more. Moreover, by implication, the influence of a risk assessment score also has a pronounced effect on broader social systems, including the person’s children, partners, extended families, and even entire neighborhoods—as some areas have significantly high rates of people under carceral control and cycling in and out of carceral institutions [5]. The widespread use and considerable social, financial, and human costs of action taken because of a risk assessment rating should rightly generate scrutiny of any limitations, partiality, or fairness of an assessment method. The stakes of not having such scrutiny are high. From public safety to procedural justice, the implications and consequences of predictive algorithms are significant, and it is no wonder that concerns and opposition to them are growing louder [6].

Activists, policy makers, and scholars have raised concerns and lodged compelling arguments against risk assessments. For instance, the knowledge base of risk assessments, the risk factor paradigm (RFP), has never been empirically demonstrated [7]. Developers of risk assessment instruments from the beginning have tolerated and based their practice on tenuous associations. The measures in risk assessment instruments overly simplify the link between risk factors and criminal pathways, and it is challenging to know from these measures which risk factors are more impactful to a person at a particular time (e.g., family versus peers) or with specific groups (e.g., rural vs. urban, male vs. female, and African America vs. Hispanic American). Employing generalized probabilities—particularly past arrest and conviction records—to predict the behavior of individuals can lead to false negatives, false positives, and inaccurate identification of risk.

Furthermore, legal scholars, law and society scholars, and criminologists have argued that risk instruments are biased against or unfairly calculate the risk scores of people of color, minority groups, women, and marginalized or economically impoverished people. Some scholars contend that, since risk assessment instruments contribute to the race and class bias that already plagues the US justice system [8]. Others find that risk assessment instruments cover up class and racial biases against poor, marginalized groups [9]. Moreover, risk assessment tools transform social and economic problems into individual ones, thus shifting the responsibility onto individual offenders and away from structural issues, social policies, and crime policies themselves [10]. At the least, there are serious questions about risk assessment instruments. It is little wonder why scholars, practitioners, and even policy makers such as former US Attorney General Eric Holder [11] contend that risk assessment tools stigmatize people based on belonging to a specific group. As Holder stated at an annual meeting of the National Association of Criminal Defense Lawyers:

“[Risk assessment instruments] may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and in our society. Criminal sentences must be based on the facts, the law, the actual crimes committed, the circumstances surrounding each individual case, and the defendant’s history of criminal conduct. They should not be based on unchangeable factors that a person cannot control, or on the possibility of a future crime that has not taken place.”

Yet, justice system agencies continue to implement risk assessments and expand their use in new settings, while experts continue to re-tool, evaluate, and sell their instruments, and researchers continue to publish articles about their efficacy and predictive validity. Too often, while scholars who are testing or evaluating risk assessments tend to acknowledge that critics have concerns about risk assessments, they nevertheless proceed with arguments that are predicated on an assumption of the unquestioned validity of these assessments. Suppose the goal of a risk assessment is to “create a fair and accurate prediction of risk that an inmate will commit a crime” [12]. In that case, it is not enough to tip your hat at these critiques and concerns and then proceed unencumbered by them. As geographic- and person-based prediction tools continue to proliferate, proponents must develop risk management evaluations that are technically neutral vis à vis race, class, economic status, and geographic location. In line with this, critical scholars must also contribute to this effort by showing, for instance, how risk-based assessments interrelate with social, economic, and political artifacts, how to improve them and, if we discontinue their use, what should replace them to guide decision making.

In line with this, in this paper, I argue against the rather uncritical, continued use of risk assessment instruments to guide decision making by carceral agencies. Using secondary data, I conclude that risk domains and their outputs are highly influenced and even produced by discretionary decisions of justice system personal, and the domains are highly influenced by historical and contemporary social forces, social policy, and crime policy, all of which are classed and racialized in many regions around the world. In other words, a substantial volume of information recorded by risk assessment instruments are highly contextualized, such as Black people having a greater probability than their White peers to be stopped by law enforcement, detained pre-trial, or sentence in secure confinement. Conversely, risk assessment instruments categorize people at a lower level of risk by virtue of being in a class or racial group that is exempt from over- or hyper-exposure to discretionary decisions by law enforcement and justice system personnel at multiple points in the criminal justice process. Therefore, risk assessment instruments convey an incomplete story about all groups of people and therefore should be considered inadequate ‘evidence’ in justice system decision making.

To reach this conclusion, in what follows, I detail the process of predicting future (undesirable) behavior, followed by a brief overview of the research base of risk assessment instruments and critics of this research base. Following this, I illustrate risk assessment instruments’ reliance on criminal history, biased data, and the racially disparate results from the reliance on criminal history.

2. The Process of Predicting the Likelihood of Future Undesirable Behavior

There are dozens of varieties of person-based risk assessment instruments. Some risk assessment instruments are used only for children, and others are only used for adults. There are pretrial risk assessment instruments, community supervision instruments, custodial intake instruments, and reentry instruments for each group. Some risk assessment instruments include the Youth Level of Service/Case Management Inventory, the Level of Service/Case Management Inventory (LS/CMI), Asset and AssetPlus, the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS), and the Risk Management Systems (RMS). These are just a sample of innumerable variations of assessments being used throughout the world.

In terms of terminology, ‘risk assessment instruments’, ‘risk assessment tools’, or ‘risk and needs assessments’, are the most common. For clarity, I will use the phrase risk assessment instrument to denote any individualized battery of binary-scored questions used by ‘front-end’ justice system agencies in the decision-making process, specifically in decisions about whether someone is likely to return to court, recidivate, or assist in decisions related to sanctions, sentences, treatment needs, or the level and duration of detention. Since my argument does not directly target risk assessments to determine reentry eligibility for those currently incarcerated and only marginally relates to prison intake tools, I will not discuss these ‘back-end’ risk assessment instruments.

In predicting future behavior, risk assessment instruments group raw risk scores, known as a summary risk score, into ordered categories of low risk, moderate risk, high risk, and very high risk. For instance, the widely used Youth Level of Service/Case Management Inventory (YLS/CMI) organizes items into eight criminogenic domains where 42 dichotomous items are scored 0 or 1 and then tallied to yield a total score that places youth into one of four categories. A score of 0–8 is assigned low risk, a score in the range of 9–22 is assigned moderate risk, 23–34 is assigned high risk, and 35–42 is assigned the rating of very high risk [13]. Justice system personnel then take into consideration this summary risk score and rating in the decision to release a person or keep them in jail to constrain them for one or more reasons that include the risk that they will defy orders or restrictions, fail to appear back in court, not comply with probation, or commit violent offenses.

In many jurisdictions dealing with children, after an arrest and before an initial hearing or arraignment, young people are evaluated using an algorithm-based risk instrument. These algorithm instruments generate scores based on a fixed set of factors [14] heavily weighted by criminal history. During the arraignment, a judge may also look to school attendance records, child welfare history, alcohol or drug use, and concerns with a living situation to decide what to do with the child. The algorithm-based risk instrument provides justice system personnel, e.g., prosecutors, judges, defense counsel, a preliminary indication of societal danger, discharge eligibility, and the likelihood of returning to court in the future. At a later date, a young person will be given a more robust risk instrument that scores a level of risk based on justice system personnel’s structured professional judgment (SPJ). This assessment requires the system personnel to interview the child to collect information about static and dynamic risk factors, which include personality characteristics, peer networks, living situation, family life, carceral involvement of parents and siblings, and more.

Static risk factors are generally scored by asking about prior arrests and convictions. Static, unchangeable risk factors also include age, gender, marital status, history of substance abuse, and history of justice system contact (e.g., arrests, convictions, incarceration). In contrast, dynamic risk factors—commonly referred to as criminogenic needs—are scored by asking about circumstances that can change over time and thus are amenable to treatment [15].

Dynamic risk factors can be changed and divided into stable and acute groups [16]. ‘Stable’ dynamic risk factors are ones that may change gradually change over time (e.g., attitudes, values, beliefs, and personality), while ‘acute’ dynamic risk factors can change rapidly (e.g., drug use). Additional dynamic factors that are changeable and ostensibly the targets for programming or case management include lifestyle (e.g., leisure activities), friendship and peer networks, family-related matters, educational attainment history, employment history, and financial resources. A small number of risk factors are described as the ‘central eight’ and the ‘big four’ because they have received consistent empirical support [15]. The so-called central eight include ‘history of antisocial behavior’, ‘antisocial personality pattern’, ‘antisocial cognition’, ‘antisocial associates’, ‘family or marital problems’, ‘school or work problems’, ‘leisure or recreation,’ and ‘substance abuse.’ The first four in this list, ‘history of antisocial behavior’, ‘antisocial personality pattern’, ‘antisocial cognition,’ and ‘antisocial associates,’ are the so-called ‘big four’ [15].

Risk assessment instruments now also measure responsivity, which includes the readiness of an individual to change and the likelihood that an individual will respond to a particular treatment method or program [17,18]. The 4th-generation risk assessment instruments can locate several areas of need and identify gaps and opportunities at different intercept points within the arc of justice system processing. Proponents suggest that risk tools could help develop a comprehensive picture of how people with mental illness, substance abuse, and co-occurring mental health or substance use disorders transfer through the justice system.

In summary, the promise of risk assessment tools is that they are research driven, constrain bias, and predict (better than other available options) the likelihood that a person will re-offend and comply with legal orders and directives. Whether scored by a human or computer, risk assessment tools promise to be the best (i.e., most accurate, objective, and cost-effective) mechanism to guide decisions about the placement and treatment of pre-offenders, offenders, and reentrants. The hope is that risk assessment instruments also avoid unnecessarily exposing people to the harms and criminogenic conditions of jails and prisons and the invisible punishments that formerly incarcerated people live with once released [19]. Risk assessment instruments are said to remove a significant degree of discretion, and proponents hope that they can help solve many of the problems within justice systems, including the race and class disparity that plagues the US justice system.

3. The Research Base of Risk Assessment Instruments

The measures encoded in instruments are grounded in the risk factor prevention paradigm—a set of individual, family, peer, school, and community factors found in longitudinal studies to be linked to delinquency and violence [20,21,22]. The general understanding of risk factor research is that incremental increases or decreases in causal, determinate, or predictor variables produce increases or decreases in offending. Proponents claim that “risk-focused prevention is easy to understand and to communicate, and it is readily accepted by policymakers, practitioners, and the general public” [23].

Risk factor research identifies the key correlates—sometimes talked about as if they are causes—found within the individual (e.g., physical health, mental health, attitudes) or in relation to their social environment (e.g., neighborhood, family, peers) and life circumstances (e.g., living situation, employment status) [24]. This correlation may include individual experiences, personality dispositions, and attitudes [25,26]. The research suggests that early interventions focused on correcting these deficiencies are likely to reduce the offending outcomes. For example, researchers identify the risk factor of inadequate parental supervision as a critical determinant or predictor in the life experience of offenders. Thus, interventions such as parental management training should be undertaken to improve parental supervision. The implications are that any agency or community-based group can prevent offending by implementing measures designed to counteract the risk factors found within an individual, their families, friends, schools, and community. Echoing this, Farrington claims that “A key advantage of the risk factor prevention paradigm is that it links explanation and prevention, fundamental and applied research, and scholars and practitioners” [21].

Numerous scholars argue that the risk factor prevention paradigm has troubling, biased elements and is methodologically flawed [27,28,29,30,31,32,33,34,35,36]. Case and Haines [7] revealed several methodological, conceptual, and applied flaws with the empirical basis of the risk factor prevention paradigm. O’Mahony reviews its methodological weaknesses and concludes that risk factor research is “…prone to merely feeding confirmation back to researchers of the theoretical assumptions they brought to the research” [35]. Moreover, risk-focused research is condemned of ‘psycho-reductionism,’ operationalizing explanations of delinquency that ignore social and structural influences or relegate them to second order ‘distal’ factors without valid reason or justification [34,37]. The discourse of risk brushes to the side what we know about the complexity of the social world [32] and, in so doing, relegates approaches that account for such complexity [31]. It is “easy to understand and to communicate” [21] only because it is oversimplified knowledge [32] that sells simplistic solutions to complex social, economic, and cultural problems [37]. With these and other shortcomings, risk factor research and the list of risk factors are of limited utility to inform policy and practice [34].

Furthermore, while on the surface the label at-risk is race- and class-neutral, in the United States, risk factors are differentially attached to poor people of color who are from inner-city communities [8,10] and in Canada and Australia to First Nation and Aboriginal youth [38,39,40]. Relatedly, critics contend that the label of ‘risk’ reinforces the criminalization and exclusion of the most vulnerable and often most impoverished people in society [28]. One such example is removing at-risk students from class to be counseled, punished, or suspended for aggressive or rule-breaking behavior. In practice, risk-based policies too often exclude, marginalize, and criminalize at-risk people—particularly immigrants and people of color [33,41,42].

Risk assessments instruments, I wish to argue, suffer from similar problems as the risk factor paradigm. The tallying of risk factors removes important features of the social world that has the effect of concealing unfair justice system conduct and inequitable economic and social conditions that illuminate problematic, destructive, or harmful behaviors. It leads to blaming ‘risky’ individuals for their contexts. Furthermore, while risk assessment instruments, and their summary risk categorization, appear race- and class-neutral, some children and adults will invariably be assessed as a higher risk. Those from racial or ethnic groups who are historically neglected, marginalized, and mistreated are most likely to be assessed as high risk. The particular groups vary by region. For example, in the US, generationally mistreated and marginalized people are Black and Indigenous Americans. In Canada, Australia, and New Zealand, these are First Nation people and Aboriginal people, respectively [43]. Yet, many scholars and policy makers still believe that risk assessment instruments can minimize incarceration rates because instruments predict better than chance and within acceptable margins of error when incarceration is necessary, even though sometimes people of color receive higher scores on tools than non-minorities [44]. This is problematic for various reasons.

4. Risk Assessment Instruments’ Reliance on Criminal History

A problem lies with the data that risk assessment instruments rely on. Some critics view these tools as having only a veneer of objectivity [9] or, taking the implications one step further, perpetuate biased decision making that disfavors Black people and poor people, and thus perpetuate inequities in society [8,10]. To examine this, we should look to the data that risk assessment relies on, determine whether the input data contain disparities and, if so, disentangle the nature of the disparities.

Various risk assessments used throughout the justice-system process rely heavily on criminal history operationalized in multiple ways for adults and children. A criminal history can include ‘age at first conviction’, ‘the number of prior periods of supervision’, ‘the number of prior probation or parole revocations’, ‘number of felony convictions’, ‘conviction or juvenile adjudication of a serious offense,’ and ‘conviction or juvenile adjudication in, for example, the last 12 months.’ For children, different risk assessment instruments also use ‘prior adjudication for felony changes’, ‘prior arrest’, ‘prior referrals to court.’ For each of these, a set number of events is given a higher score. For example, probation officers might tally 3 points for one prior adjudicated felony charge and 5 points for three or more prior adjudicated felony charges. Risk assessment instruments also score ‘prior escapes from confinement or custody, ‘failures to appear,’ and ‘pending cases or petitions.’ Though controversial and often not recommended [45], in some jurisdictions, children are scored based on simple misdemeanors, technical probation violations, status offenses such as curfew violations, and other low-level behaviors.

Frustratingly, many risk assessment instruments are proprietary and unavailable for inspection, whereas, in many jurisdictions, justice system agencies configure a unique risk assessment instrument by borrowing from a standard menu of risk factors that have been tested in other contexts and jurisdictions [45]. This customization can make it challenging to speak accurately about all instruments as a group. Regardless, there are many commonalities. For example, in addition to the above criminal history measures, justice system agents can count events known as aggravating factors. These factors can include ‘documented history of criminal gang involvement’, ‘no known local community ties,’ or ‘intoxicated upon referral.’ Alternatively, justice system agents can reduce scores by tallying mitigation factors (labeled on instruments as mitigation points). These might include ‘safety and stability of the home as an alternative to secure detention’, ‘record of regular school attendance,’ or ‘no arrest or referral for detention within the last 12 months’ [45]. Lastly, risk assessments instruments often include on its checklist ‘having a parent with a criminal history’, ‘having a parent who is or has been incarcerated,’ and ‘having a sibling with a criminal history.’

The risk assessments that heavily weigh offending history, such as initial court decisions, pretrial and post-conviction level of custody, sentencing, and the like, are regularly used at the ‘front-end’ of system involvement, whereas criminal history before incarceration is often not tallied in prison release risk assessment instruments. Instead, these instruments tally criminal history while incarcerated. Using criminal history to assess risk levels is logical, enjoys historical precedent, and is supported empirically. Logic notwithstanding, there are data that disfavor its efficacy and competing reasons that undercut the soundness of relying so heavily on criminal history in the assessment process. Moreover, unlike in the medical field, where it is straightforward to use empirically supported risk factors to predict a higher probability of an adverse health outcome, external biases feature in many justice systems.

Proponents would respond that criminal history as a measure of risk has been validated in multiple studies across multiple jurisdictions and is associated with both official records and self-reported illegal activity. This point is valid; however, the accuracy of a criminal history is not the problem. When justice system agencies assess the chance of unwanted behavior in the future, some risk domains are differentially prevalent and more frequently experienced by some people. Much of this disparity is caused by, or due to, forces external to the person being assessed. As such, it is not as straightforward as assessing someone’s risk for heart disease by using history and frequency of cigarette smoking, where smoking a pack of cigarettes means the same thing for anyone who smokes 20 cigarettes a day. In other words, 20 cigarettes are 20 cigarettes.

In the US, and indeed in many other regions worldwide, minority, marginalized, and disadvantaged groups are systematically treated differently by external forces that lead to misrepresentations of various risk factors, for instance, criminal history. Therefore, there is not a linear association between some risk domains and an objective probability rating, i.e., an impartial summary risk score, in the same way that there is a linear association between, say, the number of cigarettes smoked per day and for how long, and risk classification for heart disease. Racial biases and systemic socio-economic maltreatment of, and discrimination against, Black and Indigenous Americans in the US inflate criminal history, which inflates the overall summary risk scores. In turn, inflated summary risk scores magnify the chance of being assigned a higher risk categorization. The score and categorization sway significant life-altering decisions about whether an individual is eligible for secure detention, non-secure detention, alternative programming, or release.

5. Sources of Biased Criminal Histories

Detection and prosecution of crime and delinquency involve significant discretion from police and other agents of social control, particularly those involving minor or lower-level crimes such as drug law violations. However, discretion, which often involves incomplete knowledge, combined with human cognition, invariably involves presumptions, partiality, and implicit or even explicit bias. Thus, obtaining a criminal history is infused with discretionary choices by justice system agents, who are influenced by the fallibility of human cognition, whether intentional or unintentional. We see this manifest in several instances of policing and diversion choices.

Moreover, the persistence of racial disparities in encounters with the police, prosecutorial charging, and bail and sentencing decisions reveal that racial bias penetrates all corners of the US justice system, and they penetrate the domain of risk assessment and its instruments. The consequence is earmarking for heightened surveillance and punishment people of color, particularly those living in low-income, high violent crime areas. As such, the US justice system is an impactful source of biased data. As a side note, consistent and generational disparate treatment of some groups is not unique to the US. Therefore, the same reasoning will likely hold in other regions that rely on risk assessment instruments in carceral settings, and do not mitigate the factors that mediate scores on these assessments.

In the US, a large body of scholarship has found that Black children and adults are disproportionately patrolled, stopped, searched, arrested, prosecuted, and sentenced to confinement for ‘street crimes’ [46,47,48,49]. To illustrate this in more detail, I draw on just a portion of insightful studies.

Researchers at the Stanford Open Policing Project started in 2015 collecting and standardizing over 200 million traffic stop and search data records from across the US [50]. After analyzing 100 million traffic stops from 21 state patrol agencies and 29 municipal police departments and controlling for the drivers’ age and gender, they found that Black drivers are generally more likely to be stopped than White drivers. Combining information on both search rates and finding illegal drugs or weapons rates, Pierson et al. [50] found that police are typically more willing to stop Black and Hispanic drivers based on less suspicious behavior than White drivers. Analyzing the post-stop phase of the stop decision, Ridgeway [51] found that although citation rates and consent search rates were racially proportional, Black drivers’ stops were longer, and they were twice as likely to be searched as non-Black drivers.

Numerous other studies similarly find that local police disproportionately stop people of color while driving [52]. In analyzing the New York City Police Department’s stop-and-frisk policy, Gelman, Fagan, and Kiss [53] observed racial disparities in stops across police precincts and communities. When controlling for relevant factors, including the crime rate in neighborhoods where stops occur and race-specific estimates of crime participation, Black and Hispanic people were stopped at disproportionately higher rates compared to their White counterparts, specifically, 1.5-fold to 2.5-fold higher [53].

Beyond preemptive stops of people in public places such as sidewalks or while driving, arrests related to drug enforcement are significant when considering the criminal history and risk assessment instruments (i.e., racialized arrests causing racialized criminal records). For example, Beckett et al. [54] conducted a study of the policing of drug markets in the US city of Seattle in 1999 and 2001. They found that 51% of those arrested in Seattle for drug offenses were Black, which by far outpaced their demographic share in the city (8.4%). In a comprehensive study of marijuana arrests by the American Civil Liberties Union [55], Black Americans were found to be 3.7-fold more likely than White Americans to be arrested for possession and, in some places, 10-fold more likely, even though both groups used marijuana at comparable rates. As Provine [56] points out, “because drug dealing is a furtive, largely consensual activity, officers must seek it out. Their attitudes about where the “hunting” is good inevitably affect arrest statistics.”

Beckett and Brydolf-Horwitz [57] recently observed a modest decline in drug arrests over the last ten years in the US and the number of drug arrests involving Black people declined 31.3%. Nevertheless, they note that Black people remain “notably over-represented among drug arrestees” [57]. As an aside, they observed that drug arrests of Asian/Pacific Islanders increased 64.4% between 2007 and 2018 and drug arrests involving Native Americans increased 59.5%. Both groups primarily inhabit suburban and rural areas, rather than the urban areas that have recently experienced drug policy reforms, adding further support for how significant discretionary decisions can be in drug policy and drug enforcement.

The overwhelming racial disparities in rates of police stops and arrests in certain areas point to implicit or explicit racial bias in officer discretion. Study after study, researchers observe these disparities. Using data from the Division of Criminal Justice Services, the New York City Police Department, and the US Census, researchers from the John Jay College of Criminal Justice [58] documented patterns in misdemeanor arrests in New York City from 1990 to 2013. The misdemeanor arrest rate for 16- and 17-year-olds nearly quadrupled between 1990 and 2010, before declining some in 2013. The exponential increase expanded the already wide racial disparities of misdemeanor arrests. Black children’s arrest rates for misdemeanors during these three years were 6.2 percent, 24.7 percent, and 17.5 percent, respectively, whereas the corresponding arrest rates of White children during the same three periods were just 1.3 percent, 6.1 percent, and 3.7 percent, respectively. Chauhan et al. [58] observed a similar pattern for males 18–20 years old. The misdemeanor arrest rate for Black versus White males was 7.1 percent versus 2.1 percent in 1990, 28.2 percent versus just 8.9 percent in 2010, and 20.4 percent versus just 5.7 percent in 2013.

Using nationally representative, individual-level data, Steven and Morash [59] assessed trends in youth likelihood of arrest and court involvement while controlling for delinquency. They aimed to determine whether racial or ethnic minority youth have been disproportionately affected by the ‘tough on crime’ punitive trend towards children between 1980 and 2000. They found that Black and Hispanic boys’ probabilities of arrest and subsequent court involvement, net of effects of delinquency, were distinctively prominent. In 2000, Black male children had the highest likelihood of being charged with a crime, regardless of self-reported offending. Additionally, both Black and Hispanic boys were more likely than White boys to be placed in a correctional institution, irrespective of the frequency of delinquency self-reported by the children [59]. Although no single study should close the book on the issue, these data provide further support that racially-biased decisions by justice system personnel and agencies, not differences in crime commission by race, drive the observable racial disparity in stops and arrests.

An additional issue related to criminal history is that some groups are involved in, and victims of, meaningfully higher rates of serious crimes than others. As a result, police watch, respond to calls, investigate, and proactively intervene in neighborhoods associated with high crime rates. For example, in the US, violent crime rates, particularly homicide, are higher in Black disadvantaged communities than all other areas and populations—significantly so, as young Black males are 15-fold more likely to die of violence than White males [60]. Hence, Black people in these communities have more contact with police and police policy. However, as described above, young men of color experience higher arrest rates when controlling for crime commission by race, suggesting that proactive policing strategies in crime “hot spots” contribute overwhelmingly to the disproportionately higher arrest rates. Moreover, serious crimes do not exist in a vacuum, nor are they undergirded by biological explanations or proclivities of a particular group. Instead, the harmful effects of profound “levels of deprivation, insecurity and thwarted opportunity” explain why impoverished Black people experience higher rates of serious crime and victimization than other groups [60]. As such, police allocate resources, receive more calls for service, and spend more time where poor Black people live.

Multiple studies have concluded that Black people are treated more severely than Whites at several decision points in the prosecution and sentencing of criminal defendants [61,62,63]. Soon after the arrest, the judge determines whether a defendant should be released or detained pretrial. If the latter, and someone is detained pretrial, they are more likely to be convicted of a felony or accept a less favorable plea deal [64]. In other words, pretrial detention has the effect of increasing the odds of being sentenced to jail or prison, and for a longer time, compared to those released, for example, on bail, before conviction and sentencing [61,65]. Importantly, these studies find that racial disparities in earlier decisions increased disparities in later decisions and sentencing outcomes [63]. As such, beyond policing, justice system processing and sentencing influence individuals’ criminal histories in racialized ways.

The effect of all of this on the minority communities is bleak. Black and Hispanic Americans are disproportionately locked up in US jails and prisons, making up 56% of the incarcerated population even though their share of the total population is just 32%. Put another way, in 2014, Black people made up 13% of the US population and 40% of the people incarcerated (a rate of 2306 per 100,000) [66]. Compared to White people, these figures are astounding, as White people made up 64% of the US population in 2014, but only 39% of the people of incarcerated people (a rate of 450 per 100,000) [66]. Compared to the global rates of incarceration that hover around, but often less than 100 per 100,000, the Black incarceration rate of 2306 per 100,000 in the US is shocking.

Looking at age groups is similarly eye-opening. Using Bureau of Justice Statistics data from 2019, researchers at the Prison Policy Initiative [67] disentangled racial disparities in men’s imprisonment. For ages 18–19, the rate (per 100,000) of White men locked up in US prisons is 58, while the rate for Black men in this age category is 720, a rate 12-fold greater than for White men. For ages 20–24, the rate (per 100,000) of White men in US state and federal prisons is 347, while the rate for Black men in the age category is 2772, which is approximately 8-fold greater. The pattern continues for every age category, with the highest overall rates being in the age category of 35–36. The rate (per 100,000) in this age group of White men is 958, while the rate for Black men in the age category is 4832, which is over 5-fold greater than for White men [67]. Using somewhat different sources of data and only looking at state prisons but finding a similar pattern, Lerman and Mooney [68] found substantial variation in populations over thirteen years:

“From 2005 to 2018, the number of people in prison nationwide declined by 15%. (from 452 to 393 per 100,000 population). This change was uneven across racial/ethnic groups. Prison rates among Latinx and Black Americans declined by 28% and 33%, respectively (from 1463 to 1097 among Blacks, and 474 to 371 among Latinx), largely driving the overall trend, while White prison rates remained stable across the decade (266 in 2005 and 263 in 2018). Despite these declines, however, the incarceration rate among Blacks was still four-fold that of Whites in 2018.”

Although levels of youth confinement have significantly declined in recent years, the rate at which Black children are confined is 6-fold more than that of White children, and the commitment rate is 4-fold more than that of White children [69]. Black male children account for 42% of children in juvenile facilities, and Black female children account for 35%, while only 14% of children in the US are Black [70]. The NAACP’s [71] fact sheet summarizes this aptly: “African American children represent 32% of children who are arrested, 42% of children who are detained, and 52% of children whose cases are judicially waived to criminal court. African American children represent 14% of the population.”

6. Racial Disparate Results from the Reliance on Criminal History

From the brief review of just some of what we know from decades of research and analysis, the racial disparities in the adult and youth justice systems derive from preemptive policing, policing deployment, and discretionary choices by justice system personnel at multiple decision points. Furthermore, but not discussed here, pretrial dynamics, sentencing, trial dynamics, and several crime policies frequently disadvantage people of color because of their disproportionate share of those who live, generationally, with economic and social disadvantage. The implication of discernable racial disparities in the adult and youth justice systems in the US and many other regions is that the numerous criminal history items contained in risk assessment instruments will capture and replicate racial disparities in the scores and categorizations of risk. Specifically, a great number of prior arrests, convictions, and the like, increases risk scores, which increases the likelihood of formal or punitive sanctioning. Given that Black people in the US have a greater number of prior arrests, convictions, because – in significant part – of discretionary choices by justice system personnel, Black people in the US will have higher risk scores, and hence, a higher likelihood of formal or punitive sanctioning. Consequently, risk assessment instruments play a role in repeating and amplifying the likelihood of future racial disparities in punitive sentencing and sanctions, thus exacerbating carceral and social inequalities [72].

Criminal history is an artifact of two pillars of bias: the differential exposure and differential justice system reactions in policing (e.g., traffic stops and searches, stop and frisk, arrests), court processing (pretrial decisions, sentencing), and prosecution (discretionary decisions in bail, plea bargaining, charges). Yet, risk assessment instruments treat past reactions by justice system personnel as objective data points, and therefore as objective indicators of the probability to re-offend when they are neither. As US law professor, Sandra Mayson [73] poignantly illustrates the problem with risk assessment instruments in this anecdote:

“When I worked in New Orleans as a public defender, the significance of arrest there varied by race. If a black man had three arrests in his past, it suggested only that he had been living in New Orleans. Black men were arrested all the time for trivial things. If a white man, however, had three past arrests, it suggested that he was really bad news! White men were hardly ever arrested; three past arrests indicated a highly unusual tendency to attract law enforcement attention. A race-blind algorithm would not observe this difference. It would treat the two men as posing an identical risk. The algorithm could not consider the arrests in the context of disparate policing patterns and recognize that arrests were a much less significant indicator of risk for a black man than for a white man. It would perpetuate the historical inequality by overestimating the black man’s relative riskiness and underestimating the relative riskiness of the white man.”

7. Reconsidering the Meaning of Risk Scores

This article poses a serious challenge to using risk assessment instruments in carceral settings by showing that they are inadvertently, but carelessly biased predictions of whether a particular child or adult will behave in an unwanted way in the future. In the process of tallying a summary risk score and determining a risk classification to inform justice system decisions, some risk domains, particularly criminal history, are too easily understood and communicated as benign and neutral evidence when they are neither. Risk domains are highly influenced and even produced by discretionary decisions of justice system personnel, in addition to historical and contemporary social forces, social policy, and crime policy, which are frequently classed and racialized. A certain margin of error is acceptable, but a substantial volume of information entered into risk assessment instruments are highly contextualized, such as Black people having a greater probability of being stopped, detained, or placed in secure confinement than their White peers. However, this context is erased by an instrument in the conversion to an easy-to-read and understand summary risk score and risk classification. The implication is that risk assessment instruments, just as the risk factor paradigm, disguise important contextualized factors—and not only racial disparities in carceral contact and discretionary decisions by carceral agents, but a range of economic, political, and socio-structural dynamics [74].

The core of the problem is that risk assessment instruments, under the guise of scientific impartiality and neutrality, systematically disfavor disadvantage. Additionally, by inference, risk instruments systematically favor advantage. They categorize people at a lower level of risk by virtue of being in a class or racial group that is exempt from over- or hyper-exposure to police deployment and patrolling decisions, preemptive monitoring and checks, and detrimental discretionary choices by justice system personnel at multiple points in the criminal justice process. That is to say, economically, socially, politically, and racially advantaged people will inevitably fair better in risk assessments of the probability of future misconduct. Therefore, risk assessment instruments convey an incomplete story about all groups of people and therefore should be considered inadequate ‘evidence’ in justice system decision making, for example, to deny or grant liberty, extend or reduce a sentence, or qualify or disqualify to be diverted from incapacitation.

While not excusing harmful and destructive behaviors that are consequential to other people and communities, risk scores and the risk classification do more than disfavor disadvantage and favor advantage. They reduce an understanding of destructive, harmful behavior into something devoid of deep analysis. They reduce criminogenic issues such as understandable school failure when attending an inadequately funded school, anger towards law enforcement agents when frequently being stopped and questioned, and a host of less than skillful coping behaviors to adverse neighborhood conditions, into a numerical value, similar to an insurance rating. Risk assessment instruments may predict better than a flip of a coin or clinical judgment, but they dissuade (or at least discourage) considerations for why a person scores in the way they do. In other words, they disassociate problematic, destructive, or harmful behaviors from the social world.

At the same time, risk assessments generate a form of evidence, a risk score and classification, that then informs legal and carceral alternatives that highly impact the lives of children and adults. This generative evidential feature has the appearance of scientific endorsement that certifies that those who were previously disproportionately watched, detained, arraigned, processed, sentenced, and punished will be categorized as riskier. As such, going forward, they will inevitably be similarly scrutinized and controlled as justified by the score and classification.

8. Reversing the Viewpoint of What Is Evaluated

To upend this cycle, there are going to be difficult choices to confront. Risk domains could be used entirely differently, one that turns the risk gaze away from individuals towards the carceral state and to its reactions to unwanted behavior. Namely, risk factors would inform state- or local-level policies on delinquency and crime, as well as racialized justice system reactions to them (for an analogous point but in the context of protecting children from human trafficking, see Fedina et al.) [75]. In this way, risk domains would only inform decisions on how to reform the justice system itself while assisting agencies in developing prevention and intervention responses to social and economic correlations to delinquency and crime and the material deprivations necessary for rehabilitative programming to stick [76]. As such, racially disproportionate risk scores could be used locally to push reforms in policing and sentencing to reduce racial disparities, for example, make racial profiling illegal, better screen police officers, end the over-policing of urban and low-income communities, and prosecute laws equally.

If we ascribe to fair and accurate prediction of risk, Eckhouse and colleagues may have it right that we will need to rethink the enterprise of assessments and “… use human insight and human judgment to decide what [risk scores and categorizations] mean and when we should use them” [77]. Unlike in the UK, where risk assessment instruments are no longer popular as they once were, in many regions, including the US and Canada, they are deeply ingrained into the adult and juvenile justice systems. Given this, assessing risk is ongoing for now, so in the short term, justice system personnel must rethink the meaning and utility of a risk score and risk categorization. At the very least, they should consider the contextual and situational influences that undergird risk categorizations when planning interventions [78]. Although this article is not intended to detail solutions and recommendations, what is for sure, is that no longer should justice system personnel consider these instruments as providing unbiased evidence. As it stands, without a radical shift in the use of risk scores and categorizations, it is difficult to see how in places where inequity, implicit biases, and overt discrimination are sewn into the fabric of society that the ‘evidence’ that risk assessment instrument offer can ever be impartial, unbiased, or non-discriminatory.