Judgment by algorithm

Back to TopCommentsE-mailPrintBookmark and Share
focus-algorithm-shutterstock-459015925-15col.jpg (

Eric Loomis just wants to know how, exactly, an algorithm determined he posed a high risk to the community.

The Wisconsin man was charged in connection with a February 2013 drive-by shooting in La Crosse. He has continued to maintain he was home cooking dinner at the time of the incident, but he did plead guilty to two related offenses of trying to flee police and driving a motor vehicle without the owner’s permission.

Sentencing him to six years of incarceration and five years of extended supervision, the trial judge pointed to Loomis’ history of recidivism and failure to take responsibility. Also, the court noted the result of an actuarial assessment tool that labeled him likelier to reoffend.

Loomis filed a motion for post-conviction relief, arguing, in part, that his due process rights were violated because the court relied on the computer-based assessment program COMPAS (Correctional Offender Management Profiling for Alternative Sanctions). A central question is whether the claims by COMPAS developer Northpointe Inc. — that the calculations were proprietary — prevented Loomis from challenging the scientific validity of the assessment.

Now the Supreme Court of the United States is considering taking a closer look. Loomis filed a petition for writ of certiorari in October 2016 after the Wisconsin Supreme Court denied his motion for resentencing. In March 2017, the court invited the acting solicitor general to file a brief on the case.

Risk assessments and algorithms are not new. They have been used routinely by criminal justice systems in many states to predict how likely the defendant or offender is to commit another crime.

Indiana has a set of such tools to help determine conditions for pretrial release, community supervision, prison intake, and re-entry. The Indiana Risk Assessment System scores an individual’s responses to a series of questions that range from criminal history and substance abuse to employment and social support.

Like the Wisconsin tool, the IRAS scores are based on algorithms.

Both the Indiana Public Defender Council and the Indiana Prosecuting Attorneys Council expressed reservations about IRAS. The organizations note IRAS scores have not been validated as to their accuracy

when applied to the Indiana offenders. Moreover, they worry judges will rely more on the numerical score than their own discretion.

IRAS scores are actuarial and not individualized. The assessment tools put the defendants or offenders in certain groups based on shared characteristics. The IRAS-Pretrial Assessment Tool labels an individual as low-, medium- or high-risk, depending on the typical behavior of the group.

Larry Landis, executive director of the Indiana Public Defender Council, compared the categorization to auto insurance companies charging higher premiums for teenage male drivers because, as a group, they are involved in more car accidents.

“Maybe for insurance it’s OK, but we’re talking about liberty when using this tool to predict human behavior,” Landis said. “Nobody’s good at that.”

Human element still required

Edward Latessa, director and professor of the School of Criminal Justice at the University of Cincinnati, led the design of the IRAS. He based the tools on the Ohio Risk Assessment System he developed in 2006.

Latessa and his research team interviewed 1,800 offenders in Indiana to determine the factors that indicated a risk of additional criminal acts. The researchers are now preparing to do a validity study of IRAS using a database of information collected from 350,000 offenders.

Currently, 11 counties in Indiana are testing the IRAS Pretrial Assessment Tool as part of a pilot program to reduce the imposition of bail. Courts in the pilot counties are using the IRAS-PAT designed to help determine the conditions of release without cash or bond. The tool asks seven questions to rank the likelihood of the individual failing to appear in court and committing another offense.

Latessa maintained the IRAS score is based on information courts typically gather piecemeal. When using the Indiana tools, the interviewers, such as probation officers, still must use their personal expertise in evaluating the individuals.

Noting risk assessment tools are not going anywhere, David Powell, executive director of the Indiana Prosecuting Attorneys Council, said they must provide a valid, accountable, objective analysis and the outcomes have to be measured.

Algorithms cannot replace personal judgment, he said, giving the example that IRAS does not consider the nature of the crime. A person charged with a heinous act might be deemed low-risk if it was the first crime he or she committed. Prosecutors who turn the decision-making over to the victims have been disciplined for transfer of discretion, and Powell sees the same possibility for running afoul of court rules when judges rely too much on IRAS.

“Prosecutors are not big fans of IRAS, by and large,” he said. “The fear we have is the tool will be used in lieu of judicial discretion.”

Allen Superior Judge John Surbeck echoed Latessa by emphasizing the IRAS scores are simply additional information for the bench to review and should not be given more weight.

He acknowledged the concern, even from judges, that the assessment tools will replace judicial discretion, but he still believes it will enhance decision-making. Judges will continue to enlist their expertise when considering a defendant’s criminal history and nature of the current offense. Furthermore, they will have guidance from the state’s sentencing statute, which clearly identifies the mitigating and aggravating factors to be considered.

“It’s possible (a judgment) could be reduced to a numerical but most of my colleagues, I think, are truly dedicated to doing what they’re supposed to do,” Surbeck said. “It would be tempting but I trust my colleagues to do better than that.”

Let it percolate

News reports, including a series done by ProPublica in 2016, have focused on the potential for bias against minorities built into these kinds of assessment tools. Landis described particular questions in the IRAS, which he said ask the age of the offender’s first arrest and whether the offender has any relatives in prison, as being racially biased.

Latessa disputed the allegations. He pointed to two 2016 academic studies led by University of California researchers that repudiated assessments are skewed toward certain groups. Also, he noted relying on intuition or personal judgment alone can result in biased decisions.

In its respondent brief, Wisconsin advised the Supreme Court to let the risk assessment algorithms percolate longer in the lower courts since, to date, the question has been addressed by only one other state — Indiana.

The case, Malenchik v. Indiana, 928 N.E.2d 564 (Ind. 2010), considered Tippecanoe Superior Court’s use of the results from risk evaluation and assessment instruments when sentencing the defendant, Dan Malenchik.

Writing for the court, Justice Brent Dickson explained that neither assessment tool the trial court used “are intended nor recommended to substitute for the judicial function of determining the length of sentence appropriate for each offender. But such evidence-based assessment instruments can be significant sources of valuable information for judicial consideration.”•


Post a comment to this story

We reserve the right to remove any post that we feel is obscene, profane, vulgar, racist, sexually explicit, abusive, or hateful.
You are legally responsible for what you post and your anonymity is not guaranteed.
Posts that insult, defame, threaten, harass or abuse other readers or people mentioned in Indiana Lawyer editorial content are also subject to removal. Please respect the privacy of individuals and refrain from posting personal information.
No solicitations, spamming or advertisements are allowed. Readers may post links to other informational websites that are relevant to the topic at hand, but please do not link to objectionable material.
We may remove messages that are unrelated to the topic, encourage illegal activity, use all capital letters or are unreadable.

Messages that are flagged by readers as objectionable will be reviewed and may or may not be removed. Please do not flag a post simply because you disagree with it.

Sponsored by
Subscribe to Indiana Lawyer
  1. One can only wonder whether Mr. Kimmel was paid for his work by Mr. Burgh ... or whether that bill fell to the citizens of Indiana, many of whom cannot afford attorneys for important matters. It really doesn't take a judge(s) to know that "pavement" can be considered a deadly weapon. It only takes a brain and some education or thought. I'm glad to see the conviction was upheld although sorry to see that the asphalt could even be considered "an issue".

  2. In response to bryanjbrown: thank you for your comment. I am familiar with Paul Ogden (and applaud his assistance to Shirley Justice) and have read of Gary Welsh's (strange) death (and have visited his blog on many occasions). I am not familiar with you (yet). I lived in Kosciusko county, where the sheriff was just removed after pleading in what seems a very "sweetheart" deal. Unfortunately, something NEEDS to change since the attorneys won't (en masse) stand up for ethics (rather making a show to please the "rules" and apparently the judges). I read that many attorneys are underemployed. Seems wisdom would be to cull the herd and get rid of the rotting apples in practice and on the bench, for everyone's sake as well as justice. I'd like to file an attorney complaint, but I have little faith in anything (other than the most flagrant and obvious) resulting in action. My own belief is that if this was medicine, there'd be maimed and injured all over and the carnage caused by "the profession" would be difficult to hide. One can dream ... meanwhile, back to figuring out to file a pro se "motion to dismiss" as well as another court required paper that Indiana is so fond of providing NO resources for (unlike many other states, who don't automatically assume that citizens involved in the court process are scumbags) so that maybe I can get the family law attorney - whose work left me with no settlement, no possessions and resulted in the death of two pets (etc ad nauseum) - to stop abusing the proceedings supplemental and small claims rules and using it as a vehicle for harassment and apparently, amusement.

  3. Been on social security sense sept 2011 2massive strokes open heart surgery and serious ovarian cancer and a blood clot in my lung all in 14 months. Got a letter in may saying that i didn't qualify and it was in form like i just applied ,called social security she said it don't make sense and you are still geting a check in june and i did ,now i get a check from my part D asking for payment for july because there will be no money for my membership, call my prescription coverage part D and confirmed no check will be there.went to social security they didn't want to answer whats going on just said i should of never been on it .no one knows where this letter came from was California im in virginia and been here sense my strokes and vcu filed for my disability i was in the hospital when they did it .It's like it was a error . My ,mothers social security was being handled in that office in California my sister was dealing with it and it had my social security number because she died last year and this letter came out of the same office and it came at the same time i got the letter for my mother benefits for death and they had the same date of being typed just one was on the mail Saturday and one on Monday. . I think it's a mistake and it should been fixed instead there just getting rid of me .i never got a formal letter saying when i was being tsken off.

  4. Employers should not have racially discriminating mind set. It has huge impact on the society what the big players do or don't do in the industry. Background check is conducted just to verify whether information provided by the prospective employee is correct or not. It doesn't have any direct combination with the rejection of the employees. If there is rejection, there should be something effective and full-proof things on the table that may keep the company or the people associated with it in jeopardy.

  5. Unlike the federal judge who refused to protect me, the Virginia State Bar gave me a hearing. After the hearing, the Virginia State Bar refused to discipline me. VSB said that attacking me with the court ADA coordinator had, " all the grace and charm of a drive-by shooting." One does wonder why the VSB was able to have a hearing and come to that conclusion, but the federal judge in Indiana slammed the door of the courthouse in my face.