Why Do We Keep Creating Racist Robots?
Written by Fizzah Mansoor. A discussion on the use of machine learning in risk assessment systems.
It is almost impossible to talk about Artificial Intelligence without bringing up the myriad ways it could go wrong. Concerns about algorithms exhibiting discriminatory behavior in their predictions have led to extensive literature that tries to “blind” the algorithm to avoid exacerbating existing unfairnesses in society, which is achieved by removing demographic data, including race and gender.
A risk assessment score predicts whether someone accused of a crime is likely to commit another crime. A low-risk defendant is deemed unlikely to commit another crime, while a high-risk defendant is considered to be very likely to commit another crime. Risk assessments are used at various stages in the legal system, from assigning bond amounts to determining sentences. According to the investigative journalism organization ProPublica, they are also prone to mistakenly label black defendants as likely to re-offend — wrongly flagging them at almost twice the rate as white people (45% to 24%). (Kirchner et al., 2016)
Risk assessment algorithms are pointedly not trained on data that includes the race of defendants, yet they learn to have a racial bias; a machine learning algorithm that’s trained on current arrest data learns to be biased against defendants based on their past crimes, since it doesn’t have a way to realize which of those past arrests resulted from biased systems and humans. (Kleinberg et al., 2018) More specifically, the algorithm reconstructs race as a feature through other features, such as the type of crime committed, etc. (Green & Viljoen, 2020)
The problem, in part, seems to lie in our approach towards designing machine learning algorithms. Green and Viljeon diagnose the dominant mode of algorithmic reasoning as “algorithmic formalism” and describe how formalist orientations lead to harmful algorithmic interventions.
Algorithmic formalism involves the following orientations: objectivity, neutrality, internalism, and universalism. Although often reasonable and valuable within the context of traditional algorithmic work, these orientations can lead to algorithmic interventions that entrench existing social conditions, narrow the range of possible reforms, and impose algorithmic logic at the expense of others. While useful in a purely mathematical, abstract sense, these orientations cannot overcome the embedded biases in the data fed to the algorithm. (Green & Viljoen, 2020)
Addressing the ways in which algorithms reproduce injustice requires pursuing a new mode of algorithmic thinking that is attentive to the social concerns that fall beyond the bounds of algorithmic formalism and actively addresses social issues.
In Algorithmic Fairness, Kleinberg, Ludwig, Mullainathan & Rambachan empirically demonstrate how departing from algorithmic formalism to algorithmic realism accounts for factors that should not affect predicted outcomes while improving accuracy. By using nationally representative data on college students, their research underlines how the inclusion of a protected variable (specifically, race) not only improved predicted GPAs of admitted students (efficiency), but could also improve outcomes such as the fraction of admitted students who are black (equity).
The reason for this result is straightforward. Equity preferences involve increasing the fraction of black applicants admitted. However, within this notion, society is still served best by ranking as much as possible using the best possible predictions. This approach, ultimately, aids both equity and efficiency. Additionally, a preference for fairness should not change the choice of the estimator; accounting for the race should not mean that race becomes an estimator, but instead, it is used as a parameter for addressing the injustices that cause racialized groups to be at a disadvantage in the first place.
This approach entails a new mode of algorithmic thinking — “algorithmic realism” — that provides tools for computer scientists to account for the realities of social life and algorithmic impacts. Although not foolproof, these realist approaches will better equip computer scientists to reduce algorithmic harms and reason well about doing good. (Green & Viljoen, 2020)
References
Green, B., & Viljoen, S. (2020). Algorithmic Realism: Expanding the Boundaries of Algorithmic Thought. ACM Conference on Fairness, Accountability, and Transparency. https://dl.acm.org/doi/pdf/10.1145/3351095.3372840
Kirchner, L., Mattu, S., Larson, J., & Angwin, J. (2016, May 23). Machine Bias. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Kleinberg, J., Ludwig, J., Mullainathan, S., & Rambachan, A. (2018). Algorithmic Fairness. American Economic Association, 108, 22–27. https://www.aeaweb.org/articles?id=10.1257/pandp.20181018
