AP English Language — Mixed Skills II — Drill 4

The following text is adapted from a modern essay on the ethics of algorithmic decision-making. The most consequential thing we have done with artificial intelligence is not teach it to play chess, or recognize faces, or generate plausible paragraphs. The most consequential thing we have done is teach it to make decisions about people — about who gets a loan, who is flagged as a flight risk, who is recommended for a job interview — and then describe those decisions as objective. The word objective is doing a great deal of work here, and almost none of it is honest. An algorithmic decision is not objective in the way that a mathematical proof is objective. It is the product of a training process that reflects the data it was trained on, the choices made by engineers about what to optimize for, and the values — often implicit, rarely examined — embedded in the decisions of the humans whose behavior the algorithm has learned to replicate. When an algorithm trained on historical hiring data recommends fewer women for senior roles, it is not discovering a truth about women's qualifications. It is replicating a pattern that human bias produced. The algorithm does not introduce the bias. It launders it. The appeal of algorithmic decision-making is understandable. Human judges are inconsistent. Human interviewers are susceptible to irrelevant factors — attractiveness, accent, the quality of a handshake. The algorithm, at least, is consistent. It will make the same decision in the same circumstances every time. But consistency is not fairness. An algorithm that consistently disadvantages one group is not more fair than a human who inconsistently disadvantages the same group. It is more efficient at the disadvantage. There is a further problem. Human decision-makers can be questioned. A judge can be asked to explain her reasoning. A hiring manager can be challenged on her criteria. The algorithm, in most current deployments, cannot. It produces an output without an explanation — a score, a flag, a recommendation — and the people subject to those outputs often have no meaningful way to contest them. The opacity of algorithmic systems is not an accident of their technical complexity. In many cases, it is a design choice: explainability is expensive, and the costs of unexplainability fall most heavily on the people with the least power to demand it. What we need is not a moratorium on algorithmic decision-making — the efficiency gains are real, and some of the human inconsistencies these systems replace are genuinely harmful. What we need is a reckoning with the word objective: an acknowledgment that algorithmic systems encode values, that those values can be examined and contested, and that the people most affected by these decisions have a right to understand and challenge them.