Bias and discrimination

The Fairest Decision-Maker?

Here's a system that's never met anyone, holds no opinions, and applies the same rule to every application. It sounds like the fairest decision-maker ever built. However, it can also be the most reliably discriminatory, and the reason isn't what most people assume. In this lesson, we'll explore how bias enters AI, why simple fixes fail, and why this creates a significant commercial exposure for your clients.

A Mirror, Not a Judge

To understand AI bias, think of a model as a mirror, not a judge. If you show a mirror ten years of your firm's past decisions, it'll learn to reproduce them perfectly—including the parts nobody would defend out loud. It doesn't know which patterns you're proud of and which ones reflect historical prejudices. A mirror shows you the bias plainly; a model applies it silently and at machine speed, which is far more dangerous.

How Bias Enters the System

Bias doesn't usually happen because someone programmed a prejudice into the code. It enters through three main routes. First, historical decisions. Models learn from examples, so they inherit the biases of the humans who made those decisions in the past. Second, unrepresentative data. If the data doesn't reflect the diversity of the real world, the model struggles with minority groups. Finally, the objective. If we tell a model to 'maximize profit' without other constraints, it might find that the most efficient path involves discrimination.

The Proxy Variable Problem

Here's the core point: removing a protected characteristic does not remove the bias. You might think that by deleting the 'Gender' or 'Ethnicity' column, you've made the system blind. You haven't. Other fields, known as proxy variables, carry that information anyway. Postcode can stand in for ethnicity. Device type can signal age. The model doesn't need the column you deleted because the information routes around the deletion through these proxies.

Recap: Routes and Proxies

Let's recap what we've covered so far. AI is a mirror of the past, inheriting bias through data and objectives. Most importantly, simply deleting a sensitive column won't protect you, as proxy variables like postcode or device type allow the bias to route around your deletions.

Liability at Scale

Bias is a commercial risk, not just an ethical one. When a human is biased, the damage is limited to one desk. When a model is biased, the error is replicated across an entire caseload at speed. This creates several layers of exposure. Disparate impact can trigger legal action even without intent. Furthermore, biased patterns in AI are often easier to prove in litigation because the data provides a clear trail. Remember: 'the model decided' is not a legal shield. You are responsible for your tools.

Recap: Commercial Risk

To summarize the commercial side: AI scales bias from a single desk to an entire portfolio. This creates discoverable systemic risk where intent is irrelevant to liability, and the organization remains fully responsible for every automated decision.

The Explicit Stop

Testing a model for bias is a real discipline, and it belongs to the next course. What you take from here is narrower and more useful: a system trained on your past will repeat your past, and deleting one column will not stop it. You're now ready to recognize the silent, systemic nature of AI bias.