Drift

The Map and the Moving Roads

Welcome to our exploration of one of AI's most deceptive traits: the ability to fail without ever breaking. Here's a strange property of these systems: a model can be working perfectly on the day it goes live and be quietly wrong eighteen months later, without a single line of it having changed. Nobody touched it. The world moved instead. Think of a model like a printed map. A good map is perfectly accurate the day it's printed. It stays exactly the same—the ink doesn't move and the paper doesn't change. But over time, that map becomes wrong because the roads change around it. New highways are built, old streets are closed, and traffic patterns shift. Nobody notices the map's failure until someone drives into a field. Here's where the analogy breaks down: a map usually has its publication date printed in the corner. A model carries no visible date, and it sounds just as confident as the day you bought it.

Two Flavors of Drift

In the AI world, we call this drift. It's the process where the world the model was trained on stops matching the world it now operates in. As we've discussed, AI failure is often probabilistic and silent. Drift is the ultimate example: the system doesn't crash; it just slowly loses its grip on reality. First, there's Data Drift. This is when the input data changes. Imagine a model built to predict luxury travel demand. If a sudden economic downturn changes the 'typical' traveler's profile, the data the model sees today looks nothing like the data it learned from. Then, there's Concept Drift. This is when the relationship the model learned changes. A model might know that 'high credit utilization' usually means 'high risk.' But if a new government subsidy program changes how people use credit, that old relationship—that 'concept'—is no longer true.

Practice: Identifying the Cause

Let's test your instincts. A model built three years ago is now producing poor decisions. Where should you look first for the cause? It's a common thought, but that's usually not the culprit. In drift, the code hasn't changed at all. Look outside the system instead. Exactly. The instinct is to ask what someone did to the model, but usually the answer is nothing at all. Drift is caused by the environment: customer behavior shifted, a product changed, or a market moved. The model is still doing exactly what it learned to do; that's the problem.

Business Accelerants and Silent Decay

A common trap is believing that because a model was thoroughly checked at launch, it's safe. But model validation is a photograph, not a guarantee. It tells you the model matched the world on the day you checked it. Every day after that, the world moves and the model stays still. The most dangerous property is the silent period. Because the system doesn't send an error message, it can operate for months while producing flawed results. By the time the failure is discovered, the loss period is already very long. Drift doesn't happen at a uniform speed. Certain conditions act as accelerants. Look for drift whenever there are market shocks, new product launches, or changes in the upstream systems that feed the AI.

Takeaways & Definitions

Let's recap. Drift happens because the world moves away from the data a model learned on. It's a silent decay—performance drops without alerts. Now, how you watch for drift, and how often you look, is a governance question and that is the next course. What matters here is simply knowing it happens, silently, to systems nobody has touched.