Fabrication: the confident wrong answer

The System That Never Says 'I Don't Know'

Welcome to this lesson on fabrication. Ask a generative AI system a question it has no good answer to, and it'll very rarely say 'I don't know.' Instead, it'll give you an answer. A well-structured, confident, and professionally worded answer. This is the failure everyone's heard of and almost nobody's correctly understood, partly because the popular name for it is misleading.

The Exam-Smart Student

Think of a student who hasn't done the reading but is very good at taking exams. They've absorbed the shape of a right answer: the vocabulary, the structure, and the tone of authority. When they don't know the facts, they produce an essay with every feature of a correct answer except the actual correctness. Where the analogy breaks down is intent. The student knows they're bluffing. The AI system has no internal sense of the difference between truth and pattern, which is why it cannot warn you when it's making things up.

Defining Fabrication

First, a quick definition: Generative AI is a system that produces new text, images, or code rather than just classifying existing things. Fabrication—often called 'hallucination' by the industry—is plausible output that isn't grounded in fact. The system isn't 'lying' because it doesn't have a concept of truth; it's simply predicting the next most likely word or pixel based on a pattern.

Fluency vs. Accuracy

Here's the central point: fluency and accuracy are produced by different things. In an AI model, the 'fluency dial' and the 'accuracy dial' move independently. A system can be 100% fluent—sounding like a senior partner at a law firm—while being 0% accurate. Because the output sounds so professional, we're naturally inclined to trust it. But in the world of AI, confidence carries no information about correctness.

Recap: The Mechanics of Fabrication

Let's pause and recap. Fabrication is the result of a system that's built for fluency, not for fact-checking. It produces plausible answers that aren't grounded in reality, and it does so because it's simply following a pattern.

Where Fabrication Concentrates

Fabrication often worsens when the AI is asked for specifics. You should be most on guard when you see names, numbers, citations, dates, or quotations. These specifics are where fabrication concentrates, and they're also what make an answer feel most trustworthy.

The Reliability Reflex

Two AI-generated summaries land on your desk. One is hesitant and vaguely worded. The other is crisp, specific, and cites three sources. Which should you check more carefully? Exactly. This runs against our natural instinct. Specificity is where fabrication concentrates. Names, dates, and citations are exactly what get invented. The confident summary is the dangerous one. Actually, it's the other way around. The hesitant summary is annoying, but it's not the primary danger.

Case Study: Air Canada (2024)

In February 2024, an Air Canada website chatbot told a grieving passenger he could apply for a bereavement discount retroactively. This wasn't true. When the passenger claimed the difference, Air Canada argued in court that it wasn't responsible for what its own chatbot said. The tribunal member called this submission 'remarkable' and found the airline liable for negligent misrepresentation. The money was trivial, but the principle was not: your automated channel speaks for your business.

The Commercial Pattern

To identify where fabrication poses the highest risk, look for the commercial pattern: Fabrication becomes a loss the moment someone downstream acts on the output without checking it. As seen in the Air Canada case, an automated channel speaks for the business that runs it. You cannot distance the firm from the output of the tools it deploys.

Recap: Risks and Responsibility

Let's summarize. We've seen that fabrication is most dangerous when it's most specific. The risk isn't just the error itself, but the fact that someone acts on it. And legally, that action is the responsibility of the business, not the tool.

The Reflex

We've covered what fabrication is and why it matters commercially. There are ways to reduce fabrication, and every one of them belongs to the next course. For now, all you need is the reflex: fluent does not mean true, and confident does not mean checked.