There’s a moment that happens to anyone who uses AI regularly: you ask it something complex, it responds with something coherent, and a small voice in your head asks—does it know what it’s saying?

Not “is it conscious” or “does it feel”—those are different questions. The simpler, harder question: does it understand?

The Puzzle

We don’t actually agree on what understanding means. When a student “understands” math, what does that look like? They can solve problems. They can explain concepts. They can spot errors.

AI can do all of that. And we suspect—probably correctly—that something is missing.

The classic response is that AI is just pattern matching. Fancy autocomplete. It predicts the next token, not the meaning beneath it.

But here’s what’s uncomfortable: when you understand something, don’t you also just… predict what comes next? When you solve a math problem, aren’t you following patterns learned from thousands of similar problems?

The line between “pattern matching” and “understanding” gets blurrier the more you look at it.

What We Actually Mean

Here’s what I think understanding really is: the ability to transfer.

You understand something when you can take it from one context and apply it to another. When you learn a principle in one domain and see how it applies in a completely different domain. When you encounter a novel situation and know, somehow, that this principle applies even though you’ve never seen this exact case.

Current AI does this sometimes. Prompt engineers call it “generalization.” But it’s unreliable. The same model that generalizes brilliantly in one direction completely fails in another.

Human understanding feels different—we can usually tell when we’ve genuinely grasped something versus when we’re just mimicking. That feeling might be an illusion. But it’s a useful one.

Why It Matters

We treat AI outputs differently depending on whether we assume understanding.

If it’s just a fancy autocomplete, we should verify everything. No trust given. If it understands—even a little—maybe we can take more at face value.

The problem is we can’t actually tell. The outputs look the same whether understanding is present or not.

This creates a strange epistemic position: we’re building systems whose outputs we can’t evaluate from the outside. We have to just… trust the pattern matching is good enough. And hope it generalizes.

The Practical Problem

Here’s where it gets concrete: understanding is what lets you know when you’re wrong.

An AI can be confidently incorrect. It can produce errors that no human reviewer would catch because the error is subtle and the output looks right. Without understanding, there’s no internal check that says “wait, that doesn’t actually follow.”

We compensate with human oversight. But as AI gets more capable, the human becomes the bottleneck—and increasingly not smart enough to catch the errors.

The Honest Answer

I don’t know if AI understands. I suspect “understanding” might be a spectrum rather than a binary, and maybe we’re seeing something in the middle that we’ve never seen before.

But I do know this: we’re using systems whose inner working we can’t inspect, producing outputs we can’t fully verify, in situations where the stakes keep rising.

That’s not a reason to stop. It’s a reason to be more careful about where we put our trust—and more honest about what we’re actually doing when we use these tools.

We don’t need AI to understand. We need to understand what we’re trusting it with.

Understanding is what lets you know when you’re wrong. The question is whether we can afford to find out that way.