Do LLMs really understand?

Is prediction really understanding?

You ask a question. The model gives a thoughtful, nuanced, accurate response. It seems to understand. But does it?

This question has occupied philosophers, cognitive scientists, and AI researchers. The answer isn't settled, and that's the honest starting point. Anyone who tells you definitively that LLMs do or don't understand is overstating what we know.

What we observe

LLMs exhibit behaviors we associate with understanding:

• Answer questions accurately
• Explain concepts in multiple ways
• Apply knowledge to new situations
• Reason through problems step by step
• Recognize and correct errors
• Discuss their own limitations

If a human did these things, we'd say they understand. Do we apply the same standard to machines?

The Chinese Room argument

Philosopher John Searle proposed this thought experiment in 1980:

Imagine you're in a room with a rulebook for responding to Chinese characters. You don't understand Chinese, but by following the rules, you produce responses that fluent Chinese speakers find meaningful. From outside, it looks like you understand Chinese. But you're just manipulating symbols.

Searle argued this is what computers do: symbol manipulation without understanding. LLMs are (very sophisticated) Chinese Rooms. They process symbols according to learned rules without genuine comprehension.

Behavioral vs phenomenal understanding

Two different questions:

Behavioral: Does the system behave as if it understands? Can it answer questions, solve problems, explain concepts? By this standard, LLMs clearly show understanding-like behavior.

Phenomenal: Is there "something it is like" to be the system? Does it have subjective experience, consciousness, qualia? This is much harder to assess.

Behavioral understanding is measurable. Phenomenal understanding may be fundamentally private. We can observe behavior; we can't observe experience.

Does it matter?

Practically, maybe not. If an LLM helps you code, answers your questions, and converses meaningfully, does it matter whether it "truly" understands?

Ethically, perhaps yes. If LLMs had genuine experience, their treatment would matter morally. We'd need to consider their welfare, not just their utility.

For capability prediction, probably yes. If understanding is emergent and linked to capability, then future systems might develop deeper understanding with greater scale. If it's impossible in principle, there are hard limits to what these systems can achieve.

Where the field stands

Most researchers hold uncertain positions:

• Functionalists: Understanding is what understanding does. LLMs that behave as if they understand, understand.
• Skeptics: LLMs are sophisticated pattern matchers. Behavior resembles understanding but isn't genuine.
• Emergentists: Understanding might emerge from sufficient scale and architecture. We're watching it happen.
• Mysterians: We don't understand understanding well enough to know what LLMs have or lack.

None of these positions is clearly refuted. The honest answer is: we don't know.

Living with uncertainty

We may never resolve this question to everyone's satisfaction. That's okay. We can:

• Use LLMs for their practical value without settled metaphysics
• Stay open to evidence that changes our view
• Treat the systems with appropriate caution given our uncertainty
• Continue research into interpretability and system understanding

The question "do LLMs understand?" matters. That we can't definitively answer it matters too. It's an invitation to humility about minds, machines, and what separates them.