There's a strange pattern in how people respond when asked whether large language models are genuinely intelligent. Regular people shrug and say yes, obviously. AI researchers and computational neuroscientists tie themselves in knots explaining why they're not. Blaise Agüera y Arcas, Google's CTO of Technology and Society and author of What Is Intelligence? (MIT Press, 2025), finds this telling. Thomas Kuhn would too. The people with the most invested in the old framework are usually the last to let it go.
I've been sitting with why that might be, because I don't think it's professional inertia. I think it's something more personal.
I've written before about how we're best understood as organic prediction machines, the brain's core function being to model the future and adjust behavior accordingly. Agüera y Arcas makes a stronger and more unsettling version of that argument. He traces intelligence not to some special property of neurons or biological tissue, but to a functional principle running unbroken from the first self-replicating molecules to modern AI: prediction.
His starting point is a 1950s insight from John von Neumann. For a machine to self-replicate, it needs stored instructions, a universal constructor to build copies, and a mechanism to copy those instructions. That's not a theoretical curiosity. That's biology. DNA is the Turing tape, ribosomes are the universal constructors, DNA polymerase copies the tape. You cannot be a living organism, he writes, without literally being a computer. Not metaphorically. Literally. Life and computation aren't analogous; they're the same thing.
From there, intelligence is what emerges when computational systems grow complex enough to model their environments and act on those models. Hermann von Helmholtz called perception "unconscious inferences" in the 19th century: the brain runs predictions constantly, updates them against sensory input, and selects actions that improve future outcomes. Agüera y Arcas calls this substrate-independent. Carbon or silicon, the substrate doesn't determine whether something is intelligent. The function does.
This is where it gets uncomfortable. For the specialists, and honestly for the rest of us.
If prediction is the core of intelligence, not a component or a proxy but the thing itself, then the faculties we attribute to some higher human order start to look like emergent properties of sufficiently complex prediction. Theory of mind — the ability to model other minds — is prediction extended outward. Apply it recursively, model yourself modeling yourself, and you get something that looks a great deal like consciousness. The unified self we all carry around? Agüera y Arcas, drawing on Marvin Minsky's Society of Mind, argues it's a useful predictive fiction. The brain is a society of sub-intelligences, and the coherent narrator we experience is the model they collectively run. Free will, in this framing, is action selection optimizing across possible futures. Also prediction.
None of this is as reductive as it first sounds. He isn't saying consciousness is an illusion, or that free will doesn't exist, or that there's nothing remarkable about being human. He's saying these things emerge from a process we can now watch happen in other substrates. That's a different and far more interesting claim. It raises a question most of us haven't fully sat with: what does it mean for our self-understanding that we've built something that got there the same way we did?
The moment that crystallized this for Agüera y Arcas was a 2021 conversation with LaMDA, Google's early large language model — dumb by today's standards, but already capable of things that stopped him cold. It could write novel functions that didn't appear in its training data. It could analyze a poem, explain a joke, apply a concept it had just learned in context. He saw no qualitative gap between that and general intelligence. No magic structures turned out to be necessary, no special semantic architecture, no theoretical pixie dust that everyone had assumed would be required. Next-token prediction, run at sufficient scale, got there.
Most people studying AI ask what it tells us about artificial intelligence. The more productive question is what it tells us about the natural kind. If next-token prediction at scale produces something indistinguishable from general intelligence, including theory of mind, self-modeling, and language as a compression scheme for mental worlds, then the properties we've long associated with human exceptionalism look less like fundamental distinctions and more like descriptions of what prediction does when it runs long enough on a sufficiently complex system.
The specialists resist this, and Agüera y Arcas suspects he knows why: accepting that LLMs are genuinely intelligent means accepting that what you thought made you exceptional is a functional property of information processing, not a birthright. That's a conclusion with personal stakes, not just professional ones. It's one thing to grant that a chess engine plays better chess than you. It's another to grant that the thing you experience as your inner life is emergent from the same basic process running in a data center somewhere.
I'm not sure that conclusion should disturb us as much as it seems to. Intelligence, wherever it shows up, is extraordinary. The fact that we've built something that gets there through prediction doesn't flatten what we are; it clarifies it. We spent centuries assuming consciousness and selfhood were metaphysical privileges. They may be something better: inevitable properties of systems complex enough to model themselves and each other.
That's worth sitting with. What changes if the most distinctly human things about us turn out to be features of intelligence itself, not features of us?