A tool that provides a coherent answer to any question in any language sure sounds intelligent, but if appearances were everything, we would all be living in glorious Potemkin villages. Facades that look realistic and lively, but belie an underlying emptiness.1
LLMs2 have some Potemkin traits to them. They certainly sound pretty impressive; you just ask it a vague question in any form of human language and it’ll come back with an answer that reads like it came back from a human that understood you, cares about you, and wants to be the best little sidequest buddy in your adventure.
It understands societal trends, it’s across news and current events, it effortlessly recites topics from all over the world, and it can deftly discuss deep and dark human subject matter as well. It probably helps when nearly ⅔ of its content is sourced from the great minds of the Wikipedia and the other minds of Reddit.3 How did it get so good at coding? It did what the rest of us did and just glued shit together from Stack Overflow.4
But the name gives away the game here. Large Language Model. It’s a process whereby the input question gets turned into tokens, tokens are run through a statistical process to generate an output comprised of different tokens, and those tokens get turned into whatever language the question was in, and that’s your answer.5 Take input, put through randomised slot machine, get result that, statistically, was the best answer it could come up with. The ‘U’ in LLM means ‘Understanding’.
So when it seems like it’s got a rich comprehension of the topic that it’s talking about, it doesn’t. There is no strong grasp about anything tethered to reality, because it’s not tethered to reality. It’s been force-fed the entirety of Wikipedia so yes you can ask it anything (in a language with lots of training content, ideally) but it’s not going to be able to compensate for any weaknesses in what it’s been trained on. And it’s absolutely not a considered, reasoned view on the real world at all.
That’s not to say that LLMs are void of all worldly context, they’re not. It’s just that the only context that it knows is context that you provide it. If you keep a ChatGPT thread alive long enough, and keep conversing with it, it will take information it has gleaned from what you’ve said, and work that into its answers. But that’s a mechanical process; every time you ask a question from ChatGPT, Claude, or any chatbot at all, the entire conversation’s tokens get wrapped up, the new question or input tokens get tacked on, and the entire token bundle is put through the LLM inference process. How much context you can provide it is called the token window, and there’s a finite window size.6
And it cannot be overstated that this is not intelligence. It’s rote learning on steroids; critical thinking plays no part in LLM processing. And, crucially, it never will. An LLM model is a model that is created at a point in time, and it does not change after that. The only way to add to it is to create a whole new model and change your underlying system to use that instead. It doesn’t evolve to new concepts, because it cannot. It doesn’t grow when exposed to new ideas, because it cannot. It doesn’t seek to fill in its knowledge gaps with new information, because it cannot. It glows different colours when you shine different tokens through it, but that’s not an intelligent process.
Why does this matter? Because A.I. is a term that is a couple of things. Not only is it largely a marketing term, but it’s also not the final goal of the entire Artificial economy. That, rather, would be AGI.7 In a nutshell, AGI is the promise of an all-capable, general purpose form of Artificial Intelligence that can take on any task that a human could. And as long as you don’t pay attention to the fact that it’s an entirely hypothetical set of technologies, it’s already destined to uproot society as we know it.8
As much as I love reading AGI Sci-Fi, I’m pretty confident that fiction is all it will amount to. If we are to get to something that approximates AGI, it won’t be through the use of LLMs. The outputs of today’s LLMs are certainly impressive, and I have no doubt that they’ll find their way into invaluable everyday use, but for now it remains a firm Potemkintelligence.
-
Large Language Models. If you see something marketed as ‘A.I.’ without further definition, like an A.I. chatbot, it’s probably an LLM. They generate text that looks like a human wrote it. It’s a statistical model of large amounts of language. ↩︎
-
IBM. Artificial General Intelligence; something that I would have thought needs, at minimum, to have structured, independent thought and cognition. ↩︎
