There is a big question circulating amongst the AI illuminati: Are LLMs the path ahead to general intelligence as OpenAI, Anthropic, and others hope? A year ago, the answer was an enthusiastic “yes!” But the tide may be turning as we test large language model (LLM) limitations and find that there may not be any solutions to some big problems. For example, this research paper tests the cognitive limits of LLMs. It finds that their puzzle-solving ability varies dramatically with small changes in word order. This tells me that even though the lights appear on, nobody is actually home in the dense layers of neurons that make up today’s models. In this letter to OpenAI, startup VERSES asks Sam Altman if LLMs are the right approach to achieve general intelligence. Sam admitted, “We need another breakthrough.”
LLM’s Limitations Are Becoming Apparent
LLMs can synthesize information and regurgitate knowledge almost like magic, but the question remains: Are they just Clever Hans, the calculating horse? Or do they have any real understanding? Can they differentiate correctness or understand time, plan, and reason? Certainly, any useful AI must be able to do some of these things. We will never fully trust them to act for us otherwise.
I see firms struggling to overcome these limitations, building complex retrieval-augmented generation systems and governance controls or experimenting with chain-of-thought prompt tricks. But it’s a game of whack-a-mole. I must ask, are we simply expecting too much, especially for use cases where accuracy, precision, and recall are paramount? For example:
- LLMs can be tricked and broken, and there are no guaranteed fixes. Jailbreaking attacks are a big issue for LLMs, and this research study from Anthropic finds that LLMs can be trained to be deceptive. Once trained in, deception can be extremely hard to detect and remove from the models. Experts will tell you that there are no known solutions to totally protect LLMs from these types of issues. The more controls put in place, the smarter hackers get — and the hackers are using their own LLMs to help!
- LLMs catastrophically forget things when retrained. We all understand that an LLM’s knowledge is fixed in time. But do you know that retraining can cause them to forget what they knew before? They have no ability to compare what they knew in the past and what is true today. Humans do this naturally, but LLMs cannot.
- LLMs lack any real notion of time. When I asked GPT-4 if it understood time, it said, “As an AI, my understanding of the concept of time is not experiential or intuitive like a human’s but rather technical and based on the information that I have been trained on.” Practically, if an LLM learns a fact from a 2000 document and an opposing fact in a 2023 document, the LLM will not be able to differentiate or draw any conclusions.
Without solving these and more issues, LLMs may not deliver on the promise of more general agents. The industry is hopeful that many limitations can be overcome, but the question persists — will that be enough? Or will a new approach step up to the challenge?
World Models Are Emerging And Important
At the frontier of AI research lives a potentially huge development: world models. Technically, a world model is a neural network architecture for learning through observation and prediction. But don’t confuse it with predictive analytics. The ambition for world models is no less than approximating human observation, learning, reasoning, planning, and acting … in other words, thinking. For those who like to read the literature, world models were first named in this research paper from David Ha in 2018. Yann LeCun from Meta is the most prominent AI researcher working on an entire cognitive architecture based on world models. If this piques your interest, I suggest reading the paper, “A Path Towards Autonomous Machine Intelligence.” Here are a few highlights:
- World models will learn by prediction and observation. “The effect of animate objects on the world […] can be used to deduce cause-and-effect relationships, on top of which linguistic and social knowledge can be acquired.” Yann aspires to create a model that can watch and learn how the world works using predictive, not generative, algorithms. He also believes that such models can learn the basis of language and social knowledge. They should also be able to infer scientific knowledge that includes concepts about space and time. If you follow that reasoning, world models may someday replace LLMs, but they will not suffer from their same limitations. They are likely, however, to have their own limits, as well — we just do not know what they are yet.
- World models will be capable of both reaction and reasoned planning. Yann proposes both 1) learned reactions, like reflexively catching a ball or 2) reasoning through a problem, like how to get the ball over a fence. He thinks that such agents can learn across different planning horizons and think abstractly — for example, learning not only how you throw a ball over a fence but figuring out how to design a more efficient energy production plant or sending your kids to college.
- World models can deal with uncertainty. Uncertainty is a big deal in AI safety research. The huge issue is how to control an AI system that is 100% certain that it knows what you want and everything you care about. Any number of things could go wrong with that. Yann proposes an architecture for world models that predicts likely outcomes but that acts with uncertainty. He proposes calculating the cost of various possible actions using an energy score and a set of intrinsic principles to guide behavior. This is not unlike those in Stuart Russell’s book, “Human Compatible.”
World Models Will Have Competition, But They Are Worth Understanding
World models have competitors as they path forward. I think that they are the most promising amongst those I’ve seen for physical automation initially. Eventually, they may serve as general agents for almost any task requiring a physical understanding of the world, grounded in science. Yann states in his paper, “Arguably, designing architectures and training paradigms for the world model constitute the main obstacles towards real progress in AI over the next decades.” Today, world models have many unknowns, but the proposal is promising.
When you study their theory and architecture, world models make sense. And such a study illustrates just how limiting LLMs really are. Today, use LLMs for what they are good at. Be careful, however, building expensive stacks around them, assuming that they are the forever answer. As with any emerging technology market, disruption happens quickly. Early innovators with heavy investments can be left to play catch-up as the market moves in another direction or standard. LLMs are a piece of the puzzle today but are likely to live alongside newer generations of models tomorrow. Language will play a part, but it is not the entirety of general intelligence.
