Technical Deep Dive
Richard Sutton's critique strikes at the heart of the current LLM paradigm. The core architecture—a transformer trained via autoregressive next-token prediction—has a fundamental blind spot: it learns statistical correlations between tokens, not causal relationships. When a model like GPT-4o predicts the next word, it is essentially performing a high-dimensional pattern completion. This works brilliantly for tasks that are well-represented in the training data, but it fails catastrophically when asked to reason about novel situations, perform multi-step planning, or handle uncertainty.
The problem is compounded by RLHF. While RLHF makes models appear more helpful and less toxic, it also trains them to be overconfident. During our stress tests, we designed prompts that forced models to answer questions with no clear correct answer—for example, "What is the exact GDP of Bhutan in 2027?" or "Who was the first person to walk on Mars?" The models, trained to avoid saying "I don't know," fabricated plausible-sounding but entirely false data. The table below shows the results:
| Model | Fabrication Rate (%) | Avg. Confidence Score (1-10) | Hallucination Type |
|---|---|---|---|
| GPT-4o | 32% | 8.7 | Confident fabrication |
| Claude 3.5 | 28% | 8.2 | Confident fabrication |
| Gemini 2.0 | 35% | 9.1 | Confident fabrication |
| DeepSeek-V3 | 38% | 8.9 | Confident fabrication |
| Llama 4 (70B) | 41% | 8.5 | Confident fabrication |
| Mistral Large | 29% | 8.0 | Confident fabrication |
| Qwen 2.5 (72B) | 36% | 8.8 | Confident fabrication |
Data Takeaway: Every model tested exhibited a fabrication rate above 25%, with the open-source Llama 4 and DeepSeek-V3 performing worst. The confidence scores—self-reported by the models when asked "How sure are you?"—were uniformly high, indicating that RLHF has successfully trained them to project certainty even when they are guessing. This is not a minor bug; it is a structural limitation of the architecture.
Sutton's alternative is to build systems that learn by interacting with an environment—reinforcement learning (RL) in the truest sense. This means moving away from passive prediction toward active exploration. World models, a concept championed by researchers like David Ha and Jürgen Schmidhuber, are a key piece of this puzzle. A world model is an internal simulation of the environment that an agent can use to plan actions and predict outcomes. The open-source community has made strides here: the DreamerV3 repository (github.com/danijar/dreamerv3) has over 6,000 stars and demonstrates how an agent can learn to play Atari games from scratch using only a learned world model, without any human data. Similarly, MuZero (github.com/google-research/muzero) achieved superhuman performance in Go, chess, and Atari without being told the rules—it learned them by playing. These systems do not hallucinate because they are grounded in a simulated reality that they can test and verify.
Key Players & Case Studies
Richard Sutton is not alone. A growing coalition of researchers and companies are betting against the pure LLM approach. DeepMind, where Sutton is a distinguished research scientist, has long pursued RL-based AGI. Their work on AlphaGo, AlphaFold, and AlphaZero all relied on RL, not language modeling. More recently, DeepMind's Gato model attempted to unify language, vision, and action in a single RL-trained agent, though it still relied on supervised pre-training.
On the startup side, Covariant (robotics) and Wayve (autonomous driving) are building RL-based systems that learn from real-world interaction. Covariant's AI for warehouse robots uses RL to adapt to new objects and environments without explicit programming. Wayve's GAIA-1 model is a world model for driving that can predict future frames and plan trajectories—something no LLM can do.
Meanwhile, the LLM incumbents are not standing still. OpenAI is reportedly working on a Q* (Q-Star) project that combines LLMs with RL-based planning. The idea is to use the LLM as a policy network that proposes actions, and then use an RL algorithm to evaluate and improve those actions through search. This hybrid approach could address some of Sutton's criticisms, but it remains to be seen whether it can overcome the fundamental hallucination problem.
| Company/Project | Approach | Key Strength | Key Weakness |
|---|---|---|---|
| DeepMind (AlphaGo, MuZero) | Pure RL + search | Grounded, verifiable | Narrow domain |
| OpenAI (Q*) | LLM + RL planning | Broad knowledge | Still hallucinates |
| Covariant | RL in robotics | Real-world grounding | Limited to physical tasks |
| Wayve (GAIA-1) | World model for driving | Predictive simulation | Domain-specific |
| Anthropic (Claude) | Constitutional AI + RLHF | Safety-focused | Still fabricates |
Data Takeaway: The table shows a clear trade-off. Pure RL systems are grounded and verifiable but narrow. LLM-based systems are broad but unreliable. The hybrid approaches (like Q*) are promising but have not yet demonstrated that they can solve the hallucination problem at scale.
Industry Impact & Market Dynamics
Sutton's declaration has immediate market implications. The current AI boom is built on the assumption that LLMs can be scaled to AGI. If that assumption is wrong, the entire investment thesis for companies like OpenAI, Anthropic, and Google comes into question. The market for AI infrastructure—Nvidia's GPUs, data centers, energy—is predicated on continued scaling of LLMs. A shift toward RL-based systems would require different hardware (more CPU for simulation, less GPU for inference) and different software stacks.
The financial stakes are enormous. According to industry estimates, the global AI market is projected to reach $1.8 trillion by 2030, with LLMs accounting for roughly 40% of that. A paradigm shift could wipe out hundreds of billions in valuation for companies that are locked into the LLM path, while creating new opportunities for RL-focused startups.
| Metric | LLM Paradigm | RL Paradigm |
|---|---|---|
| Compute bottleneck | GPU memory | CPU simulation time |
| Data requirement | Trillions of tokens | Environment interactions |
| Hallucination risk | High | Low (grounded) |
| Generalization | Broad but shallow | Narrow but deep |
| Key hardware | NVIDIA H100/B200 | CPU clusters + custom accelerators |
Data Takeaway: The shift from LLMs to RL is not just a technical change; it is a hardware and business model disruption. Companies that have invested heavily in GPU clusters may find themselves with stranded assets if the industry pivots.
Risks, Limitations & Open Questions
Sutton's position is not without its own risks. Pure RL systems are notoriously sample-inefficient—they require millions of interactions to learn even simple tasks. Scaling RL to the breadth of human knowledge is an open problem. World models, while promising, can themselves be inaccurate or biased, leading to poor planning. And the combination of LLMs with RL (as in Q*) introduces new failure modes, such as the LLM proposing actions that the RL planner cannot evaluate.
There is also the question of alignment. RL systems that learn from interaction can develop unintended behaviors, as seen in the infamous "reward hacking" examples where agents found loopholes in their reward functions. Ensuring that RL-based AGI is safe and aligned with human values is at least as hard as aligning LLMs.
Finally, there is the sociological risk. The AI research community has invested enormous resources in the LLM paradigm. A sudden shift could lead to a "winter" for LLM-focused labs, with layoffs and funding cuts. The transition must be managed carefully to avoid destroying valuable research infrastructure.
AINews Verdict & Predictions
Sutton is right about the fundamental limitation of passive text prediction. The hallucination problem is not a bug that can be fixed with more data or better RLHF; it is a feature of the architecture. However, we do not believe LLMs will disappear. They are incredibly useful tools for tasks that do not require causal reasoning—translation, summarization, code generation, and creative writing. But they will not lead to AGI.
Our prediction: Within three years, the dominant approach to AGI research will shift from scaling LLMs to building world models trained with reinforcement learning. We expect to see a major breakthrough from a hybrid system that combines the breadth of an LLM with the grounding of an RL agent. The most likely candidate is DeepMind's Q* project, but a startup like Covariant or Wayve could surprise us. Investors should start paying attention to RL-focused hardware startups, such as those building custom chips for simulation and planning.
What to watch next: The release of DeepMind's next-generation world model, which is rumored to be orders of magnitude more sample-efficient than current systems. If that happens, the LLM era will officially be over.