At 69, it’s the perfect age to create!
Richard Sutton, a pioneer in reinforcement learning and the 2024 Turing Award recipient, has announced that he and his student Khurram Javed have left Keen Technologies, founded by John Carmack, to establish a new company called Oak Lab.
Richard Sutton is widely regarded as the founding figure of modern reinforcement learning:
He earned his undergraduate degree in psychology from Stanford and pursued his doctorate under Andrew Barto, a pioneer in reinforcement learning. Early in his career, he conducted research at GTE Labs and AT&T Labs.
He proposed the temporal difference algorithm and co-authored with his advisor Andrew Barto the globally recognized textbook "Reinforcement Learning: An Introduction";
Since 2003, Sutton has served as a full-time professor at the University of Alberta, where he founded the Reinforcement Learning and Artificial Intelligence Laboratory (RLAI).
Served as a Distinguished Research Scientist at DeepMind from 2017 to 2023, leading the establishment of the DeepMind Edmonton research team.
At the same time, over the past forty years, Sutton has cultivated a large number of top-tier talents in the AI industry.
Including AlphaGo’s lead designer David Silver, DeepMind Montreal head Doina Precup, game AI expert Michael Bowling, and co-founder Khurram Javed.
From the tweet, the reason Sutton is going solo is straightforward:
He believes current deep learning approaches are weak and inefficient, requiring not incremental improvements, but entirely new foundational ideas and a complete overhaul.
In other words, Sutton believes that the current AI approach struggles to advance further toward higher-level general intelligence.
And Oak Lab's ultimate goal is:
Build an agent with a trillion-parameter scale that can learn and plan in real time, with a total power consumption of only 20 watts.
20 watts, exactly matching the energy consumption level of the human brain.
While the entire AI industry is still stacking GPUs and expanding data centers, the father of reinforcement learning is preparing to redefine what intelligence means.
Break up with Card God
To understand why Sutton left, we first need to clarify where he left from.
John Carmack, founder of Keen Technologies, is the legendary programmer known as "Card God," creator of Doom and Quake, and former CTO of Oculus.
After leaving Meta in 2022, he fully dedicated himself to starting a company in AI, focusing on reinforcement learning.
In September 2023, Sutton chose to join Keen after Google DeepMind shut down the laboratory he had co-founded in Edmonton, Canada.
At the time, the two of them working together was also a dream team:
One is a legendary figure in foundational systems engineering, and the other is a pioneer of reinforcement learning theory; they plan to build a prototype system exhibiting "AGI lifelike characteristics" before 2030.
Now, after less than three years of partnership, Sutton has chosen to step away.
But he reserved the first sentence of his tweet for Carmack:
I can't praise John Carmack and Keen Technologies enough.
In other words, the departure isn't because Carmack is inadequate, but because there's a disagreement on how to reach the end goal.
In Sutton's view, the current entire development path of deep learning is not viable.
The model does not require endless iterative fine-tuning; the entire industry urgently needs a paradigm-shifting overhaul.
What does Oak Lab do?
Sutton's core bet in this venture can be summarized in one sentence:
Intelligence comes from continuous experience generated at runtime.
The current mainstream large models primarily operate by spending months and incurring substantial costs to complete offline pre-training using vast amounts of text data;
After training is complete, the model parameters are largely fixed, and the model is then deployed for use.
But even when interacting daily with hundreds of millions of users, the vast majority of these conversations cannot be transformed into entirely new capabilities.
The model can only reuse knowledge learned during training or temporarily retain information within the context of a conversation, but it cannot continuously update itself through ongoing perception of the external world, as humans and animals do.
But the agents Oak Lab aims to build are different.
It must perceive its surroundings, take appropriate actions, and adjust its behavior based on the outcomes;
Learning occurs simultaneously whenever new experiences are generated, without the need to wait long periods before initiating a new training cycle.
As Sutton himself said:
Every moment that AI operates should be a process of learning.
Currently, Oak Lab has unveiled its core research roadmap, centered around an architecture called OaK.
OaK stands for Options and Knowledge, meaning skills and knowledge.
The purpose of this architecture is to enable agents to discover abstract structures with temporal跨度 from their own experiences and transform them into skills that can be verified, planned, and repeatedly invoked.
For example, when a robot goes to the kitchen for water for the first time, the entire process involves a series of actions: identifying the room, avoiding obstacles, picking up a glass, and turning on the faucet.
Traditional AI treats all steps as a single decision task;
The OaK architecture enables agents to decompose practical actions into high-level skills such as "go to the kitchen," "pick up the cup," and "fill with water."
When encountering similar objectives in the future, the agent will directly retrieve existing skills and adapt the approach flexibly based on the current environment.
This method of condensing past experiences along the time dimension is called temporal abstraction, enabling AI to emulate humans by transforming a sequence of discrete actions into refined skills, which are then combined to accomplish more complex tasks.
In addition, the Oak architecture has a design goal that is fundamentally different from current deep learning:
The learning phase does not store historical data or replay past experiences.
Current deep reinforcement learning often stores large amounts of historical experience in a buffer and repeatedly samples from it for training.
Oak Lab envisions real-time learning with a batch size of 1, updating immediately after each new experience is obtained.
The team believes that if such algorithms are combined with event-driven neural networks, the computational requirements and energy consumption of the system could decrease by several orders of magnitude, making continuous, real-time learning truly feasible.
Thus, the long-term goal became: trillion-parameter scale, real-time learning, real-time planning, and 20-watt power consumption.
Of course, this is still just an idea for now.
Behind Oak Lab is another theoretical foundation: the large world hypothesis proposed by Sutton and Javed.
The core idea is that the real world is always more complex than AI. Even as models become increasingly powerful, the volume of data from the external environment will continue to surge accordingly.
Models trained on pre-organized data cannot keep up with real-world changes.
AI must learn to selectively remember useful information, appropriately forget outdated data, and continuously learn online to adapt to the real world.
From bitter lessons to entrepreneurship
People familiar with Sutton would not be surprised by the above perspective.
In 2019, he wrote the widely circulated essay in the AI field, "The Bitter Lesson."
The article reviews the development of chess, Go, speech recognition, and computer vision, concluding that:
General learning and search methods that scale with computational power will ultimately outperform systems reliant on human-crafted knowledge.
In the era of large models, this approach has been strongly validated: larger models, more data, and greater computing power have propelled AI capabilities to unprecedented heights.
But Sutton remains unsatisfied with today's mainstream deep learning.
In his view, the current system still heavily relies on data that is produced, filtered, and organized by humans.
The model learns from content that humans have previously written, filmed, or annotated.
True agents must generate new experiences through their own actions and use those experiences to pursue long-term goals.
This also explains why he moved from "bitter lessons" to the "era of experience."
In 2025, he collaborated with David Silver, a key figure behind AlphaGo, proposing that AI would gradually shift from relying on human-generated data to learning from experiences generated through agent-environment interactions.
Oak Lab is the entrepreneurial realization of this research framework.
Forty years ago, when Sutton wrote “temporal credit assignment in reinforcement learning” in his doctoral thesis, reinforcement learning was still an obscure field.
Forty years later, as the entire world chases the commercialization wave of large models, he still asks the same question—
Where does intelligence really come from?
One More Thing
The first stop after the old man started his business: Shanghai WAIC.
At the WAIC Thinkers Forum, Sutton will deliver a keynote titled "The First Principles of Reinforcement Learning: Cultivating Superintelligence from Experience."
Reference link: https://x.com/RichardSSutton/status/2076663628301058329
This article is from the WeChat public account "Quantum Bit," authored by: Focused on Frontier Technology