Google Breaks RSI with AI 'Dreaming' Strategy

Google may have already cracked RSI!

Just now, Google, in collaboration with Google DeepMind, the University of Maryland, and the University of Virginia, published a groundbreaking paper.

This time, they presented a completely different RSI path: Can AI evolve itself by dreaming?

This time, Google had the agent store every step it took as a tree, then "dream" within that tree.

Practice better exploration strategies in your dreams, then return to reality and keep going.

Paper URL: https://arxiv.org/abs/2609.14858

The effect of Dream-RSI is truly impressive:

For the algorithm optimization task, mainstream evolutionary search systems require 51,200 generations, while Dream-RSI achieves the same level with only 317 calls.

Solved the circle packing problem, matching Google's own AlphaEvolveV2's top record.

GPU kernel optimization reduces the number of generations by 2.43 times while maintaining the same performance.

The underlying weights remain unchanged and can be applied to both Gemini 3.1 Pro and 3.7 Flash.

Additionally, there is another more interesting conclusion: giving AI guidance can actually lead to worse results than giving no guidance at all.

As soon as the paper was posted on arXiv, X exploded. User Mark Kretschmann exclaimed: “Google may have just cracked recursive self-improvement!”

The first author of the paper, Tong Zheng, is a second-year Ph.D. student at the University of Maryland who joined Google this summer as a student researcher. He is supported by a team of 17 co-authors, including DeepMind researchers and distinguished professors.

To achieve recursive self-improvement, agents must learn to dream—and history is the world in which they dream.

Tong Zheng left this statement on Hugging Face, which encapsulates the core of the entire paper.

Dream-RSI addresses the most critical bottleneck in the entire evolutionary search field.

Systems like AlphaEvolve have already demonstrated that letting coding agents run evolutionary searches can uncover algorithms humans haven't found. However, they have a critical flaw: their exploration strategies are entirely hardcoded by humans.

Which branch should receive more computational power, which branch should be pruned early, and when to enable parallel processing—all these decisions are pre-defined by engineers. When the search space becomes large, fixed strategies waste computational resources on dead ends and fail to learn from past mistakes.

What if we let the strategy evolve on its own? Theoretically, yes—but the issue lies in the evaluation cost.

Evaluating a piece of candidate code can be done with just one run to determine its quality. But evaluating an exploration strategy requires letting it guide the agent through hundreds or even thousands of full episodes, then assessing the final results to score the strategy. Each time you modify the strategy, you must rerun the entire discovery process and wait days for a score—no one can afford that.

The solution for Dream-RSI is to save the execution process and, when evaluating new strategies, replay the saved process instead of rerunning it.

Specifically, it separates “doing the work” from “deciding how to do it” into two layers.

At the bottom level is the 'Discovery Agent,' responsible for execution, powered by Gemini 3.1 Pro or 3.7 Flash. It takes over a workspace, reads the context, generates a new candidate version of the code, and submits it for scoring by the evaluator.

The top layer is a lightweight orchestration layer containing an "exploration strategy" (a piece of Python code) that determines which node to proceed from in each round, how many parallel attempts to launch, and when to cut losses.

The underlying model remains unchanged; only this strategy code is subject to self-evolution.

The result of each real exploration is stored in a "Discovery Tree." The root node represents the initial workspace, and each child node records the complete context of a trial, including a filesystem snapshot, generated artifacts, evaluation diagnostics, and scores.

This tree is the simulator.

The strategy only needs to decide one thing: which nodes to continue from and how many to open at once. The outcomes of following each node are already recorded in the tree.

So when you need to iterate on a strategy, there’s no need to run code in a live environment—just retrace the historical path on the grown tree.

To try taking “two more steps down this path,” read the corresponding records in the tree; to try “abandoning that path earlier,” read fewer nodes.

The entire evaluation process does not invoke the model even once, does not run a single line of code, and relies solely on historical data.

The paper refers to this mechanism as "dreaming." Like a chess player reviewing past games—without needing to set up the board and play again—just flipping through the game records is enough.

The closed loop of Dream-RSI

Based on this insight, Google developed the Dream-RSI framework. Its self-improvement cycle consists primarily of three stages:

1. Real Exploration: The Latest Exploration Strategies

Guide the Agent and execute it in a real environment (e.g., invoking an LLM, compiling and running code), and record all paths, decisions, and execution feedback generated during exploration as a new structured exploration tree.

Add to historical database

Mid.

2. Build the simulator: Use the updated historical database

Convert into a reusable "replay simulator pool".

3. Dream-based Strategy Enhancement: A specialized LLM strategy development agent begins wildly "dreaming" within this simulator. It continuously modifies and rewrites the code of exploration strategies, generating thousands of candidate strategies and rapidly replaying them on the replay simulator.

The dream score consists of three components: first, the optimal score found in this round of dreaming; second, the cost deducted for the number of attempts made; and third, a parallel reward that encourages strategies to run multiple attempts in a single round.

A high-scoring strategy means it’s fast, efficient, effective, and cost-effective.

The entity responsible for modifying the strategy is another "strategy development agent."

It monitors three things: the path and score the current strategy has taken in the simulation, feedback from previous iterations, and the entire discovery tree. Then, after reviewing, it directly rewrites the strategy code.

Several versions are generated in each round, and each version is sent back to the dream for scoring; the version with the highest score becomes the official strategy for the next round.

This means the lower bound of the new strategy is "standing still," while the upper bound is "infinite evolution."

The RSI feedback loop is now complete.

Why will this strategy never get worse?

During the offline improvement phase, the strategy development agent undergoes multiple rounds of code iteration. Since the current strategy

Already included in the candidate pool, a new policy is only adopted if its overall replay score on the simulator—balancing exploration quality, computational cost, and parallel efficiency—exceeds that of the current policy.

will prevail and be deployed.

Therefore, mathematically, the new strategy deployed by Dream-RSI will never perform worse than the previous generation. As the new strategy is redeployed to explore the real world, it brings back a deeper and broader exploration tree, enabling the simulator to evolve alongside it.

The agent and its perceived "world" grow together, like interlocking gears.

After improving the strategy, deploy it to discover new algorithms. With more historical data, the dreams become more accurate—and as the dreams become more accurate, the strategy continues to evolve. If a branch doesn’t exist in the tree, it cannot be reached in the dream; thus, every real-world exploration expands the territory for the next dream.

The first version of the strategy was simple—it would just launch multiple independent workspaces and let them run in isolation. By later rounds, the strategy learned to be more efficient.

During the ConvDiv kernel experiment, when performance was on the rise, the strategy proactively reduced the number of attempts per round from 110 to 50; once performance plateaued, it maximized exploration efforts, causing the score to rise again.

No one taught it to do this—it figured it out on its own from its historical experience.

The paper was empirically tested in three areas.

1. Algorithm Engineering: Lasso Regularization Path Solver

The first area is algorithmic engineering, with the task being the Lasso regularization path. This is a long-standing challenge in high-dimensional statistics, requiring the entire path to be computed as quickly as possible while ensuring numerical accuracy.

Running Dream-RSI with Gemini 3.1 Pro reduced the solver's average runtime from 3,587.1 ms to 2,931.0 ms using only 317 calls.

In comparison, the same model using a fixed strategy requires 550 calls to reach this level. The baseline SimpleTES generates a total of 51,200 samples to achieve a similar result.

After switching to the more affordable Gemini 3.7 Flash, the model made 1,879 calls and successfully reduced the runtime to 2,350.6 milliseconds.

Because evolution occurs at this layer and is unrelated to the size of the underlying model, smaller models can also benefit.

The solver it discovers follows the baseline approach of choosing between LARS and coordinate descent based on problem dimensionality. In contrast, Dream-RSI’s solution integrates adaptability directly into the active set optimization.

Specifically, first filter features using strong rules, then apply Cauchy-Schwarz inequality for KKT pruning; only when the upper bound cannot determine a feature’s status is the exact gradient computed.

2. Mathematical Optimization: Three Challenging Mathematical Tasks

In the field of mathematical optimization, the gap has widened further.

In three highly computationally intensive discrete and geometric optimization problems—“Sum-Difference Problem,” “Circle Packing,” and “Autocorrelation Inequality”—Dream-RSI achieves or surpasses strong baseline performance within 1,000 generations.

Compared to SimpleTES, which requires a computational cost of 51,200 tokens, Dream-RSI saves over 50 times the computational budget while achieving equal or even better results.

3. GPU Operator Engineering: KernelBench Extreme Stress Testing

The third area is GPU kernel engineering, which is also Google's own core focus.

Here, the paper tested a total of four kernels.

VGG16 and LayerNorm achieve the same performance with significantly fewer resources; Dream-RSI requires only 1/2.43 and 1/1.79 of the generation steps needed by the others, respectively.

Compared to ConvDiv and ConvMax, with the same budget, the kernel performance optimized by Dream-RSI is 2.09 times and 1.44 times that of fixed exploration, respectively.

Finally, the research team distilled the previous historical trajectories into directional cues and embedded them into the prompt to guide the agent step by step.

The results were somewhat unexpected: under the same budget, the version with human guidance consistently performed worse than the one without.

In-depth analysis: Does directly telling AI to "summarize the experience" actually lead to worse results?

In the paper, there is a highly counterintuitive experiment.

Researchers tried the "experience summarization method," enabling large models to distill historical data into high-level guidelines to inform the next round of exploration.

However, the experimental results were surprising: the agent infused with high-level experience summaries performed significantly worse in exploration than the original baseline with no guidance at all!

The reason is that once you rigidly abstract experience into "high-level principles," they immediately become cognitive biases for AI, overly constraining its search space, stifling exploration diversity, and trapping AI in local optima.

In contrast, Dream-RSI's "dreaming" mechanism provides no "empty truths"; instead, it directly lets new strategies collide with specific historical trees, optimizing at the code level through micro-level parallel control, branch prioritization, and early stopping rules.

This approach preserves the openness and diversity of the exploration space, far surpassing the former.

First author Tong Zheng graduated with a bachelor’s degree from Northeastern University in 2021, during which time he conducted research at China’s most prestigious machine translation group (led by Professor Xiao Tong).

In 2024, he enrolled in a Ph.D. program at the University of Maryland under Professor Huang Heng. Dream-RSI was developed during his summer 2026 internship as a Student Researcher at Google.

From speculative decoding at ICLR 2025, where the model generates multiple drafts at once, to Parallel-R1 and Parallel-Probe at ICLR 2026 and ICML 2026—using reinforcement learning to teach the model to think in parallel—he has consistently pursued the same goal: never let the model stick to a single path.

The "Parallel Reward" in the Dream-RSI score follows the same principle. Previously, it involved parallelizing the model's reasoning; now, it involves parallelizing the agent's exploration.

Advisor Huang Heng, the inaugural Brendan Iribe Chair Professor at the University of Maryland, has authored approximately 300 top-tier conference papers and leads seven NSF projects and three NIH projects. Over the past two years, students in Huang Heng’s lab have immersed themselves in large model reasoning.

Among the 17 authors, the corresponding authors Xidong Wu and Zheng Zhang are joined by Wang-Cheng Kang, the author of SASRec, who is well known in the recommendation systems community at Google and DeepMind.

A summer intern listed as first author, with a team of senior researchers behind them—Google’s internal emphasis on this approach speaks for itself.

Has the secret passage to ASI been opened?

Previously, Google product lead Logan Kilpatrick revealed major news: Google is closely monitoring the cutting edge and has already spotted early signs of RSI!

He cited Gemini 3.5 to 3.8, updated roughly every three to four weeks, as evidence, and stated that Gemini 4 will be the largest and most ambitious pre-trained model to date, aiming to reclaim its leading position.

He emphasized that computing power and human resources should be directed toward writing code, conducting research, and advancing science, as these are most likely to make the next round of training faster and cheaper.

Why is Google so convinced that RSI is occurring?

Because the paradigm of AI self-improvement is quietly changing. Over the past decade, when people in the industry mentioned RSI, they always thought of "AI automatically modifying its own weights to train the next generation of gods."

But Dream-RSI demonstrates another possibility: the model’s weights don’t need to change first—it can become more powerful by self-rewriting its surrounding “Harness.”

When a model possesses this powerful capability, it uses the cheapest computing power to solve the most difficult problems.

As stated at the end of the paper:

An agent must learn to dream in order to recursively self-improve, and history is the entire world in which it dreams.

As ultra-large pre-trained models like Gemini 4 draw near, this RSI of “dreaming in virtual history” brings us one step closer to the impending intelligence explosion.

Reference materials:

https://dream-rsi.com/

This article is from the WeChat public account "New Intelligence Yuan," authored by Moses Aeneas.