NVIDIA open-sources SoL-Pi harness to reduce AI token costs by 64%

After OpenAI, Huang also made a move!

Finally, NVIDIA has officially open-sourced its own Harness—SoL-Pi.

It uses Pi as the foundation and rebuilds a harness efficiency enhancement layer to enable AI to self-iterate and optimize.

In this rigorous automated pipeline, AI has also become a researcher.

They observe how the Agent operates, identify which steps are unnecessarily consuming tokens, propose solutions, fix bugs, and then submit the solutions for testing.

The AI's entire search began with 152 candidate directions and ultimately narrowed down to four killer architecture mechanisms.

Actual test results, astonishing—

Token consumption reduced by up to 64%, and API call costs dropped by 50%-54%.

GitHub link: https://github.com/NVlabs/SoL-Pi

In professional research scenarios, you can directly save between $8.75 and $13.50 per hour. NVIDIA has open-sourced the switch to save money with AI.

Setup is extremely simple—just one line of code: 'pip install git+https://github.com/NVlabs/SoL-Pi' to install it on your existing Pi.

Before letting the AI recursively improve itself, could we first ask it to itemize the bill for its self-improvement?

Harness, the complete execution framework used by models during operation, is currently the best solution.

The model handles reasoning, while Harness organizes tools, context, execution feedback, and task workflows to enable the model to read files, modify code, run tests, and take further actions based on the results.

The same model, placed in different harnesses, can vary significantly in efficiency.

The current issue is that the tasks assigned to programming agents are becoming increasingly long.

From adding a few lines of code, to fixing issues across repositories, to working independently for extended periods, a single task can last several hours.

At this point, small, often overlooked token wastes will continue to accumulate.

After revising the file, the next step should be running tests, yet the model still performs another inference round. A large file has already been read, but subsequent requests continue to repeatedly carry it.

Amid thousands of lines of logs, only a few may actually impact decision-making, yet expensive models must read from the beginning.

Recursive Self-Improvement (RSI) also cannot avoid this cost. Each time the AI attempts to improve the system, it consumes tokens. Even failed proposals are charged.

The emergence of SoL-Pi is designed to solve exactly this issue.

The final results show that SoL-Pi uses 45% to 49% fewer tokens compared to the base Pi, reduces costs by approximately one-third, and retains around 94% of the average score.

Compared to the original harness for Codex and Claude Code, it uses 35% to 64% fewer tokens and has a marked cost reduction of 50% to 54%.

How did SoL-Pi achieve such impressive performance? Let’s take a detailed look at its four core mechanisms.

NVIDIA's pipeline ultimately retained four mechanisms, each carefully designed to avoid the core workflow and specifically target repetitive tasks.

Layer 1: Action Fusion—Edit and validate in a single call

Action Fusion directly targets the unnecessary model decision phase between two tool calls.

In the basic Pi execution flow, the most common sequence is modifying a file, receiving the result, and then invoking a command to test or build.

Since the subsequent commands are already very clear, there is significant room to compress the back-and-forth of this intermediate model round.

Action fusion retains a single edit and its subsequent commands within a local sequence, eliminating the need for intermediate model decisions.

Action Fusion encapsulates editing and subsequent commands into a single local execution sequence.

Harness makes the modifications at the underlying level, executes the command, and returns the merged result all at once. The test runs as usual, and the result is retrieved normally, but the intermediate model request is successfully skipped.

This combines two separate requests into a single, efficient closed-loop operation.

Layer 2: Online Context Compression with Dynamic Cost Allocation by Subtask

Online Context Compact addresses the core pain point of "when to trigger compression."

The longer the context, the more likely historical data becomes a burden on computational power and capital.

However, compression comes at a cost: rewriting the context may disrupt existing KV-cache reuse, requiring the processing cost to be paid again.

Therefore, compressing too early may not save money, and past strategies often involved delaying compression as much as possible.

Online context compression treats completed subtasks as potential compression points and waits until subsequent requests can offset the rewrite cost before executing.

SoL-Pi has restructured its timing assessment by breaking down large tasks into subtasks and re-evaluating after each step is completed.

The core logic lies in precise calculation: the system will only execute the compression operation if the "estimated future cost savings" can cover the "cost of this rewrite."

Layer 3: Output archiving and indexing, with large files recalled on demand

ObservationPack, specifically designed to address the issue of "repeated billing" caused by lengthy tool outputs.

A large file or an extremely long result, if retained in the context after the initial read, will be carried along unchanged with every subsequent request, continuously consuming cache resources.

ObservationPack replaces repetitive full outputs with stable handles while preserving the original content for pagination-based retrieval.

SoL-Pi's approach is to archive the full content directly to the local disk, leaving only a stable short handle and a brief excerpt in the context.

It’s like filing away the full report after reading it and keeping only the index and key summaries on hand. When the model needs to review details, it can simply retrieve the relevant pages using the index.

Layer 4: Initial screening logs using small models, with the introduction of a rigorous evidence verification mechanism

The core of the Evidence-Preserving Reducer is to delegate the first pass through long logs to a less expensive model.

Log files can be tens of thousands of words long, but often only a few lines of error messages truly impact the next steps in decision-making. Having cutting-edge large models read through the entire text each time is extremely costly; however, delegating summarization directly to smaller models risks introducing hallucinations.

The evidence retention reducer only accepts compact diagnostic receipts when their evidence can be verified against archived logs.

SoL-Pi has inserted a strict safeguard in the middle—evidence verification. After the auxiliary model reviews the logs, it must output a concise diagnostic acknowledgment.

The system will compare this receipt item by item with the archived original logs. Only exact matches will be approved and passed on to the frontier large model.

This ensures that the main model receives only verified information, cutting off error propagation at the source.

152 ideas compete, leaving only 4.

Why these four mechanisms? This brings us to NVIDIA’s true ambition: RSI (Recursive Self-Improvement).

Since AI can modify code, it should also be able to transform the systems that produce AI.

But RSI is too costly—each trial requires a token fee. So NVIDIA shifted its focus: instead of making AI smarter right away, first make it more cost-efficient.

Compared to directly chasing points, token efficiency is harder to cheat. Gaming task scores is easy to overfit—just write rules for specific questions—but optimizations like removing duplicate contexts, compressing tool outputs, and merging invalid decisions can be transferred across different tasks and models.

Thus, a pipeline of "Agent researching Agent" was initiated:

The team first set up 535 verifiable environments, then had AI propose 152 optimization directions, followed by three rounds of screening—

First, estimate returns using historical data and eliminate those with no potential; then let the AI modify its own code and run experiments, while the Reviewer Agent critiques the results.

Final freeze scheme: Validate on a completely isolated held-out task; performance must not degrade, efficiency must improve.

152 ideas, only 4 survived—on average, about 40 are needed to produce one.

Thus, SoL-Pi’s core methodology became clear: have AI generate numerous hypotheses, then automatically conduct experiments, eliminate, and validate them to identify the few truly effective mechanisms.

Behind SoL-Pi is the Efficient AI team led by Professor Han Song, an associate professor at MIT and director of research at NVIDIA.

For them, the current token savings from SoL-Pi are merely a阶段性 footnote.

What they truly care about is how much further this "AI researching AI" flywheel can spin.

On the official project homepage, the team has also planted two highly ambitious long-term hints:

One is "closed-loop pretraining."

The current 535 environments are still manually constructed; in the future, agents will autonomously collect tasks, set up environments, validate, and update their harnesses across the entire web.

Once computational power and environmental diversity increase, harness is likely to develop its own scaling law.

The other is "efficiency for efficiency."

Use the streamlined harness to run larger-scale, lower-cost automated research cycles, thereby identifying more efficient mechanisms and creating a compounding effect of cost reduction.

In this feedback loop, humans are only responsible for providing an initial directional input; once the cycle begins, there is zero intervention throughout the entire process from research to validation. Once the AI has successfully implemented the mechanism, humans return to understand what the AI discovered and refactor the machine-generated rough code into an industry-standard version.

Researchers are still debating the scaling laws of models; however, some have already begun moving forward with Harness’s scaling law.

And the one racing down the track is AI itself.

Reference materials:

https://nvlabs.github.io/SoL-Pi/

https://github.com/NVlabs/SoL-Pi

https://x.com/MaxForAI/status/2098050525279478059

This article is from the WeChat public account "New Intelligence Yuan," authored by ASI Revelation, edited by Peach and Moses.