Leading AI Researchers Explore AI-Driven AI Development

How to have AI develop the next generation of AI for you

Article author and source: WeChat official account "APPSO"

If a Chinese AI engineer has strong technical skills, an impressive resume, and graduated from a top-tier university like Peking University, they could potentially do anything—except not work.

Shortly after leaving Thinking Machines Lab, where she cited health and life as “the most important alignment work,” Weng Li has reportedly returned to OpenAI. According to a company spokesperson, she will lead a team accelerating internal research, contributing to OpenAI’s exploration of “recursive self-improvement”—enabling AI to participate in training and refining its own next-generation models.

Similarly, Safe Superintelligence, founded by Ilya Sutskever, another former key figure at OpenAI, has finally let a glimpse through its tightly sealed research. After gaining rare access to its internal research, NVIDIA decided to invest in SSI and expand its computing power by an order of magnitude using its new Vera Rubin system.

NVIDIA is confident that SSI has been pursuing a new research direction over the past two years, distinct from existing large model approaches. Ilya did not provide further details, only stating: "We have research worth scaling."

Curious onlookers became even more enthusiastic, and a reverse analysis of the public clues appeared on social media, suggesting that SSI was not seeking the next larger pretraining recipe, but rather a new method of iteration and evolution: enabling models to continuously generate stronger next-generation versions through feedback, experimentation, and ongoing learning—not remaining static after training concludes.

This assessment has not been confirmed by SSI, but it aligns closely with Ilya’s longstanding positions. He has previously stated multiple times that the era of scaling solely through more data and compute is nearing its limits, and that AI needs to relearn how to form deeper understanding with less experience—growing through interaction with its environment, much like humans do.

These highly accomplished top AI talents all seem to be looking in the same direction: how to get AI to develop the next generation of AI for them.

When mentioning recursive self-improvement, it’s easy to conjure a sci-fi image: an AI suddenly becomes aware of its own flaws, directly modifies its neural network weights, and wakes up ten times smarter than it was the day before.

The real starting point is much more humble. AI development is inherently a continuous cycle: researchers read papers and experiment logs, formulate a hypothesis, modify training code, queue up experiments, wait for results, compare metrics, identify failure causes, and then design the next round of experiments.

Today’s large models can already participate in many of these steps: they can read codebases, reproduce papers, write experimental scripts, invoke computing power to run tests, organize dozens of sets of results, analyze which parameters might be problematic, and then continue modifying the code. What once took a team days to complete in one cycle could, in the future, be accomplished by multiple agents simultaneously exploring dozens of paths, with only the most promising results submitted to humans for evaluation.

In a long article titled "Harness Engineering for Self-Improvement," published in early July, Weng Li almost foresaw the research direction she would soon lead. She argues that the most practical path to self-improvement in the near term is not for models to directly rewrite their own brains, but rather for AI to first learn to transform its own working environment: automatically analyzing failure logs, adjusting context management, tool usage, and workflows, then further refining this system based on new experimental results. More mature harnesses can drive automated research, while stronger models can, in turn, simplify the harnesses themselves—creating a virtuous cycle.

This creates an interesting parallel between Weng Li and Ilya’s two approaches: OpenAI appears to be gradually handing over its existing R&D processes to AI, allowing it to first become a functioning research engineer; SSI, on the other hand, may be seeking new learning paradigms at a more fundamental level, enabling the model itself to continuously iterate and evolve.

AI doesn't need to start with the ability to directly modify its own weights. By improving the training process, filtering data, designing evaluations, or optimizing the harness surrounding the model, it is already participating in creating its own successor.

The former moves inward from the engineering loop, while the latter moves outward from the model mechanism; both aim to shorten the gap between one generation of AI and the next.

Just as major players were eager to accelerate their bets on this path, another voice emerged within the industry.

This week, over 1,100 professionals from companies including OpenAI, Anthropic, Google, and Meta signed a statement calling for the early development of technologies and coordination tools to control the pace of automated AI research. Participants include senior researchers such as OpenAI Chief Scientist Jakub Pachocki, Anthropic co-founders Jack Clark and Jared Kaplan, and Ilya himself.

The petition does not call for a halt to AI research, nor does it deny the potential value of self-improvement. It expresses concern that once AI begins to significantly accelerate AI development, capability growth could outpace humanity’s ability to understand and control the system.

Consider this carefully: this seemingly contradictory back-and-forth comes from the same group of people—even the same individual. At the same time, they can hold two judgments: they believe automated research may be key to AI’s next leap forward, and they also believe that once this feedback loop is truly operational, the industry must have the capacity to slow down when necessary.

Once this cycle is established, it differs significantly from the past approach of releasing stronger models. In the past, the pace of progress was ultimately limited by human research teams—humans had to formulate questions, design experiments, and interpret results, and progress could stall due to budget constraints, manpower shortages, or organizational processes.

The goal of automated AI R&D is to gradually eliminate these bottlenecks. The stronger the model, the more it can help the team rapidly develop the next generation of models; in turn, each next-generation model becomes a better researcher, further shortening the time for the subsequent development cycle.

Thus, despite appearing to be contradictory forces, both share the same premise: automated AI research is transitioning from a distant concept into a concrete engineering goal.

Weng Jing's return to OpenAI indicates that recursive self-improvement has begun to attract dedicated organizations and leadership; NVIDIA's willingness to invest substantial funds and cutting-edge computational power after reviewing SSI's secret research suggests that the new learning pathway has at least reached a stage worthy of expanded validation.

The petitioners are calling for preparedness, not because they believe the initiative won’t succeed, but precisely because they are beginning to seriously consider what will happen if it does.

The real disagreement is not whether AI should assist humans in research—code generation, paper retrieval, and experiment automation have long been part of laboratories. Rather, it is about at which points humans should retain decision-making authority as AI evolves from a tool that enhances researchers’ efficiency into a system capable of independently proposing hypotheses, conducting experiments, evaluating results, and modifying subsequent workflows.

This raises a series of issues that need to be addressed: Who sets the optimization goals? Who determines whether a change that appears to improve benchmark scores has compromised security boundaries? When multiple labs are in competition, which company would voluntarily pause an effective iteration? If the model development cycle truly shortens from a year to months or even weeks, can external evaluations and internal security teams keep up?

AI companies are no longer just competing to create breakout models—they are competing to build a research system capable of consistently producing even stronger models.

On one side are top researchers like Ilya and Weng Li, seeking new ways to involve AI in their own evolution; on the other, thousands of researchers in the same field are calling for the preparation of a slowdown mechanism. Between these two positions, alliances shift constantly, and attitudes are not directly opposed—everyone faces the same fundamental challenge: when research tools begin accelerating research itself, who gets to decide when the next generation of AI should arrive?