Huawei Noah's Ark Lab has launched the RoboHarness system, which uses a Coding Agent to coordinate heterogeneous strategies such as VLA, WAM, RL, and TAMP, enabling zero-shot completion of long-horizon tasks. The system incorporates three core skills: understanding, memory, and evolution. The Memory Bridge module addresses state distribution mismatches during strategy transitions, achieving an 86% success rate in experiments. This work represents a shift in embodied intelligence from single-model approaches toward systematic orchestration.Author and source: Leiphone
Imagine a task: open the cabinet door, find the blocks inside, and build them into a bridge.
For humans, this is a simple task, with a few natural, seamless movements that even a kindergarten child can complete.
But for a robot, this task spans multiple entirely different capabilities: finding blocks requires visual understanding and following language instructions; opening a cabinet door requires complex interaction with the environment; and stacking blocks demands geometric planning, precise manipulation, and the ability to anticipate environmental changes.
More complicatedly, these capabilities belong to models from multiple distinct “schools”—VLA (Vision-Language-Action models), WAM (World-Action models), RL policies (Reinforcement Learning), and TAMP (Task and Motion Planning). Each has its own strengths but remains isolated, differing in training methods, input formats, and state spaces.
In today’s robotics landscape, capabilities are abundant, but collaboration is lacking.
Huawei's Noah's Ark Lab recently published a paper introducing RoboHarness, a system designed to address the issue of incompatible strategies. The paper's authors shared insights on this work with us (Leiphone).
Paper: https://arxiv.org/abs/2607.18060
Project homepage: https://www.robo-harness.com/
01 The Gap Between the Illusion and Reality of the Universal Model
This summer, several key works centered on Harness emerged in the field of robotics research, collectively pointing to a growing consensus: scaling up models alone is no longer sufficient—robots require an execution orchestration system.
Professor Yu Chao’s team at Tsinghua University released Harness VLA in July, introducing the Harness Layer—widely used in digital intelligence—to embodied intelligence. This allows a frozen VLA to focus on its strongest suit: contact-intensive manipulation, while a Harness layer learns when and how to invoke it, as well as how to reset and retry after VLA failure. In the LIBERO-Pro perturbation benchmark, this system increased success rates from 50% to 82.4%.
Conceptually, Harness VLA addresses how a model can maintain stable performance under out-of-distribution perturbations, focusing on the stability of a single policy.
Huawei's Noah's Ark Lab's RoboHarness faces a different layer of challenge: what do you do when a task exceeds the capability of any single model?
Both are called Harness and both use the Coding Agent as the orchestration layer, but they address challenges at different dimensions. The former makes a single specialist more reliable, while the latter enables a team of specialists, each with their own strengths, to collaborate effectively. As the complexity of robotic tasks continues to rise, the latter challenge is becoming increasingly impossible to ignore.
The root of this issue lies in the capabilities and limitations of various models.
The VLA model excels at understanding open-ended language instructions and generalizing to new environments, but its end-to-end generation of actions struggles to maintain consistency over long-horizon tasks. Reinforcement learning policies can develop stable closed-loop behaviors within specific training scenarios, but this stability heavily depends on the training distribution and may fail with even minor environmental changes. TAMP is strong in symbolic reasoning and geometric constraints, yet requires predefined action primitives and quickly struggles when faced with open-ended scenarios and ambiguous goals. WAM can assist long-horizon decision-making by predicting future states, but is prone to error accumulation as the prediction horizon increases.
Complex real-world tasks simultaneously demand semantic understanding, geometric planning, closed-loop control, long-horizon reasoning, and anticipation of environmental changes—capabilities that correspond to distinct training objectives, sometimes even conflicting with one another. A profound gap remains, yet to be truly bridged, between achieving breakthroughs in each individual capability and developing a single model that can consistently and stably integrate all of them in open-ended environments.
The core idea behind RoboHarness is this: rather than waiting for this gap to be bridged, why not enable the existing models—each with their own strengths—to work together effectively? The author believes that until one model can completely outperform all others and demonstrate absolute dominance, such orchestration architectures remain highly valuable.
This work did not emerge out of thin air. The authors tell us (LeiFengNet) that the prototype of RoboHarness was born from their experience participating last year in the BEHAVIOR Challenge, a global robotics competition. The competition tasks were long and extremely challenging; the team found that neither end-to-end training of VLA models nor behavior cloning approaches could adequately address the complexity of the tasks. This setback helped the team clearly identify the core issue at hand.
02 How the Scheduling Brain Decides Who Gets to Play
The core idea of RoboHarness is to encapsulate independently developed control systems—such as VLA, WAM, RL strategies, and TAMP—into agentic skills that can be uniformly orchestrated, with the Coding Agent handling high-level decision-making.
This design is based on one key premise: no modification or retraining of any underlying strategy. Each strategy retains its original implementation and does not need to share model architecture, action space, or training data. RoboHarness simply adds a layer of orchestration on top of them.
Choosing the right strategy isn't as easy as it seems.
For a coding agent, it is difficult to reliably determine who should be assigned this task based solely on raw image input, because this decision involves many quantitative questions that language models cannot answer through semantic understanding. It can comprehend instructions like “place the cup on the plate,” but it cannot calculate the similarity between the current scene and a policy’s training distribution from an RGB image, nor can it assess the reliability of the sensor data at this moment.
To address this issue, the system has been designed with three types of auxiliary skills specifically to extract the information needed for the Coding Agent to make decisions.
Understanding Skills is responsible for transforming raw inputs into quantifiable signals, such as the similarity between image embeddings and training data for each strategy, the stability of object pose within a time window, and whether current lighting and image quality meet thresholds. These metrics serve as the true basis for strategy routing. The author believes that the core function of this module is to evaluate, from multiple dimensions, whether each strategy is truly suitable for the current task.
Memory Skills retrieve historical execution experiences to inform strategy selection and use Memory Bridge to address state distribution mismatches between strategies—this is the system’s core design, detailed below.
Evolution Skills continuously updates strategy metadata and orchestration logic using online feedback, enabling the system to learn from each execution rather than making zero-based judgments for every new situation.
The information flows of the three skill categories interact with each other to jointly assist the Coding Agent in decision-making. In practice, this mechanism serves two primary functions: first, task decomposition, which breaks down long instructions into subtasks suited for different strategies; and second, dynamic switching, where the system promptly transitions to a more appropriate strategy as the current one approaches its capability limits, enabling complementary strengths across strategies.
The paper presents a series of ablation studies illustrating the individual contributions of these two layers: simply using the language model to decompose the task and then reassign it to pi0.5 already significantly increases success rate; further integrating multiple heterogeneous strategies leads to a substantial additional improvement. Getting the decomposition right is the first step, and assigning it accurately is the second—both are essential.
But there is a third issue, the most difficult one, which cannot be resolved by strategy selection alone.
The moment of the baton exchange determines the outcome of the relay race.
Two independently trained strategies typically inhabit their own data worlds. They have different input formats and distinct empirical distributions over robot states. The position where the TAMP operation stops may well fall precisely within a state space that the VLA has never encountered and cannot stably initiate.
This is a unique challenge inherent to heterogeneous strategy orchestration.
Therefore, the Memory Bridge, responsible for ensuring a stable transition between the two strategies, is listed by the author as the most critical module in RoboHarness. He admits that if a future general model could perfectly cover the entire state space, the Memory Bridge might become unnecessary; however, under current conditions, any model is prone to fall outside its capability distribution at the moment of transition.
Memory Bridge operates in three steps.
Retrieve: Based on the next subtask and current observations, identify the top-K similar trajectories from the multimodal memory bank where the target policy was previously successfully executed, using text and visual similarity. Extract robot states such as end-effector pose and joint angles.
Modeling: Assign supervision signals based on the relative progress of each state within the trajectory, and use online regression to continuously fit the range of states where the target policy "habitually takes over."
Bridging: Sample and score candidate states by balancing confidence in entering the target policy's comfort zone and motion cost, select the most suitable bridging target, and generate a transition trajectory; once the robot reaches this state, control is handed over to the next policy.
This mechanism relies entirely on online learning from historical execution data, enabling plug-and-play compatibility with any strategy without requiring modifications to the underlying implementation or joint training.
In the ablation study, removing the Memory Bridge resulted in only a modest decline in task progress, indicating that strategy selection remained largely correct; however, the overall success rate dropped sharply from 86.0% to 60.4%. Many tasks reached the final step but failed due to incompatible state transitions, thereby demonstrating the value of the Memory Bridge.
04 Two technical lineages, a quiet convergence
Broadening our perspective beyond this paper, we aim to describe a larger landscape of embodied technologies.
The author of RoboHarness observed that, against the backdrop of rapid advancements in VLA over the past three years, traditional TAMP and hierarchical planning had briefly faded from community attention due to their significantly weaker generalization capabilities in open-ended scenarios compared to end-to-end VLAs. However, with the recent rapid progress in agentic systems and coding agents, hierarchical architectures have regained new vitality: high-level agents can decompose tasks, invoke tools, and perform dynamic planning using composable skills, thereby substantially enhancing the generalization and adaptability of traditional hierarchical approaches. As a result, the academic community is once again paying attention and beginning to explore how to combine strong generalizing high-level reasoning with heterogeneous low-level policies.
The author of this article is affiliated with multiple universities and research institutions in eastern Canada. The author observes that robotics research in North America has long followed two distinct technical pathways with clear academic origins: on the U.S. West Coast, represented by Stanford, Berkeley, and others, there is a stronger emphasis on learning and scaling, focusing on end-to-end learning, data scale, and model generalization; whereas in the U.S. Northeast and Canada, represented by MIT, the University of Toronto, MILA, and others, there has been a longstanding emphasis on hierarchical architectures, symbolic reasoning, logical planning, and structured decision-making.
The authors of RoboHarness are situated on the extension of the latter spectrum.
The divergence in technical approaches has left a clear imprint on the embodied intelligence industry. In recent years, rapidly developing teams have gradually formed a technological spectrum ranging from foundation model scaling to agentic hierarchies: one camp emphasizes general-purpose embodied foundation models, VLA, and world models, continuously expanding generalization boundaries through unified architectures, larger datasets, and enhanced model capabilities; the other prioritizes reusable operational skills and hierarchical architectures, coordinating task planning, skill composition, and low-level control to accomplish complex tasks. Domestically, teams led by Baidu’s Zhiyuan AI align more closely with the former approach, while Suduo Technology and Magic Atom emphasize the latter. Abroad, Physical Intelligence’s Pi series has long represented the scaling pathway, whereas Gemini Robotics continues the hierarchical tradition of “high-level reasoning and planning + low-level skill execution.”
Interestingly, the divergence of this technological lineage is not a static opposition but rather an evolving complementarity. Over the past few months, these two technical pathways have begun to show clear convergence. For instance, Physical Intelligence has elevated controllability and skill composition to a more central role in pi0.7; meanwhile, teams like NVIDIA have consistently introduced robot agentic systems with more hierarchical structures, integrating foundational model understanding and reasoning capabilities with VLA. Overall, foundation model scaling and agentic hierarchy are gradually evolving from two relatively independent technical paths into complementary capabilities within a unified robotic system.
05 Take Orchestration as the Path Toward More General Intelligence
RoboHarness has clear limitations: it can only schedule existing strategies and cannot accomplish what no strategy can achieve. The reliability of Memory Bridge depends on sufficient historical execution data, and the early-stage performance evaluation of newly integrated strategies carries significant uncertainty.
The core argument of the paper is not to deny the necessity of a unified large model.
The authors emphasize that orchestrating long-horizon tasks is not merely a combination of existing capabilities, but also continuously generates cross-capability and cross-scenario execution data that individual models struggle to obtain independently. This data captures transitions between different strategies, seamless衔接, failure recovery, and collaboration processes—potentially becoming a vital source for training the next generation of general-purpose robotic models. Building on this insight, the RoboHarness team is also exploring how to reuse the data generated during hybrid strategy orchestration and execution to further train underlying policies, allowing heterogeneous orchestration experiences to further enhance the foundational capabilities of the models.
From seeking the strongest models to organizing the most suitable capabilities, the competition in embodied intelligence is quietly extending to system-level design. Heterogeneous strategy orchestration and unified large models are like two separate mountain ridges, both leading toward the same horizon: enabling robots with more general and reliable intelligence.