Huawei has had four papers accepted at IJCAI 2026, demonstrating the technological trend of AI shifting from "scale density" to "design density."

Author and source: Leiphone

AI moving into real-world scenarios, systematic engineering capability, is both a bottleneck and a breakthrough.

IJCAI-ECAI 2026 will be held in Bremen, Germany, from August 15 to 21, 2026. As a premier, comprehensive conference in the field of AI, IJCAI has long served as a critical benchmark for evaluating the cutting-edge research achievements of major companies over the past year: who has made real progress in which areas, and which technical approaches are converging into consensus—papers provide the most direct answers.

As the conference takes place, AI Science Review is systematically scanning the papers accepted at this year’s top conference to identify technological trends and industry directions.

A clear trend is that AI computing costs remain high, and the marginal returns from scaling parameters have long failed to keep pace with marginal costs. This economic reality is reshaping the logic of technological innovation—from “can we make it bigger?” to “can we make it more efficient?”

Four papers from Huawei accepted by IJCAI 2026 provide a technical roadmap for this shift: using more sophisticated architecture designs and training strategies to offset data scarcity bottlenecks and achieve higher intelligent output per unit of computing power.

This shift in technological orientation also reflects the deeper evolution of AI deployment: as technology moves out of the lab, competition is no longer about isolated breakthroughs in a single capability, but about the systemic coordination of accuracy, generalization, data, and computing power—each component can become a bottleneck, and also a breakthrough point.

These four papers precisely demonstrate Huawei's systematic engineering capabilities and technical choices across these four areas.

01 Breaking Scale Barriers: Architecture and Training Redefine the Capabilities of Hierarchical ViT

In the scaling race for vision foundation models, there has always been an invisible "ceiling." Standard ViTs have surged forward, continuously breaking records from 6B to 22B parameters. Yet, hierarchical ViTs have remained stuck below 2B parameters—not because they don’t want to scale, but because they can’t. Hierarchical models demand far more from data and training strategies than standard ViTs; simply applying standard ViT scaling approaches doesn’t work. This creates a structural contradiction: hierarchical ViTs are naturally suited for multi-scale representation and dense prediction tasks (object detection, segmentation, video understanding), but their inability to scale locks in their performance ceiling.

The Huawei team's paper, "Distilling and Scaling Hierarchical Vision Transformer to 30B Parameters," pushes the scale limit from 2B directly to 30B.

How is it done? The answer is to redesign both the model architecture and the training strategy—both are essential.

The core architectural design is the Efficient Hierarchical ViT architecture, which balances multi-scale feature extraction capability with computational efficiency.

Based on this architecture, the team trained dense models ranging from 200M to 5B parameters and introduced a sparse Mixture of Experts (SMoE) variant, pushing the total parameter count to 30B—the largest publicly known parameter scale in the hierarchical ViT domain to date.

For training, the team designed a two-stage process:

In the first phase, perform MAE self-supervised pretraining on ImageNet-21K to enable the model to learn the fundamental principles of visual representation;

In phase two, knowledge is distilled from multiple state-of-the-art general-purpose foundation models on a dataset of 27 million images to achieve a leap in capabilities.

The experimental results provide the strongest evidence. On the ImageNet-1K linear evaluation, the MoE model with 30B total parameters (EHV-5B-MoE) activates only 6.7B parameters during inference and achieves an accuracy of 89.0%, surpassing the larger EVA-CLIP-18B model under the standard ViT pathway. More importantly, its performance gains are not limited to image classification—it also excels in video analysis and dense prediction tasks. This demonstrates that EHV has learned not just classification features, but truly high-quality, generalizable visual representations.

The Huawei team demonstrated through this work that hierarchical ViTs can not only be scaled up but also achieve higher accuracy with fewer activation parameters on the inference side. Architecture design determines the model's upper bound in capability, while training strategies determine how much of that upper bound can be unlocked.

02 Input Pre-filtering: A Paradigm Shift in Learnable Frame Selectors

The ceiling for model backbones has been raised, but computational consumption is not limited to the model itself—the data fed into the model also consumes vast amounts of computing resources. When processing videos, video-large language models (Video-LLMs) primarily incur computational costs in frame encoding. To control costs, existing models almost universally employ uniform sampling—extracting frames at regular intervals.

However, key events in videos are never evenly distributed. A crucial action may last only one or two seconds and, if it falls between two sampling points, it is completely missed. In contrast, lengthy static scenes are repeatedly encoded, wasting valuable computational resources.

Another approach is query-driven retrieval—fetching relevant frames based on specific questions. This works well in video question-answering scenarios, but detailed video description is a “query-free” task, making this method unsuitable here.

Uniform sampling is too coarse, and query-driven methods aren't applicable. How to solve this? Huawei's AI Data team proposes a learnable frame selector, LFS (Learnable Frame Selector), to address this issue.

Paper URL: https://arxiv.org/pdf/2601.14594v1

Their core idea is straightforward: instead of relying on manual rules to select frames, let the model learn for itself what to look at.

This is a "goal-oriented" training paradigm: instead of learning which frames achieve higher scores at intermediate stages, LFS learns which frames enable the large model to generate better descriptions. How is this achieved? Three key design elements:

First, train a scoring network to assign an "event importance" score to each frame.

Second, segment-based frame selection instead of global ranking. Instead of simply selecting the top K frames with the highest scores, divide the entire video into multiple time segments and select the most important frame from each segment. This approach captures key events while ensuring an even distribution of frames across the timeline.

Third, and most crucially, the training approach: after LFS selects the frames, it feeds them into a frozen Video-LLM to generate descriptions, compares those generated descriptions with human-annotated ground truth descriptions to compute a loss, and then backpropagates to update the parameters of the frame selector. Throughout this process, the large language model remains completely unchanged; LFS functions like a plug-and-play external module.

In addition, the research team identified a significant gap between existing detailed video captioning benchmarks and human cognitive understanding. To address this, they developed a new benchmark, ICH-CC, using videos sourced exclusively from real-world commercial settings of Chinese intangible cultural heritage cooking (such as restaurant kitchens and cooking tutorials), with all captions and questions carefully written by humans to better reflect authentic use cases.

How did the experiment turn out?

LFS brings consistent and stable improvements across different models and benchmarks. The LFS-enhanced versions of all models outperform the baseline models on all test benchmarks, demonstrating that LFS not only achieves strong results on individual benchmarks but also exhibits considerable generalizability.

This also addresses the paper’s core proposition: rather than forcing the model to compute on uniformly sampled frames, perform intelligent selection at the input stage—choosing the right frames leads to improved performance in downstream tasks.

03 Task Adaptation: Representational Modular Intervention Replaces Full Model Fine-Tuning

The cross-task generalization capability of large language models has always been a problem that’s important in theory but difficult in practice. The current mainstream approach is per-token dynamic routing—each token must choose among multiple LoRA adapters during processing, deciding which path to take. While this mechanism is effective, it comes at a significant cost: high computational demand and persistent memory usage, making the model both slow and memory-intensive to run.

Another approach—Representation Fine-Tuning (ReFT)—does not modify model parameters but instead only edits the representations of prefix and suffix tokens to achieve single-task adaptation, offering significantly higher efficiency than LoRA. However, it has two shortcomings: first, the tokens themselves carry semantic ambiguity, so modifying only the beginning and end is not sufficiently precise; second, it lacks a “self-guided” mechanism, meaning the model does not know which layer or which representation to adjust when faced with unseen tasks.

The Huawei Cloud team, in collaboration with multiple universities, proposed the Representation-Aware Modularity (RaMod) framework, successfully extending the ReFT approach to cross-task generalization scenarios.

The core approach can be broken down into two parts:

The first part is dual-module representation and parameter fine-tuning. ReFT can only modify the beginning and end, while RaMod is much more precise—it selects a strategy-filtered subset from the hidden representations in intermediate layers to perform modular interventions. Where and how to modify are chosen strategically, enabling more accurate guidance of the model to solve unseen tasks.

The second part is the asynchronous scheduler. The biggest concern for cross-task generalization is insufficient VRAM, so RaMod designs an active scheduling mechanism: allocate VRAM only when interventions are needed, and actively release it when done. This minimizes storage overhead.

The effect is immediate. Experiments show that RaMod not only achieves better cross-task generalization but also significantly reduces costs: compared to original LLMs, this method reduces extra pre-filling time by 83%, lowers generation latency by 100%, and decreases GPU memory usage by 79%.

In other words, RaMod transforms task adaptation from “modifying the model” into “adjusting representations”—keeping the core model unchanged while making targeted edits to the hidden states at key layers. If this approach holds, it could rewrite the economics of large model deployment: a single base model, combined with several lightweight intervention modules, could serve vastly different task scenarios at low cost, without requiring separate training or loading of full model copies for each scenario.

However, before “rewriting,” this representation fine-tuning approach still has key questions to address, such as whether it can maintain accuracy while significantly reducing computational costs in complex reasoning and other challenging scenarios.

04 Data Breakthrough: Language Experts Specialize, Training Progresses Step by Step

The first three papers all address the question of how to use models more efficiently. But many tasks face an even more fundamental practical challenge: what if you don’t even have enough training data?

Code-switching (CS) speech translation tasks require translating speech that mixes multiple languages into a target language. This task is not only complex in semantic modeling, but the scarcity of CS data is also a major challenge.

Previous research primarily relied on two approaches: either letting the model independently "discover" semantic representations, resulting in unpredictable performance, or investing heavily in manual annotation, which was too costly and difficult to scale.

The team from Huawei's Translation Service Center, in collaboration with Xiamen University and the University of Macau, proposes a novel approach in this paper: instead of letting the model independently discover semantic representations across different languages, actively align them by assigning a dedicated “expert team” to each language to handle its own semantic modeling, followed by unified alignment.

Paper URL: https://arxiv.org/pdf/2511.10670

There are two key design elements in the specific implementation:

The first is the MoE (Mixture of Experts) speech projector. Traditional projectors treat all languages the same, resulting in suboptimal performance. The MoE version assigns a dedicated set of “experts” to each language, with each expert group responsible solely for modeling fine-grained speech features of its assigned language. The routing mechanism automatically directs speech features to the corresponding language’s expert group.

To ensure routing stays on track, the paper introduces two auxiliary losses: a language-specific loss to ensure each expert truly learns the features of its corresponding language, and an intra-group load balancing loss to prevent all tokens from being routed to the same expert.

The second is a multi-stage training paradigm. When data is insufficient, use strategies to compensate. The entire training process consists of four steps: First, pre-train the projector for each language separately using automatic speech recognition (ASR) data to establish a foundation for speech-text alignment; then assemble the projectors from all languages into an MoE structure and jointly optimize it with language-specific loss and load-balancing loss; next, gradually transition from ASR data to monolingual speech translation (ST) data using a transition loss to ensure smooth adaptation; finally, adapt from ST data to code-switched speech translation data.

The brilliance of this progressive design lies in not forcing the model to directly tackle scarce CS data, but instead first building a solid foundation using abundant ASR and ST data, then gradually shifting toward the target task.

The experiment compared multiple strong baseline models, including Whisper, SeamlessM4T, and LLaST. On the four test sets of Fisher and NTUML2021, the proposed method outperformed all other models, achieving the highest BLEU score of 39.52 and the highest COMET score of 81.33 among them.

The message conveyed by this work is clear: in cross-lingual scenarios with scarce data, sophisticated architectural design and progressive training strategies may be more effective than simply accumulating more data.

It should be noted that this approach is an effective "sharp tool" in scenarios with limited language support and controllable data quality, but its scalability in a general-purpose multilingual translation system requiring coverage of dozens of languages still needs further validation.

05 Conclusion: Trade design density for computational cost

The 30B hierarchical ViT restructured the model backbone, enabling 6.7 billion activated parameters to surpass a 18-billion-parameter counterpart on ImageNet;

LFS performs a subtraction at the input, blocking a large amount of meaningless frame encoding overhead before computation;

RaMod compresses full fine-tuning into定点 editing of the representation layer, reducing both memory usage and latency across tasks;

Code-switching speech translation uses MoE specialization and progressive training, proving in the most data-scarce domain that architectural intelligence can sometimes compensate for data scarcity.

Four papers, four directions, all pointing to the same core: Huawei is replacing “scale density” with “design density.” The four papers come from foundational research, AI data, cloud services, and the translation service center, demonstrating that prioritizing efficiency is no longer just a preference of a single team—this logic has been embedded into technical decisions across every stage.

If the AI race of the past two years was like a “mining contest,” 2026 is now revealing a turning point: the era of “smelting technology” has begun. Sparse activation, intelligent filtering, modular adaptation, and progressive training—these approaches do not rely on more chips or computing power, but on a deeper understanding of the problems themselves.

Whether large corporations or startups, the era of parameter arms races is long over; in the second half of AI, true victory lies in understanding real-world implementation challenges and the engineering capability to systematically solve them.