MiniMax M3 open-sourced with native multimodal support and 1M context length

According to Beating Monitor, Chinese large model provider MiniMax has officially open-sourced the native multimodal Mixture of Experts (MoE) model, MiniMax M3, on Hugging Face. The MiniMax M3 has a total of 428 billion parameters, with 23 billion parameters activated per token, and natively supports up to 1 million tokens of context length. To reduce deployment memory overhead, the development team has also released an MXFP8 quantized version compatible with mainstream inference frameworks such as SGLang, vLLM, and Transformers. In terms of multimodal design, MiniMax M3 undergoes joint training of text, images, and video during pre-training to achieve native semantic fusion, rather than performing multimodal alignment in post-training. The model offers two inference modes: a “Thinking” mode for complex reasoning and tool orchestration, and a “Non-thinking” mode for low-latency dialogue and code generation. The underlying kernel enabling the million-token context length is MiniMax Sparse Attention (MSA), a lightweight attention kernel library also open-sourced simultaneously. Official data shows that MSA employs a Grouped Query Attention (GQA) with chunked retrieval mechanism. When optimized for NVIDIA Blackwell (SM100) architecture, MSA operators achieve over 9x faster prefilling and 15x faster decoding compared to traditional full attention mechanisms, while significantly reducing inference overhead.