The upcoming artificial intelligence developer MiniMax has teased its new **MiniMax M3 model**, which introduces a sparse attention mechanism to accelerate long-context processing. The company released a technical report detailing its previous M2 series of software models while previewing the architectural changes designed for the next generation.
According to the technical report, the **MiniMax M3 model** uses a custom sub-quadratic framework called MiniMax Sparse Attention (MSA). This architecture delivers a 15.6-fold speedup in decoding latency and a 9.7-fold speedup in prefilling latency at a context length of one million tokens.
Technical Evolution from M2 to M3
The previous M2 series relied on a Mixture-of-Experts layout with 229.9 billion total parameters, activating 9.8 billion parameters per token. During development, the engineering team tested various sub-quadratic scaling methods but found they reduced reasoning capabilities. Specifically, sliding window attention configurations dropped the model’s score from 90.0 to 72.0 on complex word extraction tasks.
Consequently, the developers maintained full attention mechanisms for the M2 series to preserve multi-hop reasoning. However, to make long-context deployment economically viable, the team designed the **MiniMax M3 model** to bypass these hardware bottlenecks without losing accuracy.
The MiniMax M3 model Architecture
Unlike other models that compress keys and values into a low-dimensional latent space, the new MSA framework operates on a standard Grouped Query Attention backbone. It utilizes block-level selection on real, uncompressed Key-Values to prevent precision loss. This method allows the system to process massive documents at high speeds while maintaining compatibility with prefix caching.
Agent Training and the Forge Infrastructure
To train these complex workflows, the company built a scalable reinforcement learning system named Forge. This infrastructure decouples execution into independent modules, including an agent side, a gateway server, and training engines. Notably, Forge uses prefix tree merging to eliminate redundant calculations, which produces up to a 40-fold training speedup.
This training setup previously enabled the development of the M2.7 checkpoint, which operates as an independent machine learning engineer. The system successfully managed between 30% and 50% of its own development workflow. It achieved a 66.6% medal rate on autonomous research benchmarks, matching closed-weight models in independent trials.
Future Outlook for Enterprise AI
The transition to the new sparse attention framework represents a shift toward highly efficient enterprise deployment. By reducing the computational footprint of long-context models, the company aims to expand the practical utility of autonomous agents in the digital economy.
