Xiaomi Auto Unveils Unified World Model Framework, Deeply Integrating 3D Reconstruction and Video Generation

Xiaomi Auto recently unveiled its new **Xiaomi Auto World Model** framework, which deeply integrates 3D reconstruction (WorldRec) and video generation (WorldGen) into a unified architecture characterized by “reconstruction anchoring geometry, generation filling imagination.” This approach has achieved state-of-the-art (SOTA) results on major benchmarks such as Waymo and nuScenes and has already been deployed in three core scenarios at Xiaomi Auto: synthetic data generation, simulation testing, and intelligent cockpit systems. Traditional world modeling techniques follow two separate paths: reconstruction and generation. Reconstruction recovers high-fidelity, highly consistent 3D scenes from multi-view observations but is limited to reproducing only what has already been observed. Generation, powered by diffusion models, predicts future frames and possesses the ability to “imagine” unseen viewpoints and unobserved scenarios—but lacks explicit 3D structure, often leading to drift and distortion over long sequences. The Xiaomi Auto World Model structurally fuses these two approaches: reconstruction provides stable 3D geometric anchors that constrain the generation process, while generation extends predictive boundaries to overcome reconstruction’s limitations, creating a closed-loop synergy between the two. This framework achieves synergistic gains across three key dimensions: 1. **High Stability**: Deterministic geometric constraints from reconstruction suppress error accumulation in long-sequence autoregressive generation. 2. **High Consistency**: A shared 4D scene representation ensures global consistency across frames and viewpoints. 3. **High Realism**: RGB images rendered from reconstructed geometry serve as a structural scaffold, ensuring generated content aligns with physical layouts and closely matches real sensor observations—thereby narrowing the domain gap between simulation and reality. The model is now operational in three business-critical applications: - **Synthetic Data Generation**: Over 100,000 high-quality video clips have been delivered to train perception models, significantly enhancing the vehicle’s ability to recognize hazardous scenarios. - **Simulation Testing**: A closed-loop simulation environment has been built to replay real-world accidents and enable targeted system optimization. - **Intelligent Cockpit**: Dynamic first-person driving tutorial videos are generated in real time to guide users through complex traffic situations. This feature is already live in the ADAS Driving Academy’s real-scenario simulation module across all Xiaomi vehicle models.