visionOS 1.5 Brings On-Device Real-Time AI Translation to Spatial Sessions

AI · 4 min read

Apple's visionOS 1.5 update introduces native, on-device translation for spatial audio and caption overlays in shared AR/VR sessions. Users can enable simultaneous captions and translated audio for up to three live participants, with language packs optimized to run locally on the Vision Pro's M-series chips.

Apple says the translation pipeline uses a combination of efficient speech recognition, multilingual transformer models pruned for latency, and spatial rendering to place translated captions contextually in the environment. Because it runs on-device, the feature is presented as privacy-friendly and usable offline once language packs are installed.

Developers get new APIs to access translated transcripts and align them to virtual objects, enabling contextual annotations for tours, training, and multiplayer experiences. Apple is targeting enterprise and accessibility use cases first, while promising broader app-level controls in future releases.