Apple expands Vision Pro SDK with spatial gestures and scene-aware APIs

Tech · 5 min read

The latest Vision Pro SDK update includes a suite of spatial gesture recognizers and scene-aware APIs that let apps understand object placement, occlusion, and user intent in mixed reality environments. Apple says the new APIs are optimized for low-latency on-device inference to keep interactions fluid.

Developers gain access to higher-level primitives for hand pose, multi-finger gestures, and eye-gaze anchoring, along with Scene Anchors that persist across sessions. The SDK also introduces automatic content occlusion helpers so virtual objects render naturally around real-world geometry.

Privacy remains central: all gesture and gaze processing can run entirely on-device, with explicit user consent flows and fine-grained controls exposed in the system preferences. Apple provided sample projects demonstrating collaborative spatial apps and shared anchors for multi-user sessions.