OpenAI Launches Whisper 2 Accessibility API for Real-Time Captioning and Contextual Descriptions

AI · 5 min read

OpenAI's Whisper 2 Accessibility API targets the long-standing need for reliable, scalable live captioning and on-demand contextual descriptions in products. The API introduces event-streaming endpoints optimized for sub-350ms latency in common network conditions and supports incremental transcripts plus confidence scores mapped to UI states.

Crucially for design systems, Whisper 2 returns structured accessibility hints alongside raw text: suggested button labels, inferred ARIA roles, and descriptive alt-text candidates for images and complex UI assemblies. These outputs are formatted to align with popular component libraries, enabling direct injection into Storybook stories or automated updates to design tokens that govern label content.

OpenAI also released an 'Accessibility Prompt Library' to help teams fine-tune description style (concise vs. verbose), bias control, and language conventions for screen readers. Early integrations include caption overlays for video players and assistive-description toggles in enterprise dashboards; privacy controls and on-prem options are offered for regulated industries.