Discover generated subtitles and subtitle styles

Watch on Apple Developer

TL;DR

Generated subtitles fill language and accessibility gaps

Generated subtitles are created live and locally on device during media playback. They help when content does not include a subtitle language the viewer needs, while preserving authored subtitles as the preferred source when available.

The session distinguishes authored subtitles from generated subtitles: authored subtitles are manually created as part of the media package alongside video and audio, while generated subtitles are additional tracks produced at playback time.

• Speech transcription generates subtitles from source audio using an on-device Speech-To-Text model.
• Language translation generates new subtitle languages from existing subtitles using an on-device translation model.
• Authored subtitles remain unchanged and are preferred over generated alternatives.

Availability and supported media

Apps do not need to implement anything to enable generated subtitles for supported playback scenarios. The system exposes them automatically during video playback when device, OS, language, and media conditions are supported.

• Supported playback scenarios include HTTP Live Streaming, including live streams such as TV channels.
• Video on demand content is supported, including movies, shows, travel videos, and live events such as sports.
• File-based content is supported, including app-bundled videos and downloaded media.
• Supported content types include professional content such as movies and series, and customer-created content such as iPhone camera captures and social media videos.
• Starting in iOS, macOS, tvOS, and visionOS 27, English subtitles can be generated from English audio; on iOS and macOS, multiple subtitle languages can also be generated from English subtitles.

Present subtitle selection UI

Even though generated subtitles are available automatically, apps should provide subtitle selection UI during playback so people can discover and choose authored or generated options.

• Use AVPlayerViewController on iOS for fully implemented player controls and subtitle selection.
• Use AVPlayerView on macOS for similar built-in functionality.
• Use AVLegibleMediaOptionsMenuController when adding subtitle selection to an existing player UI without adopting full player controls.
• Implement custom media selection controls when the UI needs to match the rest of the app.

Subtitle style preview

Subtitle style preview lets people change caption and subtitle styling during playback instead of leaving the app for Settings. The style menu can show built-in and custom caption styles and preview how each style will look over video.

AVPlayerViewController on iOS and AVPlayerView on macOS fully implement subtitle style preview. AVLegibleMediaOptionsMenuController can add this behavior to an existing player UI. For custom UIs, AVPlayerLayer can display a preview, and AVCaptionRenderer can provide a preview when the app handles rendering.

• Subtitle and caption styles are represented by Media Accessibility caption appearance profile IDs.
• When previewing a style, AVPlayerLayer hides existing subtitles so they do not interfere with the preview.
• Passing nil for preview text uses localized system text.
• The preview position is an offset from the default preview location, which helps avoid overlapping custom controls.
• After selection, stop the preview and set the chosen profile as the active system caption style.

Implement subtitle style preview with AVPlayerLayer

Fetch system caption style profile IDs, preview a selected style over the video, stop the preview when selection ends, and set the chosen style as the active system caption appearance.

import AVFoundation
import MediaAccessibility

func updateProfileList() {
    subtitleStyleProfileIDs = MACaptionAppearanceCopyProfileIDs() as? [String] ?? []
}

func showPreviewStyle(subtitleStyleProfileID: String) {
    playerLayer.setCaptionPreviewProfileID(
        subtitleStyleProfileID,
        position: .zero,
        text: nil
    )
}

func stopPreviewStyle() {
    playerLayer.stopShowingCaptionPreview()
}

func setSubtitleStyle(subtitleStyleProfileID: CFString) {
    MACaptionAppearanceSetActiveProfileID(subtitleStyleProfileID)
}

Demo behavior and user experience

The demo shows a video with English subtitles where generated Italian subtitles appear in the subtitle menu alongside authored options. Generated options are marked with a sparkle symbol and labeled as translated.

The style menu then previews styles during playback. Selecting a style temporarily replaces existing subtitles with a localized placeholder rendered in the candidate style; dismissing the menu applies the chosen style to the active subtitles.

• Generated subtitle choices should be discoverable in the same place as authored subtitle choices.
• Style preview helps users evaluate readability before committing to a caption style.
• Providing both subtitle selection and style preview improves accessibility for users who rely on captions for comprehension.

Resources

• What's new in HTTP Live Streaming

Unofficial, not associated with Apple. By