WWDC.ai

Explore advanced App Intents features for Siri and Apple Intelligence

Advanced App Intents techniques for polishing Siri responses, donating UI interactions, indexing/searching entities, and adding onscreen context for Apple Intelligence.

Watch on Apple Developer

TL;DR

  • Customize Siri conversations with ProvidesDialog, dialog requests, entity DisplayRepresentation, and SwiftUI snippet views while keeping voice-only devices in mind.
  • Donate real UI interactions with IntentDonationManager so Apple Intelligence can learn app preferences and understand ongoing activities started inside your app.
  • Make app content discoverable through IndexedEntity, Core Spotlight indexing, IntentValueQuery, and the .system.searchInApp App Schema.
  • Improve contextual Siri requests by annotating onscreen views, activities, notifications, Now Playing state, and AlarmKit alarms with persistent app entity identifiers.

Shape Siri conversations and responses

Siri handles natural language understanding and can generate default responses, but App Intents can refine the conversation so it matches an app's terminology and tone. Returning an empty result lets Siri respond automatically; adopting ProvidesDialog lets the intent return an IntentDialog with both a full spoken response and a shorter supporting string for UI.

The full dialog string should stand alone on voice-only devices such as AirPods. Clarifying questions can be asked from inside perform() by requesting a missing parameter value, but they should be used sparingly to avoid adding friction.

  • Use default Siri responses when they are accurate enough.
  • Use ProvidesDialog for app-specific vocabulary and tone.
  • Use dialog requests to collect missing values while an intent runs.
  • Keep responses natural and correct across visual and voice-only contexts.

Custom dialog response

Adopts ProvidesDialog and returns an IntentDialog with a full spoken response plus a shorter supporting string.

@AppIntent(schema: .audio.addToPlaylist)
struct AddToPlaylistIntent {
    func perform() async throws -> some IntentResult & ProvidesDialog {
        // Add song to playlist.
        return .result(
            dialog: IntentDialog(
                full: "Added \(song.title) to the \(playlist.title) mix tape.",
                supporting: "Added"
            )
        )
    }
}

Ask a clarifying question inside an intent

Requests an optional parameter value only when additional information is needed to complete the action.

@AppIntent(schema: .clock.createTimer)
struct CreateTimerIntent {
    var duration: Duration
    var label: String?
    var isSleepTimer: Bool

    func perform() async throws -> some ReturnsValue<TimerEntity> {
        label = try await $label.requestValue(
            "You already have a timer running. What should we call this one?"
        )
        return .result(value: timerEntity)
    }
}

Add visual identity to Siri results

An entity's DisplayRepresentation is reused throughout the system: Siri responses, disambiguation, Spotlight, Shortcuts, and questions about app content. A good representation should include enough text and imagery for users to recognize the entity quickly.

For action-specific presentation, intents can return SwiftUI snippet views with ShowsSnippetView. Snippets are useful when a particular action benefits from a custom layout, but they should complement rather than replace clear spoken dialog.

  • Start with meaningful entity titles, then add subtitles and images where helpful.
  • Use custom snippets for specific actions that need app-styled UI.
  • Test how snippets scale across Siri surfaces and platforms.
  • Do not rely on visuals for information needed on voice-only devices.

Enhanced DisplayRepresentation

Provides a title, subtitle, and image so the entity is recognizable across Siri, Spotlight, and Shortcuts.

@AppEntity(schema: .audio.song)
struct SongEntity {
    var displayRepresentation: DisplayRepresentation {
        DisplayRepresentation(
            title: "\(title)",
            subtitle: "\(artistName)",
            image: artworkImage
        )
    }
}

Return a custom snippet view

Adds ShowsSnippetView so the intent can return both dialog and a custom SwiftUI snippet.

@AppIntent(schema: .audio.addToPlaylist)
struct AddToPlaylistIntent {
    var audioEntity: AudioEntity
    var playlist: PlaylistEntity

    func perform() async throws -> some IntentResult & ProvidesDialog & ShowsSnippetView {
        let view = PlaylistSnippetView(
            playlist: updatedEntity,
            tracks: updated.tracks
        )
        return .result(dialog: dialog, view: view)
    }
}

Siri and Shortcuts interactions are already visible to the system, but actions taken through an app's own UI are not. Interaction donations represent real UI actions as schema-conforming App Intents, giving Apple Intelligence context about user preferences and ongoing app activities.

Donations should be accurate and limited to real user behavior; excessive donation can be ignored. The session also covers confirmations: Siri may automatically confirm actions with meaningful side effects, especially when they affect public or shared content. OwnershipProvidingEntity lets an app tell Siri whether an entity is shared, public, unknown, or private enough to skip some confirmations.

  • Donate UI actions, not actions already performed through Siri or Shortcuts.
  • Populate intent parameters and results before donating.
  • Use donations for preference learning and ongoing activities such as navigation sessions or stopwatches where supported by the domain.
  • Adopt OwnershipProvidingEntity only for entities that can be shared or made public, and keep ownership state current.

Donates the schema-conforming send-message action only when the action originated from the app UI.

@ModelActor
actor ModelManager {
    func sendMessage(_ /* ... */, donateIntent: Bool = false) async throws -> [Message.ID] {
        if donateIntent {
            let intent = SendMessageIntent()
            intent.destination = .recipients(conversation.recipients.map(\.entity))
            let result = messages.map(\.entity)

            Task {
                try await IntentDonationManager.shared.donate(
                    intent: intent,
                    result: .result(value: result)
                )
            }
        }
    }
}

Declare entity ownership for confirmations

Supplies ownership metadata so Siri can decide when a side-effecting action should be confirmed.

@AppEntity(schema: .calendar.event)
struct EventEntity: OwnershipProvidingEntity {
    var ownership: EntityOwnership {
        attendees.isEmpty ? .unknown : .shared
    }
}

For local app content, adopt IndexedEntity and index entities with Core Spotlight using indexAppEntities. Indexing app entities populates Spotlight's semantic index, making content discoverable by Siri, Apple Intelligence, and the Spotlight UI. The index must be kept fresh as entities are added, updated, or removed.

For large, server-side, or frequently changing datasets, use IntentValueQuery instead of indexing everything ahead of time. The system passes structured search input, such as AudioSearch, and the app returns matching entities. For user-facing searches, adopt the .system.searchInApp schema so Siri can re-run a search inside the app's own search UI.

  • Use IndexedEntity for local content suitable for ahead-of-time indexing.
  • Keep Spotlight entries current and support reindexing with IndexedEntityQuery unless existing Core Spotlight reindexing already covers it.
  • Use IntentValueQuery when search must be resolved dynamically.
  • Use .system.searchInApp to open the app and show results in the app's own search experience.

Index entities with IndexedEntity

Indexes app entities into Spotlight so Siri and Apple Intelligence can find local content semantically.

struct EntityIndexingHelper {
    func indexPlaylist(_ playlist: Playlist) async throws {
        let entity = PlaylistEntity(playlist: playlist)
        try await CSSearchableIndex(name: indexName)
            .indexAppEntities([entity])
    }
}

Structured search with IntentValueQuery

Handles structured search criteria dynamically and returns app entities without pre-indexing every item.

struct AudioIntentValueQuery: IntentValueQuery {
    func values(for input: AudioSearch) async throws -> [AudioEntity] {
        switch input.criteria {
        case .searchQuery(let query):
            return try await searchResults(for: query)
        case .unspecified:
            return try await likedSongResults()
        // Also handle URL criteria where appropriate.
        }
    }
}

Re-run Siri search in app

Adopts the renamed system search schema so Siri can show search results in the app's own UI.

@AppIntent(schema: .system.searchInApp)
struct SearchAudioLibraryIntent {
    var criteria: StringSearchCriteria

    func perform() async throws -> some IntentResult {
        navigation.searchText = criteria.term
        navigation.selectedTab = .library
        return .result()
    }
}

Provide onscreen awareness

Onscreen awareness connects visible UI to structured app entities so Siri can resolve references like "the third one" or "that conversation." Text visible in pixels is not enough for Siri to understand hidden attributes, entity identity, or possible actions.

Start with NSUserActivity for a screen dedicated to one primary entity, and use view entity annotations when an entity is one of several visible items. For large lists, collection annotations avoid per-row overhead and can preserve selection context for scrolled-off items. Custom canvas annotations cover non-standard views, and UIKit and AppKit have equivalent APIs.

Display-representation queries help Siri quickly resolve visible entities without fetching full model objects. Implement them to return only the requested display components when possible.

  • Use .userActivity for the primary entity of a screen.
  • Use .appEntityIdentifier on views representing individual entities.
  • Use .appEntityIdentifier(forSelectionType:) on lists and collections.
  • Use display-representation queries to make on-screen resolution faster.
  • Consult platform-specific APIs such as AppEntityAnnotatable, UICollectionViewAppIntentsDataSource, and appEntityUIElementProvider for UIKit/AppKit.

Onscreen awareness annotations

Shows the three common SwiftUI patterns: primary entity via NSUserActivity, individual view annotations, and collection annotations for list selection.

struct NowPlayingView: View {
    @Environment(PlaybackController.self) private var playback

    var body: some View {
        VStack { /* Player UI */ }
            .userActivity("cosmotunes.nowPlaying", isActive: playback.currentTrack) { activity in
                activity.title = playback.currentTrack?.title
                activity.appEntityIdentifier = EntityIdentifier(
                    for: SongEntity.self,
                    identifier: playback.currentTrack.id
                )
            }
    }
}

struct AlbumView: View {
    private var header: some View {
        VStack(alignment: .leading, spacing: 6) { /* ... */ }
            .appEntityIdentifier(
                EntityIdentifier(for: AlbumEntity.self, identifier: session.id.uuidString)
            )
    }
}

struct PlaylistDetailView: View {
    var body: some View {
        List {
            ForEach(playlist.tracks) { track in
                PlaylistTrackRow(track: track)
            }
        }
        .appEntityIdentifier(forSelectionType: GeneratedTrack.ID.self) { trackID in
            EntityIdentifier(for: SongEntity.self, identifier: trackID)
        }
    }
}

Component-based display representation query

Lets Siri fetch lightweight display data for visible entities instead of loading full content.

extension PlaylistQuery {
    func displayRepresentations(
        for identifiers: [PlaylistEntity.ID],
        requestedComponents: DisplayRepresentation.Components = .text
    ) async throws -> [PlaylistEntity.ID: DisplayRepresentation] {
        let entities = try await model.playlistEntities(for: identifiers)
        var result: [PlaylistEntity.ID: DisplayRepresentation] = [:]
        for entity in entities {
            result[entity.id] = await entity.displayRepresentation(with: requestedComponents)
        }
        return result
    }
}

Annotate existing system integrations

The same entity-identifier pattern can add Siri context to integrations an app may already use: user notifications, Now Playing, and AlarmKit. These annotations let Siri understand content encountered outside the app UI, such as announced notifications, currently playing media, or firing alarms.

These APIs require persistent app entities. TransientAppEntity is not appropriate because transient objects do not have stable identifiers.

  • Set UNMutableNotificationContent.appEntityIdentifiers so Siri can act on notification-related content during announcements.
  • Set Now Playing appEntityIdentifiers from most specific to least specific, such as song, artist, then playlist.
  • Pass an appEntityIdentifier when creating an AlarmKit alarm or timer configuration.
  • Do not use transient entities for these annotations.

Entity annotations on system integrations

Adds persistent entity identifiers to notifications, Now Playing content, and AlarmKit configurations.

// User notifications
let content = UNMutableNotificationContent()
content.title = author.name
content.body = message.body
content.appEntityIdentifiers = [
    EntityIdentifier(for: MessageEntity.self, identifier: message.id)
]

// Now Playing
var content = MusicContent(id: track.id.uuidString, songTitle: track.title)
content.appEntityIdentifiers = [
    EntityIdentifier(for: SongEntity.self, identifier: track.id),
    EntityIdentifier(for: ArtistEntity.self, identifier: track.session.artistName),
    EntityIdentifier(for: PlaylistEntity.self, identifier: currentPlaylist.id)
]

// AlarmKit
let configuration = AlarmManager.AlarmConfiguration<CosmoTunesAlarmMetadata>.alarm(
    schedule: schedule,
    attributes: attributes,
    appEntityIdentifier: EntityIdentifier(for: AlarmEntity.self, identifier: alarm.id),
    stopIntent: DismissAlarmIntent(),
    secondaryIntent: SnoozeAlarmIntent(),
    sound: sound
)

Resources

Unofficial, not associated with Apple. BySuperwall

On this page

Ask AI