Explore advanced App Intents features for Siri and Apple Intelligence
Advanced App Intents techniques for polishing Siri responses, donating UI interactions, indexing/searching entities, and adding onscreen context for Apple Intelligence.
TL;DR
- Customize Siri conversations with
ProvidesDialog, dialog requests, entityDisplayRepresentation, and SwiftUI snippet views while keeping voice-only devices in mind. - Donate real UI interactions with
IntentDonationManagerso Apple Intelligence can learn app preferences and understand ongoing activities started inside your app. - Make app content discoverable through
IndexedEntity, Core Spotlight indexing,IntentValueQuery, and the.system.searchInAppApp Schema. - Improve contextual Siri requests by annotating onscreen views, activities, notifications, Now Playing state, and AlarmKit alarms with persistent app entity identifiers.
Shape Siri conversations and responses
Siri handles natural language understanding and can generate default responses, but App Intents can refine the conversation so it matches an app's terminology and tone. Returning an empty result lets Siri respond automatically; adopting ProvidesDialog lets the intent return an IntentDialog with both a full spoken response and a shorter supporting string for UI.
The full dialog string should stand alone on voice-only devices such as AirPods. Clarifying questions can be asked from inside perform() by requesting a missing parameter value, but they should be used sparingly to avoid adding friction.
- Use default Siri responses when they are accurate enough.
- Use
ProvidesDialogfor app-specific vocabulary and tone. - Use dialog requests to collect missing values while an intent runs.
- Keep responses natural and correct across visual and voice-only contexts.
Custom dialog response
Adopts ProvidesDialog and returns an IntentDialog with a full spoken response plus a shorter supporting string.
@AppIntent(schema: .audio.addToPlaylist)
struct AddToPlaylistIntent {
func perform() async throws -> some IntentResult & ProvidesDialog {
// Add song to playlist.
return .result(
dialog: IntentDialog(
full: "Added \(song.title) to the \(playlist.title) mix tape.",
supporting: "Added"
)
)
}
}Ask a clarifying question inside an intent
Requests an optional parameter value only when additional information is needed to complete the action.
@AppIntent(schema: .clock.createTimer)
struct CreateTimerIntent {
var duration: Duration
var label: String?
var isSleepTimer: Bool
func perform() async throws -> some ReturnsValue<TimerEntity> {
label = try await $label.requestValue(
"You already have a timer running. What should we call this one?"
)
return .result(value: timerEntity)
}
}Add visual identity to Siri results
An entity's DisplayRepresentation is reused throughout the system: Siri responses, disambiguation, Spotlight, Shortcuts, and questions about app content. A good representation should include enough text and imagery for users to recognize the entity quickly.
For action-specific presentation, intents can return SwiftUI snippet views with ShowsSnippetView. Snippets are useful when a particular action benefits from a custom layout, but they should complement rather than replace clear spoken dialog.
- Start with meaningful entity titles, then add subtitles and images where helpful.
- Use custom snippets for specific actions that need app-styled UI.
- Test how snippets scale across Siri surfaces and platforms.
- Do not rely on visuals for information needed on voice-only devices.
Enhanced DisplayRepresentation
Provides a title, subtitle, and image so the entity is recognizable across Siri, Spotlight, and Shortcuts.
@AppEntity(schema: .audio.song)
struct SongEntity {
var displayRepresentation: DisplayRepresentation {
DisplayRepresentation(
title: "\(title)",
subtitle: "\(artistName)",
image: artworkImage
)
}
}Return a custom snippet view
Adds ShowsSnippetView so the intent can return both dialog and a custom SwiftUI snippet.
@AppIntent(schema: .audio.addToPlaylist)
struct AddToPlaylistIntent {
var audioEntity: AudioEntity
var playlist: PlaylistEntity
func perform() async throws -> some IntentResult & ProvidesDialog & ShowsSnippetView {
let view = PlaylistSnippetView(
playlist: updatedEntity,
tracks: updated.tracks
)
return .result(dialog: dialog, view: view)
}
}Donate UI interactions and model ownership
Siri and Shortcuts interactions are already visible to the system, but actions taken through an app's own UI are not. Interaction donations represent real UI actions as schema-conforming App Intents, giving Apple Intelligence context about user preferences and ongoing app activities.
Donations should be accurate and limited to real user behavior; excessive donation can be ignored. The session also covers confirmations: Siri may automatically confirm actions with meaningful side effects, especially when they affect public or shared content. OwnershipProvidingEntity lets an app tell Siri whether an entity is shared, public, unknown, or private enough to skip some confirmations.
- Donate UI actions, not actions already performed through Siri or Shortcuts.
- Populate intent parameters and results before donating.
- Use donations for preference learning and ongoing activities such as navigation sessions or stopwatches where supported by the domain.
- Adopt
OwnershipProvidingEntityonly for entities that can be shared or made public, and keep ownership state current.
Donate a UI interaction
Donates the schema-conforming send-message action only when the action originated from the app UI.
@ModelActor
actor ModelManager {
func sendMessage(_ /* ... */, donateIntent: Bool = false) async throws -> [Message.ID] {
if donateIntent {
let intent = SendMessageIntent()
intent.destination = .recipients(conversation.recipients.map(\.entity))
let result = messages.map(\.entity)
Task {
try await IntentDonationManager.shared.donate(
intent: intent,
result: .result(value: result)
)
}
}
}
}Declare entity ownership for confirmations
Supplies ownership metadata so Siri can decide when a side-effecting action should be confirmed.
@AppEntity(schema: .calendar.event)
struct EventEntity: OwnershipProvidingEntity {
var ownership: EntityOwnership {
attendees.isEmpty ? .unknown : .shared
}
}Make content discoverable with indexing and search
For local app content, adopt IndexedEntity and index entities with Core Spotlight using indexAppEntities. Indexing app entities populates Spotlight's semantic index, making content discoverable by Siri, Apple Intelligence, and the Spotlight UI. The index must be kept fresh as entities are added, updated, or removed.
For large, server-side, or frequently changing datasets, use IntentValueQuery instead of indexing everything ahead of time. The system passes structured search input, such as AudioSearch, and the app returns matching entities. For user-facing searches, adopt the .system.searchInApp schema so Siri can re-run a search inside the app's own search UI.
- Use
IndexedEntityfor local content suitable for ahead-of-time indexing. - Keep Spotlight entries current and support reindexing with
IndexedEntityQueryunless existing Core Spotlight reindexing already covers it. - Use
IntentValueQuerywhen search must be resolved dynamically. - Use
.system.searchInAppto open the app and show results in the app's own search experience.
Index entities with IndexedEntity
Indexes app entities into Spotlight so Siri and Apple Intelligence can find local content semantically.
struct EntityIndexingHelper {
func indexPlaylist(_ playlist: Playlist) async throws {
let entity = PlaylistEntity(playlist: playlist)
try await CSSearchableIndex(name: indexName)
.indexAppEntities([entity])
}
}Structured search with IntentValueQuery
Handles structured search criteria dynamically and returns app entities without pre-indexing every item.
struct AudioIntentValueQuery: IntentValueQuery {
func values(for input: AudioSearch) async throws -> [AudioEntity] {
switch input.criteria {
case .searchQuery(let query):
return try await searchResults(for: query)
case .unspecified:
return try await likedSongResults()
// Also handle URL criteria where appropriate.
}
}
}Re-run Siri search in app
Adopts the renamed system search schema so Siri can show search results in the app's own UI.
@AppIntent(schema: .system.searchInApp)
struct SearchAudioLibraryIntent {
var criteria: StringSearchCriteria
func perform() async throws -> some IntentResult {
navigation.searchText = criteria.term
navigation.selectedTab = .library
return .result()
}
}Provide onscreen awareness
Onscreen awareness connects visible UI to structured app entities so Siri can resolve references like "the third one" or "that conversation." Text visible in pixels is not enough for Siri to understand hidden attributes, entity identity, or possible actions.
Start with NSUserActivity for a screen dedicated to one primary entity, and use view entity annotations when an entity is one of several visible items. For large lists, collection annotations avoid per-row overhead and can preserve selection context for scrolled-off items. Custom canvas annotations cover non-standard views, and UIKit and AppKit have equivalent APIs.
Display-representation queries help Siri quickly resolve visible entities without fetching full model objects. Implement them to return only the requested display components when possible.
- Use
.userActivityfor the primary entity of a screen. - Use
.appEntityIdentifieron views representing individual entities. - Use
.appEntityIdentifier(forSelectionType:)on lists and collections. - Use display-representation queries to make on-screen resolution faster.
- Consult platform-specific APIs such as
AppEntityAnnotatable,UICollectionViewAppIntentsDataSource, andappEntityUIElementProviderfor UIKit/AppKit.
Onscreen awareness annotations
Shows the three common SwiftUI patterns: primary entity via NSUserActivity, individual view annotations, and collection annotations for list selection.
struct NowPlayingView: View {
@Environment(PlaybackController.self) private var playback
var body: some View {
VStack { /* Player UI */ }
.userActivity("cosmotunes.nowPlaying", isActive: playback.currentTrack) { activity in
activity.title = playback.currentTrack?.title
activity.appEntityIdentifier = EntityIdentifier(
for: SongEntity.self,
identifier: playback.currentTrack.id
)
}
}
}
struct AlbumView: View {
private var header: some View {
VStack(alignment: .leading, spacing: 6) { /* ... */ }
.appEntityIdentifier(
EntityIdentifier(for: AlbumEntity.self, identifier: session.id.uuidString)
)
}
}
struct PlaylistDetailView: View {
var body: some View {
List {
ForEach(playlist.tracks) { track in
PlaylistTrackRow(track: track)
}
}
.appEntityIdentifier(forSelectionType: GeneratedTrack.ID.self) { trackID in
EntityIdentifier(for: SongEntity.self, identifier: trackID)
}
}
}Component-based display representation query
Lets Siri fetch lightweight display data for visible entities instead of loading full content.
extension PlaylistQuery {
func displayRepresentations(
for identifiers: [PlaylistEntity.ID],
requestedComponents: DisplayRepresentation.Components = .text
) async throws -> [PlaylistEntity.ID: DisplayRepresentation] {
let entities = try await model.playlistEntities(for: identifiers)
var result: [PlaylistEntity.ID: DisplayRepresentation] = [:]
for entity in entities {
result[entity.id] = await entity.displayRepresentation(with: requestedComponents)
}
return result
}
}Annotate existing system integrations
The same entity-identifier pattern can add Siri context to integrations an app may already use: user notifications, Now Playing, and AlarmKit. These annotations let Siri understand content encountered outside the app UI, such as announced notifications, currently playing media, or firing alarms.
These APIs require persistent app entities. TransientAppEntity is not appropriate because transient objects do not have stable identifiers.
- Set
UNMutableNotificationContent.appEntityIdentifiersso Siri can act on notification-related content during announcements. - Set Now Playing
appEntityIdentifiersfrom most specific to least specific, such as song, artist, then playlist. - Pass an
appEntityIdentifierwhen creating anAlarmKitalarm or timer configuration. - Do not use transient entities for these annotations.
Entity annotations on system integrations
Adds persistent entity identifiers to notifications, Now Playing content, and AlarmKit configurations.
// User notifications
let content = UNMutableNotificationContent()
content.title = author.name
content.body = message.body
content.appEntityIdentifiers = [
EntityIdentifier(for: MessageEntity.self, identifier: message.id)
]
// Now Playing
var content = MusicContent(id: track.id.uuidString, songTitle: track.title)
content.appEntityIdentifiers = [
EntityIdentifier(for: SongEntity.self, identifier: track.id),
EntityIdentifier(for: ArtistEntity.self, identifier: track.session.artistName),
EntityIdentifier(for: PlaylistEntity.self, identifier: currentPlaylist.id)
]
// AlarmKit
let configuration = AlarmManager.AlarmConfiguration<CosmoTunesAlarmMetadata>.alarm(
schedule: schedule,
attributes: attributes,
appEntityIdentifier: EntityIdentifier(for: AlarmEntity.self, identifier: alarm.id),
stopIntent: DismissAlarmIntent(),
secondaryIntent: SnoozeAlarmIntent(),
sound: sound
)Resources
- App Intents Testing
- Donating your app's data and actions to the system
- Donations and discovery
- Making app entities available in Spotlight
- Making actions and content discoverable by Apple Intelligence
- Providing contextual cues to Apple Intelligence and Siri
- Apple Intelligence and Siri AI
Related Sessions
Bring an LLM provider to the Foundation Models framework
Implement a Foundation Models LanguageModelExecutor to package custom local or server-backed LLMs behind LanguageModelSession.
Code-along: Make your app available to Siri
Code along with a SwiftUI calendar app that adopts App Schemas, Spotlight donations, onscreen awareness, and App Intents so Siri can understand and manage events.