LLM search using Core Spotlight

Watch on Apple Developer

TL;DR

Conversational search grounded in app content

The session shows how to turn Core Spotlight indexing into a retrieval-augmented experience for a Foundation Models session. A language model can answer broad questions from its own world knowledge, but questions about the user's app data need grounding in the app's Core Spotlight index.

SpotlightSearchTool adopts the Foundation Models Tool protocol. The model can decide to call it, generate search arguments, receive a description of the result set, and use that output to produce a grounded response. The app must already donate searchable content to Core Spotlight or index entities for Apple Intelligence.

• Running example: a hiking trails app with trails, locations, completion dates, and personal notes.
• SpotlightSearchTool is available on iOS, iPadOS, macOS, and visionOS.
• Use prior Core Spotlight indexing practices: donate searchable items, manage reindexing, and include metadata that supports semantic and structured search.

Ask a Foundation Models session

A plain LanguageModelSession can answer from model knowledge; adding SpotlightSearchTool grounds questions in indexed app content.

let response = try await session.respond(to: "What are some nice hikes near water?")

Configure SpotlightSearchTool and add it to a session

Adopting SpotlightSearchTool has three parts: configure the kind of search the model can perform, provide additional context while search is active, and decide how to display both generated responses and raw search results.

Configuration starts by importing CoreSpotlight and FoundationModels, creating the tool, selecting a model such as SystemLanguageModel or a model from the Model Provider APIs, and passing the tool into LanguageModelSession.

• The default tool searches the app's Core Spotlight index.
• A custom configuration can specify sources, such as file paths in the app sandbox.
• The model may call the tool one or more times before generating the final answer.

Create a default or file-backed SpotlightSearchTool

The default initializer is enough for the app's Core Spotlight index; configuration can narrow sources such as files.

import CoreSpotlight
import FoundationModels

let tool = SpotlightSearchTool()

let fileTool = SpotlightSearchTool(
    configuration: .init(
        sources: [.files]
    )
)

Add the tool to a LanguageModelSession

Once included in the session, the model can call SpotlightSearchTool when it needs app-indexed context.

import CoreSpotlight
import FoundationModels

let tool = SpotlightSearchTool()
let session = LanguageModelSession(
    model: model,
    tools: [tool],
    instructions: instructions
)

let response = try await session.respond(to: "What hikes have I gone on?")

Hydrate full items and display partial results

Some Spotlight metadata, including text content and HTML, may be stored in compact searchable representations that are not recoverable in a form the model can read. To expose complete item data during model search, implement the new delegate method that returns full CSSearchableItem values for identifiers.

For UI, the session response is usually appropriate for an assistant-style answer. For list-style search result displays, consume searchResults from SpotlightSearchTool directly. Replies arrive as an async sequence and can contain batches of results before the tool call completes.

• Implement searchableItems(forIdentifiers:) on CSSearchableIndexDelegate to hydrate full items on demand.
• Attach extra attributes during hydration when they are useful for model reasoning but not appropriate to donate for regular search.
• Use each reply's queryToken to detect a new tool query, because one model response may involve multiple Spotlight searches.

Implement an index delegate for full item recovery

The delegate lets SpotlightSearchTool recover complete searchable items by unique identifier for model reasoning.

import CoreSpotlight

class IndexDelegate: NSObject, CSSearchableIndexDelegate {
    func searchableItems(forIdentifiers identifiers: [String]) async -> [CSSearchableItem] {
        let entries = await mystore.fetchEntries(ids: identifiers)
        return entries.map { makeSearchableItem(from: $0) }
    }
}

Track partial search replies by query token

Search replies are streamed in batches; query tokens keep UI state aligned with the model's tool calls.

import CoreSpotlight
import FoundationModels

let tool = SpotlightSearchTool()

for await reply in tool.searchResults {
    if reply.queryToken != currentToken {
        currentToken = reply.queryToken
        // Start a new display section or refresh the current results UI.
    }

    switch reply.content {
    case .items(let searchItems):
        // Display this batch of CSSearchableItem results.
        break
    default:
        break
    }
}

Customize search guidance, reference resolution, and pipelines

SpotlightSearchTool exposes semantic search over text and structured search over metadata such as dates, people, and locations. Guidance profiles constrain that capability set so the model sees only what is relevant to the app, which is especially important for limited-context on-device models.

Reference resolution supplies context that is not directly in the search index. If indexed metadata includes person relationships, a ContactResolver can return identities for the current user so references such as "me" or a participant can be matched against indexed metadata.

For complex requests, the model can use pipeline search: a combination of index queries and computation stages. Apps can register custom Generable stages, such as a happiness score over hiking notes, and the tool may return pipeline outputs as partial results.

• Use GuidanceProfile to enable capabilities like text matching, dates, people, and specific attributes.
• Use focused guidance for simpler searches or constrained model contexts.
• Custom stages declare input and output SearchPipelineDataType values and are registered in the tool configuration.
• Partial reply content can include items, scored items, grouped items, counts, tables, statistics, and text, each with an LLM-generated label.

Set a dynamic guidance profile

Guidance profiles scope the search interface the model uses for generation.

import CoreSpotlight
import FoundationModels

let profile = SpotlightSearchTool.GuidanceProfile(
    textMatch: true,
    dates: true,
    people: false,
    attributes: [.title, .altitude, .completionDate]
)

let tool = SpotlightSearchTool(
    configuration: .init(
        guide: .init(level: .dynamic(profile))
    )
)

let focusedTool = SpotlightSearchTool(
    configuration: .init(
        guide: .init(level: .focused(.items))
    )
)

Provide user identity with a ContactResolver

A contact resolver lets the tool match user references to person metadata in the search index.

import CoreSpotlight
import FoundationModels

struct MyContactResolver: ContactResolver {
    func userIdentity() -> ResolvedContact {
        var contact = ResolvedContact(displayName: "Jane Doe")
        contact.emailAddresses = ["jane@example.com", "jdoe@work.com"]
        contact.names = ["Jane", "JD"]
        return contact
    }
}

tool.contactResolver = MyContactResolver()

Define and register a custom pipeline stage

Custom stages let the app run domain-specific computation over result sets for model-generated pipelines.

import CoreSpotlight
import FoundationModels

@Generable
struct HappinessStage: CustomStage {
    static var name = "happiness"
    static var description = "Scores hike by how happy the author was"
    static var inputTypes: [SearchPipelineDataType] = [.items]
    static var outputTypes: [SearchPipelineDataType] = [.scoredItems]

    @Guide(description: "Minimum happiness score (0.0-1.0) to include in results")
    var threshold: Double?

    func execute(on input: SearchPipelineData) async throws -> SearchPipelineData {
        return SearchPipelineData(payload: .scoredItems(sorted))
    }
}

let tool = SpotlightSearchTool(
    configuration: .init(
        customStages: [.happinessBoost(threshold: 0.5)]
    )
)

Evaluate tool-calling and result quality

The Evaluations framework helps measure how well the model calls SpotlightSearchTool and how meaningful the response is. The hiking trails example focuses on result coverage: given indexed data, did the model's search and response include the expected items?

The evaluation setup defines samples that include natural-language inputs, expected outputs, trajectory expectations, and expected searchable item identifiers. Seed samples can be expanded with Sample Generation APIs, then loaded into a test target that indexes items, runs the evaluation, and asserts metrics.

• Adopt ModelSampleProtocol for the evaluation dataset.
• Use TrajectoryExpectation to assert that the response includes a SpotlightSearchTool call.
• Use custom metrics such as ResultCoverage to compare expected identifiers with returned or final-response items.
• Evaluations can be used to iterate on searchable metadata, guidance profiles, custom stages, and model choice.

Define an evaluation sample type

Samples pair user prompts with expected model behavior and expected Spotlight result identifiers.

import Evaluations

struct TrailRequest: ModelSampleProtocol {
    typealias ExpectedValue = String
    typealias Expectation = TrajectoryExpectation

    var input: ModelSampleInput
    var output: ModelSampleOutput<String, TrajectoryExpectation>
    var expectedIdentifiers: [String]
}

Expect a Spotlight tool call

Trajectory expectations verify that the model used SpotlightSearchTool as part of the response.

import Evaluations

TrajectoryExpectation(
    unordered: [
        ToolExpectation(
            "searchSpotlight",
            arguments: [.keyOnly(argumentName: "query")]
        )
    ]
)

Run an evaluation test with a result coverage threshold

A test can index fixture data, run the search evaluation, and fail when aggregate quality drops below a chosen threshold.

@Test("Trail search evaluation meets quality thresholds")
func trailSearchEval() async throws {
    let items = try Self.loadItems()
    let samples = try Self.loadSamples()

    try await Self.indexDelegate.indexSearchableItems(items)

    let tool = Self.makeSearchTool()
    let evaluation = TrailSearchEvaluation(
        tool: tool,
        dataset: ArrayLoader(samples: samples)
    )

    let result = try await evaluation.run()
    let coverageMean = result.aggregateValue(.mean(of: Metric("ResultCoverage")))

    #expect(coverageMean >= 0.5, "Result coverage should be at least 50% across queries")
}

Resources

• Spotlight search tool
• Making your indexed content available to Foundation Models

Unofficial, not associated with Apple. By