WWDC 2026: Bring an LLM provider to the Foundation Models framework (Session 339)

Implement a Foundation Models LanguageModelExecutor to package custom local or server-backed LLMs behind LanguageModelSession.

Watch on Apple Developer

TL;DR

Foundation Models now supports interchangeable LLM providers through the public LanguageModel and LanguageModelExecutor protocols, covering Apple models, Core AI, MLX, and custom packages.
A provider package should be a Swift package with deliberate platform support and dependencies; each unique executor Configuration is the cache key for a session's executor store.
Executors translate Transcript entries to the model's native message format, apply ContextOptions and GenerationOptions, stream metadata/usage/text/tool events, and manage state such as loaded weights or KV cache reuse.
Server-backed providers should guide developers toward secure authentication, use Keychain for persisted tokens, consider App Attest, and expose custom metadata, custom segments, or server-side tools when needed.

Model choices through one session API

Foundation Models is extended so local and server-backed LLMs can be used through the same LanguageModelSession API. The session can be constructed with Apple's on-device SystemLanguageModel, Private Cloud Compute, Core AI resources, MLX models, or a provider's own LanguageModel implementation.

The same framework features remain available to app developers when the model conforms to LanguageModel. The session emphasizes swapping models without changing the surrounding app code.

SystemLanguageModel is the on-device Apple Foundation Model.
PrivateCloudComputeLanguageModel provides server-scale inference through Private Cloud Compute.
CoreAILanguageModel runs local model resources efficiently, including ANE support.
MLXLanguageModel can load open-source MLX models by model ID.

Swap models behind LanguageModelSession

App code talks to LanguageModelSession while the concrete model can be swapped.

import FoundationModels
import MLXFoundationModels

// On-device Apple Foundation Model
let model = SystemLanguageModel()

// Other options use the same session shape:
// let model = PrivateCloudComputeLanguageModel()
// let model = try await CoreAILanguageModel(resourcesAt: modelURL)
// let model = MLXLanguageModel(modelID: "mlx-community/my-model")

let session = LanguageModelSession(model: model)
let response = try await session.respond(to: "...")
print(response.content)

Package the provider as a Swift package

Provider integrations should be distributed as Swift packages so app developers can add the repository URL directly in Xcode. Choose supported platforms intentionally: Foundation Models supports iOS, macOS, visionOS, and watchOS, and the open-source framework can also make Linux support useful for server-side Swift.

Dependencies matter because each dependency can add shipped bytes to the developer's app. Publishing is done through normal Swift Package Manager distribution: create a git tag for a release.

Expose the public LanguageModel conformance from the package target developers import.
Keep runtime implementation details in separate targets when useful.
Minimize dependency size and surface area.
Use semantic releases that developers can consume from a repository URL.

Package.swift shape for a model package

The package exposes a model library and can isolate runtime code in an implementation target.

// Package.swift
let package = Package(
    name: "MyModel",
    platforms: [
        .macOS(.v27), .iOS(.v27), .visionOS(.v27), .watchOS(.v27)
    ],
    products: [
        .library(name: "MyModel", targets: ["MyModel"])
    ],
    dependencies: [
        .package(url: "...", .upToNextMinor(from: "1.0.0"))
    ],
    targets: [
        .target(name: "MyModelRuntime"),
        .target(name: "MyModel", dependencies: ["MyModelRuntime"]),
        .testTarget(name: "MyModelTests", dependencies: ["MyModel"])
    ]
)

Implement LanguageModel and LanguageModelExecutor

The integration has two core protocol types. LanguageModel describes the model to the framework by declaring capabilities and providing an executor configuration. LanguageModelExecutor performs the work: initialization, optional prewarming, request handling, and streaming generation back to the session.

Executor instances are cached inside each LanguageModelSession by Configuration, not by model value. If two model values produce the same Hashable configuration, they resolve to the same executor. When the session deallocates, its executor store is released, allowing provider resources such as weights or connections to be torn down naturally.

Declare only capabilities the model can support, such as tool calling, guided generation, or reasoning.
Keep the model value lightweight; put resource loading and inference behavior in the executor.
Use prewarm for expensive setup like loading weights or opening connections, but make respond work even if prewarm was never called.
For stateful integrations, compare each new full Transcript with saved prior state and preserve KV cache or session state only for the unchanged prefix.

Core protocol shape

LanguageModel supplies description and configuration; LanguageModelExecutor performs inference.

public protocol LanguageModel: Sendable {
    var capabilities: LanguageModelCapabilities { get }
    var executorConfiguration: Executor.Configuration { get }
}

public protocol LanguageModelExecutor: Sendable {
    init(configuration: Configuration) throws
    func prewarm(model: Model, transcript: Transcript)
    func respond(
        to request: LanguageModelExecutorGenerationRequest,
        model: Model,
        streamingInto channel: LanguageModelExecutorGenerationChannel
    ) async throws
}

Provider conformance sketch

A provider declares capabilities on the model and implements request execution in the executor.

public struct MyLanguageModel: LanguageModel {
    typealias Executor = MyLanguageModelExecutor

    public var capabilities: LanguageModelCapabilities {
        LanguageModelCapabilities(capabilities: [
            .toolCalling, .guidedGeneration, .reasoning
        ])
    }

    public var executorConfiguration: Executor.Configuration {
        Executor.Configuration(/* ... */)
    }
}

public struct MyLanguageModelExecutor: LanguageModelExecutor {
    public typealias Model = MyLanguageModel

    public struct Configuration: Hashable, Sendable { /* ... */ }

    public init(configuration: Configuration) throws { /* ... */ }

    public func respond(
        to request: LanguageModelExecutorGenerationRequest,
        model: MyLanguageModel,
        streamingInto channel: LanguageModelExecutorGenerationChannel
    ) async throws {
        /* ... */
    }
}

Translate transcripts, options, and streaming responses

The executor receives the full conversation as a Transcript and must translate entries into the provider's native message format. Foundation Models transcript entries cover instructions, prompts, tool calls, tool outputs, responses, and reasoning; the provider decides how these map to its roles, such as system, user, assistant, or a dedicated tool role.

Each request also carries developer intent through ContextOptions and GenerationOptions. Context options affect prompt/context construction, including reasoning level and response schema. Generation options affect decoding, including sampling strategy, temperature, and maximum response length.

Responses are always implemented as streaming events, even when the developer uses a one-shot API. The executor should send useful metadata first, then usage information, then token/tool/reasoning deltas as they arrive.

Map transcript entries carefully; tool calls, tool outputs, and reasoning may all map to assistant for models without dedicated roles.
Read contextOptions and generationOptions from LanguageModelExecutorGenerationRequest before calling the model.
Send metadata such as modelID and requestID early for logging and debugging.
Send prompt token usage before generation when available so developers can account for cost without waiting for the full stream.

Read request options

Executors are responsible for applying both context and decoding options.

func respond(
    to request: LanguageModelExecutorGenerationRequest,
    model: MyLanguageModel,
    streamingInto channel: LanguageModelExecutorGenerationChannel
) async throws {
    let reasoningLevel = request.contextOptions.reasoningLevel
    let temperature = request.generationOptions.temperature
    let maxTokens = request.generationOptions.maximumResponseTokens

    // Pass these through to the provider's inference call.
}

Stream metadata, usage, and text deltas

The recommended stream ordering gives developers identifiers and accounting before generated content.

await channel.send(.response(action: .updateMetadata([
    "modelID": "my-model-2026-06-08",
    "requestID": request.id.uuidString
])))

await channel.send(.response(action: .updateUsage(
    input: .init(totalTokenCount: promptTokens, cachedTokenCount: cachedTokens),
    output: .init(totalTokenCount: 0, reasoningTokenCount: 0)
)))

for try await token in tokens {
    await channel.send(.response(action: .appendText(token)))
}

Handle unsupported requests and provider errors

If a model cannot exactly satisfy a request, the executor can approximate when that still honors the developer's intent. For example, a service that accepts only temperature can map greedy sampling to temperature 0. If there is no honest approximation, throw an error instead.

Foundation Models provides built-in LanguageModelError cases for common cross-model failures. Custom errors should be reserved for service-specific failures such as subscription limits, unavailable account features, or account state problems.

Use LanguageModelError.contextSizeExceeded when the transcript exceeds the context window.
Use LanguageModelError.unsupportedCapability or unsupportedGenerationGuide when a requested feature cannot be honored.
Use rateLimited, refusal, guardrailViolation, unsupportedTranscriptContent, unsupportedLanguageOrLocale, or timeout when those cases fit.
Define custom LocalizedError cases only for failures unique to the provider service.

Approximate or throw

Approximate compatible intent where possible; otherwise fail with a meaningful framework error.

if request.generationOptions.sampling?.kind == .greedy {
    serviceRequest.temperature = 0
}

if let schema = request.schema,
   let budget = request.generationOptions.maximumResponseTokens,
   budget < minimumTokens(for: schema) {
    throw LanguageModelError.unsupportedCapability(
        .init(
            capability: .guidedGeneration,
            debugDescription: "Token budget too small to satisfy this schema."
        )
    )
}

Authentication, metadata, custom segments, and server-side tools

Server-backed model packages should steer developers toward secure credential flows instead of encouraging raw API key strings in initializers. If the package fetches or persists access tokens, store them securely with Keychain. Providers should also consider App Attest for device verification, tamper detection, signed payloads, and fraud signals for cloud traffic.

Customization lets a provider expose differentiating capabilities without forcing developers to leave LanguageModelSession. Providers can attach custom metadata, define custom Transcript segments for new modalities, and surface server-side tool results at different levels of detail.

Custom response metadata can report service-specific data such as tokensPerSecond and timeToFirstToken.
Custom segments are typed PromptRepresentable values that can appear in prompts and responses, supporting modalities such as audio or video.
Server-side tools can run privately, annotate text with metadata such as citations, or emit their structured output as custom segments.
Privacy characteristics differ significantly between on-device and cloud-backed packages; developers and users should understand which model type is in use.

Custom segment for audio input and output

Custom segments carry typed non-text content through LanguageModelSession and the executor channel.

public struct AudioSegment: Transcript.CustomSegment {
    public var id: String
    public var content: URL
}

let recording = AudioSegment(
    id: UUID().uuidString,
    content: URL(filePath: "/path/to/recording.m4a")
)

let response = try await session.respond {
    "Where was Frank Lloyd Wright's original architecture school located?"
    recording
}

for try await event in stream {
    switch event {
    case .audioFileGenerated(let file):
        await channel.send(.response(action: .updateCustomSegment(
            AudioSegment(id: file.id, content: file.url)
        )))
    }
}

Surface server-side tool output

A provider can expose server-side tools as typed model options and stream their structured results.

public struct MyLanguageModel: LanguageModel {
    public struct ServerTool: Sendable {
        public static let webSearch: ServerTool = ...
    }

    public init(serverTools: [ServerTool] = []) { }
}

let client = MyServerClient(serverTools: model.serverTools)
let response = try await client.send(prompt: .init(request))

for try await chunk in response {
    switch chunk {
    case .webSearch(let webSearch):
        await channel.send(.response(action: .updateCustomSegment(
            WebSearchSegment(url: webSearch.url, content: webSearch.html)
        )))
    case .textDelta(let textDelta):
        await channel.send(.response(action: .appendText(
            textDelta.text,
            tokenCount: textDelta.tokenCount
        )))
    }
}

Bring an LLM provider to the Foundation Models framework