Build AI-powered scripts with the fm CLI and Python SDK

Watch on Apple Developer

TL;DR

New macOS access paths for Apple Foundation Models

The session introduces two macOS 27 ways to use Apple Foundation Models outside Swift apps: the preinstalled fm command line tool and the Foundation Models SDK for Python. Both are intended to make prompt development, automation, and evaluation faster without rebuilding an Xcode project for every iteration.

The tools build on Foundation Models framework concepts from Swift, including guided generation for structured output and tool calling for letting the model interact with app or script context. macOS 27 and iOS 27 also add framework support for image inputs and access to server models through the same Swift API.

• fm is available from Terminal on macOS 27.
• By default, fm uses the on-device model available with macOS.
• fm can also target the Apple Foundation Model on Private Cloud Compute, which is larger and intended for more complex problems but has usage limits.
• The Python SDK requires Python 3.10+, Xcode, and an Apple Silicon Mac.

Using the fm command line tool

Running fm shows available commands. fm chat starts an interactive terminal conversation with the on-device model, supports follow-up turns, and includes slash commands such as /model to switch to Private Cloud Compute and /save to save a conversation for later.

For inline terminal use and automation, fm respond takes a prompt and writes the model response to stdout. It supports options such as --model, --image, --schema, and --help.

• Use fm chat for exploratory prompt testing.
• Use fm respond when a script needs a single response as command output.
• Use fm schema object to define structured JSON output for downstream shell processing.
• Use --model pcc when the task needs the Private Cloud Compute model.

Prompt the model, use images, and request structured output

fm respond can target different models, include an image, and constrain output with a schema generated by fm schema object.

$ fm respond "Provide a basic regex in Swift to parse an email address"

$ fm respond "Provide a comprehensive regex in Swift to parse an email address" --model pcc

$ fm respond "What app is the user using in this screenshot?" --model pcc --image Screenshot.png

$ fm schema object --name AppsIdentified --string app_names --array > schema.json
$ fm respond "What apps are the user actively using in this screenshot?" \
  --image Screenshot.png --model pcc --schema schema.json
# {"app_names": ["Messages", "Mail", "Calendar"]}

$ fm respond --help

Automating file management with structured fm output

The automation demo uses fm inside a shell script to classify files in a messy project folder as final files or drafts based on their names. The model returns schema-constrained JSON, and the script uses that JSON to copy final files to backup storage and move drafts to an archive.

The important pattern is to keep the model responsible for fuzzy classification while the shell script remains responsible for deterministic file operations. Structured output makes the model result machine-readable enough to drive the rest of the script.

• Create a schema with two array fields: final_files and draft_files.
• Prompt the model with clear instructions and the full file list.
• Pipe JSON output through jq to iterate over each returned filename.
• Use standard shell commands such as cp and mv for the actual file changes.

Classify files with fm and route them with jq

The model produces structured classification data; the script consumes that data to perform safe, explicit file operations.

fm schema object --name "TriagedFileList" \
  --string 'final_files' --array \
  --string 'draft_files' --array > /tmp/schema.json

output=$(fm respond \
  --instructions "I just completed a project, and I need help triaging the latest version of the files from the previous versions. I will give you a list of files. Return a list of the latest files (i.e., all files that, you can infer from their name in the list, are the latest versions), and then return separately a list of all draft files (i.e., all files that weren't considered final)." \
  "This is the list of all files:\n\n${files_list}" \
  --schema /tmp/schema.json)

echo "${output}" | jq -r '.final_files[]' | while read -r file; do
  cp "${DIRECTORY_TO_TRIAGE}/${file}" "${FINAL_FILES_STORAGE_DIRECTORY}"
done

echo "${output}" | jq -r '.draft_files[]' | while read -r file; do
  mv "${DIRECTORY_TO_TRIAGE}/${file}" "${DRAFT_FILES_STORAGE_DIRECTORY}"
done

Foundation Models SDK for Python basics

The Python SDK gives Python code access to Apple Foundation Models on macOS. It mirrors core Foundation Models framework features: text and image prompts, streaming responses, guided generation, and tool calling.

The session uses a grocery-ordering app prototype to show how a developer can iterate on prompts in Python before moving an implementation to Swift. A LanguageModelSession is created with instructions, and session.respond returns the model output.

• Install with pip install apple_fm_sdk or another Python package manager.
• Import the SDK as apple_fm_sdk.
• Use LanguageModelSession to hold instructions and tools.
• Call session.respond(...) asynchronously to prompt the model.

Install the SDK

Installs the Foundation Models SDK for Python in a local Python environment.

pip install apple_fm_sdk

Create a session and respond to a prompt

Creates a language model session with app-specific instructions and returns a response for a user prompt.

import apple_fm_sdk as fm

INSTRUCTIONS = "You're an AI assistant for Cupertino Mart, a grocery store with in-app ordering."

async def answer_question(prompt: str) -> str:
    session = fm.LanguageModelSession(instructions=INSTRUCTIONS)
    return await session.respond(prompt)

Tool calling and guided generation in Python

The SDK lets Python developers expose tools that the model can call. In the grocery example, a tool fetches recent orders so the model can provide personalized suggestions based on user context.

Guided generation uses the fm.generable decorator to define structured output types. The model can then generate directly into a typed object, such as an item suggestion object containing a list of item names.

• Subclass fm.Tool to define a callable tool.
• Expose argument structure with @fm.generable and generation_schema().
• Use fm.guide(...) to describe fields for generation.
• Pass a structured type as the generating argument to session.respond.

Define a model-callable tool

A Python tool exposes past-order lookup to the model with a generated argument schema.

class GetPastOrdersTool(fm.Tool):
    name = "get_past_orders"
    description = "Retrieves information about this user's past orders."

    @fm.generable("Past orders query parameter")
    class Arguments:
        number_orders: str = fm.guide("How many of the last orders to retrieve")

    @property
    def arguments_schema(self) -> fm.GenerationSchema:
        return self.Arguments.generation_schema()

    async def call(self, args: fm.GeneratedContent) -> str:
        number_orders = args.value(int, for_property="number_orders")
        return await Orders.load_last_orders(user_id=user_id, amount=number_orders)

Generate a structured item suggestion object

@fm.generable constrains the response to an ItemsSuggestion structure instead of free-form text.

@fm.generable("Suggested items")
class ItemsSuggestion:
    item_names: list[str] = fm.guide("Names of the suggested items")

INSTRUCTIONS = "You're an AI assistant tasked with returning potential grocery items that the user might be interested in."

async def generate_suggested_cart_items(user_input: Optional[str]) -> ItemsSuggestion:
    session = fm.LanguageModelSession(instructions=INSTRUCTIONS, tools=load_tools())
    prompt = """Using the tools to load the user's previous orders, \
return a list of items the user has already ordered \
and that they might be interested in again \
as they're getting ready to place a new grocery order."""
    if user_input is not None:
        prompt += f"\nAccount for the following request from the user: {user_input}"
    return await session.respond(prompt, generating=ItemsSuggestion)

Evaluation pipelines with Python tooling

A major motivation for the Python SDK is integration with the Python data science ecosystem. The session demonstrates an evaluation workflow in a Jupyter Notebook for a cart-completion feature: generate evaluation inputs, run several prompt implementations, store results in a Pandas DataFrame, grade outputs with a server judge model, and visualize metrics with matplotlib.

The example compares three prompt styles: minimal, more descriptive, and comprehensive with rules. The resulting charts show tradeoffs such as generation errors, excess items, missing expected items, and hallucinated items. Those measurements guide prompt iteration without rebuilding the app.

• Use the on-device model to generate outputs for each prompt implementation.
• Use a server model as a judge to score outputs against criteria selected for the feature.
• Track inputs, outputs, and grades in a Pandas DataFrame.
• Use matplotlib charts to compare implementations and identify prompt regressions.
• Swift developers can also use the Evaluations framework in Xcode 27 for similar measurement workflows.

Resources

• Foundation Models SDK for Python on GitHub
• Foundation Models SDK for Python Documentation on GitHub

Related Sessions

• Bring an LLM provider to the Foundation Models framework
• Build agentic app experiences with the Foundation Models framework
• Build with the new Apple Foundation Model on Private Cloud Compute
• What's new in the Foundation Models framework

Unofficial, not associated with Apple. By