WWDC 2026: Run local agentic AI on the Mac using MLX (Session 232)

Build fully local agentic AI workflows on Mac with MLX-LM Server, OpenAI-compatible agents, continuous batching, and distributed inference.

Watch on Apple Developer

TL;DR

MLX-LM Server exposes local language models through an OpenAI-compatible chat completions API, so tools like OpenCode and Xcode can use a Mac-hosted model instead of a cloud endpoint.
A local agentic loop can run commands, read files, call APIs, inspect results, and iterate while keeping model inference and code context on the Mac.
MLX improves local agent performance with M5 Neural Accelerators for prompt processing, continuous batching for concurrent subagents, and distributed inference across multiple Macs.
Setup is three steps: install mlx-lm, launch mlx_lm.server with a tool-calling model, then configure the agent's base URL to http://127.0.0.1:8080/v1.

The session explains how to run agentic AI workflows entirely on a Mac using MLX. Unlike a simple chat interaction, an agent loops between the user request, the language model, and tools such as shell commands, file reads, GitHub CLI calls, APIs, build tools, and test commands.

Running this loop locally gives developers privacy, offline availability, low latency, and no per-token API usage cost. Network access is still possible for tool calls, such as fetching GitHub pull requests, but model inference and local project context stay on the machine.

Agentic loop: user → agent → model → tools → observations → model → repeat until done.
Useful for coding workflows that require reading a project, building, fixing compile errors, or summarizing repository changes.
Demonstrated with OpenCode summarizing MLX pull requests using a locally hosted model and standard command-line tools.

The four-layer MLX agent stack

The local stack has four layers. MLX is the Apple silicon array framework handling low-level computation, Metal acceleration, and memory management. MLX-LM adds model loading, inference, quantization, fine-tuning, CLI tools, and a Python API for language models.

MLX-LM Server provides the agent-facing layer: a persistent OpenAI-compatible HTTP server with support for structured tool calling and reasoning models. At the top, any agent or coding tool that speaks the OpenAI chat completions protocol can connect to the local server.

Foundation: MLX, optimized for Apple silicon.
Model layer: MLX-LM, with support for thousands of Hugging Face models.
Server layer: MLX-LM Server, an OpenAI-compatible local endpoint.
Agent layer: OpenCode, Xcode, custom scripts, or other OpenAI-compatible agent frameworks.
Related ecosystem tools mentioned include Ollama, LM Studio, and vLLM.

Set up a local agent

Setup is intentionally small: install MLX-LM, start mlx_lm.server with a model that supports tool calling, then configure the agent to use the local server as its model provider. Starting with a small model is recommended for validating the setup.

The server listens on localhost and exposes a /v1/chat/completions API compatible with existing OpenAI-style clients. The agent generally does not need to know whether the model is local or cloud-hosted; it just needs the base URL and model name.

Install mlx-lm with pip.
Launch mlx_lm.server with a tool-calling model.
Point the agent provider configuration at http://127.0.0.1:8080/v1.

Install MLX-LM, start the server, and test chat completions

Starts a local MLX-LM Server and verifies the OpenAI-compatible chat completions endpoint.

# Step 1: Install MLX-LM
pip install mlx-lm

# Step 2: Start the server
mlx_lm.server --model mlx-community/Qwen-3.5-4B-8bit

# Step 3: Point your agent to the server
curl -X POST \
  http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"default_model","messages":[{"role":"user","content":"Hello!"}]}'

Configure OpenCode for a local MLX provider

Defines an OpenAI-compatible local provider and routes OpenCode's primary and small-model traffic to MLX-LM Server.

{
  "$schema": "https://opencode.ai/config.json",
  "model": "mlx/default_model",
  "small_model": "mlx/default_model",
  "provider": {
    "mlx": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "MLX (local)",
      "options": {
        "baseURL": "http://127.0.0.1:8080/v1"
      },
      "models": {
        "default_model": {
          "name": "Default MLX Model"
        }
      }
    }
  }
}

Performance: prompt processing, batching, and scale-out

Agentic workloads repeatedly process large tool outputs and accumulated context. The session emphasizes that these sessions can involve hundreds of thousands of tokens, and most of that work is prompt processing rather than token generation.

On M5, MLX can target Neural Accelerators for matrix multiplication, which the session states is four times faster than M4 for this work. With MLX multiplication and attention kernels, that improvement translates closely to prompt-processing speedups without requiring code changes or special flags.

MLX-LM Server also uses continuous batching so concurrent agent or subagent requests can be grouped dynamically on the GPU. For models too large for one Mac, MLX distributed inference can shard a model across multiple Macs over Thunderbolt or Ethernet.

Neural Accelerators help agents read codebases and process tool results faster on supported M5 hardware.
Continuous batching lets new requests join in-progress batches instead of waiting for one request to finish.
Distributed inference can run larger models and parallelize prompt processing across Macs.
Starting with macOS 26.2, Thunderbolt RDMA provides low-latency, high-bandwidth communication for distributed MLX inference.

Launch distributed inference with MLX

Runs MLX-LM Server through mlx.launch, using a hostfile to shard the model across multiple Macs.

mlx.launch --hostfile hosts.json \
  --backend jaccl \
  /remote/path/to/mlx_lm.server \
  --model mlx-community/Qwen-3.5-122B-A3B-8bit

Coding demos with OpenCode and Xcode

The session demonstrates using OpenCode with a local MLX server to generate a SwiftUI drawing app from a blank Xcode project. The agent inspects the project structure, plans the implementation, writes files, runs builds, and fixes compile errors using standard development tools such as xcodebuild.

A second demo connects Xcode directly to the already-running MLX server. In Xcode Settings, the Intelligence tab can add a locally hosted chat provider by selecting the local port, such as 8080. Xcode then uses the local model to inspect project files, understand build errors, and make targeted fixes.

OpenCode demo: generate and iterate on a SwiftUI iPad drawing app entirely on-device.
Xcode demo: add a Locally Hosted chat provider pointing at the MLX server port.
Local coding agents can use normal project files and build tools while keeping code context on the Mac.

Run local agentic AI on the Mac using MLX