Enhance RAW image processing with Core Image

Watch on Apple Developer

TL;DR

Core Image RAW support and RAW 9

RAW files require a processing pipeline before display: metadata parsing and sensor unpacking, demosaic from a color-filter mosaic into RGB pixels, denoising, sharpening/local contrast convolutions, and final white balance, exposure, color, and tone adjustments. Core Image and Image IO provide system RAW support across Apple platforms, so many apps and frameworks get basic viewing automatically.

RAW 9 is a major update to the Core Image RAW pipeline in iOS, iPadOS, macOS, and visionOS 27. It uses a tiled Core ML model that combines demosaic and denoise, running on the Apple Neural Engine where available, to improve sharpness, color definition, and noise handling.

• System RAW support has grown from 21 camera models in 2006 to 784 models across major camera vendors.
• Older RAW pipeline versions remain available where supported so existing rendering choices can be preserved.
• RAW 9 supports hundreds of models at launch, can grow through operating-system updates, and is automatically supported for cameras that shoot DNG natively, including Apple iPhones.

Enable RAW 9 and expose CIRAWFilter editing controls

RAW 9 is not enabled by default. Load RAW files with CIRAWFilter, check whether the filter's supportedDecoderVersions includes version9, then set decoderVersion to version9 when available. Use the supportedCameraModels class method to obtain the list of camera models supported by a decoder version.

CIRAWFilter's calibrated editing properties are the main app-facing way to expose RAW controls. The session highlights exposure, luminanceNoiseReductionAmount, sharpnessAmount, and contrastAmount as important controls for user-facing RAW editing.

• colorNoiseReductionAmount has no effect in RAW 9 because the Core ML model handles color noise reduction automatically.
• detailAmount and moireReductionAmount are no longer needed or supported in RAW 9.
• Use the filter's supported-property checks to determine whether a property is available for a given filter instance.

Performance guidance for interactive editing

RAW 9 is more resource intensive than previous RAW versions, but Core Image can keep repeated edits responsive by caching intermediate work. This is especially important when a single RAW is rendered repeatedly at screen resolution while users adjust CIRAWFilter properties.

• Set CIRAWFilter scaleFactor when displaying a reduced-size image so Core Image does not render more pixels than needed.
• Use one CIContext per view and set cacheIntermediates to true for interactive editing.
• Add the Extended Virtual Addressing entitlement when appropriate so Core Image can use more memory for caching between renders.
• Render directly to Metal-backed views, such as MTKView, so repeated renders can overlap GPU work more effectively.

Performance guidance for export

Exporting is a different workload: multiple RAW files are usually rendered once at full resolution to formats such as HEIF or JPEG. For this case, caching intermediate results is less useful and may waste memory.

• Create an export CIContext with cacheIntermediates set to false.
• Tune the CIContext memoryLimit option; iOS defaults to a conservative 256 MB, while 512 MB or 1024 MB can improve export performance when memory allows.
• Prefer Core Image's heifRepresentation and jpegRepresentation context methods for additional memory savings instead of calling Image IO directly.

CIContext options for export

Creates an export-oriented CIContext that disables intermediate caching and raises Core Image's memory limit.

let exportCtx = CIContext(options: [
    .cacheIntermediate: false,
    .memoryLimit: 512
])

New CIImageProcessor capabilities

RAW 9 uses CIImageProcessor because it can combine Core ML with other Core Image kernels. The session introduces two CIImageProcessor improvements useful for custom image processing: explicit output tile sizes and managed temporary pixel buffers.

Explicit tiling lets a processor choose output regions instead of relying entirely on Core Image's memory-driven tiling. Temporary buffers let processors request scratch CVPixelBuffers from Core Image's cache, which avoids repeatedly allocating and destroying buffers for each tile.

• In a CIImageProcessorKernel process callback, operate only on input.region and output.region.
• Use apply(withTiledExtent:inputs:arguments:) with an array of tile rectangles to control output tile layout.
• Use CIImageProcessorOutput temporaryPixelBuffer with an identifier when a process callback needs one or more scratch buffers.
• Core Image manages temporary buffer lifetime and can recycle buffers across tiles.

CIImageProcessor with explicit output tile sizes

Builds 512×512 output tiles covering an image extent and applies a CIImageProcessorKernel using those explicit output regions.

import CoreImage

class MyProcessor: CIImageProcessorKernel {
    override class func roi(forInput input: Int32,
                            arguments: [String: Any]?,
                            outputRect: CGRect) -> CGRect {
        return outputRect
    }

    override class func process(with inputs: [CIImageProcessorInput]?,
                                arguments: [String: Any]?,
                                output: CIImageProcessorOutput) throws {
        guard let input = inputs?.first,
              let iBuffer = input.pixelBuffer,
              let oBuffer = output.pixelBuffer else { return }

        let iRegion = input.region
        let oRegion = output.region
        // MyCopyBuffer(iBuffer, iRegion, oBuffer, oRegion)
    }
}

let extent = inImg.extent
let tileSize = 512.0
var tiles: [CIVector] = []

for y in stride(from: extent.minY, to: extent.maxY, by: tileSize) {
    for x in stride(from: extent.minX, to: extent.maxX, by: tileSize) {
        let tile = CGRect(x: x, y: y,
                          width: min(tileSize, extent.maxX - x),
                          height: min(tileSize, extent.maxY - y))
        tiles.append(CIVector(cgRect: tile))
    }
}

let result = try MyProcessor.apply(withTiledExtent: tiles,
                                   inputs: [inImg],
                                   arguments: [:])

CIImageProcessor temporary pixel buffer

Requests a Core Image-managed scratch CVPixelBuffer for tile processing, avoiding repeated manual allocation.

import CoreImage

class MyProcessor: CIImageProcessorKernel {
    override class func process(with inputs: [CIImageProcessorInput]?,
                                arguments: [String: Any]?,
                                output: CIImageProcessorOutput) throws {
        guard let input = inputs?.first,
              let srcPixelBuffer = input.pixelBuffer,
              let dstPixelBuffer = output.pixelBuffer else { return }

        guard let scratch = output.temporaryPixelBuffer(
            identifier: "myScratch",
            format: kCVPixelFormatType_64RGBAHalf,
            width: Int(output.region.width),
            height: Int(output.region.height),
            pixelBufferAttributes: nil
        ) else { return }

        // Step 1: copy input CVPixelBuffer → scratch
        // Step 2: process pixels in scratch
        // Step 3: copy scratch → output CVPixelBuffer
    }
}

Resources

• Extended Virtual Addressing Entitlement

Related Sessions

• Display EDR content with Core Image, Metal, and SwiftUI
• Capture and process ProRAW images

Unofficial, not associated with Apple. By