WWDC 2026: Design immersive environments for visionOS apps and the spatial web (Session 234)

Design photoreal immersive visionOS and spatial web environments with intentional pre-production, high-quality capture, 3D cleanup, motion, and Spatial Audio.

Watch on Apple Developer

TL;DR

Immersive environments are not flat panoramas: they need depth, parallax, motion, lighting, and audio to feel like believable places in visionOS.
Pre-production should define intent, viewer use cases, composition, layers, removable elements, motion needs, and spatial audio opportunities before expensive capture or CG work begins.
Production guidance emphasizes source imagery quality: leveled tripod capture, 360° coverage, bracketed exposures, secondary cameras, photogrammetry/LiDAR reference, lighting charts, and over-capturing resolution.
Post-production turns panoramas into textured 3D meshes, cleans up distractions, preserves fidelity with A/B checks, and uses efficient shader techniques such as UV flow maps, scrolling masks, flip books, layered sine waves, and normal maps.

What makes an environment immersive

visionOS system environments are described as photorealistic natural landscapes built for spatial computing, not simple backgrounds. Unlike flat media or a panoramic image, an immersive environment should respond to perspective changes with depth and parallax.

The session frames the design process around three phases: pre-production, production, and post-production. Examples from Apple environments such as Mount Hood, the Moon, Jupiter, Yosemite, Thorsmork, and Bora Bora illustrate how intent, source capture, 3D asset work, sound, and motion combine into a convincing illusion.

Design for a primary viewpoint, but account for the full 360° scene.
Expect a fully immersed viewer to see approximately 81° of the scene in their field of view.
Treat sound and motion as core immersion elements, not decorative add-ons.
Use visual reference to guide both artistic direction and technical shader controls.

Pre-production: define intent and composition

Start by answering why the environment exists, what qualities it should bring to life, and how the viewer will use the space. A media-focused environment may need terrain sculpted for a large screen and a carefully researched viewing center line, while a presentation-practice environment may intentionally omit sound and concentrate lighting on a stage.

Scouting is important for real locations. Decide where the viewer should be placed, what appears in front of them, and what they see when turning around. Scouting also reveals unwanted elements such as roads, dense vegetation, or other distractions that should be replaced or removed later.

For inaccessible or invented environments, collect reference materials such as photography, videography, mission imagery, or data sources. The Moon environment used Apollo photography; Yosemite planning used Digital Elevation Models and Earth-orbit information to study lighting and determine capture dates and times.

Visualize the scene in layers from background to foreground.
Identify elements that need motion and where spatial audio may be attached.
Flag features to omit, replace, simplify, or emphasize before production starts.
Use pre-production discoveries to update requirements; for Jupiter, a scale model led to a design requirement for passage of time.

Production: capture source material for 3D work

High-quality primary photography makes the later 3D asset process easier. For Yosemite, the team chose a viewpoint with clear mid-distance framing because foreground detail can be added with CG, while obstructing elements are harder to remove.

Plan for season, weather, and time of day, especially around sunrise and sunset where lighting changes quickly. The recommendation is to spend more time on location and shoot more material than expected, because every image can become useful for 3D asset creation.

Use a tripod with the camera leveled 1 meter off the ground and ensure deep depth of field.
If possible, trigger a second camera 2 meters off the ground at the same time to help fill areas not visible from the primary view.
Use a rig and lens setup that covers all views and can produce a stitched 360° panorama.
Shoot bracketed exposures to capture detail across high dynamic range from sun to shadow.
For visionOS, target sharpness is 40 pixels per degree; an ideal 360° panorama target is 14,400 × 7,200 pixels.

Secondary capture and reference data

Secondary photography, photogrammetry, LiDAR, lighting reference, and video reference provide measurements and context for CG work. Point clouds can be meshed as starting points for assets and can reveal real-world distances or terrain slopes that affect modeling decisions.

Lighting references such as Macbeth charts, chrome spheres, and gray spheres should be captured at the same time as primary photography so CG assets can be integrated consistently. Video of moving scene elements helps shader authors reproduce believable motion, and sound notes help source appropriate audio.

Use photogrammetry and LiDAR point clouds for geometry starts and distance measurements.
Capture lighting references alongside the main panorama photography.
Record motion reference for water, vegetation, clouds, or other dynamic elements.
If photographic panorama capture is impossible, create a rendered panorama in a digital content creation tool for full scene control.

Post-production: clean up, texture, and preserve fidelity

Post-production begins with building a high-resolution source panorama, then cleaning it up. Obvious removals include the camera rig, footprints, and people; composition-driven removals may include busy vegetation, dominant bushes, or other elements identified during pre-production.

To create parallax and depth, the refined panorama is moved onto textures in UV space for a 3D mesh. Areas not visible in the panorama need to be filled with secondary photography or CG renders. Fidelity checks are critical because texture transfer can introduce sharpness, color, value, or data-loss problems.

Use iterative digital matte painting and CG rendering to refine the panorama.
Maintain color balance and lighting consistency throughout cleanup.
A/B compare the 3D asset against the refined panorama to verify texture quality.
Keep scene element sharpness consistent with nearby objects so scale feels correct.
Flop the scene, or test extreme gamma and gain values, to reveal composition issues, value inconsistencies, or texture-transfer losses.

Motion, lighting, and Spatial Audio on a real-time budget

A textured mesh forms the visual base, but immersion also depends on sound and motion. Spatial audio emitters can be placed at meaningful scene locations, such as a rippling-water sound positioned where a river flows around rocks.

For real-time rendering, the session emphasizes artful approximations over expensive dynamic simulation. Bora Bora is used as an example with evolving clouds, connected cloud shadows, palm movement, waves, and time-varying water appearance, all designed to achieve the visual intent within a real-time rendering budget.

Use UV flow maps for evolving clouds or wind-driven vegetation at low cost.
Use scrolling masks to darken terrain textures for cloud shadows instead of rendering dynamic lights.
Use pre-rendered flip book textures to approximate dense, soft palm shadows.
Use hierarchical vertex animation and layered sine waves to combine low-frequency trunk/frond sway with higher-frequency leaflet motion.
Layer normal maps, scrolling textures, and hue/saturation/brightness modulation to simulate water movement and light interaction.

Design immersive environments for visionOS apps and the spatial web