realitykit

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

RealityKit

Build AR experiences on iOS using RealityKit for rendering and ARKit for world tracking. Covers

RealityView

, entity management, raycasting, scene understanding, and gesture-based interactions. Targets Swift 6.3 / iOS 26+.

在iOS平台上使用RealityKit完成渲染、ARKit完成世界跟踪来构建AR体验。涵盖

RealityView

、实体管理、光线投射、场景理解以及基于手势的交互。适配Swift 6.3 / iOS 26+。

Setup

环境配置

Project Configuration

项目配置

Add
```
NSCameraUsageDescription
```
to Info.plist
For iOS, RealityKit uses the device camera by default via
```
RealityViewCameraContent
```
(iOS 18+, macOS 15+)
No additional capabilities required for basic AR on iOS

在Info.plist中添加
```
NSCameraUsageDescription
```
配置
对于iOS系统，RealityKit默认通过
```
RealityViewCameraContent
```
（iOS 18+, macOS 15+）调用设备相机
iOS基础AR功能无需额外的能力配置

Device Requirements

设备要求

AR features require devices with an A9 chip or later. Always verify support before presenting AR UI.

swift

import ARKit

guard ARWorldTrackingConfiguration.isSupported else {
    showUnsupportedDeviceMessage()
    return
}

AR功能需要搭载A9及以上芯片的设备。在展示AR UI前请务必验证设备支持性。

swift

import ARKit

guard ARWorldTrackingConfiguration.isSupported else {
    showUnsupportedDeviceMessage()
    return
}

Key Types

核心类型

Type	Platform	Role
`RealityView`	iOS 18+, visionOS 1+	SwiftUI view that hosts RealityKit content
`RealityViewCameraContent`	iOS 18+, macOS 15+	Content displayed through the device camera
`Entity`	All	Base class for all scene objects
`ModelEntity`	All	Entity with a visible 3D model
`AnchorEntity`	All	Tethers entities to a real-world anchor

类型	支持平台	作用
`RealityView`	iOS 18+, visionOS 1+	承载RealityKit内容的SwiftUI视图
`RealityViewCameraContent`	iOS 18+, macOS 15+	通过设备相机展示的内容
`Entity`	全平台	所有场景对象的基类
`ModelEntity`	全平台	携带可见3D模型的实体
`AnchorEntity`	全平台	将实体绑定到现实世界锚点

RealityView Basics

RealityView基础

RealityView

is the SwiftUI entry point for RealityKit. On iOS, it provides

RealityViewCameraContent

which renders through the device camera for AR.

swift

import SwiftUI
import RealityKit

struct ARExperienceView: View {
    var body: some View {
        RealityView { content in
            // content is RealityViewCameraContent on iOS
            let sphere = ModelEntity(
                mesh: .generateSphere(radius: 0.05),
                materials: [SimpleMaterial(
                    color: .blue,
                    isMetallic: true
                )]
            )
            sphere.position = [0, 0, -0.5]  // 50cm in front of camera
            content.add(sphere)
        }
    }
}

RealityView

是使用RealityKit的SwiftUI入口。在iOS平台上，它提供

RealityViewCameraContent

，通过设备相机渲染实现AR效果。

swift

import SwiftUI
import RealityKit

struct ARExperienceView: View {
    var body: some View {
        RealityView { content in
            // content在iOS上为RealityViewCameraContent对象
            let sphere = ModelEntity(
                mesh: .generateSphere(radius: 0.05),
                materials: [SimpleMaterial(
                    color: .blue,
                    isMetallic: true
                )]
            )
            sphere.position = [0, 0, -0.5]  // 位于相机前方50cm处
            content.add(sphere)
        }
    }
}

Make and Update Pattern

创建与更新模式

Use the

update

closure to respond to SwiftUI state changes:

swift

struct PlacementView: View {
    @State private var modelColor: UIColor = .red

    var body: some View {
        RealityView { content in
            let box = ModelEntity(
                mesh: .generateBox(size: 0.1),
                materials: [SimpleMaterial(
                    color: .red,
                    isMetallic: false
                )]
            )
            box.name = "colorBox"
            box.position = [0, 0, -0.5]
            content.add(box)
        } update: { content in
            if let box = content.entities.first(
                where: { $0.name == "colorBox" }
            ) as? ModelEntity {
                box.model?.materials = [SimpleMaterial(
                    color: modelColor,
                    isMetallic: false
                )]
            }
        }

        Button("Change Color") {
            modelColor = modelColor == .red ? .green : .red
        }
    }
}

使用

update

闭包响应SwiftUI的状态变化：

swift

struct PlacementView: View {
    @State private var modelColor: UIColor = .red

    var body: some View {
        RealityView { content in
            let box = ModelEntity(
                mesh: .generateBox(size: 0.1),
                materials: [SimpleMaterial(
                    color: .red,
                    isMetallic: false
                )]
            )
            box.name = "colorBox"
            box.position = [0, 0, -0.5]
            content.add(box)
        } update: { content in
            if let box = content.entities.first(
                where: { $0.name == "colorBox" }
            ) as? ModelEntity {
                box.model?.materials = [SimpleMaterial(
                    color: modelColor,
                    isMetallic: false
                )]
            }
        }

        Button("更改颜色") {
            modelColor = modelColor == .red ? .green : .red
        }
    }
}

Loading and Creating Entities

加载与创建实体

Loading from USDZ Files

从USDZ文件加载

Load 3D models asynchronously to avoid blocking the main thread:

swift

RealityView { content in
    if let robot = try? await ModelEntity(named: "robot") {
        robot.position = [0, -0.2, -0.8]
        robot.scale = [0.01, 0.01, 0.01]
        content.add(robot)
    }
}

异步加载3D模型，避免阻塞主线程：

swift

RealityView { content in
    if let robot = try? await ModelEntity(named: "robot") {
        robot.position = [0, -0.2, -0.8]
        robot.scale = [0.01, 0.01, 0.01]
        content.add(robot)
    }
}

Programmatic Mesh Generation

程序化生成网格

swift

// Box
let box = ModelEntity(
    mesh: .generateBox(size: [0.1, 0.2, 0.1], cornerRadius: 0.005),
    materials: [SimpleMaterial(color: .gray, isMetallic: true)]
)

// Sphere
let sphere = ModelEntity(
    mesh: .generateSphere(radius: 0.05),
    materials: [SimpleMaterial(color: .blue, roughness: 0.2, isMetallic: true)]
)

// Plane
let plane = ModelEntity(
    mesh: .generatePlane(width: 0.3, depth: 0.3),
    materials: [SimpleMaterial(color: .green, isMetallic: false)]
)

swift

// 立方体
let box = ModelEntity(
    mesh: .generateBox(size: [0.1, 0.2, 0.1], cornerRadius: 0.005),
    materials: [SimpleMaterial(color: .gray, isMetallic: true)]
)

// 球体
let sphere = ModelEntity(
    mesh: .generateSphere(radius: 0.05),
    materials: [SimpleMaterial(color: .blue, roughness: 0.2, isMetallic: true)]
)

// 平面
let plane = ModelEntity(
    mesh: .generatePlane(width: 0.3, depth: 0.3),
    materials: [SimpleMaterial(color: .green, isMetallic: false)]
)

Adding Components

添加组件

Entities use an ECS (Entity Component System) architecture. Add components to give entities behavior:

swift

let box = ModelEntity(
    mesh: .generateBox(size: 0.1),
    materials: [SimpleMaterial(color: .red, isMetallic: false)]
)

// Make it respond to physics
box.components.set(PhysicsBodyComponent(
    massProperties: .default,
    material: .default,
    mode: .dynamic
))

// Add collision shape for interaction
box.components.set(CollisionComponent(
    shapes: [.generateBox(size: [0.1, 0.1, 0.1])]
))

// Enable input targeting for gestures
box.components.set(InputTargetComponent())

实体采用ECS（实体组件系统）架构。通过添加组件为实体赋予行为能力：

swift

let box = ModelEntity(
    mesh: .generateBox(size: 0.1),
    materials: [SimpleMaterial(color: .red, isMetallic: false)]
)

// 使其支持物理效果
box.components.set(PhysicsBodyComponent(
    massProperties: .default,
    material: .default,
    mode: .dynamic
))

// 添加碰撞形状以支持交互
box.components.set(CollisionComponent(
    shapes: [.generateBox(size: [0.1, 0.1, 0.1])]
))

// 启用手势输入 targeting
box.components.set(InputTargetComponent())

Anchoring and Placement

锚定与放置

AnchorEntity

Use

AnchorEntity

to anchor content to detected surfaces or world positions:

swift

RealityView { content in
    // Anchor to a horizontal surface
    let floorAnchor = AnchorEntity(.plane(
        .horizontal,
        classification: .floor,
        minimumBounds: [0.2, 0.2]
    ))

    let model = ModelEntity(
        mesh: .generateBox(size: 0.1),
        materials: [SimpleMaterial(color: .orange, isMetallic: false)]
    )
    floorAnchor.addChild(model)
    content.add(floorAnchor)
}

使用

AnchorEntity

将内容锚定到检测到的平面或世界坐标位置：

swift

RealityView { content in
    // 锚定到水平平面
    let floorAnchor = AnchorEntity(.plane(
        .horizontal,
        classification: .floor,
        minimumBounds: [0.2, 0.2]
    ))

    let model = ModelEntity(
        mesh: .generateBox(size: 0.1),
        materials: [SimpleMaterial(color: .orange, isMetallic: false)]
    )
    floorAnchor.addChild(model)
    content.add(floorAnchor)
}

Anchor Targets

锚定目标

Target	Description
`.plane(.horizontal, ...)`	Horizontal surfaces (floors, tables)
`.plane(.vertical, ...)`	Vertical surfaces (walls)
`.plane(.any, ...)`	Any detected plane
`.world(transform:)`	Fixed world-space position

目标	描述
`.plane(.horizontal, ...)`	水平平面（地面、桌面）
`.plane(.vertical, ...)`	垂直平面（墙面）
`.plane(.any, ...)`	任意检测到的平面
`.world(transform:)`	固定的世界空间位置

Raycasting

光线投射

Use

RealityViewCameraContent

to convert between SwiftUI view coordinates and RealityKit world space. Pair with

SpatialTapGesture

to place objects where the user taps on a detected surface.

使用

RealityViewCameraContent

在SwiftUI视图坐标和RealityKit世界空间之间转换。搭配

SpatialTapGesture

可以将对象放置在用户点击的检测到的平面位置。

Gestures and Interaction

手势与交互

Drag Gesture on Entities

实体拖拽手势

swift

struct DraggableARView: View {
    var body: some View {
        RealityView { content in
            let box = ModelEntity(
                mesh: .generateBox(size: 0.1),
                materials: [SimpleMaterial(color: .blue, isMetallic: true)]
            )
            box.position = [0, 0, -0.5]
            box.components.set(CollisionComponent(
                shapes: [.generateBox(size: [0.1, 0.1, 0.1])]
            ))
            box.components.set(InputTargetComponent())
            box.name = "draggable"
            content.add(box)
        }
        .gesture(
            DragGesture()
                .targetedToAnyEntity()
                .onChanged { value in
                    let entity = value.entity
                    guard let parent = entity.parent else { return }
                    entity.position = value.convert(
                        value.location3D,
                        from: .local,
                        to: parent
                    )
                }
        )
    }
}

swift

struct DraggableARView: View {
    var body: some View {
        RealityView { content in
            let box = ModelEntity(
                mesh: .generateBox(size: 0.1),
                materials: [SimpleMaterial(color: .blue, isMetallic: true)]
            )
            box.position = [0, 0, -0.5]
            box.components.set(CollisionComponent(
                shapes: [.generateBox(size: [0.1, 0.1, 0.1])]
            ))
            box.components.set(InputTargetComponent())
            box.name = "draggable"
            content.add(box)
        }
        .gesture(
            DragGesture()
                .targetedToAnyEntity()
                .onChanged { value in
                    let entity = value.entity
                    guard let parent = entity.parent else { return }
                    entity.position = value.convert(
                        value.location3D,
                        from: .local,
                        to: parent
                    )
                }
        )
    }
}

Tap to Select

点击选择

swift

.gesture(
    SpatialTapGesture()
        .targetedToAnyEntity()
        .onEnded { value in
            let tappedEntity = value.entity
            highlightEntity(tappedEntity)
        }
)

swift

.gesture(
    SpatialTapGesture()
        .targetedToAnyEntity()
        .onEnded { value in
            let tappedEntity = value.entity
            highlightEntity(tappedEntity)
        }
)

Scene Understanding

场景理解

Per-Frame Updates

逐帧更新

Subscribe to scene update events for continuous processing:

swift

RealityView { content in
    let entity = ModelEntity(
        mesh: .generateSphere(radius: 0.05),
        materials: [SimpleMaterial(color: .yellow, isMetallic: false)]
    )
    entity.position = [0, 0, -0.5]
    content.add(entity)

    _ = content.subscribe(to: SceneEvents.Update.self) { event in
        let time = Float(event.deltaTime)
        entity.position.y += sin(Float(Date().timeIntervalSince1970)) * time * 0.1
    }
}

订阅场景更新事件实现连续处理逻辑：

swift

RealityView { content in
    let entity = ModelEntity(
        mesh: .generateSphere(radius: 0.05),
        materials: [SimpleMaterial(color: .yellow, isMetallic: false)]
    )
    entity.position = [0, 0, -0.5]
    content.add(entity)

    _ = content.subscribe(to: SceneEvents.Update.self) { event in
        let time = Float(event.deltaTime)
        entity.position.y += sin(Float(Date().timeIntervalSince1970)) * time * 0.1
    }
}

visionOS Note

visionOS注意事项

On visionOS, ARKit provides a different API surface with

ARKitSession

WorldTrackingProvider

, and

PlaneDetectionProvider

. These visionOS-specific types are not available on iOS. On iOS, RealityKit handles world tracking automatically through

RealityViewCameraContent

在visionOS上，ARKit提供了不同的API接口，包含

ARKitSession

、

WorldTrackingProvider

和

PlaneDetectionProvider

。这些visionOS专属的类型在iOS上不可用。在iOS上，RealityKit通过

RealityViewCameraContent

自动处理世界跟踪。

Common Mistakes

常见错误

DON'T: Skip AR capability checks

不要跳过AR能力检测

Not all devices support AR. Showing a black camera view with no feedback confuses users.

swift

// WRONG -- no device check
struct MyARView: View {
    var body: some View {
        RealityView { content in
            // Fails silently on unsupported devices
        }
    }
}

// CORRECT -- check support and show fallback
struct MyARView: View {
    var body: some View {
        if ARWorldTrackingConfiguration.isSupported {
            RealityView { content in
                // AR content
            }
        } else {
            ContentUnavailableView(
                "AR Not Supported",
                systemImage: "arkit",
                description: Text("This device does not support AR.")
            )
        }
    }
}

不是所有设备都支持AR功能。展示无反馈的黑色相机界面会让用户产生困惑。

swift

// 错误写法 -- 未做设备检测
struct MyARView: View {
    var body: some View {
        RealityView { content in
            // 在不支持的设备上会静默失败
        }
    }
}

// 正确写法 -- 检测支持性并展示降级UI
struct MyARView: View {
    var body: some View {
        if ARWorldTrackingConfiguration.isSupported {
            RealityView { content in
                // AR内容
            }
        } else {
            ContentUnavailableView(
                "不支持AR功能",
                systemImage: "arkit",
                description: Text("当前设备不支持AR功能。")
            )
        }
    }
}

DON'T: Load heavy models synchronously

不要同步加载大体积模型

Loading large USDZ files on the main thread causes frame drops and hangs. The

make

closure of

RealityView

async

-- use it.

swift

// WRONG -- synchronous load blocks the main thread
RealityView { content in
    let model = try! Entity.load(named: "large-scene")
    content.add(model)
}

// CORRECT -- async load
RealityView { content in
    if let model = try? await ModelEntity(named: "large-scene") {
        content.add(model)
    }
}

在主线程加载大体积USDZ文件会导致掉帧和卡顿。

RealityView

的

make

闭包是

async

的，请利用该特性异步加载。

swift

// 错误写法 -- 同步加载阻塞主线程
RealityView { content in
    let model = try! Entity.load(named: "large-scene")
    content.add(model)
}

// 正确写法 -- 异步加载
RealityView { content in
    if let model = try? await ModelEntity(named: "large-scene") {
        content.add(model)
    }
}

DON'T: Forget collision and input target components for interactive entities

不要忘记为可交互实体添加碰撞和输入目标组件

Gestures only work on entities that have both

CollisionComponent

and

InputTargetComponent

. Without them, taps and drags pass through.

swift

// WRONG -- entity ignores gestures
let box = ModelEntity(mesh: .generateBox(size: 0.1))
content.add(box)

// CORRECT -- add collision and input components
let box = ModelEntity(
    mesh: .generateBox(size: 0.1),
    materials: [SimpleMaterial(color: .red, isMetallic: false)]
)
box.components.set(CollisionComponent(
    shapes: [.generateBox(size: [0.1, 0.1, 0.1])]
))
box.components.set(InputTargetComponent())
content.add(box)

手势仅能作用于同时拥有

CollisionComponent

和

InputTargetComponent

的实体。缺少这两个组件时，点击和拖拽事件会直接穿透。

swift

// 错误写法 -- 实体无法响应手势
let box = ModelEntity(mesh: .generateBox(size: 0.1))
content.add(box)

// 正确写法 -- 添加碰撞和输入组件
let box = ModelEntity(
    mesh: .generateBox(size: 0.1),
    materials: [SimpleMaterial(color: .red, isMetallic: false)]
)
box.components.set(CollisionComponent(
    shapes: [.generateBox(size: [0.1, 0.1, 0.1])]
))
box.components.set(InputTargetComponent())
content.add(box)

DON'T: Create new entities in the update closure

不要在update闭包中创建新实体

The

update

closure runs on every SwiftUI state change. Creating entities there duplicates content on each render pass.

swift

// WRONG -- duplicates entities on every state change
RealityView { content in
    // empty
} update: { content in
    let sphere = ModelEntity(mesh: .generateSphere(radius: 0.05))
    content.add(sphere)  // Added again on every update
}

// CORRECT -- create in make, modify in update
RealityView { content in
    let sphere = ModelEntity(mesh: .generateSphere(radius: 0.05))
    sphere.name = "mySphere"
    content.add(sphere)
} update: { content in
    if let sphere = content.entities.first(
        where: { $0.name == "mySphere" }
    ) as? ModelEntity {
        // Modify existing entity
        sphere.position.y = newYPosition
    }
}

update

闭包会在每次SwiftUI状态变化时执行。在该位置创建实体会导致每次渲染都重复添加内容。

swift

// 错误写法 -- 每次状态变化都会重复添加实体
RealityView { content in
    // 空实现
} update: { content in
    let sphere = ModelEntity(mesh: .generateSphere(radius: 0.05))
    content.add(sphere)  // 每次update都会重复添加
}

// 正确写法 -- 在make中创建，在update中修改
RealityView { content in
    let sphere = ModelEntity(mesh: .generateSphere(radius: 0.05))
    sphere.name = "mySphere"
    content.add(sphere)
} update: { content in
    if let sphere = content.entities.first(
        where: { $0.name == "mySphere" }
    ) as? ModelEntity {
        // 修改已有实体
        sphere.position.y = newYPosition
    }
}

DON'T: Ignore camera permission

不要忽略相机权限

RealityKit on iOS needs camera access. If the user denies permission, the view shows a black screen with no explanation.

swift

// WRONG -- no permission handling
RealityView { content in
    // Black screen if camera denied
}

// CORRECT -- check and request permission
struct ARContainerView: View {
    @State private var cameraAuthorized = false

    var body: some View {
        Group {
            if cameraAuthorized {
                RealityView { content in
                    // AR content
                }
            } else {
                ContentUnavailableView(
                    "Camera Access Required",
                    systemImage: "camera.fill",
                    description: Text("Enable camera in Settings to use AR.")
                )
            }
        }
        .task {
            let status = AVCaptureDevice.authorizationStatus(for: .video)
            if status == .authorized {
                cameraAuthorized = true
            } else if status == .notDetermined {
                cameraAuthorized = await AVCaptureDevice
                    .requestAccess(for: .video)
            }
        }
    }
}

iOS上的RealityKit需要相机访问权限。如果用户拒绝权限，视图会展示黑屏且无任何说明。

swift

// 错误写法 -- 无权限处理
RealityView { content in
    // 相机权限被拒绝时会显示黑屏
}

// 正确写法 -- 检查并请求权限
struct ARContainerView: View {
    @State private var cameraAuthorized = false

    var body: some View {
        Group {
            if cameraAuthorized {
                RealityView { content in
                    // AR内容
                }
            } else {
                ContentUnavailableView(
                    "需要相机访问权限",
                    systemImage: "camera.fill",
                    description: Text("请在设置中开启相机权限以使用AR功能。")
                )
            }
        }
        .task {
            let status = AVCaptureDevice.authorizationStatus(for: .video)
            if status == .authorized {
                cameraAuthorized = true
            } else if status == .notDetermined {
                cameraAuthorized = await AVCaptureDevice
                    .requestAccess(for: .video)
            }
        }
    }
}

Review Checklist

审核检查清单

```
NSCameraUsageDescription
```
set in Info.plist
AR device capability checked before presenting AR views
Camera permission requested and denial handled with a fallback UI
3D models loaded asynchronously in the
```
make
```
closure
Entities created in
```
make
```
, modified in
```
update
```
(not created in
```
update
```
)
Interactive entities have both
```
CollisionComponent
```
and
```
InputTargetComponent
```
Collision shapes match the visual size of the entity
```
SceneEvents.Update
```
subscriptions used for per-frame logic (not SwiftUI timers)
Large scenes use
```
ModelEntity(named:)
```
async loading, not
```
Entity.load(named:)
```
Anchor entities target appropriate surface types for the use case
Entity names set for lookup in the
```
update
```
closure

Info.plist中已设置
```
NSCameraUsageDescription
```
展示AR视图前已完成AR设备能力检测
已请求相机权限，并对权限拒绝场景做了降级UI处理
3D模型在
```
make
```
闭包中异步加载
实体在
```
make
```
中创建，在
```
update
```
中修改（不在
```
update
```
中创建）
可交互实体同时拥有
```
CollisionComponent
```
和
```
InputTargetComponent
```
碰撞形状与实体的视觉尺寸匹配
逐帧逻辑使用
```
SceneEvents.Update
```
订阅实现（不使用SwiftUI定时器）
大场景使用
```
ModelEntity(named:)
```
异步加载，不使用
```
Entity.load(named:)
```
锚定实体适配业务场景的对应平面类型
实体已设置名称，方便在
```
update
```
闭包中查找

References

参考资料

Extended patterns (physics, animations, lighting, ECS): references/realitykit-patterns.md
RealityKit framework
RealityView
RealityViewCameraContent
Entity
ModelEntity
AnchorEntity
ARKit framework
ARKit in iOS
ARWorldTrackingConfiguration
Loading entities from a file

扩展模式（物理、动画、光照、ECS）: references/realitykit-patterns.md
RealityKit framework
RealityView
RealityViewCameraContent
Entity
ModelEntity
AnchorEntity
ARKit framework
ARKit in iOS
ARWorldTrackingConfiguration
Loading entities from a file

realitykit

Original

Translation

RealityKit

RealityKit

Contents

目录

Setup

环境配置

Project Configuration

项目配置

Device Requirements

设备要求

Key Types

核心类型

RealityView Basics

RealityView基础

Make and Update Pattern

创建与更新模式

Loading and Creating Entities

加载与创建实体

Loading from USDZ Files

从USDZ文件加载

Programmatic Mesh Generation

程序化生成网格

Adding Components

添加组件

Anchoring and Placement

锚定与放置

AnchorEntity

AnchorEntity

Anchor Targets

锚定目标

Raycasting

光线投射

Gestures and Interaction

手势与交互

Drag Gesture on Entities

实体拖拽手势

Tap to Select

点击选择

Scene Understanding

场景理解

Per-Frame Updates

逐帧更新

visionOS Note

visionOS注意事项

Common Mistakes

常见错误

DON'T: Skip AR capability checks

不要跳过AR能力检测

DON'T: Load heavy models synchronously

不要同步加载大体积模型

DON'T: Forget collision and input target components for interactive entities

不要忘记为可交互实体添加碰撞和输入目标组件

DON'T: Create new entities in the update closure

不要在update闭包中创建新实体

DON'T: Ignore camera permission

不要忽略相机权限

Review Checklist

审核检查清单

References

参考资料