dockkit
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseDockKit
DockKit
Framework for integrating with motorized camera stands and gimbals that
physically track subjects by rotating the iPhone. DockKit handles motor
control, subject detection, and framing so camera apps get 360-degree pan
and 90-degree tilt tracking with no additional code. Apps can override
system tracking to supply custom observations, control motors directly,
or adjust framing. iOS 17+, Swift 6.3.
用于集成电动相机支架和云台的框架,可通过旋转iPhone实现对主体的物理追踪。DockKit负责电机控制、主体检测和取景,因此相机应用无需额外代码即可实现360度平移和90度倾斜追踪。应用可以覆盖系统追踪逻辑,提供自定义观测结果、直接控制电机或调整取景。支持iOS 17+、Swift 6.3。
Contents
目录
Setup
安装设置
Import DockKit:
swift
import DockKitDockKit requires a physical DockKit-compatible accessory and a real device.
The Simulator cannot connect to dock hardware.
No special entitlements or Info.plist keys are required. The framework
communicates with paired accessories automatically through the DockKit
system daemon.
The app must use AVFoundation camera APIs. DockKit hooks into the camera
pipeline to analyze frames for system tracking.
导入DockKit:
swift
import DockKitDockKit需要物理上兼容DockKit的配件和真机设备,模拟器无法连接到底座硬件。
不需要特殊的权限或Info.plist配置项,该框架会通过DockKit系统守护进程自动与已配对的配件通信。
应用必须使用AVFoundation相机API,DockKit会接入相机管线来分析帧以实现系统追踪。
Discovering Accessories
发现配件
Use to observe dock connections:
DockAccessoryManager.sharedswift
import DockKit
func observeAccessories() async throws {
for await stateChange in try DockAccessoryManager.shared.accessoryStateChanges {
switch stateChange.state {
case .docked:
guard let accessory = stateChange.accessory else { continue }
// Accessory is connected and ready
configureAccessory(accessory)
case .undocked:
// iPhone removed from dock
handleUndocked()
@unknown default:
break
}
}
}accessoryStateChangesAsyncSequenceDockAccessory.StateChange- --
stateor.docked.undocked - -- the
accessoryinstance (present when docked)DockAccessory - -- whether the physical tracking button is active
trackingButtonEnabled
使用监听底座连接状态:
DockAccessoryManager.sharedswift
import DockKit
func observeAccessories() async throws {
for await stateChange in try DockAccessoryManager.shared.accessoryStateChanges {
switch stateChange.state {
case .docked:
guard let accessory = stateChange.accessory else { continue }
// 配件已连接并就绪
configureAccessory(accessory)
case .undocked:
// iPhone已从底座取出
handleUndocked()
@unknown default:
break
}
}
}accessoryStateChangesAsyncSequenceDockAccessory.StateChange- --
state(已连接)或.docked(已断开).undocked - --
accessory实例(连接状态下存在)DockAccessory - -- 物理追踪按钮是否处于激活状态
trackingButtonEnabled
Accessory Identity
配件标识
Each has an with:
DockAccessoryidentifier- -- human-readable accessory name
name - --
category(currently the only category).trackingStand - -- unique identifier for this accessory
uuid
Hardware details are available via and .
firmwareVersionhardwareModel每个都有一个,包含:
DockAccessoryidentifier- -- 人类可读的配件名称
name - --
category(目前唯一的分类).trackingStand - -- 该配件的唯一标识符
uuid
可通过和获取硬件详情。
firmwareVersionhardwareModelSystem Tracking
系统追踪
System tracking is DockKit's default mode. When enabled, the system
analyzes camera frames through built-in ML inference, detects faces and
bodies, and drives the motors to keep subjects in frame. Any app using
AVFoundation camera APIs benefits automatically.
系统追踪是DockKit的默认模式。启用后,系统会通过内置ML推理分析相机帧,检测人脸和人体,并驱动电机将主体保持在画面内。任何使用AVFoundation相机API的应用都可以自动受益。
Enable or Disable
启用或禁用
swift
// Enable system tracking (default)
try await DockAccessoryManager.shared.setSystemTrackingEnabled(true)
// Disable system tracking for custom control
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)Check current state:
swift
let isEnabled = DockAccessoryManager.shared.isSystemTrackingEnabledSystem tracking state does not persist across app termination, reboots,
or background/foreground transitions. Set it explicitly whenever the app
needs a specific value.
swift
// 启用系统追踪(默认)
try await DockAccessoryManager.shared.setSystemTrackingEnabled(true)
// 禁用系统追踪以进行自定义控制
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)检查当前状态:
swift
let isEnabled = DockAccessoryManager.shared.isSystemTrackingEnabled系统追踪状态不会在应用终止、设备重启或前后台切换时持久化,每当应用需要特定状态时请显式设置。
Tap to Select Subject
点击选择主体
Allow users to select a specific subject by tapping:
swift
// Select the subject at a screen coordinate
try await accessory.selectSubject(at: CGPoint(x: 0.5, y: 0.5))
// Select specific subjects by identifier
try await accessory.selectSubjects([subjectUUID])
// Clear selection (return to automatic selection)
try await accessory.selectSubjects([])允许用户通过点击选择特定的追踪主体:
swift
// 选择屏幕坐标对应的主体
try await accessory.selectSubject(at: CGPoint(x: 0.5, y: 0.5))
// 通过标识符选择特定主体
try await accessory.selectSubjects([subjectUUID])
// 清除选择(回到自动选择模式)
try await accessory.selectSubjects([])Custom Tracking
自定义追踪
Disable system tracking and provide your own observations when using
custom ML models or the Vision framework.
当使用自定义ML模型或Vision框架时,可以禁用系统追踪并提供你自己的观测结果。
Providing Observations
提供观测结果
Construct values from your inference output
and pass them to the accessory at 10-30 fps:
DockAccessory.Observationswift
import DockKit
import AVFoundation
func processFrame(
_ sampleBuffer: CMSampleBuffer,
accessory: DockAccessory,
device: AVCaptureDevice
) async throws {
let cameraInfo = DockAccessory.CameraInformation(
captureDevice: device.deviceType,
cameraPosition: device.position,
orientation: .corrected,
cameraIntrinsics: nil,
referenceDimensions: nil
)
// Create observation from your detection output
let observation = DockAccessory.Observation(
identifier: 0,
type: .humanFace,
rect: detectedFaceRect, // CGRect in normalized coordinates
faceYawAngle: nil
)
try await accessory.track(
[observation],
cameraInformation: cameraInfo
)
}从你的推理输出中构造值,以10-30fps的频率传递给配件:
DockAccessory.Observationswift
import DockKit
import AVFoundation
func processFrame(
_ sampleBuffer: CMSampleBuffer,
accessory: DockAccessory,
device: AVCaptureDevice
) async throws {
let cameraInfo = DockAccessory.CameraInformation(
captureDevice: device.deviceType,
cameraPosition: device.position,
orientation: .corrected,
cameraIntrinsics: nil,
referenceDimensions: nil
)
// 从你的检测输出创建观测结果
let observation = DockAccessory.Observation(
identifier: 0,
type: .humanFace,
rect: detectedFaceRect, // 归一化坐标下的CGRect
faceYawAngle: nil
)
try await accessory.track(
[observation],
cameraInformation: cameraInfo
)
}Observation Types
观测类型
| Type | Use |
|---|---|
| Preserves system multi-person tracking and framing optimizations |
| Full body tracking |
| Arbitrary objects (pets, hands, barcodes, etc.) |
The uses normalized coordinates with a lower-left origin (same
coordinate system as Vision framework -- no conversion needed).
rect| 类型 | 用途 |
|---|---|
| 保留系统多人追踪和取景优化能力 |
| 全身追踪 |
| 任意物体(宠物、手、条形码等) |
rectCamera Information
相机信息
DockAccessory.CameraInformation.correctedcameraIntrinsicsreferenceDimensionsTrack variants also accept instead of observations,
and an optional for enhanced tracking.
[AVMetadataObject]CVPixelBufferDockAccessory.CameraInformation.correctedcameraIntrinsicsreferenceDimensionstrack方法也支持接收而不是观测结果,以及可选的来增强追踪效果。
[AVMetadataObject]CVPixelBufferFraming and Region of Interest
取景与感兴趣区域
Framing Modes
取景模式
Control how the system frames tracked subjects:
swift
try await accessory.setFramingMode(.center)| Mode | Behavior |
|---|---|
| System decides optimal framing |
| Keep subject centered (default) |
| Frame subject in left third |
| Frame subject in right third |
Read the current mode:
swift
let currentMode = accessory.framingModeUse or when graphic overlays occupy part of the frame.
.left.right控制系统对追踪主体的取景方式:
swift
try await accessory.setFramingMode(.center)| 模式 | 行为 |
|---|---|
| 系统决定最优取景 |
| 保持主体居中(默认) |
| 将主体放在画面左侧三分之一区域 |
| 将主体放在画面右侧三分之一区域 |
读取当前模式:
swift
let currentMode = accessory.framingMode当图形叠加层占用部分画面时可以使用或模式。
.left.rightRegion of Interest
感兴趣区域
Constrain tracking to a specific area of the video frame:
swift
// Normalized coordinates, origin at upper-left
let squareRegion = CGRect(x: 0.25, y: 0.0, width: 0.5, height: 1.0)
try await accessory.setRegionOfInterest(squareRegion)Read the current region:
swift
let currentROI = accessory.regionOfInterestUse region of interest when cropping to a non-standard aspect ratio
(e.g., square video for conferencing) so subjects stay within the
visible area.
将追踪限制在视频帧的特定区域内:
swift
// 归一化坐标,原点在左上角
let squareRegion = CGRect(x: 0.25, y: 0.0, width: 0.5, height: 1.0)
try await accessory.setRegionOfInterest(squareRegion)读取当前区域:
swift
let currentROI = accessory.regionOfInterest当裁剪为非标准长宽比时(比如会议用的方形视频)可以使用感兴趣区域,保证主体始终在可见区域内。
Motor Control
电机控制
Disable system tracking before controlling motors directly.
直接控制电机前请先禁用系统追踪。
Angular Velocity
角速度
Set continuous rotation speed in radians per second:
swift
import Spatial
// Pan right at 0.2 rad/s, tilt down at 0.1 rad/s
let velocity = Vector3D(x: 0.1, y: 0.2, z: 0.0)
try await accessory.setAngularVelocity(velocity)
// Stop all motion
try await accessory.setAngularVelocity(Vector3D())Axes:
- -- pitch (tilt). Positive tilts down on iOS.
x - -- yaw (pan). Positive pans right.
y - -- roll (if supported by hardware).
z
设置持续旋转速度,单位为弧度/秒:
swift
import Spatial
// 向右平移0.2 rad/s,向下倾斜0.1 rad/s
let velocity = Vector3D(x: 0.1, y: 0.2, z: 0.0)
try await accessory.setAngularVelocity(velocity)
// 停止所有运动
try await accessory.setAngularVelocity(Vector3D())坐标轴说明:
- -- 俯仰(倾斜),iOS上正值代表向下倾斜
x - -- 偏航(平移),正值代表向右平移
y - -- 滚动(如果硬件支持)
z
Set Orientation
设置朝向
Move to a specific position over a duration:
swift
let target = Vector3D(x: 0.0, y: 0.5, z: 0.0) // Yaw 0.5 rad
let progress = try accessory.setOrientation(
target,
duration: .seconds(2),
relative: false
)Also accepts for quaternion-based orientation. Set
to move relative to the current position. The returned
object tracks completion.
Rotation3Drelative: trueProgress在指定时长内移动到特定位置:
swift
let target = Vector3D(x: 0.0, y: 0.5, z: 0.0) // 偏航0.5弧度
let progress = try accessory.setOrientation(
target,
duration: .seconds(2),
relative: false
)也支持接收来设置基于四元数的朝向。设置可以相对于当前位置移动,返回的对象可以追踪完成进度。
Rotation3Drelative: trueProgressMotion State
运动状态
Monitor the accessory's current position and velocity:
swift
for await state in accessory.motionStates {
let positions = state.angularPositions // Vector3D
let velocities = state.angularVelocities // Vector3D
let time = state.timestamp
if let error = state.error {
// Motor error occurred
}
}监听配件当前的位置和速度:
swift
for await state in accessory.motionStates {
let positions = state.angularPositions // Vector3D
let velocities = state.angularVelocities // Vector3D
let time = state.timestamp
if let error = state.error {
// 发生电机错误
}
}Setting Limits
设置限制
Restrict range of motion and maximum speed per axis:
swift
let yawLimit = try DockAccessory.Limits.Limit(
positionRange: -1.0 ..< 1.0, // radians
maximumSpeed: 0.5 // rad/s
)
let limits = DockAccessory.Limits(yaw: yawLimit, pitch: nil, roll: nil)
try accessory.setLimits(limits)限制每个轴的运动范围和最大速度:
swift
let yawLimit = try DockAccessory.Limits.Limit(
positionRange: -1.0 ..< 1.0, // 弧度
maximumSpeed: 0.5 // 弧度/秒
)
let limits = DockAccessory.Limits(yaw: yawLimit, pitch: nil, roll: nil)
try accessory.setLimits(limits)Animations
动画
Built-in character animations that move the dock expressively:
swift
// Disable system tracking before animating
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
let progress = try await accessory.animate(motion: .kapow)
// Wait for completion
while !progress.isFinished && !progress.isCancelled {
try await Task.sleep(for: .milliseconds(100))
}
// Restore system tracking
try await DockAccessoryManager.shared.setSystemTrackingEnabled(true)| Animation | Effect |
|---|---|
| Nodding motion |
| Shaking motion |
| Startup-style motion |
| Dramatic pendulum swing |
Animations start from the accessory's current position and execute
asynchronously. Always restore tracking state after completion.
内置的角色动画,可以让底座做出富有表现力的运动:
swift
// 动画前先禁用系统追踪
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
let progress = try await accessory.animate(motion: .kapow)
// 等待动画完成
while !progress.isFinished && !progress.isCancelled {
try await Task.sleep(for: .milliseconds(100))
}
// 恢复系统追踪
try await DockAccessoryManager.shared.setSystemTrackingEnabled(true)| 动画 | 效果 |
|---|---|
| 点头动作 |
| 摇头动作 |
| 启动风格的动作 |
| 夸张的钟摆摆动 |
动画从配件的当前位置开始异步执行,完成后请务必恢复追踪状态。
Tracking State and Subject Selection
追踪状态与主体选择
iOS 18+ exposes ML-derived tracking signals through .
Each contains a timestamp and a list of
values ( or ). Persons include
, , and
(1 = most important, lower is more salient).
trackingStatesTrackingStateTrackedSubjectType.person.objectspeakingConfidencelookingAtCameraConfidencesaliencyRankswift
for await state in accessory.trackingStates {
for subject in state.trackedSubjects {
switch subject {
case .person(let person):
let speaking = person.speakingConfidence // 0.0 - 1.0
let saliency = person.saliencyRank
case .object(let object):
let saliency = object.saliencyRank
}
}
}Use to lock tracking onto specific subjects by UUID.
Pass an empty array to return to automatic selection. See
references/dockkit-patterns.md for speaker-tracking and saliency
filtering recipes.
selectSubjects(_:)iOS 18+通过暴露ML生成的追踪信号。每个包含时间戳和值列表(或)。人物信息包含(说话置信度)、(看向相机置信度)和(显著性排名,1=最重要,数值越低越显著)。
trackingStatesTrackingStateTrackedSubjectType.person.objectspeakingConfidencelookingAtCameraConfidencesaliencyRankswift
for await state in accessory.trackingStates {
for subject in state.trackedSubjects {
switch subject {
case .person(let person):
let speaking = person.speakingConfidence // 0.0 - 1.0
let saliency = person.saliencyRank
case .object(let object):
let saliency = object.saliencyRank
}
}
}使用可以通过UUID锁定特定主体的追踪,传入空数组可以回到自动选择模式。查看references/dockkit-patterns.md获取说话人追踪和显著性过滤的实现方案。
selectSubjects(_:)Accessory Events
配件事件
Physical buttons on the dock trigger events:
swift
for await event in accessory.accessoryEvents {
switch event {
case .cameraShutter:
// Start or stop recording / take photo
break
case .cameraFlip:
// Switch front/back camera
break
case .cameraZoom(factor: let factor):
// factor > 0 means zoom in, < 0 means zoom out
break
case .button(id: let id, pressed: let pressed):
// Custom button with identifier
break
@unknown default:
break
}
}For first-party apps (Camera, FaceTime), shutter/flip/zoom events work
automatically. Third-party apps receive these events and implement
behavior through AVFoundation.
底座上的物理按钮会触发事件:
swift
for await event in accessory.accessoryEvents {
switch event {
case .cameraShutter:
// 开始/停止录制 / 拍照
break
case .cameraFlip:
// 切换前后摄像头
break
case .cameraZoom(factor: let factor):
// factor > 0 代表放大,< 0 代表缩小
break
case .button(id: let id, pressed: let pressed):
// 带标识符的自定义按钮
break
@unknown default:
break
}
}对于第一方应用(相机、FaceTime),快门/切换/缩放事件会自动生效,第三方应用需要接收这些事件并通过AVFoundation实现对应行为。
Battery Monitoring
电量监控
Monitor the dock's battery status (iOS 18+). A dock can report multiple
batteries, each identified by :
nameswift
for await battery in accessory.batteryStates {
let level = battery.batteryLevel // 0.0 - 1.0
let charging = battery.chargeState // .charging, .notCharging, .notChargeable
let low = battery.lowBattery
}监听底座的电量状态(iOS 18+)。一个底座可能会报告多块电池,每块电池通过标识:
nameswift
for await battery in accessory.batteryStates {
let level = battery.batteryLevel // 0.0 - 1.0
let charging = battery.chargeState // .charging(充电中), .notCharging(未充电), .notChargeable(不可充电)
let low = battery.lowBattery
}Common Mistakes
常见错误
DON'T: Control motors without disabling system tracking
不要:未禁用系统追踪就控制电机
swift
// WRONG -- system tracking fights manual commands
try await accessory.setAngularVelocity(velocity)
// CORRECT -- disable system tracking first
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
try await accessory.setAngularVelocity(velocity)swift
// 错误 -- 系统追踪会和手动命令冲突
try await accessory.setAngularVelocity(velocity)
// 正确 -- 先禁用系统追踪
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
try await accessory.setAngularVelocity(velocity)DON'T: Assume tracking state persists across lifecycle events
不要:假设追踪状态会在生命周期事件中持久化
swift
// WRONG -- state may have reset after backgrounding
func applicationDidBecomeActive() {
// Assume custom tracking is still active
}
// CORRECT -- re-set tracking state on foreground
func applicationDidBecomeActive() {
Task {
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
}
}swift
// 错误 -- 应用回到后台后状态可能已重置
func applicationDidBecomeActive() {
// 假设自定义追踪仍处于激活状态
}
// 正确 -- 应用回到前台时重新设置追踪状态
func applicationDidBecomeActive() {
Task {
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
}
}DON'T: Call track() outside the recommended rate
不要:在推荐频率外调用track()
swift
// WRONG -- calling once per second is too slow
try await accessory.track(observations, cameraInformation: cameraInfo)
// (called at 1 fps)
// CORRECT -- call at 10-30 fps
// Hook into AVCaptureVideoDataOutputSampleBufferDelegate for per-frame callsswift
// 错误 -- 每秒调用一次太慢
try await accessory.track(observations, cameraInformation: cameraInfo)
// (调用频率为1 fps)
// 正确 -- 以10-30fps的频率调用
// 接入AVCaptureVideoDataOutputSampleBufferDelegate实现逐帧调用DON'T: Forget to restore tracking after animations
不要:动画完成后忘记恢复追踪
swift
// WRONG -- tracking stays disabled after animation
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
let progress = try await accessory.animate(motion: .kapow)
// CORRECT -- restore tracking when animation completes
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
let progress = try await accessory.animate(motion: .kapow)
while !progress.isFinished && !progress.isCancelled {
try await Task.sleep(for: .milliseconds(100))
}
try await DockAccessoryManager.shared.setSystemTrackingEnabled(true)swift
// 错误 -- 动画结束后追踪仍处于禁用状态
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
let progress = try await accessory.animate(motion: .kapow)
// 正确 -- 动画完成后恢复追踪
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
let progress = try await accessory.animate(motion: .kapow)
while !progress.isFinished && !progress.isCancelled {
try await Task.sleep(for: .milliseconds(100))
}
try await DockAccessoryManager.shared.setSystemTrackingEnabled(true)DON'T: Use DockKit in Simulator
不要:在模拟器中使用DockKit
DockKit requires a physical DockKit-compatible accessory. Guard
initialization and provide fallback behavior when no accessory is
available.
DockKit需要物理上兼容DockKit的配件,当初始化时没有可用配件请提供降级逻辑。
Review Checklist
检查清单
- present where needed
import DockKit - Subscribed to to detect dock/undock events
accessoryStateChanges - Handled both and
.dockedstates.undocked - System tracking disabled before custom tracking or motor control
- System tracking restored after animations complete
- Custom observations supplied at 10-30 fps
- Observation uses normalized coordinates (lower-left origin)
rect - Camera information matches the active capture device
- handled in all switch statements over DockKit enums
@unknown default - Motion limits set if restricting accessory range of motion
- Tracking state re-applied after app returns to foreground
- Accessory events handled for physical button integration
- Battery state monitored when showing dock status to user
- No DockKit code paths executed in Simulator builds
- 在需要的地方引入
import DockKit - 订阅检测连接/断开事件
accessoryStateChanges - 同时处理和
.docked两种状态.undocked - 自定义追踪或电机控制前已禁用系统追踪
- 动画完成后已恢复系统追踪
- 自定义观测结果的提供频率为10-30fps
- 观测结果的使用归一化坐标(原点在左下角)
rect - 相机信息与当前激活的采集设备匹配
- 所有遍历DockKit枚举的switch语句都处理了
@unknown default - 如果需要限制配件运动范围已设置运动限制
- 应用回到前台后已重新应用追踪状态
- 已处理物理按钮对应的配件事件
- 向用户展示底座状态时已监听电量状态
- 模拟器构建版本中不会执行DockKit相关代码路径
References
参考资料
- Extended patterns (Vision integration, service architecture, custom animations): references/dockkit-patterns.md
- DockKit framework
- DockAccessoryManager
- DockAccessory
- Controlling a DockKit accessory using your camera app
- Track custom objects in a frame
- Modify rotation and positioning programmatically
- Integrate with motorized iPhone stands using DockKit -- WWDC23
- What's new in DockKit -- WWDC24
- 扩展方案(Vision集成、服务架构、自定义动画):references/dockkit-patterns.md
- DockKit framework
- DockAccessoryManager
- DockAccessory
- Controlling a DockKit accessory using your camera app
- Track custom objects in a frame
- Modify rotation and positioning programmatically
- Integrate with motorized iPhone stands using DockKit -- WWDC23
- What's new in DockKit -- WWDC24