dockkit

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

DockKit

DockKit

Framework for integrating with motorized camera stands and gimbals that physically track subjects by rotating the iPhone. DockKit handles motor control, subject detection, and framing so camera apps get 360-degree pan and 90-degree tilt tracking with no additional code. Apps can override system tracking to supply custom observations, control motors directly, or adjust framing. iOS 17+, Swift 6.3.
用于集成电动相机支架和云台的框架,可通过旋转iPhone实现对主体的物理追踪。DockKit负责电机控制、主体检测和取景,因此相机应用无需额外代码即可实现360度平移和90度倾斜追踪。应用可以覆盖系统追踪逻辑,提供自定义观测结果、直接控制电机或调整取景。支持iOS 17+、Swift 6.3。

Contents

目录

Setup

安装设置

Import DockKit:
swift
import DockKit
DockKit requires a physical DockKit-compatible accessory and a real device. The Simulator cannot connect to dock hardware.
No special entitlements or Info.plist keys are required. The framework communicates with paired accessories automatically through the DockKit system daemon.
The app must use AVFoundation camera APIs. DockKit hooks into the camera pipeline to analyze frames for system tracking.
导入DockKit:
swift
import DockKit
DockKit需要物理上兼容DockKit的配件和真机设备,模拟器无法连接到底座硬件。
不需要特殊的权限或Info.plist配置项,该框架会通过DockKit系统守护进程自动与已配对的配件通信。
应用必须使用AVFoundation相机API,DockKit会接入相机管线来分析帧以实现系统追踪。

Discovering Accessories

发现配件

Use
DockAccessoryManager.shared
to observe dock connections:
swift
import DockKit

func observeAccessories() async throws {
    for await stateChange in try DockAccessoryManager.shared.accessoryStateChanges {
        switch stateChange.state {
        case .docked:
            guard let accessory = stateChange.accessory else { continue }
            // Accessory is connected and ready
            configureAccessory(accessory)
        case .undocked:
            // iPhone removed from dock
            handleUndocked()
        @unknown default:
            break
        }
    }
}
accessoryStateChanges
is an
AsyncSequence
that emits
DockAccessory.StateChange
values. Each change includes:
  • state
    --
    .docked
    or
    .undocked
  • accessory
    -- the
    DockAccessory
    instance (present when docked)
  • trackingButtonEnabled
    -- whether the physical tracking button is active
使用
DockAccessoryManager.shared
监听底座连接状态:
swift
import DockKit

func observeAccessories() async throws {
    for await stateChange in try DockAccessoryManager.shared.accessoryStateChanges {
        switch stateChange.state {
        case .docked:
            guard let accessory = stateChange.accessory else { continue }
            // 配件已连接并就绪
            configureAccessory(accessory)
        case .undocked:
            // iPhone已从底座取出
            handleUndocked()
        @unknown default:
            break
        }
    }
}
accessoryStateChanges
是一个
AsyncSequence
,会输出
DockAccessory.StateChange
值,每次状态变更包含:
  • state
    --
    .docked
    (已连接)或
    .undocked
    (已断开)
  • accessory
    --
    DockAccessory
    实例(连接状态下存在)
  • trackingButtonEnabled
    -- 物理追踪按钮是否处于激活状态

Accessory Identity

配件标识

Each
DockAccessory
has an
identifier
with:
  • name
    -- human-readable accessory name
  • category
    --
    .trackingStand
    (currently the only category)
  • uuid
    -- unique identifier for this accessory
Hardware details are available via
firmwareVersion
and
hardwareModel
.
每个
DockAccessory
都有一个
identifier
,包含:
  • name
    -- 人类可读的配件名称
  • category
    --
    .trackingStand
    (目前唯一的分类)
  • uuid
    -- 该配件的唯一标识符
可通过
firmwareVersion
hardwareModel
获取硬件详情。

System Tracking

系统追踪

System tracking is DockKit's default mode. When enabled, the system analyzes camera frames through built-in ML inference, detects faces and bodies, and drives the motors to keep subjects in frame. Any app using AVFoundation camera APIs benefits automatically.
系统追踪是DockKit的默认模式。启用后,系统会通过内置ML推理分析相机帧,检测人脸和人体,并驱动电机将主体保持在画面内。任何使用AVFoundation相机API的应用都可以自动受益。

Enable or Disable

启用或禁用

swift
// Enable system tracking (default)
try await DockAccessoryManager.shared.setSystemTrackingEnabled(true)

// Disable system tracking for custom control
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
Check current state:
swift
let isEnabled = DockAccessoryManager.shared.isSystemTrackingEnabled
System tracking state does not persist across app termination, reboots, or background/foreground transitions. Set it explicitly whenever the app needs a specific value.
swift
// 启用系统追踪(默认)
try await DockAccessoryManager.shared.setSystemTrackingEnabled(true)

// 禁用系统追踪以进行自定义控制
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
检查当前状态:
swift
let isEnabled = DockAccessoryManager.shared.isSystemTrackingEnabled
系统追踪状态不会在应用终止、设备重启或前后台切换时持久化,每当应用需要特定状态时请显式设置。

Tap to Select Subject

点击选择主体

Allow users to select a specific subject by tapping:
swift
// Select the subject at a screen coordinate
try await accessory.selectSubject(at: CGPoint(x: 0.5, y: 0.5))

// Select specific subjects by identifier
try await accessory.selectSubjects([subjectUUID])

// Clear selection (return to automatic selection)
try await accessory.selectSubjects([])
允许用户通过点击选择特定的追踪主体:
swift
// 选择屏幕坐标对应的主体
try await accessory.selectSubject(at: CGPoint(x: 0.5, y: 0.5))

// 通过标识符选择特定主体
try await accessory.selectSubjects([subjectUUID])

// 清除选择(回到自动选择模式)
try await accessory.selectSubjects([])

Custom Tracking

自定义追踪

Disable system tracking and provide your own observations when using custom ML models or the Vision framework.
当使用自定义ML模型或Vision框架时,可以禁用系统追踪并提供你自己的观测结果。

Providing Observations

提供观测结果

Construct
DockAccessory.Observation
values from your inference output and pass them to the accessory at 10-30 fps:
swift
import DockKit
import AVFoundation

func processFrame(
    _ sampleBuffer: CMSampleBuffer,
    accessory: DockAccessory,
    device: AVCaptureDevice
) async throws {
    let cameraInfo = DockAccessory.CameraInformation(
        captureDevice: device.deviceType,
        cameraPosition: device.position,
        orientation: .corrected,
        cameraIntrinsics: nil,
        referenceDimensions: nil
    )

    // Create observation from your detection output
    let observation = DockAccessory.Observation(
        identifier: 0,
        type: .humanFace,
        rect: detectedFaceRect,       // CGRect in normalized coordinates
        faceYawAngle: nil
    )

    try await accessory.track(
        [observation],
        cameraInformation: cameraInfo
    )
}
从你的推理输出中构造
DockAccessory.Observation
值,以10-30fps的频率传递给配件:
swift
import DockKit
import AVFoundation

func processFrame(
    _ sampleBuffer: CMSampleBuffer,
    accessory: DockAccessory,
    device: AVCaptureDevice
) async throws {
    let cameraInfo = DockAccessory.CameraInformation(
        captureDevice: device.deviceType,
        cameraPosition: device.position,
        orientation: .corrected,
        cameraIntrinsics: nil,
        referenceDimensions: nil
    )

    // 从你的检测输出创建观测结果
    let observation = DockAccessory.Observation(
        identifier: 0,
        type: .humanFace,
        rect: detectedFaceRect,       // 归一化坐标下的CGRect
        faceYawAngle: nil
    )

    try await accessory.track(
        [observation],
        cameraInformation: cameraInfo
    )
}

Observation Types

观测类型

TypeUse
.humanFace
Preserves system multi-person tracking and framing optimizations
.humanBody
Full body tracking
.object
Arbitrary objects (pets, hands, barcodes, etc.)
The
rect
uses normalized coordinates with a lower-left origin (same coordinate system as Vision framework -- no conversion needed).
类型用途
.humanFace
保留系统多人追踪和取景优化能力
.humanBody
全身追踪
.object
任意物体(宠物、手、条形码等)
rect
使用原点在左下角的归一化坐标(和Vision框架的坐标系一致,无需转换)。

Camera Information

相机信息

DockAccessory.CameraInformation
describes the active camera. Set orientation to
.corrected
when coordinates are already relative to the bottom-left corner of the screen. Optional
cameraIntrinsics
and
referenceDimensions
improve tracking accuracy.
Track variants also accept
[AVMetadataObject]
instead of observations, and an optional
CVPixelBuffer
for enhanced tracking.
DockAccessory.CameraInformation
描述当前激活的相机。当坐标已经相对于屏幕左下角时,将orientation设置为
.corrected
。可选的
cameraIntrinsics
referenceDimensions
可以提升追踪精度。
track方法也支持接收
[AVMetadataObject]
而不是观测结果,以及可选的
CVPixelBuffer
来增强追踪效果。

Framing and Region of Interest

取景与感兴趣区域

Framing Modes

取景模式

Control how the system frames tracked subjects:
swift
try await accessory.setFramingMode(.center)
ModeBehavior
.automatic
System decides optimal framing
.center
Keep subject centered (default)
.left
Frame subject in left third
.right
Frame subject in right third
Read the current mode:
swift
let currentMode = accessory.framingMode
Use
.left
or
.right
when graphic overlays occupy part of the frame.
控制系统对追踪主体的取景方式:
swift
try await accessory.setFramingMode(.center)
模式行为
.automatic
系统决定最优取景
.center
保持主体居中(默认)
.left
将主体放在画面左侧三分之一区域
.right
将主体放在画面右侧三分之一区域
读取当前模式:
swift
let currentMode = accessory.framingMode
当图形叠加层占用部分画面时可以使用
.left
.right
模式。

Region of Interest

感兴趣区域

Constrain tracking to a specific area of the video frame:
swift
// Normalized coordinates, origin at upper-left
let squareRegion = CGRect(x: 0.25, y: 0.0, width: 0.5, height: 1.0)
try await accessory.setRegionOfInterest(squareRegion)
Read the current region:
swift
let currentROI = accessory.regionOfInterest
Use region of interest when cropping to a non-standard aspect ratio (e.g., square video for conferencing) so subjects stay within the visible area.
将追踪限制在视频帧的特定区域内:
swift
// 归一化坐标,原点在左上角
let squareRegion = CGRect(x: 0.25, y: 0.0, width: 0.5, height: 1.0)
try await accessory.setRegionOfInterest(squareRegion)
读取当前区域:
swift
let currentROI = accessory.regionOfInterest
当裁剪为非标准长宽比时(比如会议用的方形视频)可以使用感兴趣区域,保证主体始终在可见区域内。

Motor Control

电机控制

Disable system tracking before controlling motors directly.
直接控制电机前请先禁用系统追踪。

Angular Velocity

角速度

Set continuous rotation speed in radians per second:
swift
import Spatial

// Pan right at 0.2 rad/s, tilt down at 0.1 rad/s
let velocity = Vector3D(x: 0.1, y: 0.2, z: 0.0)
try await accessory.setAngularVelocity(velocity)

// Stop all motion
try await accessory.setAngularVelocity(Vector3D())
Axes:
  • x
    -- pitch (tilt). Positive tilts down on iOS.
  • y
    -- yaw (pan). Positive pans right.
  • z
    -- roll (if supported by hardware).
设置持续旋转速度,单位为弧度/秒:
swift
import Spatial

// 向右平移0.2 rad/s,向下倾斜0.1 rad/s
let velocity = Vector3D(x: 0.1, y: 0.2, z: 0.0)
try await accessory.setAngularVelocity(velocity)

// 停止所有运动
try await accessory.setAngularVelocity(Vector3D())
坐标轴说明:
  • x
    -- 俯仰(倾斜),iOS上正值代表向下倾斜
  • y
    -- 偏航(平移),正值代表向右平移
  • z
    -- 滚动(如果硬件支持)

Set Orientation

设置朝向

Move to a specific position over a duration:
swift
let target = Vector3D(x: 0.0, y: 0.5, z: 0.0)  // Yaw 0.5 rad
let progress = try accessory.setOrientation(
    target,
    duration: .seconds(2),
    relative: false
)
Also accepts
Rotation3D
for quaternion-based orientation. Set
relative: true
to move relative to the current position. The returned
Progress
object tracks completion.
在指定时长内移动到特定位置:
swift
let target = Vector3D(x: 0.0, y: 0.5, z: 0.0)  // 偏航0.5弧度
let progress = try accessory.setOrientation(
    target,
    duration: .seconds(2),
    relative: false
)
也支持接收
Rotation3D
来设置基于四元数的朝向。设置
relative: true
可以相对于当前位置移动,返回的
Progress
对象可以追踪完成进度。

Motion State

运动状态

Monitor the accessory's current position and velocity:
swift
for await state in accessory.motionStates {
    let positions = state.angularPositions   // Vector3D
    let velocities = state.angularVelocities // Vector3D
    let time = state.timestamp
    if let error = state.error {
        // Motor error occurred
    }
}
监听配件当前的位置和速度:
swift
for await state in accessory.motionStates {
    let positions = state.angularPositions   // Vector3D
    let velocities = state.angularVelocities // Vector3D
    let time = state.timestamp
    if let error = state.error {
        // 发生电机错误
    }
}

Setting Limits

设置限制

Restrict range of motion and maximum speed per axis:
swift
let yawLimit = try DockAccessory.Limits.Limit(
    positionRange: -1.0 ..< 1.0,   // radians
    maximumSpeed: 0.5               // rad/s
)
let limits = DockAccessory.Limits(yaw: yawLimit, pitch: nil, roll: nil)
try accessory.setLimits(limits)
限制每个轴的运动范围和最大速度:
swift
let yawLimit = try DockAccessory.Limits.Limit(
    positionRange: -1.0 ..< 1.0,   // 弧度
    maximumSpeed: 0.5               // 弧度/秒
)
let limits = DockAccessory.Limits(yaw: yawLimit, pitch: nil, roll: nil)
try accessory.setLimits(limits)

Animations

动画

Built-in character animations that move the dock expressively:
swift
// Disable system tracking before animating
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)

let progress = try await accessory.animate(motion: .kapow)

// Wait for completion
while !progress.isFinished && !progress.isCancelled {
    try await Task.sleep(for: .milliseconds(100))
}

// Restore system tracking
try await DockAccessoryManager.shared.setSystemTrackingEnabled(true)
AnimationEffect
.yes
Nodding motion
.no
Shaking motion
.wakeup
Startup-style motion
.kapow
Dramatic pendulum swing
Animations start from the accessory's current position and execute asynchronously. Always restore tracking state after completion.
内置的角色动画,可以让底座做出富有表现力的运动:
swift
// 动画前先禁用系统追踪
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)

let progress = try await accessory.animate(motion: .kapow)

// 等待动画完成
while !progress.isFinished && !progress.isCancelled {
    try await Task.sleep(for: .milliseconds(100))
}

// 恢复系统追踪
try await DockAccessoryManager.shared.setSystemTrackingEnabled(true)
动画效果
.yes
点头动作
.no
摇头动作
.wakeup
启动风格的动作
.kapow
夸张的钟摆摆动
动画从配件的当前位置开始异步执行,完成后请务必恢复追踪状态。

Tracking State and Subject Selection

追踪状态与主体选择

iOS 18+ exposes ML-derived tracking signals through
trackingStates
. Each
TrackingState
contains a timestamp and a list of
TrackedSubjectType
values (
.person
or
.object
). Persons include
speakingConfidence
,
lookingAtCameraConfidence
, and
saliencyRank
(1 = most important, lower is more salient).
swift
for await state in accessory.trackingStates {
    for subject in state.trackedSubjects {
        switch subject {
        case .person(let person):
            let speaking = person.speakingConfidence   // 0.0 - 1.0
            let saliency = person.saliencyRank
        case .object(let object):
            let saliency = object.saliencyRank
        }
    }
}
Use
selectSubjects(_:)
to lock tracking onto specific subjects by UUID. Pass an empty array to return to automatic selection. See references/dockkit-patterns.md for speaker-tracking and saliency filtering recipes.
iOS 18+通过
trackingStates
暴露ML生成的追踪信号。每个
TrackingState
包含时间戳和
TrackedSubjectType
值列表(
.person
.object
)。人物信息包含
speakingConfidence
(说话置信度)、
lookingAtCameraConfidence
(看向相机置信度)和
saliencyRank
(显著性排名,1=最重要,数值越低越显著)。
swift
for await state in accessory.trackingStates {
    for subject in state.trackedSubjects {
        switch subject {
        case .person(let person):
            let speaking = person.speakingConfidence   // 0.0 - 1.0
            let saliency = person.saliencyRank
        case .object(let object):
            let saliency = object.saliencyRank
        }
    }
}
使用
selectSubjects(_:)
可以通过UUID锁定特定主体的追踪,传入空数组可以回到自动选择模式。查看references/dockkit-patterns.md获取说话人追踪和显著性过滤的实现方案。

Accessory Events

配件事件

Physical buttons on the dock trigger events:
swift
for await event in accessory.accessoryEvents {
    switch event {
    case .cameraShutter:
        // Start or stop recording / take photo
        break
    case .cameraFlip:
        // Switch front/back camera
        break
    case .cameraZoom(factor: let factor):
        // factor > 0 means zoom in, < 0 means zoom out
        break
    case .button(id: let id, pressed: let pressed):
        // Custom button with identifier
        break
    @unknown default:
        break
    }
}
For first-party apps (Camera, FaceTime), shutter/flip/zoom events work automatically. Third-party apps receive these events and implement behavior through AVFoundation.
底座上的物理按钮会触发事件:
swift
for await event in accessory.accessoryEvents {
    switch event {
    case .cameraShutter:
        // 开始/停止录制 / 拍照
        break
    case .cameraFlip:
        // 切换前后摄像头
        break
    case .cameraZoom(factor: let factor):
        // factor > 0 代表放大,< 0 代表缩小
        break
    case .button(id: let id, pressed: let pressed):
        // 带标识符的自定义按钮
        break
    @unknown default:
        break
    }
}
对于第一方应用(相机、FaceTime),快门/切换/缩放事件会自动生效,第三方应用需要接收这些事件并通过AVFoundation实现对应行为。

Battery Monitoring

电量监控

Monitor the dock's battery status (iOS 18+). A dock can report multiple batteries, each identified by
name
:
swift
for await battery in accessory.batteryStates {
    let level = battery.batteryLevel       // 0.0 - 1.0
    let charging = battery.chargeState     // .charging, .notCharging, .notChargeable
    let low = battery.lowBattery
}
监听底座的电量状态(iOS 18+)。一个底座可能会报告多块电池,每块电池通过
name
标识:
swift
for await battery in accessory.batteryStates {
    let level = battery.batteryLevel       // 0.0 - 1.0
    let charging = battery.chargeState     // .charging(充电中), .notCharging(未充电), .notChargeable(不可充电)
    let low = battery.lowBattery
}

Common Mistakes

常见错误

DON'T: Control motors without disabling system tracking

不要:未禁用系统追踪就控制电机

swift
// WRONG -- system tracking fights manual commands
try await accessory.setAngularVelocity(velocity)

// CORRECT -- disable system tracking first
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
try await accessory.setAngularVelocity(velocity)
swift
// 错误 -- 系统追踪会和手动命令冲突
try await accessory.setAngularVelocity(velocity)

// 正确 -- 先禁用系统追踪
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
try await accessory.setAngularVelocity(velocity)

DON'T: Assume tracking state persists across lifecycle events

不要:假设追踪状态会在生命周期事件中持久化

swift
// WRONG -- state may have reset after backgrounding
func applicationDidBecomeActive() {
    // Assume custom tracking is still active
}

// CORRECT -- re-set tracking state on foreground
func applicationDidBecomeActive() {
    Task {
        try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
    }
}
swift
// 错误 -- 应用回到后台后状态可能已重置
func applicationDidBecomeActive() {
    // 假设自定义追踪仍处于激活状态
}

// 正确 -- 应用回到前台时重新设置追踪状态
func applicationDidBecomeActive() {
    Task {
        try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
    }
}

DON'T: Call track() outside the recommended rate

不要:在推荐频率外调用track()

swift
// WRONG -- calling once per second is too slow
try await accessory.track(observations, cameraInformation: cameraInfo)
// (called at 1 fps)

// CORRECT -- call at 10-30 fps
// Hook into AVCaptureVideoDataOutputSampleBufferDelegate for per-frame calls
swift
// 错误 -- 每秒调用一次太慢
try await accessory.track(observations, cameraInformation: cameraInfo)
// (调用频率为1 fps)

// 正确 -- 以10-30fps的频率调用
// 接入AVCaptureVideoDataOutputSampleBufferDelegate实现逐帧调用

DON'T: Forget to restore tracking after animations

不要:动画完成后忘记恢复追踪

swift
// WRONG -- tracking stays disabled after animation
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
let progress = try await accessory.animate(motion: .kapow)

// CORRECT -- restore tracking when animation completes
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
let progress = try await accessory.animate(motion: .kapow)
while !progress.isFinished && !progress.isCancelled {
    try await Task.sleep(for: .milliseconds(100))
}
try await DockAccessoryManager.shared.setSystemTrackingEnabled(true)
swift
// 错误 -- 动画结束后追踪仍处于禁用状态
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
let progress = try await accessory.animate(motion: .kapow)

// 正确 -- 动画完成后恢复追踪
try await DockAccessoryManager.shared.setSystemTrackingEnabled(false)
let progress = try await accessory.animate(motion: .kapow)
while !progress.isFinished && !progress.isCancelled {
    try await Task.sleep(for: .milliseconds(100))
}
try await DockAccessoryManager.shared.setSystemTrackingEnabled(true)

DON'T: Use DockKit in Simulator

不要:在模拟器中使用DockKit

DockKit requires a physical DockKit-compatible accessory. Guard initialization and provide fallback behavior when no accessory is available.
DockKit需要物理上兼容DockKit的配件,当初始化时没有可用配件请提供降级逻辑。

Review Checklist

检查清单

  • import DockKit
    present where needed
  • Subscribed to
    accessoryStateChanges
    to detect dock/undock events
  • Handled both
    .docked
    and
    .undocked
    states
  • System tracking disabled before custom tracking or motor control
  • System tracking restored after animations complete
  • Custom observations supplied at 10-30 fps
  • Observation
    rect
    uses normalized coordinates (lower-left origin)
  • Camera information matches the active capture device
  • @unknown default
    handled in all switch statements over DockKit enums
  • Motion limits set if restricting accessory range of motion
  • Tracking state re-applied after app returns to foreground
  • Accessory events handled for physical button integration
  • Battery state monitored when showing dock status to user
  • No DockKit code paths executed in Simulator builds
  • 在需要的地方引入
    import DockKit
  • 订阅
    accessoryStateChanges
    检测连接/断开事件
  • 同时处理
    .docked
    .undocked
    两种状态
  • 自定义追踪或电机控制前已禁用系统追踪
  • 动画完成后已恢复系统追踪
  • 自定义观测结果的提供频率为10-30fps
  • 观测结果的
    rect
    使用归一化坐标(原点在左下角)
  • 相机信息与当前激活的采集设备匹配
  • 所有遍历DockKit枚举的switch语句都处理了
    @unknown default
  • 如果需要限制配件运动范围已设置运动限制
  • 应用回到前台后已重新应用追踪状态
  • 已处理物理按钮对应的配件事件
  • 向用户展示底座状态时已监听电量状态
  • 模拟器构建版本中不会执行DockKit相关代码路径

References

参考资料