click-target

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Click Target in XR

在 XR 中点击目标

Find a target object in the scene and click it using a controller, then verify the click registered.
在场景中查找目标对象,使用控制器点击它,然后验证点击是否已记录。

Arguments

参数

$ARGUMENTS
should be a description of the target to find (e.g., "the RESTART button", "the scoreboard", "the settings panel").
$ARGUMENTS
应为对要查找目标的描述(例如,“RESTART 按钮”、“记分板”或“设置面板”)。

Workflow

工作流

1. Get Scene Hierarchy

1. 获取场景层级

Use
mcp__iwsdk-runtime__scene_get_runtime_hierarchy
to find the target object's UUID.
  • Look for objects with entityIndex (these are ECS entities)
  • PanelUI elements typically have nested children for their content
使用
mcp__iwsdk-runtime__scene_get_runtime_hierarchy
查找目标对象的 UUID。
  • 查找带有 entityIndex 的对象(这些是 ECS 实体)
  • PanelUI 元素通常包含嵌套的内容子元素

2. Get Target Transform

2. 获取目标变换

Use
mcp__iwsdk-runtime__scene_get_object_transform
with the target UUID.
  • Use
    positionRelativeToXROrigin
    for all positioning operations
  • Note the position for the next steps
使用
mcp__iwsdk-runtime__scene_get_object_transform
和目标 UUID 获取目标变换。
  • 所有定位操作均使用
    positionRelativeToXROrigin
  • 记下当前位置,供后续步骤使用

3. Position Headset to Look at Target

3. 调整头显位置以朝向目标

Use
mcp__iwsdk-runtime__xr_look_at
with device
headset
and the target position.
  • This orients the headset to face the target
使用
mcp__iwsdk-runtime__xr_look_at
,将设备设为
headset
并传入目标位置。
  • 这会使头显朝向目标

4. Screenshot to Verify Target Visible

4. 通过截图验证目标可见

Use
mcp__iwsdk-runtime__browser_screenshot
to verify:
  • Target is visible in the view
  • Target is not occluded by other objects
  • If not visible, use
    mcp__iwsdk-runtime__xr_look_at
    with
    moveToDistance
    to get closer
使用
mcp__iwsdk-runtime__browser_screenshot
验证:
  • 目标在视野中可见
  • 目标未被其他对象遮挡
  • 如果目标不可见,使用带有
    moveToDistance
    的
    mcp__iwsdk-runtime__xr_look_at
    靠近目标

5. Position Controller

5. 调整控制器位置

Use
mcp__iwsdk-runtime__xr_get_transform
to check controller position.
  • If controller is visible in screenshot and not occluding target, proceed
  • If controller is far away or not visible:
    1. Get headset position with
      mcp__iwsdk-runtime__xr_get_transform
    2. Set controller to headset position with
      mcp__iwsdk-runtime__xr_set_transform
    3. Micro-adjust: offset x by +0.25 (right) or -0.25 (left), y by -0.1, z by -0.3
使用
mcp__iwsdk-runtime__xr_get_transform
检查控制器位置。
  • 如果控制器在截图中可见且未遮挡目标,则继续执行
  • 如果控制器距离较远或不可见:
    1. 使用
      mcp__iwsdk-runtime__xr_get_transform
      获取头显位置
    2. 使用
      mcp__iwsdk-runtime__xr_set_transform
      将控制器设置到头显位置
    3. 微调:将 x 偏移 +0.25(向右)或 -0.25(向左),y 偏移 -0.1,z 偏移 -0.3

6. Point Controller at Target

6. 让控制器指向目标

Use
mcp__iwsdk-runtime__xr_look_at
with the controller device and target position.
  • For UI elements on panels, you may need to target a specific child element
  • If the target is a button on a panel, the button may be offset from the panel center
使用
mcp__iwsdk-runtime__xr_look_at
,将设备设为控制器并传入目标位置。
  • 对于面板上的 UI 元素,可能需要指定具体的子元素
  • 如果目标是面板上的按钮,按钮的位置可能与面板中心存在偏移

7. Click

7. 点击

Use
mcp__iwsdk-runtime__xr_select
with the controller device.
使用
mcp__iwsdk-runtime__xr_select
并将设备设为控制器。

8. Verify Click

8. 验证点击

Use
mcp__iwsdk-runtime__browser_get_console_logs
with a pattern to check for expected log messages.
  • If no logs match, the click may have missed - adjust target position and retry
  • Common adjustments: change x or z by 0.05-0.1 to hit child elements
使用
mcp__iwsdk-runtime__browser_get_console_logs
并传入匹配模式,以检查预期的日志消息。
  • 如果没有匹配的日志,点击可能未命中——调整目标位置后重试
  • 常见调整:将 x 或 z 更改 0.05-0.1,以命中子元素

Tips

提示

  • For PanelUI buttons, the button position is often offset from the panel center
  • Use
    mcp__iwsdk-runtime__scene_get_object_transform
    on child elements to find exact button positions
  • Always verify with console logs rather than assuming from visuals
  • The ray visual in screenshots can be misleading - test with actual clicks
  • Either controller (left or right) can be used
  • 对于 PanelUI 按钮,按钮位置通常与面板中心存在偏移
  • 对子元素使用
    mcp__iwsdk-runtime__scene_get_object_transform
    查找精确的按钮位置
  • 始终通过控制台日志进行验证,不要仅凭视觉效果做出判断
  • 截图中的射线可视化效果可能产生误导——应使用实际点击进行测试
  • 可以使用任意一个控制器(左或右)

Example

示例

To click the RESTART button on the pong scoreboard:
  1. Find scoreboard entity (entityIndex 6 in the pong game)
  2. Scoreboard is at (0, -0.798, -1.25)
  3. Button is at approximately (0.05, -0.798, -1.109) - offset toward player
  4. Point controller at button position and click
  5. Verify "[BUTTON CLICKED]" appears in console logs
点击 pong 记分板上的 RESTART 按钮:
  1. 查找记分板实体(pong 游戏中的 entityIndex 为 6)
  2. 记分板位于 (0, -0.798, -1.25)
  3. 按钮大约位于 (0.05, -0.798, -1.109)——向玩家方向偏移
  4. 将控制器指向按钮位置并点击
  5. 验证控制台日志中出现 "[BUTTON CLICKED]"