analyze-selective-testing

Original：🇺🇸 English

Translated

Analyzes Xcode selective testing effectiveness for a test run, showing which test targets were skipped or ran, and diagnosing regressions in test selection. Can compare two test runs to identify what changed.

2installs

Sourcetuist/agent-skills

Added on2026-05-09

NPX Install

npx skill4agent add tuist/agent-skills analyze-selective-testing

SKILL.md Content

View Translation Comparison →

Analyze Selective Testing

Quick Start

Run
```
tuist test list --json
```
to find test runs.

Run

tuist test xcode target list <test-run-id> --json

to see per-target selective testing status.

If effectiveness dropped, compare targets between a known-good run and the regressed run.

Step 1: Resolve Test Runs

Find the test run to analyze

List recent test runs, optionally filtering by branch:

bash

tuist test list --json --page-size 10
tuist test list --git-branch feature-x --json --page-size 5

Get basic info for a specific test run:

bash

tuist test show <test-run-id> --json

Use this to confirm the run's branch, status, scheme, and environment (

xcode_version

,

macos_version

).

Defaults

If no test run is specified, use the most recent CI test run on the current branch.
For comparison, use the most recent CI test run on the project's default branch.

Step 2: Drill Into Selective Testing Targets

List per-target selective testing data for a test run:

bash

tuist test xcode target list <test-run-id> --json

Each target includes:

name: The test target name
hit_status:
```
miss
```
(ran),
```
local
```
(skipped via local cache), or
```
remote
```
(skipped via remote cache)
hash: The selective testing hash for this target

Filter by status to focus your analysis:

bash

tuist test xcode target list <test-run-id> --hit-status miss --json
tuist test xcode target list <test-run-id> --hit-status local --json
tuist test xcode target list <test-run-id> --hit-status remote --json

Assess effectiveness

Count the targets by status:

effectiveness = (local_hits + remote_hits) / total_targets * 100

Effectiveness	Verdict
60-100%	Healthy — most unchanged tests are being skipped
30-60%	Moderate — some cache invalidation occurring, worth investigating
0-30%	Low — likely a core dependency changed or the cache was invalidated
0%	Cold cache — first run, environment change, or full invalidation

Step 3: Diagnose Regressions

If effectiveness dropped, investigate:

Check for global invalidation

If nearly all targets show as

miss

, the entire selective testing cache was invalidated. Common causes:

Cause	How to verify
Xcode version change	Compare `xcode_version` between good and bad runs via `tuist test show`
CI environment change	Compare `macos_version` , `model_identifier` between runs
Project graph or dependency change	Check git history for changes to project manifests, dependency versions, or target configurations
Core dependency change	A widely-depended-on target changed, invalidating all its dependents

Note: Tuist CLI version upgrades rarely cause hash invalidation — the hash version is not updated on every release.

Check for cascade invalidation

If only some targets show as

miss

, a dependency change likely cascaded:

Run

tuist test xcode target list <good-run-id> --json

and

tuist test xcode target list <bad-run-id> --json

.

Match targets by name — identify those that changed from
```
local
```
/
```
remote
```
to
```
miss
```
.
Compare hashes for changed targets — if a target's hash differs, its sources or dependencies changed.
Look for a common dependency among the invalidated targets.

Compare two test runs

For each target:

Match by
```
name
```
Compare
```
hit_status
```
(hit -> miss = invalidated, miss -> hit = improved)
Compare
```
hash
```
(different hash = target or its dependencies changed)

Step 4: Recommendations

Based on the diagnosis:

Environment change: Ensure CI uses consistent Xcode/macOS versions. This is the most common cause of full cache invalidation.
Cascade from dependency change: Consider if the change to the root target was necessary. If the root target is a widely-shared module, consider splitting it.
Cold cache on new branch: Run tests once to warm the cache. Subsequent runs will benefit from selective testing.

Summary Format

Produce a summary with:

Overall: effectiveness percentage, verdict (healthy/moderate/low/cold)
Target breakdown: targets grouped by hit status
Comparison (if applicable): targets that changed status between runs, with hash diffs
Root cause: what caused the regression
Recommendations: actionable next steps

Example:

Selective Testing Analysis: test run abc123 on feature-x

Effectiveness: 15% (3/20 targets skipped) -- LOW
  Local hits:  2 targets (CoreTests, UtilTests)
  Remote hits: 1 target (NetworkTests)
  Misses:      17 targets

Comparison with baseline (run def456, 75% effectiveness):
  14 targets changed from hit to miss
  All changed targets have different hashes

Root cause: Xcode version changed from 15.2 to 16.0 between runs.
This changed all target hashes, invalidating the entire cache.

Recommendations:
- Align CI Xcode version with the version used in the baseline run.
- If the upgrade is intentional, run tests once to re-warm the cache.

Done Checklist

Identified the test run(s) to analyze
Drilled into per-target selective testing status via
```
tuist test xcode target list
```
Diagnosed root cause if effectiveness is low
Compared targets with baseline if regression detected
Provided actionable recommendations

analyze-selective-testing

NPX Install

Tags

SKILL.md Content

Analyze Selective Testing

Quick Start

Step 1: Resolve Test Runs

Find the test run to analyze

Defaults

Step 2: Drill Into Selective Testing Targets

Assess effectiveness

Step 3: Diagnose Regressions

Check for global invalidation

Check for cascade invalidation

Compare two test runs

Step 4: Recommendations

Summary Format

Done Checklist