Loading...
Loading...
AI-powered testability assessment using 10 principles of intrinsic testability with Playwright and optional Vibium integration. Evaluates web applications against Observability, Controllability, Algorithmic Simplicity, Transparency, Stability, Explainability, Unbugginess, Smallness, Decomposability, and Similarity. Use when assessing software testability, evaluating test readiness, identifying testability improvements, or generating testability reports.
npx skill4agent add proffesor-for-testing/agentic-qe testability-scoring# Run assessment on any URL
TEST_URL='https://example.com/' npx playwright test tests/testability-scoring/testability-scoring.spec.js --project=chromium --workers=1
# Or use shell script wrapper
.claude/skills/testability-scoring/scripts/run-assessment.sh https://example.com/| Principle | Weight | Key Question |
|---|---|---|
| Observability | 15% | Can we see what's happening? |
| Controllability | 15% | Can we control the application? |
| Algorithmic Simplicity | 10% | Are behaviors predictable? |
| Algorithmic Transparency | 10% | Can we understand what it does? |
| Algorithmic Stability | 10% | Does behavior remain consistent? |
| Explainability | 10% | Is the interface understandable? |
| Unbugginess | 10% | How error-free is it? |
| Smallness | 10% | Are components appropriately sized? |
| Decomposability | 5% | Can we test parts in isolation? |
| Similarity | 5% | Is the tech stack familiar? |
| Method | Command | When to Use |
|---|---|---|
| Shell Script | | One-time assessment |
| ENV Override | | CI/CD integration |
| Config File | Update | Repeated runs |
| Principle | Measures | Indicators |
|---|---|---|
| Observability | State visibility, logging, monitoring | Console output, network tracking, error visibility |
| Controllability | Input control, state manipulation | API access, test data injection, determinism |
| Principle | Measures | Indicators |
|---|---|---|
| Simplicity | Predictable behavior | Clear I/O relationships, low complexity |
| Transparency | Understanding what system does | Visible processes, readable code |
| Stability | Consistent behavior | Change resilience, maintainability |
| Explainability | Interface understanding | Good docs, semantic structure, help text |
| Unbugginess | Error-free operation | Console errors, warnings, runtime issues |
| Smallness | Component size | Element count, script bloat, page complexity |
| Principle | Measures | Indicators |
|---|---|---|
| Decomposability | Isolation testing | Component separation, modular design |
| Similarity | Technology familiarity | Standard frameworks, known patterns |
1. Navigate to URL → 2. Collect Metrics → 3. Score Principles
↓
4. Generate JSON ← 5. Calculate Grades ← 6. Apply Weights
↓
7. Generate HTML Report with Radar Chart
↓
8. Open in Browser (auto-opens)tests/reports/
├── testability-results-<timestamp>.json # Raw data
├── testability-report-<timestamp>.html # Visual report
└── latest.json # Symlink# GitHub Actions
- name: Testability Assessment
run: |
timeout 180 .claude/skills/testability-scoring/scripts/run-assessment.sh ${{ env.APP_URL }}
- name: Upload Reports
uses: actions/upload-artifact@v3
with:
name: testability-reports
path: tests/reports/testability-*.html// Combine testability with QX analysis
const qxAnalysis = await Task("QX Analysis", {
target: 'https://example.com',
integrateTestability: true
}, "qx-partner");
// Returns combined insights:
// - QX Score: 78/100
// - Testability Integration: Observability 72/100
// - Combined Insight: Low observability may mask UX issuesimport { runTestabilityAssessment } from './testability';
const results = await runTestabilityAssessment('https://example.com');
console.log(`Overall: ${results.overallScore}/100 (${results.grade})`);
console.log('Recommendations:', results.recommendations);// Run testability assessment
const assessment = await Task("Testability Assessment", {
url: 'https://example.com',
generateReport: true,
openBrowser: true
}, "qe-quality-analyzer");
// Use with QX Partner for holistic analysis
const qxReport = await Task("Full QX Analysis", {
target: 'https://example.com',
integrateTestability: true,
detectOracleProblems: true
}, "qx-partner");claude mcp add vibium -- npx -y vibium| Principle | Vibium Enhancement | Benefit |
|---|---|---|
| Observability | Auto-wait duration tracking | Measures DOM stability (30s timeout, 100ms polling) |
| Controllability | Element interaction success rate | Validates automation readiness via MCP |
| Stability | Screenshot consistency | Visual regression detection for layout stability |
| Explainability | Element attribute extraction | ARIA labels, semantic HTML validation |
// Testability assessment using both engines
const assessment = {
// Playwright: Comprehensive metrics
playwright: await runPlaywrightAssessment(url),
// Vibium: Stability metrics
vibium: {
elementStability: await measureAutoWaitDuration(url),
visualConsistency: await compareScreenshots(url),
accessibilityAttributes: await extractARIALabels(url)
}
};
// Enhanced Observability Score
const observability =
(assessment.playwright.consoleErrors * 0.6) +
(assessment.vibium.elementStability * 0.4);// 1. Element Stability Measurement
const browser = await browser_launch();
await browser_navigate({ url });
const startTime = Date.now();
const element = await browser_find({ selector: ".critical-element" });
const autoWaitDuration = Date.now() - startTime;
// Lower duration = better stability
// 2. Visual Consistency Check
const screenshot1 = await browser_screenshot();
await browser_navigate({ url }); // Reload
const screenshot2 = await browser_screenshot();
const visualDiff = compareImages(screenshot1.png, screenshot2.png);
// Lower diff = better stability
// 3. Accessibility Attribute Extraction
const elements = await browser_find({ selector: "button, a, input" });
const ariaLabels = elements.map(el => el.attributes["aria-label"]);
const semanticScore = (ariaLabels.filter(Boolean).length / elements.length) * 100;aqe/testability/
├── assessments/* - Assessment results by URL
├── historical/* - Historical scores for trend analysis
├── recommendations/* - Improvement recommendations
├── integration/* - QX integration data
└── vibium/* - Vibium-specific metrics (optional)const testabilityFleet = await FleetManager.coordinate({
strategy: 'testability-assessment',
agents: [
'qe-quality-analyzer', // Primary assessment
'qx-partner', // UX integration
'qe-visual-tester' // Visual validation
],
topology: 'sequential'
});| Issue | Solution |
|---|---|
| Tests timing out | Increase timeout: |
| Partial results | Check console errors, increase network timeout |
| Report not opening | Use |
| Config not updating | Use |
| Vibium not available | Install via |
| Hybrid mode errors | Vibium is optional; assessments work without it |
claude mcp add vibium -- npx -y vibium