Loading...
Loading...
Compare original and translation side by side
sdlc-testingsdlc-testingtemplatestemplatesSKILL.mdSKILL.md| Category | Artifact | Format | Example |
|---|---|---|---|
| Correctness | Test plan compliant with BS ISO/IEC/IEEE 29119-3:2013 | Markdown doc per | |
| 类别 | 工件 | 格式 | 示例 |
|---|---|---|---|
| 正确性 | 符合BS ISO/IEC/IEEE 29119-3:2013标准的测试计划 | 遵循IEEE测试标准的Markdown文档,参考 | |
templates/templates/world-class-engineeringadvanced-testing-strategyworld-class-engineeringadvanced-testing-strategyandroid-tddai-error-handlingvibe-security-skillfeature-planningsdlc-planningsdlc-designandroid-tddai-error-handlingvibe-security-skillfeature-planningsdlc-planningsdlc-design| # | Document | File | Purpose | Audience | Phase |
|---|---|---|---|---|---|
| 1 | Software Test Plan | | Testing strategy, tools, environments, schedule, completion criteria | QA leads, PMs, devs | After SRS + SDD |
| 2 | Test Case Specifications | | Normative 29119-3 test cases: ID, objective, priority, traceability, preconditions, input, expected result | Test engineers, devs | During development |
| 3 | Validation & Verification Plan | | V&V approach (built right + right product) | QA mgrs, PMs, compliance | After SRS + SDD |
| 4 | Validation Test Report | | Test execution results and Go/No-Go release decision | PMs, stakeholders, QA | Before release |
| 5 | Peer Review Report | | Code, design, and document review findings | Dev team, tech leads | Throughout SDLC |
| 6 | Incident Report | | Anomaly record: ID, timing, context, description, impact, urgency, status | QA, dev leads | During execution |
| 7 | Test Completion Report | | Test summary, deviations, completion criteria met, residual risks, lessons learned | PMs, stakeholders, compliance | End of test phase |
| 序号 | 文档名称 | 文件 | 用途 | 受众 | 阶段 |
|---|---|---|---|---|---|
| 1 | 软件测试计划 | | 测试策略、工具、环境、进度、完成标准 | QA负责人、项目经理、开发人员 | SRS与SDD完成后 |
| 2 | 测试用例规范 | | 符合29119-3标准的测试用例:ID、目标、优先级、可追溯性、前置条件、输入、预期结果 | 测试工程师、开发人员 | 开发过程中 |
| 3 | 验证与确认计划 | | V&V方法(正确构建+构建正确产品) | QA经理、项目经理、合规人员 | SRS与SDD完成后 |
| 4 | 验证测试报告 | | 测试执行结果与上线/不上线(Go/No-Go)发布决策 | 项目经理、利益相关者、QA | 发布前 |
| 5 | 同行评审报告 | | 代码、设计与文档评审发现 | 开发团队、技术负责人 | SDLC全流程 |
| 6 | 事件报告 | | 异常记录:ID、时间、上下文、描述、影响、紧急程度、状态 | QA、开发负责人 | 执行过程中 |
| 7 | 测试完成报告 | | 测试总结、偏差、完成标准达成情况、剩余风险、经验教训 | 项目经理、利益相关者、合规人员 | 测试阶段结束时 |
[VERIFIABILITY-FAIL][VERIFIABILITY-FAIL]android-tddandroid-tdd / UI/E2E \ 10% - Compose Testing, Espresso, browser E2E
/------------\
/ Integration \ 20% - Room+MockWebServer, API with real DB
/----------------\
/ Unit Tests \ 70% - JUnit 5, MockK, PHPUnit (fast, isolated)
/====================\ / UI/E2E \ 10% - Compose Testing、Espresso、浏览器端E2E测试
/------------\
/ 集成测试 \ 20% - Room+MockWebServer、带真实数据库的API测试
/----------------\
/ 单元测试 \ 70% - JUnit 5、MockK、PHPUnit(快速、隔离)
/====================\| Activity | Traditional Phase | Shift-Left Phase |
|---|---|---|
| Unit testing | After coding | During coding (TDD) |
| Security testing | Pre-release | During design + development |
| Performance testing | Pre-release | During development (benchmarks) |
| Code review | After feature complete | Per-commit (PR-based) |
| Requirements review | After SRS finalized | During requirements gathering |
| 活动 | 传统阶段 | 左移后阶段 |
|---|---|---|
| 单元测试 | 编码后 | 编码期间(TDD) |
| 安全测试 | 发布前 | 设计与开发期间 |
| 性能测试 | 发布前 | 开发期间(基准测试) |
| 代码评审 | 功能完成后 | 每次提交(基于PR) |
| 需求评审 | SRS定稿后 | 需求收集期间 |
Prerequisite: SRS from sdlc-planning + SDD from sdlc-design
|
Step 1: Software Test Plan (overall strategy)
|
Step 2: Validation & Verification Plan (V&V approach)
|
Step 3: Test Case Specifications (detailed test cases)
|
[--- Development & testing execution happens here ---]
|
Step 4: Peer Review Reports (during development)
|
Step 5: Validation Test Report (before release)前置条件:来自sdlc-planning的SRS + 来自sdlc-design的SDD
|
步骤1:软件测试计划(整体策略)
|
步骤2:验证与确认计划(V&V方法)
|
步骤3:测试用例规范(详细测试用例)
|
[--- 开发与测试执行在此阶段进行 ---]
|
步骤4:同行评审报告(开发期间)
|
步骤5:验证测试报告(发布前)| Input | Source | Required? |
|---|---|---|
| Software Requirements Spec (SRS) | | Yes |
| Software Design Document (SDD) | | Recommended |
| Quality Assurance Plan | | Recommended |
| Risk Management Plan | | Recommended |
| Feature specifications | | If feature-level |
| 输入 | 来源 | 是否必需? |
|---|---|---|
| 软件需求规格说明书(SRS) | | 是 |
| 软件设计文档(SDD) | | 推荐 |
| 质量保证计划 | | 推荐 |
| 风险管理计划 | | 推荐 |
| 功能规格说明 | | 若为功能级需求则需要 |
| Platform | Framework | Scope | Speed |
|---|---|---|---|
| Android | JUnit 5 + MockK + Turbine | ViewModels, UseCases, Repositories, Mappers | <1ms each |
| PHP | PHPUnit | Services, validators, business logic, helpers | <10ms each |
| 平台 | 框架 | 范围 | 速度 |
|---|---|---|---|
| Android | JUnit 5 + MockK + Turbine | ViewModels、UseCases、Repositories、Mappers | 每个<1ms |
| PHP | PHPUnit | 服务、验证器、业务逻辑、辅助工具 | 每个<10ms |
| Platform | Framework | Scope |
|---|---|---|
| Android | Room in-memory DB + MockWebServer | DAO queries, API contracts, auth flows |
| PHP | PHPUnit + real MySQL test DB | API endpoints, stored procedures, multi-table ops |
| 平台 | 框架 | 范围 |
|---|---|---|
| Android | Room内存数据库 + MockWebServer | DAO查询、API契约、认证流程 |
| PHP | PHPUnit + 真实MySQL测试数据库 | API端点、存储过程、多表操作 |
| Platform | Framework | Scope |
|---|---|---|
| Android | Compose Testing + Espresso | Screen rendering, navigation, user flows |
| Web | Browser testing (manual + automated) | CRUD workflows, form validation, responsive layout |
| 平台 | 框架 | 范围 |
|---|---|---|
| Android | Compose Testing + Espresso | 屏幕渲染、导航、用户流程 |
| Web | 浏览器测试(手动+自动化) | CRUD工作流、表单验证、响应式布局 |
| Stage | Location | Testers | Focus |
|---|---|---|---|
| Alpha | Controlled environment (internal/QA lab) | Internal testers, stakeholders | Functional correctness, defect discovery |
| Beta | Real-world environment (staging/limited production) | Selected external users | Real-world usability, edge-case discovery, performance under real load |
| 阶段 | 地点 | 测试人员 | 重点 |
|---|---|---|---|
| Alpha | 受控环境(内部/QA实验室) | 内部测试人员、利益相关者 | 功能正确性、缺陷发现 |
| Beta | 真实环境(预发布/有限生产环境) | 选定的外部用户 | 真实场景可用性、边缘案例发现、真实负载下的性能 |
| Step | Owner | Action |
|---|---|---|
| 1 | Tester | Reproduce — determine exact minimal sequence; check for intermittence |
| 2 | Tester | Discriminate — test bug or system bug? |
| 3 | Tester | Isolate — identify external factors (config, data, workflows) affecting symptoms |
| 4 | Developer | Root cause — find cause in code, hardware, network, or environment |
| 5 | Developer | Repair — fix without introducing new problems |
| 6 | Developer | Verify fix — confirm the fix is clean before handoff |
| 7 | Tester | Confirm + Regression — does it pass the failing test? Does everything else still work? |
| 步骤 | 负责人 | 操作 |
|---|---|---|
| 1 | 测试人员 | 重现 ——确定精确的最小操作序列;检查是否存在间歇性问题 |
| 2 | 测试人员 | 区分 ——是测试Bug还是系统Bug? |
| 3 | 测试人员 | 隔离 ——识别影响症状的外部因素(配置、数据、工作流) |
| 4 | 开发人员 | 根本原因 ——在代码、硬件、网络或环境中找到问题根源 |
| 5 | 开发人员 | 修复 ——修复问题且不引入新问题 |
| 6 | 开发人员 | 验证修复 ——在交接前确认修复干净 |
| 7 | 测试人员 | 确认+回归测试 ——修复后是否通过之前失败的测试?其他功能是否仍正常工作? |
## Test Data Managementfranchise_id## 测试数据管理franchise_idfranchise_idfranchise_id| Area | Method | Reference |
|---|---|---|
| Tenant isolation | Automated + manual: cross-tenant access denied | |
| Auth bypass | Token manipulation, session hijacking attempts | |
| Injection | SQL injection, XSS, CSRF payloads | OWASP Top 10 |
| Data exposure | API response auditing, error message review | |
| 领域 | 方法 | 参考 |
|---|---|---|
| 租户隔离 | 自动化+手动测试:禁止跨租户访问 | |
| 认证绕过 | 令牌篡改、会话劫持尝试 | |
| 注入攻击 | SQL注入、XSS、CSRF载荷 | OWASP Top 10 |
| 数据泄露 | API响应审计、错误消息审查 | |
| Metric | Target | Tool |
|---|---|---|
| API response time | < 500ms (P95) | curl timing, load test tools |
| Web page load | < 2s (first contentful paint) | Browser DevTools, Lighthouse |
| Android cold start | < 3s | Android Profiler |
| Database query | < 100ms (P95) | MySQL slow query log, EXPLAIN |
| 指标 | 目标 | 工具 |
|---|---|---|
| API响应时间 | < 500ms(P95) | curl计时、负载测试工具 |
| 网页加载时间 | < 2s(首次内容绘制) | 浏览器开发者工具、Lighthouse |
| Android冷启动时间 | < 3s | Android Profiler |
| 数据库查询时间 | < 100ms(P95) | MySQL慢查询日志、EXPLAIN |
| Skill | Relationship |
|---|---|
| Provides SRS (requirement IDs for traceability), QA Plan (quality standards), Risk Plan (test-specific risks). |
| Provides SDD (architecture to verify), database design (schema to test), API design (contracts to validate). |
| Raw requirements gathered via interview. Feed into SRS before creating test docs. |
| 技能 | 关系 |
|---|---|
| 提供SRS(用于可追溯性的需求ID)、QA计划(质量标准)、风险计划(测试特定风险)。 |
| 提供SDD(待验证的架构)、数据库设计(待测试的 schema)、API设计(待验证的契约)。 |
| 通过访谈收集的原始需求。在创建测试文档前需输入至SRS。 |
| Skill | Relationship |
|---|---|
| Actual TDD implementation patterns (Red-Green-Refactor, layer-specific tests). This skill documents; |
| 5-layer validation stack for AI-generated code. Complements this skill's formal V&V processes. |
| "Trust but verify" patterns. Use alongside peer review processes. |
| Security testing patterns, OWASP mapping. Reference in test plans and security test cases. |
| Feature-level testing strategy. This skill covers project-level testing. |
| 技能 | 关系 |
|---|---|
| 实际TDD实现模式(红-绿-重构、分层测试)。本技能负责文档记录; |
| AI生成代码的5层验证栈。补充本技能的正式V&V流程。 |
| “信任但验证”模式。与同行评审流程配合使用。 |
| 安全测试模式、OWASP映射。在测试计划和安全测试用例中参考。 |
| 功能级测试策略。本技能覆盖项目级测试。 |
| Skill | Relationship |
|---|---|
| Play Store compliance testing. Uses test results from this skill's reports. |
| 技能 | 关系 |
|---|---|
| Google Play商店合规测试。使用本技能报告中的测试结果。 |
| Skill | Phase | Status |
|---|---|---|
| Planning & Management | Available |
| Design & Architecture | Available |
| Testing & Quality | This Skill |
| Delivery & Deployment | Available |
| 技能 | 阶段 | 状态 |
|---|---|---|
| 规划与管理 | 可用 |
| 设计与架构 | 可用 |
| 测试与质量 | 本技能 |
| 交付与部署 | 可用 |
| Aspect | Multi-Tenant SaaS | Standalone App |
|---|---|---|
| Tenant isolation tests | Required (franchise_id in every query) | Not applicable |
| RBAC test cases | Full matrix (super_admin, owner, staff) | Simple role tests |
| Cross-tenant security | Dedicated test suite | Omit |
| Test data setup | Per-tenant fixtures (franchise_id = 1, 2) | Single dataset |
| 方面 | 多租户SaaS | 独立应用 |
|---|---|---|
| 租户隔离测试 | 必需(每个查询包含franchise_id) | 不适用 |
| RBAC测试用例 | 完整矩阵(超级管理员、所有者、员工) | 简单角色测试 |
| 跨租户安全 | 专用测试套件 | 省略 |
| 测试数据设置 | 按租户准备固定数据(franchise_id = 1、2) | 单一数据集 |
| Aspect | Android + Web | Web-Only |
|---|---|---|
| Test frameworks | JUnit 5, MockK, Compose Testing + PHPUnit | PHPUnit only |
| UI testing | Compose tests + browser tests | Browser tests only |
| Offline testing | Room caching, network error scenarios | N/A |
| Device matrix | API levels 26-35, multiple screen sizes | Browser matrix |
| 方面 | Android + Web | 仅Web |
|---|---|---|
| 测试框架 | JUnit 5、MockK、Compose Testing + PHPUnit | 仅PHPUnit |
| UI测试 | Compose测试 + 浏览器测试 | 仅浏览器测试 |
| 离线测试 | Room缓存、网络错误场景 | 不适用 |
| 设备矩阵 | API级别26-35、多种屏幕尺寸 | 浏览器矩阵 |
| Stage | Trigger | Tests Run |
|---|---|---|
| Pre-commit | Local commit | Unit tests (fast) |
| PR validation | PR created/updated | Unit + integration tests |
| Merge to develop | PR merged | Full test suite + coverage |
| Release candidate | Tag created | Full suite + security + performance |
| Post-deploy | Production deploy | Smoke tests only |
| 阶段 | 触发条件 | 运行的测试 |
|---|---|---|
| 提交前 | 本地提交 | 单元测试(快速) |
| PR验证 | PR创建/更新 | 单元+集成测试 |
| 合并至develop分支 | PR合并 | 完整测试套件+覆盖率测试 |
| 发布候选版本 | 创建标签 | 完整套件+安全+性能测试 |
| 部署后 | 生产环境部署 | 仅冒烟测试 |
vibe-security-skillvibe-security-skill| Anti-Pattern | Why It Fails | Do This Instead |
|---|---|---|
| No formal test plan | Ad-hoc testing misses critical paths | Write STP before testing begins |
| Test cases without expected results | Can't determine pass/fail | Every TC has explicit expected results (test oracle) |
| Vague expected results ("response looks OK") | Not a test oracle; tester must interpret | State exact output: value, format, timing, error code |
| No traceability to requirements | Can't prove coverage | Map every TC to FR-xxx or NFR-xxx |
| Testing only happy paths | Edge cases cause production failures | Include negative, boundary, and error cases |
| No test data strategy | Inconsistent, flaky tests | Define fixtures, factories, seed data with tenant isolation |
| Skipping security testing | Vulnerabilities ship to production | Include security test suite in every release |
| No peer review process | Bugs caught late, inconsistent code | Standardize reviews with checklists |
| Rubber-stamp reviews | Reviews provide no value | Require findings documented, metrics tracked |
| Testing in production only | Users find bugs, not testers | Test in staging first, smoke test prod |
| No regression suite | Passing tests break silently between releases | Define regression suite, trigger on every RC |
| No incident tracking | Anomalies lose context | Open an Incident Report for every anomaly during execution |
| No Test Completion Report | Phase never formally closes | Produce TCR before passing to next lifecycle phase |
| 反模式 | 失败原因 | 正确做法 |
|---|---|---|
| 无正式测试计划 | 临时测试会遗漏关键路径 | 测试开始前编写软件测试计划(STP) |
| 测试用例无预期结果 | 无法判定通过/失败 | 每个测试用例(TC)都有明确的预期结果(测试准则) |
| 模糊的预期结果(如“响应看起来正常”) | 不属于测试准则;测试人员需主观解读 | 明确输出内容:数值、格式、时间、错误码 |
| 与需求无关联 | 无法证明覆盖范围 | 将每个测试用例映射至FR-xxx或NFR-xxx |
| 仅测试正常路径 | 边缘案例会导致生产故障 | 包含负面、边界和错误案例 |
| 无测试数据策略 | 测试结果不一致、不稳定 | 定义固定数据、工厂类、种子数据并实现租户隔离 |
| 跳过安全测试 | 漏洞会被发布至生产环境 | 每个版本都包含安全测试套件 |
| 无同行评审流程 | Bug发现晚、代码不一致 | 使用检查清单标准化评审流程 |
| 形式化评审 | 评审无实际价值 | 要求记录发现问题、跟踪指标 |
| 仅在生产环境测试 | 用户发现Bug而非测试人员 | 先在预发布环境测试,生产环境仅做冒烟测试 |
| 无回归套件 | 通过的测试会在版本间无声失效 | 定义回归套件,每个发布候选版本触发回归测试 |
| 无事件跟踪 | 异常会丢失上下文 | 执行过程中为每个异常创建事件报告 |
| 无测试完成报告 | 阶段从未正式结束 | 进入下一个生命周期阶段前生成测试完成报告(TCR) |