Back to Details

ascendc-operator-testcase-gen

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

AscendC Operator Testcase Gen Skill

AscendC Operator Testcase Gen Skill

根据算子设计文档（design.md）生成验证用例和标杆，供后续UT代码生成、精度验证以及性能验证skill使用。
Generate verification testcases and benchmarks based on the operator design document (design.md) for subsequent use by the ascendc-operator-ut-gen, ascendc-operator-precision-eval, and ascendc-operator-performance-eval skills.

使用场景

Usage Scenarios

在 
ascendc-operator-design
创建
design.md
之后、
ascendc-operator-ut-gen
/
ascendc-operator-precision-eval
/
ascendc-operator-performance-eval
调用之前调用。
Call this skill after creating 
design.md
in 
ascendc-operator-design
and before invoking 
ascendc-operator-ut-gen
/
ascendc-operator-precision-eval
/
ascendc-operator-performance-eval
.

调用接口

Calling Interface

必需参数

Required Parameters

调用此技能时，必须明确提供以下参数：
参数1：设计文档
  • 根据用户提供的
    design.md
    设计用例
  • 必须包含算子逻辑、输入输出、约束条件、所支持数据类型
  • 如果未指定，默认项目路径下的design.md
When calling this skill, you must explicitly provide the following parameters:
Parameter 1: Design Document
  • Design testcases based on the 
    design.md
    provided by the user
  • Must include operator logic, input/output, constraints, and supported data types
  • If not specified, the default is design.md in the project path

设计流程

Design Process

1. 算子设计文档解析

1. Operator Design Document Parsing

  1. 读取设计文档中[算子接口]、[计算逻辑]、[参考实现]章节
  2. 根据算子实际计算逻辑以及pyTorch已有实现、参考文档等，生成算子标杆，了解算子各个输入输出之间shape的相互约束。
  3. 基于约束，生成模型场景使用该算子时的常见shape典型用例、以及根据边界条件生成的泛化用例。
  1. Read the [Operator Interface], [Computation Logic], and [Reference Implementation] sections in the design document
  2. Generate operator benchmarks based on the actual computation logic of the operator, existing PyTorch implementations, reference documents, etc., to understand the mutual constraints on the shapes of various inputs and outputs of the operator.
  3. Based on the constraints, generate typical testcases with common shapes when the operator is used in model scenarios, as well as generalized testcases generated according to boundary conditions.

2. 测试用例生成

2. Testcase Generation

MANDATORY: 在生成测试用例之前，必须读取以下参考文档：
  1. 必读:
  • templates/test-cases-template.md
    — 测试用例格式参考
  • design.md
    — 用户提供的算子设计文档
MANDATORY: Before generating testcases, you must read the following reference documents:
  1. Required Reading:
  • templates/test-cases-template.md
    — Testcase format reference
  • design.md
    — Operator design document provided by the user

2.1 设计原则

2.1 Design Principles

  1. 全 dtype 覆盖：每个 shape / 每个边界值都遍历算子支持的全部dtype，
    SUPPORTED_DTYPES
    必须包含
    design.md
    中[参数说明]支持的所有数据类型
  2. shape 由算子决定：根据算子支持的维度选择合适的 shape，不要写固定维度
  3. shape 不要过大：单个用例元素数控制在合理范围，避免不必要的大 tensor
  4. 用例总数 = (len(TEST_SHAPES) + len(GENERAL_SHAPES)) × len(SUPPORTED_DTYPES) ≥ 30
  1. Full dtype Coverage: Traverse all dtypes supported by the operator for each shape / each boundary value. 
    SUPPORTED_DTYPES
    must include all data types supported in the [Parameter Description] section of 
    design.md
  2. Shape Determined by Operator: Select appropriate shapes based on the dimensions supported by the operator; do not use fixed dimensions
  3. Avoid Oversized Shapes: Control the number of elements in a single testcase within a reasonable range to avoid unnecessary large tensors
  4. Total Number of Testcases = (len(TEST_SHAPES) + len(GENERAL_SHAPES)) × len(SUPPORTED_DTYPES) ≥ 30

Part A: 常规 Shape 测试（TEST_SHAPES）

Part A: Regular Shape Testing (TEST_SHAPES)

根据算子支持的维度以及算子类型，从以下维度池中选取适合的 shape，组成 
TEST_SHAPES
列表。
MUST 根据算子实际支持的维度来选，不支持的维度不要选。
Select suitable shapes from the following dimension pool based on the dimensions supported by the operator and operator type to form the 
TEST_SHAPES
list.
MUST select based on the actual dimensions supported by the operator; do not select unsupported dimensions.

shape 选择参考池

Shape Selection Reference Pool

维度推荐 shape适用算子类型
1D(128,), (1024,), (4096,), (8192,)elementwise, reduction
2D(32, 512), (64, 768), (128, 1024)elementwise, matmul, linear
3D(8, 16, 64), (4, 128, 256)elementwise, attention, conv1d
4D(4, 8, 32, 16), (2, 64, 32, 32)conv2d, elementwise
5D(2, 3, 4, 5, 6)conv3d, elementwise
shape 不要过大：推荐单个 shape 元素数 ≤ 200K。
DimensionRecommended ShapeApplicable Operator Type
1D(128,), (1024,), (4096,), (8192,)elementwise, reduction
2D(32, 512), (64, 768), (128, 1024)elementwise, matmul, linear
3D(8, 16, 64), (4, 128, 256)elementwise, attention, conv1d
4D(4, 8, 32, 16), (2, 64, 32, 32)conv2d, elementwise
5D(2, 3, 4, 5, 6)conv3d, elementwise
Avoid Oversized Shapes: It is recommended that the number of elements in a single shape ≤ 200K.

TEST_SHAPES 格式

TEST_SHAPES Format

python
TEST_SHAPES = [
    ("category_name", "description", (dim0, dim1, ...)),
    # ...
]
Category 名称自定义，建议按维度或场景命名。示例：
python
undefined
python
TEST_SHAPES = [
    ("category_name", "description", (dim0, dim1, ...)),
    # ...
]
Category names can be customized, and it is recommended to name them by dimension or scenario. Example:
python
undefined

elementwise 算子（支持任意维度）

elementwise operator (supports any dimension)

TEST_SHAPES = [ ("1D", "128 elements", (128,)), ("1D", "1024 elements", (1024,)), ("1D", "4096 elements", (4096,)), ("1D", "8192 elements", (8192,)), ("2D", "32x512", (32, 512)), ("3D", "8x16x64", (8, 16, 64)), ("3D", "4x128x256", (4, 128, 256)), ("4D", "4x8x32x16", (4, 8, 32, 16)), ]
undefined
TEST_SHAPES = [ ("1D", "128 elements", (128,)), ("1D", "1024 elements", (1024,)), ("1D", "4096 elements", (4096,)), ("1D", "8192 elements", (8192,)), ("2D", "32x512", (32, 512)), ("3D", "8x16x64", (8, 16, 64)), ("3D", "4x128x256", (4, 128, 256)), ("4D", "4x8x32x16", (4, 8, 32, 16)), ]
undefined

Part B: 泛化Shape 测试（GENERAL_SHAPES）

Part B: Generalized Shape Testing (GENERAL_SHAPES)

根据算子支持的维度以及算子类型，从以下维度池中选取适合的 shape，组成 
GENERAL_SHAPES
列表。
MUST 根据算子实际支持的维度来选，不支持的维度不要选。
Select suitable shapes from the following dimension pool based on the dimensions supported by the operator and operator type to form the 
GENERAL_SHAPES
list.
MUST select based on the actual dimensions supported by the operator; do not select unsupported dimensions.

shape 选择参考池

Shape Selection Reference Pool

小Shape场景（边界测试、极小值、非对齐测试）
维度小 shape测试目的
1D(1,), (2,), (4,), (8,)极小元素、边界值
1D(3,), (5,), (7,)非对齐长度
2D(1, 1), (2, 2), (4, 4)极小2D tensor
2D(1, 128), (2, 256)单行/单列场景
3D(1, 1, 1), (2, 2, 2)极小3D tensor
3D(1, 8, 64), (2, 16, 128)单batch场景
大Shape场景（生产环境、压力测试、模型典型场景）
维度大 shape适用模型元素数
1D(3072,), (4096,)BERT FFN中间层3K-4K
1D(5120,), (6400,)GPT-2 FFN中间层5K-6K
2D(512, 768)BERT-base full sequence393K
2D(512, 1024)BERT-large full sequence524K
2D(1024, 768)GPT-2 sequence786K
2D(1024, 1024)GPT-2 medium sequence1M
2D(1024, 1600)GPT-2 XL sequence1.6M
3D(8, 512, 768)BERT-base batch3.1M
3D(8, 197, 768)ViT-base batch1.2M
3D(16, 1024, 1024)GPT-2 large batch16.7M
注意事项：
  • 生产环境、模型典型场景推荐shape应视每个算子具体应用场景而变化
  • 小shape用于边界测试，确保算子在极小输入下正确工作
  • 大shape用于性能测试和验证大规模数据的正确性
  • 元素数超过200K的shape仅用于泛化测试，不在常规测试中使用
Small Shape Scenarios (Boundary Testing, Minimum Values, Misalignment Testing)
DimensionSmall ShapeTesting Purpose
1D(1,), (2,), (4,), (8,)Minimum elements, boundary values
1D(3,), (5,), (7,)Misaligned lengths
2D(1, 1), (2, 2), (4, 4)Tiny 2D tensor
2D(1, 128), (2, 256)Single row/column scenario
3D(1, 1, 1), (2, 2, 2)Tiny 3D tensor
3D(1, 8, 64), (2, 16, 128)Single batch scenario
Large Shape Scenarios (Production Environment, Stress Testing, Typical Model Scenarios)
DimensionLarge ShapeApplicable ModelNumber of Elements
1D(3072,), (4096,)BERT FFN middle layer3K-4K
1D(5120,), (6400,)GPT-2 FFN middle layer5K-6K
2D(512, 768)BERT-base full sequence393K
2D(512, 1024)BERT-large full sequence524K
2D(1024, 768)GPT-2 sequence786K
2D(1024, 1024)GPT-2 medium sequence1M
2D(1024, 1600)GPT-2 XL sequence1.6M
3D(8, 512, 768)BERT-base batch3.1M
3D(8, 197, 768)ViT-base batch1.2M
3D(16, 1024, 1024)GPT-2 large batch16.7M
Notes:
  • Recommended shapes for production environments and typical model scenarios should vary according to the specific application scenario of each operator
  • Small shapes are used for boundary testing to ensure the operator works correctly with minimal inputs
  • Large shapes are used for performance testing and verification of correctness with large-scale data
  • Shapes with more than 200K elements are only used for generalized testing and not in regular testing

GENERAL_SHAPES 格式

GENERAL_SHAPES Format

python
GENERAL_SHAPES = [
    ("Small", "description", (dim0, dim1, ...)),
    ("Large", "description", (dim0, dim1, ...)),
    # ...
]
Category 建议使用 "Small" 和 "Large" 区分场景。示例：
python
undefined
python
GENERAL_SHAPES = [
    ("Small", "description", (dim0, dim1, ...)),
    ("Large", "description", (dim0, dim1, ...)),
    # ...
]
It is recommended to use "Small" and "Large" as categories to distinguish scenarios. Example:
python
undefined

elementwise 算子（支持任意维度）

elementwise operator (supports any dimension)

GENERAL_SHAPES = [ # 小Shape场景 ("Small", "single element", (1,)), ("Small", "tiny vector 2", (2,)), ("Small", "tiny vector 4", (4,)), ("Small", "unaligned length 3", (3,)), ("Small", "unaligned length 5", (5,)), ("Small", "2x2 matrix", (2, 2)), ("Small", "1x128 single row", (1, 128)), ("Small", "1x1x1 scalar", (1, 1, 1)), ("Small", "1x8x64 single batch", (1, 8, 64)),
# 大Shape场景（生产环境）
("Large", "BERT-base FFN 3072",     (3072,)),
("Large", "BERT-large FFN 4096",    (4096,)),
("Large", "BERT-base 512x768",      (512, 768)),
("Large", "BERT-large 512x1024",    (512, 1024)),
("Large", "GPT-2 1024x768",         (1024, 768)),
("Large", "GPT-2 1024x1024",        (1024, 1024)),
("Large", "ViT-base 8x197x768",     (8, 197, 768)),
]
undefined
GENERAL_SHAPES = [ # Small Shape Scenarios ("Small", "single element", (1,)), ("Small", "tiny vector 2", (2,)), ("Small", "tiny vector 4", (4,)), ("Small", "unaligned length 3", (3,)), ("Small", "unaligned length 5", (5,)), ("Small", "2x2 matrix", (2, 2)), ("Small", "1x128 single row", (1, 128)), ("Small", "1x1x1 scalar", (1, 1, 1)), ("Small", "1x8x64 single batch", (1, 8, 64)),
# Large Shape Scenarios (Production Environment)
("Large", "BERT-base FFN 3072",     (3072,)),
("Large", "BERT-large FFN 4096",    (4096,)),
("Large", "BERT-base 512x768",      (512, 768)),
("Large", "BERT-large 512x1024",    (512, 1024)),
("Large", "GPT-2 1024x768",         (1024, 768)),
("Large", "GPT-2 1024x1024",        (1024, 1024)),
("Large", "ViT-base 8x197x768",     (8, 197, 768)),
]
undefined

2.2 设计用例输出

2.2 Design Testcase Output

基于收集的信息，读取 
templates/test-cases-template.md
模板，填充所有章节，输出到 
csrc/ops/[op-name]/test/[op-name]-test-cases.md
。
输出位置: 
ascend-kernel/csrc/ops/[op-name]/test/[op-name]-test-cases.md
Based on the collected information, read the 
templates/test-cases-template.md
template, fill in all sections, and output to 
csrc/ops/[op-name]/test/[op-name]-test-cases.md
.
Output Location: 
ascend-kernel/csrc/ops/[op-name]/test/[op-name]-test-cases.md

注意事项

Notes

  1. 严格遵循模板章节：只输出模板中定义的章节（算子标杆、典型用例、泛化用例、使用说明）
  2. 禁止输出无关内容：
    • ❌ 精度验证标准（由 
      ascendc-operator-precision-eval
      负责）
    • ❌ 性能验证标准（由 
      ascendc-operator-performance-eval
      负责）
    • ❌ UT测试代码（由其他skill负责）
    • ❌ 任何超出模板章节定义的内容
  3. 输出文件位置：
    ascend-kernel/csrc/ops/[op-name]/test/[op-name]-test-cases.md
  1. Strictly Follow Template Sections: Only output sections defined in the template (operator benchmark, typical testcases, generalized testcases, usage instructions)
  2. Prohibit Output of Irrelevant Content:
    • ❌ Precision verification standards (handled by 
      ascendc-operator-precision-eval
      )
    • ❌ Performance verification standards (handled by 
      ascendc-operator-performance-eval
      )
    • ❌ UT test code (handled by other skills)
    • ❌ Any content beyond the sections defined in the template
  3. Output File Location: 
    ascend-kernel/csrc/ops/[op-name]/test/[op-name]-test-cases.md