AscendC Operator Design Skill

Usage Scenarios

ascendc-operator-project-init

ascendc-operator-code-gen

Design Process

1. Operator Requirement Analysis

acosh

rms_norm

torch.acosh

torch.softmax

2. Select Implementation Path

New operators default to the AscendC Kernel path unless matrix multiplication is explicitly involved.

3. Generate Detailed Design Document

1. Required:

   templates/design-template.md

   — Design document template
2. Optional reading by operator type:
   • Element-wise operations (add/relu/acosh/sigmoid...) →

     references/elementwise-tiling.md

   • Reduction operations (softmax/layernorm...) →

     references/reduction-tiling.md

3. General Reference:

   references/general-tiling-principles.md

3.1 Design Document Structure

1. Operator Interface Definition — Function signature, parameter description, supported data types
2. Computation Logic Design — Algorithm description, AscendC API call pseudocode, implementation path selection
3. Tiling Strategy — Two-level Tiling design (Block-level + UB-level), UB allocation table, tileLength calculation
4. Workspace Requirements — Workspace size calculation
5. Performance Optimization — Key optimization points, operator characteristic analysis
6. Kernel-side Implementation Key Points — Offset calculation, execution process, FP16/BF16 precision lifting process
7. Implementation Checklist — File structure, code key points, test key points

3.2 Computation Logic Pseudocode (Key Output)

Add(dst, src0, src1, len)

Sub(dst, src0, src1, len)

Mul(dst, src0, src1, len)

Div(dst, src0, src1, len)

Adds(dst, src, scalar, len)

Muls(dst, src, scalar, len)

Abs(dst, src, len)

Exp(dst, src, len)

Ln(dst, src, len)

Sqrt(dst, src, len)

Reciprocal(dst, src, len)

Rsqrt(dst, src, len)

Tanh(dst, src, len)

Relu(dst, src, len)

Max(dst, src0, src1, len)

Min(dst, src0, src1, len)

Cast(dst, src, CAST_NONE, len)

Cast(dst, src, CAST_ROUND, len)

Mul(tmp, x, x, len);        // tmp = x²
Adds(tmp, tmp, -1.0f, len); // tmp = x² - 1
Sqrt(tmp, tmp, len);         // tmp = sqrt(x² - 1)
Add(tmp, tmp, x, len);      // tmp = x + sqrt(x² - 1)
Ln(y, tmp, len);             // y = ln(x + sqrt(x² - 1))

Note: If the dst and src are the same in a certain step of the computation sequence (in-place operation), most AscendC APIs support this, but you need to confirm the specific API.

3.2 Tiling Strategy

┌─────────────────────────────────────────────────────────────┐
│                    Global Memory (GM)                        │
│  ┌─────────────────────────────────────────────────────┐   │
│  │              Data of totalLength elements               │   │
│  └─────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘
                           │
          ┌────────────────┼────────────────┐
          ▼                ▼                ▼
    ┌──────────┐     ┌──────────┐     ┌──────────┐
    │  Core 0  │     │  Core 1  │ ... │ Core 39  │   ← Block-level Tiling (Inter-core Splitting)
    └──────────┘     └──────────┘     └──────────┘
          │                │                │
          ▼                ▼                ▼
    ┌──────────┐     ┌──────────┐     ┌──────────┐
    │   UB 0   │     │   UB 1   │     │  UB 39   │   ← UB-level Tiling (Intra-core Splitting)
    └──────────┘     └──────────┘     └──────────┘

• Allocate data to multiple AI Cores for parallel processing
• Load Balancing: Full-core/tail-core strategy, the data volume processed by the tail core is less than or equal to that of the full core

• Split data for processing within each Core
• UB Alignment: 32 bytes
• UB Capacity: Do not exceed UB_SIZE_LIMIT (obtained via interface during actual coding, example value: 192KB)

• Element-wise Operations: Read

  references/elementwise-tiling.md

  (includes complete two-level Tiling implementation)
• Reduction Operations: Read

  references/reduction-tiling.md

• General Principles: Refer to

  references/general-tiling-principles.md

3.3 Hardware Constraint Description

• UB Buffer: Must be aligned to 32 bytes, even if only a small amount of data needs to be stored logically
• Reduction Operators: A single-value buffer requires 32B of space
• Precision Processing:
  • FP32 Input: No precision lifting needed, compute directly
  • FP16 Input: Must lift precision to FP32 for computation to ensure calculation accuracy
  • BF16 Input: Must lift precision to FP32 for computation, vector computation units do not support direct bfloat16 computation
• Workspace Requirements:
  • Elementwise Operators: SYSTEM_WORKSPACE_SIZE (usually 16MB)
  • Other Operator Types: Calculate based on actual tiling data size

3.4 Quick Reference Table for UB Allocation of Common Operator Types

Practical Experience: bufferCoefficient is the most critical parameter in the code-gen phase. The design document must clearly specify the value for each dtype, otherwise code generation cannot calculate tileLength correctly.

3.5 Generate Design Document

templates/design-template.md

csrc/ops/<op_name>/design.md

ascend-kernel/csrc/ops/<op_name>/design.md

Interaction Process

1. Receive operator name and function description
2. Automatically select implementation path
3. Read reference documents and generate complete design document
4. Output to design.md

1. Requirement Collection: Understand operator requirements through dialogue
2. Solution Recommendation: Recommend implementation path based on requirements
3. Detailed Design: Generate complete design document
4. Check and Confirm: Confirm design key points with the user
5. Handover to Development: Generate checklist and prepare for coding phase

Notes

Tiling Parameter Design Principles

1. Parameter Structuring:
   cpp

   // Good practice: Use struct
   struct MyOperatorTilingData {
       int64_t totalLength;        // Total data length
   
       int64_t formerNum;          // Number of full cores
       int64_t formerLength;       // Data length per full core
       int64_t tailNum;            // Number of tail cores
       int64_t tailLength;         // Data length per tail core
   
       int64_t tileLength;         // UB single processing length
   };
   
   // Avoid: Use a large number of independent parameters
   void KernelFunc(int64_t totalLength, int64_t tileNum, int64_t tileLength, ...);

2. Two-level Alignment:
   cpp

   // Block-level: Cache Line alignment (512 bytes)
   constexpr int64_t CACHE_LINE_BYTE_LENGTH = 512;
   int64_t totalLengthCoreAlign = ((totalLengthCore + CACHE_LINE_BYTE_LENGTH - 1) / CACHE_LINE_BYTE_LENGTH) * CACHE_LINE_BYTE_LENGTH;
   
   // UB-level: 32-byte alignment
   int64_t ubAlignElements = 32 / dtypeSize;
   int64_t tileLengthAligned = ((tileLength + ubAlignElements - 1) / ubAlignElements) * ubAlignElements;

3. UB Allocation Table: Each operator design must include a UB allocation table, clearly listing:
   • Names and purposes of all buffers
   • Size (in bytes) of each buffer
   • Number of buffers (single buffer or double buffer)
   • Total UB usage and constraint verification
4. Double Buffer: Use BUFFER_NUM=2 to implement double buffer and hide memory latency

Other Notes

1. Data Type Alignment: Ensure PyTorch tensor types match AscendC kernel types (half ↔ float16, float ↔ float32)
2. Memory Alignment: AscendC requires memory address alignment (UB 32B, Cache Line 512B)
3. Shape Constraints: Some operators have special shape requirements (e.g., need to be divisible by tile size)
4. Performance Trade-off: Find a balance between code complexity and performance
5. Interface Definition: Check if similar operator interfaces exist in libraries like PyTorch/Numpy. If they exist, refer to PyTorch/Numpy for interface definition
6. Test Input Range: Specify the valid input range of the operator in the design document (e.g., acosh requires x >= 1), and test cases should generate data accordingly

Definition of Done (DoD)

• Function Signature: Complete declaration of

  at::Tensor op_name(const at::Tensor &self, ...)

• Supported Data Types: Clearly listed (e.g., float16, float32)
• AscendC API Call Pseudocode: Map each computation step to a specific API
• UB Allocation Table: Buffer layout and bufferCoefficient for each dtype
• Tiling Parameter Struct: Field definitions and calculation formulas
• FP16/BF16 Precision Lifting Process: If half-precision is supported, the Cast path must be described

Next Step

ascendc-operator-code-gen