authoring-mwaa-workflow
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseAuthoring MWAA Workflows
编写 MWAA 工作流
AWS MCP server (optional but recommended): running the AWS CLI commands in this skill through the AWS MCP server gives sandboxed execution and audit logging. Every command here also works with the plain AWS CLI, so the skill does not require the MCP server or any MCP-only tools.
Author production-grade workflow artifacts for Amazon MWAA. Routes to one
of two paths: Python DAG (provisioned) or YAML workflow (Serverless).
Execution note — poll in discrete steps: whenever you wait for an AWS operation to reach a terminal or ready state, issue one status check per call and decide in your own loop whether to check again. Never block a single command or script on the wait (no+whileuntil done), regardless of the operation or how long it takes.sleep
**AWS MCP server(可选但推荐):**通过 AWS MCP server 运行此 skill 中的 AWS CLI 命令可获得沙箱化执行和审计日志。此处每条命令 也可使用普通 AWS CLI 运行,因此该 skill 不需要 MCP server 或任何 仅限 MCP 的工具。
为 Amazon MWAA 编写生产级工作流工件。编写流程会选择两条路径之一:
Python DAG(provisioned)或 YAML 工作流(Serverless)。
**执行说明——分步轮询:**每当需要等待 AWS 操作达到终态或就绪状态时,每次调用仅执行 一次状态检查, 并自行在循环中判断是否再次检查。绝不要让单条命令或脚本 阻塞等待完成(不要使用+while一直等到结束), 无论操作是什么或需要多长时间。sleep
Guardrail — where this skill's own files live (MCP vs local install)
防护规则——此 skill 自带的文件位于何处(MCP 与本地安装)
This skill can be loaded two ways, and they resolve the skill's own bundled
files from different places. Determine how the skill was loaded before reading
a reference:
- Loaded through the AWS MCP tool: The skill is not installed on the local filesystem. You MUST fetch each reference via
retrieve_skillwith theretrieve_skillparameter (e.g.file) and read the returned content. Do NOTfile="references/authoring-provisioned-dag.md"these paths locally — they do not exist on disk.file_read - Installed locally (e.g. or
.kiro/skills/authoring-mwaa-workflow/): Read the files from the local skill directory using relative paths.~/.claude/skills/authoring-mwaa-workflow/
This distinction applies only to the skill's own packaged files. User data and
session artifacts are always read from and written to the user's working
directory. Never fetch or write customer data through .
retrieve_skill此 skill 有两种加载方式,它们会从不同位置解析该 skill 自带的捆绑文件。读取参考文件前,先确定此 skill 的加载方式:
- **通过 AWS MCP 工具加载:**此 skill 未安装在本地文件系统中。必须通过
retrieve_skill并使用retrieve_skill参数获取每个参考文件(例如file),然后读取返回的内容。不要在本地对这些路径使用file="references/authoring-provisioned-dag.md"——这些文件在磁盘上不存在。file_read - 本地安装(例如 或
.kiro/skills/authoring-mwaa-workflow/):使用相对路径从本地 skill 目录读取文件。~/.claude/skills/authoring-mwaa-workflow/
此区别仅适用于该 skill 自带的打包文件。用户数据和会话工件始终从用户工作目录读取,并写入用户工作目录。绝不要通过 获取或写入客户数据。
retrieve_skillStep 0: Route to Path
第 0 步:选择路径
Evaluate in this order:
- Resolvable target provided? A target reference is a definitive
routing signal regardless of other keywords:
- A provisioned environment (ARN like
, or a name resolvable via
arn:aws:airflow:<region>:<account>:environment/<name>) → go to Path A.aws mwaa get-environment - A Serverless workflow ARN
() → go to Path B.
arn:aws:airflow-serverless:<region>:<account>:workflow/<name>
- A provisioned environment (ARN like
- Both-path keywords present? If the request contains keywords from
both paths and no resolvable target, disambiguate by intent:
- is supported on Serverless, so an operator-level
PythonOperatorcue (python,PythonOperator, "Python function/task") alongside a Path B signal (python_callable,yaml, workflow ARN) is NOT ambiguous → go to Path B.serverless - Conversion context ("convert my Python DAG to serverless") → this skill does not apply; conversion is out of scope.
- A genuine provisioned cue (,
Python DAG) alongside a Serverless cue with no target → ask the clarifying question.provisioned
- Exactly one path keyword? Treat only deployment-target terms as Path A
signals: ,
provisioned, or "aPython DAGfor my environment" → go to Path A..pyoryaml→ go to Path B. Operator-level Python mentions are not Path A signals.serverless - No routing signal? "DAG" or "Workflow" alone is ambiguous — it does NOT indicate a path. Ask: is the target MWAA provisioned (Python DAG) or MWAA Serverless (YAML)?
按以下顺序评估:
- **提供了可解析的目标?**无论其他关键词如何,目标引用都是确定路由的信号:
- provisioned 环境(ARN 格式为
,或名称可通过
arn:aws:airflow:<region>:<account>:environment/<name>解析)→ 路径 A。aws mwaa get-environment - Serverless workflow ARN
() → 路径 B。
arn:aws:airflow-serverless:<region>:<account>:workflow/<name>
- provisioned 环境(ARN 格式为
- **同时出现两条路径的关键词?**如果请求同时包含两条路径的关键词且没有可解析的目标,则根据意图消除歧义:
- 在 Serverless 上受支持,因此 operator 级别的
PythonOperator提示(python、PythonOperator、“Python function/task”)与路径 B 信号(python_callable、yaml、workflow ARN)同时出现,并不存在歧义 → 路径 B。serverless - 转换语境(“将我的 Python DAG 转换为 serverless”)→ 此 skill 不适用;转换不在范围内。
- 真正的 provisioned 提示(“Python DAG”、“provisioned”)与 Serverless 提示同时出现且没有目标 → 询问澄清问题。
- **恰好出现一个路径关键词?**仅将部署目标术语视为路径 A 信号:、
provisioned或“为我的环境提供一个Python DAG”→ 路径 A。.py或yaml→ 路径 B。operator 级别的 Python 提及不属于路径 A 信号。serverless - 没有路由信号?仅出现 “DAG” 或 “Workflow” 存在歧义——它不会指示路径。询问:目标是 MWAA provisioned(Python DAG) 还是 MWAA Serverless(YAML)?
Paths
路径
Follow the reference for the path you routed to (you do not need the other path's reference):
- Path A — Python DAG (MWAA Provisioned): references/authoring-provisioned-dag.md
- Path B — YAML Workflow (MWAA Serverless): references/authoring-serverless-workflow.md
After the routed path's Write step, continue with Deploy & Test below.
遵循为所选路径提供的参考文件(无需另一条路径的参考文件):
- 路径 A——Python DAG(MWAA Provisioned): references/authoring-provisioned-dag.md
- 路径 B——YAML 工作流(MWAA Serverless): references/authoring-serverless-workflow.md
完成所选路径的 编写 步骤后,继续执行下方的 部署与测试。
Deploy & Test (optional, after Write)
部署与测试(可选,在编写之后)
Authoring owns all deployment and redeployment. Detail in
references/deploying-mwaa.md.
编写阶段负责所有部署和重新部署。详细信息见
references/deploying-mwaa.md。
Steps
步骤
-
Ask — present options based on whether the artifact has a schedule. Frame the question using path-appropriate language:
- Provisioned: "deploy this DAG to an environment" (DAGs are uploaded to an environment's S3 bucket).
- Serverless: "deploy this workflow" (workflows are standalone resources — never say "deploy to an environment").
If the DAG/workflow has a schedule:- Deploy and test — deploy, unpause, trigger a run now
- Deploy and unpause — deploy, unpause, let it run on schedule (no immediate trigger)
- Deploy only — upload to S3, leave paused
If the DAG/workflow has no schedule (manual-trigger only):- Deploy and test — deploy, trigger a run now
- Deploy only — upload to S3, leave paused (no "unpause" option — nothing to schedule)
The user may also decline all options. -
Deploy:
- Provisioned: upload the DAG to the environment's /
SourceBucketArn. Run post-deploy verification (see deploying-mwaa.md) to confirm the scheduler parsed the new file without import errors or dag_id conflicts. If no environment exists and the user approves, create one inline (plan-validate-execute + explicit confirmation), then pollDagS3Path->CREATING(~20-40 min). The user may instead supply an existing environment.AVAILABLE - Serverless: (new) or
CreateWorkflow(redeploy); the YAML is validated synchronously here. If the workflow usesUpdateWorkflow/PythonOperator, first build and upload the code package to S3 and pass it viaBashOperator(see references/serverless-code-packaging.md and references/deploying-mwaa.md). The user may instead supply an existing ARN.--code - Redeploy (fix loop): the same upload / path, reused when testing-mwaa-workflow delegates an ARTIFACT or ENVIRONMENT fix.
UpdateWorkflow
- Provisioned: upload the DAG to the environment's
-
If "Deploy and unpause" selected — deploy per step 2, then unpause. Do not trigger a run or invoke testing-mwaa-workflow.
-
If "Deploy and test" selected — deploy per step 2, unpause if applicable, then invoke testing-mwaa-workflow with the resolved target (env name +, or workflow ARN). That hand-off is testing's delegated invocation mode.
dag_id
HARD GATE: If testing is requested — whether upfront ("deploy and test") or later in the conversation ("test it", "run it", "try it") — you MUST invoke testing-mwaa-workflow. Do NOT trigger, monitor, or verify DAG runs manually. "Deploy and unpause" is NOT a test request — it is a deploy-only action.
- Production safety: ,
create-environment,update-environment, andcreate-workflowmutate state — confirm each with its impact stated. Warn on prod-named targets.update-workflow
-
询问——根据工件是否具有调度计划给出选项。使用与路径相符的语言表述问题:
- Provisioned:“将此 DAG 部署到环境”(DAG 会上传到环境的 S3 bucket)。
- Serverless:“部署此 workflow”(workflow 是独立资源——绝不要说“部署到环境”)。
如果 DAG/workflow 有调度计划:- 部署并测试——部署、取消暂停(unpause)、立即触发一次运行
- 部署并取消暂停——部署、取消暂停,让它按调度计划运行(不立即触发)
- 仅部署——上传到 S3,保持暂停(paused)
如果 DAG/workflow 没有调度计划(仅手动触发):- 部署并测试——部署、立即触发一次运行
- 仅部署——上传到 S3,保持暂停(没有“取消暂停(unpause)”选项——没有可调度内容)
用户也可以拒绝所有选项。 -
部署:
- **Provisioned:**将 DAG 上传到环境的 /
SourceBucketArn。执行部署后验证(见DagS3Path),确认 scheduler 已解析新文件,且没有导入错误或deploying-mwaa.md冲突。如果不存在环境且用户批准,则内联创建一个(plan-validate-execute + 显式确认),随后轮询dag_id->CREATING(约 20–40 分钟)。用户也可以改为提供现有环境。AVAILABLE - Serverless:(新建)或
CreateWorkflow(重新部署);此处会同步验证 YAML。如果 workflow 使用UpdateWorkflow/PythonOperator,请先构建代码包并上传到 S3,再通过BashOperator传入(见 references/serverless-code-packaging.md 和 references/deploying-mwaa.md)。用户也可以改为提供现有 ARN。--code - **重新部署(修复循环):**使用相同的上传/路径;当 testing-mwaa-workflow 委派 ARTIFACT 或 ENVIRONMENT 修复时,复用该路径。
UpdateWorkflow
- **Provisioned:**将 DAG 上传到环境的
-
如果选择“部署并取消暂停”——按步骤 2 部署,然后取消暂停。不要触发运行,也不要调用 testing-mwaa-workflow。
-
如果选择“部署并测试”——按步骤 2 部署,在适用时取消暂停,然后使用解析后的目标(环境名称 +,或 workflow ARN)调用 testing-mwaa-workflow。该移交是 testing 的委托调用模式。
dag_id
**硬性门禁:**如果请求测试——无论是开始时提出(“部署并测试”)还是在对话后续提出(“测试它”、“运行它”、“尝试一下”)——你必须调用 testing-mwaa-workflow。不要手动触发、监控或验证 DAG 运行。“部署并取消暂停”不是测试请求——它只是仅部署操作。
生产安全:、create-environment、update-environment和create-workflow会改变状态——每次操作都必须明确说明影响并获得确认。对名称中含 prod 的目标发出警告。update-workflow
Troubleshooting
故障排查
| Error | Cause | Fix |
|---|---|---|
| dagrun_timeout kills DAG early | < timeout set in service called | Raise dagrun_timeout or lower service timeout |
| YAML validation rejects workflow | Wrong type or param | Use timedelta format; check allowlist |
| Operator not found in Serverless | Not allowlisted | Use a supported operator, PythonOperator/BashOperator, or Lambda |
| Serverless run: cannot extract code / corrupt env | Bad code package | Files at zip root, no |
| Serverless Python task ImportError | Missing dep or wrong-platform wheel | Bundle as |
| Template variable undefined | Version mismatch | Check vars for exact Airflow version |
| 错误 | 原因 | 修复 |
|---|---|---|
| 所调用服务中设置的 timeout 更短 | 提高 |
| YAML 验证拒绝 workflow | 类型或参数错误 | 使用 timedelta 格式;检查 allowlist |
| Serverless 中找不到 Operator | 未加入 allowlist | 使用受支持的 Operator: |
| Serverless 运行:无法提取代码 / 环境损坏 | 代码包有问题 | 文件放在 zip 根目录,不含 |
| Serverless Python task 出现 ImportError | 缺少依赖或 wheel 平台不匹配 | 打包为 |
| 模板变量未定义 | 版本不匹配 | 检查变量,确保与确切的 Airflow 版本匹配 |
References
参考资料
- references/dag-patterns.md — Python DAG templates
- references/yaml-schema.md — MWAA Serverless format
- references/deploying-mwaa.md — deploy, inline env creation, and redeploy
- references/serverless-code-packaging.md — Python/Bash code packaging, pre-installed packages, limits
- references/authoring-provisioned-dag.md — Path A: provisioned Python-DAG authoring steps (A1–A7)
- references/authoring-serverless-workflow.md — Path B: Serverless YAML authoring steps (B1–B5)
- references/dag-patterns.md——Python DAG 模板
- references/yaml-schema.md——MWAA Serverless 格式
- references/deploying-mwaa.md——部署、内联环境创建和重新部署
- references/serverless-code-packaging.md——Python/Bash 代码打包、预装包、限制
- references/authoring-provisioned-dag.md——路径 A:provisioned Python DAG 编写步骤(A1–A7)
- references/authoring-serverless-workflow.md——路径 B:Serverless YAML 编写步骤(B1–B5)
Security Considerations
安全注意事项
- State-mutating operations (/
create,update-environment/create, S3 DAG upload) require explicit confirmation with impact stated; warn on prod-named targets (see the Deploy HARD-GATE).update-workflow - Least-privilege IAM: the A5 check adds only the exact /
Actionpairs the artifact needs — neverResourceor*FullAccess.service:* - No hardcoded secrets/endpoints: use Airflow Variables/Connections backed by Secrets Manager or SSM Parameter Store; never emit credentials in DAG code or CLI examples.
- Serverless code packages ship only the user's own modules plus pinned, platform-matched wheels — no unreviewed third-party binaries.
- Data protection: keep sensitive data out of SNS/CloudWatch notification payloads and logs; rely on their encryption.
- Secure defaults for inline-created resources: encrypt and lock down any S3/MWAA/SNS/CloudWatch resource this skill creates — see deploying-mwaa.md "Secure defaults".
- AWS security best practices: verify the security posture against the MWAA User Guide's Security best practices page (and the MWAA Serverless equivalent) at runtime — AWS updates them over time; see deploying-mwaa.md "Secure defaults".
- 改变状态的操作(/
create、update-environment/create、S3 DAG 上传)需要获得明确确认并说明影响;对 名称中含 prod 的目标发出警告(见“部署”部分的硬性门禁)。update-workflow - 最小权限 IAM:A5 检查只添加工件所需的准确 /
Action配对——绝不使用Resource或*FullAccess。service:* - 不硬编码密钥/端点:使用由 Secrets Manager 或 SSM Parameter Store 支持的 Airflow Variables/Connections;绝不在 DAG 代码或 CLI 示例中 输出凭据。
- Serverless 代码包仅包含用户自己的模块以及固定版本、平台匹配的 wheels——不包含未经审查的第三方二进制文件。
- 数据保护:不要在 SNS/CloudWatch 通知内容和日志中放入敏感数据;依靠其加密。
- 内联创建资源的安全默认值:加密并严格限制此 skill 创建的任何 S3/MWAA/SNS/CloudWatch 资源的访问——见 references/deploying-mwaa.md 中的“Secure defaults”。
- AWS 安全最佳实践:运行时对照 MWAA User Guide 的 Security best practices 页面(以及 MWAA Serverless equivalent) 验证安全态势——AWS 会随时间更新这些内容;见 references/deploying-mwaa.md 中的“Secure defaults”。