coral-create-source-spec

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Create Source Spec

创建源规格

Use this skill when the task is to author or repair a Coral source spec.
当需要编写或修复Coral源规格时,使用此技能。

Goal

目标

Produce a valid, queryable Coral source spec that works with:
  • coral source lint <path>
  • coral source add --file <path>
  • coral source test <name>
  • coral sql
  • coral.tables
    and
    coral.columns
  • coral.inputs
    for source variables and secrets
生成有效的、可查询的Coral源规格,使其可用于:
  • coral source lint <path>
  • coral source add --file <path>
  • coral source test <name>
  • coral sql
  • coral.tables
    coral.columns
  • coral.inputs
    (用于源变量和密钥)

Default Mode

默认模式

Default to standalone source authoring for external developers.
That means:
  • create a YAML source spec file
  • lint it early with
    coral source lint <path>
  • add it to Coral with
    coral source add --file <path>
    when you need to exercise it as a source
  • validate by querying it
  • iterate until the shape is correct
Only switch to repo-bundled layout when the user is explicitly editing the Coral repo.
默认面向外部开发者采用独立源编写模式。
这意味着:
  • 创建一个YAML源规格文件
  • 尽早使用
    coral source lint <path>
    进行校验
  • 当需要将其作为源使用时,通过
    coral source add --file <path>
    添加到Coral中
  • 通过查询验证规格
  • 迭代调整直到结构正确
仅当用户明确编辑Coral仓库时,才切换为仓库捆绑布局模式。

Output Modes

输出模式

  • External authoring:
    • create a standalone source spec such as
      ./my-source.yaml
    • validate structure with
      coral source lint ./my-source.yaml
    • load it with
      coral source add --file ./my-source.yaml
      when you need to query it through Coral
  • Coral repo contribution:
    • write the source spec to
      sources/<name>/manifest.yaml
    • add representative
      test_queries
      for a basic smoke/connection check of the source
    • validate with
      coral source test <name>
      and repo checks
  • 外部编写模式:
    • 创建独立源规格文件,例如
      ./my-source.yaml
    • 使用
      coral source lint ./my-source.yaml
      验证结构
    • 当需要通过Coral查询时,使用
      coral source add --file ./my-source.yaml
      加载
  • Coral仓库贡献模式:
    • 将源规格写入
      sources/<name>/manifest.yaml
    • 添加代表性的
      test_queries
      ,用于对源进行基础的连通性冒烟测试
    • 使用
      coral source test <name>
      和仓库校验工具验证

Workflow

工作流程

  1. Read the provider API docs or inspect the local dataset.
  2. Start with one small table and a few columns.
  3. Define:
    • source metadata
    • backend
    • base URL or file location
    • auth
    • variables and secrets
    • tables
    • filters
    • response extraction
    • pagination
    • typed columns
  4. Lint the source spec:
    • coral source lint <path>
  5. Validate the source in the right mode:
    • standalone specs:
      coral source add --file <path>
      and inspect with
      coral sql
    • coral source add
      is non-interactive by default: each input
      key
      is read from the matching environment variable. Export required variables and secrets before running, or pass
      --interactive
      to be prompted.
    • named or repo-bundled sources:
      coral source test <name>
  6. Inspect the exposed shape:
    • inspect
      coral.tables
    • inspect
      coral.columns
    • inspect
      coral.inputs
      to verify variables, secrets, defaults, hints, and required flags
  7. Query representative tables with
    coral sql
    .
  8. If you are relying on
    coral source test
    , make sure
    test_queries
    gives you a basic smoke/connection check for the source.
  9. Refine the spec and repeat.
  1. 阅读提供商API文档或检查本地数据集。
  2. 从一个小表和少量列开始。
  3. 定义以下内容:
    • 源元数据
    • 后端
    • 基础URL或文件位置
    • 认证方式
    • 变量和密钥
    • 过滤器
    • 响应提取规则
    • 分页规则
    • 带类型的列
  4. 校验源规格:
    • coral source lint <path>
  5. 在对应模式下验证源:
    • 独立规格:使用
      coral source add --file <path>
      并通过
      coral sql
      检查
    • 默认情况下
      coral source add
      是非交互式的:每个输入
      key
      会从匹配的环境变量中读取。运行前导出所需变量和密钥,或传递
      --interactive
      参数以获取提示。
    • 命名源或仓库捆绑源:使用
      coral source test <name>
  6. 检查暴露的结构:
    • 查看
      coral.tables
    • 查看
      coral.columns
    • 查看
      coral.inputs
      以验证变量、密钥、默认值、提示和必填标记
  7. 使用
    coral sql
    查询代表性的表。
  8. 如果依赖
    coral source test
    ,确保
    test_queries
    能对源进行基础的连通性冒烟测试。
  9. 优化规格并重复上述步骤。

Authoring Rules

编写规则

  • Start small and expand table coverage incrementally.
  • Use the source manifest schema as both inspiration for authoring and validation of structure: https://github.com/withcoral/coral/blob/main/crates/coral-spec/src/schema/source_manifest.schema.json
  • Use source variables for non-secret configuration.
  • Use source secrets for credentials.
  • Keep table names stable and SQL-friendly.
  • Mark filters as required only when the API truly requires them.
  • Prefer explicit pagination when the API shape is known.
  • Verify pagination with actual row fetches, not only
    COUNT(*)
    .
  • Add or update
    test_queries
    when you want
    coral source test
    to perform a basic smoke/connection check.
  • 从小规模开始,逐步扩展表的覆盖范围。
  • 将源清单架构作为编写灵感和结构验证依据:https://github.com/withcoral/coral/blob/main/crates/coral-spec/src/schema/source_manifest.schema.json
  • 使用源变量存储非机密配置。
  • 使用源密钥存储凭证。
  • 保持表名称稳定且符合SQL规范。
  • 仅当API确实要求时,才将过滤器标记为必填项。
  • 当API分页模式已知时,优先使用显式分页配置。
  • 通过实际获取行数据来验证分页,而不仅仅依赖
    COUNT(*)
  • 当需要
    coral source test
    执行基础连通性冒烟测试时,添加或更新
    test_queries

Metadata UX Rules

元数据UX规则

Use these rules for top-level source metadata so source discovery and setup are consistent.
遵循以下顶级源元数据规则,确保源发现和设置的一致性。

description

description

  • Start with
    Query ...
    .
  • Make the first sentence capability-first: list the key entities users can query.
  • Preferred template:
    • Query <entities> from <Provider> (<Cloud or self-hosted when relevant>).
  • Keep
    description
    focused on data coverage, not setup steps.
  • Do not use vague phrasing such as:
    • REST API v3
    • OpenAPI provider
    • ... and more
  • Move auth/setup/permission details to input hints, not description text.
  • Query ...
    开头。
  • 第一句以能力为核心:列出用户可查询的关键实体。
  • 推荐模板:
    • Query <entities> from <Provider> (<Cloud or self-hosted when relevant>).
  • description
    聚焦于数据覆盖范围,而非设置步骤。
  • 避免使用模糊表述,例如:
    • REST API v3
    • OpenAPI provider
    • ... and more
  • 将认证/设置/权限细节移至输入提示,而非描述文本中。

Input hints (
inputs.<KEY>.hint
)

输入提示(
inputs.<KEY>.hint

Each hint should tell the user:
  • what value is expected
  • how to obtain it
  • minimum scope/permission guidance
  • one concrete format/example when useful
Specific guidance:
  • For URL/base inputs:
    • say what the default means
    • include at least one concrete example
    • include self-hosted guidance when supported
  • For secrets:
    • name the exact credential type (API key, PAT, application key, etc.)
    • include format constraints when relevant (for example, token prefixes)
    • include least-privilege scope guidance
  • For derived secrets (for example Basic auth blobs):
    • include a short shell example (for example a Base64 command)
  • Prefer stable documentation links.
    • Use official docs links and stable settings pages.
    • Avoid brittle click-path instructions as the primary guidance.
Keep hints concise and directly actionable.
每个提示应告知用户:
  • 预期的值是什么
  • 如何获取该值
  • 最小权限范围指导
  • 必要时提供一个具体格式/示例
具体指导:
  • 对于URL/基础输入:
    • 说明默认值的含义
    • 至少包含一个具体示例
    • 支持自托管时包含相关指导
  • 对于密钥:
    • 指定确切的凭证类型(API密钥、PAT、应用密钥等)
    • 相关时包含格式约束(例如令牌前缀)
    • 包含最小权限范围指导
  • 对于派生密钥(例如Basic认证Blob):
    • 包含简短的Shell示例(例如Base64命令)
  • 优先使用稳定的文档链接。
    • 使用官方文档链接和稳定的设置页面。
    • 避免将脆弱的点击路径说明作为主要指导。
保持提示简洁且直接可操作。

Validation Loop

验证循环

Use this loop during authoring:
sh
undefined
编写过程中使用以下循环:
sh
undefined

Export any required inputs first (key matches the input
key
in the spec),

先导出所有必填输入(key与规格中的输入
key
匹配),

or pass --interactive to be prompted.

或传递--interactive参数以获取提示。

coral source lint ./my-source.yaml coral source add --file ./my-source.yaml coral sql "SELECT * FROM coral.tables WHERE schema_name = 'my_source'" coral sql "SELECT * FROM coral.columns WHERE schema_name = 'my_source'" coral sql "SELECT key, kind, value, default_value, hint, required, is_set FROM coral.inputs WHERE schema_name = 'my_source' ORDER BY key"

For repo-bundled or already-named sources, add `test_queries` for a basic smoke/connection check and run:

```sh
coral source test my_source
Then run targeted table queries until the source behaves correctly.
coral source lint ./my-source.yaml coral source add --file ./my-source.yaml coral sql "SELECT * FROM coral.tables WHERE schema_name = 'my_source'" coral sql "SELECT * FROM coral.columns WHERE schema_name = 'my_source'" coral sql "SELECT key, kind, value, default_value, hint, required, is_set FROM coral.inputs WHERE schema_name = 'my_source' ORDER BY key"

对于仓库捆绑源或已命名的源,添加`test_queries`进行基础连通性冒烟测试并运行:

```sh
coral source test my_source
然后运行针对性的表查询,直到源行为符合预期。

HTTP Sources

HTTP源

For HTTP-backed sources:
  • define
    backend: http
  • define
    base_url
  • define auth headers
  • define request path, query, and body only where needed
  • define response
    rows_path
  • define pagination explicitly when the provider pattern is known
  • define typed columns
  • add
    test_queries
    once you know which simple query or queries should confirm the source basically works
Read
references/http-source-checklist.md
when you need table-shape and pagination guidance.
If your HTTP source uses an Authorization header with a prefix (e.g.
Authorization: Bearer <token>
), you can use a secret input for the token and define the header as a template:
yaml
auth:
  headers:
    - name: Authorization
      from: template
      template: Bearer {{input.FOOBAR_API_KEY}}
对于基于HTTP的源:
  • 定义
    backend: http
  • 定义
    base_url
  • 定义认证头
  • 仅在需要时定义请求路径、查询参数和请求体
  • 定义响应
    rows_path
  • 当提供商分页模式已知时,显式定义分页规则
  • 定义带类型的列
  • 确定哪个简单查询可确认源基本可用后,添加
    test_queries
需要表结构和分页指导时,阅读
references/http-source-checklist.md
如果你的HTTP源使用带前缀的Authorization头(例如
Authorization: Bearer <token>
),你可以为令牌使用密钥输入,并将头定义为模板:
yaml
auth:
  headers:
    - name: Authorization
      from: template
      template: Bearer {{input.FOOBAR_API_KEY}}

Local Data Sources

本地数据源

For local file-backed sources:
  • define the file backend
  • define the source location
  • define file selection patterns if applicable
  • define typed columns
对于基于本地文件的源:
  • 定义文件后端
  • 定义源位置
  • 适用时定义文件选择模式
  • 定义带类型的列

Deliverable

交付物

Report:
  • source spec path
  • lint / add / test commands used
  • validation commands run
  • assumptions made
  • blocked or unverified endpoints
报告内容包括:
  • 源规格路径
  • 使用的校验/添加/测试命令
  • 运行的验证命令
  • 做出的假设
  • 阻塞或未验证的端点