omni-to-databricks-metric-view
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseOmni → Databricks Metric View
Omni → Databricks Metric View
Converts an Omni topic into a Databricks Metric View by exploring the Omni model via API, translating its field definitions into the Databricks Metric View embedded YAML format, and executing via the Databricks CLI.
See FIELD-MAPPING.md for full before/after translation examples and YAML-REFERENCE.md for the complete YAML structure, aggregate type, and format mapping tables.
通过API探索Omni模型,将其字段定义转换为Databricks Metric View的嵌入式YAML格式,并通过Databricks CLI执行,从而将Omni主题转换为Databricks Metric View。
查看FIELD-MAPPING.md获取完整的转换前后示例,查看YAML-REFERENCE.md获取完整的YAML结构、聚合类型和格式映射表。
Prerequisites
前提条件
bash
undefinedbash
undefinedVerify the Omni CLI is installed — if not, ask the user to install it
Verify the Omni CLI is installed — if not, ask the user to install it
command -v omni >/dev/null || echo "ERROR: Omni CLI is not installed."
```bashcommand -v omni >/dev/null || echo "ERROR: Omni CLI is not installed."
```bashShow available profiles and select the appropriate one
Show available profiles and select the appropriate one
omni config show
omni config show
If multiple profiles exist, ask the user which to use, then switch:
If multiple profiles exist, ask the user which to use, then switch:
omni config use <profile-name>
```bashomni config use <profile-name>
```bashDatabricks CLI — verify installed and check profiles
Databricks CLI — verify installed and check profiles
databricks --version
cat ~/.databrickscfg
---
> **Tip**: Use `-o json` to force structured output for programmatic parsing, or `-o human` for readable tables. The default is `auto` (human in a TTY, JSON when piped).databricks --version
cat ~/.databrickscfg
---
> **提示**:使用`-o json`强制输出结构化内容以便程序化解析,或使用`-o human`输出易读的表格。默认值为`auto`(在终端中输出人类可读格式,管道传输时输出JSON)。Workflow
工作流程
Step 1 — Gather Requirements
步骤1 — 收集需求
Ask the user:
- Which Omni topic do they want to convert? (e.g., )
orders - What is the Unity Catalog destination? () (e.g.,
catalog.schema)main.sales - What is the Databricks SQL Warehouse ID? (run to find it)
databricks sql warehouses list - Is this a new metric view or does one already exist at ?
catalog.schema.[topic_name]_mv - Which Databricks CLI profile to use (optional — only if the user has multiple profiles)?
⚠️ STOP — Confirm all answers before proceeding. The metric view will be namedby default.[topic_name]_mv
询问用户:
- 他们想要转换哪个Omni主题?(例如:)
orders - Unity Catalog目标位置是什么?()(例如:
catalog.schema)main.sales - Databricks SQL仓库ID是什么?(运行查找)
databricks sql warehouses list - 这是一个新的metric view,还是在位置已存在一个?
catalog.schema.[topic_name]_mv - 使用哪个Databricks CLI配置文件(可选 — 仅当用户有多个配置文件时)?
⚠️ 停止 — 继续前确认所有答案。Metric View默认命名为。[topic_name]_mv
Step 2 — Explore the Omni Model
步骤2 — 探索Omni模型
2a. Find the model ID
2a. 查找模型ID
bash
omni models list --modelkind SHAREDIdentify the Shared Model and note its . Always prefer the Shared Model over Schema or Workbook models.
idbash
omni models list --modelkind SHARED识别共享模型并记录其。优先选择共享模型而非Schema或工作簿模型。
id2b. Fetch the topic file
2b. 获取主题文件
bash
omni models yaml-get <modelId> --filename <topic_name>.topicFrom the topic file extract: , , , , , .
base_viewjoinsfieldsalways_filterai_contextsample_queriesbash
omni models yaml-get <modelId> --filename <topic_name>.topic从主题文件中提取:、、、、、。
base_viewjoinsfieldsalways_filterai_contextsample_queries2c. Fetch the relationships file
2c. 获取关系文件
bash
omni models yaml-get <modelId> --filename relationshipsbash
omni models yaml-get <modelId> --filename relationships2d. Fetch each view file referenced in the topic
2d. 获取主题中引用的每个视图文件
For every view in and :
base_viewjoinsbash
omni models yaml-get <modelId> --filename <view_name>.viewIf a view is prefixed with, fetch the file starting withomni_dbt_. Skip any view backed byomni_dbt_— it has no physical table.derived_table.sql
对于和中的每个视图:
base_viewjoinsbash
omni models yaml-get <modelId> --filename <view_name>.view如果视图前缀为,则获取以omni_dbt_开头的文件。跳过任何由omni_dbt_支持的视图 — 它没有物理表。derived_table.sql
Step 3 — Identify Tables and Joins
步骤3 — 识别表与关联
Map view names to fully-qualified Databricks table references ():
catalog.schema.table| Omni view name | Databricks table |
|---|---|
| |
| |
The separator maps to schema (left) and table (right). Confirm the catalog prefix with the user.
__The indentation defines the join chain — a view indented beneath another joins into its parent:
joinsyaml
joins:
user_order_facts: {} # skip — derived CTE
ecomm__users: {} # joins to base_view
ecomm__inventory_items: # joins to base_view
ecomm__products: # joins to inventory_itemsFind the dimension with in each view — list it first among that table's dimensions.
primary_key: true✋ STOP — Confirm the full table list and join hierarchy with the user before continuing.
将视图名称映射到Databricks的全限定表引用():
catalog.schema.table| Omni视图名称 | Databricks表 |
|---|---|
| |
| |
__joinsyaml
joins:
user_order_facts: {} # skip — derived CTE
ecomm__users: {} # joins to base_view
ecomm__inventory_items: # joins to base_view
ecomm__products: # joins to inventory_items在每个视图中找到带有的维度 — 将其列为该表维度的第一个。
primary_key: true✋ 停止 — 继续前与用户确认完整的表列表和关联层级。
Step 4 — Resolve the Field List
步骤4 — 解析字段列表
| Syntax | Meaning |
|---|---|
(no | Include all fields from all views |
| Include all fields from all views / named view |
| Include all fields with this tag |
| Include this specific field |
| Exclude this field (always wins over wildcard inclusions) |
Process inclusions first, then apply exclusions. Also remove any field with unless explicitly included by name.
hidden: true| 语法 | 含义 |
|---|---|
(无 | 包含所有视图中的全部字段 |
| 包含所有视图/指定视图中的全部字段 |
| 包含带有此标签的所有字段 |
| 包含此特定字段 |
| 排除此字段(始终优先于通配符包含) |
先处理包含规则,再应用排除规则。同时移除所有带有的字段,除非通过名称明确包含。
hidden: trueStep 5 — Build Join Definitions
步骤5 — 构建关联定义
Using the hierarchy from Step 3 and , extract join columns from and build the clause. Use the view name as the join .
relationships.yamlon_sqlon:nameStar schema (single-level):
yaml
joins:
- name: ecomm__users
source: catalog.ecomm.users
'on': source.user_id = ecomm__users.idSnowflake schema (multi-hop):
yaml
joins:
- name: ecomm__inventory_items
source: catalog.ecomm.inventory_items
'on': source.inventory_item_id = ecomm__inventory_items.id
joins:
- name: ecomm__products
source: catalog.ecomm.products
'on': ecomm__inventory_items.product_id = ecomm__products.id⚠️is a YAML 1.1 reserved word — always single-quote the key ason. Columns from nested (2+ level) joins cannot be used in'on':— flatten them through a denormalized direct join instead.expr
使用步骤3中的层级和,从中提取关联列并构建子句。使用视图名称作为关联。
relationships.yamlon_sqlon:name星型模式(单层级):
yaml
joins:
- name: ecomm__users
source: catalog.ecomm.users
'on': source.user_id = ecomm__users.id雪花型模式(多层级):
yaml
joins:
- name: ecomm__inventory_items
source: catalog.ecomm.inventory_items
'on': source.inventory_item_id = ecomm__inventory_items.id
joins:
- name: ecomm__products
source: catalog.ecomm.products
'on': ecomm__inventory_items.product_id = ecomm__products.id⚠️是YAML 1.1的保留字 — 始终将键用单引号括起来,即on。嵌套(2级及以上)关联中的列不能用于'on':— 改为通过非规范化直接关联来展平它们。expr
Step 6 — Map Dimensions and Measures
步骤6 — 映射维度与度量
For each field that survived Step 4, translate it using the rules below. See FIELD-MAPPING.md for full examples.
Dimension quick reference:
| Omni field type | Databricks translation |
|---|---|
| Standard string/number | |
| Single timestamp dimension |
| One |
| |
| |
| |
| BOOLEAN dimension (not a filter; omit |
Measure quick reference:
| Omni measure type | Databricks translation |
|---|---|
| |
| |
| |
| Derived (refs other measures) | |
| |
Strip Omni's refs to bare column names (or for joined fields). Use for the Omni , for , and carry directly. See YAML-REFERENCE.md for format and aggregate type mapping tables.
${view.column}join_name.columndisplay_namelabelcommentdescriptionsynonymsIf the topic has , carry it into the metric view's top-level .
ai_contextcomment✋ STOP — Review all dimensions, measures, and join definitions with the user before generating the final output.
对于步骤4中保留的每个字段,使用以下规则进行转换。查看FIELD-MAPPING.md获取完整示例。
维度速查:
| Omni字段类型 | Databricks转换结果 |
|---|---|
| 标准字符串/数字 | |
| 单个时间戳维度 |
| 每个时间范围对应一个 |
| |
| |
| |
| BOOLEAN维度(不是过滤器;省略 |
度量速查:
| Omni度量类型 | Databricks转换结果 |
|---|---|
| |
| |
| |
| 派生(引用其他度量) | |
度量上的 | |
将Omni的引用简化为裸列名(或关联字段使用)。使用对应Omni的,对应,直接保留。查看YAML-REFERENCE.md获取格式和聚合类型映射表。
${view.column}join_name.columndisplay_namelabelcommentdescriptionsynonyms如果主题包含,将其带入metric view顶级的中。
ai_contextcomment✋ 停止 — 生成最终输出前,与用户审核所有维度、度量和关联定义。
Step 7 — Check for Existing Metric View
步骤7 — 检查现有Metric View
bash
databricks api post /api/2.0/sql/statements \
--json "{\"warehouse_id\": \"<WAREHOUSE_ID>\", \"statement\": \"SHOW VIEWS IN <catalog>.<schema> LIKE '%_mv'\", \"wait_timeout\": \"30s\", \"catalog\": \"<CATALOG>\", \"schema\": \"<SCHEMA>\"}"- View does not exist → use
CREATE OR REPLACE VIEW ... WITH METRICS - View already exists → use
ALTER VIEW ... AS $$ ... $$
bash
databricks api post /api/2.0/sql/statements \
--json "{\"warehouse_id\": \"<WAREHOUSE_ID>\", \"statement\": \"SHOW VIEWS IN <catalog>.<schema> LIKE '%_mv'\", \"wait_timeout\": \"30s\", \"catalog\": \"<CATALOG>\", \"schema\": \"<SCHEMA>\"}"- 视图不存在 → 使用
CREATE OR REPLACE VIEW ... WITH METRICS - 视图已存在 → 使用
ALTER VIEW ... AS $$ ... $$
Step 8 — Generate and Execute the SQL
步骤8 — 生成并执行SQL
Write the SQL to a temp file:
sql
-- CREATE (new view)
CREATE OR REPLACE VIEW catalog.schema.orders_mv
WITH METRICS
LANGUAGE YAML
AS $$
version: 1.1
comment: "..."
source: catalog.ecomm.order_items
joins:
- name: ecomm__users
source: catalog.ecomm.users
'on': source.user_id = ecomm__users.id
dimensions:
- name: id
expr: id
display_name: "Order ID"
- name: status
expr: status
display_name: "Order Status"
measures:
- name: order_count
expr: COUNT(*)
display_name: "Order Count"
- name: total_sale_price
expr: SUM(sale_price)
display_name: "Total Sale Price"
format:
type: currency
currency_code: USD
$$sql
-- ALTER (existing view)
ALTER VIEW catalog.schema.orders_mv AS $$
version: 1.1
...
$$Execute via the SQL Statements API ( does not exist in CLI v0.295.0+):
databricks sql executebash
databricks api post /api/2.0/sql/statements \
--json "{
\"warehouse_id\": \"<WAREHOUSE_ID>\",
\"statement\": $(cat /tmp/orders_mv.sql | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()))'),
\"wait_timeout\": \"50s\",
\"catalog\": \"<CATALOG>\",
\"schema\": \"<SCHEMA>\"
}"Check the response for . If , read and see the Troubleshooting section below.
"state": "SUCCEEDED""state": "FAILED"status.error.message✋ STOP — Confirm which group or user should receive access before running the GRANT. This is a permission change visible to others.
Grant access:
bash
databricks api post /api/2.0/sql/statements \
--json "{\"warehouse_id\": \"<WAREHOUSE_ID>\", \"statement\": \"GRANT SELECT ON VIEW catalog.schema.orders_mv TO \`group_name\`\", \"wait_timeout\": \"30s\", \"catalog\": \"<CATALOG>\", \"schema\": \"<SCHEMA>\"}"将SQL写入临时文件:
sql
-- CREATE (new view)
CREATE OR REPLACE VIEW catalog.schema.orders_mv
WITH METRICS
LANGUAGE YAML
AS $$
version: 1.1
comment: "..."
source: catalog.ecomm.order_items
joins:
- name: ecomm__users
source: catalog.ecomm.users
'on': source.user_id = ecomm__users.id
dimensions:
- name: id
expr: id
display_name: "Order ID"
- name: status
expr: status
display_name: "Order Status"
measures:
- name: order_count
expr: COUNT(*)
display_name: "Order Count"
- name: total_sale_price
expr: SUM(sale_price)
display_name: "Total Sale Price"
format:
type: currency
currency_code: USD
$$sql
-- ALTER (existing view)
ALTER VIEW catalog.schema.orders_mv AS $$
version: 1.1
...
$$通过SQL Statements API执行(CLI v0.295.0+中不存在):
databricks sql executebash
databricks api post /api/2.0/sql/statements \
--json "{
\"warehouse_id\": \"<WAREHOUSE_ID>\",
\"statement\": $(cat /tmp/orders_mv.sql | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()))'),
\"wait_timeout\": \"50s\",
\"catalog\": \"<CATALOG>\",
\"schema\": \"<SCHEMA>\"
}"检查响应中的。如果,查看并参考下方的故障排除部分。
"state": "SUCCEEDED""state": "FAILED"status.error.message✋ 停止 — 运行GRANT前,确认哪些组或用户应获得访问权限。这是对他人可见的权限变更。
授予访问权限:
bash
databricks api post /api/2.0/sql/statements \
--json "{\"warehouse_id\": \"<WAREHOUSE_ID>\", \"statement\": \"GRANT SELECT ON VIEW catalog.schema.orders_mv TO \`group_name\`\", \"wait_timeout\": \"30s\", \"catalog\": \"<CATALOG>\", \"schema\": \"<SCHEMA>\"}"Troubleshooting
故障排除
When the SQL Statements API returns , read :
"state": "FAILED"status.error.message| Error message contains | Likely cause | Fix |
|---|---|---|
| Invalid YAML field or value | Check the field name against the valid keys ( |
| Warehouse is stopped or wrong ID | Start the warehouse in the Databricks UI or verify the ID with |
| The CLI profile lacks privileges | Check the profile's permissions on the catalog/schema with |
| A source or join table doesn't exist in Unity Catalog | Verify each table reference with |
| | Always write |
| Timeout out of range | |
If the error message is truncated, run the same statement with to get the full synchronous error response.
"wait_timeout": "5s"当SQL Statements API返回时,查看:
"state": "FAILED"status.error.message| 错误消息包含 | 可能原因 | 修复方案 |
|---|---|---|
| YAML字段或值无效 | 检查字段名称是否符合有效键( |
| 仓库已停止或ID错误 | 在Databricks UI中启动仓库,或通过 |
| CLI配置文件缺少权限 | 通过 |
| 源表或关联表在Unity Catalog中不存在 | 通过 |
| | 始终写入 |
| 超时超出范围 | |
如果错误消息被截断,使用运行相同语句以获取完整的同步错误响应。
"wait_timeout": "5s"Critical Rules
关键规则
- Naming: Name the metric view (snake_case, lowercase)
[topic_name]_mv - CREATE vs ALTER: Check for existence first — for new,
CREATE OR REPLACEfor existingALTER VIEW - Version: Always use (requires Databricks Runtime 17.2+)
version: 1.1 - Skip derived CTEs: Views with have no physical table — skip and warn the user
derived_table.sql - Confirm before executing: Show the full generated SQL to the user before running
- Boolean fields: Map as BOOLEAN dimensions — not filters.
type: yesnois not a valid field — omit itdata_type - Composed measures: Use syntax; define atomic measures before composed ones
MEASURE() - YAML quoting: is a YAML 1.1 reserved word — always write
on(single-quoted)'on': - No SELECT *: All fields must be explicitly defined
- MAP columns: Skip joins to tables containing type columns — not supported
MAP - Nested join refs: Only direct star join columns (1 level) can be used in . Flatten snowflake schema joins through a denormalized direct join
expr - Warehouse ID required: Always confirm before execution — cannot be inferred
- Exclusions win: always overrides any wildcard inclusion
-view.field - Format type values are lowercase: ,
number,currency,date,date_time,percentagebyte - Date format required: and
type: dateboth requiretype: date_timedate_format - Currency format: Use not
currency_code: USDiso_code: USD - unsupported: Omit it entirely — causes a parse error
decimal_places - CLI execution: Use ;
databricks api post /api/2.0/sql/statementsmust bewait_timeout–5s50s - Omni CLI flag: Use (not
--filename)--file-name - Field description key: Use not
comment:—description:is not a recognized field and causes a parse errordescription
- 命名:Metric View命名为(蛇形命名法,小写)
[topic_name]_mv - CREATE vs ALTER:先检查是否存在 — 新视图使用,现有视图使用
CREATE OR REPLACEALTER VIEW - 版本:始终使用(需要Databricks Runtime 17.2+)
version: 1.1 - 跳过派生CTE:带有的视图没有物理表 — 跳过并提醒用户
derived_table.sql - 执行前确认:运行前向用户展示完整的生成SQL
- 布尔字段:将映射为BOOLEAN维度 — 不是过滤器。
type: yesno不是有效字段 — 省略它data_type - 组合度量:使用语法;先定义原子度量再定义组合度量
MEASURE() - YAML引号:是YAML 1.1的保留字 — 始终写入
on(单引号括起)'on': - **禁止SELECT ***:所有字段必须显式定义
- MAP列:跳过关联到包含类型列的表 — 不支持
MAP - 嵌套关联引用:只有直接星型关联列(1级)可用于。通过非规范化直接关联展平雪花模式关联
expr - 仓库ID必填:执行前必须确认 — 无法推断
- 排除优先:始终覆盖任何通配符包含
-view.field - 格式类型值小写:、
number、currency、date、date_time、percentagebyte - 日期格式必填:和
type: date都需要type: date_timedate_format - 货币格式:使用而非
currency_code: USDiso_code: USD - 不支持:完全省略它 — 会导致解析错误
decimal_places - CLI执行:使用;
databricks api post /api/2.0/sql/statements必须为wait_timeout–5s50s - Omni CLI标志:使用(而非
--filename)--file-name - 字段描述键:使用而非
comment:—description:不是可识别字段,会导致解析错误description