compare-csv
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseSnapLogic Compare CSV Skill - Complete Guide
SnapLogic 比较CSV Skill - 完整指南
Overview
概述
This skill creates Robot Framework test cases for comparing actual vs expected CSV output files. The comparison:
- Validates that pipeline output matches expected baseline
- Supports excluding dynamic columns (timestamps, IDs, etc.)
- Can ignore row ordering for flexible comparison
- Provides detailed difference reports
- Supports key-based row matching
本Skill创建用于比较实际与预期CSV输出文件的Robot Framework测试用例。该比较功能：
- 验证流水线输出是否与预期基线一致
- 支持排除动态列（如时间戳、ID等）
- 可忽略行顺序以实现灵活比较
- 提供详细的差异报告
- 支持基于关键字的行匹配
Key Keywords
核心关键字
Compare CSV Files With Exclusions Template
Compare CSV Files With Exclusions TemplateCompare CSV Files With Exclusions Template
Compare CSV Files With Exclusions TemplateLocation:
test/resources/common/files.resourceArguments:
| Argument | Description | Example |
|---|---|---|
| Path to actual output CSV file | |
| Path to expected output CSV file | |
| Whether to ignore row order | |
| Whether to show detailed differences | |
| Expected comparison result | |
| Columns to exclude from comparison | |
| Additional options | |
Example Usage:
robot
Compare CSV Files With Exclusions Template
... ${actual_output_file}
... ${expected_output_file}
... ${FALSE} # ignore_order
... ${TRUE} # show_details
... IDENTICAL # expected_status
... @{excluded_columns_for_comparison}位置：
test/resources/common/files.resource参数：
| 参数 | 描述 | 示例 |
|---|---|---|
| 实际输出CSV文件的路径 | |
| 预期输出CSV文件的路径 | |
| 是否忽略行顺序 | |
| 是否显示详细差异 | |
| 预期的比较结果 | |
| 需从比较中排除的列 | |
| 附加选项 | |
示例用法：
robot
Compare CSV Files With Exclusions Template
... ${actual_output_file}
... ${expected_output_file}
... ${FALSE} # ignore_order
... ${TRUE} # show_details
... IDENTICAL # expected_status
... @{excluded_columns_for_comparison}Compare CSV Files Template
Compare CSV Files TemplateCompare CSV Files Template
Compare CSV Files TemplateLocation:
test/resources/common/files.resourceUse this simpler keyword when you don't need to exclude columns.
Arguments:
| Argument | Description | Example |
|---|---|---|
| Path to actual output CSV file | |
| Path to expected output CSV file | |
| Whether to ignore row order | |
| Whether to show detailed differences | |
| Expected comparison result | |
位置：
test/resources/common/files.resource当不需要排除列时，可使用这个更简单的关键字。
参数：
| 参数 | 描述 | 示例 |
|---|---|---|
| 实际输出CSV文件的路径 | |
| 预期输出CSV文件的路径 | |
| 是否忽略行顺序 | |
| 是否显示详细差异 | |
| 预期的比较结果 | |
Expected Status Values
预期状态值
| Status | Description |
|---|---|
| Files must match exactly (after exclusions) |
| Files are expected to differ |
| File1 is expected to be a subset of File2 |
| 状态 | 描述 |
|---|---|
| 文件必须完全匹配（排除指定列后） |
| 文件预期存在差异 |
| 文件1预期是文件2的子集 |
Database-Specific Examples
数据库特定示例
Oracle CSV Comparison
Oracle CSV比较
robot
*** Variables ***robot
*** Variables ***Output file paths
输出文件路径
${actual_output_file} ${CURDIR}/../../test_data/actual_expected_data/actual_output/oracle/oracle_actual_output.csv
${expected_output_file} ${CURDIR}/../../test_data/actual_expected_data/expected_output/oracle/expected_output.csv
${actual_output_file} ${CURDIR}/../../test_data/actual_expected_data/actual_output/oracle/oracle_actual_output.csv
${expected_output_file} ${CURDIR}/../../test_data/actual_expected_data/expected_output/oracle/expected_output.csv
Columns to exclude (dynamic values that change between runs)
需排除的列（不同运行间会变化的动态值）
@{excluded_columns_for_comparison}
... CREATED_DATE
... MODIFIED_TIMESTAMP
... UNIQUE_ID
*** Test Cases ***
Compare Oracle Actual vs Expected CSV Output
[Documentation] Validates data integrity by comparing actual Oracle export against expected output.
... This test case performs a comprehensive file comparison to ensure that data processed
... through the Oracle pipeline matches the expected results exactly.
...
... 📋 PREREQUISITES:
... • Export Oracle Data To CSV test case completed successfully
... • Expected output file exists at: test/suite/test_data/actual_expected_data/expected_output/oracle/expected_output.csv
...
... 📋 ARGUMENT DETAILS:
... • Argument 1: file1_path - Path to the actual output CSV file from Oracle
... • Argument 2: file2_path - Path to the expected output CSV file (baseline)
... • Argument 3: ignore_order - Boolean flag to ignore row ordering
... ${TRUE} = Compare without considering row order
... ${FALSE} = Rows must match in exact order
... • Argument 4: show_details - Boolean flag to display detailed differences
... ${TRUE} = Show all differences in console output
... ${FALSE} = Show only summary
... • Argument 5: expected_status - Expected comparison result
... IDENTICAL = Files must match exactly
... DIFFERENT = Files expected to differ
... SUBSET = File1 is subset of File2
... • Argument 6: exclude_columns (Optional) - List of columns to exclude from comparison
... Useful for dynamic columns like timestamps that change between runs
...
... 📋 OUTPUT:
... • Test passes if files are IDENTICAL (or match the expected_status)
... • Detailed differences are displayed in console when show_details=${TRUE}
[Tags] oracle verification comparison
[Template] Compare CSV Files With Exclusions Template
# Test Data: file1_path file2_path ignore_order show_details expected_status exclude_columns
${actual_output_file} ${expected_output_file} ${FALSE} ${TRUE} IDENTICAL @{excluded_columns_for_comparison}undefined@{excluded_columns_for_comparison}
... CREATED_DATE
... MODIFIED_TIMESTAMP
... UNIQUE_ID
*** Test Cases ***
Compare Oracle Actual vs Expected CSV Output
[Documentation] 通过比较实际导出的Oracle数据与预期输出，验证数据完整性。
... 该测试用例会执行全面的文件比较，确保通过Oracle流水线处理的数据与预期结果完全匹配。
...
... 📋 前置条件：
... • Export Oracle Data To CSV测试用例已成功完成
... • 预期输出文件存在于：test/suite/test_data/actual_expected_data/expected_output/oracle/expected_output.csv
...
... 📋 参数详情：
... • 参数1：file1_path - Oracle实际输出CSV文件的路径
... • 参数2：file2_path - 预期输出CSV文件（基线）的路径
... • 参数3：ignore_order - 忽略行顺序的布尔标志
... ${TRUE} = 不考虑行顺序进行比较
... ${FALSE} = 行必须完全按顺序匹配
... • 参数4：show_details - 显示详细差异的布尔标志
... ${TRUE} = 在控制台输出中显示所有差异
... ${FALSE} = 仅显示摘要
... • 参数5：expected_status - 预期的比较结果
... IDENTICAL = 文件必须完全匹配
... DIFFERENT = 文件预期存在差异
... SUBSET = 文件1是文件2的子集
... • 参数6：exclude_columns（可选） - 需从比较中排除的列列表
... 适用于时间戳等不同运行间会变化的动态列
...
... 📋 输出：
... • 若文件符合IDENTICAL（或匹配expected_status），则测试通过
... • 当show_details=${TRUE}时，控制台会显示详细差异
[Tags] oracle verification comparison
[Template] Compare CSV Files With Exclusions Template
# 测试数据：file1_path file2_path ignore_order show_details expected_status exclude_columns
${actual_output_file} ${expected_output_file} ${FALSE} ${TRUE} IDENTICAL @{excluded_columns_for_comparison}undefinedSnowflake CSV Comparison
Snowflake CSV比较
robot
*** Variables ***robot
*** Variables ***Output file paths
输出文件路径
${actual_output_file} ${CURDIR}/../../test_data/actual_expected_data/actual_output/snowflake/snowflake_actual_output.csv
${expected_output_file} ${CURDIR}/../../test_data/actual_expected_data/expected_output/snowflake/expected_output.csv
${actual_output_file} ${CURDIR}/../../test_data/actual_expected_data/actual_output/snowflake/snowflake_actual_output.csv
${expected_output_file} ${CURDIR}/../../test_data/actual_expected_data/expected_output/snowflake/expected_output.csv
Dynamic columns to exclude from comparison
需从比较中排除的动态列
@{excluded_columns_for_comparison}
... SnowflakeConnectorPushTime
... unique_event_id
... event_timestamp
... /MARKETING-NOTIFICATIONS/CONTENT
*** Test Cases ***
Compare Snowflake Actual vs Expected CSV Output
[Documentation] Validates Snowflake pipeline output against expected baseline.
[Tags] snowflake verification comparison
[Template] Compare CSV Files With Exclusions Template
${actual_output_file} ${expected_output_file} ${FALSE} ${TRUE} IDENTICAL @{excluded_columns_for_comparison}undefined@{excluded_columns_for_comparison}
... SnowflakeConnectorPushTime
... unique_event_id
... event_timestamp
... /MARKETING-NOTIFICATIONS/CONTENT
*** Test Cases ***
Compare Snowflake Actual vs Expected CSV Output
[Documentation] 验证Snowflake流水线输出是否符合预期基线。
[Tags] snowflake verification comparison
[Template] Compare CSV Files With Exclusions Template
${actual_output_file} ${expected_output_file} ${FALSE} ${TRUE} IDENTICAL @{excluded_columns_for_comparison}undefinedComparison with Match Key (Row Matching by Column)
基于匹配关键字的比较（按列匹配行）
robot
*** Test Cases ***
Compare CSV With Key-Based Matching
[Documentation] Compares CSV files using a specific column to match rows.
... Useful when row order varies but rows can be matched by a unique identifier.
[Tags] comparison key_match
Compare CSV Files With Exclusions Template
... ${actual_output_file}
... ${expected_output_file}
... ${TRUE} # ignore_order
... ${TRUE} # show_details
... IDENTICAL # expected_status
... @{excluded_columns_for_comparison}
... match_key=headers.profile_id # Match rows by this columnrobot
*** Test Cases ***
Compare CSV With Key-Based Matching
[Documentation] 使用特定列来匹配行，比较CSV文件。
... 适用于行顺序可能变化但可通过唯一标识符匹配行的场景。
[Tags] comparison key_match
Compare CSV Files With Exclusions Template
... ${actual_output_file}
... ${expected_output_file}
... ${TRUE} # ignore_order
... ${TRUE} # show_details
... IDENTICAL # expected_status
... @{excluded_columns_for_comparison}
... match_key=headers.profile_id # 按该列匹配行Complete Test File Template
完整测试文件模板
robot
*** Settings ***
Documentation CSV Comparison Test Suite
... Compares actual pipeline output against expected baseline files.
Library OperatingSystem
Library Collections
Resource ../../../resources/common/files.resource
*** Variables ***robot
*** Settings ***
Documentation CSV比较测试套件
... 比较流水线实际输出与预期基线文件。
Library OperatingSystem
Library Collections
Resource ../../../resources/common/files.resource
*** Variables ***Pipeline configuration
流水线配置
${pipeline_name} my_pipeline
${pipeline_name} my_pipeline
Actual output file (generated by export)
实际输出文件（由导出生成）
${actual_output_file_name} ${pipeline_name}_actual_output.csv
${actual_output_file} ${CURDIR}/../../test_data/actual_expected_data/actual_output/${pipeline_name}/${actual_output_file_name}
${actual_output_file_name} ${pipeline_name}_actual_output.csv
${actual_output_file} ${CURDIR}/../../test_data/actual_expected_data/actual_output/${pipeline_name}/${actual_output_file_name}
Expected output file (baseline)
预期输出文件（基线）
${expected_output_file_name} expected_output.csv
${expected_output_file} ${CURDIR}/../../test_data/actual_expected_data/expected_output/${pipeline_name}/${expected_output_file_name}
${expected_output_file_name} expected_output.csv
${expected_output_file} ${CURDIR}/../../test_data/actual_expected_data/expected_output/${pipeline_name}/${expected_output_file_name}
Columns to exclude from comparison (dynamic values)
需从比较中排除的列（动态值）
@{excluded_columns_for_comparison}
... created_timestamp
... modified_date
... unique_id
... session_id
*** Test Cases ***
Compare Actual vs Expected CSV Output
[Documentation] Validates data integrity by comparing actual export against expected output.
...
... 📋 PREREQUISITES:
... • Export data test case completed successfully
... • Expected output file exists
...
... 📋 ARGUMENT DETAILS:
... • file1_path - Actual output CSV file
... • file2_path - Expected output CSV file (baseline)
... • ignore_order - ${FALSE} for exact order, ${TRUE} to ignore order
... • show_details - ${TRUE} to show differences
... • expected_status - IDENTICAL, DIFFERENT, or SUBSET
... • exclude_columns - Columns to exclude from comparison
[Tags] verification comparison csv
[Template] Compare CSV Files With Exclusions Template
# file1_path file2_path ignore_order show_details expected_status exclude_columns
${actual_output_file} ${expected_output_file} ${FALSE} ${TRUE} IDENTICAL @{excluded_columns_for_comparison}
---@{excluded_columns_for_comparison}
... created_timestamp
... modified_date
... unique_id
... session_id
*** Test Cases ***
Compare Actual vs Expected CSV Output
[Documentation] 通过比较实际导出数据与预期输出，验证数据完整性。
...
... 📋 前置条件：
... • 导出数据测试用例已成功完成
... • 预期输出文件已存在
...
... 📋 参数详情：
... • file1_path - 实际输出CSV文件
... • file2_path - 预期输出CSV文件（基线）
... • ignore_order - ${FALSE} 表示严格匹配顺序，${TRUE} 表示忽略顺序
... • show_details - ${TRUE} 表示显示差异
... • expected_status - IDENTICAL、DIFFERENT或SUBSET
... • exclude_columns - 需从比较中排除的列
[Tags] verification comparison csv
[Template] Compare CSV Files With Exclusions Template
# file1_path file2_path ignore_order show_details expected_status exclude_columns
${actual_output_file} ${expected_output_file} ${FALSE} ${TRUE} IDENTICAL @{excluded_columns_for_comparison}
---Variables Section Template
变量部分模板
robot
*** Variables ***robot
*** Variables ***Pipeline name for file naming
用于文件命名的流水线名称
${pipeline_name} oracle_pipeline
${pipeline_name} oracle_pipeline
Actual output file (generated by Export DB Table Data To CSV)
实际输出文件（由Export DB Table Data To CSV生成）
${actual_output_file_name} ${pipeline_name}_actual_output_file_from_db.csv
${actual_output_file} ${CURDIR}/../../test_data/actual_expected_data/actual_output/oracle/${actual_output_file_name}
${actual_output_file_name} ${pipeline_name}_actual_output_file_from_db.csv
${actual_output_file} ${CURDIR}/../../test_data/actual_expected_data/actual_output/oracle/${actual_output_file_name}
Expected output file (user-provided baseline)
预期输出文件（用户提供的基线）
${expected_output_file_name} expected_output_file.csv
${expected_output_file} ${CURDIR}/../../test_data/actual_expected_data/expected_output/oracle/${expected_output_file_name}
${expected_output_file_name} expected_output_file.csv
${expected_output_file} ${CURDIR}/../../test_data/actual_expected_data/expected_output/oracle/${expected_output_file_name}
Columns to exclude from comparison (dynamic values that change between runs)
需从比较中排除的列（不同运行间会变化的动态值）
@{excluded_columns_for_comparison}
... CREATED_DATE # Timestamp when record was created
... MODIFIED_TIMESTAMP # Last modification time
... SnowflakeConnectorPushTime # Snowflake-specific timestamp
... unique_event_id # Auto-generated unique ID
... event_timestamp # Event time
... /MARKETING-NOTIFICATIONS/CONTENT # Nested JSON path
---@{excluded_columns_for_comparison}
... CREATED_DATE # 记录创建时间戳
... MODIFIED_TIMESTAMP # 最后修改时间
... SnowflakeConnectorPushTime # Snowflake特定时间戳
... unique_event_id # 自动生成的唯一ID
... event_timestamp # 事件时间
... /MARKETING-NOTIFICATIONS/CONTENT # 嵌套JSON路径
---Common Exclusion Patterns
常见排除模式
Timestamp Columns
时间戳列
robot
@{excluded_columns_for_comparison}
... created_date
... modified_date
... timestamp
... event_timestamp
... SnowflakeConnectorPushTime
... last_updatedrobot
@{excluded_columns_for_comparison}
... created_date
... modified_date
... timestamp
... event_timestamp
... SnowflakeConnectorPushTime
... last_updatedAuto-Generated IDs
自动生成的ID
robot
@{excluded_columns_for_comparison}
... unique_id
... unique_event_id
... session_id
... transaction_id
... uuidrobot
@{excluded_columns_for_comparison}
... unique_id
... unique_event_id
... session_id
... transaction_id
... uuidSnowflake-Specific
Snowflake特定列
robot
@{excluded_columns_for_comparison}
... SnowflakeConnectorPushTime
... unique_event_id
... event_timestamp
... RECORD_METADATArobot
@{excluded_columns_for_comparison}
... SnowflakeConnectorPushTime
... unique_event_id
... event_timestamp
... RECORD_METADATAOracle-Specific
Oracle特定列
robot
@{excluded_columns_for_comparison}
... CREATED_DATE
... MODIFIED_DATE
... ROWID
... ORA_ROWSCNrobot
@{excluded_columns_for_comparison}
... CREATED_DATE
... MODIFIED_DATE
... ROWID
... ORA_ROWSCNNested JSON Paths
嵌套JSON路径
robot
@{excluded_columns_for_comparison}
... /MARKETING-NOTIFICATIONS/CONTENT
... /headers/timestamp
... /metadata/created_atrobot
@{excluded_columns_for_comparison}
... /MARKETING-NOTIFICATIONS/CONTENT
... /headers/timestamp
... /metadata/created_atDirectory Structure
目录结构
test/
├── suite/
│ ├── pipeline_tests/
│ │ ├── oracle/
│ │ │ ├── oracle_comparison_tests.robot
│ │ │ └── COMPARE_CSV_README.md
│ │ └── snowflake/
│ │ ├── snowflake_comparison_tests.robot
│ │ └── COMPARE_CSV_README.md
│ └── test_data/
│ └── actual_expected_data/
│ ├── actual_output/
│ │ ├── oracle/
│ │ │ └── oracle_actual_output.csv # Generated by export
│ │ └── snowflake/
│ │ └── snowflake_actual_output.csv
│ └── expected_output/
│ ├── oracle/
│ │ └── expected_output.csv # User-provided baseline
│ └── snowflake/
│ └── expected_output.csv
└── resources/
└── common/
└── files.resource # Contains comparison keywordstest/
├── suite/
│ ├── pipeline_tests/
│ │ ├── oracle/
│ │ │ ├── oracle_comparison_tests.robot
│ │ │ └── COMPARE_CSV_README.md
│ │ └── snowflake/
│ │ ├── snowflake_comparison_tests.robot
│ │ └── COMPARE_CSV_README.md
│ └── test_data/
│ └── actual_expected_data/
│ ├── actual_output/
│ │ ├── oracle/
│ │ │ └── oracle_actual_output.csv # 导出生成
│ │ └── snowflake/
│ │ └── snowflake_actual_output.csv
│ └── expected_output/
│ ├── oracle/
│ │ └── expected_output.csv # 用户提供的基线
│ └── snowflake/
│ └── expected_output.csv
└── resources/
└── common/
└── files.resource # 包含比较关键字Combining with Export
与导出功能结合使用
Export Then Compare Flow
导出后比较流程
robot
*** Test Cases ***
Export Data To CSV
[Documentation] Exports data from database table to CSV file.
[Tags] export csv
Export DB Table Data To CSV
... ${table_name}
... ${order_by_column}
... ${actual_output_file}
Compare Actual vs Expected CSV Output
[Documentation] Compares exported data against expected baseline.
[Tags] verification comparison
[Template] Compare CSV Files With Exclusions Template
${actual_output_file} ${expected_output_file} ${FALSE} ${TRUE} IDENTICAL @{excluded_columns_for_comparison}robot
*** Test Cases ***
Export Data To CSV
[Documentation] 将数据库表数据导出为CSV文件。
[Tags] export csv
Export DB Table Data To CSV
... ${table_name}
... ${order_by_column}
... ${actual_output_file}
Compare Actual vs Expected CSV Output
[Documentation] 比较导出数据与预期基线。
[Tags] verification comparison
[Template] Compare CSV Files With Exclusions Template
${actual_output_file} ${expected_output_file} ${FALSE} ${TRUE} IDENTICAL @{excluded_columns_for_comparison}Best Practices
最佳实践
- Always Exclude Dynamic Columns: Timestamps, auto-generated IDs, and session-specific values should be excluded
- Use : Unless row order is truly non-deterministic, keep strict ordering
ignore_order=${FALSE} - Enable : This helps debug failures by showing exact differences
show_details=${TRUE} - Create Expected Baseline Once: Run the pipeline once, verify output manually, then save as expected baseline
- Use Meaningful File Names: Include pipeline name and purpose in file names
- Store Expected Files in Version Control: Keep expected output files in the repo for comparison
- 始终排除动态列：时间戳、自动生成的ID和会话特定值应被排除
- 使用：除非行顺序确实不确定，否则保持严格顺序匹配
ignore_order=${FALSE} - 启用：这有助于通过显示确切差异来调试失败
show_details=${TRUE} - 仅创建一次预期基线：运行一次流水线，手动验证输出，然后保存为预期基线
- 使用有意义的文件名：在文件名中包含流水线名称和用途
- 将预期文件存储在版本控制中：将预期输出文件保存在代码仓库中以便比较
Troubleshooting
故障排除
Files Show as Different When They Should Match
文件本应匹配但显示存在差异
- Check if dynamic columns need to be excluded
- Verify column names match exactly (case-sensitive)
- Check for whitespace differences
- Verify row order if
ignore_order=${FALSE}
- 检查是否需要排除动态列
- 验证列名是否完全匹配（区分大小写）
- 检查是否存在空格差异
- 若，验证行顺序是否一致
ignore_order=${FALSE}
Key-Based Matching Not Working
基于关键字的匹配不生效
- Ensure the column exists in both files
match_key - Verify column name matches exactly
- Check that key values are unique
- 确保列在两个文件中都存在
match_key - 验证列名是否完全匹配
- 检查关键字值是否唯一
Nested JSON Path Exclusions
嵌套JSON路径排除
Use forward slashes for nested paths:
robot
@{excluded_columns}
... /parent/child/field
... /headers/timestamp对嵌套路径使用正斜杠：
robot
@{excluded_columns}
... /parent/child/field
... /headers/timestampRelated Skills
相关Skills
- — Export database data to CSV before comparison
/export-data-to-csv - — Verify record counts before export
/verify-data-in-db - — Complete end-to-end pipeline setup with comparison
/end-to-end-pipeline-verification
- — 比较前将数据库数据导出为CSV
/export-data-to-csv - — 导出前验证记录数
/verify-data-in-db - — 包含比较步骤的完整端到端流水线设置
/end-to-end-pipeline-verification
Invoke with: /compare-csv
/compare-csv调用方式：/compare-csv
/compare-csv