configuring-filters

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Configuring Filters

配置过滤器

A filter is a record gate that decides which records continue through the pipeline and which are silently dropped. Filters are configured directly on exports, imports, and flow router branches -- not as standalone resources.

Concerns when configuring a filter:

Placement -- where in the pipeline the filter runs (output filter on export, input filter on lookup, pre-import filter, branch routing condition)
Mechanism -- expression-based rules (declarative, no code) vs. script-based (full JavaScript control)
Expression syntax -- prefix-notation S-expressions with operators, field access (
```
extract
```
), type coercions, and logical combinators (
```
and
```
/
```
or
```
/
```
not
```
)
Settings references -- dynamic filter values pulled from flow, export, import, or integration settings via
```
settings
```
accessor
Script alternative -- when expression rules cannot handle the logic (multi-step conditionals, date math, external lookups), use a filter script instead

过滤器是一个记录闸门，用于决定哪些记录可以继续在管道中流转，哪些会被静默丢弃。过滤器直接配置在导出、导入和流程路由器分支上——并非作为独立资源存在。

配置过滤器时需要关注的要点：

放置位置——过滤器在管道中的运行位置（导出的输出过滤器、查找的输入过滤器、导入前过滤器、分支路由条件）
实现机制——基于表达式的规则（声明式，无需代码） vs 基于脚本的规则（完全由JavaScript控制）
表达式语法——使用前缀表示法的S表达式，包含运算符、字段访问（
```
extract
```
）、类型转换以及逻辑组合符（
```
and
```
/
```
or
```
/
```
not
```
）
设置引用——通过
```
settings
```
访问器从流程、导出、导入或集成设置中获取动态过滤值
脚本替代方案——当表达式规则无法处理复杂逻辑（多步骤条件判断、日期运算、外部查找）时，改用过滤器脚本

Four Places Filters Appear

过滤器的四种应用场景

Output filter (export

filter

)

输出过滤器（导出

filter

）

Applied after records are retrieved from the source system. Records that match continue through the flow; records that do not match are silently dropped. Available on all export types.

在从源系统检索到记录后应用。匹配的记录会继续流转；不匹配的记录会被静默丢弃。适用于所有导出类型。

Input filter (lookup export

inputFilter

)

输入过滤器（查找导出

inputFilter

）

Applied to incoming records before they trigger a lookup API call. Records that do not match skip the lookup step entirely -- they pass through without enrichment. This reduces unnecessary API calls. Only relevant when

isLookup: true

在记录触发查找API调用前应用于传入记录。不匹配的记录会完全跳过查找步骤——它们会直接流转，不会被丰富数据。这可以减少不必要的API调用。仅当

isLookup: true

时生效。

Import filter (import

filter

)

导入过滤器（导入

filter

）

Applied to records before they are sent to the destination system. Records that do not match are dropped and never submitted to the destination. Useful for conditional writes (e.g., only import records with a non-empty email).

在记录发送到目标系统前应用。不匹配的记录会被丢弃，永远不会提交到目标系统。适用于条件写入场景（例如，仅导入邮箱非空的记录）。

Branch input filter (router branch

inputFilter

)

分支输入过滤器（路由器分支

inputFilter

）

Applied per-branch in a flow router to route records to different processing paths. Each branch has its own filter expression. The last branch can omit a filter to serve as a catch-all. Configured on the flow's

routers[].branches[]

entries, not on exports or imports.

在流程路由器的每个分支上应用，用于将记录路由到不同的处理路径。每个分支都有自己的过滤表达式。最后一个分支可以省略过滤器，作为兜底分支。配置在流程的

routers[].branches[]

条目上，而非导出或导入资源。

Quick Reference

快速参考

Filter Type Decision Matrix

过滤器类型决策矩阵

Situation	Use	Configured on
Skip records from the source based on field values	Output filter ( `filter` )	Export resource
Skip lookup calls for records missing required fields	Input filter ( `inputFilter` )	Export resource ( `isLookup: true` )
Skip import for records that should not be written	Import filter ( `filter` )	Import resource
Route records to different branches by field values	Branch filter ( `inputFilter` )	Flow `routers[].branches[]`
Complex multi-step logic, date math, external calls	Script filter	Export or import `filter.type: "script"`

场景	使用类型	配置位置
根据字段值跳过源系统中的记录	输出过滤器（ `filter` ）	导出资源
跳过缺少必填字段的记录的查找调用	输入过滤器（ `inputFilter` ）	导出资源（ `isLookup: true` ）
跳过不应写入的记录的导入操作	导入过滤器（ `filter` ）	导入资源
根据字段值将记录路由到不同分支	分支过滤器（ `inputFilter` ）	流程 `routers[].branches[]`
复杂多步骤逻辑、日期运算、外部调用	脚本过滤器	导出或导入 `filter.type: "script"`

Expression vs. Script

表达式 vs 脚本

Mechanism	When to use	Config
Expression ( `type: "expression"` )	Standard field comparisons, pattern matching, empty checks, combining with and/or	`filter.expression.rules` array
Script ( `type: "script"` )	Complex business logic, cross-record state, date calculations, external API calls	`filter.script._scriptId` + `filter.script.function`

Prefer expressions -- they are simpler, faster, and do not require a separate script resource. Use scripts only when expression operators cannot handle the logic.

机制	使用场景	配置方式
表达式（ `type: "expression"` ）	标准字段比较、模式匹配、空值检查、使用and/or组合条件	`filter.expression.rules` 数组
脚本（ `type: "script"` ）	复杂业务逻辑、跨记录状态、日期计算、外部API调用	`filter.script._scriptId` + `filter.script.function`

优先使用表达式——它们更简单、更快，且不需要独立的脚本资源。仅当表达式运算符无法处理逻辑时才使用脚本。

Operator Quick Reference

运算符快速参考

Operator	Meaning	Operands
`equals`	Exact match	field, value
`notequals`	Not equal	field, value
`greaterthan`	Greater than	field, value
`greaterthanequals`	Greater than or equal	field, value
`lessthan`	Less than	field, value
`lessthanequals`	Less than or equal	field, value
`contains`	Substring match	field, substring
`doesnotcontain`	No substring match	field, substring
`startswith`	Prefix match	field, prefix
`endswith`	Suffix match	field, suffix
`matches`	Pattern match	field, pattern
`notempty`	Field has a value	field
`empty`	Field is null/empty	field
`and`	All conditions true	condition, condition, ...
`or`	Any condition true	condition, condition, ...
`not`	Negate condition	condition

Field access:

["extract", "fieldName"]

for record fields,

["settings", "flow.settingName"]

for configuration values.

Type coercions:

["number", ...]

["string", ...]

["boolean", ...]

["epochtime", ...]

Transformations:

["lowercase", ...]

["uppercase", ...]

["floor", ...]

["ceiling", ...]

["abs", ...]

运算符	含义	操作数
`equals`	精确匹配	字段, 值
`notequals`	不相等	字段, 值
`greaterthan`	大于	字段, 值
`greaterthanequals`	大于等于	字段, 值
`lessthan`	小于	字段, 值
`lessthanequals`	小于等于	字段, 值
`contains`	子串匹配	字段, 子串
`doesnotcontain`	无匹配子串	字段, 子串
`startswith`	前缀匹配	字段, 前缀
`endswith`	后缀匹配	字段, 后缀
`matches`	模式匹配	字段, 模式
`notempty`	字段有值	字段
`empty`	字段为null/空值	字段
`and`	所有条件都为真	条件, 条件, ...
`or`	任一条件为真	条件, 条件, ...
`not`	否定条件	条件

字段访问：记录字段使用

["extract", "fieldName"]

，配置值使用

["settings", "flow.settingName"]

。

类型转换：

["number", ...]

["string", ...]

["boolean", ...]

["epochtime", ...]

。

转换操作：

["lowercase", ...]

["uppercase", ...]

["floor", ...]

["ceiling", ...]

["abs", ...]

。

Schema Index

架构索引

Schema	Contents
filter.yml	Filter object -- type, expression (version, rules), script (_scriptId, function), all operators and examples

架构	内容
filter.yml	过滤器对象——类型、表达式（版本、规则）、脚本（_scriptId、function）、所有运算符及示例

Related Skills

How to Configure a Filter

如何配置过滤器

1. Determine filter placement

1. 确定过滤器放置位置

Where in the pipeline should filtering happen?

After export retrieval -- use an output filter on the export (
```
filter
```
)
Before a lookup call -- use an input filter on the lookup export (
```
inputFilter
```
)
Before import submission -- use a filter on the import (
```
filter
```
)
For branch routing -- use branch input filters on the flow router

过滤操作应在管道的哪个位置执行？

导出检索后——在导出上使用输出过滤器（
```
filter
```
）
查找调用前——在查找导出上使用输入过滤器（
```
inputFilter
```
）
导入提交前——在导入上使用过滤器（
```
filter
```
）
分支路由——在流程路由器上使用分支输入过滤器

2. Choose expression vs. script

2. 选择表达式或脚本

Can the logic be expressed as field comparisons combined with and/or? Use an expression. Need loops, date math, cross-field calculations, or external API calls? Use a script.

逻辑是否可以表示为字段比较并结合and/or？使用表达式。需要循环、日期运算、跨字段计算或外部API调用？使用脚本。

3. Build the expression rules

3. 构建表达式规则

Expression rules use prefix (S-expression) notation. Every rule is an array where the first element is the operator and remaining elements are operands.

Pattern: Single condition

[operator, [extract, fieldName], value]

Pattern: Multiple conditions (all must match)

[and, [condition1], [condition2], ...]

Pattern: Multiple conditions (any must match)

[or, [condition1], [condition2], ...]

Pattern: Field access with type coercion

[operator, [number, [extract, fieldName]], numericValue]

Pattern: Dynamic value from settings

[equals, [string, [extract, fieldName]], [string, [settings, flow.settingName]]]

表达式规则使用前缀（S表达式）表示法。每个规则都是一个数组，第一个元素是运算符，其余元素是操作数。

模式：单个条件

[operator, [extract, fieldName], value]

模式：多个条件（全部必须匹配）

[and, [condition1], [condition2], ...]

模式：多个条件（任一匹配即可）

[or, [condition1], [condition2], ...]

模式：带类型转换的字段访问

[operator, [number, [extract, fieldName]], numericValue]

模式：从设置中获取动态值

[equals, [string, [extract, fieldName]], [string, [settings, flow.settingName]]]

4. Set the filter on the resource

4. 在资源上设置过滤器

Use the CLI

set

command to add or update a filter on an existing export or import. The

set

command does GET-modify-PUT automatically.

bash

undefined

使用CLI的

set

命令在现有导出或导入上添加或更新过滤器。

set

命令会自动执行GET-修改-PUT操作。

bash

undefined

Set an expression filter on an export (output filter)

在导出上设置表达式过滤器（输出过滤器）

celigo exports set <id> filter.type=expression filter.expression.version=1 filter.expression.rules='["notempty",["extract","email"]]'

Set an expression filter on an import

在导入上设置表达式过滤器

celigo imports set <id> filter.type=expression filter.expression.version=1 filter.expression.rules='["notequals",["extract","status"],"cancelled"]'


For complex expressions, use JSON input:

```bash

celigo imports set <id> filter.type=expression filter.expression.version=1 filter.expression.rules='["notequals",["extract","status"],"cancelled"]'


对于复杂表达式，使用JSON输入：

```bash

Create or update the full export with filter included

创建或更新包含过滤器的完整导出配置

celigo exports update <id> < export-with-filter.json

undefined

celigo exports update <id> < export-with-filter.json

undefined

5. Wire a script filter (when expressions are insufficient)

5. 关联脚本过滤器（当表达式不足以处理时）

Create the script resource first, then reference it:

bash

undefined

先创建脚本资源，再引用它：

bash

undefined

Create the filter script

创建过滤器脚本

celigo scripts create < filter-script.json

Wire it to the export

将其关联到导出

celigo exports set <id> filter.type=script filter.script._scriptId=<scriptId> filter.script.function=filterRecords


The script function receives `options.record` and must return `true` (process) or `false` (skip). See [writing-scripts > Record-Level Processors](../writing-scripts/SKILL.md#record-level-processors-on-export-or-import) for the full function signature.

celigo exports set <id> filter.type=script filter.script._scriptId=<scriptId> filter.script.function=filterRecords


脚本函数接收`options.record`，必须返回`true`（处理）或`false`（跳过）。完整函数签名请参见[writing-scripts > Record-Level Processors](../writing-scripts/SKILL.md#record-level-processors-on-export-or-import)。

6. Test the filter

6. 测试过滤器

bash

undefined

bash

undefined

Invoke the export to see which records pass through

调用导出以查看哪些记录可以通过

celigo exports invoke <id>

Run the flow and check record counts

运行流程并检查记录数量

celigo flows run <flowId> -y celigo jobs list --flow <flowId> --limit 1


Check `numSuccess`, `numIgnore`, and `numError` on the job. Records dropped by a filter show up in `numIgnore`, not `numError`.

celigo flows run <flowId> -y celigo jobs list --flow <flowId> --limit 1


查看作业的`numSuccess`、`numIgnore`和`numError`字段。被过滤器丢弃的记录会显示在`numIgnore`中，而非`numError`。

CLI Commands

CLI命令

Filters are configured on exports and imports, not as standalone resources. Use export and import CRUD commands.

bash

undefined

过滤器配置在导出和导入上，而非作为独立资源。使用导出和导入的CRUD命令。

bash

undefined

Read current filter config

读取当前过滤器配置

celigo exports get <id> celigo imports get <id>

Set filter via key=value (GET-modify-PUT)

通过key=value设置过滤器（自动执行GET-修改-PUT）

celigo exports set <id> filter.type=expression filter.expression.version=1 filter.expression.rules='[...]' celigo imports set <id> filter.type=expression filter.expression.version=1 filter.expression.rules='[...]'

Set input filter on a lookup export

在查找导出上设置输入过滤器

celigo exports set <id> inputFilter.type=expression inputFilter.expression.version=1 inputFilter.expression.rules='[...]'

Full JSON update (for complex filters)

完整JSON更新（适用于复杂过滤器）

celigo exports update <id> < export.json celigo imports update <id> < import.json

Remove a filter (set to empty)

移除过滤器（设置为空）

celigo exports set <id> filter=null celigo imports set <id> filter=null

Test

测试

celigo exports invoke <id> celigo flows run <flowId> -y


<!-- TIER:3 -->

celigo exports invoke <id> celigo flows run <flowId> -y


<!-- TIER:3 -->

Pre-Submit Checklist

提交前检查清单

Before creating or updating a filter, verify:

filter.type
matches the config block --

"expression"

requires

filter.expression

"script"

requires

filter.script

expression.version
is
"1"
-- the only supported version
Operator names are lowercase --
```
"notempty"
```
, not
```
"isNotEmpty"
```
or
```
"NOTEMPTY"
```
extract
references valid field names -- field names must match the actual record structure; use dot notation for nested fields (e.g.,
```
"customer.email"
```
)
Type coercions are applied where needed -- comparing a string field to a number requires
```
["number", ["extract", "field"]]
```
Script filter returns strict boolean --
```
true
```
or
```
false
```
, not truthy/falsy values
Input filter is on a lookup export --
```
inputFilter
```
only applies when
```
isLookup: true
```

在创建或更新过滤器前，请验证：

filter.type
与配置块匹配——

"expression"

需要

filter.expression

，

"script"

需要

filter.script

expression.version
为
"1"
——这是唯一支持的版本
运算符名称为小写——使用
```
"notempty"
```
，而非
```
"isNotEmpty"
```
或
```
"NOTEMPTY"
```
extract
引用有效的字段名——字段名必须与实际记录结构匹配；嵌套字段使用点表示法（例如
```
"customer.email"
```
）
在需要时应用类型转换——将字符串字段与数字比较时，需要使用
```
["number", ["extract", "field"]]
```
脚本过滤器返回严格布尔值——返回
```
true
```
或
```
false
```
，而非真值/假值
输入过滤器配置在查找导出上——
```
inputFilter
```
仅在
```
isLookup: true
```
时生效

Gotchas

常见陷阱

Operators are
notempty
/
empty
, not
isnotempty
/
isempty
. The spec documents both forms but the platform uses the shorter names. Using
```
isnotempty
```
or
```
isempty
```
will silently fail to match.
Filtered records are silently dropped, not errored. There is no error log entry for filtered records. They appear as
```
numIgnore
```
in job stats, which makes debugging "missing records" problems harder. Check the filter first when records disappear without errors.
Import filter drops records before submission. Records filtered out on an import never reach the destination -- they are not sent, not rejected. This differs from a server-side rejection which would appear in errors.
Input filter skips the step, not the record. On a lookup export, a filtered record skips the lookup but continues through subsequent flow steps. It is not removed from the pipeline.
Branch filters use the same expression syntax but live on the flow. Branch
```
inputFilter
```
is configured on
```
routers[].branches[]
```
in the flow, not on the export or import. Use
```
celigo flows set
```
or
```
celigo flows update
```
to modify them.
Expression and script filters cannot coexist. Setting
```
filter.type
```
to
```
"script"
```
ignores any
```
expression
```
config, and vice versa. Switching types does not auto-clear the other config block.
PUT erases omitted fields. Always GET the export/import first, modify the filter, then PUT the complete object. The
```
set
```
command handles this automatically.

settings
accessor requires scoped prefix. Use

flow.settingName

export.settingName

import.settingName

, or

integration.settingName

-- not bare

settingName

String comparison is case-sensitive by default. Use
```
["lowercase", ["string", ["extract", "field"]]]
```
before comparison to make it case-insensitive. The
```
matches
```
operator also does case-sensitive matching.

运算符为
notempty
/
empty
，而非
isnotempty
/
isempty
。文档中可能同时列出两种形式，但平台使用短名称。使用
```
isnotempty
```
或
```
isempty
```
会导致匹配失败且无提示。
被过滤的记录会被静默丢弃，不会报错。 被过滤的记录不会生成错误日志条目。它们会显示在作业统计的
```
numIgnore
```
中，这会增加调试“记录丢失”问题的难度。当记录无故消失时，请先检查过滤器。
导入过滤器在提交前丢弃记录。 被导入过滤器过滤掉的记录永远不会到达目标系统——它们不会被发送，也不会被拒绝。这与服务器端拒绝不同，后者会显示在错误中。
输入过滤器跳过步骤，而非丢弃记录。 在查找导出中，被过滤的记录会跳过查找步骤，但会继续后续流程步骤。不会从管道中移除。
分支过滤器使用相同的表达式语法，但配置在流程上。 分支
```
inputFilter
```
配置在流程的
```
routers[].branches[]
```
中，而非导出或导入。使用
```
celigo flows set
```
或
```
celigo flows update
```
进行修改。
表达式过滤器和脚本过滤器不能共存。 将
```
filter.type
```
设置为
```
"script"
```
会忽略任何
```
expression
```
配置，反之亦然。切换类型不会自动清除另一个配置块。
PUT操作会删除未包含的字段。 请始终先获取导出/导入的配置，修改过滤器，然后PUT完整对象。
```
set
```
命令会自动处理此操作。

settings
访问器需要带作用域前缀。使用

flow.settingName

、

export.settingName

、

import.settingName

或

integration.settingName

——不能直接使用

settingName

。

字符串比较默认区分大小写。 在比较前使用
```
["lowercase", ["string", ["extract", "field"]]]
```
使其不区分大小写。
```
matches
```
运算符同样区分大小写。

Common Errors

常见错误

Error / Symptom	Cause	Fix
Records silently disappear, no errors	Filter is dropping them; check `numIgnore` in job stats	Review the filter expression; temporarily remove it to confirm
Filter passes all records (no filtering)	Wrong field name in `extract` , or type mismatch (comparing string to number without coercion)	Verify field name matches record structure; add type coercion
Filter blocks all records	Logic inverted (e.g., `equals` instead of `notequals` ), or field is always empty/null	Test with a known record; check if the field exists in the data
"Function not found" on script filter	`function` name in `filter.script.function` does not match an exported function in the script	Verify exact function name (case-sensitive) in the script resource
Script filter returns inconsistent results	Function returns truthy/falsy instead of strict `true` / `false`	Return explicit `true` or `false`
`inputFilter` has no effect	Export is not a lookup ( `isLookup` is not `true` )	Set `isLookup: true` on the export, or move the filter to `filter` instead

错误/症状	原因	修复方案
记录无故消失，无错误提示	过滤器正在丢弃记录；检查作业统计中的 `numIgnore`	检查过滤器表达式；暂时移除过滤器以确认
过滤器允许所有记录通过（无过滤效果）	`extract` 中的字段名错误，或类型不匹配（未转换类型就将字符串与数字比较）	验证字段名与记录结构匹配；添加类型转换
过滤器阻止所有记录	逻辑反转（例如使用 `equals` 而非 `notequals` ），或字段始终为空/null	使用已知记录进行测试；检查数据中是否存在该字段
脚本过滤器提示“Function not found”	`filter.script.function` 中的函数名与脚本中导出的函数不匹配	验证脚本资源中的函数名（区分大小写）
脚本过滤器返回结果不一致	函数返回真值/假值而非严格的 `true` / `false`	返回明确的 `true` 或 `false`
`inputFilter` 无效果	导出不是查找类型（ `isLookup` 不为 `true` ）	在导出上设置 `isLookup: true` ，或将过滤器移至 `filter`

configuring-filters

Original

Translation

Configuring Filters

配置过滤器

Four Places Filters Appear

过滤器的四种应用场景

Output filter (export filter)

输出过滤器（导出filter）

Input filter (lookup export inputFilter)

输入过滤器（查找导出inputFilter）

Import filter (import filter)

导入过滤器（导入filter）

Branch input filter (router branch inputFilter)

分支输入过滤器（路由器分支inputFilter）

Quick Reference

快速参考

Filter Type Decision Matrix

过滤器类型决策矩阵

Expression vs. Script

表达式 vs 脚本

Operator Quick Reference

运算符快速参考

Schema Index

架构索引

Related Skills

相关技能

How to Configure a Filter

如何配置过滤器

1. Determine filter placement

1. 确定过滤器放置位置

2. Choose expression vs. script

2. 选择表达式或脚本

3. Build the expression rules

3. 构建表达式规则

4. Set the filter on the resource

4. 在资源上设置过滤器

Set an expression filter on an export (output filter)

在导出上设置表达式过滤器（输出过滤器）

Set an expression filter on an import

在导入上设置表达式过滤器

Create or update the full export with filter included

创建或更新包含过滤器的完整导出配置

5. Wire a script filter (when expressions are insufficient)

5. 关联脚本过滤器（当表达式不足以处理时）

Create the filter script

创建过滤器脚本

Wire it to the export

将其关联到导出

6. Test the filter

6. 测试过滤器

Invoke the export to see which records pass through

调用导出以查看哪些记录可以通过

Run the flow and check record counts

运行流程并检查记录数量

CLI Commands

CLI命令

Read current filter config

读取当前过滤器配置

Set filter via key=value (GET-modify-PUT)

通过key=value设置过滤器（自动执行GET-修改-PUT）

Set input filter on a lookup export

在查找导出上设置输入过滤器

Full JSON update (for complex filters)

完整JSON更新（适用于复杂过滤器）

Remove a filter (set to empty)

移除过滤器（设置为空）

Test

测试

Pre-Submit Checklist

提交前检查清单

Gotchas

常见陷阱

Common Errors

常见错误

Output filter (export
`filter`
)

输出过滤器（导出
`filter`
）

Input filter (lookup export
`inputFilter`
)

输入过滤器（查找导出
`inputFilter`
）

Import filter (import
`filter`
)

导入过滤器（导入
`filter`
）

Branch input filter (router branch
`inputFilter`
)

分支输入过滤器（路由器分支
`inputFilter`
）