differential-fuzzer

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Differential Fuzzer

差分模糊测试工具

Always load Debugging skill for reference

The differential fuzzer compares Turso results against SQLite for generated SQL statements to find correctness bugs.

请始终加载参考用调试技能

差分模糊测试工具会将Turso执行生成的SQL语句的结果与SQLite的结果进行对比，以发现正确性问题。

Location

位置

testing/differential-oracle/fuzzer/

testing/differential-oracle/fuzzer/

Running the Fuzzer

运行模糊测试工具

Single Run

单次运行

bash

undefined

bash

undefined

Basic run (100 statements, random seed)

cargo run --bin differential_fuzzer

With specific seed for reproducibility

cargo run --bin differential_fuzzer -- --seed 12345

More statements with verbose output

cargo run --bin differential_fuzzer -- -n 1000 --verbose

Keep database files after run (for debugging)

cargo run --bin differential_fuzzer -- --seed 12345 --keep-files

All options

cargo run --bin differential_fuzzer --
--seed <SEED> # Deterministic seed -n <NUM> # Number of statements (default: 100) -t <NUM> # Number of tables (default: 2) -c <NUM> # Columns per table (default: 5) --verbose # Print each SQL statement --keep-files # Persist .db files to disk

undefined

undefined

Continuous Fuzzing (Loop Mode)

持续模糊测试（循环模式）

bash

undefined

bash

undefined

Run forever with random seeds

cargo run --bin differential_fuzzer -- loop

Run 50 iterations

cargo run --bin differential_fuzzer -- loop 50

undefined

cargo run --bin differential_fuzzer -- loop 50

undefined

Docker Runner (CI/Production)

Docker运行器（CI/生产环境）

bash

undefined

bash

undefined

Build and run from repo root

docker build -f testing/differential-oracle/fuzzer/docker-runner/Dockerfile -t fuzzer . docker run -e GITHUB_TOKEN=xxx -e SLACK_WEBHOOK_URL=xxx fuzzer


Environment variables for docker-runner:
- `TIME_LIMIT_MINUTES` - Total runtime (default: 1440 = 24h)
- `PER_RUN_TIMEOUT_SECONDS` - Per-run timeout (default: 1200 = 20min)
- `NUM_STATEMENTS` - Statements per run (default: 1000)
- `LOG_TO_STDOUT` - Print fuzzer output (default: false)
- `GITHUB_TOKEN` - For auto-filing issues
- `SLACK_WEBHOOK_URL` - For notifications

docker build -f testing/differential-oracle/fuzzer/docker-runner/Dockerfile -t fuzzer . docker run -e GITHUB_TOKEN=xxx -e SLACK_WEBHOOK_URL=xxx fuzzer


Docker运行器的环境变量：
- `TIME_LIMIT_MINUTES` - 总运行时长（默认值：1440 = 24小时）
- `PER_RUN_TIMEOUT_SECONDS` - 单次运行超时时间（默认值：1200 = 20分钟）
- `NUM_STATEMENTS` - 每次运行的语句数量（默认值：1000）
- `LOG_TO_STDOUT` - 打印模糊测试工具输出（默认值：false）
- `GITHUB_TOKEN` - 用于自动提交Issue
- `SLACK_WEBHOOK_URL` - 用于发送通知

Output Files

输出文件

All output goes to

simulator-output/

directory:

File	Description
`test.sql`	All executed SQL statements. Failed statements prefixed with `-- FAILED:` , errors with `-- ERROR:`
`schema.json`	Database schema at end of run (or at failure)
`test.db`	Turso database file (only with `--keep-files` )
`test-sqlite.db`	SQLite database file (only with `--keep-files` )

所有输出都会保存到

simulator-output/

文件	说明
`test.sql`	所有已执行的SQL语句。失败的语句会以 `-- FAILED:` 为前缀，错误信息以 `-- ERROR:` 为前缀
`schema.json`	运行结束（或失败时）的数据库结构
`test.db`	Turso数据库文件（仅在使用 `--keep-files` 参数时生成）
`test-sqlite.db`	SQLite数据库文件（仅在使用 `--keep-files` 参数时生成）

Reproducing Errors

复现错误

Always follow these steps

Find the seed in the error output:

INFO: Starting differential_fuzzer with config: SimConfig { seed: 12345, ... }

Re-run with that seed:

bash

cargo run --bin differential_fuzzer -- --seed 12345 --verbose --keep-files

Check output files:
- ```
simulator-output/test.sql
```
  - Find the failing statement (look for
```
-- FAILED:
```
  )
- ```
simulator-output/schema.json
```
  - Check table structure at failure time
Create a minimal reproducer
- Create reproducer in
```
.sqltest
```
  or in
```
.rs
```
  always load Debugging skill for reference
Compare behavior manually: If needed try to compare the behaviour and produce a report in the end. Always write to a tmp file first with Edit tool to test the sql and then pass it to the binaries.
bash
```
# Run failing SQL against SQLite
sqlite3 :memory: < simulator-output/test.sql

# Run against tursodb CLI
tursodb :memory: < simulator-output/test.sql
```

请始终遵循以下步骤

在错误输出中找到种子值：

INFO: Starting differential_fuzzer with config: SimConfig { seed: 12345, ... }

使用该种子值重新运行：

bash

cargo run --bin differential_fuzzer -- --seed 12345 --verbose --keep-files

检查输出文件：
- ```
simulator-output/test.sql
```
  - 查找失败的语句（搜索
```
-- FAILED:
```
  ）
- ```
simulator-output/schema.json
```
  - 查看失败时的表结构
创建最小化复现用例
- 在
```
.sqltest
```
  或
```
.rs
```
  文件中创建复现用例，请始终加载参考用调试技能
手动对比行为：如有需要，尝试对比两者的行为并最终生成报告。请先使用编辑工具将SQL写入临时文件进行测试，再传递给二进制文件。
bash
```
# 在SQLite上运行失败的SQL
sqlite3 :memory: < simulator-output/test.sql

# 在tursodb CLI上运行
tursodb :memory: < simulator-output/test.sql
```

Understanding Failures

理解错误类型

Oracle Failure Types

对比验证失败类型

Row set mismatch - Turso returned different rows than SQLite
Turso errored but SQLite succeeded - Turso rejected valid SQL
SQLite errored but Turso succeeded - Turso accepted invalid SQL
Schema mismatch - Tables/columns differ after DDL

行集不匹配 - Turso返回的行与SQLite不同
Turso报错但SQLite执行成功 - Turso拒绝了合法的SQL
SQLite报错但Turso执行成功 - Turso接受了非法的SQL
结构不匹配 - 执行DDL后表/列结构不同

Warning (non-fatal)

警告（非致命）

Unordered LIMIT mismatch - LIMIT without ORDER BY may return different valid rows

无排序LIMIT不匹配 - 未搭配ORDER BY的LIMIT可能返回不同的合法行

Key Source Files

核心源文件

File	Purpose
`main.rs`	CLI parsing, entry point
`runner.rs`	Main simulation loop, executes statements on both DBs
`oracle.rs`	Compares Turso vs SQLite results
`schema.rs`	Introspects schema from both databases
`memory/`	In-memory IO for deterministic simulation

文件	用途
`main.rs`	CLI参数解析、入口文件
`runner.rs`	主要模拟循环，在两个数据库上执行语句
`oracle.rs`	对比Turso与SQLite的结果
`schema.rs`	解析两个数据库的结构
`memory/`	用于确定性模拟的内存IO

Tracing

追踪日志

Set

RUST_LOG

for more detailed output:

bash

RUST_LOG=debug cargo run --bin differential_fuzzer -- --seed 12345

设置

RUST_LOG

环境变量以获取更详细的输出：

bash

RUST_LOG=debug cargo run --bin differential_fuzzer -- --seed 12345