compare-codebases

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

compare-codebases

代码库比较

jscpd --compare A B

pairs every function of folder

with the function of folder

that does the same job, and lists the functions of each folder that have no counterpart. The two folders may be in one language or in two (Python and TypeScript, Kotlin and Swift, Java and Rust). This skill explains how the comparison works and how to run one well. For porting code with the comparison as the progress measure, use the code-migration skill; for the rest of jscpd, the jscpd skill.

jscpd --compare A B

会将文件夹

中的每个函数与文件夹

中实现相同功能的函数配对，并列出每个文件夹中没有对应项的函数。两个文件夹的代码可以是同一种语言，也可以是两种不同语言（如 Python 与 TypeScript、Kotlin 与 Swift、Java 与 Rust）。本技能将说明该比较功能的工作原理以及如何高效执行比较。如果要以比较结果作为进度指标进行代码移植，请使用代码迁移技能；如需了解 jscpd 的其他功能，请参考jscpd技能。

How the comparison works

比较功能的工作原理

jscpd finds the functions of JavaScript, TypeScript, JSX, TSX, Vue, Svelte, Astro, Python, Rust, Go, Java, Kotlin, C#, C, C++, PHP, Ruby, Scala and Swift files in both folders. It turns the code of each function into a vector with a code embedding model that runs inside jscpd (CodeRankEmbed by default), so two functions that do the same job point the same way even when their languages, names and structure differ. Functions of one folder are never compared with each other.

Functions pair in two steps:

By code. Two functions pair when each is the other's closest match in the other folder, their cosine similarity reaches the model's threshold (0.4125 across languages and 0.6375 within one language, with CodeRankEmbed), and the similarity stands out from the function's other matches. A function close to the best one also pairs when it reaches a higher bar, so a feature written twice on one side gets two pairs. Functions shorter than
```
--min-tokens
```
(30 with
```
--compare
```
) or
```
--min-lines
```
(5) stay out of this step, because a short function resembles too many others.
By name. A function left over pairs with a function of the other folder under the same name, once case and underscores are ignored (
```
encodeBinary
```
,
```
encode_binary
```
,
```
_encode_binary
```
), when their similarity reaches the
```
medium
```
level (0.5625 across languages with CodeRankEmbed). A name pair skips the closest-match and stand-out checks of the first step, so it needs more than that step's threshold; otherwise every
```
load
```
and
```
init
```
of two codebases would pair. Size does not matter here, so a short port is found. A name pair has to stay within modules the first step linked. A module is the folder right under the deepest folder all files of a side share (
```
notification
```
in
```
android/notification/…
```
); a file that sits higher than the rest, such as a build script, does not move that folder up. Two modules link when one holds the most of the other's code pairs. So
```
checkPermissions
```
of one plugin does not pair with its namesake in another.

Each pair gets a level on the scale of the model, because a cosine that is high for one model is low for another:

Level	With CodeRankEmbed, across languages	Meaning
`high`	0.7125 and up	almost always the same function
`medium`	0.5625 to 0.7125	usually the same function, restructured
`low`	0.4125 to 0.5625	read both: related code pairs here too

Totals count the functions of at least

--min-tokens

tokens and

--min-lines

lines; smaller ones appear only as partners. Anonymous functions (callbacks, closures) take no part. Types, constants, SQL and UI markup are not compared.

jscpd 会在两个文件夹中识别 JavaScript、TypeScript、JSX、TSX、Vue、Svelte、Astro、Python、Rust、Go、Java、Kotlin、C#、C、C++、PHP、Ruby、Scala 和 Swift 文件中的函数。它通过 jscpd 内置的代码嵌入模型（默认为 CodeRankEmbed）将每个函数的代码转换为向量，因此即使两个函数的语言、名称和结构不同，只要实现相同功能，它们的向量方向就会一致。同一文件夹内的函数之间不会进行比较。

函数配对分为两个步骤：

按代码匹配。当两个函数互为对方文件夹中的最匹配项，且它们的余弦相似度达到模型阈值（使用 CodeRankEmbed 时，跨语言阈值为 0.4125，同语言阈值为 0.6375），同时该相似度在该函数的所有匹配结果中显著突出时，二者会配对。如果与最佳匹配项接近的另一个函数达到了更高的阈值，也会完成配对，因此一侧中重复实现的某个功能会对应两个配对。长度短于
```
--min-tokens
```
（使用
```
--compare
```
时默认为 30）或
```
--min-lines
```
（默认为 5）的函数不会参与此步骤，因为短函数与其他函数相似的概率过高。
按名称匹配。未在第一步中配对的函数，若忽略大小写和下划线后（如
```
encodeBinary
```
、
```
encode_binary
```
、
```
_encode_binary
```
）与另一文件夹中的函数名称相同，且二者相似度达到
```
medium
```
（中等）级别（使用 CodeRankEmbed 跨语言时为 0.5625），则会完成配对。按名称配对会跳过第一步中的“最匹配项”和“显著性”检查，因此它的相似度要求高于第一步的阈值；否则两个代码库中所有名为
```
load
```
和
```
init
```
的函数都会被配对。此步骤中函数长度不影响匹配，因此可以识别出短函数的移植版本。名称配对必须发生在第一步中已建立关联的模块之间。模块指的是某一侧所有文件共享的最深层目录的下一级文件夹（例如
```
android/notification/…
```
中的
```
notification
```
）；位置比其他文件更高的文件（如构建脚本）不会抬高该共享目录的层级。当一个模块包含了另一个模块的大部分代码配对时，两个模块就会建立关联。因此，一个插件中的
```
checkPermissions
```
函数不会与另一个插件中的同名函数配对。

每个配对都会根据模型的评分等级获得一个级别，因为不同模型的余弦相似度分值标准不同：

级别	使用 CodeRankEmbed 跨语言时的分值	含义
`high`	0.7125 及以上	几乎可以确定是同一个函数
`medium`	0.5625 至 0.7125	通常是同一个函数，经过了结构调整
`low`	0.4125 至 0.5625	建议同时阅读两端代码：相关代码也可能出现在此级别中

统计总数时仅计入长度至少满足

--min-tokens

和

--min-lines

要求的函数；更小的函数仅会作为配对的另一方出现。匿名函数（回调、闭包）不参与比较。类型、常量、SQL 和 UI 标记语言不在比较范围内。

A way to compare two folders

两个文件夹的比较流程

1. Pick the folders

1. 选择待比较的文件夹

Point at the code, not the repositories:
```
app/src/main/java
```
and
```
ios/Sources
```
, not the two repository roots. Build output, vendored code and generated files dilute the result; jscpd already skips what
```
.gitignore
```
excludes, and
```
--ignore
```
takes more globs.
Two folders are required, and they must not overlap:
```
app/
```
and
```
app/android/
```
is refused.
Keep parallel structures when you can (
```
ios/<module>
```
and
```
android/<module>
```
). Modules steer the name step, so matching folder names help.
For a port, put the source first and the target second, so the first line of the report is the port's progress. For two implementations that both live on, the order does not matter.
Decide about tests. Production code and tests answer different questions: compare them separately (
```
src/
```
with
```
src/
```
,
```
tests/
```
with
```
tests/
```
), or leave tests out with
```
--ignore "**/__tests__/**,**/*.test.*,**/test/**"
```
.

指向代码目录而非仓库根目录：应选择
```
app/src/main/java
```
和
```
ios/Sources
```
这类目录，而非两个仓库的根目录。构建产物、第三方依赖代码和生成的文件会稀释比较结果；jscpd 会自动跳过
```
.gitignore
```
排除的内容，还可以通过
```
--ignore
```
参数添加更多 glob 模式来忽略文件。
必须提供两个文件夹，且二者不能有重叠：例如
```
app/
```
和
```
app/android/
```
会被拒绝。
尽量保持文件夹结构平行（如
```
ios/<模块名>
```
和
```
android/<模块名>
```
）。模块会影响名称匹配步骤，因此对应的文件夹名称有助于提升匹配准确度。
如果是代码移植场景，将源文件夹放在前面，目标文件夹放在后面，这样报告的第一行就会显示移植进度。如果是两个并行维护的实现，顺序无关紧要。
决定是否包含测试代码。生产代码和测试代码对应的问题不同：可以分开比较（
```
src/
```
与
```
src/
```
比，
```
tests/
```
与
```
tests/
```
比），或者通过
```
--ignore "**/__tests__/**,**/*.test.*,**/test/**"
```
参数排除测试文件。

2. Get the model

2. 获取模型

The first run needs the model (548 MB). Ask the user before downloading it:

bash

npx jscpd --semantic-download

jscpd caches the vectors per pair of folders, so later runs embed only the functions whose code changed.

首次运行需要下载模型（大小为 548 MB）。下载前请先询问用户：

bash

npx jscpd --semantic-download

jscpd 会按文件夹对缓存向量，因此后续运行时只会对代码发生变更的函数重新生成嵌入。

3. Read the overview

3. 查看概览报告

bash

npx jscpd --compare billing-py/ billing-ts/

text

 71% 5 of 7 functions in billing-py/ have a counterpart in billing-ts/
 80% 4 of 5 functions in billing-ts/ have a counterpart in billing-py/

billing-py/
  file         paired  similarity  counterpart
  billing.py   4 / 5   0.89        billing.ts
  shipping.py  1 / 2   0.91        shipping.ts

billing-ts/
  file         paired  similarity  counterpart
  billing.ts   3 / 4   0.89        billing.py
  shipping.ts  1 / 1   0.91        shipping.py

Paired under other names (1):
  billing-py/                   billing-ts/             similarity
  billing.py:28 tax_for_region  billing.ts:27 salesTax  0.87 high

Only in billing-py/ (2):
  billing.py (1)
    46  due_date                6 lines
  shipping.py (1)
    18  estimate_delivery_days  8 lines

Only in billing-ts/ (1):
  billing.ts (1)
    39  toCurrency  8 lines

The two top lines give the share of each folder's functions that have a counterpart in the other.
The file tables give each file's paired functions, the mean similarity of its pairs (with the number of
```
low
```
pairs, as in
```
0.62, 1 low
```
), and the file on the other side that holds most of its counterparts.
"Paired under other names" lists the pairs whose names differ even once case and underscores are ignored. A search by name never finds these.
"Only in" lists the functions with no counterpart, per folder, grouped by file with the number of each file's functions, then each function's first line, name and length.
In colour, levels are green (
```
high
```
), yellow (
```
medium
```
) and red (
```
low
```
), and the shares are green when complete, yellow when partial and red when nothing is paired. Pass
```
--no-colors
```
when you parse the console output, or read the JSON report instead.
If one folder has no functions, the report is one line of totals and
```
<folder> has no functions yet
```
. Check the path and the languages before concluding anything else.

bash

npx jscpd --compare billing-py/ billing-ts/

text

 71% 5 of 7 functions in billing-py/ have a counterpart in billing-ts/
 80% 4 of 5 functions in billing-ts/ have a counterpart in billing-py/

billing-py/
  file         paired  similarity  counterpart
  billing.py   4 / 5   0.89        billing.ts
  shipping.py  1 / 2   0.91        shipping.ts

billing-ts/
  file         paired  similarity  counterpart
  billing.ts   3 / 4   0.89        billing.py
  shipping.ts  1 / 1   0.91        shipping.py

Paired under other names (1):
  billing-py/                   billing-ts/             similarity
  billing.py:28 tax_for_region  billing.ts:27 salesTax  0.87 high

Only in billing-py/ (2):
  billing.py (1)
    46  due_date                6 lines
  shipping.py (1)
    18  estimate_delivery_days  8 lines

Only in billing-ts/ (1):
  billing.ts (1)
    39  toCurrency  8 lines

最上方的两行显示每个文件夹中在另一文件夹里有对应项的函数占比。
文件表格展示每个文件的已配对函数数量、配对的平均相似度（同时标注
```
low
```
级别的配对数，例如
```
0.62, 1 low
```
），以及另一侧包含其最多对应项的文件。
“异名配对（Paired under other names）”部分列出了即使忽略大小写和下划线后名称仍不相同的配对。这类配对无法通过名称搜索找到。
“仅存在于（Only in）”部分按文件夹列出没有对应项的函数，按文件分组并标注每个文件的函数数量，随后是每个函数的起始行号、名称和长度。
在彩色输出中，级别分别用绿色（
```
high
```
）、黄色（
```
medium
```
）和红色（
```
low
```
）表示；占比的颜色则是：全部配对为绿色，部分配对为黄色，无配对为红色。如果你需要解析控制台输出，请添加
```
--no-colors
```
参数，或者直接读取 JSON 报告。
如果其中一个文件夹没有函数，报告只会显示一行统计信息和
```
<folder> has no functions yet
```
（该文件夹暂无函数）。在得出其他结论前，请先检查路径和支持的语言是否正确。

4. Check the pairs

4. 检查配对结果

bash

npx jscpd --compare billing-py/ billing-ts/ -r console-full

console-full

adds every pair with its similarity and level, and marks the pairs found by name. Before reporting the comparison as reliable, read a sample:

two or three
```
high
```
pairs, to confirm the pairing works on this code;
every
```
low
```
pair, since related code pairs there too (a client call and the endpoint it calls, a function counting UTF-8 bytes and one converting a string to them);
the pairs marked
```
by name
```
, since a same-named function can do something else.

When a sample pair is wrong, say which one and why.

bash

npx jscpd --compare billing-py/ billing-ts/ -r console-full

console-full

模式会输出所有配对的相似度和级别，并标记出通过名称匹配的配对。在判定比较结果可靠之前，请抽样检查以下内容：

抽取两到三个
```
high
```
级别的配对，确认配对逻辑适用于当前代码；
检查所有
```
low
```
级别的配对，因为相关代码也可能出现在这个级别中（例如客户端调用和它调用的接口、统计 UTF-8 字节数的函数和将字符串转换为 UTF-8 字节的函数）；
检查标记为
```
by name
```
（按名称匹配）的配对，因为同名函数可能实现的是不同的功能。

如果抽样发现某个配对是错误的，请说明具体是哪一个以及错误原因。

5. Check the functions with no counterpart

5. 检查无对应项的函数

A function in an "Only in" list is either missing on the other side, or its counterpart was not recognized. Before calling it missing, search the other folder: in its counterpart file, under other names, as part of a larger function, or replaced by a library or platform call. The known gaps in pairing:

constructors across languages (a Java constructor and Rust's
```
new
```
, Kotlin's
```
constructor
```
, Swift's
```
init
```
) pair only when their code is similar enough;
a short function renamed on the other side (
```
add_history
```
for
```
_finder_penalty_add_history
```
);
one function split into several, or several merged into one: only the closest part may pair.

Functions that exist on one side only by design are expected: platform glue (an iOS delegate callback, an Android notification channel), helpers a language needs and another does not, features one side dropped.

“Only in”列表中的函数，要么是另一侧确实缺失该函数，要么是其对应项未被识别。在判定函数缺失之前，请在另一个文件夹中搜索确认：可能在对应的文件中、可能使用了其他名称、可能被整合到了一个更大的函数中，也可能被库或平台调用替代了。目前已知的配对盲区包括：

跨语言的构造函数（例如 Java 的构造函数与 Rust 的
```
new
```
、Kotlin 的
```
constructor
```
、Swift 的
```
init
```
）只有在代码足够相似时才会配对；
短函数在另一侧被重命名（例如
```
add_history
```
对应
```
_finder_penalty_add_history
```
）；
一个函数被拆分为多个，或多个函数被合并为一个：只有最相似的部分可能会配对。

有些函数仅存在于一侧是设计使然，属于正常情况：例如平台适配代码（iOS 的代理回调、Android 的通知渠道）、某一种语言需要而另一种语言不需要的辅助函数、某一侧已下线的功能。

6. Report

6. 输出报告

For the user, summarize in a few lines: the two percentages, the notable renamed pairs, the

low

pairs you checked and what you found, and the real gaps on each side, grouped by file or module. For a document or an issue, write the Markdown report and attach or paste it:

bash

npx jscpd --compare billing-py/ billing-ts/ -r markdown -o .jscpd-compare

For your own processing, read the JSON report (

-r json

, written to

jscpd-compare.json

). It has

sides[0]

and

sides[1]

, each with

path

functions

matched

percentage

files

(

file

functions

matched

counterpart

similarity

lowPairs

) and

unmatched

(

file

name

start

end

), and

pairs

, each with

similarity

level

renamed

and

matchedBy

. Paths are relative to each folder. Write reports outside the repository or add the folder to

.gitignore

向用户反馈时，用几句话总结：两个文件夹的配对占比、值得注意的异名配对、你检查过的

low

级别配对及发现的问题，以及每一侧真正缺失的函数（按文件或模块分组）。如果需要写入文档或 Issue，可以生成 Markdown 报告并附件或粘贴：

bash

npx jscpd --compare billing-py/ billing-ts/ -r markdown -o .jscpd-compare

如果需要自行处理数据，可以读取 JSON 报告（通过

-r json

参数生成，写入

jscpd-compare.json

文件）。报告包含

sides[0]

和

sides[1]

两个数组，每个元素包含

path

（路径）、

functions

（函数总数）、

matched

（已配对数）、

percentage

（占比）、

files

（文件列表，每个文件包含

file

、

functions

、

matched

、

counterpart

、

similarity

、

lowPairs

字段）和

unmatched

（未配对函数列表，每个函数包含

file

、

name

、

start

、

end

字段）；此外还有

pairs

数组，每个配对包含

、

similarity

、

level

、

renamed

和

matchedBy

字段。所有路径均相对于对应文件夹。请将报告生成在仓库外部，或者将报告文件夹加入

.gitignore

。

Options that change the result

影响结果的可选参数

```
--min-tokens
```
and
```
--min-lines
```
set which functions count. Raise them to focus on substantial functions.
```
--ignore
```
and
```
--format
```
narrow the files, for example to leave out tests or build scripts.
```
--semantic-model
```
picks another model, and
```
--semantic-url
```
an OpenAI-compatible embeddings API;
```
npx jscpd --semantic-models
```
lists the models with calibrated thresholds.
```
--semantic-threshold
```
and
```
--semantic-same-threshold
```
move the thresholds; change them only after reading pairs on both sides of the new value.
```
--semantic-rebuild-cache
```
embeds everything again, which is needed only when the cache is suspect.

Keep the folders, their order, the options and the model the same when you compare runs over time, or the numbers stop being comparable. The exit code is 0 whatever the result.

```
--min-tokens
```
和
```
--min-lines
```
用于设定参与统计的函数门槛。调高这两个参数可以聚焦于更核心的函数。
```
--ignore
```
和
```
--format
```
用于缩小文件范围，例如排除测试文件或构建脚本。
```
--semantic-model
```
用于选择其他模型，
```
--semantic-url
```
用于指定兼容 OpenAI 格式的嵌入 API；
```
npx jscpd --semantic-models
```
命令可以列出所有已校准阈值的模型。
```
--semantic-threshold
```
和
```
--semantic-same-threshold
```
用于调整阈值；只有在查看过新阈值上下的配对结果后再修改这两个参数。
```
--semantic-rebuild-cache
```
会强制重新生成所有嵌入的向量，仅在怀疑缓存有问题时使用。

如果需要多次运行比较以追踪进度，请保持文件夹、文件夹顺序、参数和模型都一致，否则统计数据将不具备可比性。无论比较结果如何，命令的退出码始终为 0。

Limits

局限性

Similarity does not see small differences in behavior: two versions that drifted apart still pair, often at
```
high
```
. Only tests and reading the code catch drift.
The more one side is restructured, the fewer of its functions pair by code.
Only functions are compared.
The model runs on the CPU. The first run embedded about 25 functions a second on an Apple M1 (261 functions in 11 seconds) and can be several times slower on a small CI machine; later runs reuse the cache.

相似度无法识别行为上的细微差异：两个已经产生分歧的版本仍可能配对，而且通常是
```
high
```
级别。只有通过测试和代码审查才能发现功能偏差。
一侧的代码重构程度越高，通过代码匹配到的函数数量就越少。
仅对函数进行比较。
模型在 CPU 上运行。在 Apple M1 芯片上，首次运行每秒可处理约 25 个函数（261 个函数耗时 11 秒）；在配置较低的 CI 机器上速度可能慢数倍；后续运行会复用缓存，速度更快。