api-performance-api-performance
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseBackend Performance Optimization
后端性能优化
Quick Guide: Optimize backend performance through database query optimization (indexes, prepared statements, avoiding N+1), caching strategies (cache-aside, write-through), connection pooling, and non-blocking async patterns. Always measure before optimizing -- run EXPLAIN ANALYZE, check event loop lag, and track cache hit rates before adding complexity.
<critical_requirements>
快速指南: 通过数据库查询优化(索引、预编译语句、避免N+1查询)、缓存策略(旁路缓存、写穿透)、连接池和非阻塞异步模式来优化后端性能。优化前务必先进行度量——在增加复杂度之前,先运行EXPLAIN ANALYZE、检查事件循环延迟并跟踪缓存命中率。
<critical_requirements>
CRITICAL: Before Using This Skill
重要提示:使用此技能之前
All code must follow project conventions in CLAUDE.md (kebab-case, named exports, import ordering,, named constants)import type
(You MUST always release database connections back to the pool using blocks)
finally(You MUST use eager loading or batching (DataLoader) to prevent N+1 queries -- never lazy load in loops)
(You MUST set TTL on all cached data to prevent stale data and memory exhaustion)
(You MUST offload CPU-intensive work to Worker Threads -- blocking the event loop degrades all requests)
</critical_requirements>
Detailed Resources:
- examples/core.md - Database patterns: connection pooling, N+1 prevention, indexing, prepared statements, pagination
- examples/caching.md - Cache-aside, write-through, invalidation, key strategies, TTL guidance
- examples/async.md - Event loop optimization, worker threads, chunked processing, concurrency control
- reference.md - Decision frameworks, performance monitoring
Auto-detection: connection pool, query optimization, database index, N+1, caching, cache invalidation, prepared statement, worker threads, event loop, CPU-bound, latency, throughput, performance tuning, EXPLAIN ANALYZE, keyset pagination, cache-aside, write-through
When to use:
- Database queries taking > 100ms
- High-traffic endpoints with repeated data fetches
- API responses with multiple related entities (N+1 risk)
- CPU-intensive operations blocking request handling
- Need to reduce database load via caching
When NOT to use:
- Premature optimization without measuring first
- Simple CRUD with low traffic (adds complexity without benefit)
- Data that changes frequently and must always be fresh (caching adds staleness)
- Development/debugging (caching obscures issues)
Key patterns covered:
- Database indexing strategies (composite, partial, covering)
- Connection pooling with guaranteed release
- N+1 query prevention (eager loading, DataLoader)
- Caching strategies (cache-aside, write-through, invalidation)
- Event loop optimization (async I/O, setImmediate chunking)
- Worker threads for CPU-bound operations
- Keyset pagination for large datasets
<philosophy>
所有代码必须遵循CLAUDE.md中的项目约定(短横线命名、命名导出、导入顺序、、命名常量)import type
(必须始终使用块将数据库连接释放回连接池)
finally(必须使用预加载或批处理(DataLoader)来防止N+1查询——绝不在循环中使用懒加载)
(必须为所有缓存数据设置TTL,以防止数据过期和内存耗尽)
(必须将CPU密集型工作卸载到Worker Threads——阻塞事件循环会降低所有请求的性能)
</critical_requirements>
详细资源:
- examples/core.md - 数据库模式:连接池、N+1查询预防、索引、预编译语句、分页
- examples/caching.md - 旁路缓存、写穿透、缓存失效、键策略、TTL指南
- examples/async.md - 事件循环优化、Worker Threads、分块处理、并发控制
- reference.md - 决策框架、性能监控
自动检测项: 连接池、查询优化、数据库索引、N+1查询、缓存、缓存失效、预编译语句、Worker Threads、事件循环、CPU密集型、延迟、吞吐量、性能调优、EXPLAIN ANALYZE、键集分页、旁路缓存、写穿透
适用场景:
- 数据库查询耗时超过100ms
- 高流量端点存在重复数据获取
- API响应包含多个关联实体(存在N+1查询风险)
- CPU密集型操作阻塞请求处理
- 需要通过缓存降低数据库负载
不适用场景:
- 未先进行度量就过早优化
- 低流量的简单CRUD操作(增加复杂度却无收益)
- 频繁变更且必须始终保持最新的数据(缓存会导致数据过期)
- 开发/调试阶段(缓存会掩盖问题)
涵盖的核心模式:
- 数据库索引策略(复合索引、部分索引、覆盖索引)
- 确保连接释放的连接池
- N+1查询预防(预加载、DataLoader)
- 缓存策略(旁路缓存、写穿透、缓存失效)
- 事件循环优化(异步I/O、setImmediate分块)
- 用于CPU密集型操作的Worker Threads
- 大数据集的键集分页
<philosophy>
Philosophy
理念
Backend performance optimization follows one core principle: measure first, optimize second. Premature optimization wastes development time and adds complexity without evidence of benefit.
The Three Pillars of Backend Performance:
- Database Optimization - Indexes, query planning, N+1 prevention, pagination
- Caching - Reduce repeated expensive operations with TTL-bounded cache
- Async Efficiency - Never block the event loop
When to optimize:
- Response times exceed SLA thresholds
- Database CPU/memory approaching limits
- Metrics show specific bottlenecks (EXPLAIN ANALYZE, event loop lag)
- Load testing reveals scaling issues
When NOT to optimize:
- "It might be slow someday" (premature)
- Optimizing cold paths (rarely executed code)
- Before profiling identifies the actual bottleneck
<patterns>
后端性能优化遵循一个核心原则:先度量,后优化。过早优化会浪费开发时间,且在无性能问题证据的情况下增加复杂度。
后端性能三大支柱:
- 数据库优化 - 索引、查询规划、N+1查询预防、分页
- 缓存 - 用带TTL限制的缓存减少重复的高开销操作
- 异步效率 - 绝不阻塞事件循环
何时优化:
- 响应时间超出SLA阈值
- 数据库CPU/内存接近上限
- 指标显示存在特定瓶颈(EXPLAIN ANALYZE、事件循环延迟)
- 负载测试暴露出扩展问题
何时不优化:
- “某天可能会变慢”(过早优化)
- 优化冷路径(极少执行的代码)
- 未通过性能分析确定实际瓶颈之前
<patterns>
Core Patterns
核心模式
Pattern 1: Connection Pooling with Guaranteed Release
模式1:确保连接释放的连接池
Connection pooling reuses database connections instead of creating new ones per request. A PostgreSQL handshake takes 20-30ms -- pooling eliminates this overhead.
Key rules:
- Use for simple queries (auto-manages connection lifecycle)
pool.query() - For transactions, manually checkout with and always release in
pool.connect()finally - Listen for pool errors (idle clients can still emit errors)
typescript
// Transaction with guaranteed connection release
async function createUserWithProfile(
userData: UserData,
profileData: ProfileData,
) {
const client = await pool.connect();
try {
await client.query("BEGIN");
const userResult = await client.query(
"INSERT INTO users (name, email) VALUES ($1, $2) RETURNING id",
[userData.name, userData.email],
);
await client.query("INSERT INTO profiles (user_id, bio) VALUES ($1, $2)", [
userResult.rows[0].id,
profileData.bio,
]);
await client.query("COMMIT");
return userResult.rows[0];
} catch (error) {
await client.query("ROLLBACK");
throw error;
} finally {
client.release(); // CRITICAL: Always release back to pool
}
}Why good: guarantees connection release even on error, preventing pool exhaustion
finallySee examples/core.md for full pool configuration, sizing formula, and external pooler guidance.
连接池复用数据库连接,而非为每个请求创建新连接。PostgreSQL握手耗时20-30ms——连接池可消除此开销。
关键规则:
- 简单查询使用(自动管理连接生命周期)
pool.query() - 事务处理时,使用手动取出连接,并始终在
pool.connect()中释放finally - 监听连接池错误(空闲客户端仍可能触发错误)
typescript
// 确保连接释放的事务处理
async function createUserWithProfile(
userData: UserData,
profileData: ProfileData,
) {
const client = await pool.connect();
try {
await client.query("BEGIN");
const userResult = await client.query(
"INSERT INTO users (name, email) VALUES ($1, $2) RETURNING id",
[userData.name, userData.email],
);
await client.query("INSERT INTO profiles (user_id, bio) VALUES ($1, $2)", [
userResult.rows[0].id,
profileData.bio,
]);
await client.query("COMMIT");
return userResult.rows[0];
} catch (error) {
await client.query("ROLLBACK");
throw error;
} finally {
client.release(); // 重要:务必释放回连接池
}
}优势: 可确保即使发生错误也能释放连接,避免连接池耗尽
finally查看examples/core.md获取完整的连接池配置、大小计算公式和外部连接池器指南。
Pattern 2: N+1 Query Prevention
模式2:N+1查询预防
The N+1 problem occurs when fetching N records triggers N additional queries for related data. With 100 records, that's 101 database round-trips.
Two solutions:
- Eager loading (ORM ) -- single query with JOINs for known relationships
.with() - DataLoader -- batches calls into single query per tick, ideal for GraphQL
.load()
typescript
// Eager loading: single query fetches jobs + companies + skills
const jobs = await db.query.jobs.findMany({
where: and(eq(jobs.isActive, true), isNull(jobs.deletedAt)),
with: {
company: { with: { locations: true } },
jobSkills: { with: { skill: true } },
},
});typescript
// BAD: N+1 anti-pattern -- one query per job
for (const job of jobs) {
job.company = await db.query.companies.findFirst({
where: eq(companies.id, job.companyId),
});
}Why bad: 1 query for jobs + N queries for companies, latency grows linearly with data size
See examples/core.md for DataLoader batching pattern.
N+1问题指获取N条记录时会触发N次额外的关联数据查询。若有100条记录,就会产生101次数据库往返。
两种解决方案:
- 预加载(ORM的)——通过JOIN的单条查询获取已知关联数据
.with() - DataLoader——将调用批量处理为每轮事件循环一次查询,适用于GraphQL
.load()
typescript
// 预加载:单条查询获取职位+公司+技能
const jobs = await db.query.jobs.findMany({
where: and(eq(jobs.isActive, true), isNull(jobs.deletedAt)),
with: {
company: { with: { locations: true } },
jobSkills: { with: { skill: true } },
},
});typescript
// 错误示例:N+1反模式——每个职位一次查询
for (const job of jobs) {
job.company = await db.query.companies.findFirst({
where: eq(companies.id, job.companyId),
});
}弊端: 1次职位查询 + N次公司查询,延迟随数据量线性增长
查看examples/core.md获取DataLoader批处理模式。
Pattern 3: Database Indexing
模式3:数据库索引
Indexes speed up queries by avoiding full table scans. Index columns used in WHERE, JOIN, and ORDER BY clauses.
typescript
// Strategic indexes on a table definition
export const jobs = pgTable(
"jobs",
{
id: uuid("id").primaryKey().defaultRandom(),
companyId: uuid("company_id").notNull(),
country: varchar("country", { length: 100 }),
employmentType: varchar("employment_type", { length: 50 }),
isActive: boolean("is_active").default(true),
createdAt: timestamp("created_at").defaultNow(),
deletedAt: timestamp("deleted_at"),
},
(table) => [
// Composite index for common filter combination
index("jobs_country_employment_idx").on(
table.country,
table.employmentType,
),
// Partial index -- only indexes active non-deleted jobs
index("jobs_active_idx")
.on(table.isActive, table.createdAt)
.where(sql`${table.deletedAt} IS NULL`),
// Foreign key index for JOIN performance
index("jobs_company_id_idx").on(table.companyId),
],
);Index Decision Framework:
| Column Usage | Index Type | When to Use |
|---|---|---|
| WHERE equality | B-tree (default) | High-selectivity columns |
| WHERE range (>, <, BETWEEN) | B-tree | Date ranges, numeric ranges |
| WHERE multiple columns | Composite | Queries always filter by same columns together |
| WHERE on subset | Partial | Most queries filter on active/non-deleted |
| Full-text search | GIN/GiST | Text search with LIKE, tsvector |
| JSON field access | GIN | JSONB column queries |
Composite index column order: Equality conditions first, range conditions last, high selectivity first.
See examples/core.md for EXPLAIN ANALYZE examples, index monitoring queries, and unused index detection.
索引通过避免全表扫描来加速查询。为WHERE、JOIN和ORDER BY子句中使用的列创建索引。
typescript
// 表定义中的策略性索引
export const jobs = pgTable(
"jobs",
{
id: uuid("id").primaryKey().defaultRandom(),
companyId: uuid("company_id").notNull(),
country: varchar("country", { length: 100 }),
employmentType: varchar("employment_type", { length: 50 }),
isActive: boolean("is_active").default(true),
createdAt: timestamp("created_at").defaultNow(),
deletedAt: timestamp("deleted_at"),
},
(table) => [
// 针对常用筛选组合的复合索引
index("jobs_country_employment_idx").on(
table.country,
table.employmentType,
),
// 部分索引——仅索引活跃且未删除的职位
index("jobs_active_idx")
.on(table.isActive, table.createdAt)
.where(sql`${table.deletedAt} IS NULL`),
// 用于JOIN性能的外键索引
index("jobs_company_id_idx").on(table.companyId),
],
);索引决策框架:
| 列用法 | 索引类型 | 适用场景 |
|---|---|---|
| WHERE等值筛选 | B树(默认) | 高选择性列 |
| WHERE范围筛选(>, <, BETWEEN) | B树 | 日期范围、数值范围 |
| WHERE多列筛选 | Composite | 查询总是同时筛选相同列的场景 |
| WHERE子集筛选 | Partial | 大多数查询筛选活跃/未删除数据的场景 |
| 全文搜索 | GIN/GiST | 使用LIKE、tsvector的文本搜索 |
| JSON字段访问 | GIN | JSONB列查询 |
复合索引列顺序: 等值条件在前,范围条件在后,高选择性列在前。
查看examples/core.md获取EXPLAIN ANALYZE示例、索引监控查询和未使用索引检测方法。
Pattern 4: Cache-Aside with TTL
模式4:带TTL的旁路缓存
The most common caching pattern. Check cache first, fetch from database on miss, store with TTL.
typescript
const CACHE_TTL_SECONDS = 300;
const CACHE_PREFIX = "app:user";
async function getUserById(userId: string): Promise<User | null> {
const cacheKey = `${CACHE_PREFIX}:${userId}`;
const cached = await cacheClient.get(cacheKey);
if (cached) return JSON.parse(cached) as User;
const user = await db.query.users.findFirst({ where: eq(users.id, userId) });
if (!user) return null;
await cacheClient.set(cacheKey, JSON.stringify(user), {
EX: CACHE_TTL_SECONDS,
});
return user;
}Why good: TTL prevents stale data accumulation, namespaced keys prevent collisions, early return on cache hit
See examples/caching.md for write-through, tag-based invalidation, key strategies, and TTL guidance.
最常用的缓存模式。先检查缓存,未命中时从数据库获取,存储时设置TTL。
typescript
const CACHE_TTL_SECONDS = 300;
const CACHE_PREFIX = "app:user";
async function getUserById(userId: string): Promise<User | null> {
const cacheKey = `${CACHE_PREFIX}:${userId}`;
const cached = await cacheClient.get(cacheKey);
if (cached) return JSON.parse(cached) as User;
const user = await db.query.users.findFirst({ where: eq(users.id, userId) });
if (!user) return null;
await cacheClient.set(cacheKey, JSON.stringify(user), {
EX: CACHE_TTL_SECONDS,
});
return user;
}优势: TTL防止过期数据累积,命名空间键避免冲突,缓存命中时提前返回
查看examples/caching.md获取写穿透、基于标签的缓存失效、键策略和TTL指南。
Pattern 5: Worker Threads for CPU-Bound Operations
模式5:用于CPU密集型操作的Worker Threads
Node.js uses a single thread for JavaScript. CPU-intensive work blocks ALL concurrent requests.
Rule of thumb:
| CPU Duration | Solution | Rationale |
|---|---|---|
| < 50ms | Keep on main thread | Worker overhead not worth it |
| 50-500ms | setImmediate chunking | Yields to event loop between chunks |
| > 500ms | Worker Threads | Offload completely to separate thread |
typescript
// Chunked processing with setImmediate -- yields to event loop between batches
const CHUNK_SIZE = 100;
async function processLargeArray(items: Item[]): Promise<ProcessedItem[]> {
const results: ProcessedItem[] = [];
for (let i = 0; i < items.length; i += CHUNK_SIZE) {
const chunk = items.slice(i, i + CHUNK_SIZE);
for (const item of chunk) {
results.push(expensiveTransform(item));
}
if (i + CHUNK_SIZE < items.length) {
await new Promise((resolve) => setImmediate(resolve));
}
}
return results;
}See examples/async.md for worker pool implementation, concurrency control with p-limit, and event loop lag monitoring.
Node.js使用单线程执行JavaScript。CPU密集型工作会阻塞所有并发请求。
经验法则:
| CPU耗时 | 解决方案 | 理由 |
|---|---|---|
| < 50ms | 保留在主线程 | Worker的开销得不偿失 |
| 50-500ms | setImmediate分块处理 | 在分块之间让出事件循环 |
| > 500ms | Worker Threads | 完全卸载到独立线程 |
typescript
// 使用setImmediate的分块处理——在批次之间让出事件循环
const CHUNK_SIZE = 100;
async function processLargeArray(items: Item[]): Promise<ProcessedItem[]> {
const results: ProcessedItem[] = [];
for (let i = 0; i < items.length; i += CHUNK_SIZE) {
const chunk = items.slice(i, i + CHUNK_SIZE);
for (const item of chunk) {
results.push(expensiveTransform(item));
}
if (i + CHUNK_SIZE < items.length) {
await new Promise((resolve) => setImmediate(resolve));
}
}
return results;
}查看examples/async.md获取Worker池实现、使用p-limit的并发控制和事件循环延迟监控方法。
Pattern 6: Keyset Pagination for Large Datasets
模式6:大数据集的键集分页
OFFSET pagination scans all previous rows -- at OFFSET 100,000 the database reads and discards 100,000 rows. Keyset pagination uses a cursor for constant-time performance.
sql
-- BAD: OFFSET scans all previous rows
SELECT * FROM products ORDER BY id LIMIT 20 OFFSET 100000;
-- GOOD: Keyset pagination -- constant time regardless of position
SELECT * FROM products
WHERE id > :last_seen_id
ORDER BY id LIMIT 20;When to use offset: Small datasets (< 100k rows), need total count, random page access required.
When to use keyset: Large datasets, infinite scroll, real-time data where inserts shouldn't cause duplicates.
See examples/core.md for full TypeScript implementations of both patterns.
</patterns>
<red_flags>
OFFSET分页会扫描所有之前的行——当OFFSET为100,000时,数据库会读取并丢弃100,000行。键集分页使用游标实现恒定时间性能。
sql
-- 错误示例:OFFSET会扫描所有之前的行
SELECT * FROM products ORDER BY id LIMIT 20 OFFSET 100000;
-- 正确示例:键集分页——性能与位置无关,恒定时间
SELECT * FROM products
WHERE id > :last_seen_id
ORDER BY id LIMIT 20;何时使用OFFSET: 小数据集(<10万行)、需要总行数、需要随机页面访问。
何时使用键集: 大数据集、无限滚动、插入操作不应导致重复数据的实时场景。
查看examples/core.md获取两种模式的完整TypeScript实现。
</patterns>
<red_flags>
RED FLAGS
风险警示
High Priority Issues:
- Missing connection release -- connections never returned to pool cause pool exhaustion and application hangs
- N+1 queries in loops -- fetching related data one-by-one instead of eager loading destroys performance
- Blocking event loop -- synchronous I/O or CPU-intensive work blocks all concurrent requests
- Cache without TTL -- unbounded cache grows until memory exhaustion or serves infinitely stale data
- Full table scans on large tables -- missing indexes on WHERE/JOIN columns
Medium Priority Issues:
- No index on foreign keys -- JOINs and ON DELETE CASCADE become slow
- Over-indexing -- every index slows writes; remove unused indexes with
pg_stat_user_indexes - Cache key collisions -- generic keys cause wrong data returned to wrong users
- Offset pagination on large tables -- OFFSET scans all previous rows; use keyset pagination
- Unbounded parallelism -- on 10,000 items overwhelms downstream services
Promise.all
Gotchas & Edge Cases:
- EXPLAIN shows estimates; EXPLAIN ANALYZE shows actual -- always use ANALYZE for real performance data
- Composite index (a, b) does NOT help queries filtering only on b -- column order matters
- Applying functions to indexed columns (e.g., ) prevents index use -- rewrite as range conditions
YEAR(created_at) - DataLoader caches within request -- don't reuse across requests or you get stale data
- Worker threads have ~30ms startup overhead -- don't use for fast operations
- Connection pool can cause "connection terminated unexpectedly" if set too short
idleTimeoutMillis - fetches unnecessary data and prevents covering index optimization -- select specific columns
SELECT * - Cache SET with EX option replaces existing TTL -- calling SET again resets expiration timer
</red_flags>
<critical_reminders>
高优先级问题:
- 未释放连接——连接未返回池会导致连接池耗尽和应用挂起
- 循环中的N+1查询——逐个获取关联数据而非预加载会严重破坏性能
- 阻塞事件循环——同步I/O或CPU密集型工作会阻塞所有并发请求
- 无TTL的缓存——无限制的缓存会持续增长直到内存耗尽或返回永久过期的数据
- 大数据集上的全表扫描——WHERE/JOIN列缺少索引
中优先级问题:
- 外键无索引——JOIN和ON DELETE CASCADE会变慢
- 过度索引——每个索引都会降低写入性能;使用移除未使用的索引
pg_stat_user_indexes - 缓存键冲突——通用键会导致错误的数据返回给错误的用户
- 大数据集使用OFFSET分页——OFFSET会扫描所有之前的行;使用键集分页
- 无限制并行——对10,000个项使用会压垮下游服务
Promise.all
陷阱与边缘情况:
- EXPLAIN显示估算值;EXPLAIN ANALYZE显示实际值——始终使用ANALYZE获取真实性能数据
- 复合索引(a, b)对仅筛选b的查询无帮助——列顺序很重要
- 对索引列应用函数(如)会导致索引失效——重写为范围条件
YEAR(created_at) - DataLoader在请求内缓存——不要跨请求复用,否则会获取过期数据
- Worker Threads启动开销约30ms——不要用于快速操作
- 连接池的设置过短会导致“连接意外终止”
idleTimeoutMillis - 会获取不必要的数据并阻止覆盖索引优化——选择特定列
SELECT * - 使用EX选项的Cache SET会替换现有TTL——再次调用SET会重置过期时间
</red_flags>
<critical_reminders>
CRITICAL REMINDERS
重要提醒
All code must follow project conventions in CLAUDE.md
(You MUST always release database connections back to the pool using blocks)
finally(You MUST use eager loading or batching (DataLoader) to prevent N+1 queries -- never lazy load in loops)
(You MUST set TTL on all cached data to prevent stale data and memory exhaustion)
(You MUST offload CPU-intensive work to Worker Threads -- blocking the event loop degrades all requests)
Failure to follow these rules will cause connection pool exhaustion, N+1 performance degradation, memory leaks from unbounded caches, and blocked event loops affecting all concurrent requests.
</critical_reminders>
所有代码必须遵循CLAUDE.md中的项目约定
(必须始终使用块将数据库连接释放回连接池)
finally(必须使用预加载或批处理(DataLoader)来防止N+1查询——绝不在循环中使用懒加载)
(必须为所有缓存数据设置TTL,以防止数据过期和内存耗尽)
(必须将CPU密集型工作卸载到Worker Threads——阻塞事件循环会降低所有请求的性能)
不遵守这些规则会导致连接池耗尽、N+1查询性能下降、无限制缓存导致内存泄漏,以及阻塞事件循环影响所有并发请求。
</critical_reminders>