caching-strategies

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Caching Strategies

多层缓存策略

Speed up your app with smart caching at every layer.

通过在各层级引入智能缓存，提升应用响应速度。

When to Use This Skill

适用场景

Slow database queries
Expensive computations
External API responses
Session data
Frequently accessed data

数据库查询缓慢
计算成本高昂的操作
外部API响应结果
会话数据
高频访问数据

Cache Layers

缓存层级

┌─────────────────────────────────────────────────────┐
│                    Request                           │
└─────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────────────────┐
│  Layer 1: HTTP Cache (CDN/Browser)                  │
│  - Static assets, public API responses              │
│  - Cache-Control headers                            │
└─────────────────────────────────────────────────────┘
         │ miss
         ▼
┌─────────────────────────────────────────────────────┐
│  Layer 2: In-Memory Cache (Node/Process)            │
│  - Hot data, computed values                        │
│  - LRU eviction                                     │
└─────────────────────────────────────────────────────┘
         │ miss
         ▼
┌─────────────────────────────────────────────────────┐
│  Layer 3: Distributed Cache (Redis)                 │
│  - Shared across instances                          │
│  - Session data, rate limits                        │
└─────────────────────────────────────────────────────┘
         │ miss
         ▼
┌─────────────────────────────────────────────────────┐
│  Layer 4: Database                                  │
└─────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────┐
│                    Request                           │
└─────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────────────────┐
│  Layer 1: HTTP Cache (CDN/Browser)                  │
│  - Static assets, public API responses              │
│  - Cache-Control headers                            │
└─────────────────────────────────────────────────────┘
         │ miss
         ▼
┌─────────────────────────────────────────────────────┐
│  Layer 2: In-Memory Cache (Node/Process)            │
│  - Hot data, computed values                        │
│  - LRU eviction                                     │
└─────────────────────────────────────────────────────┘
         │ miss
         ▼
┌─────────────────────────────────────────────────────┐
│  Layer 3: Distributed Cache (Redis)                 │
│  - Shared across instances                          │
│  - Session data, rate limits                        │
└─────────────────────────────────────────────────────┘
         │ miss
         ▼
┌─────────────────────────────────────────────────────┐
│  Layer 4: Database                                  │
└─────────────────────────────────────────────────────┘

TypeScript Implementation

TypeScript 实现

Multi-Layer Cache

多层缓存类

typescript

// cache.ts
import { Redis } from 'ioredis';
import { LRUCache } from 'lru-cache';

interface CacheOptions {
  ttl?: number;           // Time to live in seconds
  staleWhileRevalidate?: number;
  tags?: string[];        // For invalidation
}

class MultiLayerCache {
  private memory: LRUCache<string, { value: unknown; expires: number }>;
  private redis: Redis;

  constructor(redis: Redis) {
    this.redis = redis;
    this.memory = new LRUCache({
      max: 1000,
      ttl: 60 * 1000, // 1 minute default
    });
  }

  async get<T>(key: string): Promise<T | null> {
    // Layer 1: Memory
    const memoryHit = this.memory.get(key);
    if (memoryHit && memoryHit.expires > Date.now()) {
      return memoryHit.value as T;
    }

    // Layer 2: Redis
    const redisValue = await this.redis.get(key);
    if (redisValue) {
      const parsed = JSON.parse(redisValue) as T;
      // Populate memory cache
      this.memory.set(key, { value: parsed, expires: Date.now() + 60000 });
      return parsed;
    }

    return null;
  }

  async set<T>(key: string, value: T, options: CacheOptions = {}): Promise<void> {
    const ttl = options.ttl || 3600; // 1 hour default

    // Set in Redis
    await this.redis.setex(key, ttl, JSON.stringify(value));

    // Set in memory (shorter TTL)
    this.memory.set(key, {
      value,
      expires: Date.now() + Math.min(ttl * 1000, 60000),
    });

    // Track tags for invalidation
    if (options.tags) {
      for (const tag of options.tags) {
        await this.redis.sadd(`cache:tag:${tag}`, key);
      }
    }
  }

  async getOrSet<T>(
    key: string,
    fetcher: () => Promise<T>,
    options: CacheOptions = {}
  ): Promise<T> {
    const cached = await this.get<T>(key);
    if (cached !== null) {
      return cached;
    }

    // Prevent cache stampede with lock
    const lockKey = `lock:${key}`;
    const acquired = await this.redis.set(lockKey, '1', 'EX', 10, 'NX');

    if (!acquired) {
      // Another process is fetching, wait and retry
      await new Promise(resolve => setTimeout(resolve, 100));
      return this.getOrSet(key, fetcher, options);
    }

    try {
      const value = await fetcher();
      await this.set(key, value, options);
      return value;
    } finally {
      await this.redis.del(lockKey);
    }
  }

  async invalidate(key: string): Promise<void> {
    this.memory.delete(key);
    await this.redis.del(key);
  }

  async invalidateByTag(tag: string): Promise<void> {
    const keys = await this.redis.smembers(`cache:tag:${tag}`);
    if (keys.length > 0) {
      await this.redis.del(...keys);
      for (const key of keys) {
        this.memory.delete(key);
      }
    }
    await this.redis.del(`cache:tag:${tag}`);
  }
}

export { MultiLayerCache, CacheOptions };

typescript

// cache.ts
import { Redis } from 'ioredis';
import { LRUCache } from 'lru-cache';

interface CacheOptions {
  ttl?: number;           // Time to live in seconds
  staleWhileRevalidate?: number;
  tags?: string[];        // For invalidation
}

class MultiLayerCache {
  private memory: LRUCache<string, { value: unknown; expires: number }>;
  private redis: Redis;

  constructor(redis: Redis) {
    this.redis = redis;
    this.memory = new LRUCache({
      max: 1000,
      ttl: 60 * 1000, // 1 minute default
    });
  }

  async get<T>(key: string): Promise<T | null> {
    // Layer 1: Memory
    const memoryHit = this.memory.get(key);
    if (memoryHit && memoryHit.expires > Date.now()) {
      return memoryHit.value as T;
    }

    // Layer 2: Redis
    const redisValue = await this.redis.get(key);
    if (redisValue) {
      const parsed = JSON.parse(redisValue) as T;
      // Populate memory cache
      this.memory.set(key, { value: parsed, expires: Date.now() + 60000 });
      return parsed;
    }

    return null;
  }

  async set<T>(key: string, value: T, options: CacheOptions = {}): Promise<void> {
    const ttl = options.ttl || 3600; // 1 hour default

    // Set in Redis
    await this.redis.setex(key, ttl, JSON.stringify(value));

    // Set in memory (shorter TTL)
    this.memory.set(key, {
      value,
      expires: Date.now() + Math.min(ttl * 1000, 60000),
    });

    // Track tags for invalidation
    if (options.tags) {
      for (const tag of options.tags) {
        await this.redis.sadd(`cache:tag:${tag}`, key);
      }
    }
  }

  async getOrSet<T>(
    key: string,
    fetcher: () => Promise<T>,
    options: CacheOptions = {}
  ): Promise<T> {
    const cached = await this.get<T>(key);
    if (cached !== null) {
      return cached;
    }

    // Prevent cache stampede with lock
    const lockKey = `lock:${key}`;
    const acquired = await this.redis.set(lockKey, '1', 'EX', 10, 'NX');

    if (!acquired) {
      // Another process is fetching, wait and retry
      await new Promise(resolve => setTimeout(resolve, 100));
      return this.getOrSet(key, fetcher, options);
    }

    try {
      const value = await fetcher();
      await this.set(key, value, options);
      return value;
    } finally {
      await this.redis.del(lockKey);
    }
  }

  async invalidate(key: string): Promise<void> {
    this.memory.delete(key);
    await this.redis.del(key);
  }

  async invalidateByTag(tag: string): Promise<void> {
    const keys = await this.redis.smembers(`cache:tag:${tag}`);
    if (keys.length > 0) {
      await this.redis.del(...keys);
      for (const key of keys) {
        this.memory.delete(key);
      }
    }
    await this.redis.del(`cache:tag:${tag}`);
  }
}

export { MultiLayerCache, CacheOptions };

Cache-Aside Pattern

旁路缓存模式

typescript

// user-service.ts
class UserService {
  constructor(private cache: MultiLayerCache) {}

  async getUser(id: string): Promise<User> {
    return this.cache.getOrSet(
      `user:${id}`,
      async () => {
        return db.users.findUnique({ where: { id } });
      },
      { ttl: 3600, tags: ['users', `user:${id}`] }
    );
  }

  async updateUser(id: string, data: Partial<User>): Promise<User> {
    const user = await db.users.update({
      where: { id },
      data,
    });

    // Invalidate cache
    await this.cache.invalidate(`user:${id}`);

    return user;
  }

  async deleteUser(id: string): Promise<void> {
    await db.users.delete({ where: { id } });
    await this.cache.invalidateByTag(`user:${id}`);
  }
}

typescript

// user-service.ts
class UserService {
  constructor(private cache: MultiLayerCache) {}

  async getUser(id: string): Promise<User> {
    return this.cache.getOrSet(
      `user:${id}`,
      async () => {
        return db.users.findUnique({ where: { id } });
      },
      { ttl: 3600, tags: ['users', `user:${id}`] }
    );
  }

  async updateUser(id: string, data: Partial<User>): Promise<User> {
    const user = await db.users.update({
      where: { id },
      data,
    });

    // 失效缓存
    await this.cache.invalidate(`user:${id}`);

    return user;
  }

  async deleteUser(id: string): Promise<void> {
    await db.users.delete({ where: { id } });
    await this.cache.invalidateByTag(`user:${id}`);
  }
}

HTTP Caching Middleware

HTTP缓存中间件

typescript

// http-cache-middleware.ts
import { Request, Response, NextFunction } from 'express';

interface HttpCacheOptions {
  maxAge?: number;
  sMaxAge?: number;
  staleWhileRevalidate?: number;
  private?: boolean;
  vary?: string[];
}

function httpCache(options: HttpCacheOptions = {}) {
  return (req: Request, res: Response, next: NextFunction) => {
    const directives: string[] = [];

    if (options.private) {
      directives.push('private');
    } else {
      directives.push('public');
    }

    if (options.maxAge !== undefined) {
      directives.push(`max-age=${options.maxAge}`);
    }

    if (options.sMaxAge !== undefined) {
      directives.push(`s-maxage=${options.sMaxAge}`);
    }

    if (options.staleWhileRevalidate !== undefined) {
      directives.push(`stale-while-revalidate=${options.staleWhileRevalidate}`);
    }

    res.setHeader('Cache-Control', directives.join(', '));

    if (options.vary) {
      res.setHeader('Vary', options.vary.join(', '));
    }

    next();
  };
}

// Usage
app.get('/api/products',
  httpCache({ maxAge: 60, sMaxAge: 300, staleWhileRevalidate: 86400 }),
  async (req, res) => {
    const products = await getProducts();
    res.json(products);
  }
);

typescript

// http-cache-middleware.ts
import { Request, Response, NextFunction } from 'express';

interface HttpCacheOptions {
  maxAge?: number;
  sMaxAge?: number;
  staleWhileRevalidate?: number;
  private?: boolean;
  vary?: string[];
}

function httpCache(options: HttpCacheOptions = {}) {
  return (req: Request, res: Response, next: NextFunction) => {
    const directives: string[] = [];

    if (options.private) {
      directives.push('private');
    } else {
      directives.push('public');
    }

    if (options.maxAge !== undefined) {
      directives.push(`max-age=${options.maxAge}`);
    }

    if (options.sMaxAge !== undefined) {
      directives.push(`s-maxage=${options.sMaxAge}`);
    }

    if (options.staleWhileRevalidate !== undefined) {
      directives.push(`stale-while-revalidate=${options.staleWhileRevalidate}`);
    }

    res.setHeader('Cache-Control', directives.join(', '));

    if (options.vary) {
      res.setHeader('Vary', options.vary.join(', '));
    }

    next();
  };
}

// 用法
app.get('/api/products',
  httpCache({ maxAge: 60, sMaxAge: 300, staleWhileRevalidate: 86400 }),
  async (req, res) => {
    const products = await getProducts();
    res.json(products);
  }
);

Python Implementation

Python 实现

python

undefined

python

undefined

cache.py

import json import time from typing import TypeVar, Callable, Optional from functools import lru_cache import redis

T = TypeVar('T')

class MultiLayerCache: def init(self, redis_client: redis.Redis): self.redis = redis_client self._memory: dict[str, tuple[any, float]] = {}

def get(self, key: str) -> Optional[any]:
    # Layer 1: Memory
    if key in self._memory:
        value, expires = self._memory[key]
        if expires > time.time():
            return value
        del self._memory[key]

    # Layer 2: Redis
    redis_value = self.redis.get(key)
    if redis_value:
        parsed = json.loads(redis_value)
        self._memory[key] = (parsed, time.time() + 60)
        return parsed

    return None

def set(self, key: str, value: any, ttl: int = 3600, tags: list[str] = None):
    self.redis.setex(key, ttl, json.dumps(value))
    self._memory[key] = (value, time.time() + min(ttl, 60))

    if tags:
        for tag in tags:
            self.redis.sadd(f"cache:tag:{tag}", key)

async def get_or_set(
    self,
    key: str,
    fetcher: Callable[[], T],
    ttl: int = 3600,
) -> T:
    cached = self.get(key)
    if cached is not None:
        return cached

    # Simple lock for stampede prevention
    lock_key = f"lock:{key}"
    if not self.redis.set(lock_key, "1", ex=10, nx=True):
        await asyncio.sleep(0.1)
        return await self.get_or_set(key, fetcher, ttl)

    try:
        value = await fetcher()
        self.set(key, value, ttl)
        return value
    finally:
        self.redis.delete(lock_key)

def invalidate_by_tag(self, tag: str):
    keys = self.redis.smembers(f"cache:tag:{tag}")
    if keys:
        self.redis.delete(*keys)
        for key in keys:
            self._memory.pop(key.decode(), None)
    self.redis.delete(f"cache:tag:{tag}")

undefined

import json import time from typing import TypeVar, Callable, Optional from functools import lru_cache import redis

T = TypeVar('T')

class MultiLayerCache: def init(self, redis_client: redis.Redis): self.redis = redis_client self._memory: dict[str, tuple[any, float]] = {}

def get(self, key: str) -> Optional[any]:
    # Layer 1: Memory
    if key in self._memory:
        value, expires = self._memory[key]
        if expires > time.time():
            return value
        del self._memory[key]

    # Layer 2: Redis
    redis_value = self.redis.get(key)
    if redis_value:
        parsed = json.loads(redis_value)
        self._memory[key] = (parsed, time.time() + 60)
        return parsed

    return None

def set(self, key: str, value: any, ttl: int = 3600, tags: list[str] = None):
    self.redis.setex(key, ttl, json.dumps(value))
    self._memory[key] = (value, time.time() + min(ttl, 60))

    if tags:
        for tag in tags:
            self.redis.sadd(f"cache:tag:{tag}", key)

async def get_or_set(
    self,
    key: str,
    fetcher: Callable[[], T],
    ttl: int = 3600,
) -> T:
    cached = self.get(key)
    if cached is not None:
        return cached

    # Simple lock for stampede prevention
    lock_key = f"lock:{key}"
    if not self.redis.set(lock_key, "1", ex=10, nx=True):
        await asyncio.sleep(0.1)
        return await self.get_or_set(key, fetcher, ttl)

    try:
        value = await fetcher()
        self.set(key, value, ttl)
        return value
    finally:
        self.redis.delete(lock_key)

def invalidate_by_tag(self, tag: str):
    keys = self.redis.smembers(f"cache:tag:{tag}")
    if keys:
        self.redis.delete(*keys)
        for key in keys:
            self._memory.pop(key.decode(), None)
    self.redis.delete(f"cache:tag:{tag}")

undefined

Decorator Pattern

装饰器模式

python

undefined

python

undefined

cache_decorator.py

from functools import wraps

def cached(key_template: str, ttl: int = 3600, tags: list[str] = None): def decorator(func): @wraps(func) async def wrapper(*args, **kwargs): # Build cache key from template key = key_template.format(*args, **kwargs)

        cached_value = cache.get(key)
        if cached_value is not None:
            return cached_value

        result = await func(*args, **kwargs)
        cache.set(key, result, ttl=ttl, tags=tags)
        return result
    return wrapper
return decorator

from functools import wraps

        cached_value = cache.get(key)
        if cached_value is not None:
            return cached_value

        result = await func(*args, **kwargs)
        cache.set(key, result, ttl=ttl, tags=tags)
        return result
    return wrapper
return decorator

Usage

用法

@cached("user:{user_id}", ttl=3600, tags=["users"]) async def get_user(user_id: str) -> User: return await db.users.find_unique(where={"id": user_id})

undefined

@cached("user:{user_id}", ttl=3600, tags=["users"]) async def get_user(user_id: str) -> User: return await db.users.find_unique(where={"id": user_id})

undefined

Cache Invalidation Strategies

缓存失效策略

1. Time-Based (TTL)

1. 基于时间（TTL）

typescript

await cache.set('key', value, { ttl: 3600 }); // Expires in 1 hour

typescript

await cache.set('key', value, { ttl: 3600 }); // 1小时后过期

2. Event-Based

2. 基于事件

typescript

// On data change
eventBus.on('user.updated', async (userId) => {
  await cache.invalidate(`user:${userId}`);
});

typescript

// 数据变更时触发
eventBus.on('user.updated', async (userId) => {
  await cache.invalidate(`user:${userId}`);
});

3. Tag-Based

3. 基于标签

typescript

// Set with tags
await cache.set(`product:${id}`, product, { tags: ['products', `category:${categoryId}`] });

// Invalidate all products in category
await cache.invalidateByTag(`category:${categoryId}`);

typescript

// 设置时关联标签
await cache.set(`product:${id}`, product, { tags: ['products', `category:${categoryId}`] });

// 失效分类下所有产品缓存
await cache.invalidateByTag(`category:${categoryId}`);

4. Write-Through

4. 写穿透模式

typescript

async function updateUser(id: string, data: Partial<User>) {
  const user = await db.users.update({ where: { id }, data });
  await cache.set(`user:${id}`, user); // Update cache immediately
  return user;
}

typescript

async function updateUser(id: string, data: Partial<User>) {
  const user = await db.users.update({ where: { id }, data });
  await cache.set(`user:${id}`, user); // 立即更新缓存
  return user;
}

Best Practices

最佳实践

Cache at the right layer - Don't cache everything in Redis
Use appropriate TTLs - Balance freshness vs performance
Prevent stampedes - Use locks or stale-while-revalidate
Monitor hit rates - Track cache effectiveness
Plan for invalidation - Use tags for related data

选择合适的缓存层级 - 不要将所有数据都存入Redis
设置合理的TTL - 在数据新鲜度和性能之间取得平衡
预防缓存击穿 - 使用锁机制或stale-while-revalidate策略
监控缓存命中率 - 跟踪缓存有效性
规划失效策略 - 为关联数据使用标签分组

Common Mistakes

常见错误

Caching user-specific data without proper keys
No cache invalidation strategy
TTLs too long (stale data) or too short (no benefit)
Caching errors or null values
Not handling cache failures gracefully

未使用唯一键缓存用户专属数据
未制定缓存失效策略
TTL设置过长（数据过期）或过短（无性能收益）
缓存错误结果或空值
未优雅处理缓存故障