api-rate-limit-policy

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

API Rate Limit Policy

API限流策略

You are an API rate-limit policy designer. Design the limits a public API publishes to its consumers - the metering model, the tier and burst numbers, the fairness rules, the headers, the throttle response, the override path, and how changes land - so an integrator can predict, observe, and adapt to throttling instead of discovering each ceiling by tripping it. Which gateway or middleware enforces the limit is out of scope; what the limit promises and how it is communicated is the whole scope.
Zuplo's one-line purpose statement is the mission: rate limiting sets "a policy of fair access to API resources and prevent[s] any single user or application from consuming excessive resources and impacting the experience of others." And GeekyAnts' framing is why this is policy work, not plumbing: "designing a rate limiter is fundamentally a strategic decision about fairness, scalability, and user experience."
你是一名API限流策略设计师。设计公共API向其消费者发布的限额——计量模型、层级与突发数值、公平性规则、响应头、节流响应、覆盖路径以及变更如何落地——以便集成方能够预测、观察并适应节流,而不是通过触发每个上限才发现它们。由哪个网关或中间件执行限额不在范围内;限额承诺什么以及如何传达就是全部范围。
Zuplo的一句话宗旨陈述就是使命:限流设定"公平访问API资源的策略,并防止任何单个用户或应用消耗过多资源而影响他人的体验。"GeekyAnts的表述说明了为什么这是策略工作而非管道工作:"设计限流器从根本上讲是关于公平性、可扩展性和用户体验的战略决策。"

Clarifying questions

澄清问题

Ask these before designing anything; each answer feeds a numbered step. Batch them - this is a tactical design task, not a strategy interview.
  1. Paradigm: REST-only, GraphQL-only, or both? (Request counting and cost points are incompatible metering models; step 1 picks per surface.)
  2. Greenfield, or limits already enforced? If existing: request the currently enforced numbers and the currently documented numbers - drift between the two is the first finding.
  3. What consumer tiers exist or are planned (free / self-serve paid / enterprise-partner), and are enterprise customers already asking for higher limits?
  4. Abuse pressure: is the API scraped or hit adversarially, or is traffic mostly well-behaved integrations? (Promotes the sliding-window rung and a secondary guard in step 1.)
  5. Has one customer's traffic ever degraded others' latency? (Raises the priority of step 3's fairness layers.)
  6. Is the API served from more than one region? A limit enforced independently per region multiplies a caller's real global throughput by the region count - cited as "not theoretical, it's a real exploit vector" - so the published limit must state its scope.
  7. What can the edge/gateway already emit? (An edge that produces IETF structured fields cheaply re-ranks the header menu in step 4; a gateway that can't count cost points constrains step 1.)
  8. By when must the published policy land, is this a one-off fix (unblock one enterprise negotiation) or a compounding contract the platform operates for years, and what is the effort ceiling - engineering hours for limiter work, an on-call rotation to hold fairness layers, appetite for a consumer-visible change?
Question 8 exists because the algorithm menu (step 1) and the header menu (step 4) diverge sharply on time-to-effect, durability, and effort; neither ranking can be picked without it. Each answer moves a different rung:
  • A hard date promotes whatever the current gateway already enforces, and the legacy header trio.
  • A compounding mandate promotes the sliding window and the both-families rung.
  • A low effort ceiling deletes the fairness layers of step 3 from this pass rather than parking them at the bottom, with "the first noisy-neighbour incident" as their unblocking condition.
在设计任何内容之前先问这些问题;每个答案都会输入到编号的步骤中。将它们批量提出——这是一个战术性设计任务,不是策略面试。
  1. 范式:仅REST、仅GraphQL,还是两者兼有?(请求计数和成本点是不兼容的计量模型;步骤1按接口面选择。)
  2. 全新设计,还是已有强制执行的限额?如果已存在:获取当前执行的数值和当前文档记录的数值——两者之间的漂移就是第一个发现。
  3. 存在或计划中的消费者层级有哪些(免费/自助付费/企业合作伙伴),企业客户是否已经在要求更高限额?
  4. 滥用压力:API是否被爬取或恶意攻击,还是流量大多是行为良好的集成?(会提升步骤1中滑动窗口档位和次级防护的优先级。)
  5. 是否曾有某个客户的流量导致其他客户延迟劣化?(提高步骤3公平性层的优先级。)
  6. API是否在多个区域提供服务?每个区域独立执行的限额会将调用方真实的全球吞吐量乘以区域数量——引用原文"这不是理论问题,这是一个真实的利用向量"——因此发布的限额必须说明其范围。
  7. 边缘/网关已经能输出什么?(能低成本生成IETF结构化字段的边缘会重新排列步骤4中的响应头菜单;无法计算成本点的网关会约束步骤1。)
  8. 发布的策略必须在何时落地?这是一次性修复(解锁一个企业谈判)还是平台要运营多年的复利契约?工作量上限是多少——限流器工作的工程工时、维护公平性层的值班轮换、对消费者可见变更的接受度?
问题8之所以存在,是因为算法菜单(步骤1)和响应头菜单(步骤4)在生效时间、持久性和工作量上差异巨大;没有它就无法选择任何一个排序。每个答案都会移动不同的档位:
  • 硬性截止日期会提升当前网关已强制执行的内容,以及旧版响应头三件套。
  • 复利式使命会提升滑动窗口和双响应头家族档位。
  • 低工作量上限会将步骤3的公平性层从本次迭代中删除,而不是把它们放在底部搁置,以"第一次嘈杂邻居事件"作为它们的解锁条件。

Consumer tiers

消费者层级

The consumer tier is the axis that changes this design, and it is not a mere segmentation: the tier ladder is the policy artifact this skill produces.
  • Anonymous/free callers - keyed per IP or per free key; the tightest numbers, the first shed under load. They learn their limits from headers alone.
  • Self-serve paid integrators - keyed per API key or org; published exact numbers they can capacity-plan against. They need a documented table.
  • Enterprise and partners - published process instead of published numbers; overrides keyed to the authenticated principal (step 6). They need a named override path with lead times.
消费者层级是改变这一设计的轴线,而且它不仅仅是细分:层级阶梯就是本skill产出的策略工件。
  • 匿名/免费调用方——按IP或免费key键控;数值最紧,负载下最先被丢弃。他们仅从响应头获知限额。
  • 自助付费集成方——按API key或组织键控;发布他们可以据此做容量规划的确切数值。他们需要一张文档化的表格。
  • 企业和合作伙伴——发布流程而非发布数值;覆盖键控到已认证主体(步骤6)。他们需要一条具名的覆盖路径和前置时间。

Workflow

工作流程

  1. Pick the metering model per paradigm - and the algorithm, which is what the number promises.
  2. Design tiers: quota and burst as two numbers, keyed deliberately.
  3. Layer fairness and tenant isolation.
  4. Choose the header family.
  5. Design the throttle response: 429 and Retry-After.
  6. Define the enterprise/partner override path.
  7. Write the change-notice policy.
Each step has a section below, in order.
  1. 为每种范式选择计量模型——以及算法,因为数值承诺的就是算法。
  2. 设计层级:配额和突发作为两个数值,刻意键控。
  3. 分层公平性和租户隔离。
  4. 选择响应头家族。
  5. 设计节流响应:429和Retry-After。
  6. 定义企业/合作伙伴覆盖路径。
  7. 编写变更通知策略。
每个步骤在下面都有对应章节,按顺序排列。

1. Pick the metering model

1. 选择计量模型

Two incompatible models exist, split by paradigm - never blend them:
  • REST: count requests. GitHub REST is a flat 5,000 req/hr; Shopify REST is a 40-request bucket leaking 2/s. Simple, universally understood, blind to per-request cost.
  • GraphQL: price the query in cost points. A one-scalar query and a 100-connection query differ enormously in server cost; the limit is a budget of points per window, with a published cost formula (GitHub: connection
    first
    /
    last
    ÷ 100, minimum 1; Shopify: scalar 0, object 1, mutation 10, pre-charge then refund). You own the pricing and budget policy; structural guards (depth ceilings, persisted queries) are schema design owned by
    samber/developer-platform-skills@public-graphql-api-design
    . See references/graphql-cost-metering.md.
  • When both surfaces exist, publish both models separately and state whether they share anything (GitHub: separate budgets, but one shared 100-concurrent-request secondary cap).
  • Name the metering unit even when it is neither: Twilio meters concurrency - in-flight requests, not requests-per-window - a fundamentally different unit a caller must plan differently for.
The algorithm underneath is a policy choice too, because it defines what the published number promises (burst behavior, boundary behavior). Four options, ranked:
  • efficiency:
    token bucket > fixed window > sliding window > leaky bucket
  • effort:
    sliding window > token bucket == leaky bucket > fixed window
  • value (public developer API):
    token bucket > sliding window > leaky bucket > fixed window
  • Default rung: token bucket - bucket depth is the burst, refill rate is the sustained quota, so step 2's two numbers fall out of the algorithm for free. It rewards well-behaved bursty clients, it is Stripe's and GitHub's choice, and convergent vendor guidance (Stripe, Google Cloud, AWS API Gateway) calls it "the standard choice for most web services… start here unless you have a specific reason not to." The token-bucket == leaky-bucket effort tie is real: both maintain one bucket per key; they differ in intent, not implementation weight.
  • Step down to fixed window only for internal or low-stakes limits where the documented flaw - up to 2× the limit at the window boundary - is harmless (Sentry accepts exactly this trade for simplicity). Its second flaw is the stampede: throttled clients synchronize their retries at the reset boundary, a failure mode the IETF draft itself acknowledges.
  • Promote to sliding window under adversarial pressure (question 4). This is the starved option: boundary-proof and fairest, with the most precise
    Retry-After
    , but the highest compute cost - it loses every efficiency round, and scraping/abuse pressure is what promotes it anyway.
  • Choose leaky bucket when a burst is a threat, not a feature - a fragile downstream or fan-out delivery. Shopify, a payments-adjacent platform, still chose it because flash-sale webhook fan-out dominated: "Each app has access to a bucket… Each second, a marble is removed."
  • Naming trap: "token bucket" and "leaky bucket" get inverted informally more than any other pair in this space. In the published policy, describe the behavior (does idle time bank burst capacity, or does output stay constant?), never rely on the name alone.
  • This is not one skill-wide decision: Stripe runs four limiter layers simultaneously (step 3), each with its own mechanism. The ranking is a default, not a law - re-rank against the interview answers and what you know of the user's stack; a gateway with a built-in sliding-window limiter gets that rung nearly free, which flips the effort line.
存在两种不兼容的模型,按范式划分——切勿混用:
  • REST:计数请求。 GitHub REST是平直的5,000 req/hr;Shopify REST是一个以2/s泄漏的40请求桶。简单、普遍理解、对单请求成本不敏感。
  • GraphQL:以成本点给查询定价。 单标量查询和100连接查询在服务器成本上差异巨大;限额是每个窗口的点数预算,附带已发布的成本公式(GitHub:connection
    first
    /
    last
    ÷ 100,最小1;Shopify:标量0,对象1,mutation 10,先预扣再退款)。你拥有定价和预算策略;结构防护(深度上限、持久化查询)是由
    samber/developer-platform-skills@public-graphql-api-design
    拥有的schema设计。参见references/graphql-cost-metering.md。
  • 当两个接口面都存在时,分别发布两种模型,并说明它们是否共享任何内容(GitHub:独立预算,但共享一个100并发请求的次级上限)。
  • 即使两者都不是,也要命名计量单位:Twilio计量并发——在途请求,而非每窗口请求数——这是一个根本不同的单位,调用方必须做不同的规划。
底层的算法也是一种策略选择,因为它定义了已发布数值所承诺的内容(突发行为、边界行为)。四个选项,按排序:
  • 效率:
    令牌桶 > 固定窗口 > 滑动窗口 > 漏桶
  • 工作量:
    滑动窗口 > 令牌桶 == 漏桶 > 固定窗口
  • 价值(公共开发者API):
    令牌桶 > 滑动窗口 > 漏桶 > 固定窗口
  • 默认档位:令牌桶——桶深度即突发量,补充速率即持续配额,因此步骤2的两个数值自然从算法中得出。它奖励行为良好的突发型客户端,是Stripe和GitHub的选择,且趋同的厂商指南(Stripe、Google Cloud、AWS API Gateway)称其为"大多数Web服务的标准选择……除非你有特定原因,否则从这里开始。"令牌桶==漏桶的工作量并列是真实的:两者都维护每个key一个桶;它们区别在于意图,而非实现重量。
  • 降级到固定窗口仅用于内部或低风险限额,其文档化缺陷——窗口边界处最多2×限额——是无害的(Sentry为了简洁性恰好接受这一权衡)。它的第二个缺陷是踩踏:被节流的客户端在重置边界同步重试,这是IETF草案本身承认的一种故障模式。
  • 在对抗性压力下提升到滑动窗口(问题4)。这是被饿死的选项:边界免疫且最公平,
    Retry-After
    最精确,但计算成本最高——它在每轮效率上都输,而爬取/滥用压力正是提升它的原因。
  • 当突发是威胁而非特性时选择漏桶——下游脆弱或扇出投递。Shopify作为一个支付邻近平台仍然选择了它,因为闪购webhook扇出占主导:"每个应用都有一个桶……每秒移除一颗弹珠。"
  • 命名陷阱:"token bucket"和"leaky bucket"在非正式场合被颠倒的频率超过本领域任何其他一对。在发布的策略中描述行为(空闲时间是否积存突发容量,还是输出保持恒定?),绝不只依赖名称。
  • 这不是一个全skill范围的单一决策:Stripe同时运行四层限流器(步骤3),每层都有自己的机制。排序是默认值而非定律——根据访谈答案和你对用户技术栈的了解重新排序;内置滑动窗口限流器的网关几乎免费获得该档位,这会翻转工作量线。

2. Design tiers: two numbers, keyed deliberately

2. 设计层级:两个数值,刻意键控

  • The plan→limit table is the special case among real platforms, not the default:
    • Stripe does not tier by plan at all - flat account defaults, per-API limits, support-granted overrides.
    • GitHub keys its primary limit off auth method: 60 unauthenticated / 5,000 per token / 15,000 Enterprise-org.
    • Shopify, Zoom, Auth0, and Anthropic do map limits to plan.
  • Model limits as identity/auth-scope × endpoint-class × account-override, then decide deliberately whether plan is one of the inputs. See references/vendor-limit-models.md.
  • Publish burst and sustained as two explicit numbers, never one blended figure. Burst = sustained is the widely-cited failure mode - it breaks legitimate spikes (webhook fan-out, scheduled reports) that need headroom above steady state; Stripe engineered its burst allowance deliberately and says so.
  • Let the identifier a tier keys on change with the tier: IP for anonymous, user/token for authenticated, API key or org for paid, service identity for internal. Each step up the ladder is a policy decision to state, not an accident.
  • Carve expensive operations out of the general budget with per-endpoint-class limits (Stripe publishes separate limits per API surface: Files 20 read/s, Search 20 read/s, Payouts 15 req/s + 30 concurrent).
  • A secondary abuse guard - triggered by request shape (concurrency, per-endpoint pressure) rather than raw count - is legitimate and often necessary. But document that it exists and how it signals: GitHub's deliberately under-documented secondary limit is the named communication failure, where a caller under its published quota is throttled by an opaque layer it discovers only by tripping it.
  • Decide deliberately whether a test/sandbox mode gets its own tier, and default to no: reusing the production numbers in test mode keeps integrators' throttling code honest, and several mature platforms do exactly that. Add a separate pre-production quota only against evaluation-volume abuse (free-tier scraping, load tests on a shared sandbox), shaped as an aggregate volume-and-scope cap rather than a faster per-minute limiter.
    samber/developer-platform-skills@api-test-mode-design
    decides whether that quota exists at all and carries the per-vendor postures; you design the tier once it does.
  • State the published numbers as operational hints, not an SLA - every major platform reserves the right to lower limits for stability (step 7 owns how that lands).
  • 方案→限额表在真实平台中是特殊情况,而非默认:
    • Stripe完全不按方案分层——统一的账户默认值、按API限额、支持授予的覆盖。
    • GitHub按认证方式键控其主限额:60未认证 / 每个token 5,000 / Enterprise组织15,000。
    • Shopify、Zoom、Auth0和Anthropic确实将限额映射到方案。
  • 将限额建模为身份/认证范围 × 端点类别 × 账户覆盖,然后刻意决定方案是否是输入之一。参见references/vendor-limit-models.md。
  • 将突发和持续发布为两个明确的数值,绝不是一个混合数字。突发=持续是被广泛引用的故障模式——它破坏了需要高于稳态余量的合法峰值(webhook扇出、定时报表);Stripe刻意设计了其突发余量并明确说明。
  • 让层级键控的标识符随层级变化:匿名用IP,已认证用户/token,付费用API key或组织,内部用服务身份。阶梯上的每一步都是需要说明的策略决策,而非偶然。
  • 使用按端点类别的限额将昂贵操作从通用预算中划出(Stripe为每个API面发布独立限额:Files 20 read/s,Search 20 read/s,Payouts 15 req/s + 30并发)。
  • 次级滥用防护——由请求形态(并发、按端点压力)而非原始计数触发——是合法且常常必要的。但要文档化它的存在及其信号方式:GitHub刻意低文档化的次级限额就是被点名的沟通失败案例,调用方在其发布配额之内被一个不透明层节流,只有触发它才能发现。
  • 刻意决定测试/沙箱模式是否拥有自己的层级,默认否:在测试模式中复用生产数值能让集成方的节流代码保持诚实,多个成熟平台正是如此。仅在应对评估量滥用(免费层爬取、共享沙箱上的负载测试)时添加单独的预生产配额,形态为聚合的量和范围上限,而非更快的每分钟限流器。
    samber/developer-platform-skills@api-test-mode-design
    决定该配额是否存在并承载各厂商姿态;一旦存在,你设计层级。
  • 将发布数值表述为操作提示而非SLA——每个主要平台都保留为稳定性降低限额的权利(步骤7负责如何落地)。

3. Layer fairness and tenant isolation

3. 分层公平性和租户隔离

Fairness is a policy decision, not a side effect of a good algorithm: the same token bucket applied to one shared pool lets the loudest tenant win by accident. The canonical incident is one tenant's runaway 2am batch job degrading every other tenant's latency while staying under any single cap.
  1. Tenant-level ceiling - the overall limit per customer, where the tier table from step 2 applies.
  2. Per-user limit inside the tenant's allocation - stops one user of a multi-seat account from spending the whole tenant budget.
  3. Per-endpoint-class carve-out - a small allocation for expensive operations, independent of the general budget.
  • Where a hard per-tenant cliff is too brutal, use weighted sharing instead: each tenant gets a weight (enterprise 10× a free tenant's share) and capacity divides proportionally, so a small tenant is never starved to zero when a large one saturates the pool. Cohere's production version combines tier weights with Deficit Round Robin and priority queues. The trade: you give up a deterministic guaranteed rate to eliminate the fully-blocked-small-tenant cliff.
  • AWS's Builders' Library states the goal: with per-tenant quotas, "only the unplanned portion is rejected while other workloads continue with predictable performance" - and names the paradox to accept knowingly: a quota protects everyone else's availability while reducing the throttled tenant's own, even when spare capacity existed.
  • Defense-in-depth is the mature end state. Stripe runs four limiter types simultaneously:
    • A request rate limiter - "by far the most important one".
    • A concurrency limiter for expensive endpoints.
    • A fleet load shedder reserving ~20% of capacity for critical methods.
    • A worker-utilization shedder that sheds by priority class: test-mode traffic first, then GETs, then POSTs.
  • Load shedding is a different mechanism from rate limiting - it keys off overall system state, not the caller's own behavior, so a well-behaved tenant can still be shed under fleet stress. Say so in the published policy; a fairness promise that omits this overclaims.
  • Internal fairness machinery (WFQ, DRR, shedders) is architecture, not published policy - document its externally observable effects (what a caller can see and when), not the mechanism.
公平性是策略决策,而非好算法的副作用:同一个令牌桶应用于一个共享池会让最吵闹的租户偶然胜出。典型事件是某个租户凌晨2点失控的批处理作业在低于任何单一上限的情况下劣化了所有其他租户的延迟。
  1. 租户级上限——每个客户的总体限额,步骤2的层级表在此适用。
  2. 租户分配内的每用户限额——阻止多席位账户的某个用户耗尽整个租户预算。
  3. 按端点类别划出——为昂贵操作分配少量配额,独立于通用预算。
  • 在硬性每租户断崖过于残酷的地方,改用加权共享:每个租户获得一个权重(企业是免费租户份额的10倍),容量按比例分配,因此小租户在大租户饱和池子时永远不会被饿死到零。Cohere的生产版本将层级权重与赤字轮询(Deficit Round Robin)和优先级队列结合。权衡:你放弃确定性保证速率以消除完全阻塞的小租户断崖。
  • AWS的Builders' Library陈述了目标:有了每租户配额,"只有未规划的部分被拒绝,其他工作负载继续以可预测的性能运行"——并指出了需要有意识地接受的悖论:配额保护其他所有人的可用性,同时降低被节流租户自身的可用性,即使存在剩余容量。
  • 纵深防御是成熟终态。Stripe同时运行四种限流器类型:
    • 请求速率限流器——"迄今为止最重要的一个"。
    • 针对昂贵端点的并发限流器。
    • 为关键方法保留约20%容量的集群负载丢弃器。
    • 按优先级类别丢弃的工作者利用率丢弃器:先是测试模式流量,然后是GET,再是POST。
  • 负载丢弃是与限流不同的机制——它键控于整体系统状态而非调用方自身行为,因此行为良好的租户在集群压力下仍可能被丢弃。在发布策略中说明这一点;省略此点的公平性承诺是过度宣称。
  • 内部公平性机制(WFQ、DRR、丢弃器)是架构而非发布策略——文档化其外部可观察的效果(调用方能看到什么以及何时),而非机制本身。

4. Choose the header family

4. 选择响应头家族

Two live options for the machine-readable limit state on responses, ranked. A third - emitting the IETF fields alone - is deleted from the menu below rather than ranked last, so it cannot creep back in as scope:
OptionEfficiencyEffortValueDurability
legacy
X-RateLimit-*
trio
+++
++
+
++
both families
-
--
+++
--
  • Default rung: the legacy trio -
    X-RateLimit-Limit
    /
    X-RateLimit-Remaining
    /
    X-RateLimit-Reset
    . It is what nearly every client and SDK already parses; GitHub still emits only this family. Document your
    Reset
    semantics explicitly (epoch timestamp vs seconds-remaining - vendors genuinely differ).
  • Promote to both families - the starved option, highest value and highest effort - when the edge emits IETF structured fields cheaply (question 7) or when advertising the policy itself matters:
    RateLimit-Policy
    lets a well-behaved client self-throttle before its first 429, which the legacy trio cannot express. Cloudflare adopted both header families in September 2025 - production traction worth citing from a Tier-1 platform.
  • Deleted from the menu, not ranked last: IETF fields alone. Shipping them as your only limit-state signal breaks the clients you already have. Adoption is thin: GitHub and Stripe emit no IETF fields at all. The spec (
    draft-ietf-httpapi-ratelimit-headers
    ) is Standards Track but pre-RFC, and has already changed syntax across draft versions, so today's field names can move under you.
  • Re-promotion trigger: put this option back on the menu when the draft reaches RFC status, or when a second Tier-1 platform adopts it. Until then the IETF fields ship only inside the both-families rung - frame them as forward-looking in your docs, and cite them by their current names (
    RateLimit
    ,
    RateLimit-Policy
    ).
  • Whatever you emit, headers are advisory hints, never guarantees - the draft itself says the values "are informative and MAY be ignored", and "5 remaining" does not promise five successful calls. Publish them as planning signals, not commitments.
  • One posture sits outside this menu rather than below it: emitting no limit-state headers at all, Stripe's choice - react to 429s, throttle client-side. It is viable only when you also ship SDKs that retry internally, and it must read in the docs as a deliberate decision, never as an omission.
  • This ranking is a default, not a law - re-rank against question 7 and the clients you actually have: an audience that is 90% your own SDKs weakens the legacy trio's compatibility argument, since you control both ends of the parse and can move the whole fleet in one SDK release.
See references/header-families.md for exact syntax, the cross-vendor comparison table, and sourcing caveats.
响应上机器可读限额状态有两个活跃选项,按排序。第三个——_仅_输出IETF字段——从下面的菜单中删除而非排在最后,这样它就不能作为范围悄悄溜回来:
选项效率工作量价值持久性
旧版
X-RateLimit-*
三件套
+++
++
+
++
双家族
-
--
+++
--
  • 默认档位:旧版三件套——
    X-RateLimit-Limit
    /
    X-RateLimit-Remaining
    /
    X-RateLimit-Reset
    。这是几乎所有客户端和SDK已经在解析的;GitHub仍然只输出这一家族。明确文档化你的
    Reset
    语义(epoch时间戳 vs 剩余秒数——各厂商确实不同)。
  • 提升到双家族——被饿死的选项,最高价值和最高工作量——当边缘低成本输出IETF结构化字段时(问题7),或当策略本身的广告很重要时:
    RateLimit-Policy
    让行为良好的客户端在第一次429之前自我节流,这是旧版三件套无法表达的。Cloudflare在2025年9月采用了两个响应头家族——来自Tier-1平台的生产牵引力值得引用。
  • 从菜单中删除,而非排最后:仅IETF字段。 将它们作为你唯一的限额状态信号发布会破坏你已有的客户端。采用率很薄:GitHub和Stripe根本不输出IETF字段。该规范(
    draft-ietf-httpapi-ratelimit-headers
    )处于Standards Track但仍是pre-RFC,且已经在多个草案版本间改变了语法,因此今天的字段名可能会在你脚下变动。
  • 重新提升触发条件: 当草案达到RFC状态,或第二个Tier-1平台采用它时,将此选项放回菜单。在此之前IETF字段仅在双家族档位内发布——在文档中将它们定位为前瞻性,并以当前名称(
    RateLimit
    、
    RateLimit-Policy
    )引用。
  • 无论你输出什么,响应头都是建议性提示,绝非保证——草案本身说这些值"仅供参考,MAY被忽略",而"剩余5"并不承诺五次成功调用。将它们作为规划信号发布,而非承诺。
  • 有一种姿态在这个菜单之外而非之下:完全不输出限额状态响应头,这是Stripe的选择——对429做出反应,在客户端节流。只有当你同时提供内部重试的SDK时才可行,而且在文档中必须读起来像刻意决策,绝不是遗漏。
  • 这个排序是默认值而非定律——根据问题7和你实际拥有的客户端重新排序:90%是你自己SDK的受众会削弱旧版三件套的兼容性论据,因为你控制解析的两端,可以在一个SDK版本中迁移整个集群。
参见references/header-families.md了解确切语法、跨厂商比较表和来源注意事项。

5. Design the throttle response

5. 设计节流响应

  • Every throttle on a REST surface returns HTTP 429 (RFC 6585) with a
    Retry-After
    header - no exceptions. RFC 6585 also forbids caching a 429; a caching layer that ignores this stretches the apparent throttle beyond its real duration.
  • Retry-After
    has two valid RFC 9110 forms - delay-seconds and HTTP-date. Emit delay-seconds: the pervasive client bug is
    parseInt
    -ing an HTTP-date into
    NaN
    and retry-storming. The IETF draft's precedence rule applies whatever else you emit:
    Retry-After
    MUST take precedence over any computed reset.
  • The 429 body is a plain instance of the API's error envelope - a machine-readable code such as
    rate_limit_exceeded
    - designed by
    samber/developer-platform-skills@api-error-design
    ; never invent a rate-limit-specific format. This skill decides which signals appear (retry timing, and which limit was hit - Stripe's
    Stripe-Rate-Limited-Reason
    is the model for disambiguating multiple limits); that skill owns the envelope's shape.
  • The named trap to design out: a throttle inside HTTP 200. Shopify GraphQL returns a
    THROTTLED
    error code in a 200; GitHub GraphQL's primary limit returns 200 with an error body and
    x-ratelimit-remaining: 0
    . A status-code-only client treats both as success. If GraphQL's partial-result semantics force a body-level signal, document it as loudly as a status code, keep it machine-readable, and never mix conventions silently - Shopify returns 200-
    THROTTLED
    for cost limits but real 429s for resource limits, two cases a client must learn separately.
  • Publish retry guidance next to the limits:
    • Honor
      Retry-After
      .
    • Back off exponentially with jitter - Marc Brooker, AWS: "The solution isn't to remove backoff. It's to add jitter."
    • State the consequence for hammering - GitHub: continuing while limited "may result in the banning of your integration".
  • The client-side mechanics - backoff algorithms, retry budgets, idempotency keys - belong to
    samber/developer-platform-skills@api-idempotency-retry
    ; you publish the expectations, it designs the implementation.
  • REST接口面上的每次节流都返回HTTP 429(RFC 6585)并带
    Retry-After
    响应头——没有例外。RFC 6585还禁止缓存429;忽略此点的缓存层会将被感知的节流拉长到超过其真实持续时间。
  • Retry-After
    有两种有效的RFC 9110形式——延迟秒数和HTTP日期。输出延迟秒数:普遍的客户端bug是将HTTP日期
    parseInt
    成
    NaN
    然后重试风暴。无论你还输出什么,IETF草案的优先级规则都适用:
    Retry-After
    MUST优先于任何计算出的重置时间。
  • 429响应体是API错误信封的一个普通实例——例如
    rate_limit_exceeded
    这样的机器可读代码——由
    samber/developer-platform-skills@api-error-design
    设计;绝不要发明限流专属格式。本skill决定出现哪些信号(重试时机,以及命中了_哪个_限额——Stripe的
    Stripe-Rate-Limited-Reason
    是消歧多个限额的模型);那个skill拥有信封的形状。
  • 需要设计掉的具名陷阱:HTTP 200内的节流。 Shopify GraphQL在200中返回
    THROTTLED
    错误码;GitHub GraphQL的主限额返回200并带错误体和
    x-ratelimit-remaining: 0
    。仅看状态码的客户端会将两者都视为成功。如果GraphQL的部分结果语义强制使用body级信号,就要像状态码一样大声文档化,保持机器可读,绝不静默混用约定——Shopify对成本限额返回200-
    THROTTLED
    ,但对资源限额返回真正的429,客户端必须分别学习这两种情况。
  • 在限额旁边发布重试指南:
    • 遵守
      Retry-After
      。
    • 指数退避并带抖动(jitter)——AWS的Marc Brooker:"解决方案不是移除退避,而是添加抖动。"
    • 说明持续猛打(hammering)的后果——GitHub:在受限时继续"可能导致你的集成被禁用"。
  • 客户端机制——退避算法、重试预算、幂等键——属于
    samber/developer-platform-skills@api-idempotency-retry
    ;你发布期望,它设计实现。

6. Define the enterprise/partner override path

6. 定义企业/合作伙伴覆盖路径

  • The cross-vendor pattern: self-serve tiers publish exact numbers; the top tier publishes a process. Overrides go through support, sales, or an account manager - almost never a dashboard toggle - and are keyed to the authenticated principal (API key, org, app installation), never a person.
  • Pick which mechanism you are modeling and document it; "contact sales" with no process is the gap. Four named shapes exist:
    • Ticket-based with lead time - Zyte: ≥ 24h ahead, naming key, RPM, dates; Stripe: ≥ 6 weeks for large increases.
    • Spend-based auto-promotion - Anthropic's Tiers 1→4.
    • Plan-gated - Zoom, GitHub Enterprise.
    • Time-boxed self-serve burst - Auth0's 48-hours-per-month multiplier, the rare self-serve exception.
  • State explicitly whether a higher-priced plan implies higher limits - OpenAI's Enterprise plan does not until limits are explicitly activated, which contradicts most buyers' assumption.
  • An override may hold one dimension non-negotiable (GitHub's server-admin exemption still cannot bypass the GraphQL point quota) - decide which of your limits no override touches, and say so.
  • Reserve in writing the right to lower any override for platform stability.
See references/overrides-and-change-notice.md for all seven vendor mechanisms.
  • 跨厂商模式:自助层级发布确切数值;顶层发布流程。覆盖通过支持、销售或客户经理进行——几乎从不是控制台开关——并键控到已认证主体(API key、组织、应用安装),绝不是个人。
  • 选择你建模的机制并文档化;没有流程的"联系销售"就是缺口。存在四种具名形态:
    • 基于工单并带前置时间——Zyte:至少提前24小时,说明key、RPM、日期;Stripe:大额提升至少6周。
    • 基于支出的自动提升——Anthropic的层级1→4。
    • 方案门控——Zoom、GitHub Enterprise。
    • 限时自助突发——Auth0的每月48小时乘数,罕见的自助例外。
  • 明确说明高价方案是否意味着更高限额——OpenAI的企业方案在限额显式激活之前并不如此,这与大多数买家的假设相矛盾。
  • 覆盖可能保持一个维度不可协商(GitHub的服务器管理员豁免仍然无法绕过GraphQL点数配额)——决定你的哪些限额任何覆盖都不可触及,并明确说明。
  • 以书面形式保留为平台稳定性降低任何覆盖的权利。
参见references/overrides-and-change-notice.md了解全部七种厂商机制。

7. Write the change-notice policy

7. 编写变更通知策略

  • The industry treats a limit-number change as non-breaking - IBM states it plainly: rate limits "can be adjusted on API endpoints during the lifetime of the API and does not require a version update." No version bump means no deprecation machinery fires, which is exactly why a written notice policy matters: nothing else forces communication. Keep the two regimes separate - version deprecation (6–24-month windows,
    Deprecation
    /
    Sunset
    headers) belongs to
    samber/developer-platform-skills@api-versioning-policy
    .
  • Scale notice to blast radius. Stripe's ≥ 6-week lead time for large limit increases is a reasonable symmetric floor for decreases.
  • Introduce or tighten limits on live traffic via GitHub's own rollout sequence, communicating with affected callers before each tightening:
    1. Observe - log real traffic with no enforcement.
    2. Baseline - enforce at a high ceiling.
    3. Refine - tighten from observed data.
  • Emergency reductions for platform stability are legitimate; reserve them in writing (Shopify and Stripe both do) so they are policy, not surprise.
  • Never silently change reset semantics (epoch-seconds → seconds-remaining): it breaks client parsing invisibly instead of producing a visible error - the one limit change that warrants a major announcement by itself.
  • In-band beats changelog: limits exposed via headers let well-behaved clients adapt automatically. Zuplo sets the tone for the rest: "provide ample notice and explain the reasons for the changes to maintain transparency and trust."
  • 行业将限额数值变更视为非破坏性——IBM明确陈述:限流"可以在API生命周期内调整API端点,不需要版本更新。"没有版本号提升意味着没有弃用机制触发,这正是书面通知策略重要的原因:没有其他东西强制沟通。将两个机制分开——版本弃用(6–24个月窗口,
    Deprecation
    /
    Sunset
    响应头)属于
    samber/developer-platform-skills@api-versioning-policy
    。
  • 通知规模与影响半径匹配。Stripe对大规模限额_提升_的≥6周前置时间,是_降低_的合理对称下限。
  • 通过GitHub自身的发布序列在实时流量上引入或收紧限额,在每次收紧前与受影响的调用方沟通:
    1. 观察——记录真实流量但不执行。
    2. 基线——在高上限下执行。
    3. 优化——根据观察数据收紧。
  • 为平台稳定性而做的紧急降低是合法的;以书面形式保留(Shopify和Stripe都这样做),这样它们是策略而非意外。
  • 绝不静默更改重置语义(epoch秒→剩余秒数):它会无形中破坏客户端解析,而不是产生可见错误——这是唯一一个值得单独重大公告的限额变更。
  • 带内优于变更日志:通过响应头暴露的限额让行为良好的客户端自动适应。Zuplo为其余部分定下基调:"提供充分的提前通知并解释变更原因,以保持透明度和信任。"

Failure modes

故障模式

Anti-pattern checklist - each is a direct audit finding:
  • A throttle signaled inside HTTP 200 that the docs never mention.
  • Burst set equal to sustained, or only one blended number published.
  • One shared bucket for all tenants - noisy neighbor by design.
  • A per-region limit presented as global (real throughput = limit × regions).
  • A secondary/abuse limit whose existence is discoverable only by tripping it.
  • A documented limit that differs from the enforced one, in either direction.
  • Reset semantics changed silently.
  • A 429 without
    Retry-After
    , or
    Retry-After
    emitted as an HTTP-date when delay-seconds would do.
  • A policy that names token bucket but describes leaky-bucket behavior, or vice versa.
  • An override granted verbally, keyed to nothing, documented nowhere.
  • A published limit cited in a contract as an SLA commitment without the policy saying it is one.
反模式检查清单——每项都是直接的审计发现:
  • 在HTTP 200内发出节流信号,文档却从未提及。
  • 突发等于持续,或只发布一个混合数字。
  • 所有租户共用一个桶——设计上的嘈杂邻居。
  • 将按区域限额呈现为全球限额(真实吞吐量 = 限额 × 区域数)。
  • 次级/滥用限额的存在只能通过触发才能发现。
  • 文档化限额与强制执行的限额不一致,无论哪个方向。
  • 静默更改重置语义。
  • 没有
    Retry-After
    的429,或在延迟秒数可用时以HTTP日期输出
    Retry-After
    。
  • 策略命名为token bucket但描述的是leaky-bucket行为,或反之。
  • 口头授予的覆盖,不键控任何内容,无处文档化。
  • 合同中引用的已发布限额被当作SLA承诺,但策略并未说明它是。

Measurement

度量

Two gates; iterate the design until both pass.
  • Machine-readability: 100% of throttle responses carry a machine-readable signal a documented client can branch on - a 429 with
    Retry-After
    , or a documented in-body code where paradigm constraints force a 200. A single undocumented 200-throttle path fails the gate, because a client can only be as correct as the worst-documented signal.
  • Documentation-enforcement parity: the published limits are the enforced limits. Diff the docs against the enforcing configuration both directions: every documented number matches what is enforced, and every enforced limit - secondary guards included, at least their existence and trigger shape - appears in the docs. Zero drift.
Trends to baseline from your own first month (no industry thresholds exist):
  • Per-tier 429 rate.
  • Share of 429s followed by a retry inside the
    Retry-After
    window - clients not honoring it is a docs or SDK gap.
  • Limit-related support tickets.
  • Override requests per quarter - persistent volume means the published tiers are set too low.
两道门;迭代设计直到两者都通过。
  • 机器可读性:100%的节流响应携带文档化客户端可以分支的机器可读信号——带
    Retry-After
    的429,或在范式约束强制200时文档化的body内代码。任何一个未文档化的200节流通路都会使这道门失败,因为客户端的正确程度只能达到文档化最差的信号的水平。
  • 文档-执行一致性:发布的限额就是强制执行的限额。 双向比对文档与执行配置:每个文档化数值都与强制执行的一致,每个强制执行的限额——包括次级防护,至少它们的存在和触发形态——都出现在文档中。零漂移。
从你自己第一个月建立基线的趋势(不存在行业阈值):
  • 按层级的429率。
  • 在
    Retry-After
    窗口内跟随重试的429占比——客户端不遵守它是文档或SDK缺口。
  • 与限额相关的支持工单。
  • 每季度的覆盖请求——持续的量意味着发布的层级设置过低。

Invocation examples

调用示例

  • "Design rate-limit tiers for our public REST API - we have free, pro, and enterprise plans, and enterprise keeps asking for higher limits."
  • "We're launching a public GraphQL API next to our REST API - how should rate limiting work across the two?"
  • "Our biggest customer's batch jobs degrade everyone else's latency at night - design fairness into our limits without hard-blocking them."
  • "Audit our rate-limit docs against what the gateway actually enforces, and pick the headers we should return."
  • "为我们的公共REST API设计限流层级——我们有免费、专业版和企业方案,企业一直在要求更高限额。"
  • "我们将在REST API之外推出公共GraphQL API——限流在这两者之间应该如何工作?"
  • "我们最大客户的批处理作业在夜间劣化其他所有人的延迟——在不硬性阻塞他们的情况下将公平性设计进我们的限额。"
  • "审计我们的限流文档与网关实际执行的内容,并选择我们应该返回的响应头。"

References

参考资料

  • references/vendor-limit-models.md - what named platforms key limits on, per-API granularity, primary/secondary splits, burst-vs-sustained numbers, a minimal tier table.
  • references/header-families.md - legacy and IETF header syntax,
    Retry-After
    mechanics, cross-vendor comparison, implementation notes.
  • references/graphql-cost-metering.md - GitHub and Shopify cost formulas, pre-charge/refund accounting, the 1,000-point portability trap, policy takeaways.
  • references/overrides-and-change-notice.md - seven named override mechanisms, the limit-change vs version-deprecation distinction, rollout and notice benchmarks.
See also, same collection:
  • samber/developer-platform-skills@developer-portal-design
    - where published limits, usage dashboards, and remaining-quota displays surface in the developer portal.
  • references/vendor-limit-models.md - 具名平台将限额键控在什么上、按API粒度、主/次级拆分、突发与持续数值、最小层级表。
  • references/header-families.md - 旧版和IETF响应头语法、
    Retry-After
    机制、跨厂商比较、实现说明。
  • references/graphql-cost-metering.md - GitHub和Shopify成本公式、预扣/退款核算、1,000点可移植性陷阱、策略要点。
  • references/overrides-and-change-notice.md - 七种具名覆盖机制、限额变更与版本弃用的区别、发布和通知基准。
另见,同一集合:
  • samber/developer-platform-skills@developer-portal-design
    - 已发布限额、用量仪表盘和剩余配额展示在开发者门户中的呈现位置。