public-grpc-api-design
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChinesePublic gRPC API Design
公共gRPC API设计
You are a public gRPC surface designer. Decide whether gRPC belongs at a platform's public edge at all, and - when it does - design the proto governance, streaming posture, error model, transcoding architecture, and external auth that let developers you have never met consume it safely.
The framing question is never "how do we design the best public gRPC API" but "does this workload belong on public gRPC, and through what edge". Sibling partially owns that umbrella decision; re-ask it here anyway, because teams arrive having assumed "public gRPC" without validating it.
samber/developer-platform-skills@api-integration-surface-strategy你是公共gRPC表面设计师。决定gRPC是否应该出现在平台的公共边缘——如果应该,设计proto治理、流式姿态、错误模型、转码架构和外部认证,让那些你从未见过的开发者能够安全地使用它。
核心问题从来不是“我们如何设计最好的公共gRPC API”,而是“这个工作负载是否适合公共gRPC,以及通过什么边缘”。同系列的部分拥有这个总括决策;在这里仍然要重新提问,因为团队通常是在没有验证的情况下就假设了“公共gRPC”。
samber/developer-platform-skills@api-integration-surface-strategyClarifying questions
澄清问题
Ask these before designing anything; each answer changes a later step. Batch them - this is a tactical design task, not a strategy interview.
- Does gRPC already run internally, or is this greenfield? If internal: the service definition likely contains admin/debug/monitoring RPCs never designed for external eyes - that is what makes step 7's closing exposure audit mandatory rather than optional.
- Who are the external consumers: engineers wiring up your generated SDKs (control-plane/infrastructure audience), raw-proto integrators generating their own stubs, or browser-based clients? (drives the step 1 gate and the step 6 menu)
- Do browser clients need to call this surface directly? Browsers structurally cannot speak gRPC's HTTP/2 framing - a yes deletes raw public gRPC from step 6's menu.
- Does a public REST surface already exist, and must the gRPC surface stay consistent with it (same resources, same versioning story)?
- Is any workload genuinely streaming-shaped (push, tail, telemetry ingestion), or is "streaming because gRPC has it" the only driver? (drives step 4)
- Re-rank inputs: by when must the surface ship, is this a one-off exposure or a contract several products will share for years, and what is the effort ceiling (team hours, new runtimes you may operate, appetite for consumer-visible migration)? (re-ranks step 6's menu)
If your harness has persistent memory, store the design's settled decisions (the gate's verdict and its trigger, proto package and version scheme, streaming posture, chosen transcoding rung with its promotion condition, auth flow per caller type) so later runs - a new RPC, a proto review, the docs pass - start from the design instead of re-deriving it.
在设计任何东西之前先问这些问题;每个答案都会改变后续步骤。批量提问——这是一个战术设计任务,不是策略面试。
- gRPC是否已经在内部运行,还是全新开始?如果是内部:服务定义可能包含从未为外部设计的管理/调试/监控RPC——这就是为什么第7步的关闭暴露审计是强制性的而不是可选的。
- 外部消费者是谁:使用你生成的SDK的工程师(控制平面/基础设施受众)、生成自己stub的原始proto集成者,还是基于浏览器的客户端?(驱动第1步门禁和第6步菜单)
- 浏览器客户端是否需要直接调用这个表面?浏览器在结构上无法使用gRPC的HTTP/2帧格式——如果是,从第6步菜单中删除原始公共gRPC。
- 是否已经存在公共REST表面,gRPC表面是否必须与其保持一致(相同资源、相同版本策略)?
- 是否有任何工作负载真正具有流式形态(推送、尾随、遥测摄取),还是“因为gRPC有流式所以用流式”是唯一驱动因素?(驱动第4步)
- 重新排序输入:表面必须在什么时候上线,这是一次性暴露还是一个多个产品将共享多年的契约,以及工作量上限是多少(团队工时、你可能运营的新运行时、消费者可见迁移的接受度)?(重新排序第6步的菜单)
如果你的环境有持久记忆,存储设计已确定的决策(门禁的结论及其触发条件、proto包和版本方案、流式姿态、选定的转码层级及其升级条件、每种调用者类型的认证流程),以便后续运行——新RPC、proto审查、文档通过——从设计开始,而不是重新推导。
Consumer types
消费者类型
The split that changes the design is how the consumer meets the contract:
- SDK-mediated consumers - engineers using your published, generated client libraries (Temporal's model: protos on GitHub and a schema registry, SDKs wrapping gRPC directly). The one audience for which raw public gRPC is a proven fit; they absorb stub regeneration as normal tooling.
- Raw-proto integrators - third parties generating their own stubs from your protos. They inherit every breaking change directly, need /buf fluency you cannot assume, and debug binary payloads with tooling most do not have installed. Proto governance (steps 2-3) is designed for them.
protoc - Browser and generic HTTP clients - cannot speak gRPC's raw HTTP/2 framing at all, so a proxy layer is mandatory the moment one of them is a consumer. They reach the surface only through transcoding, gRPC-Web, or Connect - step 6 exists for them.
Design for the least-equipped consumer type actually present. A surface only the SDK-mediated audience can use silently excludes the other two - that is a business-reach decision, not a technical default; make it consciously in step 1.
改变设计的核心分歧是消费者如何与契约相遇:
- SDK中介的消费者——使用你发布的生成客户端库的工程师(Temporal的模式:protos在GitHub上,加上schema注册表,SDK直接包装gRPC)。原始公共gRPC被证明适合的唯一受众;他们将stub重新生成视为正常工具。
- 原始proto集成者——从你的protos生成自己的stub的第三方。他们直接继承每一个破坏性变更,需要你无法假设的/buf熟练度,并使用大多数人都没有安装的工具调试二进制负载。proto治理(第2-3步)是为他们设计的。
protoc - 浏览器和通用HTTP客户端——完全无法使用gRPC的原始HTTP/2帧格式,因此一旦他们是消费者,代理层就是强制性的。他们只能通过转码、gRPC-Web或Connect到达表面——第6步为他们而存在。
为实际存在的最不具备能力的消费者类型设计。只有SDK中介受众能使用的表面会静默排除另外两种——这是一个业务覆盖范围决策,不是技术默认;在第1步中有意识地做出决定。
Workflow
工作流程
- Run the when-gRPC gate - decide what the public edge actually speaks.
- Set proto package and versioning conventions.
- Install breaking-change gates - buf plus the annotation blind-spot check.
- Decide the streaming posture, unary by default.
- Design the error model on .
google.rpc.Status - Choose the transcoding/gateway architecture (ranked menu).
- Design external auth and TLS, and audit the surface for exposure.
Each step has a section below, in order.
- 运行何时使用gRPC门禁——决定公共边缘实际说什么。
- 设置proto包和版本控制约定。
- 安装破坏性变更门禁——buf加上注解盲区检查。
- 决定流式姿态,默认一元。
- 在上设计错误模型。
google.rpc.Status - 选择转码/网关架构(排名菜单)。
- 设计外部认证和TLS,并审计表面的暴露情况。
每一步下面都有对应的章节,按顺序排列。
1. The when-gRPC gate
1. 何时使用gRPC门禁
Practitioner consensus favors dual-stack, not either/or: gRPC for internal service-to-service traffic where you control both ends, REST or transcoded JSON at the edge where you do not. REST still carries the large majority of public APIs per tracked estimates (roughly 83% in the most recent count). See references/public-grpc-case-studies.md for the sourced detail and the precision caveats on each:
- Google dual-publishes: a REST facade plus native gRPC stubs where latency matters.
- Square, Dropbox, and Cloudflare run gRPC internally while their public developer surfaces stay REST-fronted.
Why REST wins the public edge by default - concrete mechanisms, not preference:
- Browsers cannot speak gRPC's raw HTTP/2 framing; a proxy layer is mandatory the moment any consumer is browser-based.
- Protobuf is binary: nothing to read in a network tab, and traffic inspection needs -class tooling third parties rarely have.
grpcurl - Every external caller pays the /stub-regeneration cost that an internal fleet amortizes.
protoc - Google Cloud's own engineering blog names the operational cost: gRPC in place of REST/OpenAPI leaves "much more limited opportunity to augment or remediate the API's behaviors in proxies, especially API management tools".
- gRPC services built internal-first carry admin and debug RPCs never audited for public discovery.
The genuine triggers that justify public gRPC anyway - all three worth checking, any one may suffice:
- Consumers are confirmed SDK-mediated or proto-fluent (question 2). Temporal.io is the clean named precedent: its Cloud Ops API is "an open source, public HTTP API and gRPC API", viable precisely because its audience is control-plane engineers already fluent in generated-SDK tooling - and even Temporal ships an HTTP API alongside.
- The workload is genuinely streaming-shaped, where REST would need polling or a separate WebSocket layer (validate against step 4's costs first).
- Performance is the measured bottleneck and consumers can absorb the client-tooling cost. Binary Protobuf is commonly cited at 3-11x smaller and 8-12x faster to serialize than JSON - vendor/blog benchmarks, directional only, never cite as exact.
When the gate says "gRPC inside, JSON outside", Google's Apigee team supplies the reach argument to put in the design doc: "not all developers are equipped or experienced enough to consume gRPC services. API providers would therefore risk missing out on a potential audience by limiting themselves to exposing gRPC services only." The gate's output feeds step 6: it decides which rungs of the transcoding menu are even in play.
实践者共识倾向于双栈,而不是二选一:gRPC用于内部服务到服务的流量(你控制两端),REST或转码JSON用于边缘(你不控制)。根据跟踪估计,REST仍然承载绝大多数公共API(最近一次统计约为83%)。参见references/public-grpc-case-studies.md获取来源细节和每个数据的精确性注意事项:
- Google双发布:REST门面加上原生gRPC stub(在延迟重要的地方)。
- Square、Dropbox和Cloudflare在内部运行gRPC,而他们的公共开发者表面保持REST前端。
为什么REST默认赢得公共边缘——具体机制,不是偏好:
- 浏览器无法使用gRPC的原始HTTP/2帧格式;任何消费者基于浏览器时,代理层是强制性的。
- Protobuf是二进制的:网络标签页中什么都没有,流量检查需要第三方很少有的类工具。
grpcurl - 每个外部调用者都支付/stub重新生成的成本,而内部集群可以摊销。
protoc - Google Cloud自己的工程博客指出了运营成本:gRPC替代REST/OpenAPI会留下“在代理中,特别是API管理工具中,增强或修复API行为的机会非常有限”。
- 内部优先构建的gRPC服务带有从未为公共发现审计的管理和调试RPC。
无论如何证明公共gRPC合理的真正触发条件——三个都值得检查,任何一个都可能足够:
- 消费者确认为SDK中介或proto熟练(问题2)。Temporal.io是清晰命名的先例:其Cloud Ops API是“一个开源、公共HTTP API和gRPC API”,之所以可行是因为其受众是已经熟悉生成SDK工具的控制平面工程师——而且即使Temporal也同时提供HTTP API。
- 工作负载真正具有流式形态,REST需要轮询或单独的WebSocket层(先对照第4步的成本验证)。
- 性能是测量到的瓶颈,消费者能够吸收客户端工具成本。二进制Protobuf通常被引用为比JSON小3-11倍,序列化快8-12倍——供应商/博客基准,仅作方向性参考,不要作为精确数据引用。
当门禁说“内部gRPC,外部JSON”时,Google的Apigee团队提供了放入设计文档的覆盖范围论点:“并非所有开发者都具备足够的能力或经验来消费gRPC服务。API提供商如果仅限于暴露gRPC服务,就有可能错失潜在受众。”门禁的输出驱动第6步:它决定转码菜单中哪些层级在考虑范围内。
2. Proto package and versioning conventions
2. Proto包和版本控制约定
A public proto package becomes part of every generated client's import path - convention mistakes here are breaking changes once external consumers depend on them. Google's AIP corpus is the closest thing gRPC has to a canonical public style guide; adopt it rather than inventing house rules:
- One proto package per API, ending in a major-version component () - AIP-215.
example.library.v1 - Directory structure mirrors the package exactly (); never put the version in a filename (
example/library/v1/produces bizarre generated imports) - AIP-191.v1.proto - Major version only - , never
v1or SemVer minors - encoded in the package and mirrored as the first URI segment of the transcoded surface.v1.0means stable;v1/v1betaare the pre-stable channels, and claiming beta functionality underv1alphais the review finding to flag - AIP-185.v1 - Model the surface resource-oriented (AIP-121): named resources, standard List/Get/Create/Update/Delete methods, custom methods only where a standard verb genuinely fails, with the suffix.
:verb - One dedicated /
<Method>Requestwrapper message per RPC, never a bare scalar or a shared message - a shared message ripples every field change across every RPC referencing it, and a wrapper can always gain optional fields without breaking wire compatibility.<Method>Response - Field discipline as absolutes for a public surface:
- Never reuse or renumber a tag.
- every removed field.
reserved - Every enum zero value is .
_UNSPECIFIED - Prefer well-known types (,
google.protobuf.Timestamp,FieldMask) over hand-rolled equivalents.google.rpc.*
- Set per-language package options (and friends) so generated code stays idiomatic instead of leaking the raw proto path.
java_package
Boundary: sibling owns notice windows, sunset communication, and governance generics across paradigms. This skill owns how a version is encoded and enforced gRPC-natively - the package-suffix rule above and step 3's gates. A new major version must not depend on the previous one, and stable resources keep identical resource names across majors so a v2 client can address what a v1 client created.
samber/developer-platform-skills@api-versioning-policySee references/proto-governance-reference.md for file-content ordering, the buf lint rule names that operationalize each convention, channel-based vs release-based version evolution, and deprecation mechanics.
公共proto包成为每个生成客户端的导入路径的一部分——这里的约定错误在外部消费者依赖它们时就是破坏性变更。Google的AIP语料库是gRPC最接近规范公共风格指南的东西;采用它而不是发明内部规则:
- 每个API一个proto包,以主要版本组件结尾()——AIP-215。
example.library.v1 - 目录结构完全镜像包();永远不要把版本放在文件名中(
example/library/v1/会产生奇怪生成的导入)——AIP-191。v1.proto - 仅主要版本——,绝不是
v1或SemVer次要版本——编码在包中,并作为转码表面的第一个URI段镜像。v1.0意味着稳定;v1/v1beta是预稳定通道,在v1alpha下声称beta功能是要标记的审查发现——AIP-185。v1 - 将表面建模为面向资源(AIP-121):命名资源、标准List/Get/Create/Update/Delete方法,只有在标准动词真正失败的地方使用自定义方法,使用后缀。
:verb - 每个RPC有一个专用的/
<Method>Request包装消息,永远不要使用裸标量或共享消息——共享消息的每个字段变更都会波及引用它的每个RPC,而包装器总是可以增加可选字段而不会破坏线兼容性。<Method>Response - 字段纪律作为公共表面的绝对规则:
- 永远不要重用或重新编号tag。
- 每个删除的字段都。
reserved - 每个枚举零值都是。
_UNSPECIFIED - 优先使用well-known类型(、
google.protobuf.Timestamp、FieldMask)而不是手动实现的等价物。google.rpc.*
- 设置每语言包选项(等),使生成的代码保持惯用,而不是泄漏原始proto路径。
java_package
边界:同系列的拥有通知窗口、废弃沟通和跨范式的治理通用性。本skill拥有如何在gRPC原生中编码和强制版本——上面的包后缀规则和第3步的门禁。新主要版本不能依赖前一个主要版本,稳定资源在主要版本之间保持相同的资源名称,以便v2客户端可以寻址v1客户端创建的内容。
samber/developer-platform-skills@api-versioning-policy参见references/proto-governance-reference.md获取文件内容顺序、每个约定操作的buf lint规则名称、基于通道与基于发布的版本演进,以及弃用机制。
3. Breaking-change gates
3. 破坏性变更门禁
Protobuf's encoding only guarantees wire compatibility, not client-code compatibility (AIP-180's distinction) - so the gate must be tooling, not reviewer vigilance:
- Run on every PR as a hard CI gate, plus
buf breaking --against <main>with the STANDARD set. Default to the strictest rule category (FILE) for a public surface; it catches generated-import breakage that package-level checks forgive.buf lint - Know the traps the tooling catches that human reviewers miss:
- A rename is semantically remove-and-add (always breaking).
- Moving a field into or out of a breaks generated Go stubs.
oneof - Moving a message to a different file breaks generated imports - the file path is part of the contract.
- The blind spot buf leaves open: does not inspect custom options like
buf breaking, so a team can break its REST-facing URL/verb mapping while every proto gate stays green. Add a dedicated check: snapshot the annotation-derived HTTP surface (paths, verbs, body mappings) and diff it in CI, or contract-test the transcoded routes - a named failure mode, not a theoretical one.google.api.http - Test both directions across a version boundary: old client against new server and new client against old server - "does the schema parse" is not "does the cross-version RPC succeed".
- Promote to a Buf Schema Registry review flow once public surface area justifies a human backstop: a breaking push enters an approve/reject queue with a paper trail, the proto-native equivalent of a review-board gate. Teams not on buf can use Salesforce's against a checked-in
proto-backwards-compat-maven-plugin.proto.lock
Deprecation is proto-native too: (available on messages, enums, enum values, and services) plus an AIP-192 comment saying why and what replaces it. Channel expectations differ sharply:
deprecated = true- Alpha may vanish without notice.
- Beta may be removed after a recommended 180 days deprecated.
- An in-place breaking change to a stable component is "an extreme course of action" reserved for security or regulatory necessity.
The notice windows themselves are 's territory.
samber/developer-platform-skills@api-versioning-policyProtobuf的编码只保证线兼容性,不保证客户端代码兼容性(AIP-180的区分)——所以门禁必须是工具,而不是审查者的警觉:
- 在每个PR上运行作为硬性CI门禁,加上使用STANDARD集合的
buf breaking --against <main>。对于公共表面,默认使用最严格的规则类别(FILE);它可以捕获包级检查忽略的生成导入破坏。buf lint - 了解工具捕获而人类审查者遗漏的陷阱:
- 重命名在语义上是删除和添加(总是破坏性的)。
- 将字段移入或移出会破坏生成的Go stub。
oneof - 将消息移动到不同的文件会破坏生成的导入——文件路径是契约的一部分。
- buf留下的盲区: 不检查自定义选项如
buf breaking,因此团队可以在所有proto门禁保持绿色的情况下破坏其REST面对的URL/动词映射。添加专用检查:快照注解派生的HTTP表面(路径、动词、body映射)并在CI中diff,或对转码路由进行契约测试——这是一个明确的失败模式,不是理论上的。google.api.http - 在版本边界上双向测试:旧客户端对新高服务器和新客户端对旧服务器——“schema能否解析”不是“跨版本RPC是否成功”。
- 一旦公共表面面积证明需要人工后盾,升级到Buf Schema Registry审查流程:破坏性推送进入带有纸质记录的批准/拒绝队列,这是proto原生的审查板门禁。不在buf上的团队可以使用Salesforce的对照检入的
proto-backwards-compat-maven-plugin。proto.lock
弃用也是proto原生的:(可用于消息、枚举、枚举值和服务)加上AIP-192注释说明原因和替代品。通道期望差异很大:
deprecated = true- Alpha可以无通知消失。
- Beta可能在推荐的180天弃用期后移除。
- 对稳定组件的原地破坏性变更是“极端行动方案”,保留给安全或监管必要性。
通知窗口本身是的领域。
samber/developer-platform-skills@api-versioning-policy4. Streaming posture: unary by default
4. 流式姿态:默认一元
gRPC's four call types are not equally priced on a public surface. Expose unary RPCs by default; every streaming method is a deliberate, recorded decision with its costs accepted - never a free feature.
- Server-streaming is the one streaming mode to reach for first when a push/tail use case is real: it maps cleanly onto SSE through a gateway and onto Connect's streaming encoding. Notifications, log tailing, progressive results.
- Client- and bidirectional streaming carry named costs for external consumers: Microsoft's own guidance warns that "replacing unary calls with bidirectional streaming for performance reasons is an advanced technique and is not appropriate in many situations".
- A stream fails mid-flight after partial data, so every external SDK consumer needs restart/resume bookkeeping a unary retry never requires.
- Retries stop being transparently client-library-managed.
- Browsers cannot do client or bidi streaming over HTTP/1.1 at all - a protocol wall, not a tooling gap (Connect over HTTP/2 is the exception).
- Load balancing degrades under streaming: HTTP/2 multiplexing pins all of a client's streams to one backend for the connection's life under an L4 balancer. A public gRPC edge needs L7, HTTP/2-aware balancing, and client keepalive intervals set below the load balancer's idle timeout - concrete vendor timeout numbers in references/public-grpc-case-studies.md.
- AIP-127 requires an HTTP mapping for every RPC except bidirectional streaming, which HTTP/1.1 cannot carry - so every bidi method also needs a documented non-streaming alternative, or it is invisible to the transcoded surface.
Enforce the decision, don't just document it: when the surface commits to unary-only, enable buf's lint category so a future PR cannot silently introduce a streaming method the gateway, SDKs, or transport cannot carry. Sibling owns retry mechanics, deadline propagation, and backoff generics; this step only decides which call shapes the public contract offers.
UNARY_RPCsamber/developer-platform-skills@api-idempotency-retrygRPC的四种调用类型在公共表面上的代价不同。默认暴露一元RPC;每个流式方法都是一个经过深思熟虑、记录的决策,其成本被接受——从来不是免费功能。
- 服务器流式是当推送/尾随用例真实时第一个要使用的流式模式:它通过网关干净地映射到SSE,并映射到Connect的流式编码。通知、日志尾随、渐进结果。
- 客户端和双向流式对外部消费者带有命名的成本:微软自己的指南警告“用双向流式替换一元调用以提升性能是一种高级技术,在许多情况下并不合适”。
- 流在部分数据后中途失败,所以每个外部SDK消费者都需要一元重试永不需要的重启/恢复记账。
- 重试不再是透明的客户端库管理。
- 浏览器在HTTP/1.1下完全不能做客户端或双向流式——这是一个协议墙,不是工具缺口(通过HTTP/2的Connect是例外)。
- 负载均衡在流式下会退化:HTTP/2多路复用将客户端的所有流在整个连接生命周期内固定到一个后端(在L4负载均衡器下)。公共gRPC边缘需要L7、HTTP/2感知的负载均衡,客户端keepalive间隔设置在负载均衡器空闲超时之下——具体供应商超时数字在references/public-grpc-case-studies.md中。
- AIP-127要求每个RPC都有HTTP映射,除了双向流式——HTTP/1.1无法承载——所以每个双向方法也需要一个文档化的非流式替代方案,否则它对转码表面不可见。
强制执行决策,不要只文档化:当表面承诺仅一元时,启用buf的 lint类别,以便未来的PR不能静默引入网关、SDK或传输无法承载的流式方法。同系列的拥有重试机制、截止时间传播和退避通用性;本步只决定公共契约提供哪些调用形状。
UNARY_RPCsamber/developer-platform-skills@api-idempotency-retry5. Error model: google.rpc.Status mechanics
5. 错误模型:google.rpc.Status机制
Sibling owns taxonomy philosophy - code catalogs, message writing, retryability signaling. This step owns the gRPC-native mechanics and the two traps that bite public surfaces:
samber/developer-platform-skills@api-error-design- Use :
google.rpc.Status- : one of the 17 canonical codes.
code - : developer-facing English, never localized user text.
message - : typed extras from
details(google/rpc/error_details.protofirst,ErrorInfofor field violations,BadRequestpaired withRetryInfo).RESOURCE_EXHAUSTED
- Return the most specific applicable code (over
OUT_OF_RANGEwhen both apply). Defaulting everything toFAILED_PRECONDITION/UNKNOWNas a defensive habit is the named anti-pattern: it flattens every status dashboard into one bucket.INTERNAL - Sanitize outbound errors structurally, not just prose: is documented as internal-only - filter it from
DebugInfobefore the response leaves; sanitizingdetailsalone is insufficient.message - Trap one - the mapping table's direction. The gRPC project's own documentation is explicit: "servers must not use this table to determine an HTTP status code to use... the mappings are neither symmetric nor 1-to-1." The canonical gRPC↔HTTP table exists for clients interpreting responses that arrived without a ; a gateway translating errors outward needs its own deliberate mapping, confirmed against the transcoding implementation actually in use.
grpc-status - Trap two - HTTP collapses distinctions. In the common gateway mapping, three codes (,
INVALID_ARGUMENT,FAILED_PRECONDITION) all land on 400, andOUT_OF_RANGEmaps to the non-standard 499. Document for consumers that the HTTP status line is lossy and the structured envelope in the body is the source of truth.CANCELLED - Verify the envelope survives your gateway:
- Envoy needs and every detail type present in the proto descriptor (missing types drop silently).
convert_grpc_status: true - Connect serializes errors as JSON natively.
- grpc-gateway's default handler flattens the message and drops entirely - a custom error handler is mandatory there if structured detail matters.
details
- Envoy needs
Only is generally safe to auto-retry without extra signal (AIP-194); everything else needs an explicit or caller judgment. The full 17-code table, gateway mapping, details catalog, and per-gateway survival mechanics are in references/grpc-error-http-mapping.md.
UNAVAILABLERetryInfo同系列的拥有分类法哲学——代码目录、消息编写、可重试性信号。本步拥有gRPC原生机制和困扰公共表面的两个陷阱:
samber/developer-platform-skills@api-error-design- 使用:
google.rpc.Status- :17个规范代码之一。
code - :面向开发者的英文,绝不是本地化的用户文本。
message - :来自
details的类型化附加信息(google/rpc/error_details.proto优先,ErrorInfo用于字段违规,BadRequest与RetryInfo配对)。RESOURCE_EXHAUSTED
- 返回最具体的适用代码(两者都适用时优于
OUT_OF_RANGE)。将所有内容默认为FAILED_PRECONDITION/UNKNOWN作为防御性习惯是命名的反模式:它将所有状态仪表板扁平化为一个桶。INTERNAL - 结构化地清理出站错误,而不仅仅是文本:被文档化为仅内部使用——在响应离开之前在
DebugInfo中过滤它;仅清理details是不够的。message - 陷阱一——映射表的方向。 gRPC项目自己的文档明确说:“服务器不能使用此表来确定HTTP状态码……映射既不对称也非一对一。”规范的gRPC↔HTTP表用于客户端解释没有到达的响应;向外转换错误的网关需要自己的深思熟虑的映射,并对照实际使用的转码实现确认。
grpc-status - 陷阱二——HTTP折叠了区分。 在常见的网关映射中,三个代码(、
INVALID_ARGUMENT、FAILED_PRECONDITION)都落在400上,OUT_OF_RANGE映射到非标准的499。向消费者文档化HTTP状态行是有损的,body中的结构化信封是真相来源。CANCELLED - 验证信封在你的网关中幸存:
- Envoy需要以及proto描述符中存在的每个detail类型(缺失类型静默丢弃)。
convert_grpc_status: true - Connect原生地将错误序列化为JSON。
- grpc-gateway的默认处理器扁平化消息并完全丢弃——如果结构化细节重要,那里必须有自定义错误处理器。
details
- Envoy需要
只有通常可以安全自动重试而无需额外信号(AIP-194);其他所有都需要显式的或调用者判断。完整的17代码表、网关映射、details目录和每个网关的信封幸存机制在references/grpc-error-http-mapping.md中。
UNAVAILABLERetryInfo6. Transcoding and gateway architecture
6. 转码和网关架构
The design decision this step owns is what edge serves non-gRPC consumers and how the mapping is expressed - AIP-127's annotation is the convention every implementation converged on ("APIs must provide HTTP definitions for each RPC except bi-directional streaming RPCs"). The menu, ranked:
google.api.http- effort:
raw public gRPC edge > grpc-gateway == Envoy transcoder > Connect-RPC - value:
Connect-RPC > Envoy transcoder > grpc-gateway > raw public gRPC edge - efficiency:
Connect-RPC > Envoy transcoder > grpc-gateway > raw public gRPC edge
Effort here is what the team operates and regenerates; value is audience reached per proto contract, with error and streaming fidelity preserved. The effort tie is genuine: regenerating a Go proxy on every proto change and keeping a descriptor set plus filter config in sync are different work at the same order of magnitude - and "you already run Envoy" is exactly the condition that breaks it.
grpc-gateway == Envoy-
Default rung: Connect-RPC. One server speaks Connect, gRPC, and gRPC-Web:
- No proxy required.
- Browsers served natively.
- Full streaming over HTTP/2.
- Errors are JSON out of the box.
- Wire-compatible with gRPC (validated against Google's interop suite).
Entered CNCF Sandbox in April 2024, already in production at CrowdStrike, PlanetScale, RedPanda, Chick-fil-A, Bluesky, and Dropbox. Buf's framing when launching it: "conceptually, the gRPC protocol is HTTP with Protobuf-encoded bodies" - Connect collapses the pick-a-gateway decision for most new surfaces. -
Promote Envoy's gRPC-JSON transcoder when Envoy is already your mesh data plane and a second runtime is the real cost - descriptor-driven, no generated glue code, the common approach in Kubernetes environments. Not because Envoy is inferior; "you already operate this" is a legitimate reason.
-
Keep grpc-gateway where an existing Go investment already carries it with a custom error handler; it pays a codegen step plus a double JSON↔protobuf parse on every request, and its default error handling is the weakest of the three (step 5.6).
-
Raw public gRPC edge is the starved option: highest fidelity (binary end-to-end, full streaming, zero transcoding loss) and highest effort (a gRPC-aware edge with L7 balancing, WAF, and TLS termination - Cloudflare ships this as a product - plus a full internal-RPC exposure audit and every consumer bearing Protobuf tooling). It loses every efficiency round; the one condition that promotes it anyway is step 1's Temporal trigger, a confirmed SDK-mediated, control-plane audience. Even then, dual-publish an HTTP surface alongside, as Temporal does.
-
Ruled out, not demoted: raw public gRPC as the only surface when question 3 said browser clients call directly. Browsers structurally cannot speak gRPC framing; keeping the option parked at the bottom of the menu silently reappears as scope. Delete it and choose among the three above.
This ranking is a default, not a law. Re-rank against question 6 and what you know about the team:
- An Envoy-fluent platform team gets the transcoder nearly free.
- A hard ship date promotes whatever the team already operates.
- A contract several products will share for years weighs error-fidelity (value axis) over this quarter's effort.
Performance-critical consumers should be steered to native gRPC or Connect's binary mode regardless of rung - every JSON-transcoding path pays a real parse cost.
Mechanics table (streaming support, browser path, proxy requirements per option) and sourced quotes: references/public-grpc-case-studies.md.
本步拥有的设计决策是什么边缘为非gRPC消费者服务,以及映射如何表达——AIP-127的注解是每个实现都趋同的约定(“API必须为每个RPC提供HTTP定义,除了双向流式RPC”)。菜单,按排名:
google.api.http- 工作量:
原始公共gRPC边缘 > grpc-gateway == Envoy转码器 > Connect-RPC - 价值:
Connect-RPC > Envoy转码器 > grpc-gateway > 原始公共gRPC边缘 - 效率:
Connect-RPC > Envoy转码器 > grpc-gateway > 原始公共gRPC边缘
这里的工作量是指团队运营和重新生成的内容;价值是每个proto契约所达到的受众,同时保持错误和流式保真。的工作量并列是真实的:在每次proto变更时重新生成Go代理并保持描述符集和过滤器配置同步是不同的工作,但数量级相同——而“你已经运行Envoy”正是打破并列的条件。
grpc-gateway == Envoy-
默认层级:Connect-RPC。 一个服务器同时说Connect、gRPC和gRPC-Web:
- 不需要代理。
- 原生服务浏览器。
- 通过HTTP/2的完整流式。
- 错误开箱即用是JSON。
- 与gRPC线兼容(已针对Google的互操作套件验证)。
2024年4月进入CNCF Sandbox,已在CrowdStrike、PlanetScale、RedPanda、Chick-fil-A、Bluesky和Dropbox生产使用。Buf在推出时的框架:“从概念上讲,gRPC协议是带有Protobuf编码体的HTTP”——Connect为大多数新表面消除了选择网关的决策。 -
当Envoy已经是你的网格数据平面且第二个运行时是真正成本时,提升Envoy的gRPC-JSON转码器——描述符驱动,无生成胶水代码,Kubernetes环境中的常见方法。不是因为Envoy较差;“你已经运营它”是合法理由。
-
在现有的Go投资已经承载它并带有自定义错误处理器的地方保留grpc-gateway;它支付代码生成步骤加上每个请求的双重JSON↔protobuf解析,其默认错误处理是三者中最弱的(步骤5.6)。
-
原始公共gRPC边缘是受限选项: 最高保真度(二进制端到端、完整流式、零转码损失)和最高工作量(具有L7负载均衡、WAF和TLS终止的gRPC感知边缘——Cloudflare作为产品提供——加上完整的内部RPC暴露审计,每个消费者都承担Protobuf工具)。它在每轮效率上都输;无论如何提升它的一个条件是步骤1的Temporal触发,即确认的SDK中介、控制平面受众。即便如此,也要像Temporal那样同时发布HTTP表面。
-
被排除,不是降级: 当问题3说浏览器客户端直接调用时,原始公共gRPC作为_唯一_表面。浏览器在结构上无法说gRPC帧格式;将该选项停在菜单底部会静默地作为范围重新出现。删除它并从上面三个中选择。
这个排名是默认,不是法律。根据问题6和你对团队的了解重新排名:
- 熟悉Envoy的平台团队几乎免费获得转码器。
- 硬上线日期会提升团队已经运营的内容。
- 一个多个产品将共享多年的契约会在本季度的工作量之上加权错误保真度(价值轴)。
性能关键的消费者应该被引导到原生gRPC或Connect的二进制模式,无论层级如何——每个JSON转码路径都支付真实的解析成本。
机制表(流式支持、浏览器路径、每个选项的代理要求)和来源引用:references/public-grpc-case-studies.md。
7. External auth, TLS, and the exposure audit
7. 外部认证、TLS和暴露审计
The internal-auth instinct is the wrong one: mTLS authenticates workloads and scales fine inside a mesh, but issuing and revoking client certificates for developers you have never met is operationally impractical. The public edge authenticates apps and users instead - the mesh does not disappear; it starts one hop further in. Order of operations:
- TLS first, non-negotiable: credentials travel as gRPC metadata, plaintext without transport security; most implementations refuse to send credentials over an unencrypted channel at all.
- OAuth2 flow by caller type: Client Credentials for machine/service callers, Authorization Code + PKCE where a human authenticates interactively. Token as metadata, validated in a server interceptor.
authorization: Bearer <token> - Token mechanics:
- JWT as token format, strongly signed (RS256), with expiry claims.
- Token introspection against the authorization server, cached so it does not tax every call.
- An established identity provider over a hand-rolled issuer once external caller count justifies it.
- Enforce scopes at the method level - not "is this caller authenticated" but "is this token scoped for this RPC".
- API keys complement, never replace, authentication (Google's own Endpoints distinction): a key identifies a project/consumer for quota attribution and coarse gating, not an authenticated identity.
- Rate limiting signals over-quota with (429 once transcoded) paired with a
RESOURCE_EXHAUSTEDdetail so callers back off instead of hammering the same limit.RetryInfo - The exposure audit, mandatory for any internal-first service: disable server reflection in production (it is an API-discovery tool for attackers once the port is public), then enumerate the full method surface for admin, debug, and monitoring RPCs that were never designed for external visibility. TLS and auth switched on do not close this hole.
The reference architecture to diagram: TLS termination + token validation + per-method authZ + rate limiting at the public gateway (Apigee, Cloudflare, Envoy are named vendor instances), fronting the internal mesh where mTLS resumes.
内部认证本能是错误的:mTLS认证的是_工作负载_,在网格内扩展良好,但为你从未见过的开发者签发和撤销客户端证书在操作上不切实际。公共边缘改为认证应用和用户——网格不会消失;它从一跳之后开始。操作顺序:
- 首先是TLS,不可协商:凭据作为gRPC元数据传输,没有传输安全就是明文;大多数实现完全拒绝在未加密的通道上发送凭据。
- OAuth2流程按调用者类型:机器/服务调用者使用Client Credentials,人类交互式认证使用Authorization Code + PKCE。令牌作为元数据,在服务器拦截器中验证。
authorization: Bearer <token> - 令牌机制:
- JWT作为令牌格式,强签名(RS256),带过期声明。
- 对授权服务器进行令牌内省,缓存以免每次调用都征税。
- 一旦外部调用者数量证明合理,使用成熟的身份提供商而不是自建发行者。
- 在方法级别强制作用域——不是“此调用者是否已认证”而是“此令牌是否被授权用于此RPC”。
- API密钥补充而不是替代认证(Google自己的Endpoints区分):密钥标识项目/消费者用于配额归属和粗略门控,而不是认证身份。
- 速率限制用(转码后为429)加上
RESOURCE_EXHAUSTEDdetail来信号超配额,以便调用者退避而不是敲打同一个限制。RetryInfo - 暴露审计,对任何内部优先服务都是强制性的:在生产中禁用服务器反射(一旦端口公开,它是攻击者的API发现工具),然后枚举完整方法表面,查找从未为外部可见性设计的管理、调试和监控RPC。TLS和认证开启并不能关闭这个漏洞。
要绘制的参考架构:TLS终止 + 令牌验证 + 方法级授权 + 公共网关处的速率限制(Apigee、Cloudflare、Envoy是命名的供应商实例),前端是内部网格,mTLS在其中恢复。
Failure modes
失败模式
Anti-pattern checklist - each is a direct audit finding:
- The server or gateway derives HTTP status codes from the client-side gRPC↔HTTP mapping table the gRPC project explicitly forbids servers to use.
- green while a
buf breakingannotation change broke the public REST mapping - the blind spot with no dedicated check.google.api.http - grpc-gateway in production with the default error handler, silently dropping every payload.
details - Server reflection enabled on a publicly reachable service.
- Two RPCs sharing a request or response message; a bare scalar as a request type.
- A version number in a proto filename, or a directory that does not mirror the package.
- A streaming method added to a browser-facing surface with no non-streaming alternative and no recorded tradeoff decision.
- Everything failing as or
UNKNOWN;INTERNALreaching external callers.DebugInfo - An L4 balancer in front of streaming RPCs, pinning every client's streams to one backend.
- Client keepalive interval longer than the load balancer's idle timeout, so the balancer drops connections before keepalive fires.
- Beta functionality shipped inside a package.
v1
反模式检查清单——每个都是直接的审计发现:
- 服务器或网关从gRPC项目明确禁止服务器使用的客户端侧gRPC↔HTTP映射表派生HTTP状态码。
- 绿色,而
buf breaking注解变更破坏了公共REST映射——盲区没有专用检查。google.api.http - grpc-gateway在生产中使用默认错误处理器,静默丢弃每个负载。
details - 在可公开到达的服务上启用服务器反射。
- 两个RPC共享一个请求或响应消息;裸标量作为请求类型。
- proto文件名中有版本号,或目录不镜像包。
- 在面向浏览器的表面上添加流式方法,没有非流式替代方案,也没有记录的权衡决策。
- 所有失败都是或
UNKNOWN;INTERNAL到达外部调用者。DebugInfo - L4负载均衡器在流式RPC前面,将每个客户端的流固定到一个后端。
- 客户端keepalive间隔长于负载均衡器的空闲超时,因此负载均衡器在keepalive触发之前断开连接。
- Beta功能在包中发布。
v1
Measurement
测量
Pass gates, self-set from the sourced rules above (no industry-standard thresholds exist for this surface; each gate is binary and auditable):
- Breaking-change gate: zero breaking proto changes reach the default branch outside a new major version - (FILE category) green in CI on 100% of merges.
buf breaking - Annotation coverage: 100% of RPCs carrying annotations covered by a dedicated mapping diff or contract test, because buf does not check them.
google.api.http - Streaming discipline: lint enabled on a unary-only surface, or every streaming method paired with a recorded accepted-tradeoff decision and (for bidi) a non-streaming alternative.
UNARY_RPC - Exposure: reflection disabled in production and the internal-RPC audit completed before launch - both binary.
Iterate the design until all four pass. Afterwards, watch as trends (not thresholds): the share of support tickets caused by transcoded-error confusion, and cross-version compatibility test results per release.
通过门禁,根据上述来源规则自行设置(此表面没有行业标准阈值;每个门禁都是二进制的且可审计的):
- 破坏性变更门禁:在新主要版本之外,零破坏性proto变更到达默认分支——CI中100%的合并(FILE类别)绿色。
buf breaking - 注解覆盖:100%的RPC携带注解,并由专用的映射diff或契约测试覆盖,因为buf不检查它们。
google.api.http - 流式纪律:在仅一元表面上启用lint,或每个流式方法都配有记录的已接受权衡决策,并且(对于双向)有非流式替代方案。
UNARY_RPC - 暴露:在生产中禁用反射,并在启动前完成内部RPC审计——两者都是二进制的。
迭代设计直到所有四个通过。之后,作为趋势(而非阈值)观察:由转码错误混淆引起的支持工单比例,以及每个版本的跨版本兼容性测试结果。
Invocation examples
调用示例
- "Our platform team wants to expose our internal gRPC services to partners - design the public surface and tell us whether that's even the right move."
- "Design proto package and versioning conventions for a public API we'll publish protos for, with CI gates against breaking changes."
- "We're putting Connect-RPC in front of our gRPC backend for browser clients - review the error model and the transcoding architecture."
- "Audit this gRPC service before we open it to external developers."
- “我们的平台团队想将内部gRPC服务暴露给合作伙伴——设计公共表面并告诉我们这是否是正确的做法。”
- “为我们将发布protos的公共API设计proto包和版本控制约定,并带有针对破坏性变更的CI门禁。”
- “我们将Connect-RPC放在gRPC后端前面供浏览器客户端使用——审查错误模型和转码架构。”
- “在我们向外部开发者开放之前,审计这个gRPC服务。”
References
参考资料
- references/proto-governance-reference.md - AIP file/package rules in detail, buf lint rule names, AIP-180 compatibility traps, channel vs release versioning, deprecation mechanics.
- references/grpc-error-http-mapping.md - the 17 canonical codes, the full gateway HTTP mapping with its collapse points, the error-details catalog, and per-gateway envelope-survival mechanics.
- references/public-grpc-case-studies.md - Google, Square, Dropbox, Cloudflare, Temporal case studies with precision caveats; the transcoding-option mechanics table; Connect-RPC adoption facts; load-balancer keepalive numbers.
See also, same collection:
- - the umbrella decision of which surfaces to offer at all; this skill re-validates its gRPC branch and executes it.
samber/developer-platform-skills@api-integration-surface-strategy - - whole-surface consistency review for the REST/JSON side; it hands gRPC-specific design here.
samber/developer-platform-skills@public-api-design-review - - the structural analogue for a public GraphQL surface; same gate-first shape, different paradigm.
samber/developer-platform-skills@public-graphql-api-design - - notice windows, sunset communication, and governance generics; this skill owns only the gRPC-native version encoding and gates.
samber/developer-platform-skills@api-versioning-policy - - error taxonomy philosophy, message writing, retryability signaling; this skill owns only
samber/developer-platform-skills@api-error-designmechanics and the HTTP-mapping traps.google.rpc.Status - - client retry mechanics, deadline propagation, and backoff generics consumed by step 4's streaming decision.
samber/developer-platform-skills@api-idempotency-retry
- references/proto-governance-reference.md - 详细的AIP文件/包规则、buf lint规则名称、AIP-180兼容性陷阱、通道与发布版本控制、弃用机制。
- references/grpc-error-http-mapping.md - 17个规范代码、完整的网关HTTP映射及其折叠点、错误details目录,以及每个网关的信封幸存机制。
- references/public-grpc-case-studies.md - Google、Square、Dropbox、Cloudflare、Temporal案例研究及精确性注意事项;转码选项机制表;Connect-RPC采用事实;负载均衡器keepalive数字。
另见,同一系列:
- - 关于提供哪些表面的总括决策;本skill重新验证其gRPC分支并执行。
samber/developer-platform-skills@api-integration-surface-strategy - - REST/JSON侧的整体表面一致性审查;它将gRPC特定的设计移交到这里。
samber/developer-platform-skills@public-api-design-review - - 公共GraphQL表面的结构类比;相同的门禁优先形状,不同的范式。
samber/developer-platform-skills@public-graphql-api-design - - 通知窗口、日落沟通和治理通用性;本skill只拥有gRPC原生的版本编码和门禁。
samber/developer-platform-skills@api-versioning-policy - - 错误分类法哲学、消息编写、可重试性信号;本skill只拥有
samber/developer-platform-skills@api-error-design机制和HTTP映射陷阱。google.rpc.Status - - 客户端重试机制、截止时间传播和退避通用性,由第4步的流式决策消费。
samber/developer-platform-skills@api-idempotency-retry