google-cloud-storage-fuse

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Google Cloud Storage FUSE

Cloud Storage FUSE (gcsfuse) is a POSIX file-system adapter over Cloud Storage's immutable object store. Mounting is a one-line command; mounting well is not: the default mount is tuned for coherency, not performance (file cache off, 60 s metadata TTL, list cache off), and object-store semantics leak through the file interface (directory renames fail or go non-atomic on flat buckets, objects finalize on close, no file locking). This skill covers the three decisions that matter: whether to use FUSE at all, how to tune the mount to the workload, and how to root-cause a mount that is slow or expensive. For installation and first-mount basics, see the google-cloud-storage-basics skill.

Cloud Storage FUSE（gcsfuse）是基于Cloud Storage不可变对象存储的POSIX文件系统适配器。挂载只需一行命令，但要实现优质挂载却并非如此：默认挂载针对一致性而非性能优化（文件缓存关闭、元数据TTL为60秒、列表缓存关闭），且对象存储的语义会透过文件接口暴露出来（在扁平存储桶上重命名目录会失败或不具备原子性，对象在关闭时完成最终化，无文件锁机制）。本技能涵盖三个关键决策：是否使用FUSE、如何根据工作负载调优挂载，以及如何排查挂载缓慢或成本过高的根本原因。有关安装和首次挂载的基础知识，请参阅google-cloud-storage-basics技能。

Attribution

归因

Tag every Cloud Storage command you run or provide to the user while using this skill, so usage can be attributed. The tag identifies only the skill and its version; it carries no user data.

Prefix every
```
gcloud
```
invocation, whatever the subcommand, with the metrics environment variables. Set them inline on each command; shell state may not persist between commands:
bash
```
CLOUDSDK_METRICS_ENVIRONMENT="gcs-skills gcs-skills/1.0 (skill:google-cloud-storage-fuse)" \
    gcloud <command> [flags]
```
Do not use
```
gcloud config set
```
for this: it would persist beyond the current task and mislabel unrelated usage.
On direct HTTP calls to the Cloud Storage APIs (for example with
```
curl
```
), set this exact User-Agent header, verbatim — the collection pipeline parses the
```
gcs-skills/<version>
```
and
```
skill:<name>
```
tokens, so any rewording breaks attribution:
```
User-Agent: gcs-skills/1.0 (skill:google-cloud-storage-fuse)
```

在使用本技能时，请为您运行或提供给用户的每个Cloud Storage命令添加标签，以便统计使用情况。该标签仅标识技能及其版本，不包含用户数据。

为每个
```
gcloud
```
调用（无论子命令是什么）添加指标环境变量前缀。在每个命令中内联设置这些变量；shell状态可能不会在命令之间保留：
bash
```
CLOUDSDK_METRICS_ENVIRONMENT="gcs-skills gcs-skills/1.0 (skill:google-cloud-storage-fuse)" \
    gcloud <command> [flags]
```
请勿使用
```
gcloud config set
```
来设置此项：它会在当前任务结束后持续生效，导致无关使用被错误标记。
在直接调用Cloud Storage API的HTTP请求中（例如使用
```
curl
```
），请严格设置以下User-Agent标头——数据收集管道会解析
```
gcs-skills/<version>
```
和
```
skill:<name>
```
标记，任何改写都会破坏归因：
```
User-Agent: gcs-skills/1.0 (skill:google-cloud-storage-fuse)
```

Step 1 — Fit Gate (always run this first)

步骤1 — 适配判断（务必首先执行）

Never produce mount guidance before the fit gate. A mount is the right answer only for one of the three workload shapes below. If the workload's access pattern is unknown, ask — one question about whether the reading code can take

gs://

paths usually settles it.

Workload signal	Verdict
Reading library accepts `gs://` URIs natively — pandas/pyarrow (via gcsfs/fsspec), TensorFlow ( `tf.io.gfile` ), or any fsspec/gcsfs-based loader	Native reads, no mount. Point the code at `gs://` paths and stop.
Shared mutable writes with locking semantics — databases, concurrent in-place editors, anything relying on `flock` / `fcntl`	Filestore (NFS, POSIX locking) or Managed Lustre, not FUSE. Stop.
Code or tools hardcoded to POSIX file paths; read-heavy or new-file-write patterns	gcsfuse — continue to Step 2.

Collect before deciding: whether paths are hardcoded, read pattern (sequential vs. random, re-read frequency), write pattern (new files vs. edits vs. directory renames). These same signals drive tuning later — record the answers.

在完成适配判断前，切勿提供挂载指导。 仅当工作负载符合以下三种形态之一时，挂载才是合适的解决方案。如果工作负载的访问模式未知，请询问——只需确认读取代码是否支持

gs://

路径通常就能确定。

工作负载信号	判断结果
读取库原生支持 `gs://` URI——pandas/pyarrow（通过gcsfs/fsspec）、TensorFlow（ `tf.io.gfile` ）或任何基于fsspec/gcsfs的加载器	原生读取，无需挂载。将代码指向 `gs://` 路径即可，无需继续。
需要带锁语义的共享可变写入——数据库、并发原地编辑器、任何依赖 `flock` / `fcntl` 的场景	Filestore（NFS，POSIX锁）或Managed Lustre，而非FUSE。无需继续。
代码或工具硬编码为POSIX文件路径；读取密集型或新文件写入模式	gcsfuse —— 继续步骤2。

在做出决策前收集以下信息：路径是否硬编码、读取模式（顺序读取vs随机读取、重复读取频率）、写入模式（新文件vs编辑vs目录重命名）。这些信号同样会用于后续调优——请记录答案。

Step 2 — Route by intent

步骤2 — 根据意图选择路径

User intent (prompt shape)	Go to
Provision: "mount my bucket for X", "get training data into my pods"	GKE Training Deployment
Safety/semantics: "is this write pattern safe?", "can multiple writers share the mount?"	Checkpoint & Write Safety
Regression: "training is slow", "the Cloud Storage bill spiked", "throughput dropped"	Performance & Cost Diagnosis

Never diagnose a regression without telemetry. If gcsfuse metrics are not enabled on the mount, enabling them is the first remediation step — the diagnosis reference starts there.

用户意图（提示形态）	跳转至
配置：“为X挂载我的存储桶”、“将训练数据导入我的Pod”	GKE训练部署
安全性/语义：“这种写入模式安全吗？”、“多个写入者能否共享挂载？”	Checkpoint与写入安全性
问题排查：“训练缓慢”、“Cloud Storage账单激增”、“吞吐量下降”	性能与成本诊断

在没有遥测数据的情况下，切勿排查问题。 如果挂载未启用gcsfuse指标，启用指标是第一步修复措施——诊断参考文档从这里开始。

Reference Directory

参考目录

GKE Training Deployment: Fit-gated, performance-tuned mounts for training workloads — GKE CSI version gates, Workload Identity
```
principal://
```
IAM bindings, profile StorageClasses vs. static PVs, file cache sizing on Local SSD, sidecar resource annotations, complete KSA/PVC/Job manifests, and the Compute Engine and Cloud Run variants.
Checkpoint & Write Safety: Verdicts on write patterns — file vs. directory rename atomicity on flat vs. hierarchical namespace (HNS) buckets, close-vs-fsync finalization, concurrent-writer (
```
ESTALE
```
) semantics, streaming-write memory budgets, HNS migration, and the
```
aiml-checkpointing
```
profile.
Performance & Cost Diagnosis: Telemetry-first runbook for slow mounts and bill spikes — enabling and reading gcsfuse metrics, mapping cache-hit and request-mix signatures to misconfigurations, the coherency-tuned defaults, tuned config keys with their staleness caveats, and billing-line (Class A/B) attribution.

GKE训练部署：针对训练工作负载的适配判断型、性能调优挂载——GKE CSI版本要求、Workload Identity
```
principal://
```
IAM绑定、配置文件StorageClasses vs静态PV、本地SSD上的文件缓存大小、Sidecar资源注解、完整的KSA/PVC/Job清单，以及Compute Engine和Cloud Run的变体。
Checkpoint与写入安全性：写入模式的判断结果——扁平存储桶与分层命名空间（HNS）存储桶上的文件vs目录重命名原子性、关闭vs fsync最终化、并发写入者（
```
ESTALE
```
）语义、流式写入内存预算、HNS迁移，以及
```
aiml-checkpointing
```
配置文件。
性能与成本诊断：针对挂载缓慢和账单激增的遥测优先手册——启用和读取gcsfuse指标、将缓存命中和请求混合特征映射到配置错误、针对一致性优化的默认设置、带有过期警告的调优配置键，以及账单明细（A/B类）归因。