cofounder-ssh-key-rotation

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

SSH Key Rotation

SSH密钥轮换

Rotates the SSH key for all VMs of a deployed environment. After rotation, only the new key grants access -- the old key is permanently revoked from every VM.
为已部署环境的所有VM轮换SSH密钥。轮换完成后,只有新密钥可授予访问权限——旧密钥会从所有VM中永久撤销。

When to Use

使用场景

  1. Explicit request: The user asks to rotate, regenerate, or replace SSH keys.
  2. Missing local key: The agent needs to SSH into a VM (for logs, debugging, database access, etc.) but the expected key file does not exist on disk.
  1. 明确请求:用户要求轮换、重新生成或替换SSH密钥。
  2. 本地密钥缺失:Agent需要通过SSH访问VM(如查看日志、调试、数据库访问等),但磁盘上不存在预期的密钥文件。

Detecting a missing key

检测缺失的密钥

Before any SSH operation, the agent resolves the key path:
bash
REPO_NAME=$(gh repo view --json name -q .name)
在执行任何SSH操作之前,Agent会解析密钥路径:
bash
REPO_NAME=$(gh repo view --json name -q .name)

Preview environment

预览环境

SSH_KEY=~/.ssh/$REPO_NAME
SSH_KEY=~/.ssh/$REPO_NAME

Other environments (e.g., production)

其他环境(例如生产环境)

SSH_KEY=~/.ssh/$REPO_NAME-production

If the file does not exist (`test -f "$SSH_KEY"` fails), the key is missing and rotation is needed.
SSH_KEY=~/.ssh/$REPO_NAME-production

如果文件不存在(`test -f "$SSH_KEY"`执行失败),则密钥缺失,需要进行轮换。

Warnings (Always Communicate Before Proceeding)

注意事项(执行前务必告知用户)

Before starting rotation, always warn the user:
  1. Downtime: Rotation stops each VM, resets its SSH key, and restarts it. All VMs in the environment will experience downtime during the process (accessories first, then workers, then web -- to minimize user-facing downtime).
  2. Old key revoked: The old SSH key is permanently erased from all VMs'
    authorized_keys
    files. Anyone using the old key will lose access immediately.
  3. All environments are independent: Each environment (preview, production, etc.) has its own SSH key. Rotation only affects the specified environment.
Ask for explicit confirmation before proceeding.
开始轮换之前,务必向用户发出以下警告:
  1. 停机时间:轮换过程会停止每台VM,重置其SSH密钥,然后重启。环境中的所有VM在过程中都会经历停机(先处理附属服务,再处理工作节点,最后处理Web节点——以减少面向用户的停机时间)。
  2. 旧密钥被撤销:旧SSH密钥会从所有VM的
    authorized_keys
    文件中永久删除。任何使用旧密钥的用户都会立即失去访问权限。
  3. 所有环境相互独立:每个环境(预览、生产等)都有自己的SSH密钥。轮换仅影响指定的环境。
在执行前请获取用户的明确确认。

Rotation Procedure

轮换流程

Step 1: Determine the environment

步骤1:确定环境

Identify which environment needs rotation:
  • If the user specifies an environment, use it.
  • If the agent discovered a missing key during a troubleshooting attempt, use the environment that was being targeted.
  • If ambiguous, ask the user.
确定需要轮换的环境:
  • 如果用户指定了环境,则使用该环境。
  • 如果Agent在故障排查过程中发现密钥缺失,则使用目标环境。
  • 如果存在歧义,请询问用户。

Step 2: Generate a new SSH key locally

步骤2:在本地生成新的SSH密钥

Delete the old key file (if it exists) and generate a fresh one using the standard naming convention:
bash
REPO_NAME=$(gh repo view --json name -q .name)
删除旧密钥文件(如果存在),并按照标准命名规则生成新密钥:
bash
REPO_NAME=$(gh repo view --json name -q .name)

Preview environment

预览环境

rm -f ~/.ssh/$REPO_NAME ~/.ssh/$REPO_NAME.pub ssh-keygen -t ed25519 -f ~/.ssh/$REPO_NAME -N "" -C "$REPO_NAME-deploy" chmod 600 ~/.ssh/$REPO_NAME
rm -f ~/.ssh/$REPO_NAME ~/.ssh/$REPO_NAME.pub ssh-keygen -t ed25519 -f ~/.ssh/$REPO_NAME -N "" -C "$REPO_NAME-deploy" chmod 600 ~/.ssh/$REPO_NAME

Other environments (e.g., production)

其他环境(例如生产环境)

rm -f ~/.ssh/$REPO_NAME-production ~/.ssh/$REPO_NAME-production.pub ssh-keygen -t ed25519 -f ~/.ssh/$REPO_NAME-production -N "" -C "$REPO_NAME-deploy-production" chmod 600 ~/.ssh/$REPO_NAME-production
undefined
rm -f ~/.ssh/$REPO_NAME-production ~/.ssh/$REPO_NAME-production.pub ssh-keygen -t ed25519 -f ~/.ssh/$REPO_NAME-production -N "" -C "$REPO_NAME-deploy-production" chmod 600 ~/.ssh/$REPO_NAME-production
undefined

Step 3: Update the GitHub secret

步骤3:更新GitHub密钥

Upload the new private key to the corresponding GitHub secret:
bash
undefined
将新的私钥上传到对应的GitHub Secret:
bash
undefined

Preview

预览环境

gh secret set SSH_PRIVATE_KEY < ~/.ssh/$REPO_NAME
gh secret set SSH_PRIVATE_KEY < ~/.ssh/$REPO_NAME

Production (or other environment -- suffix matches env_name uppercased)

生产环境(或其他环境——后缀与大写的env_name匹配)

gh secret set SSH_PRIVATE_KEY_PRODUCTION < ~/.ssh/$REPO_NAME-production
undefined
gh secret set SSH_PRIVATE_KEY_PRODUCTION < ~/.ssh/$REPO_NAME-production
undefined

Step 4: Create and run the rotation caller workflow

步骤4:创建并运行轮换调用工作流

Create a caller workflow that invokes the reusable rotation workflow. This follows the same pattern as teardown workflows:
yaml
undefined
创建一个调用可重用轮换工作流的调用工作流,这与销毁工作流的模式相同:
yaml
undefined

.github/workflows/rotate-ssh-key-preview.yml

.github/workflows/rotate-ssh-key-preview.yml

name: Rotate SSH Key Preview on: workflow_dispatch:
permissions: contents: read
jobs: rotate: uses: locaweb/locaweb-cloud-provision/.github/workflows/rotate-ssh-key.yml@v1 with: env_name: "preview" secrets: CLOUDSTACK_API_KEY: ${{ secrets.CLOUDSTACK_API_KEY }} CLOUDSTACK_SECRET_KEY: ${{ secrets.CLOUDSTACK_SECRET_KEY }} SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}

For other environments, adjust `env_name` and the `SSH_PRIVATE_KEY` secret reference:

```yaml
name: Rotate SSH Key Preview on: workflow_dispatch:
permissions: contents: read
jobs: rotate: uses: locaweb/locaweb-cloud-provision/.github/workflows/rotate-ssh-key.yml@v1 with: env_name: "preview" secrets: CLOUDSTACK_API_KEY: ${{ secrets.CLOUDSTACK_API_KEY }} CLOUDSTACK_SECRET_KEY: ${{ secrets.CLOUDSTACK_SECRET_KEY }} SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}

对于其他环境,调整`env_name`和`SSH_PRIVATE_KEY`的Secret引用:

```yaml

.github/workflows/rotate-ssh-key-production.yml

.github/workflows/rotate-ssh-key-production.yml

name: Rotate SSH Key Production on: workflow_dispatch:
permissions: contents: read
jobs: rotate: uses: locaweb/locaweb-cloud-provision/.github/workflows/rotate-ssh-key.yml@v1 with: env_name: "production" secrets: CLOUDSTACK_API_KEY: ${{ secrets.CLOUDSTACK_API_KEY }} CLOUDSTACK_SECRET_KEY: ${{ secrets.CLOUDSTACK_SECRET_KEY }} SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY_PRODUCTION }}

Commit and push the workflow file, then trigger it:

```bash
git add .github/workflows/rotate-ssh-key-preview.yml
git commit -m "Add SSH key rotation workflow for preview"
git push
name: Rotate SSH Key Production on: workflow_dispatch:
permissions: contents: read
jobs: rotate: uses: locaweb/locaweb-cloud-provision/.github/workflows/rotate-ssh-key.yml@v1 with: env_name: "production" secrets: CLOUDSTACK_API_KEY: ${{ secrets.CLOUDSTACK_API_KEY }} CLOUDSTACK_SECRET_KEY: ${{ secrets.CLOUDSTACK_SECRET_KEY }} SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY_PRODUCTION }}

提交并推送工作流文件,然后触发它:

```bash
git add .github/workflows/rotate-ssh-key-preview.yml
git commit -m "Add SSH key rotation workflow for preview"
git push

Trigger the workflow

触发工作流

gh workflow run rotate-ssh-key-preview.yml
undefined
gh workflow run rotate-ssh-key-preview.yml
undefined

Step 5: Monitor the rotation

步骤5:监控轮换过程

bash
undefined
bash
undefined

Watch the run

查看运行列表

gh run list --workflow=rotate-ssh-key-preview.yml --limit=5 gh run watch <run-id>

Give the user a direct link to follow in the GitHub UI:

```bash
gh run list --limit=1 --json databaseId,url -q '.[0].url'
If the workflow fails, read the logs:
bash
gh run view <run-id> --log-failed
gh run list --workflow=rotate-ssh-key-preview.yml --limit=5 gh run watch <run-id>

为用户提供GitHub UI中的直接跟踪链接:

```bash
gh run list --limit=1 --json databaseId,url -q '.[0].url'
如果工作流失败,查看日志:
bash
gh run view <run-id> --log-failed

Step 6: Verify SSH access

步骤6:验证SSH访问权限

After the workflow completes successfully, verify that the new key works:
bash
REPO_NAME=$(gh repo view --json name -q .name)
工作流成功完成后,验证新密钥是否可用:
bash
REPO_NAME=$(gh repo view --json name -q .name)

Get the web IP from the latest deploy run

从最新的部署运行中获取Web节点IP

rm -rf $HOME/tmp/provision-output gh run list --workflow=deploy-preview.yml --status=success --limit=1 gh run download <run-id> --name provision-output --dir $HOME/tmp/provision-output cat $HOME/tmp/provision-output/provision-output.json
rm -rf $HOME/tmp/provision-output gh run list --workflow=deploy-preview.yml --status=success --limit=1 gh run download <run-id> --name provision-output --dir $HOME/tmp/provision-output cat $HOME/tmp/provision-output/provision-output.json

Test SSH with the new key

使用新密钥测试SSH连接

ssh -i ~/.ssh/$REPO_NAME -o ConnectTimeout=10 root@<web_ip> "echo 'SSH rotation successful'"
undefined
ssh -i ~/.ssh/$REPO_NAME -o ConnectTimeout=10 root@<web_ip> "echo 'SSH rotation successful'"
undefined

Step 7: Resume the original task

步骤7:恢复原始任务

If rotation was triggered because the agent needed SSH access for troubleshooting, resume the original operation (checking logs, debugging, database access, etc.) using the new key.
如果轮换是因为Agent需要SSH访问进行故障排查而触发的,请使用新密钥恢复原始操作(查看日志、调试、数据库访问等)。

Workflow Inputs

工作流输入参数

The reusable rotation workflow (
rotate-ssh-key.yml@v1
) accepts:
InputTypeDefaultDescription
env_name
string
"preview"
Environment name (must match the deployed environment)
Required secrets:
SecretDescription
CLOUDSTACK_API_KEY
CloudStack API key
CLOUDSTACK_SECRET_KEY
CloudStack secret key
SSH_PRIVATE_KEY
The new SSH private key (already updated in Step 3)
可重用轮换工作流(
rotate-ssh-key.yml@v1
)接受以下参数:
输入参数类型默认值描述
env_name
string
"preview"
环境名称(必须与已部署的环境匹配)
必填的Secret:
Secret描述
CLOUDSTACK_API_KEY
CloudStack API密钥
CLOUDSTACK_SECRET_KEY
CloudStack密钥
SSH_PRIVATE_KEY
新的SSH私钥(已在步骤3中更新)

What the Rotation Does (Server-Side)

轮换操作的服务器端执行内容

  1. Verifies the SSH keypair and network exist in CloudStack (safety check)
  2. Deletes the old keypair from CloudStack and registers the new public key under the same name
  3. For each VM (accessories first, workers next, web last):
    • Stops the VM
    • Resets its SSH key via CloudStack API
    • Starts the VM
    • Connects via SSH with the new key and overwrites
      authorized_keys
      with only the new key
  4. Prints a summary of results
  1. 验证SSH密钥对和网络是否存在于CloudStack中(安全检查)
  2. 从CloudStack中删除旧密钥对,并以相同名称注册新公钥
  3. 针对每个VM(先处理附属服务,再处理工作节点,最后处理Web节点):
    • 停止VM
    • 通过CloudStack API重置其SSH密钥
    • 启动VM
    • 使用新密钥通过SSH连接,并将
      authorized_keys
      覆盖为仅包含新密钥
  4. 打印结果摘要

Key File Naming Convention

密钥文件命名规则

EnvironmentLocal key pathGitHub secret
preview (default)
~/.ssh/<repo-name>
SSH_PRIVATE_KEY
production
~/.ssh/<repo-name>-production
SSH_PRIVATE_KEY_PRODUCTION
other
<env_name>
~/.ssh/<repo-name>-<env_name>
SSH_PRIVATE_KEY_<ENV_NAME>
(uppercased)
环境本地密钥路径GitHub Secret
preview(默认)
~/.ssh/<repo-name>
SSH_PRIVATE_KEY
production
~/.ssh/<repo-name>-production
SSH_PRIVATE_KEY_PRODUCTION
其他
<env_name>
~/.ssh/<repo-name>-<env_name>
SSH_PRIVATE_KEY_<ENV_NAME>
(大写)