secrets-scanner

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Secrets Scanner

密钥扫描工具

Detect and prevent leaked credentials in your codebase.
检测并阻止代码库中的凭证泄露问题。

Secret Detection Patterns

密钥检测规则

yaml
undefined
yaml
undefined

.gitleaks.toml

.gitleaks.toml

title = "Gitleaks Configuration"
[[rules]] id = "aws-access-key" description = "AWS Access Key" regex = '''(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}''' tags = ["key", "AWS"]
[[rules]] id = "aws-secret-key" description = "AWS Secret Key" regex = '''(?i)aws(.{0,20})?(?-i)['"][0-9a-zA-Z/+]{40}['"]''' tags = ["key", "AWS"]
[[rules]] id = "github-token" description = "GitHub Personal Access Token" regex = '''ghp_[0-9a-zA-Z]{36}''' tags = ["key", "GitHub"]
[[rules]] id = "github-oauth" description = "GitHub OAuth Token" regex = '''gho_[0-9a-zA-Z]{36}''' tags = ["key", "GitHub"]
[[rules]] id = "slack-webhook" description = "Slack Webhook URL" regex = '''https://hooks\.slack\.com/services/T[a-zA-Z0-9_]{8,10}/B[a-zA-Z0-9_]{8,10}/[a-zA-Z0-9_]{24}''' tags = ["webhook", "Slack"]
[[rules]] id = "private-key" description = "Private Key" regex = '''-----BEGIN (RSA|OPENSSH|DSA|EC|PGP) PRIVATE KEY-----''' tags = ["key", "private"]
[[rules]] id = "generic-api-key" description = "Generic API Key" regex = '''(?i)(api[-]?key|apikey|access[-]?key)(.{0,20})?['"][0-9a-zA-Z]{32,}['"]''' tags = ["key", "generic"]
[[rules]] id = "database-connection" description = "Database Connection String" regex = '''(?i)(postgresql|mysql|mongodb)://[^\s:]+:[^\s@]+@[^\s/]+''' tags = ["database", "credentials"]
[allowlist] description = "Allowlist" paths = [ '''node_modules/''', '''.git/''', '''.lock$''', ]
regexes = [ '''EXAMPLE_KEY_123''', '''your_api_key_here''', '''<API_KEY>''', ]
title = "Gitleaks Configuration"
[[rules]] id = "aws-access-key" description = "AWS Access Key" regex = '''(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}''' tags = ["key", "AWS"]
[[rules]] id = "aws-secret-key" description = "AWS Secret Key" regex = '''(?i)aws(.{0,20})?(?-i)['"][0-9a-zA-Z/+]{40}['"]''' tags = ["key", "AWS"]
[[rules]] id = "github-token" description = "GitHub Personal Access Token" regex = '''ghp_[0-9a-zA-Z]{36}''' tags = ["key", "GitHub"]
[[rules]] id = "github-oauth" description = "GitHub OAuth Token" regex = '''gho_[0-9a-zA-Z]{36}''' tags = ["key", "GitHub"]
[[rules]] id = "slack-webhook" description = "Slack Webhook URL" regex = '''https://hooks\.slack\.com/services/T[a-zA-Z0-9_]{8,10}/B[a-zA-Z0-9_]{8,10}/[a-zA-Z0-9_]{24}''' tags = ["webhook", "Slack"]
[[rules]] id = "private-key" description = "Private Key" regex = '''-----BEGIN (RSA|OPENSSH|DSA|EC|PGP) PRIVATE KEY-----''' tags = ["key", "private"]
[[rules]] id = "generic-api-key" description = "Generic API Key" regex = '''(?i)(api[-]?key|apikey|access[-]?key)(.{0,20})?['"][0-9a-zA-Z]{32,}['"]''' tags = ["key", "generic"]
[[rules]] id = "database-connection" description = "Database Connection String" regex = '''(?i)(postgresql|mysql|mongodb)://[^\s:]+:[^\s@]+@[^\s/]+''' tags = ["database", "credentials"]
[allowlist] description = "Allowlist" paths = [ '''node_modules/''', '''.git/''', '''.lock$''', ]
regexes = [ '''EXAMPLE_KEY_123''', '''your_api_key_here''', '''<API_KEY>''', ]

Pre-commit Hook Setup

预提交钩子配置

yaml
undefined
yaml
undefined

.pre-commit-config.yaml

.pre-commit-config.yaml

repos:
  • repo: https://github.com/gitleaks/gitleaks rev: v8.18.0 hooks:
    • id: gitleaks
    • id: detect-secrets args: ['--baseline', '.secrets.baseline']
  • repo: local hooks:
    • id: check-env-files name: Check for .env files entry: bash -c 'if git diff --cached --name-only | grep -E ".env$"; then echo "❌ .env file detected! Add to .gitignore"; exit 1; fi' language: system pass_filenames: false

```bash
repos:
  • repo: https://github.com/gitleaks/gitleaks rev: v8.18.0 hooks:
    • id: gitleaks
    • id: detect-secrets args: ['--baseline', '.secrets.baseline']
  • repo: local hooks:
    • id: check-env-files name: Check for .env files entry: bash -c 'if git diff --cached --name-only | grep -E ".env$"; then echo "❌ .env file detected! Add to .gitignore"; exit 1; fi' language: system pass_filenames: false

```bash

Install pre-commit

Install pre-commit

pip install pre-commit
pip install pre-commit

Install hooks

Install hooks

pre-commit install
pre-commit install

Run on all files

Run on all files

pre-commit run --all-files
undefined
pre-commit run --all-files
undefined

CI Integration

CI集成配置

yaml
undefined
yaml
undefined

.github/workflows/secrets-scan.yml

.github/workflows/secrets-scan.yml

name: Secrets Scan
on: push: branches: [main, develop] pull_request: branches: [main, develop]
jobs: gitleaks: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: fetch-depth: 0 # Full history for scanning
  - name: Run Gitleaks
    uses: gitleaks/gitleaks-action@v2
    env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      GITLEAKS_LICENSE: ${{ secrets.GITLEAKS_LICENSE }}
trufflehog: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: fetch-depth: 0
  - name: TruffleHog OSS
    uses: trufflesecurity/trufflehog@main
    with:
      path: ./
      base: ${{ github.event.repository.default_branch }}
      head: HEAD
      extra_args: --debug --only-verified
undefined
name: Secrets Scan
on: push: branches: [main, develop] pull_request: branches: [main, develop]
jobs: gitleaks: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: fetch-depth: 0 # Full history for scanning
  - name: Run Gitleaks
    uses: gitleaks/gitleaks-action@v2
    env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      GITLEAKS_LICENSE: ${{ secrets.GITLEAKS_LICENSE }}
trufflehog: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: fetch-depth: 0
  - name: TruffleHog OSS
    uses: trufflesecurity/trufflehog@main
    with:
      path: ./
      base: ${{ github.event.repository.default_branch }}
      head: HEAD
      extra_args: --debug --only-verified
undefined

Custom Secret Scanner

自定义密钥扫描器

typescript
// scripts/scan-secrets.ts
import * as fs from "fs";
import * as path from "path";

interface SecretPattern {
  name: string;
  regex: RegExp;
  severity: "critical" | "high" | "medium";
}

const SECRET_PATTERNS: SecretPattern[] = [
  {
    name: "AWS Access Key",
    regex: /(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}/g,
    severity: "critical",
  },
  {
    name: "Private Key",
    regex: /-----BEGIN (RSA|OPENSSH|DSA|EC|PGP) PRIVATE KEY-----/g,
    severity: "critical",
  },
  {
    name: "Generic API Key",
    regex:
      /['"]?[a-zA-Z0-9_-]*api[_-]?key['"]?\s*[:=]\s*['"][a-zA-Z0-9]{32,}['"]/gi,
    severity: "high",
  },
  {
    name: "Database URL",
    regex: /(postgresql|mysql|mongodb):\/\/[^\s:]+:[^\s@]+@[^\s\/]+/gi,
    severity: "critical",
  },
  {
    name: "JWT Token",
    regex: /eyJ[a-zA-Z0-9_-]*\.eyJ[a-zA-Z0-9_-]*\.[a-zA-Z0-9_-]*/g,
    severity: "high",
  },
];

interface SecretFinding {
  file: string;
  line: number;
  column: number;
  pattern: string;
  match: string;
  severity: string;
}

function scanFile(filePath: string): SecretFinding[] {
  const findings: SecretFinding[] = [];
  const content = fs.readFileSync(filePath, "utf-8");
  const lines = content.split("\n");

  lines.forEach((line, lineIndex) => {
    SECRET_PATTERNS.forEach((pattern) => {
      const matches = line.matchAll(pattern.regex);

      for (const match of matches) {
        findings.push({
          file: filePath,
          line: lineIndex + 1,
          column: match.index || 0,
          pattern: pattern.name,
          match: match[0].substring(0, 50) + "...",
          severity: pattern.severity,
        });
      }
    });
  });

  return findings;
}

function scanDirectory(dir: string): SecretFinding[] {
  const findings: SecretFinding[] = [];
  const files = fs.readdirSync(dir, { withFileTypes: true });

  const ignorePaths = ["node_modules", ".git", "dist", "build"];

  files.forEach((file) => {
    const fullPath = path.join(dir, file.name);

    if (file.isDirectory() && !ignorePaths.includes(file.name)) {
      findings.push(...scanDirectory(fullPath));
    } else if (file.isFile()) {
      findings.push(...scanFile(fullPath));
    }
  });

  return findings;
}

// Run scan
const findings = scanDirectory("./src");

if (findings.length > 0) {
  console.error("🚨 Secrets detected!\n");

  findings.forEach((f) => {
    console.error(
      `[${f.severity.toUpperCase()}] ${f.file}:${f.line}:${f.column}`
    );
    console.error(`  Pattern: ${f.pattern}`);
    console.error(`  Match: ${f.match}\n`);
  });

  process.exit(1);
} else {
  console.log("✅ No secrets detected");
}
typescript
// scripts/scan-secrets.ts
import * as fs from "fs";
import * as path from "path";

interface SecretPattern {
  name: string;
  regex: RegExp;
  severity: "critical" | "high" | "medium";
}

const SECRET_PATTERNS: SecretPattern[] = [
  {
    name: "AWS Access Key",
    regex: /(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}/g,
    severity: "critical",
  },
  {
    name: "Private Key",
    regex: /-----BEGIN (RSA|OPENSSH|DSA|EC|PGP) PRIVATE KEY-----/g,
    severity: "critical",
  },
  {
    name: "Generic API Key",
    regex:
      /['"]?[a-zA-Z0-9_-]*api[_-]?key['"]?\s*[:=]\s*['"][a-zA-Z0-9]{32,}['"]/gi,
    severity: "high",
  },
  {
    name: "Database URL",
    regex: /(postgresql|mysql|mongodb):\/\/[^\s:]+:[^\s@]+@[^\s\/]+/gi,
    severity: "critical",
  },
  {
    name: "JWT Token",
    regex: /eyJ[a-zA-Z0-9_-]*\.eyJ[a-zA-Z0-9_-]*\.[a-zA-Z0-9_-]*/g,
    severity: "high",
  },
];

interface SecretFinding {
  file: string;
  line: number;
  column: number;
  pattern: string;
  match: string;
  severity: string;
}

function scanFile(filePath: string): SecretFinding[] {
  const findings: SecretFinding[] = [];
  const content = fs.readFileSync(filePath, "utf-8");
  const lines = content.split("\n");

  lines.forEach((line, lineIndex) => {
    SECRET_PATTERNS.forEach((pattern) => {
      const matches = line.matchAll(pattern.regex);

      for (const match of matches) {
        findings.push({
          file: filePath,
          line: lineIndex + 1,
          column: match.index || 0,
          pattern: pattern.name,
          match: match[0].substring(0, 50) + "...",
          severity: pattern.severity,
        });
      }
    });
  });

  return findings;
}

function scanDirectory(dir: string): SecretFinding[] {
  const findings: SecretFinding[] = [];
  const files = fs.readdirSync(dir, { withFileTypes: true });

  const ignorePaths = ["node_modules", ".git", "dist", "build"];

  files.forEach((file) => {
    const fullPath = path.join(dir, file.name);

    if (file.isDirectory() && !ignorePaths.includes(file.name)) {
      findings.push(...scanDirectory(fullPath));
    } else if (file.isFile()) {
      findings.push(...scanFile(fullPath));
    }
  });

  return findings;
}

// Run scan
const findings = scanDirectory("./src");

if (findings.length > 0) {
  console.error("🚨 Secrets detected!\n");

  findings.forEach((f) => {
    console.error(
      `[${f.severity.toUpperCase()}] ${f.file}:${f.line}:${f.column}`
    );
    console.error(`  Pattern: ${f.pattern}`);
    console.error(`  Match: ${f.match}\n`);
  });

  process.exit(1);
} else {
  console.log("✅ No secrets detected");
}

Remediation Steps

泄露修复步骤

markdown
undefined
markdown
undefined

Secret Leak Remediation Checklist

密钥泄露修复清单

Immediate Actions (< 1 hour)

紧急处理(< 1小时)

  1. Revoke the compromised secret
    • Deactivate API key/token immediately
    • Rotate credentials in production
    • Update all services using the secret
  2. Remove from git history
    bash
    # Using BFG Repo-Cleaner
    bfg --replace-text secrets.txt repo.git
    git reflog expire --expire=now --all
    git gc --prune=now --aggressive
    
    # Force push (requires team coordination)
    git push --force --all

3. **Notify stakeholders**
   - [ ] Security team
   - [ ] DevOps team
   - [ ] Service owners
   - [ ] Management (if public repo)
  1. 吊销泄露的密钥
    • 立即停用API密钥/令牌
    • 轮换生产环境凭证
    • 更新所有使用该密钥的服务
  2. 从Git历史中移除
    bash
    # Using BFG Repo-Cleaner
    bfg --replace-text secrets.txt repo.git
    git reflog expire --expire=now --all
    git gc --prune=now --aggressive
    
    # Force push (requires team coordination)
    git push --force --all

3. **通知相关人员**
   - [ ] 安全团队
   - [ ] DevOps团队
   - [ ] 服务负责人
   - [ ] 管理层(若为公开仓库)

Short-term Actions (< 24 hours)

短期处理(< 24小时)

  1. Audit access logs
    • Check CloudWatch/CloudTrail for suspicious activity
    • Review API usage for unauthorized access
    • Check for data exfiltration
  2. Update secret management
    • Store in vault (AWS Secrets Manager, HashiCorp Vault)
    • Use environment variables
    • Remove hardcoded secrets
  3. Add scanning
    • Install pre-commit hooks
    • Add CI secret scanning
    • Set up monitoring alerts
  1. 审计访问日志
    • 检查CloudWatch/CloudTrail中的异常活动
    • 审查API使用记录,排查未授权访问
    • 检查是否存在数据泄露
  2. 优化密钥管理
    • 存储至密钥管理服务(AWS Secrets Manager、HashiCorp Vault)
    • 使用环境变量传递密钥
    • 移除硬编码的密钥
  3. 配置持续扫描
    • 安装预提交钩子
    • 配置CI密钥扫描
    • 设置监控告警

Long-term Actions (< 1 week)

长期优化(< 1周)

  1. Review and improve
    • Conduct security training
    • Update secret management policies
    • Implement secret rotation schedule
    • Document incident and lessons learned
undefined
  1. 复盘与改进
    • 开展安全培训
    • 更新密钥管理政策
    • 实现密钥定期轮换机制
    • 记录事件与经验教训
undefined

Secret Management Best Practices

密钥管理最佳实践

typescript
// ❌ BAD: Hardcoded secrets
const API_KEY = 'sk_live_abc123xyz789';
const db = connect('mongodb://admin:password@localhost');

// ✅ GOOD: Environment variables
const API_KEY = process.env.API_KEY;
const db = connect(process.env.DATABASE_URL);

// ✅ BETTER: Secret management service
import { SecretsManagerClient, GetSecretValueCommand } from '@aws-sdk/client-secrets-manager';

async function getSecret(secretName: string): Promise<string> {
  const client = new SecretsManagerClient({ region: 'us-east-1' });
  const response = await client.send(
    new GetSecretValueCommand({ SecretId: secretName })
  );
  return response.SecretString!;
}

const apiKey = await getSecret('prod/api/stripe-key');
typescript
// ❌ BAD: Hardcoded secrets
const API_KEY = 'sk_live_abc123xyz789';
const db = connect('mongodb://admin:password@localhost');

// ✅ GOOD: Environment variables
const API_KEY = process.env.API_KEY;
const db = connect(process.env.DATABASE_URL);

// ✅ BETTER: Secret management service
import { SecretsManagerClient, GetSecretValueCommand } from '@aws-sdk/client-secrets-manager';

async function getSecret(secretName: string): Promise<string> {
  const client = new SecretsManagerClient({ region: 'us-east-1' });
  const response = await client.send(
    new GetSecretValueCommand({ SecretId: secretName })
  );
  return response.SecretString!;
}

const apiKey = await getSecret('prod/api/stripe-key');

GitHub Secret Scanning

GitHub密钥扫描

yaml
undefined
yaml
undefined

Enable GitHub secret scanning (Enterprise)

Enable GitHub secret scanning (Enterprise)

Settings → Security & analysis → Secret scanning

Settings → Security & analysis → Secret scanning

Configure custom patterns

Configure custom patterns

.github/secret_scanning.yml

.github/secret_scanning.yml

patterns:
  • name: Company API Key pattern: "company_[a-zA-Z0-9]{32}" secret_type: company_api_key
undefined
patterns:
  • name: Company API Key pattern: "company_[a-zA-Z0-9]{32}" secret_type: company_api_key
undefined

Environment Variable Validation

环境变量验证

typescript
// config/env-validation.ts
import { z } from "zod";

const envSchema = z
  .object({
    NODE_ENV: z.enum(["development", "production", "test"]),
    DATABASE_URL: z.string().url(),
    API_KEY: z.string().min(32),
    JWT_SECRET: z.string().min(64),
    // Never allow default/example values in production
  })
  .refine((env) => {
    if (env.NODE_ENV === "production") {
      const invalidValues = ["example", "test", "localhost", "changeme"];
      return !invalidValues.some((val) =>
        Object.values(env).some((envVal) =>
          String(envVal).toLowerCase().includes(val)
        )
      );
    }
    return true;
  }, "Production environment cannot use example/test values");

// Validate on startup
try {
  envSchema.parse(process.env);
} catch (error) {
  console.error("❌ Invalid environment configuration:", error);
  process.exit(1);
}
typescript
// config/env-validation.ts
import { z } from "zod";

const envSchema = z
  .object({
    NODE_ENV: z.enum(["development", "production", "test"]),
    DATABASE_URL: z.string().url(),
    API_KEY: z.string().min(32),
    JWT_SECRET: z.string().min(64),
    // Never allow default/example values in production
  })
  .refine((env) => {
    if (env.NODE_ENV === "production") {
      const invalidValues = ["example", "test", "localhost", "changeme"];
      return !invalidValues.some((val) =>
        Object.values(env).some((envVal) =>
          String(envVal).toLowerCase().includes(val)
        )
      );
    }
    return true;
  }, "Production environment cannot use example/test values");

// Validate on startup
try {
  envSchema.parse(process.env);
} catch (error) {
  console.error("❌ Invalid environment configuration:", error);
  process.exit(1);
}

Monitoring & Alerts

监控与告警

typescript
// monitoring/secret-monitoring.ts
import {
  CloudWatchClient,
  PutMetricDataCommand,
} from "@aws-sdk/client-cloudwatch";

async function monitorSecretUsage(secretName: string) {
  const cloudwatch = new CloudWatchClient();

  await cloudwatch.send(
    new PutMetricDataCommand({
      Namespace: "Security/Secrets",
      MetricData: [
        {
          MetricName: "SecretAccess",
          Value: 1,
          Unit: "Count",
          Dimensions: [
            {
              Name: "SecretName",
              Value: secretName,
            },
          ],
        },
      ],
    })
  );
}

// Alert on unusual secret access patterns
typescript
// monitoring/secret-monitoring.ts
import {
  CloudWatchClient,
  PutMetricDataCommand,
} from "@aws-sdk/client-cloudwatch";

async function monitorSecretUsage(secretName: string) {
  const cloudwatch = new CloudWatchClient();

  await cloudwatch.send(
    new PutMetricDataCommand({
      Namespace: "Security/Secrets",
      MetricData: [
        {
          MetricName: "SecretAccess",
          Value: 1,
          Unit: "Count",
          Dimensions: [
            {
              Name: "SecretName",
              Value: secretName,
            },
          ],
        },
      ],
    })
  );
}

// Alert on unusual secret access patterns

Best Practices

最佳实践

  1. Never commit secrets: Use .gitignore for .env files
  2. Use secret managers: AWS Secrets Manager, Vault
  3. Rotate regularly: 90-day rotation policy
  4. Scan continuously: Pre-commit + CI + scheduled scans
  5. Least privilege: Minimal secret access
  6. Audit logs: Track secret access
  7. Incident response: Have remediation playbook ready
  1. 切勿提交密钥:将.env文件加入.gitignore
  2. 使用密钥管理工具:如AWS Secrets Manager、Vault
  3. 定期轮换:执行90天轮换策略
  4. 持续扫描:结合预提交钩子、CI及定时扫描
  5. 最小权限原则:限制密钥访问范围
  6. 审计日志:记录密钥访问情况
  7. 事件响应:准备好修复预案

Output Checklist

落地检查清单

  • Gitleaks configuration created
  • Pre-commit hooks installed
  • CI secret scanning configured
  • Custom scanner implemented (optional)
  • Remediation playbook documented
  • Secret management best practices
  • Environment validation
  • Monitoring and alerts
  • .gitignore includes .env files
  • Team trained on secret handling
  • 已创建Gitleaks配置
  • 已安装预提交钩子
  • 已配置CI密钥扫描
  • 已实现自定义扫描器(可选)
  • 已编写修复预案文档
  • 已推行密钥管理最佳实践
  • 已配置环境变量验证
  • 已设置监控与告警
  • .gitignore已包含.env文件
  • 已完成团队密钥处理培训