Loading...
Loading...
Secure AI coding agents (Claude Code, Cursor, Codex, Copilot) with permission boundaries, secret protection, code review gates, and safe sandbox configurations for team environments.
npx skill4agent add bagelhole/devops-security-agent-skills ai-coding-agent-guardrailsCLAUDE.md# CLAUDE.md
## Restrictions
- NEVER read or output contents of .env, .env.*, secrets.yaml, or any file matching *.pem, *.key
- NEVER execute `rm -rf`, `DROP TABLE`, `kubectl delete`, or `terraform destroy` commands
- NEVER push directly to main or master branches
- NEVER modify files in the infrastructure/, terraform/, or .github/workflows/ directories without explicit user approval
- NEVER install new dependencies without listing them first for review
- NEVER access or display API keys, tokens, passwords, or connection strings
## Allowed Operations
- Read and modify application source code in src/, lib/, and tests/
- Run test suites with `npm test`, `pytest`, `go test`
- Run linters with `eslint`, `ruff`, `golangci-lint`
- Create new branches with prefix `ai/` or `agent/`
- Create and modify files in docs/ directory
## Code Standards
- All new functions must include docstrings or JSDoc comments
- All new code must have corresponding unit tests
- Follow existing code style and patterns in the repository
- Maximum file length: 500 lines. Suggest splitting if exceeded.# .agent-permissions.yaml
agent_permissions:
allowed_commands:
- "npm test"
- "npm run lint"
- "npm run build"
- "pytest"
- "ruff check"
- "go test ./..."
- "git status"
- "git diff"
- "git log"
- "git checkout -b"
- "git add"
- "git commit"
- "ls"
- "cat"
- "head"
- "tail"
blocked_commands:
- "rm -rf"
- "curl"
- "wget"
- "ssh"
- "scp"
- "kubectl"
- "terraform"
- "aws"
- "gcloud"
- "az"
- "docker push"
- "npm publish"
blocked_paths:
- ".env*"
- "**/*.pem"
- "**/*.key"
- "**/secrets/**"
- "infrastructure/**"
- ".github/workflows/**"
allowed_paths:
- "src/**"
- "lib/**"
- "tests/**"
- "docs/**"
- "package.json"
- "pyproject.toml"#!/bin/bash
# setup-agent-workspace.sh
# Create a restricted workspace for agent execution
AGENT_USER="ai-agent"
REPO_DIR="/workspace/repo"
# Create agent user with limited permissions
useradd --system --shell /bin/bash --no-create-home "$AGENT_USER"
# Set ownership: developers own everything, agent gets read on most
chown -R root:developers "$REPO_DIR"
chmod -R 750 "$REPO_DIR"
# Grant agent write access only to safe directories
setfacl -R -m u:${AGENT_USER}:rwx "${REPO_DIR}/src"
setfacl -R -m u:${AGENT_USER}:rwx "${REPO_DIR}/tests"
setfacl -R -m u:${AGENT_USER}:rwx "${REPO_DIR}/docs"
# Deny agent access to sensitive files
setfacl -m u:${AGENT_USER}:--- "${REPO_DIR}/.env"
setfacl -R -m u:${AGENT_USER}:--- "${REPO_DIR}/infrastructure"
setfacl -R -m u:${AGENT_USER}:--- "${REPO_DIR}/.github/workflows"
echo "Agent workspace permissions configured."#!/bin/bash
# install-secret-scanning.sh
# Install git-secrets
git clone https://github.com/awslabs/git-secrets.git /tmp/git-secrets
cd /tmp/git-secrets && make install
# Initialize in repository
cd /path/to/repo
git secrets --install
# Register common secret patterns
git secrets --register-aws
# Add custom patterns for common credential formats
git secrets --add '-----BEGIN (RSA |EC |DSA )?PRIVATE KEY-----'
git secrets --add 'AKIA[0-9A-Z]{16}'
git secrets --add 'ghp_[a-zA-Z0-9]{36}'
git secrets --add 'sk-[a-zA-Z0-9]{48}'
git secrets --add 'xox[baprs]-[0-9a-zA-Z-]{10,}'
git secrets --add 'password\s*[:=]\s*["\x27][^\s]{8,}'
git secrets --add 'api[_-]?key\s*[:=]\s*["\x27][^\s]{8,}'
# Add allowed patterns (false positive exclusions)
git secrets --add --allowed 'EXAMPLE_KEY'
git secrets --add --allowed 'your-api-key-here'#!/usr/bin/env python3
"""scan_agent_output.py - Scan AI agent output for leaked secrets."""
import re
import sys
from pathlib import Path
SECRET_PATTERNS = [
(r'AKIA[0-9A-Z]{16}', 'AWS Access Key'),
(r'(?i)aws_secret_access_key\s*[:=]\s*\S+', 'AWS Secret Key'),
(r'ghp_[a-zA-Z0-9]{36}', 'GitHub Personal Access Token'),
(r'gho_[a-zA-Z0-9]{36}', 'GitHub OAuth Token'),
(r'sk-[a-zA-Z0-9]{48,}', 'OpenAI/Anthropic API Key'),
(r'xox[baprs]-[0-9a-zA-Z\-]{10,}', 'Slack Token'),
(r'-----BEGIN (RSA |EC |DSA )?PRIVATE KEY-----', 'Private Key'),
(r'(?i)(password|passwd|pwd)\s*[:=]\s*["\x27][^\s]{4,}', 'Hardcoded Password'),
(r'(?i)(api[_-]?key|apikey)\s*[:=]\s*["\x27][^\s]{8,}', 'API Key'),
(r'eyJ[a-zA-Z0-9_-]{10,}\.[a-zA-Z0-9_-]{10,}', 'JWT Token'),
(r'(?i)database_url\s*[:=]\s*\S+', 'Database Connection String'),
]
def scan_file(filepath: str) -> list[dict]:
findings = []
content = Path(filepath).read_text(errors="ignore")
for line_num, line in enumerate(content.splitlines(), 1):
for pattern, label in SECRET_PATTERNS:
if re.search(pattern, line):
findings.append({
"file": filepath,
"line": line_num,
"type": label,
"content": line.strip()[:120],
})
return findings
def main():
files = sys.argv[1:]
if not files:
print("Usage: scan_agent_output.py <file1> [file2] ...")
sys.exit(1)
all_findings = []
for f in files:
all_findings.extend(scan_file(f))
if all_findings:
print(f"BLOCKED: {len(all_findings)} potential secret(s) detected:\n")
for finding in all_findings:
print(f" [{finding['type']}] {finding['file']}:{finding['line']}")
print(f" {finding['content']}\n")
sys.exit(1)
print("OK: No secrets detected in agent output.")
sys.exit(0)
if __name__ == "__main__":
main()#!/bin/bash
# .git/hooks/pre-commit
# Block commits containing secrets from AI agents
STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACM)
if [ -z "$STAGED_FILES" ]; then
exit 0
fi
echo "Scanning staged files for secrets..."
# Run git-secrets
git secrets --pre_commit_hook -- "$@"
GIT_SECRETS_EXIT=$?
# Run custom scanner on staged files
python3 .tools/scan_agent_output.py $STAGED_FILES
SCANNER_EXIT=$?
if [ $GIT_SECRETS_EXIT -ne 0 ] || [ $SCANNER_EXIT -ne 0 ]; then
echo ""
echo "COMMIT BLOCKED: Secrets detected in staged files."
echo "If this is a false positive, use: git commit --no-verify"
exit 1
fi# Dockerfile.agent-sandbox
FROM ubuntu:24.04
RUN apt-get update && apt-get install -y \
git \
nodejs \
npm \
python3 \
python3-pip \
&& rm -rf /var/lib/apt/lists/*
# Create non-root agent user
RUN useradd -m -s /bin/bash agent && \
mkdir -p /workspace && \
chown agent:agent /workspace
# Drop capabilities
USER agent
WORKDIR /workspace
# No network by default - override at runtime if needed
# No access to Docker socket
# No access to host filesystem beyond mounted volume#!/bin/bash
# run-agent-sandbox.sh
# Launch an AI coding agent inside a locked-down container
REPO_DIR="$(pwd)"
CONTAINER_NAME="agent-sandbox-$$"
docker run \
--name "$CONTAINER_NAME" \
--rm \
--network none \
--read-only \
--tmpfs /tmp:size=512m \
--tmpfs /home/agent:size=256m \
--memory 4g \
--cpus 2 \
--pids-limit 256 \
--security-opt no-new-privileges:true \
--security-opt seccomp=seccomp-agent.json \
--cap-drop ALL \
--cap-add DAC_OVERRIDE \
-v "${REPO_DIR}/src:/workspace/src" \
-v "${REPO_DIR}/tests:/workspace/tests:rw" \
-v "${REPO_DIR}/docs:/workspace/docs:rw" \
-v "${REPO_DIR}/package.json:/workspace/package.json:ro" \
-e "NO_COLOR=1" \
agent-sandbox:latest \
"$@"{
"defaultAction": "SCMP_ACT_ERRNO",
"comment": "seccomp-agent.json - Restrictive profile for AI coding agents",
"syscalls": [
{
"names": [
"read", "write", "open", "close", "stat", "fstat", "lstat",
"poll", "lseek", "mmap", "mprotect", "munmap", "brk",
"access", "pipe", "select", "sched_yield", "mremap",
"dup", "dup2", "nanosleep", "getpid", "getuid", "getgid",
"geteuid", "getegid", "getppid", "getpgrp", "setsid",
"getgroups", "uname", "fcntl", "flock", "fsync",
"getcwd", "chdir", "readlink", "chmod", "mkdir",
"rmdir", "unlink", "rename", "symlink", "readlinkat",
"openat", "mkdirat", "newfstatat", "unlinkat", "renameat",
"faccessat", "pselect6", "ppoll", "set_robust_list",
"get_robust_list", "epoll_create1", "epoll_ctl", "epoll_wait",
"eventfd2", "pipe2", "dup3", "pread64", "pwrite64",
"futex", "clock_gettime", "clock_getres", "exit_group",
"wait4", "clone", "execve", "arch_prctl", "set_tid_address",
"exit", "getdents64", "rt_sigaction", "rt_sigprocmask",
"rt_sigreturn", "ioctl", "writev", "madvise", "getrandom"
],
"action": "SCMP_ACT_ALLOW"
},
{
"names": [
"socket", "connect", "bind", "listen", "accept",
"sendto", "recvfrom", "sendmsg", "recvmsg"
],
"action": "SCMP_ACT_ERRNO",
"comment": "Block all network syscalls"
},
{
"names": ["ptrace", "process_vm_readv", "process_vm_writev"],
"action": "SCMP_ACT_ERRNO",
"comment": "Block debugging and process inspection"
}
]
}# .github/workflows/agent-pr-review.yaml
name: Agent PR Security Review
on:
pull_request:
types: [opened, synchronize]
jobs:
detect-agent-pr:
runs-on: ubuntu-latest
outputs:
is_agent: ${{ steps.check.outputs.is_agent }}
steps:
- name: Check if PR is from an AI agent
id: check
run: |
BRANCH="${{ github.head_ref }}"
AUTHOR="${{ github.event.pull_request.user.login }}"
BODY="${{ github.event.pull_request.body }}"
IS_AGENT="false"
if [[ "$BRANCH" == ai/* ]] || [[ "$BRANCH" == agent/* ]]; then
IS_AGENT="true"
fi
if echo "$BODY" | grep -qi "generated with.*claude\|generated by.*copilot\|generated by.*cursor\|generated by.*codex"; then
IS_AGENT="true"
fi
if [[ "$AUTHOR" == *"bot"* ]] || [[ "$AUTHOR" == *"agent"* ]]; then
IS_AGENT="true"
fi
echo "is_agent=$IS_AGENT" >> "$GITHUB_OUTPUT"
security-scan:
needs: detect-agent-pr
if: needs.detect-agent-pr.outputs.is_agent == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Run Semgrep security scan
uses: semgrep/semgrep-action@v1
with:
config: >-
p/default
p/owasp-top-ten
p/command-injection
p/sql-injection
p/xss
- name: Scan for secrets with Trufflehog
uses: trufflesecurity/trufflehog@main
with:
extra_args: --only-verified
- name: Check for dependency changes
run: |
CHANGED_FILES=$(git diff --name-only origin/main...HEAD)
DEP_FILES="package.json package-lock.json requirements.txt Pipfile.lock go.sum Cargo.lock"
for dep_file in $DEP_FILES; do
if echo "$CHANGED_FILES" | grep -q "$dep_file"; then
echo "::warning::Agent modified dependency file: $dep_file"
echo "DEPENDENCY_CHANGED=true" >> "$GITHUB_ENV"
fi
done
- name: Require extra review for dependency changes
if: env.DEPENDENCY_CHANGED == 'true'
run: |
gh pr edit "${{ github.event.pull_request.number }}" \
--add-label "agent-dependency-change" \
--add-label "requires-security-review"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
enforce-review:
needs: detect-agent-pr
if: needs.detect-agent-pr.outputs.is_agent == 'true'
runs-on: ubuntu-latest
steps:
- name: Label as agent-generated
run: |
gh pr edit "${{ github.event.pull_request.number }}" \
--add-label "ai-generated"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Enforce minimum reviewers
run: |
echo "Agent-generated PR detected."
echo "This PR requires at least 2 human approvals before merge."#!/bin/bash
# configure-branch-protection.sh
# Set up branch protection rules for agent-generated PRs via GitHub API
OWNER="your-org"
REPO="your-repo"
gh api repos/${OWNER}/${REPO}/rulesets \
--method POST \
--input - <<'EOF'
{
"name": "Agent Branch Protection",
"target": "branch",
"enforcement": "active",
"conditions": {
"ref_name": {
"include": ["refs/heads/ai/*", "refs/heads/agent/*"],
"exclude": []
}
},
"rules": [
{
"type": "pull_request",
"parameters": {
"required_approving_review_count": 2,
"dismiss_stale_reviews_on_push": true,
"require_code_owner_review": true,
"require_last_push_approval": true
}
},
{
"type": "required_status_checks",
"parameters": {
"required_status_checks": [
{ "context": "security-scan" },
{ "context": "test-suite" },
{ "context": "secret-detection" }
],
"strict_required_status_checks_policy": true
}
}
]
}
EOF# .cursorrules
You are working in a production codebase. Follow these rules strictly:
## Security Rules
- Never hardcode secrets, API keys, tokens, or passwords in source code.
- Never read or display the contents of .env files or secret configuration files.
- Never disable SSL verification, CSRF protection, or authentication middleware.
- Never use eval(), exec(), or similar dynamic code execution functions.
- Never introduce SQL string concatenation; always use parameterized queries.
## File Restrictions
- Do not modify any files in: infrastructure/, terraform/, .github/workflows/, deploy/
- Do not create or modify Dockerfiles without explicit approval.
- Do not modify CI/CD configuration files.
## Code Quality
- Every new function must have a corresponding unit test.
- All error handling must be explicit; never silently swallow exceptions.
- Follow existing patterns for logging, error handling, and API responses.
- Maximum function length: 50 lines. Refactor if exceeded.
## Git Behavior
- Create branches with the prefix: ai/
- Write descriptive commit messages referencing the task or issue.
- Never force push or rebase shared branches.# .github/copilot-config.yml
#
# Note: Copilot content exclusion is configured at the org/repo level
# via GitHub settings. This file documents intended exclusions and
# can be referenced by org admins when configuring the settings at
# github.com > Org Settings > Copilot > Content Exclusions.
content_exclusions:
paths:
- "**/.env*"
- "**/secrets/**"
- "**/*.pem"
- "**/*.key"
- "**/infrastructure/**"
- "**/terraform/**"
- "**/.aws/**"
- "**/credentials*"
# Copilot content exclusion via organization settings (recommended):
# 1. Go to Organization Settings > Copilot > Content exclusion
# 2. Add repository paths:
# - ".env*"
# - "secrets/**"
# - "infrastructure/**"
# - "*.pem"
# - "*.key"<!-- codex.md or AGENTS.md for Codex CLI -->
## Rules
- Do not modify files outside of src/ and tests/ directories.
- Do not install packages or modify dependency files without listing changes first.
- Run `npm test` after every code change to verify nothing is broken.
- Never access network resources or make HTTP requests during code generation.
- All generated code must include error handling.
- Prefix all branch names with `agent/codex/`.#!/bin/bash
# agent-network-controls.sh
# Restrict network access for agent execution environments
# Create a dedicated chain for agent traffic
iptables -N AGENT_CHAIN
# Allow DNS resolution
iptables -A AGENT_CHAIN -p udp --dport 53 -j ACCEPT
iptables -A AGENT_CHAIN -p tcp --dport 53 -j ACCEPT
# Allow package registries
iptables -A AGENT_CHAIN -d registry.npmjs.org -p tcp --dport 443 -j ACCEPT
iptables -A AGENT_CHAIN -d pypi.org -p tcp --dport 443 -j ACCEPT
iptables -A AGENT_CHAIN -d files.pythonhosted.org -p tcp --dport 443 -j ACCEPT
iptables -A AGENT_CHAIN -d proxy.golang.org -p tcp --dport 443 -j ACCEPT
# Allow GitHub for git operations
iptables -A AGENT_CHAIN -d github.com -p tcp --dport 443 -j ACCEPT
iptables -A AGENT_CHAIN -d github.com -p tcp --dport 22 -j ACCEPT
# Block everything else
iptables -A AGENT_CHAIN -j DROP
# Apply to agent user
iptables -A OUTPUT -m owner --uid-owner ai-agent -j AGENT_CHAIN# /etc/squid/squid-agent.conf
# Transparent proxy for AI agent network requests
acl agent_user proxy_auth ai-agent
acl allowed_domains dstdomain .npmjs.org .pypi.org .github.com .golang.org
# Allow only specific domains
http_access allow agent_user allowed_domains
http_access deny agent_user
# Log all agent requests for auditing
access_log /var/log/squid/agent-access.log squid
log_mime_hdrs on
# Request size limits
request_body_max_size 10 MB
reply_body_max_size 50 MB
http_port 3128# docker-compose.agent.yaml
version: "3.8"
services:
agent-sandbox:
build:
context: .
dockerfile: Dockerfile.agent-sandbox
networks:
- agent-restricted
volumes:
- ./src:/workspace/src
- ./tests:/workspace/tests:rw
mem_limit: 4g
cpus: 2
pids_limit: 256
security_opt:
- no-new-privileges:true
read_only: true
tmpfs:
- /tmp:size=512m
agent-proxy:
image: ubuntu/squid:latest
networks:
- agent-restricted
- external
volumes:
- ./squid-agent.conf:/etc/squid/squid.conf:ro
ports:
- "3128:3128"
networks:
agent-restricted:
internal: true # No external access from this network
external:
driver: bridge#!/bin/bash
# git-ai-commit.sh
# Wrapper for committing agent-generated code with proper attribution
AGENT_NAME="${AI_AGENT_NAME:-unknown-agent}"
AGENT_VERSION="${AI_AGENT_VERSION:-unknown}"
TASK_ID="${AI_TASK_ID:-none}"
git commit -m "$(cat <<EOF
$1
AI-Generated-By: ${AGENT_NAME} ${AGENT_VERSION}
AI-Task-ID: ${TASK_ID}
AI-Reviewed-By: pending
Co-Authored-By: ${AGENT_NAME} <noreply@${AGENT_NAME}.ai>
EOF
)"#!/usr/bin/env python3
"""agent_audit_logger.py - Log all AI agent actions for compliance."""
import json
import logging
import os
import time
from datetime import datetime, timezone
from pathlib import Path
LOG_DIR = Path(os.environ.get("AGENT_LOG_DIR", "/var/log/ai-agents"))
LOG_DIR.mkdir(parents=True, exist_ok=True)
logger = logging.getLogger("agent_audit")
handler = logging.FileHandler(LOG_DIR / "agent-actions.jsonl")
handler.setFormatter(logging.Formatter("%(message)s"))
logger.addHandler(handler)
logger.setLevel(logging.INFO)
def log_action(
agent: str,
action: str,
target: str,
details: dict | None = None,
user: str = "system",
) -> None:
entry = {
"timestamp": datetime.now(timezone.utc).isoformat(),
"agent": agent,
"user": user,
"action": action,
"target": target,
"details": details or {},
"session_id": os.environ.get("AGENT_SESSION_ID", "unknown"),
}
logger.info(json.dumps(entry))
def log_file_read(agent: str, filepath: str) -> None:
log_action(agent, "file_read", filepath)
def log_file_write(agent: str, filepath: str, lines_changed: int) -> None:
log_action(agent, "file_write", filepath, {"lines_changed": lines_changed})
def log_command(agent: str, command: str, exit_code: int) -> None:
log_action(agent, "command_exec", command, {"exit_code": exit_code})
def log_pr_created(agent: str, pr_url: str, files_changed: list[str]) -> None:
log_action(agent, "pr_created", pr_url, {"files_changed": files_changed})
# Usage example:
# log_file_write("claude-code", "src/api/handler.py", 42)
# log_command("claude-code", "npm test", 0)
# log_pr_created("claude-code", "https://github.com/org/repo/pull/99", ["src/main.py"])#!/bin/bash
# query-agent-audit.sh
# Query agent audit logs for compliance reporting
LOG_FILE="/var/log/ai-agents/agent-actions.jsonl"
echo "=== Agent Activity Summary ==="
echo ""
echo "Actions by agent (last 24h):"
jq -r 'select(.timestamp > (now - 86400 | todate)) | .agent' "$LOG_FILE" \
| sort | uniq -c | sort -rn
echo ""
echo "File writes by agent:"
jq -r 'select(.action == "file_write") | "\(.agent) -> \(.target)"' "$LOG_FILE" \
| sort | uniq -c | sort -rn
echo ""
echo "Commands executed:"
jq -r 'select(.action == "command_exec") | "\(.agent): \(.target) (exit: \(.details.exit_code))"' "$LOG_FILE" \
| tail -20
echo ""
echo "PRs created by agents:"
jq -r 'select(.action == "pr_created") | "\(.agent): \(.target)"' "$LOG_FILE"# .github/agent-policy.yaml
# Team policy for AI coding agent usage
policy:
version: "1.0"
last_updated: "2025-06-01"
general:
- All developers may use AI coding agents for code generation
- Agent-generated code has the same quality and security bar as human code
- Developers are responsible for all code they submit, regardless of origin
required_review:
standard_code:
min_reviewers: 1
agent_generated: 2
infrastructure_changes:
min_reviewers: 2
agent_generated: "blocked" # Agents may not modify infra
security_sensitive:
min_reviewers: 2
requires: ["security-team-member"]
agent_generated: 2
additional_requires: ["security-team-lead"]
escalation:
- type: "dependency_addition"
action: "require security review"
- type: "auth_or_crypto_changes"
action: "require security team approval"
- type: "ci_cd_changes"
action: "blocked for agents"
- type: "database_migration"
action: "require DBA review"
- type: "api_contract_change"
action: "require API owner approval"
allowed_use_cases:
- "Writing unit and integration tests"
- "Implementing well-specified features with clear requirements"
- "Refactoring code with existing test coverage"
- "Writing documentation and code comments"
- "Fixing linter warnings and code style issues"
- "Generating boilerplate code from templates"
prohibited_use_cases:
- "Modifying authentication or authorization logic"
- "Writing or changing cryptographic implementations"
- "Modifying CI/CD pipelines or deployment configs"
- "Changing infrastructure-as-code without human authorship"
- "Accessing production databases or systems"
- "Modifying security controls or audit logging"# .github/CODEOWNERS
# Require specific reviewers for agent-sensitive areas
# All agent-generated branches require security team review
# (enforced via branch protection rules for ai/* and agent/* branches)
# Infrastructure is off-limits to agents and requires platform team
/infrastructure/ @platform-team
/terraform/ @platform-team
/.github/workflows/ @platform-team @security-team
# Security-sensitive code requires security team
/src/auth/ @security-team
/src/crypto/ @security-team
/src/middleware/auth* @security-team
# Dependency files require security review
package.json @security-team @tech-leads
package-lock.json @security-team
requirements.txt @security-team @tech-leads
go.sum @security-team# .github/workflows/agent-test-gate.yaml
name: Agent Code Test Gate
on:
pull_request:
types: [opened, synchronize]
jobs:
test-coverage:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Detect changed files
id: changes
run: |
FILES=$(git diff --name-only origin/main...HEAD -- '*.py' '*.js' '*.ts' '*.go')
echo "changed_files=$FILES" >> "$GITHUB_OUTPUT"
- name: Run tests with coverage
run: |
# Python
if ls tests/*.py &>/dev/null; then
pip install pytest pytest-cov
pytest --cov=src --cov-report=json --cov-fail-under=80
fi
# Node.js
if [ -f package.json ]; then
npm ci
npm test -- --coverage --coverageThreshold='{"global":{"branches":80,"functions":80,"lines":80}}'
fi
- name: Verify new code has tests
run: |
NEW_FILES=$(git diff --name-only --diff-filter=A origin/main...HEAD -- 'src/**')
for file in $NEW_FILES; do
base=$(basename "$file" | sed 's/\.[^.]*$//')
if ! find tests/ -name "*${base}*" | grep -q .; then
echo "::error::New file $file has no corresponding test file"
exit 1
fi
done
security-scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run Bandit (Python)
if: hashFiles('**/*.py') != ''
run: |
pip install bandit
bandit -r src/ -f json -o bandit-report.json || true
ISSUES=$(jq '.results | length' bandit-report.json)
if [ "$ISSUES" -gt 0 ]; then
echo "::warning::Bandit found $ISSUES security issue(s) in agent-generated code"
jq -r '.results[] | " \(.severity): \(.issue_text) in \(.filename):\(.line_number)"' bandit-report.json
fi
- name: Run ESLint security plugin (JavaScript/TypeScript)
if: hashFiles('**/*.js') != '' || hashFiles('**/*.ts') != ''
run: |
npm ci
npx eslint --no-eslintrc \
--plugin security \
--rule '{"security/detect-eval-with-expression": "error"}' \
--rule '{"security/detect-non-literal-fs-filename": "warn"}' \
--rule '{"security/detect-possible-timing-attacks": "error"}' \
--rule '{"security/detect-no-csrf-before-method-override": "error"}' \
src/ || true
mutation-testing:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run mutation testing on agent-changed files
run: |
CHANGED=$(git diff --name-only origin/main...HEAD -- 'src/**/*.py')
if [ -n "$CHANGED" ]; then
pip install mutmut
for file in $CHANGED; do
echo "Mutation testing: $file"
mutmut run --paths-to-mutate="$file" --no-progress || true
done
mutmut results
fi#!/bin/bash
# validate-agent-pr.sh
# Run all validation checks before merging an agent-generated PR
set -euo pipefail
PR_BRANCH="${1:?Usage: validate-agent-pr.sh <branch-name>}"
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
PASS=0
FAIL=0
WARN=0
check() {
local name="$1"
shift
if "$@" >/dev/null 2>&1; then
echo -e "${GREEN}PASS${NC}: $name"
((PASS++))
else
echo -e "${RED}FAIL${NC}: $name"
((FAIL++))
fi
}
warn_check() {
local name="$1"
shift
if "$@" >/dev/null 2>&1; then
echo -e "${GREEN}PASS${NC}: $name"
((PASS++))
else
echo -e "${YELLOW}WARN${NC}: $name"
((WARN++))
fi
}
echo "Validating agent PR: $PR_BRANCH"
echo "=============================="
# Check that branch follows naming convention
check "Branch naming convention" [[ "$PR_BRANCH" == ai/* || "$PR_BRANCH" == agent/* ]]
# Check for secrets in diff
check "No secrets in diff" git secrets --scan
# Check that tests pass
check "Test suite passes" npm test
# Check test coverage threshold
warn_check "Coverage above 80%" npm test -- --coverage --coverageThreshold='{"global":{"lines":80}}'
# Check for forbidden file modifications
FORBIDDEN_CHANGES=$(git diff --name-only origin/main..."$PR_BRANCH" -- \
'infrastructure/' 'terraform/' '.github/workflows/' '.env*' '*.pem' '*.key')
check "No forbidden file changes" [ -z "$FORBIDDEN_CHANGES" ]
# Check for new dependencies
DEP_CHANGES=$(git diff --name-only origin/main..."$PR_BRANCH" -- \
'package.json' 'requirements.txt' 'go.mod' 'Cargo.toml')
warn_check "No dependency changes" [ -z "$DEP_CHANGES" ]
# Check commit messages have AI trailers
MISSING_TRAILERS=$(git log origin/main.."$PR_BRANCH" --format='%B' \
| grep -cL "AI-Generated-By:" || true)
warn_check "All commits have AI attribution trailers" [ "$MISSING_TRAILERS" -eq 0 ]
echo ""
echo "=============================="
echo -e "Results: ${GREEN}${PASS} passed${NC}, ${RED}${FAIL} failed${NC}, ${YELLOW}${WARN} warnings${NC}"
if [ "$FAIL" -gt 0 ]; then
echo -e "${RED}PR validation FAILED. Address issues before merging.${NC}"
exit 1
fi
echo -e "${GREEN}PR validation passed.${NC}"| Control | Tool | Purpose |
|---|---|---|
| Permission boundaries | CLAUDE.md, .cursorrules, codex.md | Restrict agent behavior per-repo |
| Secret scanning | git-secrets, pre-commit hooks | Block credential leaks |
| Sandbox isolation | Docker, seccomp, network=none | Contain agent execution |
| Code review gates | GitHub Actions, branch protection | Enforce human review |
| Network controls | iptables, Squid proxy | Limit agent internet access |
| Audit trail | Git trailers, JSONL logger | Track AI-generated code |
| Test requirements | Coverage gates, mutation testing | Validate agent output quality |
| Team policies | agent-policy.yaml, CODEOWNERS | Govern agent usage org-wide |