healthcare-eval-harness

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Healthcare Eval Harness — Patient Safety Verification

医疗评估工具 — 患者安全验证

Automated verification system for healthcare application deployments. A single CRITICAL failure blocks deployment. Patient safety is non-negotiable.

Note: Examples use Jest as the reference test runner. Adapt commands for your framework (Vitest, pytest, PHPUnit, etc.) — the test categories and pass thresholds are framework-agnostic.

面向医疗应用部署的自动化验证系统，只要出现1个严重故障就会阻止部署，患者安全是不容妥协的。

注意： 示例使用Jest作为参考测试运行器，你可以适配自己使用的框架（Vitest、pytest、PHPUnit等）的命令 —— 测试分类和通过阈值与框架无关。

When to Use

适用场景

Before any deployment of EMR/EHR applications
After modifying CDSS logic (drug interactions, dose validation, scoring)
After changing database schemas that touch patient data
After modifying authentication or access control
During CI/CD pipeline configuration for healthcare apps
After resolving merge conflicts in clinical modules

EMR/EHR应用任何部署前
修改CDSS逻辑（药物相互作用、剂量验证、评分）后
更改涉及患者数据的数据库schema后
修改身份验证或访问控制后
配置医疗应用的CI/CD流水线期间
解决临床模块的合并冲突后

How It Works

工作原理

The eval harness runs five test categories in order. The first three (CDSS Accuracy, PHI Exposure, Data Integrity) are CRITICAL gates requiring 100% pass rate — a single failure blocks deployment. The remaining two (Clinical Workflow, Integration) are HIGH gates requiring 95%+ pass rate.

Each category maps to a Jest test path pattern. The CI pipeline runs CRITICAL gates with

--bail

(stop on first failure) and enforces coverage thresholds with

--coverage --coverageThreshold

评估工具按顺序运行5类测试，前3类（CDSS准确性、PHI泄露、数据完整性）是严重级别闸门，要求100%通过率 —— 只要1个测试失败就会阻止部署。剩下2类（临床工作流、集成合规）是高优先级闸门，要求95%以上通过率。

每个分类对应一个Jest测试路径匹配规则，CI流水线运行严重级别闸门时会带上

--bail

参数（首次失败就终止），并通过

--coverage --coverageThreshold

强制要求覆盖率阈值。

Eval Categories

评估分类

1. CDSS Accuracy (CRITICAL — 100% required)

Tests all clinical decision support logic: drug interaction pairs (both directions), dose validation rules, clinical scoring vs published specs, no false negatives, no silent failures.

bash

npx jest --testPathPattern='tests/cdss' --bail --ci --coverage

2. PHI Exposure (CRITICAL — 100% required)

Tests for protected health information leaks: API error responses, console output, URL parameters, browser storage, cross-facility isolation, unauthenticated access, service role key absence.

bash

npx jest --testPathPattern='tests/security/phi' --bail --ci

3. Data Integrity (CRITICAL — 100% required)

Tests clinical data safety: locked encounters, audit trail entries, cascade delete protection, concurrent edit handling, no orphaned records.

bash

npx jest --testPathPattern='tests/data-integrity' --bail --ci

4. Clinical Workflow (HIGH — 95%+ required)

Tests end-to-end flows: encounter lifecycle, template rendering, medication sets, drug/diagnosis search, prescription PDF, red flag alerts.

bash

tmp_json=$(mktemp)
npx jest --testPathPattern='tests/clinical' --ci --json --outputFile="$tmp_json" || true
total=$(jq '.numTotalTests // 0' "$tmp_json")
passed=$(jq '.numPassedTests // 0' "$tmp_json")
if [ "$total" -eq 0 ]; then
  echo "No clinical tests found" >&2
  exit 1
fi
rate=$(echo "scale=2; $passed * 100 / $total" | bc)
echo "Clinical pass rate: ${rate}% ($passed/$total)"

5. Integration Compliance (HIGH — 95%+ required)

Tests external systems: HL7 message parsing (v2.x), FHIR validation, lab result mapping, malformed message handling.

bash

tmp_json=$(mktemp)
npx jest --testPathPattern='tests/integration' --ci --json --outputFile="$tmp_json" || true
total=$(jq '.numTotalTests // 0' "$tmp_json")
passed=$(jq '.numPassedTests // 0' "$tmp_json")
if [ "$total" -eq 0 ]; then
  echo "No integration tests found" >&2
  exit 1
fi
rate=$(echo "scale=2; $passed * 100 / $total" | bc)
echo "Integration pass rate: ${rate}% ($passed/$total)"

1. CDSS准确性（严重级别 — 要求100%通过）

测试所有临床决策支持逻辑：药物相互作用配对（双向）、剂量验证规则、临床评分与公开规范的一致性、无假阴性、无静默故障。

bash

npx jest --testPathPattern='tests/cdss' --bail --ci --coverage

2. PHI泄露（严重级别 — 要求100%通过）

测试受保护健康信息泄露风险：API错误响应、控制台输出、URL参数、浏览器存储、跨机构隔离、未授权访问、服务角色密钥缺失。

bash

npx jest --testPathPattern='tests/security/phi' --bail --ci

3. 数据完整性（严重级别 — 要求100%通过）

测试临床数据安全性：就诊记录锁定、审计日志记录、级联删除保护、并发编辑处理、无孤立记录。

bash

npx jest --testPathPattern='tests/data-integrity' --bail --ci

4. 临床工作流（高优先级 — 要求95%以上通过）

测试端到端流程：就诊生命周期、模板渲染、药物集合、药品/诊断搜索、处方PDF、红色风险警报。

bash

tmp_json=$(mktemp)
npx jest --testPathPattern='tests/clinical' --ci --json --outputFile="$tmp_json" || true
total=$(jq '.numTotalTests // 0' "$tmp_json")
passed=$(jq '.numPassedTests // 0' "$tmp_json")
if [ "$total" -eq 0 ]; then
  echo "No clinical tests found" >&2
  exit 1
fi
rate=$(echo "scale=2; $passed * 100 / $total" | bc)
echo "Clinical pass rate: ${rate}% ($passed/$total)"

5. 集成合规性（高优先级 — 要求95%以上通过）

测试外部系统集成：HL7消息解析（v2.x）、FHIR校验、检验结果映射、异常格式消息处理。

bash

tmp_json=$(mktemp)
npx jest --testPathPattern='tests/integration' --ci --json --outputFile="$tmp_json" || true
total=$(jq '.numTotalTests // 0' "$tmp_json")
passed=$(jq '.numPassedTests // 0' "$tmp_json")
if [ "$total" -eq 0 ]; then
  echo "No integration tests found" >&2
  exit 1
fi
rate=$(echo "scale=2; $passed * 100 / $total" | bc)
echo "Integration pass rate: ${rate}% ($passed/$total)"

Pass/Fail Matrix

通过/失败规则表

Category	Threshold	On Failure
CDSS Accuracy	100%	BLOCK deployment
PHI Exposure	100%	BLOCK deployment
Data Integrity	100%	BLOCK deployment
Clinical Workflow	95%+	WARN, allow with review
Integration	95%+	WARN, allow with review

分类	阈值	失败处理
CDSS准确性	100%	阻止部署
PHI泄露	100%	阻止部署
数据完整性	100%	阻止部署
临床工作流	95%+	警告，经审核后可允许部署
集成合规	95%+	警告，经审核后可允许部署

CI/CD Integration

CI/CD集成

yaml

name: Healthcare Safety Gate
on: [push, pull_request]

jobs:
  safety-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci

      # CRITICAL gates — 100% required, bail on first failure
      - name: CDSS Accuracy
        run: npx jest --testPathPattern='tests/cdss' --bail --ci --coverage --coverageThreshold='{"global":{"branches":80,"functions":80,"lines":80}}'

      - name: PHI Exposure Check
        run: npx jest --testPathPattern='tests/security/phi' --bail --ci

      - name: Data Integrity
        run: npx jest --testPathPattern='tests/data-integrity' --bail --ci

      # HIGH gates — 95%+ required, custom threshold check
      # HIGH gates — 95%+ required
      - name: Clinical Workflows
        run: |
          TMP_JSON=$(mktemp)
          npx jest --testPathPattern='tests/clinical' --ci --json --outputFile="$TMP_JSON" || true
          TOTAL=$(jq '.numTotalTests // 0' "$TMP_JSON")
          PASSED=$(jq '.numPassedTests // 0' "$TMP_JSON")
          if [ "$TOTAL" -eq 0 ]; then
            echo "::error::No clinical tests found"; exit 1
          fi
          RATE=$(echo "scale=2; $PASSED * 100 / $TOTAL" | bc)
          echo "Pass rate: ${RATE}% ($PASSED/$TOTAL)"
          if (( $(echo "$RATE < 95" | bc -l) )); then
            echo "::warning::Clinical pass rate ${RATE}% below 95%"
          fi

      - name: Integration Compliance
        run: |
          TMP_JSON=$(mktemp)
          npx jest --testPathPattern='tests/integration' --ci --json --outputFile="$TMP_JSON" || true
          TOTAL=$(jq '.numTotalTests // 0' "$TMP_JSON")
          PASSED=$(jq '.numPassedTests // 0' "$TMP_JSON")
          if [ "$TOTAL" -eq 0 ]; then
            echo "::error::No integration tests found"; exit 1
          fi
          RATE=$(echo "scale=2; $PASSED * 100 / $TOTAL" | bc)
          echo "Pass rate: ${RATE}% ($PASSED/$TOTAL)"
          if (( $(echo "$RATE < 95" | bc -l) )); then
            echo "::warning::Integration pass rate ${RATE}% below 95%"
          fi

yaml

name: Healthcare Safety Gate
on: [push, pull_request]

jobs:
  safety-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci

      # 严重级别闸门 — 要求100%通过，首次失败就终止
      - name: CDSS Accuracy
        run: npx jest --testPathPattern='tests/cdss' --bail --ci --coverage --coverageThreshold='{"global":{"branches":80,"functions":80,"lines":80}}'

      - name: PHI Exposure Check
        run: npx jest --testPathPattern='tests/security/phi' --bail --ci

      - name: Data Integrity
        run: npx jest --testPathPattern='tests/data-integrity' --bail --ci

      # 高优先级闸门 — 要求95%以上通过，自定义阈值检查
      # 高优先级闸门 — 要求95%以上通过
      - name: Clinical Workflows
        run: |
          TMP_JSON=$(mktemp)
          npx jest --testPathPattern='tests/clinical' --ci --json --outputFile="$TMP_JSON" || true
          TOTAL=$(jq '.numTotalTests // 0' "$TMP_JSON")
          PASSED=$(jq '.numPassedTests // 0' "$TMP_JSON")
          if [ "$TOTAL" -eq 0 ]; then
            echo "::error::No clinical tests found"; exit 1
          fi
          RATE=$(echo "scale=2; $PASSED * 100 / $TOTAL" | bc)
          echo "Pass rate: ${RATE}% ($PASSED/$TOTAL)"
          if (( $(echo "$RATE < 95" | bc -l) )); then
            echo "::warning::Clinical pass rate ${RATE}% below 95%"
          fi

      - name: Integration Compliance
        run: |
          TMP_JSON=$(mktemp)
          npx jest --testPathPattern='tests/integration' --ci --json --outputFile="$TMP_JSON" || true
          TOTAL=$(jq '.numTotalTests // 0' "$TMP_JSON")
          PASSED=$(jq '.numPassedTests // 0' "$TMP_JSON")
          if [ "$TOTAL" -eq 0 ]; then
            echo "::error::No integration tests found"; exit 1
          fi
          RATE=$(echo "scale=2; $PASSED * 100 / $TOTAL" | bc)
          echo "Pass rate: ${RATE}% ($PASSED/$TOTAL)"
          if (( $(echo "$RATE < 95" | bc -l) )); then
            echo "::warning::Integration pass rate ${RATE}% below 95%"
          fi

Anti-Patterns

反模式

Skipping CDSS tests "because they passed last time"
Setting CRITICAL thresholds below 100%
Using
```
--no-bail
```
on CRITICAL test suites
Mocking the CDSS engine in integration tests (must test real logic)
Allowing deployments when safety gate is red
Running tests without
```
--coverage
```
on CDSS suites

以「上次已经通过」为由跳过CDSS测试
将严重级别闸门的阈值设置为低于100%
在严重级别测试套件中使用
```
--no-bail
```
参数
在集成测试中Mock CDSS引擎（必须测试真实逻辑）
安全闸门显示红灯时仍允许部署
运行CDSS测试套件时不带
```
--coverage
```
参数

Examples

示例

Example 1: Run All Critical Gates Locally

示例1：本地运行所有严重级别闸门

bash

npx jest --testPathPattern='tests/cdss' --bail --ci --coverage && \
npx jest --testPathPattern='tests/security/phi' --bail --ci && \
npx jest --testPathPattern='tests/data-integrity' --bail --ci

bash

npx jest --testPathPattern='tests/cdss' --bail --ci --coverage && \
npx jest --testPathPattern='tests/security/phi' --bail --ci && \
npx jest --testPathPattern='tests/data-integrity' --bail --ci

Example 2: Check HIGH Gate Pass Rate

示例2：检查高优先级闸门通过率

bash

tmp_json=$(mktemp)
npx jest --testPathPattern='tests/clinical' --ci --json --outputFile="$tmp_json" || true
jq '{
  passed: (.numPassedTests // 0),
  total: (.numTotalTests // 0),
  rate: (if (.numTotalTests // 0) == 0 then 0 else ((.numPassedTests // 0) / (.numTotalTests // 1) * 100) end)
}' "$tmp_json"

bash

tmp_json=$(mktemp)
npx jest --testPathPattern='tests/clinical' --ci --json --outputFile="$tmp_json" || true
jq '{
  passed: (.numPassedTests // 0),
  total: (.numTotalTests // 0),
  rate: (if (.numTotalTests // 0) == 0 then 0 else ((.numPassedTests // 0) / (.numTotalTests // 1) * 100) end)
}' "$tmp_json"

Expected: { "passed": 21, "total": 22, "rate": 95.45 }

undefined

undefined

Example 3: Eval Report

示例3：评估报告

undefined

undefined

Healthcare Eval: 2026-03-27 [commit abc1234]

Patient Safety: PASS

Category	Tests	Pass	Fail	Status
CDSS Accuracy	39	39	0	PASS
PHI Exposure	8	8	0	PASS
Data Integrity	12	12	0	PASS
Clinical Workflow	22	21	1	95.5% PASS
Integration	6	6	0	PASS

Category	Tests	Pass	Fail	Status
CDSS Accuracy	39	39	0	PASS
PHI Exposure	8	8	0	PASS
Data Integrity	12	12	0	PASS
Clinical Workflow	22	21	1	95.5% PASS
Integration	6	6	0	PASS

Coverage: 84% (target: 80%+)

Verdict: SAFE TO DEPLOY

undefined

undefined