signal-postmortem

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Signal Postmortem

Signal事后分析

Overview

概述

Signal Postmortem records and analyzes the outcomes of trading signals generated by the edge pipeline, screeners, and other skills. It compares predicted edge direction against 5-day and 20-day realized returns, categorizes outcomes (true positive, false positive, missed opportunity, regime mismatch), and generates feedback for edge-signal-aggregator weight adjustments and skill improvement backlog entries.

Signal事后分析用于记录并分析由edge pipeline、筛选器及其他skill生成的交易信号的结果。它会将预测的信号方向与5日和20日的实际收益进行对比，对结果进行分类（真阳性、假阳性、错失机会、市场环境不匹配），并生成反馈用于edge-signal-aggregator的权重调整和skill改进待办事项的创建。

When to Use

使用场景

After a trade has been closed and you want to record the outcome
When reviewing a batch of signals that have reached their holding period (5 or 20 days)
To identify systematic false positive patterns from specific skills
To generate feedback for edge-signal-aggregator weight calibration
When building a skill improvement backlog from decision quality metrics
For periodic (weekly/monthly) signal quality audits

交易平仓后，需要记录结果时
当一批信号达到持有期限（5天或20天），需要复盘时
识别特定skill产生的系统性假阳性模式时
为edge-signal-aggregator的权重校准生成反馈时
根据决策质量指标构建skill改进待办事项时
进行定期（每周/每月）的信号质量审计时

Prerequisites

前置条件

Python 3.9+
FMP API key (optional, for fetching realized returns if not provided manually)
Standard library +
```
requests
```
for API calls
Input: signal records in JSON format (from edge-signal-aggregator or screener outputs)

Python 3.9+
FMP API密钥（可选，若未手动提供实际收益时用于获取数据）
标准库 +
```
requests
```
用于API调用
输入：JSON格式的信号记录（来自edge-signal-aggregator或筛选器的输出）

Workflow

工作流程

Step 1: Prepare Signal Records

步骤1：准备信号记录

Gather closed or matured signal records. Each record should include:

```
signal_id
```
: Unique identifier
```
ticker
```
: Stock symbol
```
signal_date
```
: Date signal was generated
```
predicted_direction
```
: LONG or SHORT
```
source_skill
```
: Which skill generated the signal
```
entry_price
```
: Price at signal generation (optional, for manual override)

bash

undefined

收集已平仓或已到期的信号记录。每条记录应包含：

```
signal_id
```
：唯一标识符
```
ticker
```
：股票代码
```
signal_date
```
：信号生成日期
```
predicted_direction
```
：做多（LONG）或做空（SHORT）
```
source_skill
```
：生成该信号的skill
```
entry_price
```
：信号生成时的价格（可选，用于手动覆盖）

bash

undefined

Example: List signals ready for postmortem (5+ days old)

示例：列出可进行事后分析的信号（生成时间超过5天）

python3 skills/signal-postmortem/scripts/postmortem_recorder.py
--list-ready
--signals-dir state/signals/
--min-days 5

undefined

python3 skills/signal-postmortem/scripts/postmortem_recorder.py
--list-ready
--signals-dir state/signals/
--min-days 5

undefined

Step 2: Record Outcomes

步骤2：记录结果

Run the postmortem recorder to fetch realized returns and classify outcomes.

bash

python3 skills/signal-postmortem/scripts/postmortem_recorder.py \
  --signals-file state/signals/aggregated_signals_2026-03-10.json \
  --holding-periods 5,20 \
  --output-dir reports/

For manual outcome recording (when price data is already available):

bash

python3 skills/signal-postmortem/scripts/postmortem_recorder.py \
  --signal-id sig_aapl_20260310_abc \
  --exit-price 178.50 \
  --exit-date 2026-03-15 \
  --outcome-notes "Closed at target, +3.2% in 5 days" \
  --output-dir reports/

运行事后分析记录器来获取实际收益并对结果进行分类。

bash

python3 skills/signal-postmortem/scripts/postmortem_recorder.py \
  --signals-file state/signals/aggregated_signals_2026-03-10.json \
  --holding-periods 5,20 \
  --output-dir reports/

手动记录结果（当已获取价格数据时）：

bash

python3 skills/signal-postmortem/scripts/postmortem_recorder.py \
  --signal-id sig_aapl_20260310_abc \
  --exit-price 178.50 \
  --exit-date 2026-03-15 \
  --outcome-notes "Closed at target, +3.2% in 5 days" \
  --output-dir reports/

Step 3: Classify Outcomes

步骤3：结果分类

The recorder automatically classifies each signal into one of four categories:

Category	Definition
TRUE_POSITIVE	Predicted direction matched realized return sign
FALSE_POSITIVE	Predicted direction opposite to realized return
MISSED_OPPORTUNITY	Signal not taken but would have been profitable
REGIME_MISMATCH	Signal failed due to market regime change

Classification rules are documented in

references/outcome-classification.md

记录器会自动将每个信号分为以下四类之一：

分类	定义
TRUE_POSITIVE	预测方向与实际收益的符号一致
FALSE_POSITIVE	预测方向与实际收益的符号相反
MISSED_OPPORTUNITY	未执行该信号，但该信号原本可获利
REGIME_MISMATCH	信号失效是由于市场环境变化

分类规则记录在

references/outcome-classification.md

中。

Step 4: Generate Feedback Files

步骤4：生成反馈文件

Generate feedback for downstream consumers:

bash

undefined

为下游模块生成反馈：

bash

undefined

Generate weight adjustment suggestions for edge-signal-aggregator

为edge-signal-aggregator生成权重调整建议

python3 skills/signal-postmortem/scripts/postmortem_analyzer.py
--postmortems-dir reports/postmortems/
--generate-weight-feedback
--output-dir reports/

Generate skill improvement backlog entries

生成skill改进待办事项

python3 skills/signal-postmortem/scripts/postmortem_analyzer.py
--postmortems-dir reports/postmortems/
--generate-improvement-backlog
--output-dir reports/

undefined

python3 skills/signal-postmortem/scripts/postmortem_analyzer.py
--postmortems-dir reports/postmortems/
--generate-improvement-backlog
--output-dir reports/

undefined

Step 5: Review Summary Statistics

步骤5：查看汇总统计数据

Generate aggregate statistics by skill, by ticker, and by time period:

bash

python3 skills/signal-postmortem/scripts/postmortem_analyzer.py \
  --postmortems-dir reports/postmortems/ \
  --summary \
  --group-by skill,month \
  --output-dir reports/

按skill、股票代码和时间段生成汇总统计数据：

bash

python3 skills/signal-postmortem/scripts/postmortem_analyzer.py \
  --postmortems-dir reports/postmortems/ \
  --summary \
  --group-by skill,month \
  --output-dir reports/

Output Format

输出格式

Postmortem Record (JSON)

事后分析记录（JSON）

json

{
  "schema_version": "1.0",
  "postmortem_id": "pm_sig_aapl_20260310_abc",
  "signal_id": "sig_aapl_20260310_abc",
  "ticker": "AAPL",
  "signal_date": "2026-03-10",
  "source_skill": "edge-signal-aggregator",
  "predicted_direction": "LONG",
  "entry_price": 172.50,
  "realized_returns": {
    "5d": 0.032,
    "20d": 0.058
  },
  "exit_price": 178.50,
  "exit_date": "2026-03-15",
  "holding_days": 5,
  "outcome_category": "TRUE_POSITIVE",
  "regime_at_signal": "RISK_ON",
  "regime_at_exit": "RISK_ON",
  "outcome_notes": "Clean breakout, held through minor pullback",
  "recorded_at": "2026-03-17T10:30:00Z"
}

json

{
  "schema_version": "1.0",
  "postmortem_id": "pm_sig_aapl_20260310_abc",
  "signal_id": "sig_aapl_20260310_abc",
  "ticker": "AAPL",
  "signal_date": "2026-03-10",
  "source_skill": "edge-signal-aggregator",
  "predicted_direction": "LONG",
  "entry_price": 172.50,
  "realized_returns": {
    "5d": 0.032,
    "20d": 0.058
  },
  "exit_price": 178.50,
  "exit_date": "2026-03-15",
  "holding_days": 5,
  "outcome_category": "TRUE_POSITIVE",
  "regime_at_signal": "RISK_ON",
  "regime_at_exit": "RISK_ON",
  "outcome_notes": "Clean breakout, held through minor pullback",
  "recorded_at": "2026-03-17T10:30:00Z"
}

Weight Feedback (JSON)

权重反馈（JSON）

json

{
  "schema_version": "1.0",
  "generated_at": "2026-03-17T10:35:00Z",
  "analysis_period": {
    "from": "2026-02-01",
    "to": "2026-03-15"
  },
  "skill_adjustments": [
    {
      "skill": "vcp-screener",
      "current_weight": 1.0,
      "suggested_weight": 0.85,
      "reason": "15% false positive rate in RISK_OFF regime",
      "sample_size": 42
    }
  ],
  "confidence": "MEDIUM",
  "min_sample_threshold": 20
}

json

{
  "schema_version": "1.0",
  "generated_at": "2026-03-17T10:35:00Z",
  "analysis_period": {
    "from": "2026-02-01",
    "to": "2026-03-15"
  },
  "skill_adjustments": [
    {
      "skill": "vcp-screener",
      "current_weight": 1.0,
      "suggested_weight": 0.85,
      "reason": "15% false positive rate in RISK_OFF regime",
      "sample_size": 42
    }
  ],
  "confidence": "MEDIUM",
  "min_sample_threshold": 20
}

Skill Improvement Backlog Entry (YAML)

Skill改进待办事项条目（YAML）

yaml

- skill: vcp-screener
  issue_type: false_positive_cluster
  severity: medium
  evidence:
    false_positive_rate: 0.15
    sample_size: 42
    regime_correlation: RISK_OFF
  suggested_action: "Add regime filter or reduce signal confidence in RISK_OFF"
  generated_by: signal-postmortem
  generated_at: "2026-03-17T10:35:00Z"

yaml

- skill: vcp-screener
  issue_type: false_positive_cluster
  severity: medium
  evidence:
    false_positive_rate: 0.15
    sample_size: 42
    regime_correlation: RISK_OFF
  suggested_action: "Add regime filter or reduce signal confidence in RISK_OFF"
  generated_by: signal-postmortem
  generated_at: "2026-03-17T10:35:00Z"

Summary Report (Markdown)

汇总报告（Markdown）

Reports are saved to

reports/

with filenames

postmortem_summary_YYYY-MM-DD.md

报告将保存到

reports/

目录，文件名为

postmortem_summary_YYYY-MM-DD.md

。

Resources

资源

```
scripts/postmortem_recorder.py
```
-- Records individual signal outcomes
```
scripts/postmortem_analyzer.py
```
-- Generates feedback and summary statistics
```
references/outcome-classification.md
```
-- Classification rules and edge cases
```
references/feedback-integration.md
```
-- How to integrate feedback with downstream skills

```
scripts/postmortem_recorder.py
```
-- 记录单个信号的结果
```
scripts/postmortem_analyzer.py
```
-- 生成反馈和汇总统计数据
```
references/outcome-classification.md
```
-- 分类规则和边缘情况
```
references/feedback-integration.md
```
-- 如何将反馈与下游skill集成

Key Principles

核心原则

Honest Attribution -- Every outcome is attributed to its source skill for accountability
Regime Awareness -- Regime context is recorded to distinguish skill failure from market regime shifts
Minimum Sample Size -- Weight adjustments require 20+ signals for statistical validity
Feedback Loop Closure -- Results flow back to improve both signal aggregation and skill quality

明确归因 -- 每个结果都归因于其来源skill，确保可追溯
环境感知 -- 记录市场环境背景，区分skill失效与市场环境变化
最小样本量 -- 权重调整需要至少20个信号以保证统计有效性
闭环反馈 -- 结果回流用于优化信号聚合和skill质量