data-wizard

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Data Wizard

数据向导

Full-stack data science and ML engineering — from exploratory data analysis through model deployment strategy. Adapts approach based on complexity classification.
全栈数据科学与ML工程——从探索性数据分析到模型部署策略。根据复杂度分类调整方法。

Canonical Vocabulary

标准术语表

TermDefinition
EDAExploratory Data Analysis — systematic profiling and summarization of a dataset
featureAn individual measurable property used as input to a model
feature engineeringCreating, transforming, or selecting features to improve model performance
hypothesis testA statistical procedure to determine if observed data supports a claim
p-valueProbability of observing data at least as extreme as the actual results, assuming the null hypothesis is true
effect sizeMagnitude of a difference or relationship, independent of sample size
power analysisDetermining sample size needed to detect an effect of a given size
CUPEDControlled-experiment Using Pre-Experiment Data — variance reduction technique for A/B tests
MLOps maturityLevel 0 (manual), Level 1 (ML pipeline), Level 2 (CI/CD + CT), Level 3 (full automation)
data quality scoreComposite metric across completeness, consistency, accuracy, timeliness, uniqueness
profileStatistical summary of a dataset: types, distributions, missing patterns, correlations
anomalyData point or pattern deviating significantly from expected behavior
术语定义
EDAExploratory Data Analysis — 对数据集进行系统性分析和总结
feature作为模型输入的单个可测量属性
feature engineering创建、转换或选择特征以提升模型性能
hypothesis test用于判断观测数据是否支持某一主张的统计流程
p-value在原假设成立的前提下,观测到至少与实际结果同样极端数据的概率
effect size差异或关系的量级,与样本量无关
power analysis确定检测给定量级效应所需样本量的方法
CUPEDControlled-experiment Using Pre-Experiment Data — A/B测试中的方差缩减技术
MLOps maturity成熟度等级:Level 0(手动)、Level 1(ML流水线)、Level 2(CI/CD + CT)、Level 3(全自动化)
data quality score涵盖完整性、一致性、准确性、及时性、唯一性的综合指标
profile数据集的统计摘要:类型、分布、缺失模式、相关性
anomaly显著偏离预期行为的数据点或模式

Dispatch

指令调度

$ARGUMENTS
Action
eda <data>
EDA — profile dataset, summary stats, missing patterns, distributions
model <task>
Model Selection — recommend models, libraries, training plan for task
features <data>
Feature Engineering — suggest transformations, encoding, selection pipeline
stats <question>
Stats — select and design statistical hypothesis test
viz <data>
Visualization — recommend chart types, encodings, layout for data
viz plan <data> [goal]
Viz Plan — JSON chart plan from data + goal via
viz-planner.py
viz render <plan> <data>
Viz Render — PNG/HTML charts from plan via
viz-renderer.py
viz dashboard <profile>
Viz Dashboard — HTML EDA dashboard via
dashboard-builder.py
experiment <hypothesis>
Experiment Design — A/B test design, power analysis, CUPED
Viz pipeline:
viz plan
viz render
→ optional
viz dashboard
. Run
viz-planner.py
for JSON encodings,
viz-renderer.py
for PNG/HTML assets, then pass profile + plan/render JSON to
dashboard-builder.py
for a shareable EDA report. |
timeseries <data>
| Time Series — forecasting approach, decomposition, model selection | |
anomaly <data>
| Anomaly Detection — detection approach, algorithm selection, threshold strategy | |
mlops <model>
| MLOps — serving strategy, deployment pipeline, monitoring plan | | Natural language about data | Auto-detect — classify intent, route to appropriate mode | | Empty | Gallery — show common data science tasks with mode recommendations |
$参数
操作
eda <data>
EDA — 分析数据集概况、统计摘要、缺失模式、分布情况
model <task>
模型选择 — 根据任务推荐模型、库和训练方案
features <data>
特征工程 — 建议转换、编码和选择流水线
stats <question>
统计分析 — 选择并设计统计假设检验
viz <data>
数据可视化 — 根据数据推荐图表类型、编码方式和布局
viz plan <data> [goal]
可视化规划 — 通过
viz-planner.py
根据数据和目标生成JSON图表规划
viz render <plan> <data>
可视化渲染 — 通过
viz-renderer.py
根据规划生成PNG/HTML图表
viz dashboard <profile>
可视化仪表盘 — 通过
dashboard-builder.py
生成HTML格式的EDA仪表盘
experiment <hypothesis>
实验设计 — A/B测试设计、功效分析、CUPED应用
可视化流水线:
viz plan
viz render
→ 可选
viz dashboard
。运行
viz-planner.py
生成JSON编码,运行
viz-renderer.py
生成PNG/HTML资源,然后将概况+规划/渲染JSON传入
dashboard-builder.py
生成可共享的EDA报告。 |
timeseries <data>
| 时间序列 — 预测方法、分解方式、模型选择 | |
anomaly <data>
| 异常检测 — 检测方法、算法选择、阈值策略 | |
mlops <model>
| MLOps — 服务策略、部署流水线、监控方案 | | 数据相关自然语言描述 | 自动检测 — 分类意图,路由至对应模式 | | 空参数 | 示例库 — 展示常见数据科学任务及模式推荐 |

Auto-Detection Heuristic

自动检测规则

If no mode keyword matches:
  1. Mentions dataset, CSV, columns, rows, missing values → EDA
  2. Mentions predict, classify, regression, recommend → Model Selection
  3. Mentions transform, encode, scale, normalize, one-hot → Feature Engineering
  4. Mentions test, significant, p-value, hypothesis, correlation → Stats
  5. Mentions chart, plot, graph, visualize, dashboard → Visualization
  6. Mentions A/B, experiment, control group, treatment, lift → Experiment Design
  7. Mentions forecast, seasonal, trend, time series, lag → Time Series
  8. Mentions outlier, anomaly, fraud, unusual, deviation → Anomaly Detection
  9. Mentions deploy, serve, pipeline, monitor, retrain → MLOps
  10. Ambiguous → ask: "Which area: EDA, modeling, stats, or something else?"
若未匹配到模式关键词:
  1. 提及数据集、CSV、列、行、缺失值 → EDA
  2. 提及预测、分类、回归、推荐 → 模型选择
  3. 提及转换、编码、缩放、归一化、独热编码 → 特征工程
  4. 提及检验、显著性、p-value、假设、相关性 → 统计分析
  5. 提及图表、绘图、可视化、仪表盘 → 数据可视化
  6. 提及A/B、实验、对照组、处理组、提升 → 实验设计
  7. 提及预测、季节性、趋势、时间序列、滞后 → 时间序列
  8. 提及异常值、反常、欺诈、异常、偏离 → 异常检测
  9. 提及部署、服务、流水线、监控、重训练 → MLOps
  10. 模糊不清 → 询问:"您需要哪个领域的支持:EDA、建模、统计还是其他?"

Gallery (Empty Arguments)

示例库(空参数)

Present common data science tasks:
#TaskModeExample
1Profile a dataset
eda
/data-wizard eda customer_data.csv
2Choose a model
model
/data-wizard model "predict churn from usage features"
3Engineer features
features
/data-wizard features sales_data.csv
4Pick a stat test
stats
/data-wizard stats "is conversion rate different between groups?"
5Choose visualizations
viz
/data-wizard viz time_series_metrics.csv
5bPlan + render charts
viz plan
/
viz render
/data-wizard viz plan sales.csv "compare regions"
5cBuild EDA dashboard
viz dashboard
/data-wizard viz dashboard profile.json
6Design an experiment
experiment
/data-wizard experiment "new checkout flow increases conversion"
7Forecast time series
timeseries
/data-wizard timeseries monthly_revenue.csv
8Detect anomalies
anomaly
/data-wizard anomaly server_metrics.csv
9Plan deployment
mlops
/data-wizard mlops "churn prediction model"
Pick a number or describe your data science task.
展示常见数据科学任务:
序号任务模式示例
1分析数据集概况
eda
/data-wizard eda customer_data.csv
2选择模型
model
/data-wizard model "基于使用特征预测客户流失"
3特征工程
features
/data-wizard features sales_data.csv
4选择统计检验方法
stats
/data-wizard stats "不同组的转化率是否存在差异?"
5选择可视化方案
viz
/data-wizard viz time_series_metrics.csv
5b规划并渲染图表
viz plan
/
viz render
/data-wizard viz plan sales.csv "对比不同区域"
5c构建EDA仪表盘
viz dashboard
/data-wizard viz dashboard profile.json
6设计实验
experiment
/data-wizard experiment "新结账流程可提升转化率"
7时间序列预测
timeseries
/data-wizard timeseries monthly_revenue.csv
8异常检测
anomaly
/data-wizard anomaly server_metrics.csv
9规划模型部署
mlops
/data-wizard mlops "客户流失预测模型"
选择序号或描述您的数据科学任务。

Skill Awareness

技能适配检查

Before starting, check if another skill is a better fit:
SignalRedirect
Database schema, SQL optimization, indexingSuggest
database-architect
Frontend dashboard code, React/D3 componentsSuggest relevant frontend skill
Data pipeline, ETL, orchestration (Airflow, dbt)Out of scope — suggest data engineering tools
Production infrastructure, Kubernetes, scalingSuggest
devops-engineer
or
infrastructure-coder
开始前,检查是否有更合适的技能:
信号重定向建议
数据库 schema、SQL优化、索引推荐
database-architect
前端仪表盘代码、React/D3组件推荐相关前端技能
数据流水线、ETL、编排(Airflow、dbt)超出范围 — 推荐数据工程工具
生产基础设施、Kubernetes、扩容推荐
devops-engineer
infrastructure-coder

Complexity Classification

复杂度分类

Score the query on 4 dimensions (0-2 each, total 0-8):
Dimension012
Data complexitySingle table, cleanMulti-table, some nullsMessy, multi-source, mixed types
Analysis depthDescriptive statsInferential / predictiveMulti-stage pipeline, iteration
Domain specificityGeneral / well-knownDomain conventions applyDeep domain expertise needed
Tooling breadthSingle library suffices2-3 libraries neededFull ML stack integration
TotalTierStrategy
0-2QuickSingle inline analysis — eda, viz, stats
3-5StandardMulti-step workflow — features, model, experiment, timeseries, anomaly
6-8Full PipelineOrchestrated — mlops, complex multi-stage analysis
Present the scoring to the user. User can override tier.
从4个维度对查询打分(每个维度0-2分,总分0-8分):
维度0分1分2分
数据复杂度单表、数据干净多表、存在少量空值数据杂乱、多源、混合类型
分析深度描述性统计推断/预测分析多阶段流水线、迭代分析
领域特异性通用/知名领域需遵循领域惯例需深度领域专业知识
工具广度单个库即可满足需求需要2-3个库需整合完整ML栈
总分等级策略
0-2快速处理单次内联分析 — EDA、可视化、统计
3-5标准流程多步骤工作流 — 特征工程、模型、实验、时间序列、异常检测
6-8全流水线编排式处理 — MLOps、复杂多阶段分析
向用户展示打分结果,用户可覆盖等级。

Mode Protocols

模式协议

EDA (Quick)

EDA(快速处理)

  1. If file path provided, run:
    !uv run python scripts/data-profiler.py "$1"
  2. Parse JSON output — present: row/col counts, dtypes, missing patterns, top correlations
  3. Highlight: data quality issues, distribution skews, potential target leakage
  4. Recommend next steps: cleaning, feature engineering, or modeling
  1. 若提供文件路径,运行:
    !uv run python scripts/data-profiler.py "$1"
  2. 解析JSON输出 — 展示:行数/列数、数据类型、缺失模式、Top相关性
  3. 重点高亮:数据质量问题、分布偏斜、潜在目标泄漏
  4. 推荐下一步:数据清洗、特征工程或建模

Model Selection (Standard)

模型选择(标准流程)

  1. Run:
    !uv run python scripts/model-recommender.py
    with task JSON input
  2. Present ranked model recommendations with rationale
  3. Read
    references/model-selection.md
    for detailed guidance by data size and type
  4. Suggest: train/val/test split strategy, evaluation metrics, baseline approach
  1. 运行:
    !uv run python scripts/model-recommender.py
    并传入任务JSON输入
  2. 展示带理由的排序模型推荐
  3. 查阅
    references/model-selection.md
    获取按数据规模和类型划分的详细指导
  4. 建议:训练/验证/测试集拆分策略、评估指标、基准方法

Feature Engineering (Standard)

特征工程(标准流程)

  1. If file path, run data profiler first for column analysis
  2. Read
    references/feature-engineering.md
    for patterns by data type
  3. Load
    data/feature-engineering-patterns.json
    for structured recommendations
  4. Suggest: transformations, encodings, interaction features, selection methods
  1. 若提供文件路径,先运行数据概况分析工具进行列分析
  2. 查阅
    references/feature-engineering.md
    获取按数据类型划分的模式
  3. 加载
    data/feature-engineering-patterns.json
    获取结构化推荐
  4. 建议:转换方法、编码方式、交互特征、选择方法

Stats (Quick)

统计分析(快速处理)

  1. Run:
    !uv run python scripts/statistical-test-selector.py
    with question parameters
  2. Load
    data/statistical-tests-tree.json
    for decision tree
  3. Read
    references/statistical-tests.md
    for assumptions and interpretation guidance
  4. Present: recommended test, alternatives, assumptions to verify, interpretation template
  1. 运行:
    !uv run python scripts/statistical-test-selector.py
    并传入问题参数
  2. 加载
    data/statistical-tests-tree.json
    获取决策树
  3. 查阅
    references/statistical-tests.md
    获取假设和解释指导
  4. 展示:推荐检验方法、替代方案、需验证的假设、解释模板

Visualization (Quick)

数据可视化(快速处理)

  1. Load
    data/visualization-grammar.json
    for chart type selection
  2. Match data characteristics to visualization types
  3. Recommend: chart type, encoding channels, color palette, layout
  4. Read
    references/visualization.md
    when user needs executable artifacts
  1. 加载
    data/visualization-grammar.json
    进行图表类型选择
  2. 将数据特征与可视化类型匹配
  3. 推荐:图表类型、编码通道、调色板、布局
  4. 当用户需要可执行产物时,查阅
    references/visualization.md

Viz Plan (Quick)

可视化规划(快速处理)

  1. Run:
    !uv run python scripts/viz-planner.py <data> --goal "<goal>"
  2. Parse JSON — present:
    goal_category
    , chart list with encodings and rationale
  3. Confirm column mappings before rendering
  1. 运行:
    !uv run python scripts/viz-planner.py <data> --goal "<goal>"
  2. 解析JSON — 展示:
    goal_category
    、带编码和理由的图表列表
  3. 渲染前确认列映射

Viz Render (Quick)

可视化渲染(快速处理)

  1. Run:
    !uv run python scripts/viz-renderer.py <plan.json> <data> --format png|html
  2. Parse JSON — present output paths and any per-chart errors
  3. Use PNG for reports/dashboards; HTML for interactive exploration
  1. 运行:
    !uv run python scripts/viz-renderer.py <plan.json> <data> --format png|html
  2. 解析JSON — 展示输出路径和各图表的错误信息
  3. 报告/仪表盘使用PNG格式;交互式探索使用HTML格式

Viz Dashboard (Quick)

可视化仪表盘(快速处理)

  1. Ensure profile JSON exists (run
    data-profiler.py
    and optionally
    data-quality-scorer.py
    )
  2. Optional: run viz plan + render, pass
    --viz-plan
    and
    --render-result
    to builder
  3. Run:
    !uv run python scripts/dashboard-builder.py <profile.json> --output <path>
  4. Open output HTML via
    file://
    — do not read template into context
  5. Read
    references/dashboard-design.md
    for view schema
  1. 确保概况JSON已存在(运行
    data-profiler.py
    并可选运行
    data-quality-scorer.py
  2. 可选:运行可视化规划+渲染,将
    --viz-plan
    --render-result
    传入构建工具
  3. 运行:
    !uv run python scripts/dashboard-builder.py <profile.json> --output <path>
  4. 通过
    file://
    打开输出HTML — 不要将模板读入上下文
  5. 查阅
    references/dashboard-design.md
    获取视图 schema

Experiment Design (Standard)

实验设计(标准流程)

  1. Read
    references/experiment-design.md
    for A/B test patterns
  2. Design: hypothesis, metrics, sample size (power analysis), duration
  3. Address: novelty effects, multiple comparisons, CUPED variance reduction
  4. Output: experiment brief with decision criteria
  1. 查阅
    references/experiment-design.md
    获取A/B测试模式
  2. 设计:假设、指标、样本量(功效分析)、时长
  3. 考虑:新奇效应、多重比较、CUPED方差缩减
  4. 输出:带决策标准的实验简报

Time Series (Standard)

时间序列(标准流程)

  1. If file path, run data profiler for temporal patterns
  2. Assess: stationarity, seasonality, trend, autocorrelation
  3. Recommend: decomposition method, forecasting model, validation strategy
  4. Address: cross-validation for time series (walk-forward), feature lags
  1. 若提供文件路径,运行数据概况分析工具获取时间模式
  2. 评估:平稳性、季节性、趋势、自相关性
  3. 推荐:分解方法、预测模型、验证策略
  4. 考虑:时间序列交叉验证(滚动前进法)、特征滞后

Anomaly Detection (Standard)

异常检测(标准流程)

  1. Classify: point anomalies, contextual anomalies, collective anomalies
  2. Recommend: algorithm (Isolation Forest, LOF, DBSCAN, autoencoder, etc.)
  3. Address: threshold selection, false positive management, interpretability
  4. Suggest: alerting strategy, root cause investigation framework
  1. 分类:点异常、上下文异常、集体异常
  2. 推荐:算法(Isolation Forest、LOF、DBSCAN、自动编码器等)
  3. 考虑:阈值选择、误报管理、可解释性
  4. 建议:告警策略、根因分析框架

MLOps (Full Pipeline)

MLOps(全流水线)

  1. Read
    references/mlops-maturity.md
    for maturity model
  2. Assess current maturity level (0-3)
  3. Design: serving strategy (batch vs real-time), monitoring, retraining triggers
  4. Address: model versioning, A/B testing in production, rollback strategy
  5. Output: deployment architecture brief
  1. 查阅
    references/mlops-maturity.md
    获取成熟度模型
  2. 评估当前成熟度等级(0-3)
  3. 设计:服务策略(批量vs实时)、监控、重训练触发条件
  4. 考虑:模型版本控制、生产环境A/B测试、回滚策略
  5. 输出:部署架构简报

Data Quality Assessment

数据质量评估

Run:
!uv run python scripts/data-quality-scorer.py <path>
Dimensions scored:
DimensionWeightChecks
Completeness25%Missing values, null patterns
Consistency20%Type uniformity, format violations
Accuracy20%Range violations, statistical outliers
Timeliness15%Stale records, temporal gaps
Uniqueness20%Duplicates, near-duplicates
运行:
!uv run python scripts/data-quality-scorer.py <path>
打分维度:
维度权重检查项
完整性25%缺失值、空值模式
一致性20%类型一致性、格式违规
准确性20%范围违规、统计异常值
及时性15%过期记录、时间间隙
唯一性20%重复项、近似重复项

Reference File Index

参考文件索引

FileContentRead When
references/statistical-tests.md
Decision tree for test selection, assumptions, interpretationStats mode
references/model-selection.md
Model catalog by task type, data size, interpretability needsModel Selection mode
references/feature-engineering.md
Patterns by data type: numeric, categorical, temporal, text, geospatialFeature Engineering mode
references/experiment-design.md
A/B test patterns, CUPED, power analysis, multiple comparison correctionsExperiment Design mode
references/mlops-maturity.md
Maturity levels 0-3, deployment patterns, monitoring strategyMLOps mode
references/data-quality.md
Quality framework, scoring dimensions, remediation strategiesEDA mode, Data Quality Assessment
references/visualization.md
Viz plan/render pipeline, encoding rules, dependency handlingViz Plan, Viz Render, Visualization
references/dashboard-design.md
Dashboard JSON schema, builder usage, view selectionViz Dashboard
Loading rule: Load ONE reference at a time per the "Read When" column. Do not preload.
文件内容查阅时机
references/statistical-tests.md
检验选择决策树、假设、解释方法统计分析模式
references/model-selection.md
按任务类型、数据规模、可解释性需求划分的模型目录模型选择模式
references/feature-engineering.md
按数据类型划分的模式:数值型、分类型、时间型、文本型、地理空间型特征工程模式
references/experiment-design.md
A/B测试模式、CUPED、功效分析、多重比较校正实验设计模式
references/mlops-maturity.md
成熟度等级0-3、部署模式、监控策略MLOps模式
references/data-quality.md
质量框架、打分维度、修复策略EDA模式、数据质量评估
references/visualization.md
可视化规划/渲染流水线、编码规则、依赖处理可视化规划、可视化渲染、数据可视化模式
references/dashboard-design.md
仪表盘JSON schema、构建工具使用方法、视图选择可视化仪表盘模式
加载规则: 每次仅加载一个参考文件,遵循“查阅时机”列的要求。请勿预加载所有参考文件。

Critical Rules

核心规则

  1. Always run data profiler before recommending models or features — never guess at data characteristics without evidence
  2. Present classification scoring before executing analysis — user must see and can override complexity tier
  3. Never recommend a statistical test without stating its assumptions — untested assumptions invalidate results
  4. Always specify effect size alongside p-values — statistical significance without practical significance is misleading
  5. Model recommendations must include a baseline — always start with the simplest viable model (logistic regression, linear regression, naive forecast)
  6. Never skip train/test split strategy — leakage is the most common ML mistake
  7. Experiment designs must include power analysis — underpowered experiments waste resources
  8. Feature engineering must address target leakage risk — flag any feature derived from post-outcome data
  9. Time series cross-validation must use walk-forward — random splits violate temporal ordering
  10. MLOps recommendations must assess current maturity — do not recommend Level 3 automation for Level 0 teams
  11. Load ONE reference file at a time — do not preload all references into context
  12. Data quality scores must be computed, not estimated — run the scorer script on actual data
Canonical terms (use these exactly throughout):
  • Modes: "EDA", "Model Selection", "Feature Engineering", "Stats", "Visualization", "Experiment Design", "Time Series", "Anomaly Detection", "MLOps"
  • Tiers: "Quick", "Standard", "Full Pipeline"
  • Quality dimensions: "Completeness", "Consistency", "Accuracy", "Timeliness", "Uniqueness"
  • MLOps levels: "Level 0" (manual), "Level 1" (pipeline), "Level 2" (CI/CD+CT), "Level 3" (full auto)
  1. 推荐模型或特征前必须运行数据概况分析工具 — 无证据时切勿猜测数据特征
  2. 执行分析前必须展示复杂度分类打分 — 用户必须查看并可覆盖等级
  3. 推荐统计检验方法时必须说明其假设条件 — 未验证的假设会导致结果无效
  4. 展示p-value时必须同时说明效应量 — 仅统计显著性无实际意义会产生误导
  5. 模型推荐必须包含基准模型 — 始终从最简单的可行模型开始(逻辑回归、线性回归、朴素预测)
  6. 切勿跳过训练/测试集拆分策略 — 数据泄漏是ML最常见的错误
  7. 实验设计必须包含功效分析 — 功效不足的实验会浪费资源
  8. 特征工程必须解决目标泄漏风险 — 标记任何由结果后数据衍生的特征
  9. 时间序列交叉验证必须使用滚动前进法 — 随机拆分违反时间顺序
  10. MLOps推荐必须评估当前成熟度 — 切勿为Level 0团队推荐Level 3自动化方案
  11. 每次仅加载一个参考文件 — 请勿预加载所有参考文件到上下文
  12. 数据质量得分必须通过计算得出,而非估算 — 对实际数据运行打分脚本
标准术语(全程严格使用以下表述):
  • 模式:"EDA"、"模型选择"、"特征工程"、"统计分析"、"数据可视化"、"实验设计"、"时间序列"、"异常检测"、"MLOps"
  • 等级:"快速处理"、"标准流程"、"全流水线"
  • 质量维度:"完整性"、"一致性"、"准确性"、"及时性"、"唯一性"
  • MLOps等级:"Level 0"(手动)、"Level 1"(流水线)、"Level 2"(CI/CD+CT)、"Level 3"(全自动化)