data-wizard
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseData Wizard
数据向导
Full-stack data science and ML engineering — from exploratory data analysis through model deployment strategy. Adapts approach based on complexity classification.
全栈数据科学与ML工程——从探索性数据分析到模型部署策略。根据复杂度分类调整方法。
Canonical Vocabulary
标准术语表
| Term | Definition |
|---|---|
| EDA | Exploratory Data Analysis — systematic profiling and summarization of a dataset |
| feature | An individual measurable property used as input to a model |
| feature engineering | Creating, transforming, or selecting features to improve model performance |
| hypothesis test | A statistical procedure to determine if observed data supports a claim |
| p-value | Probability of observing data at least as extreme as the actual results, assuming the null hypothesis is true |
| effect size | Magnitude of a difference or relationship, independent of sample size |
| power analysis | Determining sample size needed to detect an effect of a given size |
| CUPED | Controlled-experiment Using Pre-Experiment Data — variance reduction technique for A/B tests |
| MLOps maturity | Level 0 (manual), Level 1 (ML pipeline), Level 2 (CI/CD + CT), Level 3 (full automation) |
| data quality score | Composite metric across completeness, consistency, accuracy, timeliness, uniqueness |
| profile | Statistical summary of a dataset: types, distributions, missing patterns, correlations |
| anomaly | Data point or pattern deviating significantly from expected behavior |
| 术语 | 定义 |
|---|---|
| EDA | Exploratory Data Analysis — 对数据集进行系统性分析和总结 |
| feature | 作为模型输入的单个可测量属性 |
| feature engineering | 创建、转换或选择特征以提升模型性能 |
| hypothesis test | 用于判断观测数据是否支持某一主张的统计流程 |
| p-value | 在原假设成立的前提下,观测到至少与实际结果同样极端数据的概率 |
| effect size | 差异或关系的量级,与样本量无关 |
| power analysis | 确定检测给定量级效应所需样本量的方法 |
| CUPED | Controlled-experiment Using Pre-Experiment Data — A/B测试中的方差缩减技术 |
| MLOps maturity | 成熟度等级:Level 0(手动)、Level 1(ML流水线)、Level 2(CI/CD + CT)、Level 3(全自动化) |
| data quality score | 涵盖完整性、一致性、准确性、及时性、唯一性的综合指标 |
| profile | 数据集的统计摘要:类型、分布、缺失模式、相关性 |
| anomaly | 显著偏离预期行为的数据点或模式 |
Dispatch
指令调度
| Action |
|---|---|
| EDA — profile dataset, summary stats, missing patterns, distributions |
| Model Selection — recommend models, libraries, training plan for task |
| Feature Engineering — suggest transformations, encoding, selection pipeline |
| Stats — select and design statistical hypothesis test |
| Visualization — recommend chart types, encodings, layout for data |
| Viz Plan — JSON chart plan from data + goal via |
| Viz Render — PNG/HTML charts from plan via |
| Viz Dashboard — HTML EDA dashboard via |
| Experiment Design — A/B test design, power analysis, CUPED |
Viz pipeline:→viz plan→ optionalviz render. Runviz dashboardfor JSON encodings,viz-planner.pyfor PNG/HTML assets, then pass profile + plan/render JSON toviz-renderer.pyfor a shareable EDA report. |dashboard-builder.py| Time Series — forecasting approach, decomposition, model selection | |timeseries <data>| Anomaly Detection — detection approach, algorithm selection, threshold strategy | |anomaly <data>| MLOps — serving strategy, deployment pipeline, monitoring plan | | Natural language about data | Auto-detect — classify intent, route to appropriate mode | | Empty | Gallery — show common data science tasks with mode recommendations |mlops <model>
| 操作 |
|---|---|
| EDA — 分析数据集概况、统计摘要、缺失模式、分布情况 |
| 模型选择 — 根据任务推荐模型、库和训练方案 |
| 特征工程 — 建议转换、编码和选择流水线 |
| 统计分析 — 选择并设计统计假设检验 |
| 数据可视化 — 根据数据推荐图表类型、编码方式和布局 |
| 可视化规划 — 通过 |
| 可视化渲染 — 通过 |
| 可视化仪表盘 — 通过 |
| 实验设计 — A/B测试设计、功效分析、CUPED应用 |
可视化流水线:→viz plan→ 可选viz render。运行viz dashboard生成JSON编码,运行viz-planner.py生成PNG/HTML资源,然后将概况+规划/渲染JSON传入viz-renderer.py生成可共享的EDA报告。 |dashboard-builder.py| 时间序列 — 预测方法、分解方式、模型选择 | |timeseries <data>| 异常检测 — 检测方法、算法选择、阈值策略 | |anomaly <data>| MLOps — 服务策略、部署流水线、监控方案 | | 数据相关自然语言描述 | 自动检测 — 分类意图,路由至对应模式 | | 空参数 | 示例库 — 展示常见数据科学任务及模式推荐 |mlops <model>
Auto-Detection Heuristic
自动检测规则
If no mode keyword matches:
- Mentions dataset, CSV, columns, rows, missing values → EDA
- Mentions predict, classify, regression, recommend → Model Selection
- Mentions transform, encode, scale, normalize, one-hot → Feature Engineering
- Mentions test, significant, p-value, hypothesis, correlation → Stats
- Mentions chart, plot, graph, visualize, dashboard → Visualization
- Mentions A/B, experiment, control group, treatment, lift → Experiment Design
- Mentions forecast, seasonal, trend, time series, lag → Time Series
- Mentions outlier, anomaly, fraud, unusual, deviation → Anomaly Detection
- Mentions deploy, serve, pipeline, monitor, retrain → MLOps
- Ambiguous → ask: "Which area: EDA, modeling, stats, or something else?"
若未匹配到模式关键词:
- 提及数据集、CSV、列、行、缺失值 → EDA
- 提及预测、分类、回归、推荐 → 模型选择
- 提及转换、编码、缩放、归一化、独热编码 → 特征工程
- 提及检验、显著性、p-value、假设、相关性 → 统计分析
- 提及图表、绘图、可视化、仪表盘 → 数据可视化
- 提及A/B、实验、对照组、处理组、提升 → 实验设计
- 提及预测、季节性、趋势、时间序列、滞后 → 时间序列
- 提及异常值、反常、欺诈、异常、偏离 → 异常检测
- 提及部署、服务、流水线、监控、重训练 → MLOps
- 模糊不清 → 询问:"您需要哪个领域的支持:EDA、建模、统计还是其他?"
Gallery (Empty Arguments)
示例库(空参数)
Present common data science tasks:
| # | Task | Mode | Example |
|---|---|---|---|
| 1 | Profile a dataset | | |
| 2 | Choose a model | | |
| 3 | Engineer features | | |
| 4 | Pick a stat test | | |
| 5 | Choose visualizations | | |
| 5b | Plan + render charts | | |
| 5c | Build EDA dashboard | | |
| 6 | Design an experiment | | |
| 7 | Forecast time series | | |
| 8 | Detect anomalies | | |
| 9 | Plan deployment | | |
Pick a number or describe your data science task.
展示常见数据科学任务:
| 序号 | 任务 | 模式 | 示例 |
|---|---|---|---|
| 1 | 分析数据集概况 | | |
| 2 | 选择模型 | | |
| 3 | 特征工程 | | |
| 4 | 选择统计检验方法 | | |
| 5 | 选择可视化方案 | | |
| 5b | 规划并渲染图表 | | |
| 5c | 构建EDA仪表盘 | | |
| 6 | 设计实验 | | |
| 7 | 时间序列预测 | | |
| 8 | 异常检测 | | |
| 9 | 规划模型部署 | | |
选择序号或描述您的数据科学任务。
Skill Awareness
技能适配检查
Before starting, check if another skill is a better fit:
| Signal | Redirect |
|---|---|
| Database schema, SQL optimization, indexing | Suggest |
| Frontend dashboard code, React/D3 components | Suggest relevant frontend skill |
| Data pipeline, ETL, orchestration (Airflow, dbt) | Out of scope — suggest data engineering tools |
| Production infrastructure, Kubernetes, scaling | Suggest |
开始前,检查是否有更合适的技能:
| 信号 | 重定向建议 |
|---|---|
| 数据库 schema、SQL优化、索引 | 推荐 |
| 前端仪表盘代码、React/D3组件 | 推荐相关前端技能 |
| 数据流水线、ETL、编排(Airflow、dbt) | 超出范围 — 推荐数据工程工具 |
| 生产基础设施、Kubernetes、扩容 | 推荐 |
Complexity Classification
复杂度分类
Score the query on 4 dimensions (0-2 each, total 0-8):
| Dimension | 0 | 1 | 2 |
|---|---|---|---|
| Data complexity | Single table, clean | Multi-table, some nulls | Messy, multi-source, mixed types |
| Analysis depth | Descriptive stats | Inferential / predictive | Multi-stage pipeline, iteration |
| Domain specificity | General / well-known | Domain conventions apply | Deep domain expertise needed |
| Tooling breadth | Single library suffices | 2-3 libraries needed | Full ML stack integration |
| Total | Tier | Strategy |
|---|---|---|
| 0-2 | Quick | Single inline analysis — eda, viz, stats |
| 3-5 | Standard | Multi-step workflow — features, model, experiment, timeseries, anomaly |
| 6-8 | Full Pipeline | Orchestrated — mlops, complex multi-stage analysis |
Present the scoring to the user. User can override tier.
从4个维度对查询打分(每个维度0-2分,总分0-8分):
| 维度 | 0分 | 1分 | 2分 |
|---|---|---|---|
| 数据复杂度 | 单表、数据干净 | 多表、存在少量空值 | 数据杂乱、多源、混合类型 |
| 分析深度 | 描述性统计 | 推断/预测分析 | 多阶段流水线、迭代分析 |
| 领域特异性 | 通用/知名领域 | 需遵循领域惯例 | 需深度领域专业知识 |
| 工具广度 | 单个库即可满足需求 | 需要2-3个库 | 需整合完整ML栈 |
| 总分 | 等级 | 策略 |
|---|---|---|
| 0-2 | 快速处理 | 单次内联分析 — EDA、可视化、统计 |
| 3-5 | 标准流程 | 多步骤工作流 — 特征工程、模型、实验、时间序列、异常检测 |
| 6-8 | 全流水线 | 编排式处理 — MLOps、复杂多阶段分析 |
向用户展示打分结果,用户可覆盖等级。
Mode Protocols
模式协议
EDA (Quick)
EDA(快速处理)
- If file path provided, run:
!uv run python scripts/data-profiler.py "$1" - Parse JSON output — present: row/col counts, dtypes, missing patterns, top correlations
- Highlight: data quality issues, distribution skews, potential target leakage
- Recommend next steps: cleaning, feature engineering, or modeling
- 若提供文件路径,运行:
!uv run python scripts/data-profiler.py "$1" - 解析JSON输出 — 展示:行数/列数、数据类型、缺失模式、Top相关性
- 重点高亮:数据质量问题、分布偏斜、潜在目标泄漏
- 推荐下一步:数据清洗、特征工程或建模
Model Selection (Standard)
模型选择(标准流程)
- Run: with task JSON input
!uv run python scripts/model-recommender.py - Present ranked model recommendations with rationale
- Read for detailed guidance by data size and type
references/model-selection.md - Suggest: train/val/test split strategy, evaluation metrics, baseline approach
- 运行:并传入任务JSON输入
!uv run python scripts/model-recommender.py - 展示带理由的排序模型推荐
- 查阅获取按数据规模和类型划分的详细指导
references/model-selection.md - 建议:训练/验证/测试集拆分策略、评估指标、基准方法
Feature Engineering (Standard)
特征工程(标准流程)
- If file path, run data profiler first for column analysis
- Read for patterns by data type
references/feature-engineering.md - Load for structured recommendations
data/feature-engineering-patterns.json - Suggest: transformations, encodings, interaction features, selection methods
- 若提供文件路径,先运行数据概况分析工具进行列分析
- 查阅获取按数据类型划分的模式
references/feature-engineering.md - 加载获取结构化推荐
data/feature-engineering-patterns.json - 建议:转换方法、编码方式、交互特征、选择方法
Stats (Quick)
统计分析(快速处理)
- Run: with question parameters
!uv run python scripts/statistical-test-selector.py - Load for decision tree
data/statistical-tests-tree.json - Read for assumptions and interpretation guidance
references/statistical-tests.md - Present: recommended test, alternatives, assumptions to verify, interpretation template
- 运行:并传入问题参数
!uv run python scripts/statistical-test-selector.py - 加载获取决策树
data/statistical-tests-tree.json - 查阅获取假设和解释指导
references/statistical-tests.md - 展示:推荐检验方法、替代方案、需验证的假设、解释模板
Visualization (Quick)
数据可视化(快速处理)
- Load for chart type selection
data/visualization-grammar.json - Match data characteristics to visualization types
- Recommend: chart type, encoding channels, color palette, layout
- Read when user needs executable artifacts
references/visualization.md
- 加载进行图表类型选择
data/visualization-grammar.json - 将数据特征与可视化类型匹配
- 推荐:图表类型、编码通道、调色板、布局
- 当用户需要可执行产物时,查阅
references/visualization.md
Viz Plan (Quick)
可视化规划(快速处理)
- Run:
!uv run python scripts/viz-planner.py <data> --goal "<goal>" - Parse JSON — present: , chart list with encodings and rationale
goal_category - Confirm column mappings before rendering
- 运行:
!uv run python scripts/viz-planner.py <data> --goal "<goal>" - 解析JSON — 展示:、带编码和理由的图表列表
goal_category - 渲染前确认列映射
Viz Render (Quick)
可视化渲染(快速处理)
- Run:
!uv run python scripts/viz-renderer.py <plan.json> <data> --format png|html - Parse JSON — present output paths and any per-chart errors
- Use PNG for reports/dashboards; HTML for interactive exploration
- 运行:
!uv run python scripts/viz-renderer.py <plan.json> <data> --format png|html - 解析JSON — 展示输出路径和各图表的错误信息
- 报告/仪表盘使用PNG格式;交互式探索使用HTML格式
Viz Dashboard (Quick)
可视化仪表盘(快速处理)
- Ensure profile JSON exists (run and optionally
data-profiler.py)data-quality-scorer.py - Optional: run viz plan + render, pass and
--viz-planto builder--render-result - Run:
!uv run python scripts/dashboard-builder.py <profile.json> --output <path> - Open output HTML via — do not read template into context
file:// - Read for view schema
references/dashboard-design.md
- 确保概况JSON已存在(运行并可选运行
data-profiler.py)data-quality-scorer.py - 可选:运行可视化规划+渲染,将和
--viz-plan传入构建工具--render-result - 运行:
!uv run python scripts/dashboard-builder.py <profile.json> --output <path> - 通过打开输出HTML — 不要将模板读入上下文
file:// - 查阅获取视图 schema
references/dashboard-design.md
Experiment Design (Standard)
实验设计(标准流程)
- Read for A/B test patterns
references/experiment-design.md - Design: hypothesis, metrics, sample size (power analysis), duration
- Address: novelty effects, multiple comparisons, CUPED variance reduction
- Output: experiment brief with decision criteria
- 查阅获取A/B测试模式
references/experiment-design.md - 设计:假设、指标、样本量(功效分析)、时长
- 考虑:新奇效应、多重比较、CUPED方差缩减
- 输出:带决策标准的实验简报
Time Series (Standard)
时间序列(标准流程)
- If file path, run data profiler for temporal patterns
- Assess: stationarity, seasonality, trend, autocorrelation
- Recommend: decomposition method, forecasting model, validation strategy
- Address: cross-validation for time series (walk-forward), feature lags
- 若提供文件路径,运行数据概况分析工具获取时间模式
- 评估:平稳性、季节性、趋势、自相关性
- 推荐:分解方法、预测模型、验证策略
- 考虑:时间序列交叉验证(滚动前进法)、特征滞后
Anomaly Detection (Standard)
异常检测(标准流程)
- Classify: point anomalies, contextual anomalies, collective anomalies
- Recommend: algorithm (Isolation Forest, LOF, DBSCAN, autoencoder, etc.)
- Address: threshold selection, false positive management, interpretability
- Suggest: alerting strategy, root cause investigation framework
- 分类:点异常、上下文异常、集体异常
- 推荐:算法(Isolation Forest、LOF、DBSCAN、自动编码器等)
- 考虑:阈值选择、误报管理、可解释性
- 建议:告警策略、根因分析框架
MLOps (Full Pipeline)
MLOps(全流水线)
- Read for maturity model
references/mlops-maturity.md - Assess current maturity level (0-3)
- Design: serving strategy (batch vs real-time), monitoring, retraining triggers
- Address: model versioning, A/B testing in production, rollback strategy
- Output: deployment architecture brief
- 查阅获取成熟度模型
references/mlops-maturity.md - 评估当前成熟度等级(0-3)
- 设计:服务策略(批量vs实时)、监控、重训练触发条件
- 考虑:模型版本控制、生产环境A/B测试、回滚策略
- 输出:部署架构简报
Data Quality Assessment
数据质量评估
Run:
!uv run python scripts/data-quality-scorer.py <path>Dimensions scored:
| Dimension | Weight | Checks |
|---|---|---|
| Completeness | 25% | Missing values, null patterns |
| Consistency | 20% | Type uniformity, format violations |
| Accuracy | 20% | Range violations, statistical outliers |
| Timeliness | 15% | Stale records, temporal gaps |
| Uniqueness | 20% | Duplicates, near-duplicates |
运行:
!uv run python scripts/data-quality-scorer.py <path>打分维度:
| 维度 | 权重 | 检查项 |
|---|---|---|
| 完整性 | 25% | 缺失值、空值模式 |
| 一致性 | 20% | 类型一致性、格式违规 |
| 准确性 | 20% | 范围违规、统计异常值 |
| 及时性 | 15% | 过期记录、时间间隙 |
| 唯一性 | 20% | 重复项、近似重复项 |
Reference File Index
参考文件索引
| File | Content | Read When |
|---|---|---|
| Decision tree for test selection, assumptions, interpretation | Stats mode |
| Model catalog by task type, data size, interpretability needs | Model Selection mode |
| Patterns by data type: numeric, categorical, temporal, text, geospatial | Feature Engineering mode |
| A/B test patterns, CUPED, power analysis, multiple comparison corrections | Experiment Design mode |
| Maturity levels 0-3, deployment patterns, monitoring strategy | MLOps mode |
| Quality framework, scoring dimensions, remediation strategies | EDA mode, Data Quality Assessment |
| Viz plan/render pipeline, encoding rules, dependency handling | Viz Plan, Viz Render, Visualization |
| Dashboard JSON schema, builder usage, view selection | Viz Dashboard |
Loading rule: Load ONE reference at a time per the "Read When" column. Do not preload.
| 文件 | 内容 | 查阅时机 |
|---|---|---|
| 检验选择决策树、假设、解释方法 | 统计分析模式 |
| 按任务类型、数据规模、可解释性需求划分的模型目录 | 模型选择模式 |
| 按数据类型划分的模式:数值型、分类型、时间型、文本型、地理空间型 | 特征工程模式 |
| A/B测试模式、CUPED、功效分析、多重比较校正 | 实验设计模式 |
| 成熟度等级0-3、部署模式、监控策略 | MLOps模式 |
| 质量框架、打分维度、修复策略 | EDA模式、数据质量评估 |
| 可视化规划/渲染流水线、编码规则、依赖处理 | 可视化规划、可视化渲染、数据可视化模式 |
| 仪表盘JSON schema、构建工具使用方法、视图选择 | 可视化仪表盘模式 |
加载规则: 每次仅加载一个参考文件,遵循“查阅时机”列的要求。请勿预加载所有参考文件。
Critical Rules
核心规则
- Always run data profiler before recommending models or features — never guess at data characteristics without evidence
- Present classification scoring before executing analysis — user must see and can override complexity tier
- Never recommend a statistical test without stating its assumptions — untested assumptions invalidate results
- Always specify effect size alongside p-values — statistical significance without practical significance is misleading
- Model recommendations must include a baseline — always start with the simplest viable model (logistic regression, linear regression, naive forecast)
- Never skip train/test split strategy — leakage is the most common ML mistake
- Experiment designs must include power analysis — underpowered experiments waste resources
- Feature engineering must address target leakage risk — flag any feature derived from post-outcome data
- Time series cross-validation must use walk-forward — random splits violate temporal ordering
- MLOps recommendations must assess current maturity — do not recommend Level 3 automation for Level 0 teams
- Load ONE reference file at a time — do not preload all references into context
- Data quality scores must be computed, not estimated — run the scorer script on actual data
Canonical terms (use these exactly throughout):
- Modes: "EDA", "Model Selection", "Feature Engineering", "Stats", "Visualization", "Experiment Design", "Time Series", "Anomaly Detection", "MLOps"
- Tiers: "Quick", "Standard", "Full Pipeline"
- Quality dimensions: "Completeness", "Consistency", "Accuracy", "Timeliness", "Uniqueness"
- MLOps levels: "Level 0" (manual), "Level 1" (pipeline), "Level 2" (CI/CD+CT), "Level 3" (full auto)
- 推荐模型或特征前必须运行数据概况分析工具 — 无证据时切勿猜测数据特征
- 执行分析前必须展示复杂度分类打分 — 用户必须查看并可覆盖等级
- 推荐统计检验方法时必须说明其假设条件 — 未验证的假设会导致结果无效
- 展示p-value时必须同时说明效应量 — 仅统计显著性无实际意义会产生误导
- 模型推荐必须包含基准模型 — 始终从最简单的可行模型开始(逻辑回归、线性回归、朴素预测)
- 切勿跳过训练/测试集拆分策略 — 数据泄漏是ML最常见的错误
- 实验设计必须包含功效分析 — 功效不足的实验会浪费资源
- 特征工程必须解决目标泄漏风险 — 标记任何由结果后数据衍生的特征
- 时间序列交叉验证必须使用滚动前进法 — 随机拆分违反时间顺序
- MLOps推荐必须评估当前成熟度 — 切勿为Level 0团队推荐Level 3自动化方案
- 每次仅加载一个参考文件 — 请勿预加载所有参考文件到上下文
- 数据质量得分必须通过计算得出,而非估算 — 对实际数据运行打分脚本
标准术语(全程严格使用以下表述):
- 模式:"EDA"、"模型选择"、"特征工程"、"统计分析"、"数据可视化"、"实验设计"、"时间序列"、"异常检测"、"MLOps"
- 等级:"快速处理"、"标准流程"、"全流水线"
- 质量维度:"完整性"、"一致性"、"准确性"、"及时性"、"唯一性"
- MLOps等级:"Level 0"(手动)、"Level 1"(流水线)、"Level 2"(CI/CD+CT)、"Level 3"(全自动化)