Loading...
Loading...
Found 17 Skills
You are an expert error analysis specialist with deep expertise in debugging distributed systems, analyzing production incidents, and implementing comprehensive observability solutions.
You are an expert error analysis specialist with deep expertise in debugging distributed systems, analyzing production incidents, and implementing comprehensive observability solutions.
Help the user systematically identify and categorize failure modes in an LLM pipeline by reading traces. Use when starting a new eval project, after significant pipeline changes (new features, model switches, prompt rewrites), when production metrics drop, or after incidents.
Audit an LLM eval pipeline and surface problems: missing error analysis, unvalidated judges, vanity metrics, etc. Use when inheriting an eval system, when unsure whether evals are trustworthy, or as a starting point when no eval infrastructure exists. Do NOT use when the goal is to build a new evaluator from scratch (use error-analysis, write-judge-prompt, or validate-evaluator instead).
Advanced error analysis and pattern detection specialist for identifying, analyzing, and preventing software errors
Debug experiment code with structured error analysis. Categorize errors, apply targeted fixes with retry logic, and use reflection to prevent recurring issues. Use when experiment code fails or produces incorrect results.
Generate a custom trace annotation web app for open coding during LLM error analysis. Use when the user wants to review LLM traces, annotate failures with freeform comments, and do first-pass qualitative labeling (open coding). Also use when the user mentions "annotate traces", "trace review tool", "open coding tool", "label traces", "build an annotation interface", "review LLM outputs", or wants to manually inspect pipeline traces before building a failure taxonomy. This skill produces a tailored Python web application using FastHTML, TailwindCSS, and HTMX.
Create an AI Evals Pack (eval PRD, test set, rubric, judge plan, results + iteration loop). Use for LLM evaluation, benchmarks, rubrics, error analysis/open coding, and ship/no-ship quality gates for AI features.
Use proactively when encountering Rails errors, test failures, build issues, or unexpected behavior. Analyzes errors, reproduces issues, and identifies root causes.
Identifies bugs, analyzes errors, performs root cause analysis, and proposes fixes
AscendC Operator Precision Evaluation. Generate a comprehensive precision test case set (≥30 cases) for the compiled and installed operator, run the tests and generate a precision verification report. Keywords: precision test, precision evaluation, precision report, accuracy, error analysis. After execution, YOU MUST display the overview, failure summary and key findings in the current conversation, and must not only attach the report path.
Sentry JavaScript frontend bug pattern review based on real production errors. Use when reviewing React/TypeScript frontend code for common bug patterns. Trigger keywords: "javascript bug review", "frontend errors", "react error patterns", "sentry frontend bugs".