Loading...
Loading...
Found 83 Skills
Execute tasks through systematic exploration, pruning, and expansion using Tree of Thoughts methodology with multi-agent evaluation
This skill should be used when designing autonomous agent harnesses: research loops, evaluation scaffolds, locked and editable surfaces, durable logs, novelty gates, pruning, rollback, PR preparation, and human approval boundaries.
Run scored behavioral evaluations and repair gaps. Use when the user says "evaluate this agent" or "I want proof this behaves correctly"
Evaluate, test, and track an AG2 Agent offline. Build a Suite of tasks, run the agent with run_agent, and grade answers with prebuilt scorers (final_answer_matches, tool_called, no_tool_errors, token_budget) or a custom @scorer — including the agent_judge LLM judge. Read the RunResult scorecard (pass_rate, score_stats, value_counts), gate it in CI with deterministic TestConfig cassettes, persist to store_dir and diff runs to catch regressions, and grade existing traces with evaluate_traces. Use when the user wants to evaluate, test, grade, or benchmark an agent, build a CI or regression gate, or score correctness, tool use, cost, or quality. To compare builds head-to-head or on a leaderboard, see ag2-eval-comparison.
Run a Karpathy-style LLM council (3-stage multi-agent evaluation) on top of the existing orca orchestration: N seats independently answer a question, the answers are BLIND peer-ranked (identities stripped so a model can't favour its own), and a chairman synthesizes the final answer. The orchestration makes the model calls; harness/scripts/council.py does the deterministic protocol math (anonymization, mean-rank aggregation, dissent, anchor guard, transcript). Use when the user says "run a council", "llm-council", "panel of models", "blind peer-rank these answers", "ensemble + chairman", or invokes /council. Stage-4 is a MANDATORY HTML report, rendered inside council.py itself (offline, no CDN, no external skill): every `rank` writes a NEW versioned self-contained report (llmwiki/html/council/council-report-NNN-seed<seed>.html) — opinion cards (persona name + lens), a blind-vote table, and a dashboard. Isolated in try/except so a render bug never kills the core transcript.
Turn a real agent failure into one reproducible eval-harness case plus the smallest knowledge change — failing case first, one knowledge owner, semantic assertions, before/after evidence through the repo's configured harness commands. Use for "add harness case", "agent got this wrong", "extend the harness", "rozszerz harness".
Verification Loops are systematic evaluation pipelines that validate agent outputs at every stage of execution.
Evaluate changes to agent prompts, skills, routing policies, and harnesses against reproducible baselines, held-out suites, and regression benchmarks. Use when optimizing agent system prompts, measuring skill triggering accuracy, evaluating routing changes, or running benchmark regressions — even if the user does not explicitly say "fable-eval" (e.g. "benchmark this prompt", "evaluate agent behavior", "test skill performance", "run the eval suite"). Do NOT use for routine application unit tests (use fable-verify).
The standard a computed result must meet before it may be called done. Use whenever an agent claims an integral is solved, a fit has closed, a formula is identified, or any quantitative answer is ready — the claim is not done until it passes this gate.
Skeleton for a public KiloAgent skill. Use this when creating a new skill repo that follows skills/<name>/SKILL.md.
Constitution / Operating Model / Practice / Skillがorganizational outcomeとinvariantを維持するかを、formal model・deterministic checks・cold eval・grader controls・context budgetで検証する時に使用する。