Loading...
Loading...
Found 6 Skills
Find every way users can break your AI before they do. Use when you need to red-team your AI, test for jailbreaks, find prompt injection vulnerabilities, run adversarial testing, do a safety audit before launch, prove your AI is safe for compliance, stress-test guardrails, or verify your AI holds up against adversarial users. Covers automated attack generation, iterative red-teaming with DSPy, and MIPROv2-optimized adversarial testing.
Run structured AI red team exercises for jailbreak resistance, data exfiltration risk, harmful output controls, and agent tool abuse resilience.
Generate and curate evaluation datasets — structured generation via dimensions-tuples-NL, quick from description, expansion from existing data, plus dataset maintenance through deduplication, rebalancing, and gap-filling. Use when creating eval data, expanding test coverage, or cleaning datasets. Do NOT use when sufficient real production data exists (use analyze-trace-failures instead). Do NOT use for evaluator creation (use build-evaluator).
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
Run a bounded red-teaming autoresearch loop to generate LLM guardrail training data. You (the agent running the skill) are the attacker and the judge: you craft attacks and label every response. The only model the harness calls is the target under test, over any OpenAI-compatible API (OpenRouter, Moonshot/Kimi, Fireworks, Ubicloud, OpenAI, or custom), with its key in `.red-team/.env`. Every attempt (pass and fail) is written to JSONL ready for fine-tuning guardrails in Llama Guard format. Use when asked to red-team or stress-test an LLM for harmful content, jailbreaks, prompt injection, or backdoor/trigger behavior; to build a safe/unsafe dataset; or to mine novel attacks at scale. Complements the hacker skill (code/infra exploitability): this skill targets model behavior and produces data.
Falsify software implementations and gather fresh, machine-checked acceptance proof across tests, builds, typechecks, and runtime smoke checks before completion. Use when running test suites, checking type correctness, validating acceptance criteria, or verifying post-mutation code integrity — even if the user does not explicitly say "fable-verify" (e.g. "verify my changes", "run all tests and typecheck", "check if everything passes", "prove this works"). Do NOT use for planning (use fable-plan) or diff code review (use fable-review).