A 12-minute orientation to the Advanced Evals skill path — judges, suites, and gates: the three layers that turn eval-by-vibes into a discipline that ships

Stop checking outputs by vibes — build a runnable eval with a golden dataset, deterministic scorer, and LLM judge, and read the result like an engineer

Design judges that survive CALM biases, calibrate against humans, and earn a place in your CI gate

Author, run, and visualize frontier-grade eval suites with UK AISI's open-source framework

Wire per-PR evals into GitHub Actions, pick thresholds that survive flakiness, and decide when a gate belongs on main

Orientation across the four-chapter Adversarial AI skill path — defender → threat-surface → offense → program — covering everything from your first prompt-injection patch to running an internal red-team practice

Why prompt injection exists, how attackers exploit it, and the layered defense every LLM-powered feature needs before shipping

Map and defend the agent-era attack surface — RAG poisoning, document-borne payloads, memory poisoning, and tool-output hijacking that direct-injection defenses don't reach

Stand up automated offensive tooling for AI red-teaming — DeepTeam, PyRIT, Mindgard; attacker LRMs and BYO-attacker pattern; PAIR, TAP, GCG algorithms; and trajectory caching that turns expensive discovery into cheap CI gates

Run a red-team program for production AI — taxonomy, the finding lifecycle, runtime monitoring, alignment regression, and the internal playbook

A 12-minute orientation to the AI Governance & Compliance skill path — the rules, the evidence, and the response that turn "we should be compliant" into a program regulators accept

Ship compliant general-purpose AI before 2 Aug 2026 — scope, tech docs, copyright, systemic risk, and the Code of Practice shortcut

Produce audit-ready model cards and signed, immutable logs that tie every output to its source, model version, and policy

Handle the 3am page — detect AI failures, contain blast radius, file Article 62 reports on time, and turn post-mortems into hardening

Navigate the fragmented US AI regulatory landscape — NIST AI RMF, state-level laws, post-EO federal posture, and enforcement actions that set precedent

Map the compliance terrain beyond the EU and US — UK AISI, OECD AI Principles, ISO/IEC 42001 certification, and the frameworks shaping AI governance worldwide

Comply with sector regulators — HIPAA for healthcare AI, FDA for medical devices, FINRA for financial services, and PCI-DSS for payment systems

A 12-minute orientation to the Code Testing skill path — the five feedback layers that keep Claude Code and Codex honest, and the order to learn them in

Test strategy, the testing trophy, and writing automated tests that actually catch bugs

Wire Ruff, Biome, Pyrefly, ty, Semgrep, and CodeQL as sub-second feedback sources for Claude Code and Codex

Wire format, lint, type-check, and test into a layered hook stack that forces Claude Code and Codex to fix before they commit

The two-tool feedback loop — coverage shows where tests are missing, mutation shows whether the tests you have actually catch bugs

From one-shot prompts to feedback-driven loops — drive, supervise, and review the systems that now write your tests

Move past hand-picked examples — specify invariants, let generators hunt counterexamples, and pair the discipline with agents that propose the properties you missed

A 12-minute orientation to the E2E & Multimodal Testing skill path — the four senses QA needs when agents ship multimodal products, and the order to learn them in

Master browser testing with Playwright — locators, assertions, debugging, and cross-browser automation

AI agents that browse, click, and test — Playwright MCP, Test Agents, and autonomous QA workflows

Screenshots meet LLM vision — semantic visual QA, vibe checks, design fidelity, and intelligent visual regression

Test what users hear — voice bot QA, audio quality analysis, speech pipeline testing, and multimodal CI

CI/CD pipelines, sharding, test observability, and AI-powered orchestration for production-grade testing

The map of the Production AI skill path — what each chapter teaches, how they fit together, and where to start

Build eval suites, catch regressions, and ship AI features with confidence

Deploy, monitor, and operate AI systems that stay reliable at scale

Instrument, debug, and optimize multi-step AI agents in production

Protect AI applications — from prompt injection defense to EU AI Act compliance

Master token economics, model routing, and budget governance to run AI sustainably

SSE, WebSockets, partial JSON parsing, streaming tool calls, and responsive AI interfaces

Architect reliable, scalable AI-native applications for production

Design AI features users actually trust — confidence indicators, graceful failures, and human-in-the-loop

Prepare for model deprecations, build abstraction layers, and route across providers with confidence

Why k6 and Locust lie about streaming LLMs — TTFT, ITL, goodput, GPU saturation, and SLO-gated load tests that actually predict production