When an AI feature has to survive production, not just demo: prompts, evals, agents, MCP.
Reach for this when you're putting an AI feature into production, not just prototyping one. It carries you through the real build loop: write prompts that hold up under edge cases, prove they work with rigorous evals before you ship, orchestrate multi-step agents when one call isn't enough, and expose your own tools as MCP servers Claude can call. It also covers the surrounding engineering work - deep multi-source research, sound ML feature design, and recoverable, observable error handling - so the system holds together end to end. The outcome is an AI feature you can defend in code review and trust in front of users.
Click to play with sound.
Arranged in the author's recommended order. Walk through them in sequence, or open any one on its own.
Turns a vague request into a structured, reliable prompt – role, context, task, format, failure handling, and few-shot examples – that produces consistent output across real inputs.
Design and build Model Context Protocol servers that models use correctly – choosing stdio vs stateless Streamable HTTP while still serving 2025-era clients, shaping a few task-level tools with clear descriptions, JSON Schema inputs and honest annotations.
Builds evaluation harnesses for LLM products – golden datasets, deterministic checks, calibrated LLM-as-judge rubrics, and CI regression gates that turn "seems good" into a tracked number.
Designs reliable multi-agent LLM systems – choosing a topology, writing handoff contracts between agents, deciding what runs in parallel vs series, and setting context, retry, and cost budgets that stop runaway loops.
Runs a thorough multi-source research process – decompose the question, gather independent sources, cross-check, synthesize – and delivers a structured, cited brief with confidence levels and gaps.
Designs ML features with leakage-safe pipelines, correct categorical encoding by cardinality, numeric transforms, and validation that a feature earns its place.
Designs typed, observable, recoverable error handling – an explicit taxonomy of retryable vs terminal failures, Result types at boundaries, single-point logging, and retry policies with real backoff numbers.