Evals - AI, for real

AI, for real

Closing the loop: turn every AI failure into a test

Traces, evals and guardrails only watch on their own. Connect them and your AI gets better every week. Even OpenAI learned this the hard way with a GPT-4o update it rolled back within days.

AI, for real

Silent regressions: why your AI quietly gets worse (and how to catch it)

Same model name, same prompt, completely different behaviour. Stanford researchers watched GPT-4 go from 97.6% to 2.4% on one task. Here's the cheap weekly habit that catches drift before your users do.

AI, for real

Evals vs vibes: you probably don't have AI evals (yet). Here's your first real one

Most 'evals' are a glance at the output and a shrug. Air Canada and Cursor show what that costs. Your first real eval needs no framework: read real outputs, write down what good means.

AI, for real

Does your AI actually work? Start here: the AI, for real series

Everyone is building with AI. Very few teams can prove it works. Here's the one loop this series teaches: trace it, test it, guard it, improve it.