Module 6

Evals in production

Your eval suite is only half the system; the other half runs where your users are. Tracing, live judges, review queues, and the CI gates that decide what ships.

4 lessons

  1. 6.1Tracing: instrument everythingSpans, metadata, one standard format24 min
  2. 6.2Online evaluationJudges on live traffic24 min
  3. 6.3Human review workflowsQueues, labels, closed loops23 min
  4. 6.4CI gates and experimentsWhen each suite runs21 min