Learning on Web Dev Open is free for all.

AI-Native Products > Evals are the deliverableRegression suites for prompts
Phase 08Evals are the deliverable420 of 434

Regression suites for prompts

A prompt is code with no type checker. Version it, run the suite on every change, and refuse to merge a regression.

Build40 minAI implements

Build this in your own editor

This one runs on your machine rather than in the browser workbench. Work to the outcomes below, then come back and mark it complete.

You should now be able to

  • Version prompts and tie a suite run to a version
  • Run evals in CI with a pass threshold and a report
  • Detect a regression caused by a model update rather than by you
Ask the community

Loading…

← PreviousThree kinds of grader and when each is honestEvals are the deliverableNext →Offline and online, and why you need bothEvals are the deliverable