Learning on Web Dev Open is free for all.

AI-Native Products > Trust, cost and visibilityAn AI feature in production, with the receipts
Phase 08Trust, cost and visibility429 of 434

An AI feature in production, with the receipts

A real feature real people can use, with an eval suite that can fail, a written cost model, and a switch that turns it off.

Solo project300 minAI inside

Done when

  • Live and usable by someone other than you, with a clear statement of what it does and does not guarantee
  • An eval suite of at least twenty cases drawn from real inputs, runnable with one command, with at least three cases that failed the first implementation
  • Structured output validated on arrival, with a defined behaviour for refusals and truncation
  • A cost model in the README: tokens per action, cost per active user per month, and the usage pattern that breaks it
  • Streaming or staged progress, with working cancellation that actually stops the request
  • Per-user rate limiting, a global spend ceiling, and a kill switch that does not require a deploy
  • Tracing that gets you from a complaint to the exact request, its context and its cost

Nobody marks this for you. It goes into your phase checkpoint, where a person does.

You should now be able to

  • Ship an AI feature to a live URL
  • Prove quality with an eval suite over real inputs
  • Publish a cost model and hold it under load
Ask the community

Loading…

← PreviousShipping non-determinism to real peopleTrust, cost and visibilityNext →A red-team pass on your own featureTrust, cost and visibility