Designing for graceful wrongness

A deterministic feature is either right or broken, and the interface can be confident. A probabilistic one is right most of the time, which is a different design problem: the user has to be able to tell when it went wrong and recover cheaply. That means edits in place rather than regenerate-only, visible sources for claims, undo everywhere, and no irreversible action taken on a model output without a human in the loop.

Confidence in the interface should track reliability in the system. A feature that is right eighty per cent of the time and presented as an authoritative answer will burn trust faster than one presented as a draft, and once trust is gone, users stop reading the output at all, which is worse than them never having had the feature.

The most important line is the one between suggestion and action. Drafting an email is a suggestion. Sending it is an action. Categorising a ticket is a suggestion; issuing the refund is an action. Keep model output on the suggestion side of that line unless the cost of being wrong is genuinely trivial, and if you cross it, make the action reversible and logged.