How Nova's team turned AI hallucinations into UX patterns

AI · 6 min read

Nova built a domain-specific assistant for legal researchers but early pilots showed the model hallucinated citations and statutes. The product team treated hallucinations as a predictable failure mode and designed UX patterns to reduce user reliance on unverifiable outputs.

They implemented three concurrent strategies: (1) explicit provenance UI that highlighted statements backed by sources, (2) a confidence meter tied to internal fact-checking heuristics, and (3) an inline 'verify' action that auto-runs a citation check and surfaces the evidence or correction. These changes were A/B tested against a control with no provenance indicators.

Results were immediate: users who saw provenance indicators corrected 68% of hallucinations themselves versus 24% in the control. Trust metrics shifted — users reported higher clarity about when to trust the assistant, even if overall blunt trust didn't change. Nova then made the verification action a default affordance in the premium workflow.

The team's key takeaway was that UX can make model failure modes manageable without waiting for perfect models. By combining detection signals with interaction pathways for verification and correction, Nova preserved utility while reducing downstream cost for users and the support team.