Aria AI Chat: Redesigning Assistant Prompts to Improve User Trust

AI · 6 min read

Aria AI, an assistant embedded in a productivity suite, collected user feedback showing low trust in long-form answers and frequent corrections. The product team hypothesized that the presentation of AI-generated content — not only model performance — drove trust issues.

They redesigned the response card to surface provenance (data sources and confidence score), added a one-tap “ask for sources” affordance, and created an inline correction flow where users could flag inaccuracies that fed back to the fine-tuning pipeline. The interface balanced transparency with simplicity by hiding technical details behind expandable cards to avoid overwhelming less technical users.

Post-launch telemetry showed a 47% drop in “inaccurate” feedback reports and a 20% increase in users engaging with the source tool, which correlated with longer session duration. The piece discusses trade-offs: adding provenance increased cognitive load slightly but delivered improved trust and fewer support escalations, validating the team’s decision to iterate on UI-first remedies alongside model improvements.