Redesigning an LLM-Powered Notes App: Two Iterations, One Core Decision

AI · 5 min read

The startup shipped an inline assistant that auto-summarized notes and suggested tags. In iteration one the assistant proactively surfaced summaries and rewrite suggestions at the top of any long note. Early user testing revealed that many writers felt ownership was being taken from them; they either ignored suggestions or edited them out entirely. Usage metrics were low despite high curiosity clicks.

For iteration two the team reframed the assistant as a co-author with adjustable assertiveness. Suggestions were moved into a collapsible ‘Assistant’ rail and labeled with intent and confidence scores. The UI added explicit controls: “Suggest”, “Draft”, and “Refine”. Designers also included a small hint explaining confidence scores and a link to examples showing when to trust suggestions versus when to treat them as drafts.

After the relaunch, suggestion acceptance rose 2.3x and complaints about misleading content dropped sharply. Qualitative interviews showed users trusted the assistant more when it behaved like a partner rather than an authoritative editor. The case underscores a core decision for LLM features: prioritize user agency and transparency, not just model output quality.