Designing for Ambiguity: A UX Case Study of an LLM-First Research Tool

AI · 6 min read

The product aimed to let researchers upload transcripts and ask open-ended questions of a large language model, but early users reported inconsistent outputs and felt unsure how to guide the system. Research showed two core problems: users had unclear mental models about what the AI could and couldn't do, and the interface offered too few levers for controlling result fidelity and style.

Designers introduced a structured prompt builder with presets (summarize, find themes, generate quotes) and an “explainability strip” that surfaced provenance: source excerpts, confidence indicators, and edit history for generated claims. The team also added a slider that traded concision for literalness and a quick toggle to require direct quotes vs. model-synthesized summaries.

Post-launch metrics indicated users spent more time in iteration (positive signal) and reduced manual cleanup by 35% when using the presets. Qualitative feedback praised the transparency strip for trust-building. The case study emphasizes the importance of building controls that reflect real epistemic trade-offs and designing affordances that allow experts to push the model rather than be surprised by it.