Conversational Search vs Traditional Queries: ShopSense's Checkout Search Redesign

AI · 7 min read

ShopSense's checkout optimization team observed that many users abandoned the flow while trying to find discount terms, shipping options, or support articles. Their existing search required precise keywords and often returned too many results. The team built a conversational search prototype that accepted natural language prompts, summarized options, and offered one-tap actions like apply coupon or change shipping speed.

Designers limited the model's scope to transactional intents to avoid hallucination, backed responses with exact matches from the knowledge base, and surfaced confidence scores when an interpretation was ambiguous. The experiment ran as an A/B test: half of checkout traffic saw the conversational experience, the rest used the classic search bar. Success metrics included task completion, time-to-action, and subsequent support interactions.

The conversational variant improved first-query success by 33 percent and reduced time-to-action by 24 percent, with a small uptick in users relying on the explicit 're-run as keywords' option when confidence was low. The team learned that constrained, explainable conversational interfaces can outperform free-text search in high-stakes flows if designers couple them with transparent fallbacks and clear provenance for responses.