A/B Testing Won't Save You: Lessons from a Marketplace Redesign

Tech · 6 min read

The marketplace had long leaned on A/B testing to eke out conversion improvements, iterating on button copy, color, and microcopy. While many tests produced statistical lifts, the overall transaction funnel stagnated. A cross-functional audit revealed deeper issues: listing flows assumed symmetric expertise between buyers and sellers, and key signals like delivery time and trust indicators were buried.

Designers replaced the experimentation-first approach with problem-led design: ethnographic research with power sellers, cognitive walkthroughs for new buyers, and working sessions to redesign the listing and discovery flow. They introduced asymmetric flows — streamlined listing for sellers with defaults and bulk-edit tools, and a guided discovery flow for buyers that prioritized verified delivery estimates, seller response time, and simplified dispute steps.

Within 90 days the marketplace saw a 20% lift in successful transactions and a reduction in time-to-first-listing for new sellers. The case argues that A/B testing is valuable for optimizations, but structural UX problems require qualitative research and hypothesis-driven redesigns to move the needle.