When to Let LLMs Suggest UX Patterns: Fintech Startup A/B Results

AI · 4 min read

Aurora Payments integrated an LLM into their design toolchain to generate alternative microcopy, label suggestions, and small layout variants for their KYC screens. The LLM pipeline produced dozens of concise copy variants and three layout suggestions per screen, which designers then curated into experiment buckets. The goal was faster ideation while keeping human oversight.

In a 12-week A/B test across 18,000 new signups, the AI-suggested copy improved form completion by 4.2% over the baseline, driven largely by clearer CTA phrasing and shorter error messages. However, several AI-suggested layouts performed worse, particularly those that compressed legal copy or moved critical trust signals below the fold. Designers removed those variants before broader rollout.

The team concluded that LLMs speed up iteration for microcopy and can surface novel phrasing, but they are not a substitute for design judgment on layout, visual hierarchy, and regulatory content. Operationally, the company established a guardrail policy: all AI outputs must be reviewed by a designer and pass an accessibility and compliance checklist before deployment.