Stencil.ai's Experiment: Using LLMs to Auto-Generate Microcopy and Reduce Writer's Block

AI · 5 min read

Stencil.ai, a marketplace for developer tools, piloted an LLM-based microcopy engine to assist product teams in generating CTAs, error messages, and help text. The setup used a small, curated dataset of existing product copy plus brand tone guidelines to fine-tune an open model, and a design review loop that kept a human in the loop for final selection.

During the pilot, the AI generated three copy variants for each CTA or error state. Designers typically used the suggestions as starting points, editing for brevity and brand voice. In two recent checkout experiments, the AI-assisted variants improved click-through by 12% compared with the previous baseline. However, in complex legal and onboarding flows, generated copy often missed nuance, requiring more extensive edits.

The main lesson for product leaders is that generative tools accelerate iteration but should not replace co-design with stakeholders. Stencil.ai formalized a guardrail process: define intent, generate, annotate what was changed, and measure. They also tracked hallucination incidents and added a content QA stage, reducing incorrect claims in UI copy to near zero.