Stable Systems announces DesignEval: an open benchmark for layout fidelity and token accuracy

AI · 4 min read

DesignEval includes a corpus of annotated screens, sketches, and real-world UI artifacts with ground-truth component boundaries, style tokens, and accessibility labels. Metrics measure bounding-box precision, token stability across iterations, and propensity to invent nonexistent elements.

Stable Systems wants DesignEval to be a go-to benchmark for vendors and researchers creating layout or UI-aware models. The benchmark also provides standardized evaluation scripts and a leaderboard with per-task breakdowns to identify strengths and weaknesses (e.g., icon recognition vs typography fidelity).

By publishing the dataset and tools under permissive terms, Stable Systems hopes to accelerate progress and give design teams an objective way to choose models for production workflows. They encourage contributors to add more languages, interaction patterns, and device-specific layouts to broaden coverage.