Vectary plugin converts diffusion outputs into clean 3D meshes and materials

Design · 4 min read

The plugin detects semantic regions in generated images (foreground objects, backgrounds, reflections) and constructs proxy geometry with simplified topology. It produces UVs and base PBR maps while exposing a set of parameters to refine mesh density and seam placement.

Material generation is guided by a small, learnable network that proposes albedo, roughness and normal approximations from the image, along with mask-driven layer suggestions for decals or emissive surfaces. Outputs are optimized for webGL and AR engines.

Vectary positions the plugin as a bridge between purely image-based workflows and interactive 3D experiences, reducing manual retopology and texture bake time. Designers noted that the automated meshes are not production-ready but are excellent for prototyping and rapid concept exploration.