Google's Virtual Try-On Fuses Fashion with Generative AI

Google uses massive data set to train a new image diffusion model

Google announced a new Virtual Try-on feature to help shoppers view models like themselves in clothes prior to purchase. Lilian Rincon, senior director of product for Google Shopping commented in a blog post:

While apparel is one of the most-searched shopping categories, most online shoppers agree: It’s hard to know what clothes will look like on you before you buy them. Forty-two percent of online shoppers don’t feel represented by images of models, and fifty-nine percent feel dissatisfied with an item they shopped for online because it looked different on them than expected.

Now, thanks to our new virtual try-on tool on Search, you can see whether a piece is right for you before you buy it.

Generative Modeling

Viewing clothes on virtual models or even based on a photo of a shopper is not new. Google has set out to produce higher-fidelity rendering to help overcome shopper reticence about how clothes will look when delivered by promising more realistic renderings. Rincon added:

Virtual try-on for apparel shows you how clothes look on a variety of real models. Here’s how it works: Our new generative AI model can take just one clothing image and accurately reflect how it would drape, fold, cling, stretch and form wrinkles and shadows on a diverse set of real models in various poses. We selected people ranging in sizes XXS-4XL representing different skin tones (using the Monk Skin Tone Scale as a guide), body shapes, ethnicities and hair types.

Ira Kemelmacher-Shlizerman, a senior staff research scientist for Google Shopping, added more detail about the AI model training in another blog post:

Current techniques like geometric warping can cut-and-paste and then deform a clothing image to fit a silhouette. Even so, the final images never quite hit the mark: Clothes don’t realistically adapt to the body, and they have visual defects like misplaced folds that make garments look misshapen and unnatural.

So when we set out to build a new VTO feature, we were committed to generating every pixel of a garment from scratch to produce high-quality, realistic images. We found a way with our new diffusion-based AI model.

…

To make our VTO feature as helpful and realistic as possible, we put the new AI model through rigorous training. But rather than training it with an LLM (like Imagen does), we tapped into Google’s Shopping Graph, the world’s most comprehensive data set of the latest products, sellers, brands, reviews and inventory.

The Google Shopping Graph claims more than 35 billion product listings. It is designed to help shoppers find items using specific search criteria. The Graph also provides a rich data set to use for model training. And this emphasizes one more advantage Google has over rivals when expanding its generative AI features. Its datasets are unmatched.

Google said shoppers can check out the new solution and virtually try on apparel from Anthropologie, LOFT, H&M, and Everlane, with many more coming.

What this Means

For Google, Virtual Try-On (VTO) represents a practical application for a single aspect of one of its many generative AI research projects. This research around image diffusion models has a clear commercial application.

However, it is unclear whether consumers will notice a meaningful difference in their shopping experience or if it will lead to higher sales for clothing retailers. VTO is definitely interesting and worth monitoring.