A newly developed NVIDIA AI can produce photorealistic masterpieces of landscapes from rough sketches. While the outputs were not perfect, the feat takes artificial intelligence technology a step closer to creating synthetic sceneries.
Known as the GauGAN model, the NVIDIA AI was designed to help game designers, city landscape planners, and architects build synthetic images for their projects. According to reports, GauGAN was trained on more than one million images including some 41,000 from Flickr.
The researchers claim that GauGAN acts as a smart paintbrush that fills in the necessary details on a sketch.
NVIDIA AI GauGAN
For GauGAN to create its masterpieces, a user must first draw the segmentation maps of a possible scene. Each segment must be labeled according to its role in the image like sand, sea, sky, snow, trees, pond, or land.
Then, GauGAN can fill in each segment with a corresponding image based on its label. Bryan Catanzaro, NVIDIA vice president for Applied Deep Learning Research, said in a statement:
“It’s like a coloring book picture that describes where a tree is, where the sun is, where the sky is. And then the neural network is able to fill in all of the detail and texture, and the reflections, shadows, and colors, based on what it has learned about real images.”
Even though the NVIDIA AI has no understanding of the physical world, it was structurally designed to have a cooperating pair of networks that allows it to produce realistic results.
It has a generator which creates the images presented to the discriminator. For its part, the discriminator, trained on real images, guides the generator with pixel-by-pixel feedback on how it could make its synthetic images more realistic.
Catanzaro added:
“This technology is not just stitching together pieces of other images, or cutting, and pasting textures. It’s actually synthesizing new images, very similar to how an artist would draw something.”
This is awesome. The features are a real masterpiece.