Technology 2 min read

NVIDIA AI Turns Sketches to Photorealistic Masterpieces

All of these image were synthetically created by NVIDIA's AI. ¦ Image via NVIDIA

All of these image were synthetically created by NVIDIA's AI. ¦ Image via NVIDIA

A newly developed NVIDIA AI can produce photorealistic masterpieces of landscapes from rough sketches. While the outputs were not perfect, the feat takes artificial intelligence technology a step closer to creating synthetic sceneries.

Known as the GauGAN model, the NVIDIA AI was designed to help game designers, city landscape planners, and architects build synthetic images for their projects. According to reports, GauGAN was trained on more than one million images including some 41,000 from Flickr.

The researchers claim that GauGAN acts as a smart paintbrush that fills in the necessary details on a sketch.


For GauGAN to create its masterpieces, a user must first draw the segmentation maps of a possible scene. Each segment must be labeled according to its role in the image like sand, sea, sky, snow, trees, pond, or land.

Then, GauGAN can fill in each segment with a corresponding image based on its label. Bryan Catanzaro, NVIDIA vice president for Applied Deep Learning Research, said in a statement:

“It’s like a coloring book picture that describes where a tree is, where the sun is, where the sky is. And then the neural network is able to fill in all of the detail and texture, and the reflections, shadows, and colors, based on what it has learned about real images.”

Even though the NVIDIA AI has no understanding of the physical world, it was structurally designed to have a cooperating pair of networks that allows it to produce realistic results.

It has a generator which creates the images presented to the discriminator. For its part, the discriminator, trained on real images, guides the generator with pixel-by-pixel feedback on how it could make its synthetic images more realistic.

Catanzaro added:

“This technology is not just stitching together pieces of other images, or cutting, and pasting textures. It’s actually synthesizing new images, very similar to how an artist would draw something.”

Read More: New NVIDIA AI System Creates Video Game Using Dash Cam Vids

First AI Web Content Optimization Platform Just for Writers

Found this article interesting?

Let Chelle Fuertes know how much you appreciate this article by clicking the heart icon and by sharing this article on social media.

Profile Image

Chelle Fuertes

Chelle is the Product Management Lead at INK. She's an experienced SEO professional as well as UX researcher and designer. She enjoys traveling and spending time anywhere near the sea with her family and friends.

Comment (1)
Least Recent least recent
  1. Profile Image
    Isaac Hesson March 20 at 1:25 am GMT

    This is awesome. The features are a real masterpiece.

share Scroll to top

Link Copied Successfully

Sign in

Sign in to access your personalized homepage, follow authors and topics you love, and clap for stories that matter to you.

Sign in with Google Sign in with Facebook

By using our site you agree to our privacy policy.