ShadowPlay: Using our hands to have some fun with AI

Editor's note:TensorFlow, our open source machine learning platform, is just that—open to anyone. Companies, nonprofits, researchers and developers have used TensorFlow in some pretty cool ways and at Google, we're always looking to do the same. Here's one of those stories.

Chinese shadow puppetry—which uses silhouette figures and music to tell a story—is an ancient Chinese art form that's been used by generations to charm communities and pass along cultural history. At Google, we're always experimenting with how we can connect culture with AI and make it fun, which got us thinking: can AI help put on a shadow puppet show?

So we created ShadowPlay, an interactive installation that celebrates the shadow puppetry art form. The installation, built using TensorFlow and TPUs, uses AI to recognize a person's hand gestures and then magically transform the shadow figure into digital animations representing the 12 animals of the Chinese zodiac and in an interactive show.

Attendees use their hands to make shadow figures, which transform into animated characters and creates.

We debuted ShadowPlay at the World AI Conference and Google Developers Day in Shanghai in September. To build the experience, we developed a custom machine learning model that was trained on a dataset made up of lots of examples of people's hand shadows, which could eventually recognize the shadow and match it to the corresponding animal. "In order to bring this project to life, we asked Googlers to help us train the model by making a lot of fun hand gestures. Once we saw the reaction of users seeing their hand shadows morph into characters, it was impossible not to smile!", says Miguel de Andres-Clavera, Project Lead at Google. To make sure the experience could guess what animal people were making with high accuracy, we trained the model using TPUs, our custom machine learning hardware accelerators.

• Shadowplay_AI.gif

  The ShadowPlay installation brings to life Chinese shadow puppetry with AI
  

• Shadowplay_AI_2.jpg

  A user can make shadow puppets from the Chinese zodiac, like this tiger, which the AI will recognize and turn it into an animated figure.
  

• Shadowplay_AI_3.jpg

  An attendee uses her hand to make a shadow figure, which the machine learning system recognizes as a dragon.
  

• Shadowplay_AI_4.jpg

  An attendee's shadow figure, recognized by the machine learning system, transforms into an animated silhouette
  

We had so much fun building ShadowPlay (almost as much fun as practicing our shadow puppets … ), that we'll be bringing it to more events around the world soon!

#Google #Android #Smartphones #OS #News @ndrdnws #ndrdnws #AndroidNews