The Solutioneers Week 4: Turn That Frown Into Code

BACK TO THE SOLUTIONEERS HOME PAGE

On this episode of The Solutioneers, the team is incorporating AI facial recognition software into their robot. They set up a baseline and teach the computer the differences between a smile and a frown. Gwen volunteers to keep a straight face for a whole minute to calibrate the facial markers.

On Maker Space, the Makers walk through setting up and using Microsoft’s cognitive emotion tracking services, allowing their EZ-Robots to track emotions.

And on Future Minds, David Hewlett chats with Nethra Wick, who created an app to sense and prescribe Cognitive Behavioural therapy (CBT). The app takes bodily stimulus and prescribes therapy based on a program interpreting sensory input from a wearable device.

Now it's your turn!

You think you know your family and friends? Can you read their emotions? Could you create a way of recognizing the emotions of your family and friends that a computer could understand? Let’s try it out.

Download the activity as a [PDF]

WHAT DO YOU NEED?

• A device to take pictures (smartphone, digital camera)
• Drawing materials: pencils and unlined paper 

WHAT DO YOU DO?

1. Ask a family member or a friend if you can take pictures of them. You will need to do this with two or more people. If you cannot take pictures of people in person, try using images from a video call.
2. Take five pictures of each person. Those five pictures need to show different emotional states -  showing no emotion (neutral), happy (smiling), sad, surprised and angry. You now have “collected your data.”

   

3. Your next task is to draw five faces. Each face will show one of the five emotional states. In order to draw the angry emotions correctly, you will first analyze the similarities between the angry faces of the different people you photographed. Look at the eyes, eyebrows, noses and mouths. You can also compare the angry pictures with the neutral pictures to help. Try to reproduce those features you found to be similar in your “angry face” drawing.

   

4. Repeat this process for the four other emotional states.
5. Once you have drawn your five faces, you can now test out your “emotion recognition” skills by asking people if they can identify the emotion shown in your different drawings.

WHAT’S THE POINT?

WHAT ELSE?

Go Further!

• Try hiding one part of the face on a drawing. See if your friends can still identify the emotion. What features of the face provide the best information (or data) to show an emotion?
• Can you think of other factors that could give robots a hint of how we’re feeling? (voice)
• Are you artsy? Now that you have figured out what features are important to show emotion, can you draw your own set of custom emojis?

Check out this career that involves machine learning and artificial intelligence.

• Melissa Valdez (Career profile)

Try This!

• The Seeing AI iOS app from Microsoft lets you use your camera or smartphone to describe things and people around you, even their emotions! It can also read and translate text in five languages. This is especially useful for people with visual disabilities.

Partner