Educational Resources Lets Talk Science Challenge participants

Do you see a face in that electric socket? (michaeljmc, iStockPhoto)

Do you see a face in that electric socket? (michaeljmc, iStockPhoto)

STEM in Context

Facial Recognition and the Brain

Sarah McCrackin
Format
Video Text Images
Readability
7.7
Subjects

Summary

Learn how your brain recognizes faces and why you sometimes see them in places they don’t exist!

Congratulations! You are a face expert. You’ve been training for it your entire life.

Faces are some of the most common things we see. The more you see certain things, the more visual experience you gain with them. In fact, people are such experts with faces that our brains see faces even when they shouldn’t. That’s why you might see power outlets that look like they’re frowning, cars that look happy, or even faces on the surface of Mars. 

 Front grill of an Austin-Healey Sprite AN5
Front grill of an Austin-Healey Sprite AN5 (Source: Berthold Werner [CC BY-SA] via Wikimedia Commons).

 

These are examples of pareidolia. This means seeing patterns where they don’t exist. Pareidolia happens a lot for faces because of how our brains are wired.

Did you know? 

In 2004, a customer on eBay spent $28 000 USD on a grilled cheese sandwich that appeared to have the Virgin Mary’s face in it!

How does your brain process what it sees?

Your brain can only make sense of what your eyes tell it. But this communication is more complex than you’d think. Light goes into your eyes and hits a light-sensitive surface at the back. This is called the retina. The retina captures an image of what you’re looking at using photoreceptors

But the image your brain sees is upside down! Your eye is convex, or curved outward. When light hits a convex object, it refracts. This flips the image upside-down. You also have a blind spot. That’s the point where the optic nerve (which goes to the brain) connects to the retina. It has no photoreceptors at all.

But you don’t see the world upside-down, or with a hole in the middle of it. Your brain knows it always gets images upside down, so it flips all the information automatically. And when light hits the blind spot in one eye, your brain uses information from your other eye to fill in the gap. 

What you see, in other words, is not always what your eyes are telling your brain. Your brain is always interpreting the information it gets, to give you the best picture. But sometimes it can be fooled.

Test your blind spot
Test your blind spot with this tool from Let's Talk Science

 

Seeing faces where there aren’t any

Your brain processes most of the objects you see, like cars or houses, with the lateral occipital complex (LOC). The LOC is a part of the brain located in the outward portion of the occipital lobe. The occipital lobe is located near the back of the head. 

But faces seem to have a whole region of the brain dedicated to recognizing them. This is called the fusiform face area (FFA). There is a whole network of brain areas involved in face perception, but we’re going to focus on the FFA here.

 

Location of the Fusiform Face Areas and Lateral Occipital Complexes
Location of the Fusiform Face Areas and Lateral Occipital Complexes (© 2020 Let’s Talk Science using an image by VectorMine via iStockphoto).

 

The LOC and FFA are close to each other in the brain, but they deal with information very differently. The LOC thinks about objects in terms of their parts (part-based processing). The FFA thinks about objects as a whole (holistic processing). To understand the difference between these ways of processing information, take a look at the famous Giuseppe Arcimboldo painting below. 
 

Face made of fruit
Do you see a face or a bunch of fruit? (Source: Giuseppe Arcimboldo [Public domain] via Wikimedia Commons).


Do you see a face or a bunch of fruit and vegetables? If your brain were looking at this image in a part-based way, you’d see individual vegetables and pieces of fruit. That’s how the LOC processes faces and objects — on a part by part basis. 

But if you look at the painting as a whole, or in a holistic way, you can see that the fruit and vegetables make up a face. That’s how the FFA works — by processing the relationships between the parts that make up the face. 

In the painting, you can tell that the pear is the nose and the mushroom is the ear. This is because they are in the correct positions to stand in for these parts of the face. You can see this when you compare the pear and mushroom with each other, and with all the other fruits and vegetables that make up the face. It is understanding these spatial relationships that lets us identify people, just by looking at their faces.

The Expert Brain

The name “fusiform face area” is actually a little misleading. Processing faces is the FFA’s main job, but it does other work as well. Remember how I said that you were a face expert? Well, the FFA processes any type of object that you have visual expertise with. 

For example, in one study, researchers used an fMRI scanner to see what happened in people’s brains when they looked at pictures of birds and cars. People who did a lot of bird watching used their FFAs more than their LOCs when they saw bird pictures. With pictures of cars, most people used mainly their LOCs. But car experts used more of their FFAs when they saw cars. 

One group of researchers was even able to teach people visual expertise with made-up creatures they called “greebles”. As people got better at identifying greebles, they began to use their FFAs more when they looked at them. 

 

Three of the greebles
Three of the greebles (Source: Scott Yu, I. Gauthier, I., M.J. Tarr [GFDL 1.3] via Wikimedia Commons).

 

Why do people use their FFAs more when they have more experience looking at an object? Well, scientists think the FFA is probably really good at telling the difference between members of the same object category. Like different species of birds, different types of cars, or individual humans for example.

Did you know? 

Lesions, or injuries, to the fusiform face area can cause prosopagnosia. Prosopagnosia is an inability to recognize individuals’ faces. This is also called face blindness.

Summing up with an illusion

Faces are pretty special! Our brains are so used to seeing them that we process them differently from most other types of objects. Holistic processing in the fusiform face area of our brains lets us recognize faces quickly and accurately. Even if we do slip up once and awhile! 

I’m going to leave with a famous illusion that shows us the power of holistic processing. Check out these upside-down faces:

Thatcher Effect images upside down
Upside-down images of former British Prime Minister Margaret Thatcher (Source: Photography: Rob Bogaerts; Image manipulation: Phonebox [CC0] via Wikimedia Commons).


Your brain processes upside-down faces in a part-based manner because you’re not used to seeing faces that way. Since both pictures have two eyes, a mouth, and a nose, the pictures look okay. Now let’s flip the same images right side up, so that your brain switches back to holistic processing:

Thatcher Effect images right-side up
Right-side up versions of the same images as above (Source: Photography: Rob Bogaerts; Image manipulation: Phonebox [CC0] via Wikimedia Commons).

 

The difference is immediate, and even a little terrifying! In the upside down pictures, your FFA was hard at work trying to make sense of the relationships between the different parts of each face. Now that they are right side up, it’s clear that eyes and mouth on the right-hand photograph have been flipped. 

 

This is known as the Thatcher Illusion. That’s because a British professor first used photos of former British Prime Minister Margaret Thatcher to explain it. You can have some fun with the Thatcher Illusion on this website!

Did you know? 

A more modern version of the Thatcher Illusion hit Instagram in 2016, when Adele’s face was shown upside down and photoshopped so that her eyes and mouth were the only parts right-side up.

Starting Points

Connecting and Relating
  • Do you think you are good at recognizing and remembering faces? Why? 
  • What features of a person’s face do you find easiest to recognize? 
  • Have you perceived faces in objects before? In what objects or where? 
Connecting and Relating
  • Do you think you are good at recognizing and remembering faces? Why? 
  • What features of a person’s face do you find easiest to recognize? 
  • Have you perceived faces in objects before? In what objects or where? 
Relating Science and Technology to Society and the Environment
  • Think of a career that relies on discerning visual differences in objects, artifacts or living things. How might a brain injury to the fusiform face area (FFA) impact a person’s ability to do their job? 
  • Why would it have been an evolutionary advantage for humans to have highly developed facial recognition capabilities? 
Relating Science and Technology to Society and the Environment
  • Think of a career that relies on discerning visual differences in objects, artifacts or living things. How might a brain injury to the fusiform face area (FFA) impact a person’s ability to do their job? 
  • Why would it have been an evolutionary advantage for humans to have highly developed facial recognition capabilities? 
Exploring Concepts
  • What is pareidolia? What basic patterns make a face recognisable?
  • What parts of the brain are used to recognise faces? What role does each part play? 
  • If a person is able to distinguish small visual differences in similar objects, what part of the occipital lobe is most active? 
Exploring Concepts
  • What is pareidolia? What basic patterns make a face recognisable?
  • What parts of the brain are used to recognise faces? What role does each part play? 
  • If a person is able to distinguish small visual differences in similar objects, what part of the occipital lobe is most active? 
Nature of Science/Nature of Technology
  • How has MRI technology affected our knowledge of vision and perception? 

 

Nature of Science/Nature of Technology
  • How has MRI technology affected our knowledge of vision and perception? 

 

Media Literacy
  • Find examples of part-based facial imagery used in different media forms. What is the purpose and/or effect of including this facial imagery? 
Media Literacy
  • Find examples of part-based facial imagery used in different media forms. What is the purpose and/or effect of including this facial imagery? 
Teaching Suggestions
  • This article can be used to support teaching and learning of Biology and Social science related to the human eye, vision and the nervous system. Concepts introduced include pareidolia, retina, photoreceptors, convex, occipital lobe, lateral occipital complex (LOC), fusiform face area (FFA), part-based processing, holistic processing and spatial relationships. 
  • Before reading this article, teachers could provide students with a Vocabulary Preview to engage prior knowledge and introduce new vocabulary. Ready-to-use Vocabulary Preview reproducibles for this article are available in [Google doc] and [PDF] formats. 
  • After reading this article, teachers could have students use an Importance Line learning strategy to help students note and organize the key points from the article. Ready-to-use Importance Line reproducibles for this article are available in [Google doc] and [PDF] formats.
  • For a hands on exploration and demonstration of the Thatcher Illusion, students could create photoshop images of themselves, with flipped eyes and mouths and then share them with the rest of the class or on a school bulletin board. 
  • As a Visual Arts extension, students could create facial images like those of Giuseppe Arcimboldo using real objects or images of objects in a media form of their choice.
Teaching Suggestions
  • This article can be used to support teaching and learning of Biology and Social science related to the human eye, vision and the nervous system. Concepts introduced include pareidolia, retina, photoreceptors, convex, occipital lobe, lateral occipital complex (LOC), fusiform face area (FFA), part-based processing, holistic processing and spatial relationships. 
  • Before reading this article, teachers could provide students with a Vocabulary Preview to engage prior knowledge and introduce new vocabulary. Ready-to-use Vocabulary Preview reproducibles for this article are available in [Google doc] and [PDF] formats. 
  • After reading this article, teachers could have students use an Importance Line learning strategy to help students note and organize the key points from the article. Ready-to-use Importance Line reproducibles for this article are available in [Google doc] and [PDF] formats.
  • For a hands on exploration and demonstration of the Thatcher Illusion, students could create photoshop images of themselves, with flipped eyes and mouths and then share them with the rest of the class or on a school bulletin board. 
  • As a Visual Arts extension, students could create facial images like those of Giuseppe Arcimboldo using real objects or images of objects in a media form of their choice.

Learn more

Does Someone Else Have Your Face? Doppelgängers Explained! (2018)

Video (14:14 min.) from It's Okay TO Be Smart explaining how people can mistake two people and how facial recognition works, for humans and machines.

Neuroscience: why do we see faces in everyday objects? (2014) 

This article from BBC Future features an in-depth look at the phenomenon, debates about its cause and lots of great pictures.

The Thatcher illusion: Are faces special? (2016) 

This article from The Guardian provides a more lengthy explanation of the Thatcher illusion with accompanying illustrations.

How Exactly Do Our Brains Recognize Faces? (2016) 

This Discovery News video (4:03 min.) explains what happens in our brains when we look at faces.

Bringing Recognition to Face Blindness (2010)

In this CBS video (3:10 min.) people with face blindness talk about the condition and how it affects their lives.

References

Gauthier, I. & Tarr, M. J. (1997). Becoming a “Greeble” expert: Exploring mechanisms for face recognition. Vision Research, 37(12), 1673–1682. DOI: 10.1016/s0042-6989(96)00286-6

Gauthier, I., Tarr, M. J., Anderson, A. W., Skudlarski, P., & Gore, J. C. (1999). Activation of the middle fusiform “face area” increases with expertise in recognizing novel objects. Nature Neuroscience, 2(6), 568–573. DOI: 10.1038/9224

Gauthier, I., Skudlarski, P., Gore, J. C., & Anderson, A. W. (2000). Expertise for cars and birds recruits brain areas involved in face recognition. Nature Neuroscience, 3(2), 191–197. DOI: 10.1038/72140

Kanwisher, N., & Yovel, G. (2006). The fusiform face area: A cortical region specialized for the perception of faces. Philosophical Transactions of the Royal Society B: Biological Sciences, 361(1476), 2109–2128. DOI: 10.1098/rstb.2006.1934

Liu, J., Li, J., Feng, L., Li, L., Tian, J., & Lee, K. (2014). Seeing Jesus in toast: Neural and behavioral correlates of face pareidolia. Cortex, 53, 60–77. DOI: 10.1016/j.cortex.2014.01.013

Thompson, P. (1980). Margaret Thatcher: A new illusion. Perception, 9(4), 483–484. DOI: 10.1068/p090483