Facial Recognition and the Brain

Congratulations! You are a face expert. You’ve been training for it your entire life.

Faces are some of the most common things we see. The more you see certain things, the more visual experience you gain with them. In fact, people are such experts with faces that our brains see faces even when they shouldn’t. That’s why you might see power outlets that look like they’re frowning, cars that look happy, or even faces on the surface of Mars. 

 

These are examples of pareidolia. This means seeing patterns where they don’t exist. Pareidolia happens a lot for faces because of how our brains are wired.

Did you know? 

In 2004, a customer on eBay spent $28 000 USD on a grilled cheese sandwich that appeared to have the Virgin Mary’s face in it!

How does your brain process what it sees?

Your brain can only make sense of what your eyes tell it. But this communication is more complex than you’d think. Light goes into your eyes and hits a light-sensitive surface at the back. This is called the retina. The retina captures an image of what you’re looking at using photoreceptors. 

But the image your brain sees is upside down! Your eye is convex, or curved outward. When light hits a convex object, it refracts. This flips the image upside-down. You also have a blind spot. That’s the point where the optic nerve (which goes to the brain) connects to the retina. It has no photoreceptors at all.

But you don’t see the world upside-down, or with a hole in the middle of it. Your brain knows it always gets images upside down, so it flips all the information automatically. And when light hits the blind spot in one eye, your brain uses information from your other eye to fill in the gap. 

What you see, in other words, is not always what your eyes are telling your brain. Your brain is always interpreting the information it gets, to give you the best picture. But sometimes it can be fooled.

 

Seeing faces where there aren’t any

Your brain processes most of the objects you see, like cars or houses, with the lateral occipital complex (LOC). The LOC is a part of the brain located in the outward portion of the occipital lobe. The occipital lobe is located near the back of the head. 

But faces seem to have a whole region of the brain dedicated to recognizing them. This is called the fusiform face area (FFA). There is a whole network of brain areas involved in face perception, but we’re going to focus on the FFA here.

 

 

The LOC and FFA are close to each other in the brain, but they deal with information very differently. The LOC thinks about objects in terms of their parts (part-based processing). The FFA thinks about objects as a whole (holistic processing). To understand the difference between these ways of processing information, take a look at the famous Giuseppe Arcimboldo painting below. 
 

Do you see a face or a bunch of fruit and vegetables? If your brain were looking at this image in a part-based way, you’d see individual vegetables and pieces of fruit. That’s how the LOC processes faces and objects — on a part by part basis. 

But if you look at the painting as a whole, or in a holistic way, you can see that the fruit and vegetables make up a face. That’s how the FFA works — by processing the relationships between the parts that make up the face. 

In the painting, you can tell that the pear is the nose and the mushroom is the ear. This is because they are in the correct positions to stand in for these parts of the face. You can see this when you compare the pear and mushroom with each other, and with all the other fruits and vegetables that make up the face. It is understanding these spatial relationships that lets us identify people, just by looking at their faces.

The Expert Brain

The name “fusiform face area” is actually a little misleading. Processing faces is the FFA’s main job, but it does other work as well. Remember how I said that you were a face expert? Well, the FFA processes any type of object that you have visual expertise with. 

For example, in one study, researchers used an fMRI scanner to see what happened in people’s brains when they looked at pictures of birds and cars. People who did a lot of bird watching used their FFAs more than their LOCs when they saw bird pictures. With pictures of cars, most people used mainly their LOCs. But car experts used more of their FFAs when they saw cars. 

One group of researchers was even able to teach people visual expertise with made-up creatures they called “greebles”. As people got better at identifying greebles, they began to use their FFAs more when they looked at them. 

 

 

Why do people use their FFAs more when they have more experience looking at an object? Well, scientists think the FFA is probably really good at telling the difference between members of the same object category. Like different species of birds, different types of cars, or individual humans for example.

Did you know? 

Lesions, or injuries, to the fusiform face area can cause prosopagnosia. Prosopagnosia is an inability to recognize individuals’ faces. This is also called face blindness.

Summing up with an illusion

Faces are pretty special! Our brains are so used to seeing them that we process them differently from most other types of objects. Holistic processing in the fusiform face area of our brains lets us recognize faces quickly and accurately. Even if we do slip up once and awhile! 

I’m going to leave with a famous illusion that shows us the power of holistic processing. Check out these upside-down faces:

Your brain processes upside-down faces in a part-based manner because you’re not used to seeing faces that way. Since both pictures have two eyes, a mouth, and a nose, the pictures look okay. Now let’s flip the same images right side up, so that your brain switches back to holistic processing:

 

The difference is immediate, and even a little terrifying! In the upside down pictures, your FFA was hard at work trying to make sense of the relationships between the different parts of each face. Now that they are right side up, it’s clear that eyes and mouth on the right-hand photograph have been flipped. 

 

This is known as the Thatcher Illusion. That’s because a British professor first used photos of former British Prime Minister Margaret Thatcher to explain it. You can have some fun with the Thatcher Illusion on this website!

Did you know? 

A more modern version of the Thatcher Illusion hit Instagram in 2016, when Adele’s face was shown upside down and photoshopped so that her eyes and mouth were the only parts right-side up.