In 2003, Tsao and her collaborators discovered that certain regions in the primate brain are most active when a monkey is viewing a face. The researchers dubbed these regions face patches; the neurons inside, they called face cells. Research over the past decade had revealed that different cells within these patches respond to different facial characteristics. For example, some cells respond only to faces with eyes while others respond only to faces with hair.

“But these results were unsatisfying, as we were observing only a shadow of what each cell was truly encoding about faces,” says Tsao. “For example, we would change the shape of the eyes in a cartoon face and find that some cells would be sensitive to this change. But cells could be sensitive to many other changes that we hadn’t tested. Now, by characterizing the full selectivity of cells to faces drawn from a realistic face space, we have discovered the full code for realistic facial identity.”

Two clinching pieces of evidence prove that the researchers have cracked the full code for facial identity. First, once they knew what axis each cell encoded, the researchers were then able to develop an algorithm that could decode additional faces from neural responses. In other words, they could show a monkey a new face, measure the electrical activity of face cells in the brain, and recreate the face that the monkey was seeing with high accuracy.

Second, the researchers theorized that if each cell was indeed responsible for coding only a single axis in face space, each cell should respond exactly the same way to an infinite number of faces that look extremely different but all have the same projection on this cell’s preferred axis. Indeed, Tsao and Le Chang, postdoctoral scholar and first author on the Cell paper, found this to be true.

“In linear algebra, you learn that if you project a 50-dimensional vector space onto a one-dimensional subspace, this mapping has a 49-dimensional null space,” Tsao says. “We were stunned that, deep in the brain’s visual system, the neurons are actually doing simple linear algebra. Each cell is literally taking a 50-dimensional vector space—face space—and projecting it onto a one-dimensional subspace. It was a revelation to see that each cell indeed has a 49-dimensional null space; this completely overturns the long-standing idea that single face cells are coding specific facial identities. Instead, what we’ve found is that these cells are beautifully simple linear projection machines.”

http://www.caltech.edu/news/cracking-code-facial-recognition-78508