CSCI 2033 Project : Eigenfaces

By: Nayan, Daniel, Cameron, and Mason

Intro:

Humans can abstractly represent faces in their head. A computer can have an image of a face, but it only holds pixel data which isn’t very useful. To get useful information from a picture of a face our computer needs to abstract away the structure of a face. With this structure our computer could be able to compress a face into meaningful features and allow us to do more abstract operations on it like comparison, recognition, and generation.

Method:

To solve this problem we used Eigenfaces. Eigenfaces applies singular value decomposition to a large set of faces to find the structure within them. To do this we first flatten each face into a column vector and put them into a large matrix.

Now we can apply PCA to this matrix and find its principal components. To do this we first take the average of all faces and subtract it from itself . Next we compute the sample covariance matrix from this difference: 

After the sample covariance matrix is calculated, we can calculate its eigenvalues and eigenvectors. Each eigenvector is a principal component of our original matrix, and if we reshape them back into images we’ll get eigenfaces. On the next page there's a grid of eigenfaces ranked by their eigenvalues.

The eigenfaces we computed can be used to represent the abstract structure of any face. This can be done by representing each face as a linear combination of eigenfaces. To compute this for each face we have to solve a linear systems of equations, but we can speed it up by computing a projection matrix and applying it to each face:

Eigenfaces ranked by their eigenvalues

Application:

Facial recognition is an application of eigenfaces. After the eigenfaces for a large dataset is computed,  each face could be broken down into just a linear combination of the first 100 to 200 eigenfaces. Then each face’s combination could be matched with a label and fed to a supervised clustering model like SVM. Our clustering model could then classify new unseen faces to function as a facial recognition model.

Bush, Reconstructed Bush, Differance

To prove eigenfaces are effective, every face was reconstructed with 100 eigenfaces and the difference was taken. Above is an example of this for George W. Bush, on the right is the original photo, the center is the reconstructed face, and on the left is their difference. After the difference was computed for every face, we found that 100 eigenfaces accounts for 99.4% of a face on average. This was computed by taking the mean squared error for 100 eigenfaces and for 0 eigenfaces and their quotient revealed there was a 0.6% error on average for 100 eigenfaces. So we are confident that eigenfaces would be sufficient for facial recognition.

More Eigenfaces demonstrating the importance of the first 100 Eigenfaces

Source Code:

https://colab.research.google.com/drive/1QYeSDwrC1rK1qBwcIf6hZYVudCWvoBun?usp=sharing 

Our first Eigenfaces computed