Project 3 Webpage
Selecting facial keypoints and creating a triangulation mesh of these points:
The way I implemented this part of the project was first by using the correspondence tool provided. I used a picture of me and my roommate and I labeled corresponding points. The points that I selected were relatively arbitrary: Parts of the jaw, ears, nose contour, cheekbone area, lips, and eyes. I then saved these points in a json file.
For actually creating a triangulation mesh, I used the Delaunay triangulation method that was suggested. This ensured that my triangle mesh was not overly skinny.
The first picture is output of the triangulation method on me:
Triangulation on my roommate:
Affine warp to create a midway face between your two images:
The design and implementation details for this part are structured pretty closely to the given instructions.
I first calculated the average between the corresponding points. Since they are numpy arrays, this was done in a single line of code.
I then warped both faces into that shape. This was done separately for each image. For an image, I first computed the Delaunay triangulation of the averaged points and actual points. Then for each triangle in the triangulation, I used my implementation of the computeAffine function. This function computes the affine transformation matrix for two triangles. Thus the problem boiled down to solving the least squares system and compute the matrix that transforms one triangle to another.
Then each triangle undergoes the transformation and its corresponding pixels get mapped to the “average location”. I do the process on both images and then add the images to each other and then divide by 2.
This is the output:
This looks pretty accurate in my opinion besides the difference in our smile and hairstyle
This part was essentially a repeated part of the last part but a bunch of times.
I essentially had a parameter that controlled how much the output image looked like one person or another. As such, I iterated through these alphas and computed a morphed image at that location. I maintained an array of intermediate images. I simply stitched these together to create a gif as seen below:
(Not sure why the black things are appearing when I upload here but it doesn’t show on local machine)
The mean face was computed by using a similar process to part 2 and 3. Instead of varying an alpha between 0 and 1, I simply added all of the images together without an alpha. Then I normalized it by dividing by the number of images. This essentially gave us the average image of the population. My population was 40 images from the Danes dataset, the first link provided. Since they are prelabeled, we can simply use those points for our dataset rather than relabeling everything.
This is the “mean” face of the population
Here are a couple of examples of merging an example face with this above mean face:
Example 1:
Example 1 Morphed:
Example 2:
Example 2 Morphed:
Example 3:
Example 3 Morphed:
The average face warped to my geometry (i blended the images as well)
Only warped to my geometry and not my face:
Other way around: my face on the mean face geometry (blended):
My face on the mean face geometry only:
Caricature of your face by extrapolating from the population mean you calculated in the last step.
To do this, I first saved a copy of the population mean_face from part 4. I then try to do the same thing I did in part 4 with applying my face to the geometry of the mean face. It is a similar process here but instead of simply moving my face to the sample image coordinates, I use a parameter alpha that captures some level of movement from my facial geometry to the mean face image. Thus, instead of interpolating like we did in part 4, we extrapolate and make the differences larger by multiplying the difference by some factor (1 + alpha). Thus my face will be exaggerated and look a lot like the population mean.
Here is the caricatures of my face at varying levels of alpha:
0: (Output from part 4):
0.25:
0.5:
Another caricature I was able to make by exaggerating the features of my face (particularly my eyes and my lips) from the population mean. I did this by first selecting people from the population who I thought had big smiles and eyes. The different facial expressions also helped in the Danes dataset.
An even more exaggerated version:
It becomes apparent that when we start extrapolating too much, the blockiness becomes very obvious, thus increasing the need for more correspondence points. For interpolation it isn’t as noticeable.
For my B&W, I want to overlay a feature that I don’t quite have fully, which is a beard. I want to see what I might look like with more facial hair like the snapchat filter. As such, I selected a subset of people from the dataset that had beards.
I ran the algorithm from part 4 on this subset and produced the following image.
This was the resulting image.
Combining this with me yielded the following result:
Nice! I have a very nice looking beard now!
Here I made a morphing video from pictures of me from approximately the last 4 years. It was cool to see how much my face has changed.
The way I implemented this was creating a new make_gif function that essentially stitches together 3 different morphs (from 4 different images).
This was a pretty fun one to make as it took after my attempt in part 5.
I essentially wanted to bloat my face from a given point. In this case, I did it from the center but it can be changed.
Since the bloating is circular in nature, I calculated the displacement from the given center for every pixel. For pixels closer to my center, I wanted to bloat them more than pixels farther away.
I calculated this distance by normal euclidean distance. Name this distance d.
I calculated the new distance d’ = d * (1 + 0.01 * d^(0.5)) Thus farther away values will be distorted less. I then computed the new distance of where to place this pixel by simply scaling the difference vector by d’ / d. This allowed for the following result.
No key points were used in this implementation.
I can change the center pretty easily:
