Single View Reconstruction from profile portraits

Hello everybody,

this fast post is to answer a question my friend and colleague Emanuel Demetrescu asked me. He wanted to know if the software developed by A. Jackson and his equip is working also with profile portraits.

It took me longer than I thought to answer, since it is true that the software has more difficulties in performing this kind of reconstructions and, for a while, my opinion was that it simply did not work. Finally, testing the software with random profile portraits from internet, I tried a picture of a famous actress (Jennifer Lawrence; I hope she does not mind about it, but I am also pretty sure she will never read this post :)).  This portrait was the only photo that worked since that moment and you can see the result here below.

3D reconstruction from a profile photo (Jennifer Lawrence)

Despite some minimal errors in the reconstruction (you can see them in black in the 3D model), the software finally worked, at least with a picture. Now it was the time to test it with some paints. Again I did many test that ended with no result, but than I started to analyse the photo of Jennifer Lawrence to understand why that picture, among many others, was the only one that worked.

My opinion was that, for the software, it is difficult to recognise the face within a profile portrait, probably due to the fact that it has some parameters to check and probably it is looking for two eyes (and does not expect to find just one). So I figured out that the probability to succeed would be higher if the portrait would have some characteristics: profiles with features that makes an area of the face more recognizable; profiles which are not perfectly straight, so that, for instance, it is possible to glimpse the second eye; images with a background very different from the portrait (like in the picture of Jennifer Lawrence, in which the pale face is very visible against a dark background). 

It took me a while to find profile portraits with the two first features in art history, but finally I found them. For the first test I used the face of an angel, painted by Giotto in the Dormitio Virginis. In this paint the angel is blowing on an incense burner, so that his left cheek is very recognizable. In fact the software gave me a result, which you can see here below.

3D profile reconstruction from the Dormition Virginis (Giotto)

For the second test I used a Botticelli's paint and, more precisely, the Portrait of a Young Woman. In this case the profile is not perfectly straight, so I thought there were good chances to succeed. Also in this case the software worked well, giving me the result you can see here below.

3D profile reconstruction from a paint of Sandro Botticelli

After these two positive results with ancient paints, I wanted to do one more test with a picture and especially with one in which the background was very different form the face, so I looked up form other images and I checked directly for photographic portraits. Finally I found a picture of Natalie Portman by Dan Winters. Also in this case the software worked well and here below is the result.

3D reconstruction from a profile portrait of Natalie Portman

In conclusion, after all these test, my opinion is that it is still possible to automatically reconstruct faces from profiles, using the "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression" methodology (and software) proposed by Jackman and colleagues, but the images have to be accurately selected and the success rate is very much lower than working with frontal or three-quarter portraits.

I hope that this post will be useful. Have a nice day!