Single View Reconstruction from profile portraits

Hello everybody,

this fast post is to answer a question my friend and colleague Emanuel Demetrescu asked me. He wanted to know if the software developed by A. Jackson and his equip is working also with profile portraits.

It took me longer than I thought to answer, since it is true that the software has more difficulties in performing this kind of reconstructions and, for a while, my opinion was that it simply did not work. Finally, testing the software with random profile portraits from internet, I tried a picture of a famous actress (Jennifer Lawrence; I hope she does not mind about it, but I am also pretty sure she will never read this post :)).  This portrait was the only photo that worked since that moment and you can see the result here below.

3D reconstruction from a profile photo (Jennifer Lawrence)

Despite some minimal errors in the reconstruction (you can see them in black in the 3D model), the software finally worked, at least with a picture. Now it was the time to test it with some paints. Again I did many test that ended with no result, but than I started to analyse the photo of Jennifer Lawrence to understand why that picture, among many others, was the only one that worked.

My opinion was that, for the software, it is difficult to recognise the face within a profile portrait, probably due to the fact that it has some parameters to check and probably it is looking for two eyes (and does not expect to find just one). So I figured out that the probability to succeed would be higher if the portrait would have some characteristics: profiles with features that makes an area of the face more recognizable; profiles which are not perfectly straight, so that, for instance, it is possible to glimpse the second eye; images with a background very different from the portrait (like in the picture of Jennifer Lawrence, in which the pale face is very visible against a dark background). 

It took me a while to find profile portraits with the two first features in art history, but finally I found them. For the first test I used the face of an angel, painted by Giotto in the Dormitio Virginis. In this paint the angel is blowing on an incense burner, so that his left cheek is very recognizable. In fact the software gave me a result, which you can see here below.

3D profile reconstruction from the Dormition Virginis (Giotto)

For the second test I used a Botticelli's paint and, more precisely, the Portrait of a Young Woman. In this case the profile is not perfectly straight, so I thought there were good chances to succeed. Also in this case the software worked well, giving me the result you can see here below.

3D profile reconstruction from a paint of Sandro Botticelli

After these two positive results with ancient paints, I wanted to do one more test with a picture and especially with one in which the background was very different form the face, so I looked up form other images and I checked directly for photographic portraits. Finally I found a picture of Natalie Portman by Dan Winters. Also in this case the software worked well and here below is the result.

3D reconstruction from a profile portrait of Natalie Portman

In conclusion, after all these test, my opinion is that it is still possible to automatically reconstruct faces from profiles, using the "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression" methodology (and software) proposed by Jackman and colleagues, but the images have to be accurately selected and the success rate is very much lower than working with frontal or three-quarter portraits.

I hope that this post will be useful. Have a nice day!

Faces 3D Single View Reconstruction

Hello everybody,

as you probably know, if you are a regular reader of ATOR, we worked a lot, in the past, on the topic of human faces, getting involved in aFFR projects and organizing exhibitions. 

Archaeological Foresnic Facial Reconstructions (aFFR) was the technique we used most, working with the protocol we developed through the years, thanks to our specialist Cicero Moraes. For instance, we used this methodology for St. Anthony of Padua (CARRARA et al. 2014), St. Catherine of Genua (BEZZI et al. 2019), for the father of pathological anathomy Giovanni Battista Morgagni (ZANATTA et al. 2018), for the medieval poet Petrarch (CARRARA and BEZZI 2018), for the mesolithic man of Mondeval, and for the Ptolemaic mummy of the first priest of Toth in Helipolis (CARRARA and SCATTOLIN 2018). Despite this, aFFR is not the only techniques we used in this field: between 2012 and 2013 we developed a new paleoartistic methodology in order to reconstruct the faces of our ancestors. We called this new methodology "Coherent Anatomical Deformation" (BEZZI 2016) and it is based on a x-ray tomography of a Pan troglodytes (Chimpanzee), adapted to the cranium of different hominess’s. Rarely we used also a third technique: an Iconographic Facial Reconstruction, based on a comparison of historical images of a subject. This is the case, for instance, of the Cardinal Bernardo Clesio (NEBL 2018), whose face was reconstructed on ancient paintings and sculptures, after analysing the common features all the portraits.  

Today I will consider this third techniques, speaking about a software I fond yesterday. This software is pretty recent (2017) and, unfortunately, we did not know it when we were working on the exhibition "Imago animi" (BEZZI et al. 2018), because it would have been useful for the facial reconstruction of Bernardo Clesio. This program is referenced as "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression", which is also the title of the related article (JACKSON et al. 2017). Of course the software is Open Source and here you can find the code.

Yesterday I played around with it and here below are some results. I started with some test concerning the new exhibition we are working in these days, about the history of my hometown: Cles. Our task for the exhibition is to prepare the archaeological session, so I started trying to automatically reconstruct the face of Luigi Campi, a famous archaeologist born in Cles. To do it I used an historical picture and here below is the result.

Luigi Campi: on the left the original picture, on the right the reconstructed face in 3D.

After this first positive test I wanted to try if the software was able to work also with other kind of historical iconographic sources, so I tried to find a good quality paint of someone related with Cles and finally I found the  portrait of Giuseppina Cles, painted by Giovanni Battista Lampi the Elder around 1780/1781. Also this image gave me positive results.

The reconstruction of Giuseppina Cles from an ancient paint

At this point I wanted to know if the software was also able to work with less realistic images, so I tried to reconstruct the face of the Christ Panthocrator of an ancient fresco in the church of St. Vigilio in Pez and here below is the result.

3D facial reconstruciotn of the Christ Panthocrator fo S. Vigilio in Pez

Befor to stop playing with this very interesting software, I wanted to do some more test with a couple of iconic images, so I tried out the portrait of the Boy Eutyches (from Fayum...

3D of Boy Eutyches

 ... and the Monna Lisa.

After all these test, my opinion about the software is that this tool can be very useful for IFR (Iconographic Facial Reconstruction), in order to get fast 3D model of faces from different historical source to automatize the comparison stage between the different portrait and find the common features. This second stage could be done with a face recognition software, able to compare 3D models.

I hope this post will be useful. Have a nice day!

Bibliografy

Bezzi L. (2016). "FACCE. I molti volti della storia umana".

Bezzi L., Bezzi A., Moraes C. (2019). "Ricostruzione Facciale Forense di S. Caterina Fieschi Adorno".

Bezzi L., Carrara N, Nebl M. (2018). "Imago animi. Volti dal passato".

Carrara N., Bezzi L. (2018). "Lo strano caso del cranio di Francesco Petrarca".

Carrara N., Bezzi L., Moraes C. (2014). "Il volto del Santo. La ricostruzione Facciale Forense di Sant'Antonio di Padova".

Carrara N., Scattolin G. (2018). "La mummia del primo sacerdote di Thot" 

Jackson A. S.,  Bulat A., Argyriou V.,  Tzimiropoulos G. (2017). "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression".

Nebl M. (2018). "Il volto di Bernardo Cles".

Zanatta A., Bezzi L., Carrara N., Moraes C., Thiene G., Zampieri F. (2018). "New technique in facial reconstruction: the case of Giovanni Battista Morgagni".