Faces 3D Single View Reconstruction

Hello everybody,

as you probably know, if you are a regular reader of ATOR, we worked a lot, in the past, on the topic of human faces, getting involved in aFFR projects and organizing exhibitions. 

Archaeological Foresnic Facial Reconstructions (aFFR) was the technique we used most, working with the protocol we developed through the years, thanks to our specialist Cicero Moraes. For instance, we used this methodology for St. Anthony of Padua (CARRARA et al. 2014), St. Catherine of Genua (BEZZI et al. 2019), for the father of pathological anathomy Giovanni Battista Morgagni (ZANATTA et al. 2018), for the medieval poet Petrarch (CARRARA and BEZZI 2018), for the mesolithic man of Mondeval, and for the Ptolemaic mummy of the first priest of Toth in Helipolis (CARRARA and SCATTOLIN 2018). Despite this, aFFR is not the only techniques we used in this field: between 2012 and 2013 we developed a new paleoartistic methodology in order to reconstruct the faces of our ancestors. We called this new methodology "Coherent Anatomical Deformation" (BEZZI 2016) and it is based on a x-ray tomography of a Pan troglodytes (Chimpanzee), adapted to the cranium of different hominess’s. Rarely we used also a third technique: an Iconographic Facial Reconstruction, based on a comparison of historical images of a subject. This is the case, for instance, of the Cardinal Bernardo Clesio (NEBL 2018), whose face was reconstructed on ancient paintings and sculptures, after analysing the common features all the portraits.  

Today I will consider this third techniques, speaking about a software I fond yesterday. This software is pretty recent (2017) and, unfortunately, we did not know it when we were working on the exhibition "Imago animi" (BEZZI et al. 2018), because it would have been useful for the facial reconstruction of Bernardo Clesio. This program is referenced as "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression", which is also the title of the related article (JACKSON et al. 2017). Of course the software is Open Source and here you can find the code.

Yesterday I played around with it and here below are some results. I started with some test concerning the new exhibition we are working in these days, about the history of my hometown: Cles. Our task for the exhibition is to prepare the archaeological session, so I started trying to automatically reconstruct the face of Luigi Campi, a famous archaeologist born in Cles. To do it I used an historical picture and here below is the result.

Luigi Campi: on the left the original picture, on the right the reconstructed face in 3D.

After this first positive test I wanted to try if the software was able to work also with other kind of historical iconographic sources, so I tried to find a good quality paint of someone related with Cles and finally I found the  portrait of Giuseppina Cles, painted by Giovanni Battista Lampi the Elder around 1780/1781. Also this image gave me positive results.

The reconstruction of Giuseppina Cles from an ancient paint

At this point I wanted to know if the software was also able to work with less realistic images, so I tried to reconstruct the face of the Christ Panthocrator of an ancient fresco in the church of St. Vigilio in Pez and here below is the result.

3D facial reconstruciotn of the Christ Panthocrator fo S. Vigilio in Pez

Befor to stop playing with this very interesting software, I wanted to do some more test with a couple of iconic images, so I tried out the portrait of the Boy Eutyches (from Fayum...

3D of Boy Eutyches

 ... and the Monna Lisa.

After all these test, my opinion about the software is that this tool can be very useful for IFR (Iconographic Facial Reconstruction), in order to get fast 3D model of faces from different historical source to automatize the comparison stage between the different portrait and find the common features. This second stage could be done with a face recognition software, able to compare 3D models.

I hope this post will be useful. Have a nice day!

Bibliografy

Bezzi L. (2016). "FACCE. I molti volti della storia umana".

Bezzi L., Bezzi A., Moraes C. (2019). "Ricostruzione Facciale Forense di S. Caterina Fieschi Adorno".

Bezzi L., Carrara N, Nebl M. (2018). "Imago animi. Volti dal passato".

Carrara N., Bezzi L. (2018). "Lo strano caso del cranio di Francesco Petrarca".

Carrara N., Bezzi L., Moraes C. (2014). "Il volto del Santo. La ricostruzione Facciale Forense di Sant'Antonio di Padova".

Carrara N., Scattolin G. (2018). "La mummia del primo sacerdote di Thot" 

Jackson A. S.,  Bulat A., Argyriou V.,  Tzimiropoulos G. (2017). "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression".

Nebl M. (2018). "Il volto di Bernardo Cles".

Zanatta A., Bezzi L., Carrara N., Moraes C., Thiene G., Zampieri F. (2018). "New technique in facial reconstruction: the case of Giovanni Battista Morgagni".

“FACCE. I molti volti della storia umana”, an open source exhibition

It's been more than one year since the conclusion of the Taung Project and many things have happened. Today I want to post something related with one of the most important derivations of this project: the exhibition "FACCE. I molti volti della storia umana" (en "Faces. The many aspects of human history"). The event is organized by the Anthropological Museum of the University of Padua, from an idea of Dott. Nicola Carrara, and it is a cooperation between the Museum, the University, Arc-Team and Antrocom NPO (the same actors of the Taung Project). I will not go into details of the exhibition, due to the fact it is still a work in progress being its opening planned for October 2014, but I post the preliminary presentation we did during the European Academic Heritage Day 2013 ("Down the Rabbit hole", backstage of knowledge production).

 As you ague form the title, the main topic of the exhibition is the human face, which, in the words of the French photograper Gisele Freund, is also the "the only part of the body to be exposed naked to the first comer." Actually the thematic sessions planned for the event are six:

1. "Guardiamo in faccia la diversità umana" (en "Let's face human diversity"), which is dedicated to human evolution and will take advantage of the forensic facial reconstructions of the principal actors in the phylogenetic human three, preformed by the Brazilian digital artist +Cícero Moraes.

2. "Una faccia, una razza?" (en "One face, one race?"). This section will show the division of mankind into races proposed by various authors in the 700-800, which will have more deleterious effects in the first decades of the 1900s. Nowadays, thanks especially to human genetic studies, we can say that the division of Homo sapiens into races is just a cultural and social construction, without any sense in biology.

3.  "Una faccia, un destino?" (en "A face, a fate?"), in which the main topic will be physiognomy, analyzed from the artistic aspects of the XVI century till the pseudoscientific approach in the XIX century criminology in connection with phrenology, another famous pseudoscience, well-accepted in the same period.

4. "Con quella faccia un po' così" (quote from the popular Italian song "Genova per Noi" of the singer Paolo Conte), a session in which will be proposed different forensic facial reconstructions of historical personalities connected with the city of Padua.

5. "Ma che faccia fai?" (Italian popular expression). This part of the exhibition will be dedicated to human facial expressions.

6. "Dalla faccia alla maschera: il volto simbolico" (en "From the face to the mask: the symbolic aspect"). The final session will take advantage from the contribute of the Museo Internazionale della Maschera, Amleto e Donato Sartori and will show the symbolic aspect of the face: the mask.

The contribute of Arc-Team to the exhibition, in addition to the forensic facial reconstructions that will be created by our expert (+Cícero Moraes), will cover the touristic fruition of the event through scientific interactive installations mainly based on Computer Vision (from Augmented Reality to Facial Recognition and Motion Capture). In the presentation you can see some of the projected prototype, as well as some sources of inspiration (e.g. the movie "A scanner Darkly") some examples from the software we will use (e.g. Openframeworks). 

As you probably guessed from the title of this post, the main peculiarity of this event will be the fact that it can be considered an "open source exhibition", maybe the first of its kind, due to the fact that all the produced material will be released under the therms of the Creative Commons Attribution, which is a license approved for free cultural works. Moreover open techniques will be used also to collect material for the exhibition itself, with specific crowedsourcing campaigns, while part of the budget will be probably collected wit crowdfunding. I will have time to post more news about these last two topics, by now I just hope that you'll enjoy the presentation below (like always, to navigate just click on the image and use the spacebar to browse the slides).

For a better visualization, click here