Showing posts with label SVR. Show all posts
Showing posts with label SVR. Show all posts

Friday, 10 April 2020

Faces 3D Single View Reconstruction

Hello everybody,
as you probably know, if you are a regular reader of ATOR, we worked a lot, in the past, on the topic of human faces, getting involved in aFFR projects and organizing exhibitions. 
Archaeological Foresnic Facial Reconstructions (aFFR) was the technique we used most, working with the protocol we developed through the years, thanks to our specialist Cicero Moraes. For instance, we used this methodology for St. Anthony of Padua (CARRARA et al. 2014), St. Catherine of Genua (BEZZI et al. 2019), for the father of pathological anathomy Giovanni Battista Morgagni (ZANATTA et al. 2018), for the medieval poet Petrarch (CARRARA and BEZZI 2018), for the mesolithic man of Mondeval, and for the Ptolemaic mummy of the first priest of Toth in Helipolis (CARRARA and SCATTOLIN 2018). Despite this, aFFR is not the only techniques we used in this field: between 2012 and 2013 we developed a new paleoartistic methodology in order to reconstruct the faces of our ancestors. We called this new methodology "Coherent Anatomical Deformation" (BEZZI 2016) and it is based on a x-ray tomography of a Pan troglodytes (Chimpanzee), adapted to the cranium of different hominess’s. Rarely we used also a third technique: an Iconographic Facial Reconstruction, based on a comparison of historical images of a subject. This is the case, for instance, of the Cardinal Bernardo Clesio (NEBL 2018), whose face was reconstructed on ancient paintings and sculptures, after analysing the common features all the portraits.  
Today I will consider this third techniques, speaking about a software I fond yesterday. This software is pretty recent (2017) and, unfortunately, we did not know it when we were working on the exhibition "Imago animi" (BEZZI et al. 2018), because it would have been useful for the facial reconstruction of Bernardo Clesio. This program is referenced as "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression", which is also the title of the related article (JACKSON et al. 2017). Of course the software is Open Source and here you can find the code.
Yesterday I played around with it and here below are some results. I started with some test concerning the new exhibition we are working in these days, about the history of my hometown: Cles. Our task for the exhibition is to prepare the archaeological session, so I started trying to automatically reconstruct the face of Luigi Campi, a famous archaeologist born in Cles. To do it I used an historical picture and here below is the result.

Luigi Campi: on the left the original picture, on the right the reconstructed face in 3D.
After this first positive test I wanted to try if the software was able to work also with other kind of historical iconographic sources, so I tried to find a good quality paint of someone related with Cles and finally I found the  portrait of Giuseppina Cles, painted by Giovanni Battista Lampi the Elder around 1780/1781. Also this image gave me positive results.

The reconstruction of Giuseppina Cles from an ancient paint
At this point I wanted to know if the software was also able to work with less realistic images, so I tried to reconstruct the face of the Christ Panthocrator of an ancient fresco in the church of St. Vigilio in Pez and here below is the result.

3D facial reconstruciotn of the Christ Panthocrator fo S. Vigilio in Pez
Befor to stop playing with this very interesting software, I wanted to do some more test with a couple of iconic images, so I tried out the portrait of the Boy Eutyches (from Fayum...

3D of Boy Eutyches
 ... and the Monna Lisa.





After all these test, my opinion about the software is that this tool can be very useful for IFR (Iconographic Facial Reconstruction), in order to get fast 3D model of faces from different historical source to automatize the comparison stage between the different portrait and find the common features. This second stage could be done with a face recognition software, able to compare 3D models.

I hope this post will be useful. Have a nice day!


Bibliografy


Bezzi L., Bezzi A., Moraes C. (2019). "Ricostruzione Facciale Forense di S. Caterina Fieschi Adorno".

Bezzi L., Carrara N, Nebl M. (2018). "Imago animi. Volti dal passato".

Carrara N., Bezzi L. (2018). "Lo strano caso del cranio di Francesco Petrarca".


Carrara N., Scattolin G. (2018). "La mummia del primo sacerdote di Thot

Jackson A. S.,  Bulat A., Argyriou V.,  Tzimiropoulos G. (2017). "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression".

Nebl M. (2018). "Il volto di Bernardo Cles".

Zanatta A., Bezzi L., Carrara N., Moraes C., Thiene G., Zampieri F. (2018). "New technique in facial reconstruction: the case of Giovanni Battista Morgagni".

Tuesday, 19 February 2019

4D reconstruction of Fort "Rocchetta"

Hi everyone,
it's been a long time since I've written a post. Like always the problem is the lack of time :).
I will try to be more present on ATOR and reactivate our blog, since recently some people wrote me that it has been a good resource for free research in archaeology during the last years. I think that, with the help of readers and authors, the blog can start again to share interesting information about Open Research in our discipline.
Today I will start writing about the result of a study we did in 2018, for an exhibition on the World War I in Non Valley. We were asked to work on the archaeological session of the exhibition, which was held in the town of Livo. If someone is interested in this project, here is the article (in Italian), about our session: it is basically an introduction to Modern Conflict Archaeology, with focuses on the several sub-disciplines which are used in this kind of research (Aerial Archaeology, Glacial Archaeology, High Mountain Archaeology, Underwater Archaeology and Speleoarchaeology).
Working on this project, we tried, like always, to keep an Open Research approach, so that, after the good experience of the Open Source exhibition "Imago Animi", we also organized an editathon for Wikipedia, thanks to the help of the wiki community of our region. I will write in another post about this experience. Today I will share one of the main result: the 4D reconstruction of Fort "Rocchetta", which is one of the main structure of the valley, involved in WW1.

One of the view of the reconstruction of Fort "Rocchetta"

The 4D reconstruction has been performed by our 3D specialist Cicero Moraes in Blender. Technically speaking it is a synchronic 4D, since we reconstructed the building at the moment of its construction (around 1860), but we planned a future development into a diachronic 4D (with all the historical steps that modified the structure), after a more accurate research. We speak about 4D, because this kind of archaeological reconstructions are done to recover the three spatial coordinates (x,y,z, better, n,e,z) and the fourth temporal coordinate (t).

Another view of the reconstruction of Fort "Rocchetta"


Currently the fort is in the state of ruins, so that to perform the reconstruction we started with an archaeological survey on the field, integrated with an historical (archival) research, studying the documents of the Archivio di Stato di Trento and of the Österreichisches Staatsarchiv of Wien. Thanks to this research we were able to find some old pictures, and a map, showing the fort from enough different viewpoints to try a SVR (Single View Reconstruction), based on perspective rules.
If you need to use a similar technique for your work, there are several Free/Libre and Open Source Software to perform this kind of 4D reconstruction. One of the first software of this kind was jSVR, but nowadays I would suggest to use the specific Blender addon BLAM, recently evolved into the project fSpy. BTW Cicero preferred to work directly in Blender, without the use of addons.
Here below is the final video we produce, released with an open license (CC-BY).


I hope this post was useful, have a nice day!
BlogItalia - La directory italiana dei blog Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.