Single View Reconstruction from profile portraits

Hello everybody,

this fast post is to answer a question my friend and colleague Emanuel Demetrescu asked me. He wanted to know if the software developed by A. Jackson and his equip is working also with profile portraits.

It took me longer than I thought to answer, since it is true that the software has more difficulties in performing this kind of reconstructions and, for a while, my opinion was that it simply did not work. Finally, testing the software with random profile portraits from internet, I tried a picture of a famous actress (Jennifer Lawrence; I hope she does not mind about it, but I am also pretty sure she will never read this post :)).  This portrait was the only photo that worked since that moment and you can see the result here below.

3D reconstruction from a profile photo (Jennifer Lawrence)

Despite some minimal errors in the reconstruction (you can see them in black in the 3D model), the software finally worked, at least with a picture. Now it was the time to test it with some paints. Again I did many test that ended with no result, but than I started to analyse the photo of Jennifer Lawrence to understand why that picture, among many others, was the only one that worked.

My opinion was that, for the software, it is difficult to recognise the face within a profile portrait, probably due to the fact that it has some parameters to check and probably it is looking for two eyes (and does not expect to find just one). So I figured out that the probability to succeed would be higher if the portrait would have some characteristics: profiles with features that makes an area of the face more recognizable; profiles which are not perfectly straight, so that, for instance, it is possible to glimpse the second eye; images with a background very different from the portrait (like in the picture of Jennifer Lawrence, in which the pale face is very visible against a dark background). 

It took me a while to find profile portraits with the two first features in art history, but finally I found them. For the first test I used the face of an angel, painted by Giotto in the Dormitio Virginis. In this paint the angel is blowing on an incense burner, so that his left cheek is very recognizable. In fact the software gave me a result, which you can see here below.

3D profile reconstruction from the Dormition Virginis (Giotto)

For the second test I used a Botticelli's paint and, more precisely, the Portrait of a Young Woman. In this case the profile is not perfectly straight, so I thought there were good chances to succeed. Also in this case the software worked well, giving me the result you can see here below.

3D profile reconstruction from a paint of Sandro Botticelli

After these two positive results with ancient paints, I wanted to do one more test with a picture and especially with one in which the background was very different form the face, so I looked up form other images and I checked directly for photographic portraits. Finally I found a picture of Natalie Portman by Dan Winters. Also in this case the software worked well and here below is the result.

3D reconstruction from a profile portrait of Natalie Portman

In conclusion, after all these test, my opinion is that it is still possible to automatically reconstruct faces from profiles, using the "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression" methodology (and software) proposed by Jackman and colleagues, but the images have to be accurately selected and the success rate is very much lower than working with frontal or three-quarter portraits.

I hope that this post will be useful. Have a nice day!

Space archaeology

Per aspera ad astra

When you start a new research you know where your path begins, but you do not know where it will end (and where it will take you). 

As many of you knows, we work also with 3D printing of archaeological objects: here (1 and 2) is the two post +Leonardo Zampi wrote about the Taung Project and here is a post regarding some Augmented Reality applications, in one of which a 3D printed skull was used (look the first video).

Most of these experiments are connected with the open source exhibition "Facce. I molti volti della storia umana" (please, do not forget our crowdsourcing campaign: send us your images!). For this event, whose English title is "Faces. The many aspects of human history", we planned to used 3D printed objects for different Augmented Reality applications and to expand the accessibility to the digital exhibit for the visitors (reducing the restrictions connected with disability). This post reports an preliminary overview of the event (done during the European Academic Heritage Day 2013), in which are presented the main topics of the exhibition, the problems and the solutions we planned to apply (sorry, the slides are in Italian; I will translated the text ASAP).

Today, working on a new research for this exposition, I tested different possibilities to reconstruct a 3D from a unique image. Normally our (Arc-Team) work-flow starts with a 3D model obtained from Structure from Motion and Image-Based Modeling (using different software) or from x-ray Computer Tomography (like for the paleoart or mummiology projects), but in archeology can happen to use Single View Reconstruction techniques when there are no other solutions. This post of +Cícero Moraes is a good example of a reconstruction in Blender based on perspective and vanishing points. Of course this technique is optimized for architectural documentation of structures, but is almost unusable for more irregular objects. 

To avoid this problem I studied different possibilities and I decided to use the same software, Blender, but in a different way. I looked in internet for an archaeological picture that could meet my requirements: not too simple, but with a correct light exposition. My problem in finding a good base image comes from the fact that the archaeological artifact photography has codified rules and normally the light source is located in the upper-left corner, otherwise bas-relief (convex) would appear as counter-relief (concave) and vice versa (due to the Hollow-Face optical illusion).

After a while I found this image, which has an almost correct light exposition (sorry, I do not know anymore the source of the photo and I did not find informations about the author).

The base image

I modified the picture with GIMP, in order to obtain a grayscale photo, than I imported it in Blender.

The grayscale image

There I used the "displace" modifier and I automatically obtained a fast 3D of the object (of course nothing comparable with SfM and IBM technique, but enough for my SVR needs).

The displace modifier

After some additional smoothing operation in Blender (you can directly use the related modifier), the model was ready to be saved as an stl file, loaded in Cura and printed in 3D.

The stl file in Cura

At this point I was ready to adapt the entire process to my needs in order to work for the exhibition "Facce", but here is were my research took a complete different way.

On my desk was lying a local newspaper in which was a photo of the Italian astronaut Samantha Cristoforetti, who is actually on board the International Space Station (ISS). Dr. Cristoforetti was born in Milano, but her family is originally form a town (Malé) very close to the one in which I live (Cles) and this is the reason why local press is following her scientific mission very closely. Reading the article I was thinking that it would be nice to print in 3D something that could be a tribute to her work and to the whole mission: something related with space exploration and archeology. Suddenly in my mind appeared a black and white picture, which probably most of you know and that dates back to July 20 1969, so I decided to test the process on this image and see the result.

I searched on the NASA website regarding the Apollo 11 mission and I found what I was looking for: the photo of the first footprints on the moon. I turned the picture into a grayscale image and I repeat the protocol of Single View Reconstruction with this data

The grayscale image

The video below shows all the work-flow and is a new videotutorial for the Digital Archaeological Documentation Project.

Of course the result has no metric, nor topographic value and it is more an artistic reconstruction than a 3D documentation, but this time it was just for fun and for a tribute to woman's contribution in space exploration. BTW on board the ISS astronauts are currently testing 3D printing in space (Made in Space).

If you want to print directly the .stl file I did, you can download it at this link. Otherwise in this post you can find all you need to do the process by yourself.

Have fun!