Kinect, a sleeping research branch reacivated

As you probably noticed, one of the topic of ATOR is related with hardware hacking, with the aim to build new archaeological devices from ordinary objects and tools (33).

This concept is close to the one of "reuse" (using an artefact for a purpose which is completely different from the original function), a pretty common phenomenon in archaeology; also in architecture there is something similar,  called "spolia" (but maybe our interest in hacking things is just a kind of McGiver syndrome of people grown up in the 80s).
However, this post is about hacking a common game device like Kinect to use its characteristic in archaeological 3D real-time documentation. If you are a regular reader of ATOR, you will know that we already face this challenge, performing a first test (1) with RGBDemo in February 2012, and controlling accuracy and precision of the device in March of the same year (2), after a discussion with some of the researchers of FBK, during the workshop "Low cost 3D: sensori algoriti e applicazioni". Due to the encouraging results achieved in our first experiments, we worked on the hardware in order to modify it for outdoor projects (3), but soon we experimented the limits of this technology when applied in areas with direct sunlight (4) or in documenting small objects (5, 25). Despite this drawbacks in our research, Kinect worked pretty good in indoor excavations (6), helping us in difficult situations (related with the the workplace safety), and for particular purposed, like for infra-red prospections in dark environment (7).
After all these experiences, our final advice about Kinect is that the device has a potential in archaeology, but its real employment in professional work is restricted to peculiar conditions, while in most of the cases the SfM-based techniques are the best option (due to their versatility, which makes them a perfect choice during missions abroad (8), for small finds documentation (9, 10), for underwater and aerial archaeology (11, 12, 13), considering also the speed which characterize SfM and MVSR open source software development (14) and the wide range of possibilities between the different tools (15, 16).
Well, at least this was our opinion until now... Currently we are changing our mind about Kinect, and this is due to our professional engagement in underground archaeology (17) and to our renovate interest in robotics. Let's deal with these two points separately.

Underground archaeology

Documenting an underground semisubmerged structure in Firuzabad (Iran)

Like any other operation in archaeological 3D documentation, the tolerance regarding accuracy and precision is variable and influenced by some factors, and mainly: research purposes, logistics, characteristics of the structures to be documented.
Without considering some important exceptions (e.g prehistoric rock shelter, which are often simple to document with SfM techniques), most of the structures related with underground archaeology (WW1 artificial caves, medieval mines, etc...) are connected with large scale survey projects (where it is important a "big data" approach, raising the tolerance in data acquisition to increase the number of documented structures); with logistically difficult areas (high mountains, glaciers, (18, 19) etc...); with structures often characterized by vast surfaces without important small details, which (when present) can be recorded with a targeted SfM or RTI (21, 22) documentation (e.g. for graffiti, inscriptions (20), manufacture traces, etc...). For this reason, in most of these projects, it is necessary to deal with precision in documenting (keeping checkpoints thanks to other TOF instruments, like total stations) in order to gain a real-time response from the selected device, and, under this point of view, Kinect is often a good solution, considering also that its infrared sensor helps very much in low light conditions (7).

Documenting WW1 caves in Southtirol (Italy)

Archeorobotics

Arc-Team's UAV during an aerial archaeology project in Storo (Trentino - Italy)

Since 2006, when we joined an aerial archaeological project in Armenia (23), we started to work on "archaeorobotics", trying to develop robotic devices able to help us in the most difficult archaeological missions.
The first positive results we reached in this field were related with aerial archaeology and the building of an open hardware UAV (in 2008), even if at the beginning we underestimated the time needed to practice with our new tool  (24). Soon our experience increased as we built different drones, based on open and closed solutions (like kk multicopter (26) or Naza dji (27) models). The benefits of this research branch were clear (28, 29) and soon other research institutions, like the CNR-ITAB of Rome (30), the University of Lund (31) and  the CNR-ISTI of Pisa (32), asked us to give lessons about this topic.
Another field of archaeorobotics we explored is the one related with CNC machines and especially with 3D Printers. For this topic a precious help came from the society Kentstrapper and Leonardo Zampi (aka +Exekias 87), who helped us in 3D printing the cast of the Taung Child (34, 35). Since RepRap project started (in 2005), 3D printers evolved very fast. Of course our interest regarding these machines is mainly oriented to Cultural Heritage, and this is also the reason why we built a Fa)(a 3D form scratch (36), but results with this kind of instruments can be very impressive, especially considering the wide range of scientific applications (37, 38, 39, 40, 41), even if sometimes you have to deal with difficult boolean operations (42).
However, none of the robotic projects we developed till now needed Kinect, being based on UAV, to 3D document archaeological sites, or on CNC machines, to fast replicate archaeological artefacts. Our renovate interest in Kinect for archeorobotics is due to our new challenge in developing a ROV (Remotely Operated Vehicle), in order to assist us in our underwater archaeological missions. Indeed, in the last months, we started a collaboration with the WitLab, the FabLab of Rovereto (Trentino - Italy), to develop a new Open Hardware ROV, especially designed for archaeological aims. One of the main topic in developing such an instrument is that the new robot will be oriented not only to 3D documentation, but also to the exploration of unknown areas. For such reason SfM and MVS software are no more enough, but we had to start again in testing Open Source SLAM (Simultaneous  Localization And Mapping) algorithms, due to the fact that we need to register in 3D the submerged landscape (Mapping), but also to recover the path the "ArcheoROV" did (Localization) to reach new hidden archaeological evidences (for a better planning of human operations).

Testing the ArcheoROV at night

Testing Open Source SLAM solutions

The importance of SLAM algorithms in exploring devices is the main reason why we started again to experiment Kinect. Indeed, despite Kinect cannot be used as an on-board optical device in our ArcheoROV (due to the infrared camera), this tool is the perfect system to check SLAM software.
If, you ever started in working on robotics, probably sooner or later you stepped into ROS (Robot Operating System), an Open Source (BSD License) collection of software frameworks for robots. Of course SLAM is a very important task for any robotic vehicle, and the ROS package RTAB-Map is a perfect solution to implement this capability into any autonomous or remotely operated machine, like our ArcheoROV. For this reason, before starting experiments in more sophisticated (and complicated) systems, we checked RTAB-Map performance with an old Kinect, and here is the video of the result:

As you can see, the performance of real-time 3D is pretty responsive, respect our old experiments with the Open Source software RGBDemo (also considering that the Kinect used in this video is the first version, and it is now pretty obsolete) and, most important, the localization function within SLAM algorithm works very good. As I wrote at the beginning of the post, our current impression is that this combination of hardware (Kinect) and software (ROS) can be a good solution for underground environment documentation, while the software can be the right choice for archaeological exploring robotic devices.

I hope that this long post will be useful, if you have any feedback, please just write your comment below. Have a nice day!

PS:

we will present the ArcheoROV at the ArcheoFOSS (43) of Cagliari (Sardinia - Italy), this year. Also our partner of WitLab will be with us!

Webography

ATOR:

(1) Kinect, real-time 3D; (2) Kinect accuracy and precision with RGBDemo; (3) Kinect 3D outdoor: hacking the hardware; (4) Kinect 3D outdoor: first test; (5) Kinect 3D limits: documenting small objects; Kinect 3D indoor: excavation test (6); Kinect - Infrared prospections (7); Aramus 2014: 2D and 3D documentation of archaeological excavation (8); 3D for archaeological finds (9); Taung Project: 3D with SfM & IBM (10); Extreme SfM: underwater archaeology (11); From drone-aerial pictures to DEM and ORTHOPHOTO: the case of Caldonazzo's castle (12); Documentation of a bas-relief on a cliff : the workflow (13); CMVS/PMVS2 40% faster (14); OpenMVG VS PPT (15); MicMac and PPT: two FLOSS solutions for 3D data (16); SfM for Underground Documentation (17); Archaeology as a profession (18); Glacial Archaeology: About the challange to work in extreme conditions (19); WW1: High Alpine Survey Data - Work in Progress (20); Arc-Team tries Large Scale Reflectance Transformation Imaging (RTI) (21); WebRTIViewer (22); UAVP (Universal Aerial Video Platform) (23); UAVP indoor flight (24); 3D documentation of small archaeological finds (25); Building an Xcopter (26); Arc-Team's UAVP: testing the NAZA dji (27); Xcopter drone and SFM techniques (28); ArcheOS and UAVP for archaeological remote sensing (29); Open Source Remote Sensing Platform (30); Remote sensing with UAV in archeology (lessons at Lund University) (31); Aerial archaeology with FLOS Hardware and Software (32); A DIY endoscope for emergencies during excavation fieldwork (33); 3D PRINTING THE PAST: SOME ISSUES (34); The Taung Child is now touchable, thanks to 3d printing (35); 3D printing for Cultural Heritage (36); Space archaeology (37); 3D PRINTING GOOGLE MAPS IS NOW EASY (38); When Veterinary Medicine and 3D printing meet each other (39); Three more animals are saved with the aid of Blender and 3D printing (40); Augmented Reality at Cultways (41); Boolean operations - the powerful Cork! (42); ArcheoFOSS 2016 in cagliari! (43)

Kentstrapper website: http://kentstrapper.com/

Fa)(a 3D website: http://www.falla3d.com/

WitLab website: http://www.witlab.io/

ROS website: http://www.ros.org/

RTAB-Map website: http://introlab.github.io/rtabmap/

Doing quantitative archaeology with open source software

This short post is written for archaeologists who frequently perform common data analysis and visualisation tasks in Excel, SPSS or similar commercial packages. It was motivated by my recent observations at the Society of American Archaeology meeting in San Francisco - the largest annual meeting of archaeologists in the world - where I noticed that the great majority of archaeologists use Excel and SPSS. I wrote this post to describe why those packages might not be the best choices, and explain what one good alternative might be. There’s nothing specifically about archaeology in here, so this post will likely to be relevant to researchers in the social sciences in general. It’s also cross-posted on the Software Sustainability Institute blog.

Prevailing tools for data analysis and visualization in archaeology have severe limitations

For many archaeologists, the standard tools for any kind of quantitative analysis include Microsoft Excel, SPSS, and for more exotic methods, PAST. While these software are widely used, there are a few limitations that are obvious to anyone who has worked with them for a long time, and raise the question about what alternatives are available. Here are three key limitations:

• File formats: each program has its own proprietary format, and while there is some interoperability between them, we cannot open their files in any program that we wish. And because these formats are controlled by companies rather than a community of researchers, we have no guarantee that the Excel or SPSS file format of today will be readable by any software 10 or 20 years from now. 
• Click-trails: the main interaction with these programs is by using the mouse the point and click on menus, windows, buttons and so on. These mouse actions are ephemeral and unrecorded, so that many of the choices made during a quantitative analysis in Excel are undocumented. When a researcher wants to retrace the steps of their workflow days, months or years after the original effort, they are dependent on their memory or some external record of many of the choices made in an analysis. This can make it very difficult for another person to understand how an analysis was conducted because many of the details are not recorded. 
• Black boxes: the algorithms that these programs use for generating results are not available for convenient inspection to the researcher. The programs are a classic black box, where data and settings go it, and a result comes out, as if by magic. For moderately complicated computations, this can make it difficult for the researcher to interpret their results, since they do not have access to all of the details of the computation. This black box design also limits the extent to which the researcher can customise or extend built-in methods to new applications.

How to overcome these limitations?

For a long time archaeologists had few options to deal with these problems because there were few alternative programs. The general alternative to using a point-and-click program is writing scripts to program algorithms for statistical analysis and visualisations. Writing scripts means that the data analysis workflow is documented and preserved, so it can be revisited in the future and distributed to others for them to inspect, reuse or extend. For many years this was only possible using ubiquitous but low-level computer languages such as C or Fortran (or exotic higher level languages such as S), which required a substantial investment of time and effort, and a robust knowledge of computer science. In recent years, however, there has been a convergence of developments that have dramatically increased the ease of using a high level programming language, specifically R, to write scripts to do statistical analysis and visualisations. As an open source programming language with special strengths in statistical analysis and visualisations, R has the potential to be a solution to the three problems of using software such as Excel and SPSS. Open source means that all of the code and algorithms that make the program operate are available for inspection and reuse, so that there is nothing hidden from the user about how the program operates (and the user is free to alter their copy of the program in any way they like, for example, to increase computation speed).

Three reasons why R has become easier to use

Although R was first released in 1993, it has only been in the last five years or so that it has really become accessible and a viable option for archaeologists. Until recently, only researchers steeped in computer science and fluent in other programming languages could make effective use of R. Now the barriers to getting started with R are very low, and archaeologists without any background with computers and programming can quickly get to a point where they can do useful work with R. There are three factors that are relevant to the recent increase in the usability of R, and that any new user should take advantage of:

• the release of an Integrated Development Environment, RStudio, especially for R
• the shift toward more user-friendly idioms of the language resulting from the prolific contributions of Hadley Wickham, and 
• the massive growth of an active online community of users and developers from all disciplines.

1. RStudio

For the beginner user of R, the free and open source program RStudio is by far the easiest way to quickly get to the point of doing useful work. First released in 2011, it has numerous conveniences that simplify writing and running code, and handling the output. Before RStudio, an R user had little more than a blinking command line prompt to work with, and might struggle for some time to identify efficient methods for getting data in, run code (especially if more than a few lines) and then get data and plots out for use in reports, etc. With RStudio, the barriers to doing these things are lowered substantially. The biggest help is having a text editor right next to the R console. The text editor is like a plain text editor (such as Notepad on Windows), but has many features to help with writing code. For example, it is code-aware and automatically colours the text to make it a lot easier to read (functions are one colour, objects another, etc.). The code editor has comprehensive auto-complete feature that shows suggested options while you type, and gives in-context access to the help documentation. This makes spelling mistakes rare when writing code, which is very helpful. There is a plot pane for viewing visualisations and buttons for saving them in various formats, and a workspace pane for inspecting data objects that you've created. These kinds of features lower the cognitive burden to working with a programming language, and make it easier to be productive with a limited knowledge of the language.

2. The Hadleyverse

A second recent development that makes it easier for a new user to be productive using R is a set of contributed packages affectionately known in the R user community as the Hadleyverse. User contributed packages are add-on modules that extend the functionality of base R. Base R is what you get when you download R from r-project.org, and while it is a complete programming language, the 6000-odd user contributed packages provide ready-made functions for a vast range of data analysis and visualization tasks. Because the large number of packages can make discovering relevant ones challenges, they have been organised into 'task views' that list packages relevant to specific areas of analysis. There is a task view for archaeology, providing an annotated list of R packages useful for archaeological research. Among these user-contributed packages are a set by Hadley Wickham (Chief Scientist at RStudio and adjunct Professor at Rice University) and his collaborators that make plotting better, simplify common data analysis activities, speed up importing data in R (including from Excel and SPSS files), and improve many other common tasks. The overall result is that for many people, programming in R is shifting from the base R idioms to a new set of idioms enabled by Wickham's packages. This is an advantage for the new user of R because writing code with Wickham's packages results in code that is easier to read by people, as well as being highly efficient to compute. This is because it simplifies many common tasks (so the user doesn't have to specify exotic options if they don't want to), uses common English verbs ('filter', 'arrange', etc.), and uses pipes. Pipes mean that functions are written one after the other, following the order they would appear in when you explain the code to another person in conversation. This is different from the base R idiom, which doesn't have pipes and instead has functions nested inside each other, requiring them to be read from the center (or inside of the nest) to the left (outside of the nest), and use temporary objects, which is a counter-intuitive flow for most people new to programming.

3. Big open online communities of users

A third major factor in the improved accessibility of R to new users is the growth of an active online communities of R users. There has long been an email list for R users, but more recently, user communities have former around websites such as Stackoverflow. Stackoverflow is a free question-and-answer website for programmers using any language. The unique concept is that it gamifies the process of asking and answering questions, so that if you ask a good question (ie. well-described, includes a small self-contained example of the code that is causing the problem), other users can reward your effort by upvoting your question. High quality questions can attract very quick answers, because of the size of the community active on the site. Similarly, if you post a high-quality answer to someone else's question, other users can recognise this by upvoting your answer. These voting processes make the site very useful even for the casual R user searching for answers (and who may not care for voting), because they can identify the high-quality answers by the number of votes they've received. It's often the case that if you copy and paste an error message from the R console into the google search box, the first few results will be Q&A pages on Stackoverflow. This is very different experience compared to using the r-help email list, where help can come slowly, if at all, and searching the email list, where it's not always clear which is the best solution. Another useful output from the online community of R users are blogs that document how to conduct various analyses or produce visualizations (some 500 blogs are aggregated at http://www.r-bloggers.com/). The key advantage to Stackoverflow and blogs, aside from their free availability, is that they very frequently include enough code for the casual user to reproduce the described results. They are like a method exchange, where you can collect a method in the form of someone else's code, and adapt it to suit your own research workflow.

There's no obvious single explanation for the growth of this online community of R users. Contributing factors might include a shift from SAS (a commercial product with licensing fees) to R as the software to teach students with in many academic departments, due to the Global Financial Crisis of 2008 that forced budget reductions at many universities. This led to a greater proportion of recent generations of graduates being R users. The flexibility of R as a data analysis tool, combined with  rise of data science as an attractive career path, and demand for data mining skills in the private sector may also have contributed to the convergence of people who are active online that are also R users, since so many of the user contributed packages are focused on statistical analyses.

So What?

The prevailing programs used for statistical analyses in archaeology have severe limitations resulting from their corporate origins (proprietary file formats, uninspectable algorithms) and mouse-driven interfaces (impeding reproducibility). The generic solution is an open source programming language with tools for handling diverse file types and a wide range of statistical and visualization functions. In recent years R has become the a very prominent and widely used language that fulfills these criteria. Here I have briefly described three recent developments that have made R highly accessible to the new user, in the hope that archaeologists who are not yet using it might adopt it as more flexible and useful program for data analysis and visualization than their current tools. Of course it is quite likely that the popularity of R will rise and fall like many other programming languages, and ten years from now the fashionable choice may be Julia or something that hasn't even been invented yet. However, the general principle that a scripted analyses using an open source language is better for archaeologists, and science generally, will remain true regardless of the details of the specific language.

3D printing for Cultural Heritage

As many of you know,

since last year we are working on the preparation of the open source exhibition "FACCE. I molti volti della storia umana" (here some news in ATOR: 1, 2 and here the official page and the FB page). Of course all the software we are using are Free and Open (FLOSS), but for some particular exhibits, we are working also with open hardware.

Our interest in this specific topic dates back to 2006, when we started investigating the potentialities of open hardware in archeology, especially in the field of aerial photography e rapid prototyping. We get the first results in 2008, with our UAVP prototype (here some news from ATOR: 1, 2) an all the other UAV we built (KK, etc...).

For the exhibition the main open hardware we used are 3D printing machines. We experienced different solutions: +Leonardo Zampi printed the 3D model of the Taung Child thanks to the Kentstrapper machines, an ideal choice if you want an hardware which is ready to work out of the box (here the related articles: 1, 2)...

The Taung Child printed in the Volta printer (Kentstrapper)

... and we build our own device from the scratch, a Fa)(a 3D, thanks to +Giacomo Falaschi  and +silvio tassinari , of Roma Makers.

Building the Fa)(a 3D at Roma Makers lab.

One of the main issue in 3D printing for Cultural Heritage is to replicate an object in the right scale (otherwise you will just print a souvenir) and this is why it is very important to know how to perform a 3D scan, but also how to post-process your data ( (in ATOR you will find a lot of informations about this), in order to send the right file to the machine.
Here are some example of the models we are printing for the exhibition (scale 1:1): some 3D skulls of hominids (different individuals of Homo georgicus) we were able to scan, tanks to the kindness of Prof. David Lordkipanidze, during our mission to Tiblisi and Dmanisi. We will use them to prepare some Augmented Reality interactive exhibits related to paleoart.

3D printed skull (side view)

3D printed skull (front view)

And here is a short video of our Fa)(a 3D in action:

Have a nice day!

Remote sensing with UAV in archeology (lessons at Lund University)

Since 2011, Arc-Team is teaching during the course "digital archeology" at Lund University, introducing the class to the use of Free and Open Source Software (FLOSS) in archeology (with ArcheOS). The course is held by Nicolò Dell'unto (Department of Archaeology and Ancient History). 

This year we had the opportunity to give a lesson regarding the use of UAV (Unmanned Aerial Vehicles), and more precisely quadcopters,  in remote sensing projects. Our airfield was the Swedish village of Uppåkra, where the University is undertaking an excavation. 

Here you can see a coupe of video recorded in slow motion by our friends Carolina Larson and Stefan Lindgren of the Humanities Lab. Thanks to the slow motion, it is possible to observe the flight stability ...

... and the ability to maintain the position of the last drone we built (more details here).

The use of drones in the field of archeology has also attracted interest from the local press: the magazine of the University (LUM) published an article on the topic:

Radio-controlled helicopter maps
archaeological sites from above

From Youtube to Blender: Forensic facial reconstruction of a child mummy

This post is about a practical application of a serie of studies published here in this blog.

After I started to study about forensic facial reconstruction I saw that is much more easy to find videos of CT-Scan than the DICOM files and other tomography formats.

A way to convert a video in a reconstructed mesh was described here.

Some days ago I was reading about mummies (desperate to find a CT-Scan) and I found this post:

http://erl.wustl.edu/research/imseg/mummy.html

It talks about a child mummy of St. Louis, that lived in a range of 40 BC and 130 AD. He died with 7 or 8 months.

Inside the matter had a video with some seconds of a CT-Scan slicing. I was able to convert it in a reconstructed mesh, and after I found a video on Youtbe with more qualty and I used it to make the final mesh, used in this post.

I downloaded the video with Videodownload Helper (Firefox) and it was converted in a image sequence and after in a serie of DICOM files.

Unfortunately I lost the original vetorial file and now we have only the infographic in Portuguese version, like you can see below (but it have a lot of images, that dispensing you to read it).

To make a reconstruction with historical and archaeological foundation I had the help of Moacir Elias Santos, archaeologist of the Egypt Museum and Rosacruz, from Brazil.

The animated gif above shows the extracted frames of the animation converted into a CT-Scan. I reduced the slices to make it more didatic.

I had a serie of dificulties to find landmarks to use on the child's face, cause appear that it doesn't exist. So I use a average of 3-8 year and rescale it to have at least a reference.

I use a serie of babies pictures to draw the line of the neck and ears.

Moacir sent me a compose image with the original mummy, that you can see below.

I hope you enjoy this post. I see you in the next. A big hug!

ArcheOS and UAVP for archaeological remote sensing

Hi all,

Finally I uploaded the presentation we did in the CAA Southampton 2012. Until our website is down (for maintenance), you can see it here. Inside you can find more details about the aerial archaeology project we mentioned in the post Xcopter drone and SfM techniques.

Here you can see the first slide, in the new Arc-Team theme I did playing with beamer, LaTeX :).

As you see the license is the Creative Commons Attribution 3.0 Unported (CC BY 3.0), which we are planning to adopt soon also for ATOR to facilitate content sharing.

2016-03-31 Update

Thanks to self-archiving I can now add the bibliography related with this post:

 ResearchGate: Articl

Academia: Article

I hope it will be useful, even if no more up to date it can be a starting point to work in Aerial Archeology with Open Software and Hardware.

Archaeological drawing from photomapping

IMHO one of the main benefits of photomapping is the fact that everybody is able to do good archaeological drawing. In this way it is not necessary to divide archaeologists between "diggers" (who simply excavate layer by layer)  and "drawers" (who just document). In our (Arc-Team) experience it is never a good solution when someone has to document a situation he does not know (because was exposed by someone else).
In other words, photomapping techniques and GIS allow also normal archaeologists to reach the same quality level of professionals in drawing layers or finds. As an example, the picture below shows the skeleton of a young goat found in Aramus (AM): the georeferenced photomosaic is done with photomapping techniques ("metodo Aramus") and the drawing is done by me (i am not a professional in drawing) using the GIS OpenJUMP (inside ArcheOS).

Georeferenced photomosaic and vector drawing with OpenJUMP

Archaeological automatic drawing

In 2009 we were looking for a system to get a fast and easy way to draw a lot of archaeological finds automatically. Thanks to the help of Simone Cavalieri (who found in internet Adrian Secord's algorythm) we developed a methodology which, combining different Free/Libre and Open Source Software, has speeded up our drawing techniques. The main point of the problem was to get an automatic shading through dots, to respect most rules in archaeological drawing. The solution was the software Stippler, which gave us promising results. In the image below you can see just an example of automatic shading with a test low quality picture of the Venus of Willendorf (downloaded from Wikipedia).

Shading with stippler (stippler-GUI)

2016-04-28 Post updated

In 2010 we wrote an article (in Italian) about this technique:

"Proposta per un metodo informatizzato di disegno archeologico" (here in ResearchGate and here in Academia).