ATOR (Arc-Team Open Research).
The blog spreads tests, problems and results of Arc-Team research in archaeology, following the guidelines of the OpArc (Open Archaeology) project.
It is a long time since we wrote something in this blog, but (like every year) the excavation season leaves us few time for research. For this reason, today I want to break our silence and show some results of our latest studies regarding archeorobotics (the use and development of robotic devices in archaeology).
If you are a regular reader of ATOR, you probably know that since 2012 we are working on optical sensor to achieve a real-time 3D documentation of archaeological evidences (or any kind of data we need to acquire during our projects). Since we started to work on different kind of drones (UAV, ROV, etc...), we discover the nice universe of ROS (Robot Operating System) and SLAM (Simultaneous Localization And Mapping) algorithms. In this post we summarized our research on this topic, focusing on the use of Kinect. Currently we already used this techniques on professional projects (like large scale surveys or excavations), adapting the system to work with RGB-D devices (in underground environment or during cloudy days) or stereocameras (with direct sun light conditions). For instance we helped our friend Cristian Boscaro of IUAV to test this technology in order to document the tunnels which connect the domes of the Abbay of S. Giustina in Padua. This evening I will post a video which shows a particular use of ROS and Kinect to solve a technical problem we had on the field today. We were working to assist the excavator in doing a trench for a pipeline near the Sanctuary of S. Romedio, in difficult logistic condition. Despite the absence of archaeological evidences, the Superintendence asked us to document the track of the trench, since often what is realize during the execution of this kind of work is different from what is planned in the map. Due to the fact that too few hours were left to accomplish a documentation with GPS and total station and that this strategy would have been pretty tricky (inside the gorge of the river S. Romedio) and not so accurate (for the scattering effect of the wood), we decided to use SLAM to get a real time 3D documentation of the track and later to georeference the result on the LIDAR data which the Autonomous Province of Trento releases freely. The video below shows the final result, which completely satisfies the (high) archaeological tolerance of this project.
The 21st Conference on Cultural Heritage and NEW Technologies (CHNT 21, 2016) took place in Vienna the first week of November 2016. In that occasion we gave a presentation entitled "Digitizing the excavation. Toward a real-time documentation and analysis of the archaeological record". Today I found the time to publish it in our blog, to share our research regarding this topic and in particular some interesting projects of "archeorobotics" we are working on.
Here below you can see the video of the presentation, done like always with the open source software impress.js and Strut...
... and here is a short description of each slide:
SLIDE 1
The title (strictly related with Digital Archaeology in general)
SLIDE 2
A short presentation of Arc-Team
SLIDE 3
All the work has been done thanks to Free/Libre and Open Source Software. In order to keep going on with our research regarding archaeological methodology we need the source code!
SLIDE 4
The fundamental schema of the archaeological cognitive process elaborated by G. Leonardi in 1982. The schema shows the progressive reduction of the informations regarding human actions before and during the archaeological excavation (Human activities --> Traces on the soil --> Natural and anthropological degradation of the record --> archaeological excavation --> archaeological documentation) until the interpretative knowledge starts recover information during the post-excavation stage (with analitical data interpretation and reconstructive hypothesis)
SLIDE 5
A practical example of the schema from the site of Torre dei Sicconi in Italy (a medieval castle):
1. Human activities (summarized in the building of the castle, the medieval battle and the destruction of the main structure and the controlled explosion during the Great War)
2. Traces on the soil (summarized in the evidences of the battle, of the controlled explosion and of recent agrarian activities, while just negative layers were found regarding the construction of the structure)
3. Natural and anthropological degradation (summarized in the battle, the explosion, the agrarian activities and the normal natural dynamics)
4. Archaeological excavation (the most destructive investigation: in Torre dei Sicconi all the layers concerning the tower and the main central building has been removed by this activity)
5. The importance of archaeological documentation comes from distructive analysis (excavation). Being a long term project, Torre dei Sicconi was documented both with traditional and digital methodology
6. Data analysis. During this stage our knowledge of the site started to grow again. In this case both archaeological and historical techniques have been used
7. Reconstructive hypotheses represent the maximum increase of our (interpretative) knowledge of the site. For Torre dei Sicconi this stage has been achieved just for the central part of the castle (tower and main building)
SLIDE 6
The archaeological excavation is the most critical (destructive) stage of our knowledge regarding a site.
SLIDE 7
Arc-Team's excavation strategies:
1. increasing the amount of information registered decreasing the time-consuming operation of archaeological documentation
2. on-site direct observation for a better interpretation, avoiding at the same time any kind of data selection
3. moving the lab into the field (chemical and physical analyses)
SLIDE 8
A milestone of our research: in 2006 the development of the "Metodo Aramus" gave us a better (more precise and accurate), faster and corect (equalized) 2D digital documentation with FLOSS.
SLIDE 9
Another milestone. Between 2008 and 2009 the migration from pure photogrammetric software to SfM and MVSR methods (through the development of a GUI for +Pierre Moulon's application Python Photogrammetry Suite) gave us better and faster 3D digital documentation
SLIDE 10
Even today we still use a combination of 2D and 3D techniques to meet different requirements of various archaeological projects
SLIDE 11
2D digital documentation through GIS is fast enough for on site interpretation during emergency excavation
SLIDE 12
A software like +QGIS allows a direct interpretation on the field without the necessity of long post-rpocessing
SLIDE 13
3D documentation gives better results, but needs longer processing time (even if it does not need long data acquisition on the field, which is always performed)
SLIDE 14
We achieved (a lower quality) 3D data acquisition which has the fundamental characteristic of being real-time, thanks to open hardware (archeorobotics)
SLIDE 15
Our experience in archeorobotics dates back to 2006 with our first prototype of UAV, which could be use professionally just in 2008.
SLIDE 16
Currently or archeorobotics research regards our last prototype of Archeodrone (a UAV specifically designed for aerial archaeology)...
SLIDE 17
... some CNC machines and, above all, the Fa)(a 3D, a 3D open hardware printer which without any kind of modifications was able to satisfy our archaeological needs (like 3D printing casts of unique finds or exctract and print DICOM data form x-ray CT scan)...
SLIDE 18
... and the ArcheoROV, the open hardware Remotely underwater Operated Vehicle which we developed with the +Witlab Fablab
SLIDE 19
Some pictures of the first test of the ArcheoROV
SLIDE 20
A first step into 3D real-time documentation through SLAM (Simultaneous Localization and Mapping) techniques has been done with the open source ROS (Robot Operating System) and RTAB-Map via Kinect...
SLIDE 21
... and tested for 3D real-time documentation in wooden areas (where SfM and MVSR or laserscab would have been too slow), reaching in almost one hour of work a model (with real dimension) of 75000 points.
SLIDE 22
A benefit of archaeorobotic system like these (which are ROS capable) is the possibility to change the sensor in order to adapt the hardware to different situation, using monocular or stereo cameras (for odometry) as well as LIDAR or SONAR devices.
SLIDE 23
Another benefit is the wide range of possibilities offered by the different open source software (e.g. RTAB-Map, LSD-SLAM, REMODE, Cartographer, ecc...)
SLIDE 24
Currently the precision/accuracy level of a real-time 3D archaeological documentation cannot be compared with the results achieved with post-processing through traditional SfM - MVSR systems, but there are good prospects for improvement.
SLIDE 25
Nowadays, basing on our professional experience, the best use of such devices seems to be during extreme operations, such as high mountain archaeology, glacial archaeology, underwater archaeology or speleoarchaeology
SLIDE 26
Another important step to improve the reaction time of professional archaeology, in order to avoid errors during the critical stage of the excavation, is the possibility to perform some basic archaeometrical analyses (chemical and physical) directly on the field.
SLIDE 27
Considering the composition of any archaeological layer based on two different elements, the skeleton (macroscopic) and the fine earth (microscopic), it is obvious that different analyses can be performed in different work environment.
SLIDE 28
For instance, in the case of the skeleton, a fast petrografic (ontoscopic) analysis can be easily performed directly on the field (defining allogeneic elements), while further (more specific) investigations need an equipped laboratory.
SLIDE 29
Also in the case of fine earth, some raw descriptive analyses can be performed on the field, while laboratory investigation can reach very detailed results (e.g. with the Scanning Electron Microscope).
SLIDE 30
The field analysis of the fine earth is more problematic (compared with the skeleton) the most common test (e.g. the Soil texture by feel) are anametric and subjective
SLIDE 31
For this reason, archaeometric test are the better choice (e.g the sedimentation test)
SLIDE 32
The sedimentation test on the field can be improved with basic physical analysis (e.g. considering the Stoke's Law in order to define sand, silt and clay by the tme they need to sediment)
SLIDE 33
Another implementation on the field for the sedimentation test is the possibility to directly store the data into a PostreSQL/PostGIS database (through some specific fields of the archaeological recording sheet), using the open source application geTTexture.
SLIDE 34
An example of the use of geTTexture
SLIDE 35
Other archaeometric test which are simple to perform directly during the excavation are based on basic chemical analyses, and specifically with the quantification of compounds like phosphates or nitrates.
SLIDE 36
Moreover, with some simple workarounds, it is possible to turn anametric (boolean) analyses of carbonates or organic substances, into metric (quantitative) observations.
SLIDE 37
The Archaeological excavation is a destructive process, subject to fatal (not reversible) errors. Moreover the reduced time and budget in professional and emergency archaeology increase stress conditions during decision making stages.
Real-time 3D mapping can speed up data interpretation, avoiding data selection on the field, while on-site chemical and physical analyses (geoarchaeology and archaeometry) can define a better (data-driven) digging strategy.
I hope this presentation can be useful. Have a nice day!
As you probably noticed, one of the topic of ATOR is related with hardware hacking, with the aim to build new archaeological devices from ordinary objects and tools (33).
This concept is close to the one of "reuse" (using an artefact for a purpose which is completely different from the original function), a pretty common phenomenon in archaeology; also in architecture there is something similar, called "spolia" (but maybe our interest in hacking things is just a kind of McGiver syndrome of people grown up in the 80s).
However, this post is about hacking a common game device like Kinect to use its characteristic in archaeological 3D real-time documentation. If you are a regular reader of ATOR, you will know that we already face this challenge, performing a first test (1) with RGBDemo in February 2012, and controlling accuracy and precision of the device in March of the same year (2), after a discussion with some of the researchers of FBK, during the workshop "Low cost 3D: sensori algoriti e applicazioni". Due to the encouraging results achieved in our first experiments, we worked on the hardware in order to modify it for outdoor projects (3), but soon we experimented the limits of this technology when applied in areas with direct sunlight (4) or in documenting small objects (5, 25). Despite this drawbacks in our research, Kinect worked pretty good in indoor excavations (6), helping us in difficult situations (related with the the workplace safety), and for particular purposed, like for infra-red prospections in dark environment (7).
After all these experiences, our final advice about Kinect is that the device has a potential in archaeology, but its real employment in professional work is restricted to peculiar conditions, while in most of the cases the SfM-based techniques are the best option (due to their versatility, which makes them a perfect choice during missions abroad (8), for small finds documentation (9, 10), for underwater and aerial archaeology (11, 12, 13), considering also the speed which characterize SfM and MVSR open source software development (14) and the wide range of possibilities between the different tools (15, 16).
Well, at least this was our opinion until now... Currently we are changing our mind about Kinect, and this is due to our professional engagement in underground archaeology (17) and to our renovate interest in robotics. Let's deal with these two points separately.
Underground archaeology
Documenting an underground semisubmerged structure in Firuzabad (Iran)
Like any other operation in archaeological 3D documentation, the tolerance regarding accuracy and precision is variable and influenced by some factors, and mainly: research purposes, logistics, characteristics of the structures to be documented.
Without considering some important exceptions (e.g prehistoric rock shelter, which are often simple to document with SfM techniques), most of the structures related with underground archaeology (WW1 artificial caves, medieval mines, etc...) are connected with large scale survey projects (where it is important a "big data" approach, raising the tolerance in data acquisition to increase the number of documented structures); with logistically difficult areas (high mountains, glaciers, (18, 19) etc...); with structures often characterized by vast surfaces without important small details, which (when present) can be recorded with a targeted SfM or RTI (21, 22) documentation (e.g. for graffiti, inscriptions (20), manufacture traces, etc...). For this reason, in most of these projects, it is necessary to deal with precision in documenting (keeping checkpoints thanks to other TOF instruments, like total stations) in order to gain a real-time response from the selected device, and, under this point of view, Kinect is often a good solution, considering also that its infrared sensor helps very much in low light conditions (7).
Documenting WW1 caves in Southtirol (Italy)
Archeorobotics
Arc-Team's UAV during an aerial archaeology project in Storo (Trentino - Italy)
Since 2006, when we joined an aerial archaeological project in Armenia (23), we started to work on "archaeorobotics", trying to develop robotic devices able to help us in the most difficult archaeological missions.
The first positive results we reached in this field were related with aerial archaeology and the building of an open hardware UAV (in 2008), even if at the beginning we underestimated the time needed to practice with our new tool (24). Soon our experience increased as we built different drones, based on open and closed solutions (like kk multicopter (26) or Naza dji (27) models). The benefits of this research branch were clear (28, 29) and soon other research institutions, like the CNR-ITAB of Rome (30), the University of Lund (31) and the CNR-ISTI of Pisa (32), asked us to give lessons about this topic.
Another field of archaeorobotics we explored is the one related with CNC machines and especially with 3D Printers. For this topic a precious help came from the society Kentstrapper and Leonardo Zampi (aka +Exekias 87), who helped us in 3D printing the cast of the Taung Child (34, 35). Since RepRap project started (in 2005), 3D printers evolved very fast. Of course our interest regarding these machines is mainly oriented to Cultural Heritage, and this is also the reason why we built a Fa)(a 3D form scratch (36), but results with this kind of instruments can be very impressive, especially considering the wide range of scientific applications (37, 38, 39, 40, 41), even if sometimes you have to deal with difficult boolean operations (42).
However, none of the robotic projects we developed till now needed Kinect, being based on UAV, to 3D document archaeological sites, or on CNC machines, to fast replicate archaeological artefacts. Our renovate interest in Kinect for archeorobotics is due to our new challenge in developing a ROV (Remotely Operated Vehicle), in order to assist us in our underwater archaeological missions. Indeed, in the last months, we started a collaboration with the WitLab, the FabLab of Rovereto (Trentino - Italy), to develop a new Open Hardware ROV, especially designed for archaeological aims. One of the main topic in developing such an instrument is that the new robot will be oriented not only to 3D documentation, but also to the exploration of unknown areas. For such reason SfM and MVS software are no more enough, but we had to start again in testing Open Source SLAM (Simultaneous Localization And Mapping) algorithms, due to the fact that we need to register in 3D the submerged landscape (Mapping), but also to recover the path the "ArcheoROV" did (Localization) to reach new hidden archaeological evidences (for a better planning of human operations).
Testing the ArcheoROV at night
Testing Open Source SLAM solutions
The importance of SLAM algorithms in exploring devices is the main reason why we started again to experiment Kinect. Indeed, despite Kinect cannot be used as an on-board optical device in our ArcheoROV (due to the infrared camera), this tool is the perfect system to check SLAM software.
If, you ever started in working on robotics, probably sooner or later you stepped into ROS (Robot Operating System), an Open Source (BSD License) collection of software frameworks for robots. Of course SLAM is a very important task for any robotic vehicle, and the ROS package RTAB-Map is a perfect solution to implement this capability into any autonomous or remotely operated machine, like our ArcheoROV. For this reason, before starting experiments in more sophisticated (and complicated) systems, we checked RTAB-Map performance with an old Kinect, and here is the video of the result:
As you can see, the performance of real-time 3D is pretty responsive, respect our old experiments with the Open Source software RGBDemo (also considering that the Kinect used in this video is the first version, and it is now pretty obsolete) and, most important, the localization function within SLAM algorithm works very good. As I wrote at the beginning of the post, our current impression is that this combination of hardware (Kinect) and software (ROS) can be a good solution for underground environment documentation, while the software can be the right choice for archaeological exploring robotic devices.
I hope that this long post will be useful, if you have any feedback, please just write your comment below. Have a nice day!
PS:
we will present the ArcheoROV at the ArcheoFOSS (43) of Cagliari (Sardinia - Italy), this year. Also our partner of WitLab will be with us!
Despite what I wrote at the end of this post, it looks like that Kinect is not really the best option for archaeological underground documentation, or for any other situation in which it is necessary to work in darkness.
I already tested the hardware and the software (RGBDemo) at home, simulating the light conditions of an underground environment, and the result was that Kinect scanned in 3D some parts of an object (a small table), with great difficulties.
My hope was that the infrared sensors of Kinect were enough to record the objects geometries also in darkness, as actually happened. The problem was that probably RGBDemo, to work properly, needs also RGB values (from the normal camera). Without colors information the final 3D model is obviously black (as you can see below), but (and this is the real difficulty) it seems that the software loses a fundamental parameter to keep tracking the object to document, so that the operations become too slow and, in most cases, it is not possible to complete the recording of a whole scene. In other words the documentation process often stops, so that after it is necessary to start again or simply to save different partial scans of the scene, to reassemble at a later time.
However, before discarding Kinect as an option for 3D documentation in darkness, I wanted to do one more experiment in a real archaeological excavation and, some weeks ago, I found the right test area: an acient family tomb inside a medieval church.
As you see in the movie below, the structure was partially damaged, having a small hole on the North side. This hole was big enough to insert Kinect in the tomb, so that I could try to get a fast 3D overview of the inside, also to understand its real area (which was not identifiable from the outside).
As I expected, it was problematic to record the 3D characteristics of such a dark room, but I got all the informations I needed to estimate the real perimeter. I guess that in this occasion RGBDemo worked better because of the ray of light that, entering the underground structure and illuminating a small spot of the ground, was giving the software a good reference point in order to track all the surrounding areas.
Since the poor quality video it is difficult to evaluate the low resolution of the 3D reconstruction, you can get a better idea looking this other short clip, where the final pointcloud is loaded in MeshLab.
This new test of Kinect in a real archaeological excavation seems to confirm that this technology is not (yet?) ready for documentation in complete absence of light. However the most remarkable result of the experiment was the use of one of the tool of RGBDemo, which shows directly the infrared input in a monitor. This option has been a good prospection instrument to explore and monitoring the inside of the burial structure, without other invasive methodology. As you see in the screenshot, it is possible to see the inside condition of the tomb and to recognize some of the objects that lie on the ground (e.g. wooden planks or human bones), but of course this could have been done simply with a normal endoscope and some led lights (like we did in this occasion).
RGBDemo infrared view
However, here is possible to compare what the normal RGB sensor of Kinects is able to "see" in darkness and what its infrared sensors can do:
PS
This experiment was possible thanks to the support of Gianluca Fondriest, who helped me in every single step of the workflow.
As Moreno Tiziani wrote in his post, last Monday (October 22) I was in Padua to start the "Taung Project". The first step of this research was indeed the 3D documentation of the cast of the Taung Child, preserved in the Museum of Anthropology of Padua University.
To digitally register our subject we chose SfM/IBM techniques (using ArcheOS and PPT), because, as I reported in this post, the methodology is accurate enough to document small objects. Nevertheless I brought in Padua also our hacked Kinect, to show Moreno how this system is working in 3D recording operations.
Red circle: Kinect. Green circle Taung Child's cast. Blue circle: RGBDemo compiling on ArcheOS
As we thought, the cast was too small to be documented with Kinect. The reason is clear: when Kinect is too close, it simply does not "see" the subject to record, while when the device is too far away, it register too few 3D points, so that the final mesh is not accurate enough.
Unfortunately, I did not capture a screenshot of our test, but I think the images below illustrate the concept: in the first picture my hand is to close to the sensor and it appears completely black, while in the second picture Kinect can see my hand, which appears pink, but the resolution is too low.
The sensor is too close to the subject
The distance between the sensor and the subject is adequate, but the resolution is too low
However we used Kinect to document something in the Museum of Anthropology: a wooden Egyptian sarcophagus.
As you can see in the short movie below, we registered just one side of the object, for the same reason I explained before: when Kinect is too close to the subject it does not work properly. In this case the position of the sarcophagus was too close to the wall (almost 50 cm) and to a glass showcase (almost 20 cm). It would have been possible to scan all the three visible faces and join them together in post-processing with MeshLab, but this was just an experiment, so we concentrate on the Taung cast.
However in the movie it is also possible to observe another interesting characteristic of Kinect: being an infrared based device it is not able to go through glass, which is registered like a normal opaque object.
To complete the "Kinect trilogy", today I write this post about our first test during a real archaeological fieldwork.
Also in this case we (Alessandro Bezzi and me) used our "hacked Kinect" with the external battery in connection with the rugged PC and, again, the chosen software for data acquisition was RGBDemo. This time we documented in 3D a layer during an "indoor" excavation, to avoid the problems with direct sunlight I descirbed in this post.
The video below tries to summarize this operation...
... and here are some screenshots to have an idea of the final result:
The pointcloud (frontal view)
The pointcloud (side view)
The mesh
The mesh (wireframe)
As you can see the general quality is lower respect the results we can obtain with other techniques (e.g. SfM and IBM), but Kinect and RGBDemo have the benefit to acquire and elaborate the data almost at the same moment, with the possibility to see the documentation process in real time.
Ultimately Kinect is one more option to consider for 3D indoor documentation, considering the peculiarities of the archaeological project (the light conditions, the available time, the required level of detail, etc...). Our experiments will now go on now with some tests in particoular situations, where this technique could be the best option (expecially in underground environments).
It was a sunny September Sunday, so I decided to take a walk with my wife Kathi and show her one of the hermitages located in the valley in which we live (Val di Non, Trentino, Italy).
My second thought was that the ramble was a perfect opportunity to test the hacked Kinect and try to document in 3D the main wall of S. Gallo's ruins (the remains of the hermitage). So I prepared the backpack with Kinect, the external battery and the rugged pc we normally use on the archaeological excavations.
After half an hour's walk throught apple orchards and woods we reached the hermitage. Along the way we also found a stunned rooster. That was strange! A rooster, in italian "gallo", in the S. Gallo's hermitage...
However, we began to try to document the main wall of the ruins, which you can see in the picture below...
S. Gallo's hermitage, with the rooster
... but, probably due to direct sunlight conditions, Kinect and RGBDemo where not working propertly.
In fact, as you can also read in M. Dalla Mura, M. Aravecchia and M. Zanin poster (during "LOW COST 3D: sensori, algoritmi e applicazioni" workshop), "...The main issue is due to direct Sun illumination that leads to saturation in the depth acquisition...". Moreover the software (RGBDemo) was reacting very slowly, but this was probably due to the hardware (Panasonic Thougbook), which is less powerful compared to the laptop I normally use to work. Secondly also RGBDemo seems to work better on GNU/Linux (ArcheOS), the Operating System which runs my laptop, than in Windows, the rugged PC OS (but this could be just my impression).
Not beeing satisfied with the results I get with the 3D documentation of the ruins (software too slow to manage all the data recording process, high errors on the sunny parts of the wall, etc...), I decided to check for another subject to document. Luckily in S. Gallo's hermitage are not missing the caves, so, with the help of Kathi, I did a fast digital 3D copy of the cave you see in the picture below.
S. Gallo's cave
This time the software was working good, fast enought to work on the field and with negligible errors in data acquiring. In the movie below it is possible to see the final pointcloud (not complete, but big enought to understand the quality of a 3D "field" documentation with Kinect).
After this test, our Kinect was ready to the "trial by fire" of a real (indoor) archaeological excavation, which will be the topic of one of the next posts in ATOR.
Some weeks ago I finally found the time to modify our Kinect in order to test it outdoor.
My main problem was the power supply, while in an archaeological excavation it can be difficoult to find an access to electricity. The solution was obviously an external battery, but I did not know the right voltage and amperage. Finally I found this interesting post, in Ken Mankof blog, with all the necessary informations to hack Kinect for the fieldwork.
I tried to summarize all the operations I did in the image below:
The modified Kinect
As you can see, Kinect's main cable ends with an USB (for data transfer) and with an electric plug (for power supply). I cut the cable before the elctric plug and I added a connectors pair, positioning the male one on the Kinect side and the female one on the electric plug side. Then I prepared another cable with a female connector at one end and two different female blade connectors (faston) on the other end (one for the positive and one for the negative exit). The two female blade connectors are compatible with the two terminals (red/positive and black/negative) of a lead battery (12 V, 7.2 Ah). In this way it is possible to switch Kinect form the electric net to the battery, simply plugging and unplugging the right cables.
After the hardware modification I was able to test Kinect outdoor. I will write soon a post with the results.
