Digitizing the excavation

The 21st Conference on Cultural Heritage and NEW Technologies (CHNT 21, 2016) took place in Vienna  the first week of November 2016. In that occasion we gave a presentation entitled "Digitizing the excavation. Toward a real-time documentation and analysis of the archaeological record". Today I found the time to publish it in our blog, to share our research regarding this topic and in particular some interesting projects of "archeorobotics" we are working on.

Here below you can see the video of the presentation, done like always with the open source software impress.js and Strut...

... and here is a short description of each slide:

SLIDE 1

The title (strictly related with Digital Archaeology in general)

SLIDE 2

A short presentation of Arc-Team

SLIDE 3

All the work has been done thanks to Free/Libre and Open Source Software. In order to keep going on with our research regarding archaeological methodology we need the source code!

SLIDE 4

The fundamental schema of the archaeological cognitive process elaborated by G. Leonardi in 1982. The schema shows the progressive reduction of the informations regarding human actions before and during the archaeological excavation (Human activities --> Traces on the soil --> Natural and anthropological degradation of the record --> archaeological excavation --> archaeological documentation) until the interpretative knowledge starts recover information during the post-excavation stage (with analitical data interpretation and reconstructive hypothesis)

SLIDE 5

A practical example of the schema from the site of Torre dei Sicconi in Italy (a medieval castle):

1. Human activities (summarized in the building of the castle, the medieval battle and the destruction of the main structure and the controlled explosion during the Great War)

2. Traces on the soil (summarized in the evidences of the battle, of the controlled explosion and of recent agrarian activities, while just negative layers were found regarding the construction of the structure)

3. Natural and anthropological degradation (summarized in the battle, the explosion, the agrarian activities and the normal natural dynamics)

4. Archaeological excavation (the most destructive investigation: in Torre dei Sicconi all the layers concerning the tower and the main central building has been removed by this activity)

5. The importance of archaeological documentation comes from distructive analysis (excavation). Being a long term project, Torre dei Sicconi was documented both with traditional and digital methodology

6. Data analysis. During this stage our knowledge of the site started to grow again. In this case both archaeological and historical techniques have been used

7. Reconstructive hypotheses represent the maximum increase of our (interpretative) knowledge of the site. For Torre dei Sicconi this stage has been achieved just for the central part of the castle (tower and main building)

SLIDE 6

The archaeological excavation is the most critical (destructive) stage of our knowledge regarding a site.

SLIDE 7

Arc-Team's excavation strategies:

1. increasing the amount of information registered decreasing the time-consuming operation of archaeological documentation

2. on-site direct observation for a better interpretation, avoiding at the same time any kind of data selection

3. moving the lab into the field (chemical and physical analyses)

SLIDE 8

A milestone of our research: in 2006 the development of the "Metodo Aramus" gave us a better (more precise and accurate), faster and corect (equalized) 2D digital documentation with FLOSS.

SLIDE 9

Another milestone. Between 2008 and 2009 the migration from pure photogrammetric software to SfM and MVSR methods (through the development of a GUI for +Pierre Moulon's application  Python Photogrammetry Suite) gave us better and faster 3D digital documentation

SLIDE 10

Even today we still use a combination of 2D and 3D techniques to meet different requirements of various archaeological projects

SLIDE 11

2D digital documentation through GIS is fast enough for on site interpretation during emergency excavation

SLIDE 12

A software like +QGIS  allows a direct interpretation on the field without the necessity of long post-rpocessing

SLIDE 13

3D documentation gives better results, but needs longer processing time (even if it does not need long data acquisition on the field, which is always performed)

SLIDE 14

We achieved (a lower quality) 3D data acquisition which has the fundamental characteristic of being real-time, thanks to open hardware (archeorobotics)

SLIDE 15

Our experience in archeorobotics dates back to 2006 with our first prototype of UAV, which could be use professionally just in 2008.

SLIDE 16

Currently or archeorobotics research regards our last prototype of Archeodrone (a UAV specifically designed for aerial archaeology)...

SLIDE 17

... some CNC machines and, above all, the Fa)(a 3D, a 3D open hardware printer which without any kind of modifications was able to satisfy our archaeological needs (like 3D printing casts of unique finds or exctract and print DICOM data form x-ray CT scan)...

SLIDE 18

... and the ArcheoROV, the open hardware Remotely underwater Operated Vehicle which we developed with the +Witlab Fablab 

SLIDE 19

Some pictures of the first test of the ArcheoROV

SLIDE 20

A first step into 3D real-time documentation through SLAM (Simultaneous Localization and Mapping) techniques has been done with the open source ROS (Robot Operating System) and RTAB-Map via Kinect...

SLIDE 21

... and tested for 3D real-time documentation in wooden areas (where SfM and MVSR or laserscab would have been too slow), reaching in almost one hour of work a model (with real dimension) of 75000 points.

SLIDE 22

A benefit of archaeorobotic system like these (which are ROS capable) is the possibility to change the sensor in order to adapt the hardware to different situation, using monocular or stereo cameras (for odometry) as well as LIDAR or SONAR devices.

SLIDE 23

Another benefit is the wide range of possibilities offered by the different open source software (e.g. RTAB-Map, LSD-SLAM, REMODE, Cartographer, ecc...)

SLIDE 24

Currently the precision/accuracy level of a real-time 3D archaeological documentation cannot be compared with the results achieved with post-processing through traditional SfM - MVSR systems, but there are good prospects for improvement.

SLIDE 25

Nowadays, basing on our professional experience, the best use of such devices seems to be during extreme operations, such as high mountain archaeology, glacial archaeology, underwater archaeology or speleoarchaeology

SLIDE 26

Another important step to improve the reaction time of professional archaeology, in order to avoid errors during the critical stage of the excavation, is the possibility to perform some basic archaeometrical analyses (chemical and physical) directly on the field.

SLIDE 27

Considering the composition of any archaeological layer based on two different elements, the skeleton (macroscopic) and the fine earth (microscopic), it is obvious that different analyses can be performed in different work environment.

SLIDE 28

For instance, in the case of the skeleton, a fast petrografic (ontoscopic) analysis can be easily performed directly on the field (defining allogeneic elements), while further (more specific) investigations need an equipped laboratory.

SLIDE 29

Also in the case of fine earth, some raw descriptive analyses can be performed on the field, while laboratory investigation can reach very detailed results (e.g. with the Scanning Electron Microscope).

SLIDE 30

The field analysis of the fine earth is more problematic (compared with the skeleton) the most common test (e.g. the Soil texture by feel) are anametric and subjective

SLIDE 31

For this reason, archaeometric test are the better choice (e.g the sedimentation test)

SLIDE 32

The sedimentation test on the field can be improved with basic physical analysis (e.g. considering the Stoke's Law in order to define sand, silt and clay by the tme they need to sediment)

SLIDE 33

Another implementation on the field for the sedimentation test is the possibility to directly store the data into a PostreSQL/PostGIS database (through some specific fields of the archaeological recording sheet), using the open source application geTTexture.

SLIDE 34

An example of the use of geTTexture

SLIDE 35

Other archaeometric test which are simple to perform directly during the excavation are based on basic chemical analyses, and specifically with the quantification of compounds like phosphates or nitrates.

SLIDE 36

Moreover, with some simple workarounds, it is possible to turn anametric (boolean) analyses of carbonates or organic substances, into metric (quantitative) observations.

SLIDE 37

The Archaeological excavation is a destructive process, subject to fatal (not reversible) errors. Moreover the reduced time and budget in professional and emergency archaeology increase stress conditions during decision making stages.
Real-time 3D mapping can speed up data interpretation, avoiding data selection on the field, while on-site chemical and physical analyses (geoarchaeology and archaeometry) can define a better (data-driven) digging strategy.


I hope this presentation can be useful. Have a nice day!

QGIS Time Manager, for archaeological time series

Hi all,

I am back from the CHNT conference, which was held, like every year, in Vienna. There I had many feedbacks and the possibility to speak with colleagues regarding common problems in our profession (soon I hope to report some feedbacks from the session I was attending). 

Today I would like to write a fast post about time visualization in GIS for archaeological aims, because I was asked by our friend +Undine Lieberwirth  if we ever faced with this topic and especially if we ever used TGRASS. The answer is yes and this reminded me that we never wrote something about it, so I would like to start here a series of post dedicated to chronological GIS visualization with open source GIS in general (and in particular about some non conventional and alternative use of it), considering also the 4D visualization tasks.

By the way, today I will start with something simple, just showing an interesting tool of +QGIS : the  Time Manager plugin.

The video below is just a fast demonstration of this tool, with some data coming from an excavation we performed between 2009 and 2011 in the church of S. Giovanni at Massimeno. The raster time series regards the different architectural phases of the structure (from XI to XXI century) we recognized during the excavation.

That's all for now. Have a nice day!

OpenJUMP GIS: from a local (cartesyan) system to a projeced coordinate system

Sometimes we are asked why (after 11 years) we still keep OpenJUMP in ArcheOS, since QGIS became such a functional GIS  and could cover all the feature of the other similar software. The main reason for such a choice are two:

1. QGIS developed very fast and can happen that some tools are still buggy when released (like for the newest georeferencer version)

2. the software (Polygontool) our friend +Szabolcs Köllö (aka Keulemaster) developed for us, in order to handle big data in archaeological surveys, is strictly connected with OpenJUMP

The GIS OpenJUMP

Today I just finished to package for (ArcheOS Hypatia) the last version of this GIS (OpenJUMP 1.9.1) and to upload it in our experimental repository (soon we share it), so I prepared a new videotutorial to illustrate one of the operation in which OpenJUMP is still useful, since the similar tool of QGIS are sometimes buggy: the recovering of old excavation data from a local (cartesian) coordinate system to a projected coordinate system (e.g. in the videotutorial, ETRS89 / UTM zone 32 N).

I hope this can be useful. Have a nice day!

The archaeometric excavation

Last year, on November 28, Arc-Team joined the conference "Lo scavo archeometrcio: scienza e tecnologia applicate allo scavo archeologico" (en: "The archaeometrcic excavation: science and technology applied to the archaeological excavation"), which was held in Rovereto (Italy) at the Museo Civico.

During the meeting we gave a presentation titled "Professional archaeology. Innovations and best practice with free technology. Toward an Open Research." Today I uploaded on our server the slides, so that we can share this work (like always under Creative Commons Attribution - CC BY).

As usual the presentation has been done with impress.js through the Graphical User Interface Strut (both GPL licensed) and it is optimized for Firefox or Iceweasel (better visualized here).

Here is a little explanation regarding the single slides:

SLIDE 1

A fast presentation regarding Arc-Team.

SLIDE 2

An animation representing the importance of geocoding in archaeology (from space to site).

SLIDE 3

Differential GPS and Total Station: the main tools needed by archaeologists on the field (to georeference every single element of the archaeological record).

SLIDE 4

Some examples of geocoding in archaeology: everyday work, project in extreme conditions and missions abroad...

SLIDE 5

... survay and excavations

SLIDE 6

In survay projects the geocoding tolerance for archaeology is higher, so that we are testing alternative solutions to build a low-cost and open source GPS with centimetric accuracy, using the software RTKLIB (or its port in Android)

SLIDE 7

All the recorded data (in 2D and 3D) can be imported into an open source GIS.

SLIDE 8

For aerial archaeology it since 2008 we are working with open source DIY UAV, like the UAVP or the KKcopter (in the slide).

SLIDE 9

Our last UAV prototype and an example of 3D pointcloud form aerial pictures.

SLIDE 10

Since 2014 we are testing DIY camera (using the filter of Public Lab) for NDVI and NGB pictures in archaeological remote sensing.

SLIDE 11

Just removing the IR filter, a normal camera can be used for endoscopic prospections in low light conditions.

SLIDE 12

In the field of geophysical prospections we use a DIY  machine for Electrical Resistivity Imaging. The data can be visualized in a GIS (e.g. GRASS GIS in the slide), using the east and north and the resistivity values.

SLIDE 13

Some geoarchaeological analyses can be performed directly on the field, like the settlement test (using the soil triangle) for the texture or the lithologic recognition for the skeleton.

SLIDE 14

Also some basic analytical chemistry can help during the excavation (giving indications on the ancient use of the soil), to verify the presence/absence of phosphates or of organic remains.

SLIDE 15

Other preliminary laboratory (flotation and sieving) analyses can prepare the samples for further investigation. Also in this case we use a DIY machine.

SLIDE 16

Colorimetry can be performed in many ways. Currently we are testing different options, like the open source spectrometer of Public Lab.

SLIDE 17

For some laboratory geoarchaeological analysis (e.g. microscopic morphology) we use normal optic microscopes, while for more advanced studies we externalize the service (e.g. SEM or energy dispersive x-ray spectroscopy)

SLIDE 18

Currently we are testing the potentialities of the FLOSS MorphoJ to speed up the process in carpological remains recognition

SLIDE 19

To document archaeozoological remains in the field, we use the standard digital documentation techniques (in 2 and 3D), with FLOSS (e.g. bidimensional photomapping with the Aramus method or 3D recording through SfM and MVSR)

SLIDE 20

In the evolutionary anthropology field we developed a new technique (anatomical deformation) thanks to the FLOSS Blender

SLIDE 21

The same software (Blender) is used in the process of archaeological forensic facial reconstruction

SLIDE 22

Open source GIS (e.g. GRASS) are the main software we use to process and manage the recorded data

SLIDE 23

Thanks to open source UAV and Blender we experimented new ways to disclose archaeological data in a four-dimensional way (x,y,z,t)

A more detailed explanation of the entire presentation will come soon with the related article. For the topics which were already discussed in AOTR, I suggest to read the related post (see the above bibliography). For the latest experiment (e.g. near infrared, NDVI and NGB; Electrical Resistivity Imaging; Sedimentation test; litologic recognition on the field; flotation and sieving; colorimetry; microscopic morphology; MorphoJ;), we will try to write something as soon as possible.

Bibliography

Lo scavo archeologico professionale, innovazioni e best practice mediante metodologie aperte e Open research (here on Research Gate and here in Academia)

Webography (from ATOR):

3D and 4D GIS

gvSIG

SfM and MVSR

PPT-GUI (tutorial); from pictures to mesh (PPT, MeshLab, Blender); PPT with Bundler MultiTread; PPT with turntable (not recomended); PPT VS MicMac; PPT VS openMVG; recovering old data; underground documentation; big data; low-light conditions; underwater archaeology; aerial 3D; 3D GIS; anthropology;

Aerial 3D documentation

Caldonazzo Castle (case of study);

Archaeological endoscopy

normal endoscopy; IR endoscopy;

Geoarchaeology

genetic color, framework and edge analyses of skeleton; analytical chemistry on the field;

Archaeobothany

Vegetal macro remains indentification;

Evolutionary anthropology
Anatomical Deformation Technique (ADT): validation; ADT Paranthropus boisei; ADT Homo rodhesiensis;

Archaeoanthropology
Archaeological Forensic Facial Reconstruction (AFFR); Digital AFFR: technique validation; AFFR: state of the arts; AFFR: poster;

Archaeological dissemination
Caldonazzo Castle 4D (case of study);

Project Tovel part 3: georeferencing historical maps

In many archaeological GIS a very important step is the study of historical maps. During the Project Tovel this stage has been a primary target, being strictly related to the 3D reconstruction of the underwater surface of the lake. In fact one of the best source for the bathymetry of Tovel is the plan drawn, between 1937 and 1938, by Edgardo Baldi (director of the "Istituto italiano di idrobiologia Dott. Marco De Marchi" of Pallanza, currently incorporated in the National Research Council -  Institute of Ecosystem Study).
To Import Baldi's map into my GIS, I simply used the "georeferencer" tool of +QGIS, based on the related CTP (Carta Tecnica Provinciale) I loaded previously. The short videotutorial below describes this operation:

Have a nice day!

Project Tovel part 1: open geodata

As many of you know, since some years we (Arc-Team) are supporting Prof. Tiziano Camgna's project regarding the underwater forest of Lake Tovel (1, 2) in Trentino (Italy).
In the past days we are asked to give a lesson about our contribution to the project at the Liceo Scientifico Bertrand Russel of Cles (TN, Italy). In order to prepare some data for the event, I started to work on a 3D map of the underwater surface of the lake and,  to collect material for different ArcheOS videotutorials I recorded almost the complete process. For this reason I start today a series of "minipost" to publish this material, hoping that this will be useful for some readers.
In this first minipost I will write about open geodata, which are very important for archaeologists, but also for other professionals. Before to enter the topic, I have to say that we live and work (fortunately for us) in a Italian province (Trentino) which is sensible to this matter (open data). If you can understand Italian, you should read +Maurizio Napolitano's blog, one of the real expert in this field currently working at the Fondazione Bruno Kessler.
During the Project Tovel I needed geographical open data to set up the GIS system with some basic cartography. More specifically I started with the "Carta Tecnica Provinciale" (a technical map of the province), which I downloaded form the "Portale Cartografico Trentino" of the S.I.A.T. (Sistema Informativo Ambientale e Territoriale of the Province Trentino). As you can read in the portal (sorry, just Italian), the C.T.P. (Carta Tecnica Provinciale) is available Under the Creative Commons Attribution 2.5 terms.
The videotutorial you can see below just show the simple process to access and download these data (in raster or vector) form the webgis service.

I hope this first post was youseful for some of you, have a nice day!

OpenJUMP, auto assign attribute

This post will present a new videotutorial for ArcheOS 5 (codename Theodoric), regarding the software OpenJUMP.

Like for this other article, also in this case I chose to use a real project, to show the potentialities of ArcheOS in different archaeological missions. The main objective of the work was the inspection of an high mountain area (more than 3000 meters above the sea level), in order to verify the possible presence of historical remains connected with the World War I. One of my specific needs to prepare the mission was the setting of a GIS system, updating a geological vector map of the whole province in which I was operating. Luckily this province (Trentino) is at the forefront (in Italy) for the distribution and use of geographical open data, so I had no problems in finding the base map I mentioned before; the main work has been the updating of the database, connected with the vector layer, in which I had to insert some additional informations that were stored in an external spreadsheet.

In short, what I did in OpenJUMP is:

1. Query the Area Of Interest (AOI) of my project in the vector base map, in order to visualize a numerical code that was connected with the additional data in the spreadsheet

2. Edit the database schema of the vector map to add two new fields for the additional informations

3. Check the spreadsheet to read the values connected to the numerical code of the vector map

4. Query the vector map to select all the regions with the numerical code of my AOI (which would share with it all the same additional informations)

5. Use the Auto Assign Attributes Tool to fill (for all the selected regions) the new empty database fields with the right values

6. Repeat the operations till all the vector map has been updated with the additional informations

This videotutorial shows the main operations. I hope it will be usefull for you. As usual I uploaded it also in the DADP wiki.

Have a nice day!

QGIS: exporting 3D data in threejs

Hi all,

I go on recording small videotutorial regarding the software in ArcheOS 5 (codename Theodoric), trying to collecting more material for the official documentation.

In order to avoid the creation of "wasted food" (videotutorial which are not connected with a real project risk to be useless because too theoretical and too few practical), I am collecting examples from our (Arc-Team) work.

This time I will show how to export 3D data from QGIS and visualize them in a browser thanks to the nice plugin "Qgis2threejs". I I had the necessity to do this kind of operation just to create some screenshot to complete this very simple illustration that gives a geological overview of the working area:

Of course this is not the only way to collect 3D views (I could do the same in GRASS with Nviz), but this workflow is very fast, for a small project.

Here is the videotutorial (I hope it will be useful):

As ususal, the video is uploaded also in our Digital Archaeological Documentation Project.

Have a nice day!

Polygontool

Hi all,

this fast post is intended to be an overview of the new open source software Polygontool, an application our friend +Szabolcs Köllö (aka +keulemaster) developed for Arc-Team. This tool is helping us in defining an automatic data processing protocol, in order to directly convert raw data files (collected with RTK GPS or total station during survey campaigns) into GIS readable formats. Currently the tool is under an hard test phase, being used during an interreg project (leaded by +Rupert Gietl) about the Great War between the Austrian and Italian border, but it had already positive effects on our work-flow, reducing the time expensive operations of manual data processing. The short video below is a demo to explain how the software works and what it can do.

The source code (in Python) can be found on github and it is already usable (if you want to test it) and open to contributions (if you want to help us in the development). Currently the configuration files (in the "config" folder) are optimized for our interreg project, but you can, of course, modify the terminology to make them fit to any other archaeological database.

Soon I will post other reports about Polygontool. By now I hope this preview will be useful for some of you (and maybe for us, if someone will join the project).

Have a nice day!

The Austro-Hungarian emplacements on top of Mt. Roteck

(2390m)

Dolomites / South-Tyrol 

 A case study for extensive survey and documentation on occasion of the 100th anniversary of the beginning of WW1 on the Italian front in May 2015.

As reported on ATOR in summer 2013, Arc-Team is pushing ahead the plan of  mapping extensive areas of the high alpine frontline of WW1 from the Swiss border to the Dolomites.

Our approach consists in a very detailed DGPS-survey, terrestrial structure from motion, geolocalized images, archeological description and aerial survey by our drone.

Of course we are basing the whole working process on Open Source Soft- and, where possible , also on Hardware.

Now we want to share the latest version of a presentation, given originally in occasion of the 7th Fields Of Conflict Conference in Budapest (Hungary) in October 18.-21. 2012.

It outlines the characteristics of the high alpine working environment, the nature of the WW1 remains, the challenges to meet, our project strategy and first results.

„@MAP“ the Arc-Team Mobile Mapping Platform

In Summer 2013 Arc-Team was charged with the task to survey a micro-DTM on an archaeological area of about 10.000 m2.

The underlying archaeological remains on the side cause small differences in height on the surface and the shape of a nearly 60 x 60 meters large structure was known from aerial photographs.

We've had only 10 hours of fieldwork at our disposal and exploring our options we've made some numbers games:

• Doing the job with total station would allow us to take an average of 5 points per minute, which means a (very optimistic) total amount of 3000 points in 10 hours. (2 operators)

• Using our DGPS, the stroke per minute increases up to a maximum of 15 points per minute, working with continuous point capturing mode, having an operator on the field who's stepping forward, putting down the pole and balancing the bubble eyery 4 seconds. The total amount in this case is about 9000 points. This means an average of only 0,9 points / m2. That would be far to few...

So what would we going to do?

In this occasion we've had the idea to adapt a monocycle in order to have a rollable vehicle carrying the GPS antenna and maintaining a constant distance to the ground.

The result you can admire in the illustration below.

By the help of this tool we were able to increase the stroke on 42 points / minute and a total amount of almost 25.000 points. This means an average of at least 2,5 points / m2.

The result of our efforts was quite lovely: GRASS GIS produced a high quality DTM from which we derive 3D views, isolines and shaded reliefs.

The official name of the trolley is „@MAP“ Arc-Team Mobile Mapping Platform. ;-)

Digital Archaeology at Lund University

This year, as usual since 2011, +Alessandro Bezzi and me taught some lessons during the course "Archaeology and Ancient History: Digital Archaeology, GIS in Archaeology" at Lund University, held by +nicolò dell'unto. We used the opportunity to update the presentation with which we always start the first lecture. Here below you can see its last version, done with impress.js (just click on the first slide and us the spacebar to navigate).

For a better view, click here

The main topic is digital archeology (or "computational archeology", as it is also known in Italy). 

Initially we define five main operations that are common to any archaeological project: data acquisition, processing, management, analysis and sharing. The first three steps refer to the documentation work-flow, while the last three actions are related with the real research process (of course data management is in common with both of the phases).

Thereafter we analyze each step, starting with data acquisition, which is mainly based on hardware devices. During this operation are normally registered two elements, points and pictures, in order to virtually recover what the archaeological excavation is destroying. With points and pictures it is possible to document objects (artifacts and ecofacts) and actions (basically the archaeological samplings), and their elaboration or, in some cases combination, allows the researchers to record lines, polygon, 3d surfaces and real volumes, to register also the most complex elements of the archaeological record (layers, structures, etc...).

On the contrary of what happen with data acquisition, data processing is mainly based on software. Nowadays it can be divided into two orders of operations: standard procedures (raw coordinates elaboration, 2D photomapping, 2D vector archaeological drawing) and advanced techniques (3D restitution, volume calculation and 3D modeling). The very first and basic step to visualize recorded data is to elaborate the raw coordinates, registered with a total station or a RTK GPS, into a GIS readable code (e.g. CVS or WTK). Combining points and pictures is also possible to create georeferenced photomosaic, using a photomapping techniques (e.g. the metodo Aramus, the Khovle method or the newest Corte Inferiore method). Once obtained a complete georeferenced photomosaic it is possible to use a GIS to draw over the raster level, using one or more vector layers and to connect them with a database. Advanced techniques of documentation are more directly related with 3D and can be based on different methodology to extract morphological, topological and metric informations from one or more pictures (e.g. SvR, SfM, IBM, 3D photogrammetry, etc...). With these informations it is possible to calculate the real volumes of the elements of the archaeological records and use this data to reconstruct the depositional and post-depositional processes, using, when necessary, 3D modeling. Normally, during the different work-flows that can be involved in data processing, many kind of informations are elaborated with raster, vector and voxel graphic in 2 (x,y), 3 (x,y,z) or 4 (x,y,z,t) dimensions. The final aim is to set up a system which is able to handle such a variety of data and this system is the GIS.
In fact GIS software, combined with DBMS, are extremely useful during the data management phase, exactly for their capacity to handle different kind of informations (as many as are the disciplines or sciences which help archeology in its task). The use of such instruments helps to optimize the research, especially in comparison with the traditional techniques, not only during data management, but also during the more delicate stage of data analysis (when most of the cognitive processes are involved).
Among other things, in this fourth step, it is more evident the importance of using open source software and tools to maintain a continuous control on every single process of a study that can lead to the elaboration of new theories. Of course, not all the the analysis are equally sensitive under this aspect: for the simplest researches (anastylosis, building techniques, basic geomorphology, etc...) it is not strictly mandatory to know the source code of the applications, also because in these cases the main examinations are done directly by humans. On the other hand, for more complex studies (landscape archeology and Cost Surface Analysis, statistics, advanced geomorphology, etc...), it is very important to have a complete access to  the formulas and algorithms used by the software in order to keep an human control and do not completely delegate to the computer, among difficult quantitative calculations, also more delicate qualitative investigations (in which the human operator is still essential). In this way it is possible to correctly study all the different informations collected during the archaeological research, considering, at the same time, future integrations (GIS is an open system under a temporal point of view). The last goal of data analysis is to share results with the (scientific and non) community, which is the best way to improve the archaeological discipline itself, especially exploiting the potential of internet.
This lead us to the final step of an archaeological project (data sharing), which can follow different channels, like traditional publication, e-publication (e.g. webgis), exhibitions, etc... The most important thing, at least for scientific disclosure, is to grant a public access to all the informations used for the study (not only the filtered data, but also the raw data), in order to propose new hypothesis and (at the same time) give the all the necessary elements to verify them (no dogma, no authority principle).
To summarize the meaning of this contribution, considering archeology as a science (empiric approach) and a humanity (speculative approach), we can see how computational archeology helps to improve the scientific (empiric) approach, which is often underestimated, granting a more objective data acquisition and processing respects traditional techniques, especially during the critical phase of the archaeological excavation. In fact, unlike scientific experiments, the archaeological excavation is unrepeatable, being the most destructive approach of the discipline (and, at the same time, the most important).

PS

All the screenshots were done with ArcheOS. Some of them are related with really old projects, slowly we will replace them with more updated images...

OpenJUMP: adding a vector layer with a customized db

Hi all,

today I recorder a fast videotutorial regarding the GIS OpenJUMP (which is one of the Geographic software integrated in ArcheOS).

Normally, when you add a new vector layer in your project, you get a new empty level, without any database schema, so that you can start to draw your feature, but, if you want to add some info, you have to manually describe your db schema, like in the video below:

Of course, if you have to draw many different layers with a common database schema (like always happen in a normal archelogical GIS), this operation can be time-consuming (and boring). For this reason in the videotutorial below I try to show how to write a short script which automatically add in OpenJUMP a new vector layer with a customized database schema:

Since I fear the quality of the video is too poor in Youtube, I prepared an image in which you can see better the source code of the script:

The code of the script

To work correctly, the script has to be placed in your OpenJUMP folder, in /lib/ext/BeanTools/ and, as you see in the video, you have to refresh the menu in OpenJUMP (Customize --> BeanTools --> RefreshScriptMenu) to find it (in ArccheOS Caesar you will find the script already in the menu. Just modify the code according to your needs).

Like always I added the tutorial in our ArcheOS wiki (DADP project), in order to go on in composing a free documentation system for Digital Archaeology: 

• Tutorial 1
• Tutorial 2

I also uploaded the code of the script into a specific github repository, so that, if you want, you can contribute in its development. We can use the comment space of this post for the discussion about the schema and about its possible modifications (or you can simply download the script and modify it in order to fulfill your specific needs).

I hope it was useful, have a nice day!

File conversion with TOPS (Total Open Station)

HI all,

today I had to convert some raw total station data from Trimble file format (.are) into something simpler and more readable, like a csv file (Comma Separated Value), in order to further process my points and load them inside a GIS. To perform this operation I used a specific software which you can find in ArcheOS: Total Open Station. 

This tool is developed by +Stefano Costa and Luca Bianconi. Here you can find the official website.
I record a short videotutorial about the file conversion. You can see it in the video below.

As usual, I uploaded it also in the Digital Archaeological Documentation Project of Innsbruck University. Here is the direct link.
I hope it will be useful.
Have a nice day!

An example of “local reprojection” from WGS84 to Gauss-Boaga Rome40 with SQLite/SpatiaLite

A common problem for many Italian GIS users in archaeology is to reproject vector features from WGS84 to Gauss-Boaga Rome40 (national Spatial Reference System in Italy till 2012). For shapefiles this transformation can be performed by specific GIS tools, often based on PROJ library (http://trac.osgeo.org/proj/), by OGR utilities (i.e. ogr2ogr: http://www.gdal.org/ogr/) or others standalone software.

In SQLite/SpatiaLite the reprojection between 2 different Spatial Reference Systems (SRS) is provided by “Transform” function (for complete description and syntax see: http://www.gaia-gis.it/gaia-sins/spatialite-tutorial-2.3.1.html#t5): it’s possible to convert data in all the many Spatial Reference Systems stored in SpatiaLite’s “spatial_ref_sys” table and coded by SRID (Spatial Reference Identifier) and EPSG dataset. For example, to transform a vector from WGS84 to UTM ED50 it's enough to update geometry field in your table with the following SQL command:

UPDATE your_table SET geometry_field = Transform(geomWGS84, 23032);

where “geomWGS84” is a geometry field in WGS84 system (EPSG:4326) and “23032” is EPSG value for UTM ED50 zone 32 N.

The problem is that the reprojection between global systems often yields an approximate and imprecise result. For reaching a more accurate outcome a “local reprojection” is required: it is feasible using specific parameters of transformation (translation, rotation and scaling), different for each part of territory.

My example is about Veneto Region in Northern Italy. I needed to reproject some points, representing archaeological sites of this Region, from WGS84 to Gauss-Boaga Rome40 – West (EPSG:3003). I tested this transformation in SpatiaLite simply using the Spatial Reference Systems stored in spatial_ref_sys table; so I executed the SQL command

UPDATE my_table SET geometry_field = Transform(geom4326, 3003);

where “geom4326” is the geometry field recording the geometries of my points in WGS84 and “3003” is the EPSG value for Gauss-Boaga Rome40 W. The outcome was not good: the points converted were located till 80 meters far away from the correct position.

In order to reduce this displacement, I read this post of Flavio Rigolon about reprojection with ogr2ogr and I adapted that solution for SQLite/SpatiaLite (but I think it could work also with PostGIS in similar way).

In particular, I added a new Spatial Reference System (SRS) in spatial_ref_sys table:

INSERT INTO spatial_ref_sys VALUES(30033003,'epsg',30033003,'Veneto 3003','+proj=tmerc +ellps=intl +lat_0=0 +lon_0=9 +k=0.999600 +x_0=1500000 +y_0=0 +units=m +towgs84=-104.1,-49.1,-9.9,0.971,-2.917,0.714,-11.68','');

My new SRS is identified by SRID and EPSG value “30033003”, is called “Veneto 3003” and has the same geodetic attributes (projection, ellipsoid, units, etc.) of Gauss-Boaga Rome40, but with the addition of 7 trasformation parameters (translation + rotation + scaling) defined by attribute “towgs84”.

For testing precision of my new SRS I selected 5 points of Veneto Region identifiable in WGS84 maps and in Gauss-Boaga Rome40 maps (CTR = “Carta Tecnica Regionale” at scale 1:10.000). I transformed my points from WGS84 SRS (EPSG:4326) to Gauss-Boaga Rome40 SRS (EPSG:3003) and to my new SRS “Veneto 3003” (EPSG:30033003). In the following images you can estimate the different positions for points transformed respectively in global EPSG:3003 and in local EPSG:30033003.

These two simple plots visualize the displacement (in meters) between correct position and points transformed.

The min, max and mean value of x (longitude) error of transformation from EPSG:4326 to EPSG:3003 is: 18.57, 36.26, 23.23 meters;

the min, max and mean value of y (latitude) error of transformation from EPSG:4326 to EPSG:3003 is: 63.36, 71.95, 68.74 meters;

the min, max and mean value of x (longitude) error of transformation from EPSG:4326 to EPSG:30033003 is: 0.49, 15.14, 4.31 meters;

the min, max and mean value of y (latitude) error of transformation from EPSG:4326 to EPSG:30033003 is: 3.3, 11.6, 6.8 meters.

[Part of this error is due to my test data: the 5 points in WGS84 have been selected from Googleearth and not recorded with a GPS and the same points identified in CTR are affected by the resolution of paper maps (“errore di graficismo” in Italian). I hope to do a more accurate test in the coming months...].

A mean error between 4 and 7 meters is acceptable for my purposes and in general for many archaeological works: in fact this error is not far from the best accuracy of portable GPS device (often used in archaeological surveys) and certainly smaller than positioning inaccuracy of many archaeological sites found in 19^th or in the first half of 20^th century. More accurate parameters of transformation (the 7 roto-traslation and scaling parameters of towgs84) could reduce this error, in particular in the Western and Northern part of Region where the distance from correct position seems to be greater.

That's all. If you know other (and faster) methods or if you detect mistakes in my post, please let me know. Any suggestions are welcome.

Denis Francisci

P.S. To enlarge the first image, open it in a new tab or window!

Which GIS? OpenJUMP

We are often asked why there are so many GIS inside ArcheOS (GRASS, gvSIG, OpenJUMP, Quantum GIS, SAGA GIS, Udig). Today I start answering with OpenJUMP. This is my favorite software for an excavation GIS, due to its fantastic drawing tools (that make it similar to a CAD). So, from my point of view, OpenJUMP is the best GIS to draw vector layers from georeferenced photomosaics, like in the image you see below...

... but of course there are also many other tools, which sometime are usefull to handle excavation data (below you see the "warping" utility). Moreover the database integration in the program is pretty enought for this kind of small projects.

Archaeological drawing from photomapping

IMHO one of the main benefits of photomapping is the fact that everybody is able to do good archaeological drawing. In this way it is not necessary to divide archaeologists between "diggers" (who simply excavate layer by layer)  and "drawers" (who just document). In our (Arc-Team) experience it is never a good solution when someone has to document a situation he does not know (because was exposed by someone else).
In other words, photomapping techniques and GIS allow also normal archaeologists to reach the same quality level of professionals in drawing layers or finds. As an example, the picture below shows the skeleton of a young goat found in Aramus (AM): the georeferenced photomosaic is done with photomapping techniques ("metodo Aramus") and the drawing is done by me (i am not a professional in drawing) using the GIS OpenJUMP (inside ArcheOS).

Georeferenced photomosaic and vector drawing with OpenJUMP