Automatic Soil Texture Triangle

As the regular ATOR readers know, since some years we are trying to improve our laboratory's technologies in order to achieve a better metric classification of archaeological data and to expand our research interests in other archaeological sub-disciplines and mainly in archaeo-anthropology (as well as taphonomy and mummiology), archaeo-zoology, archaeo-bothany in general (and carpology and dendrochronology in particular), geoarchaeology and archaeometry. 

Soon we will start a series of post about our lab (aka ATLAB, Arc-Team LABoratory), which is evolving fast thanks to the effort of +Gianluca Fondriest  and +Mattia Segata.

Today I want to illustrate one of the tool we developed to speed up the geo-archaeological interpretation of the soil texture of the different layers during our excavations. This project (which is now in its early stage) starts from the need to use the Soil Texture Triangle to help archaeologists (especially the new diggers) in correctly interpreting the texture components using an objective method like the sedimentation test (because often new archaeologists are not comfortable with more subjective analysis like the Ribbon test or the squeeze-ball test). I leave the explanations of the operations to perform on the field for another post, while I want here to show the small software "geTexture", which +Giuseppe Naponiello developed to use automatically the Soil Texture Triangle from internet.
To illustrate how the application works, I recorded a short videotutorial:

The software is currently just in Italian, but we will translate it in English soon. The development is still active and this can be considered just a pre-release (it will be implemented also with a tutorial explaining how to perform the sedimentation test on the excavation). If you want to help us, on GitHub you can find the source code, while if you need already to use the software, here is the link where is is accessible. In the next days I will try to program also a small app for Android devices.

Stay tuned and have a nice day!

Notes

Some of the results of ATLAB can be seen in the article "Lo Scavo archeologico professionale, innovazioni e best practice mediante metodologie aperte e Open Research" (here in ResearchGate and here in Academia)

ArcheOS Hypatia, a new tool for 3D documentation: opnMVG-GUI

In these days we are working very hard to package new software for ArcheOS v. 6 (codename Hypatia). This time we just finished to work on the new GUI +Martin Greca developed for +Pierre Moulon software, openMVG, setting up all the requested dependencies. The result is a new tool for 3D photogrammetry in +ArcheOS: openMVG-GUI. This software can be considered as the evolution of the old Python Photogrammetry ToolBox, but we are currently working to fix some bugs of this application to keep providing it in ArcheOS, since it gave the best results in underground environment documentation.

Here below you an see a fast videotutorial I did for our brand new YouTube channel:

To speed up ArcheOS Hypatia development, we set up an unofficial new repository, which we will use (by now) just internally our society, to be sure that everything works fine before to release it publicly to all the users. Anyway we will share this repository also during the university courses in which we should teach this years, like the one in Evora (Portugal) or the one in Venice, since in this conditions it is possible to work under strict control, avoiding problems in unresolved package dependencies. As soon as the new repository will be hardly tested, we will open it, adding the coordinates to the ArcheOS main branch.

The new GUI (by +Martin Greca) for openMVG (by +Pierre Moulon)

 

PS

If you are interested, there are still available places for the course in Evora (regarding open source technologies and cultural heritage). Here more infos.

Have a nice day!

Nexus 3 videotutorial

In order to help people in testing Nexus package for ArcheOS Hypatia (see the related post), I just recorded a fast videotutorial to show how this simple tool works.

Please note that the videotutorial refers to ArcheOS Hypatia's package, where Nexus binaries are stored in /usr/bin (so that they can be start from any path in the terminal). If you compiled your Nexus version somewhere in your home folder, than probably you will have to activate the binary from that directory.

Here below is the short video; I hope it will be useful.

Soon I will upload also the new 3DHOP package for ArcheOS Hypatia, with some instruction about it.

Have a nice day! 

ArcheOS new issue on GitHub repository (how to)

In order to simplify reports regarding ArcheOS from the community, I just uploaded on YouTube a short videotutorial which shows how to register a new issue on our main repository on GitHub.

Here is the video:

I am sorry for the low quality, but I think you can understand the single steps. Next time I will try to do it better :).

Have a nice day!

Turning GeoTIFF into TIFF + worldfile (QGIS)

hi all,

after some weeks I go on with the videotutorial from the Project Tovel. Until now we saw how to download some Open Data for our GIS, how to load georeferenced raster level in QGIS, how to georeference historical maps.

Today I will show something particular, that probably many of you will not need very often working on landscape archaeological project, but that will be more important to manage excavation GIS: how to turn a GeoTIFF picture into a TIFF + worldfile image.

As some of you will know a GeoTIFF is a particular kind of raster data in which the georeferencing values are embedded within the TIFF itself. This option can be a nice solution for a topographer but it is extremely annoying for archaeologists. The reason is simple: topographers often work on pictures or maps that are ready to be used, without the necessity of any photo-editing, which (on the contrary) is an important phase in archaeological photo-mapping process (e.g. for the "Aramus method"). The primary difference between a GeoTIFF and a TIFF + worldfile image is that it is not possible to modify the first one without loosing the georeferencing values (which are integrated in the picture), while it is possible to perform some photo-editing operations (change the colors, balance the brightness and contrast, etc...) in the second one, without problems, being the geolocalization data stored in a separate file (the worldfile).
For this reason working with raster images and worldfile is often the best choice for archaeological GIS (especially for excavation), where it can be useful to "erase" all the part of the photo which are outside the area of interest (e.g. outside the rectification region) and to take advantage of transparency in overlapping different raster levels (which can correspond to different stratigraphic levels).
As I wrote previously, the videotutorial I prepared using the data of Project Tovel simply shows how to turn a GeoTIFF, currently the unique option for QGIS georeferencing module) into a TIFF and a worldfile, a more useful format, without exiting the software.

Have a nice day!

Fusion + Quantum GIS: an Open Source approach for managing LIDAR data

As I was recently asked by some students here at Lund university to provide them with some "alternative" solution (compared to the ArcGIS-based workflow) to import LIDAR data in GIS, I ran into this software package named Fusion (http://forsys.cfr.washington.edu/fusion/fusionlatest.html). It has been developed by a branch of the US Forest Service to manage and analyze LIDAR data. Basically, by using Fusion, users are enabled to convert .las binary data format into a .txt file. Then, the data can be imported in QGIS and filtered based on the classification codes associated to the file itself. Here you can find a short video-tutorial which can help you in understanding the whole data-processing workflow (p.s. please ignore the MS Windows background! :-) ).

Project Tovel part 3: georeferencing historical maps

In many archaeological GIS a very important step is the study of historical maps. During the Project Tovel this stage has been a primary target, being strictly related to the 3D reconstruction of the underwater surface of the lake. In fact one of the best source for the bathymetry of Tovel is the plan drawn, between 1937 and 1938, by Edgardo Baldi (director of the "Istituto italiano di idrobiologia Dott. Marco De Marchi" of Pallanza, currently incorporated in the National Research Council -  Institute of Ecosystem Study).
To Import Baldi's map into my GIS, I simply used the "georeferencer" tool of +QGIS, based on the related CTP (Carta Tecnica Provinciale) I loaded previously. The short videotutorial below describes this operation:

Have a nice day!

OpenJUMP, auto assign attribute

This post will present a new videotutorial for ArcheOS 5 (codename Theodoric), regarding the software OpenJUMP.

Like for this other article, also in this case I chose to use a real project, to show the potentialities of ArcheOS in different archaeological missions. The main objective of the work was the inspection of an high mountain area (more than 3000 meters above the sea level), in order to verify the possible presence of historical remains connected with the World War I. One of my specific needs to prepare the mission was the setting of a GIS system, updating a geological vector map of the whole province in which I was operating. Luckily this province (Trentino) is at the forefront (in Italy) for the distribution and use of geographical open data, so I had no problems in finding the base map I mentioned before; the main work has been the updating of the database, connected with the vector layer, in which I had to insert some additional informations that were stored in an external spreadsheet.

In short, what I did in OpenJUMP is:

1. Query the Area Of Interest (AOI) of my project in the vector base map, in order to visualize a numerical code that was connected with the additional data in the spreadsheet

2. Edit the database schema of the vector map to add two new fields for the additional informations

3. Check the spreadsheet to read the values connected to the numerical code of the vector map

4. Query the vector map to select all the regions with the numerical code of my AOI (which would share with it all the same additional informations)

5. Use the Auto Assign Attributes Tool to fill (for all the selected regions) the new empty database fields with the right values

6. Repeat the operations till all the vector map has been updated with the additional informations

This videotutorial shows the main operations. I hope it will be usefull for you. As usual I uploaded it also in the DADP wiki.

Have a nice day!

QGIS: exporting 3D data in threejs

Hi all,

I go on recording small videotutorial regarding the software in ArcheOS 5 (codename Theodoric), trying to collecting more material for the official documentation.

In order to avoid the creation of "wasted food" (videotutorial which are not connected with a real project risk to be useless because too theoretical and too few practical), I am collecting examples from our (Arc-Team) work.

This time I will show how to export 3D data from QGIS and visualize them in a browser thanks to the nice plugin "Qgis2threejs". I I had the necessity to do this kind of operation just to create some screenshot to complete this very simple illustration that gives a geological overview of the working area:

Of course this is not the only way to collect 3D views (I could do the same in GRASS with Nviz), but this workflow is very fast, for a small project.

Here is the videotutorial (I hope it will be useful):

As ususal, the video is uploaded also in our Digital Archaeological Documentation Project.

Have a nice day!

Raw data to WKT: a polygon

Hi all,
this is the last (for a while) videotutorial of the mini-series regarding WKT code. This time we see how to draw a polygon. As usual, you'll find the video also on the DADP wiki. Here is the video, I hope you'll find it useful!

Raw data to WKT: a line

Hi all,

Again a short videotutorial about FLOSS in archeology. In the last one we saw how to turn raw data (from the total station) into the WKT code of a point; this time we will see how to create a line. 

This videotutorial is also available on the DADP wiki (I updated the old tutorial). 

Like before, the system I am using is a preview version of ArcheOS Theodoric, done building the iso image just following the instruction of the readme file on the github page.

Now, thanks to +Fabrizio Furnari we also have the project for an ArcheOS manual (still work in progress). If you want to help us, you will find the code on the github page (yes, we like github very much...). 

A great help it would be to translate these videotutorials into text :). If you want to participate to the documentation, you can contact us in many ways (also commenting this post), like the official IRC channel #archeos, or the developer mailing list (for more info, read the text here)

Here is the video, I hope it will be useful!

OpenJUMP: adding a vector layer with a customized db

Hi all,

today I recorder a fast videotutorial regarding the GIS OpenJUMP (which is one of the Geographic software integrated in ArcheOS).

Normally, when you add a new vector layer in your project, you get a new empty level, without any database schema, so that you can start to draw your feature, but, if you want to add some info, you have to manually describe your db schema, like in the video below:

Of course, if you have to draw many different layers with a common database schema (like always happen in a normal archelogical GIS), this operation can be time-consuming (and boring). For this reason in the videotutorial below I try to show how to write a short script which automatically add in OpenJUMP a new vector layer with a customized database schema:

Since I fear the quality of the video is too poor in Youtube, I prepared an image in which you can see better the source code of the script:

The code of the script

To work correctly, the script has to be placed in your OpenJUMP folder, in /lib/ext/BeanTools/ and, as you see in the video, you have to refresh the menu in OpenJUMP (Customize --> BeanTools --> RefreshScriptMenu) to find it (in ArccheOS Caesar you will find the script already in the menu. Just modify the code according to your needs).

Like always I added the tutorial in our ArcheOS wiki (DADP project), in order to go on in composing a free documentation system for Digital Archaeology: 

• Tutorial 1
• Tutorial 2

I also uploaded the code of the script into a specific github repository, so that, if you want, you can contribute in its development. We can use the comment space of this post for the discussion about the schema and about its possible modifications (or you can simply download the script and modify it in order to fulfill your specific needs).

I hope it was useful, have a nice day!

OpenJUMP: query to extract single points from a general vector layer

Hi all,

today I am working on data elaboration of an archaeological excavation. I decided to record a short videotutorial to show how to perform a very simple query in OpenJUMP (which is a topic I am often asked to explain during lessons about archeology and open source). Before to start, I'll write a short introduction regarding the processing of this data, just to understand why I need to perform this kind of query. 

I worked on this excavations collecting all the data in a local system (with simple 3D Cartesian coordinates), because the job was an emergency archeology project and I did not have the time to set up a geographic coordinate system on the field, before the construction site began. Just at the and of the excavation, I could come with a RTK GPS, to collect some Ground Control Points (GCP) in UTM WGS84. This is the reason why in this time, when I am processing the data, I had to put together all the daily total station downloads in a single cvs file, which I georeferenced in OpenJUMP, using the GCP I collected with the GPS. The problem is that now I will have to separate again the single points, grouping them according to their function (e.g. points for photomapping of area 1, height points of the same area, points for SfM georeferencing, find-points and so on...). To do this operation, I will simply use OpenJUMp (where I loaded and georeferenced the csv file), performing a query on the attribute (the name) of cumulative vector layer , which will tell me where are the the points I need. In this way I can select in the GIS the features I am looking for and put them in a new vector level, copying the db schema of the cumulative layer (name, y, x, z, code). At the end of the process, I will have single separated vector files which I will use in the next steps of data processing (photomapping, 3D, ecc...).

Here is the videotutorial I uploaded in the ArcheOS tutorila wiki (rom Insbruck University):

I hope it was useful, have a nice day!

Photomapping with Quantum GIS (Khovle method)

Hi everybody
Together with Alexamder Sachsenmaier and Alesandro Bezzi i found out a method to create a photomosaic just with QGIS. The problem was to export the single pictures in a good quality and in the size of the whole photomosaic not just the size of the single picture. But this works fine with the print composer of QGIS
So shortly:
1. edith the file of the ground control points to a .csv file
2. import the .csv file into QGIS (plugin is requiered)
3. change the design of the points
4. start the print composer and export the model with the points. Here its possible to set the dpi: e.g. for an area of 3x2m 500dpi gives a resolution of more or less 1mm
5. start the georeferencing plugin of QGIS and georeference the model
6. to export the wordfile from the geotiff of the model, type the following in the terminal
gdal_translate -co "TFW=YES" input_geotif.tif output_tif_tfw.tif
or open the model in OpenJUMP and close it again
7. georeference all the single pictures with QGIS
8. start the same print composer like the one for the model and export all single picures (dont move the pictures)
9. open the model in GIMP and import all single pictures as single layers
10. give the same name to the wordfile of the model and the photomosaic
Alessandro allready maked a videotutorial:

So, I hope this is helpful for somebody...

New tutorial in the Mesh Editing session of DADP

Hi all,
after my teaching in the Master Open Téchne 2013 about Computer Vision (SfM and IBM), i'm updating the Mesh Editing session (english page) inside the Digital Archaeological Documentation Project.

Up to now I inserted four videotutorials:

how to create a poisson mesh inside MeshLab

how to georeference a point cloud in CloudCompare using a Cartesian coordinates system

how to erase not useful points inside CloudCompare

MeshLab: difference between vertex color, raster color and texture 

how to import Bundler's camera positions inside MeshLab

i hope this is useful :)

ArcheOS 5 (Theodoric): testing Vokoscreen

One of the main problem in doing videotutorials for ArcheOS 4 (Caesar) regarded the possibility to record the desktop activities when using 3D applications. Here is my report about this topic in ArcheOS ML. As you can read, the software we chose (recordMyDesktop) was not able to record videos (just a series of screenshots) of windows displaying 3D objects (like other similar tools: Istanbul, Xvidcap, Shuter). 

Since I was using ArcheOS Caesar, I solved the problem with glc, before, and with ffmpeg, later (thanks to this post of Cicero Moraes). These two options have certainly high performance, but they are not user friendly for newbies. 

Now that we started again with the development of ArcheOS 5, I think we will need a simple tool to record video of 3D applications and maybe I found a good solution: Vokoscreen.

The video below shows a fast test I did with this software. The data comes from the Taung Project and were developed by Cicero Moraes.

With this new tool I hope that ArcheOS users will help us in doing videotutorials or simply in recording demonstrative videos of Theodoric in action.

New tutorial on Python Photogrammetry Toolbox: check camera database

Hi all,
i uploaded a new tutorial on Python Photogrammetry Toolbox. It is available here. It shows how to insert a new camera image sensor (CCD width) into the Camera's Database of PPT.

This is needed to elaborate pictures of a new digital camera. Look this tutorial to go on with the elaboration of the images.