#gdal

La  società ARCADIA Sistemi Informativi Territoriali srl (http://www.arcadiasit.it/) ha vinto una gara d’appalto che le permette di fornire personale al CCR – Centro Comune di Ricerca dell’Unione Europea con sede a Ispra in provincia di Varese (https://ec.europa.eu/jrc/en/about/jrc-site/ispra).

Attualmente stiamo cercando due collaboratori da inserire nello staff del CCR nei settori hazard e…

In our April session we had a great talk about light pollution and a hands on tutorial about how to use GDAL/OGR via the command line. We started with our guest Helga Kuechly (@helsche), a Berlin & Potsdam based geo-ecologist. She explained to us, “where the light is“. Helga showed what nighttime satellite imagery can be used for and gave an introduction to her research about light pollution.

We uploaded here slides to our new 😃 slideshare account: https://www.slideshare.net/MaptimeBER/where-is-the-light-helga-kuechly-maptimeberlin

Several interesting nighttime image data sources and visualizations were mentioned in her talk.

Helga for example demonstrated a comparison between lamps in Berlin. Even nowadays, decades after the fall of the wall, clearly the former east-west division of Berlin can be seen  from sky.

Very impressive is NASA´s black marble visualization. Black marble images can be found at: http://www.space.com/19049-black-marble-earth-night-photos.html

For anyone who planes to work with nighttime data, Helga recommend some very interesting URLs:

- Berlin Mosaic 2010: http://dx.doi.org/10.1594/PANGAEA.785492

- DMSP & VIIRS data: https://ngdc.noaa.gov/eog/download.html

- Nightpod ISS data: https://eol.jsc.nasa.gov/

-Loss of the night app: http://www.verlustdernacht.de/Loss_of_the_Night_App_engl.html

It should not be unmentioned, that she and her joint researchers did some interesting publications in the field of light pollution:

Light pollution literature database: http://darkshy.org/resources/research/alan-database/

1. Kuechly, H. U., Kyba, C. C. M., Ruhtz, T., Lindemann, C., Wolter, C., Fischer, J., & Hölker, F. (2012). Aerial survey and spatial analysis of sources of light pollution in Berlin, Germany. Remote Sensing of Environment, 126, 39–50.

2. Kyba, C. C. M., Garz, S., Kuechly, H., de Miguel, A. S., Zamorano, J., Fischer, J., & Hölker, F. (2015). High-resolution imagery of earth at night: New sources, opportunities and challenges. Remote Sensing, 7(1), 1–23.

Helga works for the LUP GmbH (http://www.lup-umwelt.de/en/forschung-entwicklung/) located in Potsdam and as a freelancer, e.g. for NGOs like WWF, in the field of nature conservation.

Next to Helga Harald Schernthanner (@hatschito) and Nico Bellack (@bellackn) did a hands on tutorial about the command line in general and its great use, when combined with GDAL/OGR, the geospatial data abstraction library, which is  probably the most well known program library when it comes to geospatial applications. The word cloud below shows where GDAL/OGR is inside.

The final outcome of their tutorial was a heat map, showing all ice cream shops in Berlin, completely assembled just in the command line. Here you can find the slides of their talk: https://www.slideshare.net/MaptimeBER/gdal-ogr-intro2542017 They started with the very powerful wget command. A command to retrieve content from web servers. Wget was used to filter and download OpenStreetMap data. The following request retrieves all ice cream shops in Berlin via wget and the Overpass API: https://overpass-api.de. By the way, another great tool to filter the OpenStreetMap is Overpass Turbo: http://overpass-turbo.eu/

Next you can see the command to download Berlin's ice cream shops: wget -O ice_cream.osm https://overpass-api.de/api/xapi? node[amenity=ice_cream][bbox=12.93,52.45,13.71,52.65] Here we explain point by point, how the command is constructed. In the slides you can find further commands, explained point by point. 1. Wget = command to retrieve files via HTTP / HTTPS / FTP

2. -O = write output to document

3. URL = Url to the Overpass API

4. [amenity = ice_cream] = OpenStreetMap map features:

https://wiki.openstreetmap.org/wiki/Map_Features

5. [bbox=12.93,52.45,13.71,52.65] = Bounding box for the request in Lat / Long lower-left(SW) to upper right (NE) coordinate

Based on this first example, they turned to GDAL/OGR and at the end our we had our first shell script. Here you can see and copy our final small script for your own purpose. Feel free to copy & paste 😃.

######################################################

Our First Shell Script

#######################################################  #Shebang#!/bin/bash 

#wget download data via the Overpass APIwget -O ice_cream.osm https://overpass-api.de/api/xapi?node[amenity=ice_cream][bbox=12.93,52.45,13.71,52.65]

#Convert .osm to shape fileogr2ogr -f "ESRI Shapefile" ice ice_cream.osm -explodecollections

#Reproject to UTM 32ogr2ogr -t_srs EPSG:25833 icecream_shops_UTM33.shp ice/points.shp -overwrite

#Grid the datagdal_grid -l icecream_shops_UTM33 -a count:radius1=1000:radius2=1000:angle=0:min_points=0:nodata=0 -ot Float32 icecream_shops_UTM33.shp ice_cream_shops_4000_4000_mean.tif

To initiate the script, a so called Shebang has to be written: #!/bin/bash To execute the script, store it as .sh file and run it  in the terminal via the command bash + scriptname.sh  Finally, we get an almost automatically derived ice cream heat map of Berlin. Just some styling had to be done in QGIS.

Our Maptime ice-cream shop heat map

To round up our session, Nico briefly showed a web-based GDAL application prototype, he built during a study project at the University of Potsdam, automatically interpolating air pollution data of Berlin‘s BLUME sensor network. Below you can see a flowchart of the work-flow of his prototype:

If any of you have a problem with huge dataset for aerial photo, read this source to compress your data using GDAL.

For me, it works by simply create a *.bat file in notepad++ of mentioned code. If you have a lot of photos, just do it in a batch file by making a list for the whole photos.

As a summary, do this following step:

1. Copy the path of the whole photos in a csv file

2. Run the whole list in *.py to make it run as a batch file in the next processing

3. Run the GDAL bat file

Latest post on my blogblog, feels good to be writing more for it lately, want to try and get more disciplined with this.

If you're working with open source GIS software you'll be using GDAL at some point and if you were cursed by an ancient mirror when your vanity drove you to lust over your reflection in it you may get stuck dealing with reading the proprietary ECW image.

Reviews of my book: "#Geospatial Power Tools" Really encouraging - thanks guys!

Thinking of buying my latest book?  We’ve finally got a few reviews on Amazon that might help you decide.  See my other post for more about the book.  Buy the PDF on Locate Press.com.

Reader Reviews

From Amazon.com

This book makes a great reference manual for using GDAL/OGR suite of command line …, January 24, 2015 By Leo Hsu

“The GDAL Toolkit is chuckful of ETL commandline tools for working with…

on auto-subsetting Google Maps imagery using a point shapefile and Python

I worked up the code below to extract jpegs of Google Maps images at locations defined in a shapefile. Works perfectly so far (barring API timeouts) and should be easy enough to adapt the spatial dimensions of the subset maps as need be. The example below uses GDAL/OGR, pandas, and requests Python libraries, and a sample shp that covers four cities but the code should work for any number of features.