Icelandic Open Data, lambs and maps
Two weeks ago I attended the first Open Knowledge Foundation Iceland hackathon. The main goal of the day was to work in some visualizations of already published information and, although the number of attendees was quite small as it’s usually the case with first editions, a few interesting ideas were proposed.
Between them, one that caught my imagination was bringing to life the data published by the The Icelandic Agricultural Advisory Centre about the quality of the lamb population in Iceland. This public institution provides professional consulting for farmers and every year they inspect 90% of lambs in iceland and publish a few reports with the results. Thanks to this work they can for example pinpoint the best ~300 farms in Iceland according to the quality of their lambs.
As unfortunately and usually happens with Open Data the first step to tell the story behind these reports is to convert them in machine-readable information. We know this is usually a point hard to understand by non-developers and institutions: the problem with publishing information in PDF is that is not possible to feed these documents into any kind of software in a reliable way. These are formats designed to be read by humans, not by computers. The input for algorithms ideally should an homogeneous, ornament-free, text only format. So that was our first step.
With the help of pdftotext, a utility for converting PDF files to plain text files, we did a first explorative conversion of the PDFs. As you can see the translation of a PDF like this into its text version is far from perfect but the layout is conserved on the crucial tabular data so it’s a good start. After some clean up and manual editing we got our first csv readable files:
Many people –and almost every Icelander– describe Icelandic lamb as the best lamb in the world. This is usually attributed to the fact that the animals roam free during the summer only to be collected again in September during a special meeting called Réttir where Icelandic sheep farmers invite family and friends to help out with the round-up:
Given the lambs stay in wild nature for many months subject to the weather and local climate, we thought it would be interesting to map its quality and some other features and see if we could see some geographical patterns on the data. The first problem we found was that the farm information was not geolocated.
Luckily, Iceland is a country that dedicates many resources to the management of geographical information, in part because the rift associated with the Mid-Atlantic Ridge, runs across the country from the southwest to the northeast. This rift marks the division between the European and North American tectonic plates, and moves at the not negligible spreading rate of ~2 cm per year giving Iceland its earthquakes, its volcanic nature and creating constant changes in the landscape.
The Icelandic National Registry provides access to geo information of every parcel in Iceland and although for the first tests I used some python code to query their server, we thought it would useful to extract and polish this piece of code into its own gem: skra-geo. With this working it was possible to correlate the name of a farm like Bjarteyjarsandur with its coordinates [378995, 435348] and update the csv files.
But wait a moment, those coordinates don’t look like the usual lat-lon pair… Yes, Iceland uses its own reference system called ISN93 (and its newest revision ISN2004). You can learn more about the reasons for its use in this presentation but the main reason is that the aforementioned tension in the rift deforms the network of reference GPS stations constantly, creating a context with very unique challenges. Again, we have open source’d a ruby gem called isnet93 that allows simple conversion between this system and the internationally supported WGS84.
Finally we were able to assign geographical coordinates to each farm in the reports and our csv files were ready to be put on a map. For this task we sticked to CartoDB a great mapping service that makes easy to build visualizations on top of tabulated data. After importing the data, we played with the configuration of the different visual styles and create a custom basemap. After more tweaking here you have an initial release of the map according to the fat in the lambs and the general quality:
$(document).ready(function () { $('#lamb-map-selector').change(function(){ var map = $(this).val(); $("#current-lamb-map").html($("#"+map).html()); }); $("#current-lamb-map").html($("#lamb-by-quality").html()); });
Top icelandic farms in 2013 according to lamb overall quality fat
Two interesting details are evident in the these maps:
Although the quality is very high in all the country, it’s possible to detect small clusters of farms in areas like Grímsnes where it’s possible to find the best of the best. The reason for this concentration could be connected with different reasons like good local practices in the treatment of the sheep or specially good grazing areas.
In the second map, built using the amount of fat in the lambs of the best farms, we can see clearly a front running in the SW-NE axis in the isthmus of the Vestfirðir peninsula. This could initially correlated with the harsh winter conditions in this area, that forces the animals to be better insulated, but further research to discard other factors would be interesting.
We’ve had a lot of fun creating these maps and we’ve only started to scratch the surface of the information. For example, the RML has been publishing these reports for more than a decade now, and it’ll be amazing to integrate all the data to see trends over time. Modern tech tools and fast development iterations are bringing new possibilities to every industry… sheep farming included.













