Reduce your PostgreSQL/PostGIS query time by 99%
In our search engine for renewables, we are running a bunch of PostGIS queries, e.g. to determine the wind speed at a user's location. In our alpha search engine, we noticed that the PostGIS requests caused the longest query time of all 25 queries.
While experimenting with PostGIS, we discovered a nice time saver: In the alpha version, we translated the CSV data of measurement points (lat, lng, wind speed) into a shapefile with geometry points representing every measurement in the UK. The result looked like below (Southern England with a zoom on the Isle of Wight):
We extracted the data_point for a given location with the raw SQL statement in Django/Python:
We used the raw SQL statement in Django, since it provided the fastest query response. But it was still almost 300ms. Not enough!
So we looked for an alternative solution based on polygon intersect. Instead of looking for the closest data point to the geographic coordinate of the user's location, we are now searching for the polygon which contains the user's coordinate. Thanks to the WKT technology, intersect lookups run amazingly fast - in our case in 3ms.
If you have a shapefile with the point geometries (you can use QGIS to convert your CVS measurement data into shapefiles), you can use the QGIS geoprocessing function Voronoi Polygon (Vector | Geometry Tools | Voronoi Polygons) to convert the points into polygons for each measurement point/value. The result is a fancy British spider web:
PostGIS queries based on the polygon intersects can be done as follows:
data_point = WindData.objects.get(geom__intersects = location)
Since we do not have to sort the locations of the data points to find the closest data point anymore, the intersect query runs much faster. The WKT technology allows a hash comparison to determine if a point is part of a polygon or not. This amazing development provides the great speed increase.
The performance comparison in the Django shell says it all:
Nice reduction of almost 99% in the query time.
