Between reality and cyberspace

Tumblr has made it harder and harder for me to post uisng Markdown, so I decided to give it up and run a self-hosted blog under a new name:

50 Lines of Code is built upon a bunch of Markdown files that are transformed into HTML using SuSi.

For now, I'll continue cross-posting the blog posts here, but at some point, you'll find new content exclusively on 50linesofco.de

Now that I am winding down a bit, I get to take a look back at what's been a wonderful, crazy year. I would like to take this opportunity to share with you, chronologically, the many people I got to meet or meet again, the things that happened and left their impression on me and take a glimpse into 2016. So if you wonder what's been on my mind: read on.

At Archilogic we've made a range of big leaps forward. * We've rolled out Virtual Reality support for Google Cardboard and Oculus Rift * We improved our realistic lighting pipeline, which is now faster, easily scalable and much more flexible. * We have and still constantly do improve on our user experience, giving it faster loading times, more intuitive controls and better visual quality * Besides OBJ we can now also export to DXF * We have improved our image recognition to create 3D models from 2D floorplans * We are about to launch a few amazing features that I can't talk about yet ;-) * We contributed back to open source, for instance to jQuery and Three.js

I got to play with the LEAP Motion, the Myo, the Oculus Rift and Google Tango as well.

I also had the humbling experience of visiting the LHC at CERN and go inside this impossible structure.

Also, we've grown our amazing team with great new members - it's a pleasure working with each and every one of our team members for me!

As a bonus, I got to play around with Open Data from Zurich:

I am also so absolutely blown away and grateful to have a chance to meet so many bright, kind and amazing people and talk to so many different communities all over the world.

It's been a dazzling 21 events in 17 different countries I got to visit this year! I am out of words for how humbled I am by this - I really hope that I could provide something insightful, useful and fun for each of these. I also met a lot of old and new friends and have been inspired in many surprising and different ways.

I'd like to highlight a few events that stand out for me personally, but that doesn't mean the others aren't extraordinary and great, just a few stood out a bit more: RSConf in Minsk, DaFED, JSConf UY, MediterraneaJS and JSConf US (Last Call).

RSConf was well-done, full of great talks and workshops and speakers, organisers and attendees where great, bright folks and it was a pleasure being among them

DaFED is an exceptional regular meetup run by amazing and lovely folks that gave me an incredibly warm welcome and I am proud to have been part of this incredible event where great minds and enthusiasm come together to create a better future for their local community

JSConf Uruguay had been so supportive to me and managed to balance feeding our brains with inspiring content and making everybody mingle and chat - it has been a blast!

Mediterranea JS was a well-crafted meeting of a lovely bunch of people in Barcelona to discuss tech and community matters that took the best of many other events and combined it in a fruitful gathering that felt like a family meeting, despite the broad range of topics covered

JSConf US, the last call, had all the feels. The brilliant content that pushed things to the edge combined with what matters most in all our lives: Lovely people. It's been a very special event in that it discussed a lot of issues that are bigger than tech. It was focused on enabling people to unfold their full potential, together, as a community. And last but not least: It was a farewell to Laura and Chris, who made all this happen and inspired so many other events and communities to come up and prosper.

So all in all, it's been a blast!

Sweet surprise: SoftwareRenderer

If you wonder if and how you can render a Three.js 3D scene using node.js on the server, you may have been thinking about using a canvas module, such as Automattic/Canvas or run a headless browser such as Phantom or even a full-blown Firefox or Chrome on xvfb.

But you don't need to, thanks to a modified SoftwareRenderer.

The following Gif has been rendered on a server in 1.5 seconds:

Using the new SoftwareRenderer

You find the modified SoftwareRenderer on npm and Github and it's working in the browser as well as on the server with node.js.

Here's an example that renders a spinning cube (or a single frame of the cube, if you're not in a browser) and counts the coloured pixels. This is a simple demonstration of how the renderer is used:

https://gist.github.com/AVGP/88dbd972824eb2e0b2a1

The difference to other renderers is that render() returns an object similar (but not exactly) ImageData:

{ data: Uint8ClampedArray(...), // pixels as RGBARGBARGBA... array width: Number, // width in pixels height: Number // height in pixels }

The data coming from render can be generated and used both in browser as well as on the server. So it's "isomorphic" in current lingo (I'd prefer "universal").

Adventure Time!

With the possibility to render off-GPU and on the server, we get a bunch of interesting possibilities like render in a web worker or.... render an animated GIF from a THREE.Scene on the server:

https://gist.github.com/40e545f9e1b793edbb71

Credits

The original THREE.SoftwareRenderer is part of the Three.js examples but still uses a canvas element.

When going through the sources I found that it actually uses array operations on the pixel array from the canvas context, so I decided to modify it a bit to not create (or use) a canvas and a 2D context from it, but just create and use a ray Uint8ClampedArray which, to my delight, worked just fine.

The original source can be found here.

Heroku's toolbelt is great. It allows you to access all of Herokus important features conveniently from the command line.

But there may be situations where you would like to use the toolbelt but can't, because it doesn't have the desired feature.

That's where the plugins come into play.

Plugins?

The toolbelt has a plugin architecture which allows you to install plugins via

$ heroku plugins:install <Git URL>

for example:

$ heroku plugins:install https://github.com/AVGP/heroku-build.git

The plugins will then behave exactly like built-in commands, for instance:

$ heroku build deploy.tar.gz v1.2.3

The heroku build command does not exist in the toolbelt itself, but in a plugin.

Our first plugin: Hello World

Medias in res - let's build a simple plugin that outputs the infamous "Hello World!" when we run heroku hello in a terminal.

First of all, we need to create a new directory with a Ruby file (convention seems to be init.rb as the file name).

The content of our plugin looks like this:

require 'heroku/command/base' class Heroku::Command::Hello < Heroku::Command::Base def index display "Hello World!" end end

There's not much going on so far:

We include the Heroku Command namespace

We create a new class Hello within that namespace

We inherit from Heroku::Command:Base for that class

We have a index method that displays the words "Hello World"

Cool. Now we can put it on Github and run

heroku plugins:install https://github.com/YOUR_USERNAME/YOUR_REPO.git

and then

$ heroku hello Hello World $

Yay!

Okay, but that isn't terribly useful so far. Let's do something for real!

A real plugin: List Dynos for an application

Now we want to use our newly-gained plugin developer skills to build a plugin that lists the state ("up" or "idle"), name (e.g. "web.1") and application version for each Dyno of a given app.

So we'll be doing this:

Create a new "heroku-dynos" plugin

Have it carry out an API call to the /dynos endpoint (with authentication)

List the dynos of the given application with name, state and code version

To begin, create a new directory heroku-dynos with a fresh, empty init.rb. We'll use that as our clean slate for the new plugin.

First of all, we need to inherit from Heroku::Command::BaseWithApp to get the app variable that is either the name of the current application we're in or uses the --app (or -a for short) parameter to set that name manually.

Let's start with the - adjusted - basics:

require 'heroku/command/base' class Heroku::Command::Dynos < Heroku::Command::BaseWithApp def index display "Hello #{app}!" end end

Notice the 2 changes we've made? Good. Try running it from the directory of a Heroku application or use

$ heroku dynos -a <Name>

Great. Now on to using the Heroku Platform API that has an endpoint to get all the Dynos for an application. Cool.

So we'll need to fetch the list of Dynos via HTTP and parse the JSON response. We create a make_api_request method for that:

require 'heroku/command/base' class Heroku::Command::Dynos < Heroku::Command::BaseWithApp def index display "Dynos for #{app}..." end private def make_api_request(url) url = URI.parse(url) req = Net::HTTP::Get.new(url.request_uri) req.add_field("Accept", "application/vnd.heroku+json; version=3") req.add_field("Authorization", "Bearer #{Heroku::Auth.password}") res = Net::HTTP.start(url.host, url.port, nil, nil, nil, nil, {use_ssl: true}) { |http| http.request(req) } JSON.parse(res.body) end

The new method does not only contain the HTTP request for the API but also another handy detail: Heroku::Auth.password gives us access to the API token that is needed to carry out authenticated API calls.

Now we'll write one more method that actually calls the make_api_request method and displays the Dynos as we wanted - with name, status and version of the code they run:

require 'heroku/command/base' class Heroku::Command::Dynos < Heroku::Command::BaseWithApp def index display "Dynos for #{app}..." list_dynos_for(app) end private def make_api_request(url) url = URI.parse(url) req = Net::HTTP::Get.new(url.request_uri) req.add_field("Accept", "application/vnd.heroku+json; version=3") req.add_field("Authorization", "Bearer #{Heroku::Auth.password}") res = Net::HTTP.start(url.host, url.port, nil, nil, nil, nil, {use_ssl: true}) { |http| http.request(req) } JSON.parse(res.body) end def list_dynos_for(app) dynos = make_api_request("https://api.heroku.com/apps/#{app}/dynos") dynos.each do |dyno| name = dyno["name"] state = dyno["state"] version = if dyno["release"].nil? "unkown" else dyno["release"]["version"].to_s end display "#{name} is #{state} and running version #{version}" end end end

And with that, we have a fresh plugin that gives us a list of the dynos for any of our applications on Heroku:

$ heroku dynos -a some-app Dynos for some-app... web.1 is up and running version 6 web.2 is up and running version 6

Preface

A few days ago I stumbled upon this tweet from @OpenDataZurich:

Hochaufgelöste Höhendaten des @kantonzuerich neu als #opendata verfügbar! are.zh.ch/internet/baudi… by @GISZentrumZH ^si

So I checked them out in the GIS browser and was hit by the possibilities this meant.

The data contains three interesting things:

the output files of the actual measurement done with the LIDAR

the Digital Terrain Model (i.e. the pure terrain without buildings or vegetation) data

the digital surface model (in German "Digitales Oberflächen Modell, DOM)

I am particularly interested in the latter, as it allows for a number of ideas to become possible!

Some of the things that came to my mind:

make an interactively explorable model from the data to allow getting a 3D look at Zurich

hook it up to a physics engine and do a few neat simulations

pair it with VR (e.g. the Oculus Rift or the Google Cardboard) and make it walkable

There's the data. Now what?

Alright, enough ideas, let's go! So first of all, I navigated into the directory that holds all the surface model data and hunted for the tile that I identified using the GIS browser beforehand - in my case the tile 6830_2470.tif that holds part of the city center. This is the one I'm talking about.

Okay, now I have a TIFF image that contains geo data. Now what? Googling brought me to the GeoTIFF website with a lot of information on the format, but I wondered if I really have to parse it myself.

Obviously, I didn't have to. As nearly always npm got me covered with geo-pixel-stream from the folks at MapBox! Ryan Clark, if you read this: Thanks man!

Okay, so I wrote a small script to do some evaluation of what the heck I will get from the file:

var ps = require('geo-pixel-stream'), readers = ps.createReadStreams('data.tif'); console.log(readers.length); console.log(readers[0].metadata); readers[0].on('data', function(data) { console.log("----", data.offset, data.blockSize, data.buffer[0]); });

Okay, so that gave me some information:

{ driver: 'GTiff', width: 2000, height: 2000, numBands: 1, srs: {}, geotransform: [ 683000, 0.5, 0, 248000, 0, -0.5 ], id: 1, type: 'Float32', blockSize: { x: 256, y: 256 } }

Notably:

It's a 2000x2000 "image"

It is a bunch of Float32s

It comes in blocks of 256 by 256 data points

Ah-ha. So I have a stream of floats... that's possibly the elevation, huh?

{ x: 0, y: 0 } { x: 256, y: 256 } 411.1099853515625 { x: 1, y: 0 } { x: 256, y: 256 } 407.8500061035156 { x: 2, y: 0 } { x: 256, y: 256 } 430.29998779296875 { x: 3, y: 0 } { x: 256, y: 256 } 408.260009765625

That sounds about right (checking it with the GIS browser again).

Cool, so I'll turn that into a terrain with Three.js for which I need each block of 256x256 points in one JSON array - that's what linearize.js does:

var ps = require('geo-pixel-stream'), readers = ps.createReadStreams('6830_2470.tif'), fs = require('fs'); var numPoints = 0, blocks = []; readers[0].on('data', function(data) { var blockPoints = [], blockLen = data.blockSize.x * data.blockSize.y; for(var i=0;i<blockLen;i++) { blockPoints.push(data.buffer[i]); } blocks.push(blockPoints); }); readers[0].on('end', function() { fs.writeFile('data.json', JSON.stringify(blocks), function() { console.log("Written " + blocks.length + " blocks to file."); }); });

So we're now having it all in a (large) JSON file (data.json) - ready for consumption in Three.js :)

Making it discoverable

Now that we have the data ready, it's time to put pixels on the screen, isn't it?

For that, we'll read the data.json that is created by linearize.js and create a plane for each block. Each of those planes will represent a block of 256x256 points in our case - and we'll have 8x8 blocks arranged in a grid in the data.

So let's start looking at how to create the planes containing the point data:

function createTerrainBlock(points, width, depth, blockWidth, blockDepth) { var geometry = new THREE.PlaneBufferGeometry( width, depth, blockWidth - 1, blockDepth - 1 ); geometry.applyMatrix( new THREE.Matrix4().makeRotationX( - Math.PI / 2 ) ); var vertices = geometry.attributes.position.array; for ( var i = 0, j = 0, l = vertices.length; i < l; i++, j += 3 ) { vertices[ j + 1 ] = points[ i ]; } return geometry; }

This will create a plane, rotate it so that it's horizontal and set each of its vertices (it's an array in the form of [x1, y1, z1, x2, y2, z2, ... , xn, yn, zn] so we're iterating in steps of 3 through that array) and set the y-coordinate to the point data from the GeoTIFF (i.e. the elevation of that point).

Now we'll have to walk through each of our blocks and use the geometry to create meshes to put on screen:

function createTerrain(blocks, blocksPerRow, blockSize) { var material = new THREE.MeshLambertMaterial(); var meshes = []; for(var i=0; i<blocks.length; i++) { var geometry = createTerrainBlock(blocks[i], blockSize, blockSize, 256, 256); var mesh = new THREE.Mesh(geometry, material); mesh.position.set((i%blocksPerRow) * blockSize, 0, Math.floor(i/blocksPerRow) * blockSize); } return meshes; }

And that's it for the generation and positioning. When you render it, together with some lighting, you get something similar to this:

Nice, isn't it?

Where to go next?

What you can do with this data is up to your imagination now - so go and check out the data in the GIS browser and play around with it.

Old computer artwork, based on a Sinclair.

So, I think it’s time to wrap up 2014.

The year has been a blast and I’m very grateful for the people I met and my friends and family for making this year so wonderful.

I’ve started writing an ebook on web components which has sold 208 times so far.

I had the pleasure to be part of Codebits VII, together with my friends and former colleagues Balgan, Rafael and Pedro(https://twitter.com/pfcabrita) - it’s been an honour to join the Security CTF with these guys and we made 2nd place, being 1st just until a few seconds before the end.

I also gave my “Build your own OS for fun and profit” talk in Lisbon and met a bunch of lovely people (e.g. the amazing David and the famous Christian Heilmann.

In May I had the chance to meet the brilliant community and conference-hosting genius Patrick in Barcelona and get to know Sam and Francisco who showed me around the vibrant and beautiful Barcelona.

In Barcelona I had the chance to talk about web components at FutureJS and the MEAN Stack at WeNode where I met a bunch of lovely, smart people.

A week later I was invited to the 2014 edition of The Geek Gathering in Croatia. I spend a lovely day in Zagreb before meeting the folks in Osijek - thanks to Andrej, Dennis and all the others for The Geeks Gathering and thanks in particular to my friends in Osijek over at Lovely HQ, yes, I’m looking at you Mario and Marina :)

Then the next special shoutout goes to the Swiss (JS) developer community and the Frontend Conf crowd. Thanks to Pierre and Joel for making the various meetups in and around Zurich happen that I had the pleasure to attend, meeting nice folks like Steffi, Gregor or Philip among so many others (sorry for not listing you all here)!

I also had the chance to visit a few nice german developer conferences and different crowds at enterJS where I met Sebastian and Frederic.

I also spoke at the Mobile App Europe and the Dev.Talks, both of which was nice and interesting.

I also had a visit from Jan-Erik here in Zurich and met the brilliant Googler Michael over a burger

.

I also left Centralway to continue my ongoing journey to support and foster the open web, its ecosystem and the community. I am very happy to have found the amazing team at [Archilogic] (http://archilogic.com) and that I am now part of the endeavours to push the web in general and WebGL in particular to new frontiers. for instance I got to work with the Oculus Rift and the experimental WebVR API. Amazing stuff coming out soon and I'm happy to be aboard of such a wonderful team!

I also had the opportunity to meet the lovely and welcoming developer community in Amsterdam for the DevFest NL, it's been nice to meet so many lovely folks and it was a lot of fun!

Then, last but not least, this year I got to work with a bunch of great people, some of which are still in Zurich, some of which aren't. Of those who haven't been mentioned yet, I'd like to particularly give a shoutout to my former team lead Ollie and my mentor and team mate Marco as well as the seasoned Tim and Eduardo; I learnt an awful lot from these people.

It has been a great year and I hope (and believe) 2015 will be a great year as well.

Seasons greetings from my employer and me!

With this pretty winter wonderland you can discover lovely crafted little details by my talented colleauge David or just zone out staring at the snowflakes...

If you're interested in how it's done, you find the code on Github.

What if sourcecode repositories where landscapes that you could fly through? How could that look like?

Wonder no more, try it from your browser!

Visit the codescape here

Codescap.es takes a public Git URL and then visualises the repository content as blocks of different color and height that you can interactively fly through.

Visit the codescape here

Each block represents a file, the color represents the file type and the height the number of lines in the file.

Visit the codescape here

Clicking on a block shows you the line count and name of the file it represents.

Visit the codescape here

Visit the codescape here

CodeScapes is on Github and open source so you can install it locally and use it with your own JSON files for Non-Git repositories or private repositories.

Post your own repository landscapes as reactions on this blog or mention them on Twitter to me, @g33konaut