#nytlabs

Unfortunately this is only a prototype.

Today is my last day at the New York Times. I cant really explain to the internet what this place has meant to me, the things Ive learned here, the incredible people Ive spent time with. And this video certainly doesnt accomplish any of that, but it is the view from the R&D lab, taken with the camera for our infamous mirror (once a minute, that I saved and stitched together), and there has been many a day that we've stood and watched the sky.

PUCK: PUCK of Ubiquitous Contextual Knowledge™ Internet-of-things / ubiquitous computing ... these concepts have been central to NYTLabs thinking over the last year (and related to much of my past work 1, 2). They are old ideas, but we are finally seeing their fruition in interesting consumer products on the market. I'm into two concepts in particular: Twine and GreenGoose. Both are looking at dirt simple, generalized hardware (particularly the mighty accelerometer) that wirelessly spit data up to the cloud. Server-side software can then contextualize that data to serve any number of social purposes, like alerting you when the mail comes (Twine) or keeping tabs on whether the dog is getting fed (GreenGoose). It hits that middle range of Greenfield's notion of scale — the level of the room (as opposed to the individual, building, or street) — which I think is a particularly ripe area. So in tribute I wanted to create our own version at NYTLabs (after acronym brainstorming w/ @mboggie). The idea is a small, generalized piece of hardware that you can attach to a physical thing in your world which can be programmatically assigned to monitor interesting events. It should have long battery life and report data from the accelerometer and other sensors wirelessly. Its logic should be on the server somewhere, not burned into the device, and it should have HTTP and UDP interfaces. And finally, in my twist on the concept, I want it to respond to touch, so that if you pick it up, it turns into a high-resolution controller, like a Nintendo Wii. The idea is to then have hundreds of these all over everything. I've been picking at this the last few months, and finally have a proof of concept running. The core component of PUCK is the XBee. XBees rule. A network of lightweight components is in itself aesthetically appealing, and in essence, all I want to do here is hook a sensor up to one. The XBee is smart enough that this can be done without the help of an Arduino or other microcontroller, which is essential to keeping down the cost. The path I went down and should not have is trying to figure out how to program them directly in python over the serial port. Instead, Andrew Rapp has an awesome java library and a wealth of documentation that is the way to go. I got hung up in that I didn't just want to collect data from the XBee, I wanted to reprogram its behavior on the fly, using 'API' mode — this library has all that hard work in place. In any case, the rest of the design here comprises an accelerometer, tilt switch, and LED. The tilt switch is able to wake up an XBee — it's quite sensitive, so moving the device at all can take it from a mode where its just periodically reporting data and put it into high-frequency reporting mode. Finally, the LED just indicates when the puck is awake. When we get a MakerBot I'll design a housing. On the server side, I'm pushing the data into Redis and rebroadcasting it via OSC. The next step is to figure out how to architect the signal processing and the nature of the interface. But I'm far from a hardware ninja, so I'm pleased to have something together that I can experiment with. Updates to come.

By using a special semi-reflective glass surface, the users of the mirror are able to see both a normal reflection of the real world as well as overlaid, high-contrast graphics. We've dubbed this "augmented reflection". Conceptually, the idea is that our mirror can reveal the halos of data around real-world objects, including ourselves.

JOYRIDE: STREET VIEW VIDEO FROM A STOLEN PHONE My close friend and collaborator, Sue, had her iPhone stolen earlier this month. The thief had it for 5 days, after which he ransomed it back to her. In the meantime, he had it with him as he drove around LA, presumably looking for other opportunities to be an asshole. Our phones, clearly, are really personal devices. When we talk about personal data, the mobile phone is as physical an embodiment of this as anything, a data-sensory appendage if you will. What does it mean, then, when we've been separated from the device? It feels like identity theft as much as the loss of valuable electronics. So when Sue got it back, she felt a bit estranged from it. We wondered about the life her device had had away from her, which led her to use OpenPaths to take a look at where it had been. Sure enough, the thief's home and haunts were pretty readily identifiable. Sue had also seen the last video Id made with OpenPaths and Google Street View, and we decided to make another one with her data. However, I wanted to take it a bit further. As fun as my first video attempt had been, it's a bit impressionistic — you just get this blitz of unconnected images. However, Sue's data had a very clear narrative behind it. We had a collection of points that the thief had visited with the phone, so I thought we should be able to get a smooth path between them. First, I used the Google Directions API to map the likely route that the thief would have taken between known locations, as well as filling in some intermediary points, which was @blprnt's idea from our earlier brainstorms. One of the cool things about the Street View panorama data (described by @jaimethompson) is that it shows the linkages between consecutive images taken by the Google car. So by calculating the heading from one point to the next and heuristically choosing links between panoramas headed in the right direction, we can access all the images taken along the way. Again using heading we can point the camera in the right direction, download the tiles we want, and stitch a frame together. Applying this to the thief's route, we got a complete reconstructed path that plays back much more like a continuous video than my previous experiment (it evens out after the frantic first 30 seconds). It's a bit like if Google was driving the getaway car, starting downtown where the phone was stolen, and traveling over the city until it's finally given back. Of course, we're leaving out the pauses when he wasnt moving, and the temporal displacement of Street View images make this a kind of a weird frankendata — while the video retains some relationship to the truth of the human interaction behind it, it remains a kind of data fiction. Oh, and for those who prefer the written word, theres always the driving directions. Edit: some press love from Gizmodo and Flowing Data

STREET VIEW VIDEO VIA OPENPATHS API [code] Python (2.6), GPL Brainstorming with @blprnt this morning about what people might do with the new OpenPaths API, we thought it would be pretty awesome to see every place you've ever been via Google Street View. Loading all of that up through the Google Maps interface seemed overly burdensome, so we figured there must be a way to pull the static tiles. Turns out there is (though it's unofficial). @jaimethompson breaks it down for you. From there, it was pretty straightforward to pull the points, scrape the images, and assemble the video. It includes points from September '10 to the present and a dozen or so cities, beginning in LA I think, but NYC clearly dominates. Non-urban spots arent captured well, and in Googleland it's never winter. You might also notice that the granularity of the video increases at the end. That's because at a certain point I start using the forthcoming OpenPaths app, which samples periodically, rather than the data from iTunes backups, which only looks at novel locations. The API pulls from both. Want your own? I did this with python as usual — you can grab the code here if you're interested (youll need PIL and the latest OpenCV bindings installed to export the video). This is a bit of a soft launch for the OP API as we gradually work in new features. Let me know if anyone gives it a try (especially if youre using a different language). Noncoders fear not — we'll hopefully be integrating something like this (but cooler and more blprnty) directly into the OpenPaths interface in the near future.

von [nytlabs] ist ein Tool um Sharing Verhalten zu visualisieren. Entwickelt mit Processing and MongoDB.

AN ACTIVITY HEATMAP FROM FITBIT DATA The Fitbit is a very nicely designed device, basically an accelerometer plus clock in a belt clip that will wirelessly sync to a base station. It's marketed as a fitness aid — on their site, you can check out charts and graphs that give you some sense whether you're moving as much as you should be, set goals, and add subjectively reported measures like diet and mood. It's a perfect example of a tool for a quantified self practice. Though fitness is interesting, I'm more fascinated with the possibilities of using movement data for behavior analysis. My intuition is that the patterns of how much one moves over the course of a day or week or month is enough to infer quite a bit about an individual's lifestyle and to potentially drive a predictive model. Fitbit data, once gathered by the base station, is sent to your account on their site, which is accessible through an oauth-based API. Which is awesome. However, I was disappointed that the data they expose is very much geared toward their fitness market, and is preprocessed and interpreted along these lines. Significantly, unless I've just missed something, the raw, minute-by-minute data is not available, and everything is aggregated by day. While I can understand the motivation behind this, there is lot of power in the raw stream that's not exposed. However, prior to the release of the Fitbit API, the developer Eric Blue published a hack in perl that scrapes data from your Fitbit profile page — turns out that the Flash charts they display are powered by an XML feed with significantly more resolution than what is offered via the API. Based on this discovery, I wrote a python script that pulls the granular data. Using numpy and my nascent drawing module, I created a heatmap of my average week in terms of intensity of movement (since 2011-01-01, wearing the Fitbit somewhat consistently). Behavior clearly emerges, and my bike to work each morning dominates. From here, there are a lot of possibilities. Beginning to cluster days by type and to cross-reference this with other vectors, such as location, are my next steps. We've ordered like 10 of these things for the lab, so I'm psyched to see what we can do with them.