DATASNAP.IO BLOG

Google made an announcement today that could significantly change the beacon landscape and greatly improve adoption of micro-proximity technology. Google has launched “Eddystone” an open source, cross-platform Bluetooth message format for proximity beacon messages.

The Eddystone protocol allows you to do two key things that the iBeacon protocol does not. Firstly the beacon itself sends out a url which means that objects in the real world can tell you about themselves, secondly it does not need an app to work. By removing the need for an app Google has opened up the possibility of frictionless interaction with users on mobile devices as well as providing a way for venues to reach a much larger and more differentiated audience – venues are no longer restricted to loyal customers who have downloaded their app.

This is huge! At a venue you have an audience who are already engaged with your brand. Clients are currently hobbled by only being able to reach users on apps that they control. We expect the Google release to energize the use of in-venue beacons and open up new ways of reaching visitors and new insights into in-venue behavior.

There are two other interesting pieces to the new technology. One is the implementation of the fleet management component of the protocol, which can send diagnostic information back to the developer on battery and temperature allowing you to figure out what beacons might be damaged or have a low battery. Another key platform feature is a security model called Ephemeral Identifiers (EIDs), which “change frequently” and can only be interpreted by authorized users. Check out more details on the Google developer blog here: http://googledevelopers.blogspot.com/2015/07/lighting-way-with-ble-beacons.html

The Eddystone protocol works with both Android and IOS. It is built into Google Maps and will be available soon on Google Now. It’s all open source and available on Github https://github.com/google/eddystone

datasnap.io already supports the Eddystone protocol. Check out our updated event API docs on how to send us data: http://docs.datasnapio.apiary.io/

Details on all of the new functionality are spotty, so we’re testing it all out now. Watch this space for more posts on the dirty details of implementation!

Using beacons to measure a users approximate location can result in a steady stream of sighting events.  As the number of beacons deployed continues to escalate and the number of users interacting with these beacons grow, this stream can quickly turn into a torrent.  Attempting to analyze this raw dataset without a significant effort spent on tidying and cleaning the data is not only impractical but generally culminates in results that are inaccurate at best and are downright misleading at worst.

Datasnap.io has analyzed beacon data from many different venue types, measured using various beacon technologies, and recorded on all major mobile platforms.  A number of insights gained are discussed here.

Beacons can be used for a variety of end uses including:

Creating heat maps

Generating data on when users arrived and departed the beacon area

Calculating how users are moving through venues

For the data to be "tidy" each use case should be looked at as a distinct observational unit and stored in its own table. The building blocks of each table are the same - raw sightings from beacons.  A valid sighting contains at least the following information - beacon_id, user_id and timestamp, and often contains data regarding RSSI. Various vendors speculate on the  event_type (i.e. arrival / sighting / depart), and the dwell_time (time passed since arrival event).  A single row in the datasets then generally look something like the following:

user_id | beacon_id | timestamp | event_type | dwell_time | rssi jon_snow1 | wall_beacon | 132015-03-30 17:00:34 | depart | 95 | -49

Issues frequently encountered...

Depart events with no associated arrival events (and vice versa)

Multiple arrival / depart events in a very short timescale

When a visit is defined by a timestamp and dwell time, multiple concurrent visits to the same beacon for the same user are frequently recorded

Negative dwell times have been recorded

Nonsensical rssi values have been also recorded

From the raw sighting events we can generate the observational units for answering points 1,2 and 3 above.  An example of the individual observations for heat maps is in the following table. If coordinates are given in cartesian format, then a map_id may need to be specified as well, and the precise value of the x,y coordinates can be calculated from a combination of beacon location, RSSI, time of flight, etc.

user_id | beacon_id | timestamp | location_x | location_y jon_snow1 | wall_beacon13 | 2015-03-30 17:00:34 | 51.897222 | -8.47

The observations for beacon visits are:

user_id | beacon_id | visit_start | visit_end | dwell_time jon_snow1 | wall_beacon13 | 2015-03-30 16:45:34 | 2015-03-30 17:00:29895

And for user movement:

user_id | beacon_id_from | beacon_id_to | time_of_transition jon_snow1 | wall_east | kitchen | 2015-03-30 17:15:12

When the data is in a tidy format such as this, it is easy to query and manipulate. For example, to calculate how many users are in proximity to a given beacon at a point in time is straightforward in this format and near impossible looking at the raw data.

- Gearόid O'Brien (Head of Data Science, Datasnap.io)

http://www.linkedin.com/in/gearoid

datasnap.io is a new measurement company focused on building analytics products based on data derived from customer-triggered engagement on mobile devices. While we are a relatively new company we are an experienced team and are already engaging with some of the largest companies in event promotion, sports, retail, hospitality and outdoor advertising. We aim to work with all proximity-platforms that have explicit customer opt-in. We take no ownership in our customers’ data but instead transform their raw data into rich marketing intelligence.

The ideas behind datasnap.io are based on our experience building currency data products in mobile media measurement, mobile advertising and financial sector reporting. The initial conception for the company came from our observations that the current crop of business intelligence tools lacked in terms of support for workflow and syndication of reporting. We initially sought to create better BI for companies that publish data and for whom data is a primary part of their customer value. As we socialized our ideas these resonated with many but it was clear that BI was a very crowded sector and that incumbents like Tableau and Looker were not going to sit still and would eventually solve many of these issues.

Our decision to focus on analytics for iBeacon and proximity technology came from conversations we had with Chip Austin and Brad Farkas of i-Hatch Ventures. i-Hatch was the lead investor in MMetrics, my first VC-backed start-up, and were also investors in Metaresolver, my last company. Earlier this year, i-Hatch led an investment round in Gimbal, a leading iBeacon technology vendor that they helped spin out of Qualcomm.  I met with the Gimbal guys and did research regarding BLE and its adoption in a host of verticals from live events and arenas to retail, hospitality, healthcare, and outdoor advertising. BLE seemed like an awesome free resource that was proliferated by Apple and increasingly by Android licensees. It seemed only a matter of time before Google would add native support for iBeacon like functionality at the OS layer in Android.

What appeared to be lacking in this emerging sector was real analytics and BI optimized for the data that proximity-triggered engagement would yield. Linking disparate event data into cohesive sessions would be a challenging undertaking. The event stream may include BLE (Bluetooth Low Energy) transmitter sighting; push notification receipt and open; in-app engagement, potentially via deep linking; and offline engagement. BI that integrated support for spatial data and user interaction data is needed. Valuable insight will come from new visualization that can, for example, convey a sense of movement between different parts of a venue.

The creation of derivative analytics products by linking proximity data to preceding and subsequent events has the potential unlock substantial new value but creates additional data integration challenges. For example, the potential to link opt-in sharing of time and place coordinates with Web cookies or mobile device ad IDs may yield new metrics used to understand digital ad effectiveness and omni-channel customer engagement. I had prior experience in managing teams that focused on problems inherent in linking data to support these applications and felt confident that it was solvable in this instance.

In my role heading applications for data analytics at Millennial Media I launched Omni Measurement Solutions. Omni Measurement provided metrics for ad effectiveness by linking mobile ad exposure to credit card purchase, online panel, grocery loyalty card, and consumer location data. Of these data sources panel-based metrics to link location data to points of interest like retailer outlet was probably the weakest. The panels are limited in size—up to several hundred thousand or about 0.1% of population. Handset-derived network- or GPS-based location data has error, of up to 200 meters in some cases. While substantial progress has been made in mapping of PoI, these too have error. While media buyers like its novelty, the use of handset-based location data can only support attribution metrics that are directionally correct or that deliver relative ranking of effectiveness.

BLE-derived proximity data overcomes the shortcomings of handset-based location technology linked to PoI. Technologies like WiFi MAC address sniffing can deliver accurate intelligence regarding a broader base of users. A lack of explicit consumer opt-in means these may not be used to link to user-level data and to derive user-level metrics like ad exposure lift. Apple’s iBeacon and similar implementations on Android insure explicit opt-in. While the portion of users that may be reached via BLE-triggered push notifications limit its efficacy for marketing engagement it is more than adequate to support new forms of marketing intelligence. Even with a small percentage of users that have an app that can pick up a beacon signal, that have opted in for location services, and have Bluetooth switched on, this group will be substantially higher and likely more relevant than a group of users that can be reached via panel-based methods.