#dataanlysis

source

Transform your reports with me: Let me and my team develop your … source

To Join My Skill-To-Job Transition Program related to Data Analytics, Data Science, AI, Generative AI and Data Engineering: … source

source

Follow the link below to convert your data to the necessary form to run an Actor Partner Interdependence Model (APIM):

Using beacons to measure a users approximate location can result in a steady stream of sighting events.  As the number of beacons deployed continues to escalate and the number of users interacting with these beacons grow, this stream can quickly turn into a torrent.  Attempting to analyze this raw dataset without a significant effort spent on tidying and cleaning the data is not only impractical but generally culminates in results that are inaccurate at best and are downright misleading at worst.

Datasnap.io has analyzed beacon data from many different venue types, measured using various beacon technologies, and recorded on all major mobile platforms.  A number of insights gained are discussed here.

Beacons can be used for a variety of end uses including:

Creating heat maps

Generating data on when users arrived and departed the beacon area

Calculating how users are moving through venues

For the data to be "tidy" each use case should be looked at as a distinct observational unit and stored in its own table. The building blocks of each table are the same - raw sightings from beacons.  A valid sighting contains at least the following information - beacon_id, user_id and timestamp, and often contains data regarding RSSI. Various vendors speculate on the  event_type (i.e. arrival / sighting / depart), and the dwell_time (time passed since arrival event).  A single row in the datasets then generally look something like the following:

user_id | beacon_id | timestamp | event_type | dwell_time | rssi jon_snow1 | wall_beacon | 132015-03-30 17:00:34 | depart | 95 | -49

Issues frequently encountered...

Depart events with no associated arrival events (and vice versa)

Multiple arrival / depart events in a very short timescale

When a visit is defined by a timestamp and dwell time, multiple concurrent visits to the same beacon for the same user are frequently recorded

Negative dwell times have been recorded

Nonsensical rssi values have been also recorded

From the raw sighting events we can generate the observational units for answering points 1,2 and 3 above.  An example of the individual observations for heat maps is in the following table. If coordinates are given in cartesian format, then a map_id may need to be specified as well, and the precise value of the x,y coordinates can be calculated from a combination of beacon location, RSSI, time of flight, etc.

user_id | beacon_id | timestamp | location_x | location_y jon_snow1 | wall_beacon13 | 2015-03-30 17:00:34 | 51.897222 | -8.47

The observations for beacon visits are:

user_id | beacon_id | visit_start | visit_end | dwell_time jon_snow1 | wall_beacon13 | 2015-03-30 16:45:34 | 2015-03-30 17:00:29895

And for user movement:

user_id | beacon_id_from | beacon_id_to | time_of_transition jon_snow1 | wall_east | kitchen | 2015-03-30 17:15:12

When the data is in a tidy format such as this, it is easy to query and manipulate. For example, to calculate how many users are in proximity to a given beacon at a point in time is straightforward in this format and near impossible looking at the raw data.

- Gearόid O'Brien (Head of Data Science, Datasnap.io)

http://www.linkedin.com/in/gearoid