Scribble

Ben Shneiderman - HCIL University of Maryland

www.cs.umd.edu/hcil  [email protected]  @benbendc

11th June 2012, Future Interaction Lab, Swansea University. 

  Brief History

Ben has worked on design ideas, input devices, output devices, social media, help tutorials, teaching, search and visualisations.

Pride in serving 5 Billion users with a diverse multitude of apps and interfaces.

Successfully affected the development of a wide variety of interfaces across a huge range of platforms.

  Info Vis

Visual bandwidth is huge, and the human perception of it is remarkable

Trends, clusters, outliers are easy to spot, humans very good at recognising patterns.

A lot of big business buying visualisation companies.

Eg, Spotfire: Pioneer of software that ran realtime queries, selected data, contained multi-filters over millions of data

Used in big pharma, to identify the role of Retinol in embryos and vision.

  Over time found multiple 2D displays better than fewer 3D - cordinated multiple displays highly useful, all 2D

100M pixels and more, spatially stable displays, arranged in meaningful relationships via proximity

EG, corporate headquarters and NASA control rooms

Smaller screens such have tablets and mobile phones have become increasingly popular too.

  Information Visualisation Mantra

"Overview, zoom & filter, details on demand"

Show user everything first, no matter how complex and messy

Allow user to zoom and/or filter data

Then allow user to query details on demand

  Written in paper, that then got 2k cites, and attracted lots of interest & discussion

Tried to represent a human way of navigating data similar to they way we navigate and interact with the world around us.

  Info Vis: Data Types

SciViz; 1D Linear, 2D Map, 3D World

InfoViz; Multi-Var, Temporal, Tree, Network

Multi-variant, hugely dimesional visualisations can be difficult to create, display and use in infovis

flowingdata.com, infovis.org, infoasthetics.com, infovis.net all have some great and not so great examples of visualisations

  Why Visualise?

Anscombe's Quartet:

4 sets of data, each with 12 rows of x and y points. Hard to see any patterns when just tabular formatted data.

Takes a while to identify data trends, points of interest.

Very easy to see when plot on simple charts.

Also very hard to see errors in large data sets

Hospital thought its average age statistics were out, visualised data.

Only then did they notice multiple patients had an age recorded as 999 years old!

Also found other monthly data series which were missing april's month data.

  Some Examples and Previous Projects: Multi Var and Temporal visualisations

  Timesearcher

V1.3 Designed for time series data, used for stocks, weather and genes

Users specified patterns, supported rapid search

Design goal: 200 periods, 5000 stocks, 100ms updates required

KD-tree, quad trees, gridfiles fall out after 6-8 dimensions

constructed using a rapid linear search

Uses the above mantra- allows users to see intial overview, and immediately identify POI.

     V2.0 allowed for 10,000 points, multi-var data

Allowed controlled precision of match, tightness of fit (linear, offset, noise, match)

V3.0 includes forcasting etc.

  Lifelines: Patient Histories

www.cs.umd.edu/hcil/lifelines

Historical medial data, visual overview of issues, updates and an idea of magnitude of each event

Lifelines 2: Contrast+Creatine

Large amount of patient data and histories- millions of people over 20 years.

Allows identification of generally slow or hard to spot patterns.

Designed around ARF;  Align, Rank and filter

Ability to align data by certain events, rank it and create sequence filters. 

  Lifeflow: Aggregation Strategy.

www.cs.umd.edu/hcil/lifeflow

Temporal categorisation of data > data in lifelines 2 format > tree of event sequences > lifeflow aggregation

Visualisation of sequences of hospital vists- where did people go, and what happened to them

Allows identification of "bounce backs"- Patients arrive, are treated in ICU, sent to ward and then bounced back to ICU - means staff missed something.

Can align by any event - enables identification of patients who went to ward before ICU etc

  Treemap: Gene Ontology

www.cs.umd.edu/hcil/treemap

Space filling, space limited, colour coding, size coding, but requires learning.

Practical example: www.smartmoney.com/marketmap

Provides a spatialy stable map, enables identification of differences over time, variance etc.

An example of visualisations giving you answers to questions you didn't know you had.

newsmap.jp - example of Google news treemap

hivegroup.com - example of logistics and supply-chain treemap

Spotfire.com bond portfolio analysis, NY times, gardian all used treemaps in the past.

Voronoi tree maps in NY times piece on inflation.

Also used treemaps for vis of hard drives across multiple computers in an organisation,

shows wasted trash space, directories mirrored across loads of machines.

  Network analysis

Visualcomplexity.com (not visual simplicity! Some great, some bad examples of network visualisations)

Discovery Process: Social Action

Network using links from US Senators that seem to vote similarly to each other.

Filtering weak links shows distint clustering between two communities- Republicans and Democrats

Shows strong, weak and middle postions within parties.

  NodeXL

codeplex.com/nodexl

Netowork overview tool for discovery and exploration in Excel.

Also shows senate voting pattern discussed above

Allows lots of social network data to be used by people with limited programming skills

  "Group in a box" layout: treemaps with node links

Innovation clusters: people, locations, compnies.

11k noes, 26k links

Using vis of clusters using "mouse" shows the animal, input device, mickey mouse

    Summary of projects. 

Check out analysing socialmedia with NodeXL book

All work has tried to affect the world, not just restricted to minor unconnected problems

Tried to focus on UN millenium development goals - a worthy task

Q: WRT clustering algos - How can I prove that this is a sensible network?

A: nodeXL uses 3 diff clusters algos, current fashions push for multiple community membership.

Metric for clusterings are important.  Most clusters works on network connections, but should work on node value too- starting to happen

Goal of vis is insight not pretty pictures, learning better how to do it. Vis integrated with stats is the way to go, produces clues on next stats methods to use.

nodeXL goal was allow access to stats tools without programming tools.

  Q: Have you done any work to identify and explore poor quality data sets (medical as example)?

Favourite example: Medical data showed patient admitted to hospital 14 times, but discharged twice.

Being admitted is a billing event, discharge isn't and gets less attention.

Outliers are obvious indication of POI, either genuine, or errors.

There is a overal commercial need for cleanup of data

Sub Q: If you don't know the truth for events, if there are more than one, how do you choose?

Use context: IE aligned medical data by heart attacks and rank by number of.

Someone had 6 attacks, but this was actually 2 attacks that had been reported 3 times from differnt people.

Make public aware of how poor medical data is!

  Q: What would you say are the low hanging fruit in UN millenium development goals?

Approach to research has shifted dramatically. Previously promoting imperical controled studies was the heart of HCI.

Increasingly aware these don't payout for quetions of insight.

these require much longer term programs.

Some projects require weeks & months to get used to vis tools.- 2-4 weeks with domain expert, 2-4 weeks more on their own, then to solve problem- much longer term than standard small user studies.

An answer is to find agencies, commercial entities, ask them which small problems they are having solving bigger probems, and help solve them.

Believe in breaking the typical separations of research: work should be both basic and applied, be mission driven and curiosity driven.

Getting comanies to work with you is hard, 3-4 months to get to talk to people. But resulting conversations, networking and projects are worth it.

Iv'e been playing around with my Pi for a little while now, and one of the apps I'm trying to build for it requires MongoDB. Unfortunately, the Mongo source is very x86 specific, however whilst the newest version isn't available on the ARM architecture, V1.8 is thanks to some patches from Per Ola Ingvarsson on Github at https://github.com/skrabban/mongo-nonx86.

  So, to build MongoDB V1.8 on your RaspberryPi:

Install the dependent packages, including a minimal build system and LibBoost C++ libs:

sudo apt-get install git-core build-essential scons libpcre++-dev xulrunner-dev libboost-dev libboost-program-options-dev libboost-thread-dev libboost-filesystem-dev

  Clone the Github repo:

git clone https://github.com/skrabban/mongo-nonx86

  And compile and install it! (NB these two steps take a long time (many hours) on a raspberry Pi, I left mine going overnight for this.)

cd mongo-nonx86

scons

sudo scons --prefix=/opt/mongo install

  This will install mongo in /opt/mongo, to get other programs to see it, you can add this dir to your $PATH:

PATH=$PATH:/opt/mongo/bin/

Today some friends and I hit up Derby to attend the second Maker Faire in the UK! It was great fun, a load of very cool projects, a lot of effort and passion from people into making things they love, and a great day out. Colour me inspired!!

Some photos from the faire: http://www.flickr.com/photos/elsmorian/sets/72157630034769870/

The Raspberry Pi comes with 256MB of RAM, included in the Broadcom BCM2835 System on chip, which also contains the CPU, GPU and DSP in the same package.

  That 256MB of RAM is split between the CPU and GPU at boot time of the Pi, and by default is a 50:50 split, 128MB each. This works well for video decoding and 3D graphics, but if you know you won't be using such graphically intensive applications, you can change this split to give the CPU a bit more.

  To do so, you just need to copy the pre-made .elf files to the start.elf file in /boot, and reboot:

  For 192MB for the CPU and 64MB for the GPU:

sudo cp /boot/arm192_start.elf /boot/start.elf

For 224MB for the CPU and 32MB for the GPU:

sudo cp /boot/arm224_start.elf /boot/start.elf

For the default equal split of 128MB for both CPU & GPU:

I have a few applications in mind for my Raspberry Pi, and one uses Node.JS. Compiling and installing node is not difficult, but requires a few tweaks to the standard system before it will compile.

  I'm using the Debian Squeeze image, and though installing the prerequisites and the majority of the build goes smoothly, I ran into a problem building the V8 engine:

error: #error "For thumb inter-working we require an architecture which supports blx"

This is down to Debian using a default of ARMv4 architecture, to try and target the widest range of devices by default. Unfortunately Node.JS requires ARMv5 architecture, but luckily for us, the Raspberry Pi's CPU is based on ARMv6, we just need to tell the compiler to use it!

  To build and install Node.js:

$ sudo apt-get install git-core build-essential libssl-dev (to installed needed packages)

$ mkdir ~/nodeDL && cd ~/nodeDL (to make a temporary place to download and compile)

$ git clone https://github.com/joyent/node.git . (to get a copy of node)

$ git checkout v0.6.15 (to checkout the most recent stable version (at time of writing))

  next we need to tell the compiler to use the armv6 architecture for the compilation:

$ export CCFLAGS='-march=armv6'

$ export CXXFLAGS='-march=armv6'

and then edit deps/v8/SConstruct around the line 82 mark, to add "-march=armv6":

'all': {

   'CCFLAGS':      ['$DIALECTFLAGS', '$WARNINGFLAGS', '-march=armv6'],

   'CXXFLAGS':     ['-fno-rtti', '-fno-exceptions', '-march=armv6'],

 },

Then comment out lines starting around the 157 mark, to remove the vfp3 and simulator parts. Since this is a JSON-like object, remember to remove the comma on the CPPDEFINES line!

'armeabi:softfp' : {

   'CPPDEFINES' : ['USE_EABI_HARDFLOAT=0']

  # 'vfp3:on': {

  #   'CPPDEFINES' : ['CAN_USE_VFP_INSTRUCTIONS']

  # },

  # 'simulator:none': {

  #   'CCFLAGS':     ['-mfloat-abi=softfp'],

  # }

 },

  Then the usual configure, make, make install process, NB I had to manually specify the location of the OpenSSL libpath:

$ ./configure --openssl-libpath=/usr/lib/ssl (to configure the build)

$ make (to compile node (This took 103 minutes on my RPi!))

$ sudo make install (to install)

  Thats it, you should now have a working Node.JS install!

$ node -v should show you the version number

Overall I'm very happy with my Tumblr blog, but the HTML support is something that is sadly not so great - quite a few tags are not supported, including the <br> tag.

Unfortunately this makes it hard to have short breaks in blocks of text between groups of paragraphs - empty <p></p> pairs are also stripped out :(

  However! The <code> tag is not, and using this and a "&nbsp" (non-breaking space) we can create paragraphs that don't look empty to Tumblr and so aren't stripped out, but also don't contain any visible characters, and thus look like an ordinary line break.

Earlier this week I took delivery of my Raspberry Pi, and I've been setting a few basic things up prior to properly playing around with it - I thought I'd document this to help others who were looking to do the same thing.

I'm using the Debian Squeeze image, using the instructions here (I'm using OSX). When I DD'ed it across to the SD card I got the error "dd: /dev/rdisk1: Invalid argument" but this didn't stop it working.

The following is how I resized the Raspberry Pi Debian image to the full size of my SD card, enabling the ssh server, and changing the hostname.

Resizing the SD card partition and creation of a swapfile.

The Debian image is sized to fit on a 2GB disk, if your card is larger than this you will have to resize the partitions manually. Note if this works with no problems you won't loose any data, but just incase it doesn't go well I did it straight after booting my Pi for the first time, and didn't have anything on it that I couldn't afford to loose.

This involves, firing up fdisk, rewriting the partition table, resizing the filesystem to the new partitions, and the creation and mounting of a swapfile.

If you like videos, this one carries out the exact same thing :)

  To start, run fdisk. "mmcblk0" is where my /boot was mounted (IE the SD card), yours might be different- check with "df -h"

sudo fdisk -uc /dev/mmcblk0

(Whilst running fdisk):

"p" (to print the partition table)

"d" (to delete a partition)

"2" (to delete /dev/mmcblk0p2)

"d" (to delete another partition)

"3" (to delete the existing swap partition)

"n" (for creating a new partition)

"p" (for new primary partition)

"2" (for creation of partition number 2)

"157696" (in my case, to create a partition at the same location as the ones just deleted, enter in the old start number of /dev/mmcblk0p2 printed earlier).

Just hitting enter next will use the remaining space of the card, or you can manually specify a size yourself.

"w" (to write the changes)

Reboot the system, (sudo reboot) and the we can resize the filesystem into the new partitions with:

sudo resize2fs /dev/mmcblk0p2 (this will take a while if you have a large card)

Reboot afterwards, and "df -h" will show the new filesystem using all the space of your card!

  To create the swapfile;

sudo dd if=/dev/zero of=/var/swapfile bs=1M count=128

This creates a 128MB file filled with zeros called /var/swapfile, and might take a little while depending on the speed of your card.

sudo mkswap /var/swapfile (to setup the file as swap space)

sudo swapon /var/swapfile (to enable the swap space, this also might take a little while.)

sudo reboot

  All thats left now is to edit /etc/fstab and tell the system to mount the file as swap every time it starts up;

sudo nano /etc/fstab

Whilst in nano, delete the line starting with a hash (#), and paste or type in:

/dev/mmcblk0p2  /       ext4    defaults,noatime,nodiratime     0 0

/var/swapfile   none    swap    sw                              0 0

This tells the system to mount the 2nd partition at / in the filesystem, and to mount it with "noatime,nodiratime", meaning the system will not write when files and directories were last accessed, speed things up a bit. The next line also tells the system to mount the swapfile we made. This file is read at boot time, and tells the system to mount the stated devices automatically each time the system is booted.

"CTRL+W" then "Y" to exit nano and save changes, and then a final reboot is all that's needed.

Change Hostname:

I also changed the hostname of my Pi, to do this:

sudo nano /etc/hostname (and enter the desired name, "CTRL+W" then "Y" to quit) 

sudo nano /etc/hosts and replace raspberry with the hostname you chose above

sudo /etc/init.d/hostname.sh start (to enable the changes).

Enable SSH:

ssh-keygen (I hit enter for all three options to accept defaults and no passphrase)

sudo service ssh start (to start sshd)

Recently I wanted to draw dashed lines in Protovis, this particular case was to represent missing data in a time series line chart.

Protovis doesn't support this natively, and is now no-longer maintained with the Standford Visualisation guys preferring D3.js these days.

However, adding in support for dashed lines wasn't too hard, there were a few posts on google groups mentioning how to add it to V3.2, I was using the latest V3.3.1 (the last and most recent version) so I forked it on GitHub and made the changes, mainly adding in options and defaults for the SVG1.1 stroke-dasharray attribute.

You can then do something like the following to create various dashed lines, see the SVG docs for more info.

context.add(pv.Line) .data(myData) .left(function(d) x(d.x)) .bottom(function(d) y(d.y)) .strokeStyle(function(d) d.colour) .segmented(true) .strokeDasharray('2,2') .lineWidth(2);

NB: You'll need to set the segmented attribute if you want to have parts of lines dashed and others not, etc, just like for colours and other styles.