This blog has now moved to Medium
I decided to retire this blog a while back. All new posts will now be on my Medium page:
https://medium.com/@5tuartreeves
$LAYYYTER

Discoholic đŞŠ
taylor price
Today's Document

shark vs the universe

Origami Around
almost home

Kaledo Art
Claire Keane
2025 on Tumblr: Trends That Defined the Year
Three Goblin Art

Janaina Medeiros
Xuebing Du
trying on a metaphor
let's talk about Bridgerton tea, my ask is open
h
he wasn't even looking at me and he found me

if i look back, i am lost

seen from Malaysia

seen from Chile

seen from United States
seen from United States

seen from Ukraine

seen from Poland

seen from United States

seen from Switzerland
seen from United Kingdom
seen from TĂźrkiye

seen from Singapore

seen from Malaysia

seen from Malaysia
seen from Greece
seen from United States
seen from China
seen from United States
seen from United Kingdom

seen from Ireland

seen from Poland
@notesonresearch
This blog has now moved to Medium
I decided to retire this blog a while back. All new posts will now be on my Medium page:
https://medium.com/@5tuartreeves

Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
Free to watch ⢠No registration required ⢠HD streaming
(This piece is adapted from a talk I gave at the Interaction18 conference in Lyon, Feb 2018.)
Anonymising and subtitling video with VLC and ffmpeg
This post documents a process I have used for âanonymisingâ video (FSVO âanonymousâ).
Compatibility things: macOS Sierra, VLC 2.2.6, ffmpeg 3.4 (installed via homebrew, and compiled --with-libass flag)
1. Open your video in VLC.
2. In VLC, Preferences menu option, Show All button, then go to Stream Output, Sout Stream, Transcode and select âGradient video filterâ checkbox, âImage properties filterâ checkbox, then hit Save.
2. In VLC, Window menu option, select Video Effects. Hit the Colour tab, Gradient checkbox, select Mode âGradientâ and some combination of the Colour checkbox and Cartoon checkbox. Hit the Basic tab, Image Adjust checkbox and play with contrast and brightness sliders.
3. In VLC, File menu option, Convert / Stream. Drag the video file onto Open Media (yes, should be unnecessary!). Customise, then ensure âKeep original video trackâ is unchecked. Save as File (filename Iâm using here is âanon.m4vâ), then Save... and wait for the anonymised video to transcode to disk.
4a. (I found this step was necessary to ensure the anonymised video played correctly.) Open a terminal window and runÂ
ffmpeg -i anon.m4v anon-fixed.m4v
4b. Alternatively, if you have subtitles to render (I use Aegisub), open a terminal window and runÂ
ffmpeg -i anon.m4v -vf subtitles=subs.ass  anon-subs.m4v
Example
Hereâs an example, using a video of some fool doing a talk (I fully realise that I am mixing repair with correction in this video, but it was for a non-technical audience).Â
Before:
After:
Thoughts on the UKâs Teaching Excellence Framework
On 22nd of June 2017, the UKâs Teaching Excellence Framework (TEF) results were released. While this immediately led to celebratory noises from the various âwinnersâ, the prominent âlosersâ tended to call into question the validity of the ranking as unreflective of actual teaching quality. I want to talk about this response and the context that surrounds TEF, and how my own thinking about it (and metrics in general) differs somewhat from this âaccurate vs. inaccurateâ characterisation of TEF.
Firstly, TEF is of course complex. Wonkhe has a good summary of the TEF mechanism, but Iâll give a short version here. TEF is a UK government-run assessment scheme for rating the âqualityâ of âteachingâ (I use both words advisedly). It requires UK universities submit justifying evidence alongside various metrics (e.g., National Student Survey results, student employment data, etc.). The TEF process then sorts participating universities into âGoldâ, âSilverâ or âBronzeâ categories. These categories are then tied to fees that universities may charge students (although this mechanism is not in place yet). TEF is also conceptually connected to the Research Excellence Framework (REF), which is the UKâs national evaluation of research output conducted every 4â5 years (last time 2014). REF, being older, is far more involved than TEF, relying on a significant number of âpeer reviewâ panels (using that term advisedly too) to inspect selected research output from all UK institutions in order to then rank them (which is then tied to core funding distribution).
Three points to note first:
1. I donât think my observations below are particularly original or surprising.
2. I am no expert in politics or the history of government relationship to universities or in university governance.
3. It should go without saying but quite obviously I do indeed think universities should strive to be excellent in teaching. Just not in this way.
Trend and ideology
TEF is the latest way that successive governments in the UK have sought to introduce (or impose, depending on your point of view) market dynamics on the university sector. Broadly this reflects the advance of something that looks a bit like scientific management. I believe TEF is traceable to longer term trends of ânew public managementâ and managerialism that emerged in the 1970s and 1980s to reshape the public sector, particularly through the application of metricisation, performance monitoring, and introduction of competition mechanisms. Even more broadly, if could be said that TEF is reflective of the logic of neoliberalism that has been adopted (or at least tolerated) by all recent UK governments as a model for most shaping sectors of society: i.e., privatisation, financialisation, deregulation, etc. I say this more as a matter of fact than a suggestion of right or wrong. The core question is the relevance of that kind of model to higher education and universities as institutions.
The REF informed TEF; in fact Michael Barber says as much in his recent speech on the Office for Students. But the conditions for TEFâs appearance have been steadily establishing themselves for a long time. The older version of the REF, the Research Assessment Exercise (RAE), started in the 1980s as the application of public management ideas gained traction. The introduction of university fees for students at the end of the 1990s accelerated this (it was a solution to widening participation and access). All of these incremental moves lead us towards the reconceptualisation of universities as corporations, students as customers, and the introduction of business processes to university management.
In a way this is key to understanding the logic of TEF and the form it takes, i.e., reliance on metrics. Some kind of metricisation is a key requirement for the introduction of market dynamics in that it provides the âsignalâ for market mechanisms to operate. (Note that just because the Gold-Silver-Bronze categorisation system is adjectival does not mean it is no longer functioning as a metric.)
As I mentioned, the ideology of TEF seems to be based in the assumption that market dynamics are the way to improve almost anything. Since this is the essential model to be applied to all sectors of life, there clearly can be no suspension for universities.
This leads me to two big questions. Firstly, the sheer complexity and cost of TEF and REF is enormous. The desire is for TEF to become subject specific, thus incurring even more administrative burden. It is interesting to see that a cost-benefit argument is specifically not used with either TEF or REF in order to justify the expenditure. More on that in the next section.
The second big question here is about what I guess you could call the epistemology of TEFâs instigators. It needs examining. Do they believe metricisation throws into the light some hidden naturally occurring order of things embedded into universities but hitherto invisible? Or do they view it metricisation as a mechanic to manipulate and control the university sector? Again, there is very little forthcoming on this, perhaps because even suggesting such questions threatens to shed some light on the central assumptions of TEF.
Given the trends I think it possible (but not necessarily likely) that current and future governments will gradually attempt to move TEF towards an Ofsted-like approach to assessing higher education, i.e., greater oversight, greater powers, involving in-depth inspections (preferably by peers but there a few guarantees), and stronger sanctions. The use of âTeaching Excellenceâ terminology rather than âStudent Experienceâ implies that indirect measures of teaching quality are only the beginning.Â
This leads to the following points on language and strategy.
Language and strategy
The language of TEF and indeed REF is positioned in ways that make it seem churlish to oppose. I suspect this is intentional, but I donât know. The strategic move here has been to align TEF (aspirationally) with teaching rather than âreported student experienceâ (which would be more accurate). But in line with what I noted above, this is a neat trick since TEF currently only uses indirect measures of teaching in order to map out institutions to the Gold-Silver-Bronze categorisation schemeâââin other words there is no direct examination of teaching at all (e.g., via observation).
TEF is thus explicitly affiliated with categories of âteaching excellenceâ and the rigour-inflected notion of âframeworkâ. This means opposition to TEF then tends to affiliate with inversions of those things, because thatâs how language generally works. No-one can be reasonably be against âteaching excellenceâ, therefore criticising the very idea of the TEF is met with astonishment (compare this with criticisms that merely seek to tweak its metrics to be âmore accurateâ or ârepresentativeââââi.e., that accept the very idea). To critique TEF in this fundamental wayâââessentially a critique of ways of knowing, i.e., epistemologicalâââis thus readily taken to be against âteaching excellenceâ as important (which everyone wants) and therefore âyou are badâ. It would be good to see the discourse advance a bit here.
Institutionalisation
It also seems that, with the TEF and the REF, there is a gradual process of institutionalisation taking place for academics and those working in universities. Surely we should expect academics, as critically-minded people, to be very careful when engaging in metricisation of, say, natural or social phenomena? (Itâs their job!)Â
It goes like this: At first there is outright skepticism. This then crumbles into skeptical participation (with a nudge and a wink indicating âwhat we all really think about itâ). But when the prizes are awarded, the process of acclimatisation is complete, particularly for the winners.Â
Gold-award institutions in particular come to feel that the metric has been truthful and proven what they already knew: that their teaching is indeed excellent. While schemes as TEF âhave their teething problemsâ for some there no longer is much skepticism about the fundamentals of the exercise. The role of self-congratulation is important here. As Emilie Murphy states âby applauding TEF results we implicitly accept this framework and its methodologiesâ.Â
As the acclimatisation proceeds we see an institutionalisation process taking place. Internal, institutional systems are set up to replicate the TEF or REF in order to ensure that in the next round we are favoured / improve / whatever. After this, internal-TEFs or internal-REFs are then connected with performance assessment, progression, and promotion as part of the logic of managerialism. The end result is that externally-specified and mandated metricisation mechanisms come to significantly influence and shape the guts of academic life: how papers are written and published, which research ideas are pursued, how teaching is configured, how students are treated, etc. This is often glossed as âgame playingâ but this overlooks the wider potentially deleterious effects that happen downstream as a result of that game playing.
Of course, it is unfair to paint this picture of acquiescence without noting that schemes like TEF (eventually) and REF (currently) wield huge sticks in order to get that level of compliance. If it was me in charge I accept Iâd probably have to acquiesce too. For TEF the stick is student fee levels, while for REF it is core funding. The stakes are enormous for universities: and you have little choice but to play. Rather than blaming the people involved in the acquiescence or institutionalisation, perhaps itâs more like the frog-in-boiling-water parable.
The really sad thing about TEF and institutionalisation is that universities are already saturated with ways of assessing and improving teaching. Student evaluations of modules, surveys, peer observation and feedback, connecting PGCHE certification to promotion requirements, meetings and workshops on sharing best practice, institutional support (e.g., courses) for developing teaching skills and techniques, etc. There are many obvious ways to improve teaching from a national point of view that donât involve logic of metricisation. While Iâd like to be pleasantly surprised, I do doubt whether the TEF is really going to genuinely enhance teaching.
The interactional âworkâ of video game play
Thereâs a significant and rich body of research on video games, spanning various disciplines. It addresses a wide range of aspects of video gaming, encompassing studies of video gaming cultures, the economics of video games, player motivations, creativity and video gaming, and many more things besides.Â
But little of this focusses on the fine-grained âdetailsâ of actual play as-it-happens: in other words, the âmessy stuffâ. Some time ago (2007-2009) we sought to remedy this in some small way by taking an ethnomethodological approach to making sense of video gaming practices (âExperts at play: Understanding skilled expertiseâ).
Since then, more ethnomethodological and conversation analytic (EMCA) work has been done on video games. Our paper, âVideo Gaming as Practical Accomplishment: Ethnomethodology, Conversation Analysis and Playâ (published in Topics in Cognitive Science) attempts to bring together this literature. It also seeks to do some interdisciplinary work by introducing EMCA studies of video game play to a cognitive science audience, which has, historically, often been interested in the study of games as an approach to understanding cognition (e.g., chess, which also illustrates the overlapped concern with AI).
In our paper (PDF here) we look at a number of things:
- We present a short history of ethnomethodology and conversation analysis and (video) gaming, and a primer / intro to EMCA from this perspective.
- We present a practical âtutorialâ of sorts, in order to introduce EMCA studies of video games. To do this we use a series of fragments of (video) data drawn from different EMCA studies of gaming, we look at how gaming âgets doneâ in public internet cafes, in the home, and online. By gradually âzoomingâ into these settings we try to cover both the stuff that happens âaroundâ video gaming and what happens on screen too.
- We discuss a few analytic challenges to looking at video gaming in this way, and some ideas for future work.

Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
Free to watch ⢠No registration required ⢠HD streaming
Nine questions for HCI researchers in the making
My colleagues Susanne Bødker, Kasper HornbĂŚk, Antti Oulasvirta, and I together have penned a short article that asks nine reflective questions about the practice of HCI research. We think they will be particularly useful for HCI researchers who are new to the game. But they are also the kind of questions we think more experienced HCI researchers could be asking ourselves. In that sense they are thinking tools.Â
Here are the questions:
If you could address just one problem in 10 years, what would it be?
Are you using your unique situation and resources to the fullest?
Whatâs your HCI research genre?
In one sentence, what is the contribution of your research?
Is your approach right for your research topic?
Why is your research interesting?
Can you fail in trying to answer the research problem?
Will your work open new possibilities of research?
Why do you build/prototype?
And here are the answers: Nine Questions for HCI Researchers in the Making (ACM Interactions, Jul-Aug 2016).
Is interface design âjustâ a search problem?
âCan you come up with a (concrete) design problem in interface design that can NOT be formulated as a search problem?â
This was a question posed to me recently by Antti Oulasvirta. It relates to his talk, âCan Computers Design?â, which he gave at the IXDA conference Interaction â16 (slides).
Itâs a discussion that seems to have a long history in HCI. I tried to pick apart a little of it in a recent paper looking at the notion of (scientific) design spaces in HCI research (video).
For now, though, I think there are a number of ways to answer Anttiâs question.
One way is a kind of naĂŻve approach. We take the question âas readâ, at face value, and simply try to answer it.
Answer 1: âNO, all design problems can be formulated as search problems.âÂ
We can pick any bog-standard web design type problem to talk about this. For instance, a simple interface design issue might be the question posed to many designers by their clients of the form âwhere should I my place call-to-action button on this page to increase my sales?â. (This is, of course, assuming that the way Iâm formulating a design problem fits with the sense in which it was meant in the original question! But more about that later.)
For instance, say a company is selling cars or some other high-value and quite configurable product online. They have a purchase page and a âBuyâ button, but where do they put the button to maximise their sales? This design problemâthe clientâs questionâcan be conceptualised as search quite readily by the designer. Letâs do this. One way is to tackle the problem as a simple computational search matter by generating the space defined by all possible button positions to a certain level of resolution (which could be âpixelsâ). This is following what we might call a âzero knowledgeâ model. We could make the search more dimensional through considering variables like colour of the button, the layout of other page elements that must move in accordance with our button, etc. We could then apply something like A/B type testing in order to solve this problemâi.e., treat it in a behaviourist manner. Of course this renders the search space absolutely enormous and intractable.
However, the âOulasvirta Alternativeâ as Iâll call it (see Anttiâs slides) would be a lot more smart than this. Here we could use a more sophisticated approachâdrawing on models of human performance, attention and cognition, perception and aesthetics, etc. in order to shape and therefore solve the design space in a quicker way. Along the way we would assist the shaping of the design space by selecting relevant design variables (button and page items positioning, button colour, sizes, alignments, etc.) for particular design objectives of interest that relate to increases in sales (clutter perception, motor performance, visual search performance, colour harmony, etc.). We get a result having traversed the design space and found the optimal location, colour, layout of the page, etc. We could then validate the result online via A/B testing and find we get more sales on average with the design solution we located.
Letâs try the opposite answer now. Answer 2: âYES, there are design problems that cannot be formulated (solved) as search problems.âÂ
But what if the formulation of the design problem as found in the client statement is actually problematic? For instance we might discover this via user research that we performâââhypothetically letâs say it involves showing prospective users a set of button-related prototypes. What we then (hypothetically) find is that, due to cars being high value and highly configurable items, users really want to be personally guided through a purchase via an online chat interface where they can ask critical questions about the purchase as they make it. In this case solving the original design problem statement involves respecifying the design problem entirely. Letâs now say that our solution (based on ourâââimaginaryâââresearch) is to pop up a âlive chatâ window to start this dialogue with the customer as soon as they reach the purchase page. When we test this we find that the new solution offers a higher sales output compared to the button approach. So, although we might be able to increase sales via the Answer 1 / âNOâ approach above and validate this through A/B testing or other appropriate metric, it seems to be a âlocal minimumâ.
The solution presented here was not possible to arrive at by formulating the design problem in the way that we did to start with (i.e., âwhere should I my place call-to-action button on this page to increase my sales?â) because that formulation required a particular and necessarily limiting form of specification with regard to the goal (increasing sales). We did this because we tackled the design problem as something that is not a formally-specifiable search problemâi.e., we folded user testing into the output of our initial prototype. Doing so meant we discovered that the design problem had to be reformulated as a matter of UI element modality rather than button attributes. So we then revise the design problem like this: âwhat UI element should I employ on this page to increase my sales?â. When we go back to the client with our solution they may or may not be happy with the âpivotâ we performed. They might be happy because we decided to read the design problem as one that was focussed on sales as a metric and re-scope the design problem accordingly. Or they might be unhappy because actually the technical limitations of their website architecture renders our solution impossible for some reason (they can only have buttons on this page for instance) and they really were tying the use of a button purposefully to the âincrease salesâ issue.
In Designerly Ways of Knowing (2007), Nigel Cross argues that design activities involve âabductiveâ forms of reasoning which, quoting March (in turn quoting Peirce) âmerely suggests that something may beâ (p. 37). Itâs this kind of âtalk backâ between problem and solution that is highlighted by Cross in a reference to Thomas and Carroll (1979), where he states that âa fundamental aspect [to design] is the nature of the approach taken to problems, rather than the nature of problems themselvesâ, and then quotes Thomas and Carroll: âDesign is a type of problem solving in which the problem solver views the problem or acts as though there is some ill-definedness in the goals, initial conditions or allowable transformationsâ. Cross then quotes SchĂśn, â[the designer] shapes the situation, in accordance with is initial appreciation of it; the situation âtalks backâ, and he responds to the back talkâ (p. 38). Itâs this that probably characterises the view of Answer 2 / âYESâ best.
Having said all this, I think there are problems with both answers. In both cases we must perform some âtricksâ in order to produce the right result. In the Answer 1 / âNOâ case we pretend that design problems can be atemporally specifiedâi.e., that we can formulate the design problem from the initial clientâs question in its entirety without any role for any processual, iterative way of reaching design solutions, or taking into account the idea that this might then produce new, hitherto unknown factors that then may lead to design problem reformulation. On the other hand in the Answer 2 / âYESâ case we strategically insert some hitherto purposefully âhiddenâ information (obtained via putting a prototype in front of prospective usersâthus generating the information missing from the clientâs question). Through this we construct the original design problem specified via the clientâs question (âwhere should I place myâŚâ) as deficient in some way.
Finally we might say that I myself have also performed a âtrickâ in answering as I have done above because I have excluded the possibility of iteration where it suited me (thatâs because Iâm a Bad Person).
I think we can also unpack the problems of attempting to provide any kind of answer if we think about and unpack the language of Anttiâs original question.
There is a whole set of things going on around the very idea of the formulation of design problems, such as what kind of formulation this might be, what is permissible within formulation activities (its scope) and so on. The formulation I adopted above (âwhere should I my place call-to-action button on this page to increase my sales?â) constructs the design solution in-and-through the process of formulation. For instance, concurrently with problem formulation we also specify a certain kind of inherent scope to the problemâi.e., an ontology that is concerned with buttons, pages, etc. as entities of the formulation. My âtrickâ above then traded on exploiting the language problem via certain implied boundaries of design problem scoping (i.e., ruling out the more general notion of âUI elementsâ).
This reminds me of work on programmable user models, which seemed (to me) to have highlighted this matter in the past. For instance, Butterworth and Blandford (1997) indicate that cataloging âthe knowledge necessary for a user to successfully interact with a device [has shown] that design decisions can be sensibly made without the need to actually run the modelâ (p. 13). In other words, the very work of attempting to formulate a design problem into some kind of formal / technical language itself results in possible design solutions (design decisions) emerging as a matter of that process.
This kind of issue around articulating what we mean when we talk about the formulation of design problems also extends to other concepts in the original questionâsuch as âsearchâ and âsolutionâ. What constitutes a âsolutionâ and by what criteria are we deciding that a solution has been reached? Depending upon how we answer this, we may resolve the type / form of âsearchâ that is meant when we say âsearchâ differently. Is the possibility of search dependent upon things that can be formally specifiable in models? Do we mean âsearchâ in a technical computational sense or some vernacular, ordinary sense (e.g., a âhuntâ, a âdiscovering processâ, etc.)? Mixing the two might be problematic, leading to confusions about the claims being made for computational design.
There is also a further question about what design even is and what might âcount asâ design. Antti asks âcan computers design?â but this presupposes particular senses in which we ascribe âdesignâ to certain sorts of activities. By this I mean if we look at how we might use âdesignâ in ordinary language, then âdesignâ is something that we might only properly say that people do (a bit like âinteractionâ). This sense of âdesignâ suggests human intentionality and all the attendant attributes that we might normally see as relevant (and talkable) topics in some wayâe.g., that some person is considered legally responsible for a design, that they are socially accountable for it, and that they may be the subject of praise about a design done well. None of these things could properly be said of a âmachine designâ if there were such a thing. In this ordinary sense of âdesignâ whatever aspects of design practices become automated simply no longer constitute âdesignâ because they are no longer things we would ordinarily say are âdesignedâ.Â
Said in another way, the question might be âcan machines support design?â to which the answer would be a strong âyesâ.
Talking about âinteractionâ
(This piece was originally presented at a Microsoft Research and Mobile Life workshop hosted at MSR Cambridge by the Human Experience & Design group on the 9th of March 2016. The title of the workshop was âHCI after interactionâ; it was organised in response to Alex Taylorâs ACM Interactions article âAfter interactionâ and the subsequent discussions that ensued on Alexâs blog here: http://ast.io/back-to-interaction. I have adapted and expanded my talk slightly to work more clearly on this blog.)
Reading Alex Taylorâs ACM Interactions article âAfter interactionâ along with some of the discussions online raised some immediate questions about the very idea of âinteractionâ for me:
What drives calls to go âbeyond interactionâ, or consider what might be âafter interactionâ?Â
Does âinteractionâ conceptually no longer articulate the right kinds of ideas when we talk about our (HCI) research?Â
Does it have enough expressive power for us as HCI researchers?Â
Do we as a community need a definition of âinteractionâ to proceed coherently in doing HCI research together in future?Â
... Or is there just too much baggage to the concept of âinteractionâ â has it had its day?
The idea of âinteractionâ is an interesting thing to return to, I believe. But thinking about it made me quite confused. I ended up with a sketch of a discussion about it.
I thought: maybe we should return to âfirst principlesâ, by which I mean questions like these two: 1. What jobs has âinteractionâ done for us in our HCI research communities? 2. What might we mean to say when we talk about âinteractionâ?
For nearly 40 years, âinteractionâ has been used in HCI and beyond as a way of talking about the myriad forms of use that emerge between people and computer systems.
First Iâll recap some of what I think Alexâs âAfter interactionâ article says. Alex argues that âinteractionâ has often been articulated in HCI in terms of âhuman-machine interactionsâ. Alex also argues that âinteractionâ and its sense has been tied to idea of âthe interfaceâ â and this has maybe hindered HCI conceptually. In other words, he says that âinteraction hinges on an outmoded notion of technology in useâ.
I think Alexâs use of âinteractionâ itself might be trading on certain ways of working with the word â I found this interesting, particularly, his coupling of âinteractionâ with âdiscreteâ as in âdiscrete interactionâ. I take this to mean that âinteractionâ has been lacking the kind of expressive power that might help us talk meaningfully, deeply about the embedded and fluid ways with which we are implicated in (or âentangledâ with to use his terminology) technologies in our everyday lives. Alex also ties âinteractionâ conceptually with the materiality of the user interface, arguing that âinteractionâ as a concept has led us to âconcentrate our attentions on the interfaceâ to the exclusion of other things.Â
Finally, I noted that the call sent to âHCI after interactionâ workshop participants argues that the concept of âinteractionâ implies a long âassumed binary of âuser-computerââ. So thatâs another way of expressing similar problems with âinteractionâ.
Now, on to the first question where we ask âwhat jobs âinteractionâ has done for usâ â i.e., as a matter of our discourse, our academic talk, and so on.
Firstly we could say that âinteractionâ has been and perhaps still is a usefully under-defined concept for HCI.
When we talk about âinteractionâ in HCI communities we can and do use it to say a great many things â things that may conflict and be incompatible ways of saying âinteractionâ. We can say that someone tapping a touch screen is âinteractingâ, just as we can say that posting on social media is âinteractionâ, just as we can say that someone being tracked by their location is âinteractingâ with a system, just as we can say âinteractionsâ are taking place with / around / through technologies embedded in the social life of the home, just as we can say someone hiring a Boris Bike is âinteractingâ with a network of systems and data and other people and â even things like âpoliticalâ and âethical worldsâ.
I think the concept of âinteractionâ also brokers relationships between a range of diverse research communities which dip into the HCI cauldron at some point or another (to mix some metaphors). For instance, one way of talking about âinteractionâ in HCI is a kind of software engineering oriented way, which emphasises the parallel workings and misalignments of the user and the machine (makes me think of Suchmanâs studies here). The engineering sense of âinteractionâ is framed in terms of the computerâs technical needs of formatted input, output, events, interrupts, etc. This is a nice point I think Alex makes about Englebart and the Mother of All Demos â talking about âinteractionâ like this is about bringing the machinic requirements to the foreground.
But, I think there are other disciplinary ways we talk about âinteractionâ too. For instance, psychologists and sociologists of different flavours and persuasions have added their own alternative and sometimes incommensurate ways of talking about âinteractionâ in HCI â and of course these then become ways of talking about âinteractionâ into which technologies become enmeshed when they hit the HCI community. For example we might consider how âinteractionâ can be a way of speaking of (on the one hand),
1. a model of stimulation and response between people,Â
which we might contrast with,Â
2. âinteractionâ as an interpretive process performed by members of social groupings.Â
There are of course many more examples like these. The point is that there are many ways of talking about âinteractionâ which can be âat playâ at any time in no distinctly differentiated way when we talk about it in HCI. âInteractionâ, being a promiscuous concept in this way, is probably both good and bad: it fuels a vibrant HCI community but at the same time can submerge perspectival differences.
Next: the second question, about what we might even mean when we talk about âinteractionâ.
I think we have sometimes forgotten to keep in mind that âinteractionâ is a metaphor that is doing some potentially interesting but confusing things for us by blurring the social and the technical.
I came to talk about this with Barry while doing empirical work together on how social media use is embedded into everyday life.
My point is that when we say technologies, systems, devices are âinteractiveâ, we also must necessarily embed them within mundane social order â i.e., our social interactional world. In other words we leverage ordinary understandings of âinteractionâ to talk about what people do with computational technologies at the selfsame time as we might ordinarily speak of what people do with one another.
This socio-technical blurring of âinteractionâ â this drawing of the idea of interaction from our ordinary language â then suggests a relevant family of âinteraction wordsâ like âresponseâ, âreactâ, âalertâ, âremindâ, âinterruptâ, etc. These are all things we might say machines also do.
But, we have to keep in mind these are ways of talking about machines. And these ways of talking about machines are grounded necessarily in ordinary language. This means that these ways of talking about machines, this family of âinteraction wordsâ, borrow from the everyday sense.
So, we might ask, what does it mean for a machine, a computer, a program, an app, a system, a bot, an agent, and so on, to ârespondâ? We say these things might ârespondâ to us but is this a âresponseâ in the social, human interaction sense or some other sense? What does it mean for us to use the metaphor of interaction to talk about and ascribe things like âresponsesâ to technologies â things which ordinarily we might say are things that people do?
At this point we could think about this as a problem of language in use. For instance, I think Ryle talks about something similar when he speaks of philosophical confusions around âthinking wordsâ â so, how we talk about things like our âintentionsâ or our âbeliefsâ in ordinary language compared with philosophical programmes to formally locate or define âbeliefâ or âintentionâ. Iâm left wondering whether âinteractionâ and the way we talk about it is part of a wider set of troubles around how we talk about machines in general. Take for instance the idea of calling machines âintelligentâ and compare it with how we talk about âintelligenceâ in an everyday sense. Do we want âinteractionâ to be usefully ambiguous or will we get caught up, and confused about the difference between the family of âinteraction wordsâ in ordinary language and their metaphoric application to things like machines?
Maybe we can try to sort through these language confusions: when we say a system ârespondsâ to us does this leverage the methods of âresponseâ that people employ in everyday social interactions? Are they the same? Are they different and how might they be? Maybe we should try to describe them? It suggests that we could take a closer look again at âinteractionâ and the very idea. It feels like there is a lot of this ground that has been left unexamined with HCIâs expansion.
In closing, I think there might be value in rediscovering âinteractionâ as a concept. Perhaps we might be a bit more cognisant of the multiplicity of ways in which âinteractionâ â perhaps in an often confused way â lets us talk about what is a massively varied phenomenon. In this way I think many of the valid concerns expressed by Alexâs piece can be addressed in an inside-out, âinteractionâ-first way and not necessarily by doing away with it.
Work with Mixed Reality Lab! Fully-funded 3-year PhD in Internet of Things + UX design + ubicomp research
We are offering a fully-funded 3-year PhD studentship to investigate the intersection of established industrial User eXperience (UX) and design professions, with the increasing availability of consumer âInternet of Thingsâ (IoT) technologies for the workplace, the home, and beyond. As part of this the PhD will explore the relevance of 20+ years of ubiquitous computing research to this area.
Full details can be found here:
http://www.cs.nott.ac.uk/~str/files/iot-studentship-further-info.pdf
To apply, please use the following jobs website for University of Nottingham:
http://www.nottingham.ac.uk/jobs/currentvacancies/ref/SCI1516
Closing date: 30th June 2016 â Interviews: mid July 2016 â Start date: 1st Oct 2016
Things we learned about social media from recording peopleâs use of mobile phones
Research on social media is popular. What does not seem to be so popular is finding ways to closely examine peopleâs actual, real-time use of social media, or in other words, how they use it moment-by-moment. So this is precisely what we did (Barry Brown and myself, with data collection conducted by Moira McGregor and Barry).
But how could you investigate this real-time use of social media as-it-happens? One approach Barry and colleagues employed is to get research participants to screen-capture their smartphone during use while at the same time recording ambient audio via the microphone.
The result is a richly detailed set of data captured from everyday situations where you can see considerable detail of how people interact with their smartphones, browsing Facebook, Twitter or Instagram (to name just three social media systems). Since smartphones are pretty much always with us, you can also hear how participants chat with others as they use their phone. It enabled us to develop a detailed understanding of this interaction, supported by transcription of talk and the visibility of on-screen action.Â
Hereâs a figure from our paper to illustrate:
Iâll return to this example in more detail next. Now, hereâs three things we learned from looking at this data.
1. Social media use is expansive.Â
In the data we often find the use of social media being brought to bear as an interactional resource during the everyday moments. What this means is that social media use itself can move face to face conversations on, such as through jokes or introducing new topics to talk about. What does this look like? Here's a simple example where A and B are chatting about a Facebook post that B has just made. B posts âWhee!â as a status update which A then sees and then transforms into a question for B (âwe?â, as illustrated in animated GIFs below!):
B is a bit confused by this:
B is still confused by Aâs question, but A persists (âyou said WEâ); B essentially verbalises his status update with a high-pitched âoh, wheeee!â, producing a groan of acknowledgement from A (âoh my gawd!â):
So what? In essence this suggests an alternative account to the popularly-held view that the use of social media, and smartphones use in general, constitutes a unique distraction from normal 'morally desirable' forms of social interaction (a view espoused by Sherry Turkle). Such a view is simply not borne out in our data.
The above example also illustrates the next point well.
2. Interaction with social media is finely and intricately interwoven with the proceedings of everyday life.Â
In the previous example, what A posts on Facebook occasions a brief humorous verbal interaction with B. If you look carefully, how B scrolls around his news feed fits in with their conversation. In short, itâs interwoven.
Thereâs a clearer example we can draw on. Here we join three people (a different A, B and C) while C is browsing Facebook on his smartphone. C is taking part in a conversation with A and B. What we draw attention to is just how finely and sensitively coordinated Câs talk is to this conversation and his use of Facebook.
C is scrolling through photos on Facebook. A and B are having a conversation about science fiction and fantasy books. Just as B finishes talking (âdonât they just have like sci-fi and fantasy booksâ), C takes advantage of the fact that there is a pause in the photos loading to interject. He breathes in and talks (âthere no thereâs...â) just as A also starts talking (A: âbut- thatâs the thing is that...â):
The point here is that C is not just âbrowsing Facebookâ but doing so in a way that is synchronised with the ongoing conversation he seems to be part of. He takes advantage of the slowness of Facebook to jump into the conversation at an opportune moment.
That interweaving takes place in some sense is âobviousâ, but it actually has really significant implications. It suggests that we must rethink where meaning of social media is âlocatedâ when we seek to study its use. Meaning is not necessarily to be found âinâ social media at all, but rather in the interactional process of âgearing inâ the use of social media with the mundane, deeply practical contingencies of everyday life.
In this way our research suggests that the picture gathered from either large-scale aggregations of social media data or post-hoc interviews of users may be missing something. But more about that later.
3. Thereâs a great similarity in the methods people use to interact on social media and those we use in everyday verbal talk.Â
By âmethodsâ we mean the familiar ones we all employ on a daily basis, e.g.: taking turns to speak, repairing one anotherâs utterances, selecting who to speak next, and employing common patterns of adjacent pairs of things (e.g., question/answer, summons/response, greetings, etc.). These normal methods are âtweakedâ to fit the design features of social media. So, for instance, social media users employ the â@â feature to address others, but in ways that are /a bit different/ to how we preface an utterance with someone's name when we select them as the ânext speakerâ. For instance, a lot of â@â use also is used to sort out who is âin playâ and whether they are considered to be a relevant ânext speakerâ at all.
The wider significance of this is that it means a whole gamut of concepts and findings from ethnomethodology and conversation analysis (or âEMCAâ--which has extensively studied naturally occurring talk) can usefully be applied to help make sense of the complex interactions people perform on social media.
Question 1: But isnât there lots of social media research already?
But where does this all fit into existing social media research, of which there is plenty?
When we looked at what research was out there on social media, we found that most of it seems to fit into a couple of two overlapping categories. We called these âactor-focussedâ and âaggregateâ perspectives in our paper:
âActor-focussedâ perspectives are mainly about eliciting usersâ post-hoc accounts of their behaviour on social media. So, interview studies, surveys (e.g., questionnaires) and so on.
âAggregateâ perspectives are probably more popular, and involve scraping data from social media itself. So, Twitter postings, friend network mapping, etc.
Yet productive as these approaches have been, what ends up being absent from them is an examination of how social media concretely features in the mundane âeveryday worldâ of its users; this is the kind of approach we have taken in the points made above.
Our inspiration for this different approach is ethnomethodology and conversation analysis. You can read more about this particular perspective in the paper. Suffice to say here that it relentlessly prioritises close inspection of the details of how human action (e.g, with technology) is organised.
Question 2: Does this kind of research into social media tell you what to design?
The simple answer is no, not directly.
But what we do find is that looking at moment-by-moment use throws into relief some of the design choices that have been made. Take commenting on social media for example. Most systems hide what it is you are typing until you hit âpostâ. This means that there are many âconversationalâ things that cannot be done, such as someone repairing what you are saying as you say it, or enabling someone else to quickly respond to what you are saying as you say it. In other words, online chat becomes less collaborative and slower.
You can read our CSCW paper here.

Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
Free to watch ⢠No registration required ⢠HD streaming
Recording of my talk on âHuman-computer interaction as scienceâ
Thanks to Lone Koefoed Hansen, Iâm able to post a recording of my talk on âHuman-computer interaction as scienceâ that I delivered at the Aarhus Decennial conference (Critical Alternatives 2015).
New publication:Â âHuman-computer interaction as scienceâ
My paper âHuman-computer interaction as scienceâ (PDF) is in the proceedings of Critical Alternatives 2015, which is the 5th Decennial Aarhus Conference.
The paper attempts to unpack the role of âscienceâ in HCI research, both in the discourse / language of HCI and in its practices. In the paper I examine the development of (cognitivist) scientific approaches and the notion of âdesign spacesâ in HCI, tracing this from early work on input devices.Â
The conclusion I reach is to argue that HCI research should largely stop worrying about âscienceâ and also stop worrying about âdisciplinarityâ. The reason for this is that these have proven to be very troublesome concerns which (I feel) draw attention away from the more important work of determining appropriate rigour in HCI, and engaging rigorously with HCIâs inherent interdisciplinarity. (Also see my Interactions article âLocating the âBig Holeâ in HCIâ.)
I feel that this paper is still midway through developing and in several places elides important things, misses some historical subtleties or doesnât offer a clear line of argument---in that sense a work-in-progress. So I welcome any comment, criticism, or suggestion.
I made this ironic depiction (PDF) of how some in HCI research might view âscienceâ, particularly as part of a broader ordering of knowledge. It was initially a reaction, taking the idea of CHI 2014âs âInteraction Scienceâ Spotlight (PDF) quite literally as best I could. This led to some interesting places as I attempted to (temporarily) inhabit that perspective and work out what the world might look like from standpoint of that intellectual placeâspecifically, for instance, considering how one might view âscientific knowledgeâ and the corresponding knowledge productions of a âscience of interactionâ.
Personally Iâm interested in bracketing this discussion. It is part of a broader set of questions in HCI around disciplinarity and often about a scientific disciplinarity of HCI. I have attempted to do this bracketing and address these issues in some other writings:
Locating the âBig Holeâ in HCI Research (ACM Interactions, 2015)
Human-Computer Interaction as Science (Critical Alternatives, 2015)
Is Replication Important for HCI? (RepliCHI Workshop, CHI 2013)
New publication: âLocating the âBig Holeâ in HCI Researchâ in ACM Interactions mag
Based on my earlier blog post, ACM Interactions magazine has published âLocating the âBig Holeâ in HCI Researchâ in its July / August 2015 issue.Â
Iâve made a draft copy of the article available as a PDF.
Four things we found out about live video streaming in public from Blast Theoryâs game âIâD HIDE YOUâ
With the recent interest in live mobile video streaming services like Periscope and Meerkat, I thought Iâd give a brief summary of some relevant findings from our study of Blast Theoryâs game Iâd Hide You. (Full paper here.)
1. Learning how to simultaneously manage your body and the behaviour of the camera you are broadcasting with is critical to producing âgoodâ video (i.e., interesting, compelling, watchable video). We found a bunch of methods that are used to do this that go beyond standard shot composition and framing etc. (More in the paper...)
2. On the street all actions have a 'double duty' to them. So, pointing the camera at something in the street (a person, an object) is both 'showing' this to the online viewer, and also making that thing interesting to those physically around you. In other words, your video broadcast will always have a dual orientation: to the street and to the online audience.
3. There will be tensions between the demands of the environment you are filming in and making the broadcast interesting for online viewers. And the immediacy of things happening in the street environment will fight with the priority you have for online viewers' attentions.
In I'd Hide You, this is something where the artistic director (and creative design) works with the video broadcasters. It's also important for the artistic director, together with a support team that watches the stream of each broadcaster who is out on the street, to monitor this tension during the performance. This is to make sure that video broadcasters do not get 'too involved' with the street.
4. As with every performance, there is a load of 'backstage work' that is deliberately hidden from view. Video broadcasters in public need to maintain 'backstage zones' that are somehow hidden from the online viewer.
These are not necessarily just physical spaces, but also preparatory procedures (e.g., developing a palette of talkable topics, and establishing locations where it is okay to film). Or they might be about developing 'backstage' methods for doing stuff with people outside (but alongside) the live broadcast stream (e.g., checking it's okay for someone to be filmed by you).

Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
Free to watch ⢠No registration required ⢠HD streaming
Locating âThe Big Hole in HCI Researchâ
At CHI 2014 a paper was presented (Liu et al., 2014) which sought to demonstrate, through an analysis of keywords specified in a large tranche of CHI papers (from the last 20 years), that HCI research is lacking in âmotor themesâ. According to the paper, a motor theme is a commonly addressed topic in a given academic discipline that defines the research âmainstreamâ. Motor themes themselves are in turn made from keyword clusters that emerge during a co-word analysis process performed on the collection of keywords, and are found to have particular values of centrality and density. (Note that co-word analysis was initially popularised by Michel Callon and other STS researchers for the study of scientific disciplines, based upon a conceptual backdrop of actor-network theory.) Keyword clusters would be things like âcollaborationâ and âhandheld devicesâ amongst many others (see Liu et al. (2014) for the full list), while centrality and density metrics rate âhow âcentralâ a theme is to the whole fieldâ and âthe internal cohesion of the themeâ (Kostakos, 2015).
I attended the conference talkâdelivered with panache by Vassilis Kostakosâand the paper received a curiously noisy reception; this was unusual for the normally serene audience at CHI, so I felt it must have touched a nerve for people, resonating with existing concerns some researchers were already occupied by in some way. There were audible gasps when Kostakos produced the punchline of the talk: a graph plotting the apparent absence of keyword clusters in the coveted âQuadrant Iâ, i.e., the motor theme zone (see image below). This graph was set in opposition to some published co-word analyses of other research areas (e.g., stem cell research, psychology, etc.), which provided more âhealthyâ plots.
The tenor of the talk and the response it received have to be considered alongside other activities appearing at CHI in recent yearsâfor instance, the "Interaction Science" SIG of 2014Â along with the emergence of yearly events at CHI around âreplicationâ from 2011 onwards. It was this chain of events that made me start to wonder whether a somewhat-dormant cultural undercurrent at CHI (and in HCI more broadly) had been surfaced.
Before I continue I should strongly emphasise that it is extremely important Kostakos and others (e.g., Howes et al. (2014), Wilson et al. (2011), etc.) are discussing the âstate of the fieldâ. We should applaud them for bringing up this difficult conversation when it is far easier to continue with âbusiness as usualâ and to shy away from mulling over what can be contentious arguments (which HCI and CHI in particular seems to prefer to avoidâoften confusing challenges to research with attacks on researchers). While I have very different views on the topic, the emergence of a debate about the very idea of HCI, what it is and its seeming contradictions, seems like a valuable activity for us and probably long overdue. In many ways this reflects some of the preoccupations of early HCI manifest in the exchanges between Carroll and Campbell (1985), Newell and Card (1985), and others.
What follows below is a list of my objections, highlights of what I perceive as confusions, as well as agreements (in a strange way) with Liu et al.âs paper and Kostakosâs corresponding ACM interactions magazine piece, âThe Big Hole in HCI Researchâ . There comes with this an implication that the discussion is also applicable to the broader cultural movement that I felt it represents in HCI.
Low-hanging fruit: On method
The easiest line of attack in academic work is usually âgoing for the methodsâ. Often this is a proxy for other problems that a peer reviewer canât necessarily articulate. Itâs also a strategy reviewers may employ when they have fundamental perspectival differences with the approach taken by the authors of the paper but are perhaps unable to step outside their own perspective for a moment (I would be the first to admit having done this in the past). However, in Liu et al.âs paper, understanding problems with method help us unpack what I see as deeper confusions around disciplinarity and the status of HCI in its relation to âscienceâ (which I shall relentlessly retain in quotation marks within this post, sorry!). So this is where I start: at the low-hanging fruit.
In principle co-word analysis (of academic papers) clearly has value in providing surveys of particular attributes of publication corpora. Yet we should also exercise some caution here: the claims made off the back of the co-word analysis must have attention paid to them, and therefore the basis upon which such claims are being made. First we should remind ourselves of these claims. The subsequent summary of the original CHI paper (Kostakos, 2015) states that the co-word analysis âconsiders the keywords of papers, how keywords appear together on papers, and how these relationships change over timeâ. Doing a co-word analysis, it is argued, therefore âcan map the âknowledgeâ of a scientific field by considering how concepts are linkedâ. To clarify, this is indeed a significant claim: firstly that co-word analysis of paper keywords is an adequate method for âmapping knowledgeâ for a given discipline, and secondly that it can also be used to demonstrate âgapsâ in a knowledge space.
My problem with this methodologically is that there is no obvious sense in which co-word analysis of the keywords of papers can provide an adequate overview of âthe âknowledgeâ of a scientific fieldâ (Kostakos, 2015) because it does not follow that keywords are being deployed by authors in order to provide some set of indices to some known-in-common âmapâ of disciplinary knowledge in the first place. Instead one must subscribe to the idea that authorsâ deployments of keywords are driven by some âhidden orderâ which only becomes visible through the application of the method of co-word analysis. I want to question that claim.
In order to unravel why I might question this, I should explain that I think that some confusions are being made about just what is being âdoneâ in the writing of keywords for academic papers. These confusions touch on more fundamental misunderstandings about language. Firstly we have to consider how keywords are encountered in the situation in which CHI papers are read (i.e., consider the visual practices of paper-reading CHI-formatted research): they are placed on the first page of the paper, they are prominent under the abstract (where we might start reading) and they have their own headed section. (We donât âreadâ keywords, we âlook at themâ; I think there is a distinction.) The point is that their visual organisation sets them up to do particular kinds of work for the reader of the paper only as it is read, i.e., they cannot be easily removed from their situational relevance to the surrounding text and the manner by which we as readers encounter them.
For instance, authors may deploy keywords according to myriad of possible reasons, many of which may or may not pertain to some âhidden orderââi.e., the âknowledge mapâ that is discoverable through co-word analysis. Here are some examples. Keywords may get deployed as terms for ACM Digital Library search optimisation. Keywords may be âsignalsâ for allying oneself to some sub-community of researchers or âsending a messageâ to another. Keywords can be used as discriminators of novelty perhaps via the creation of new terms (and thus claiming prospective research spaces). Keywords can be referents to established corpuses of work, framing judgements on the work through the lens of an an existing tradition in order to tell reviewers that âthis is one-of-those-papers, so judge it on those termsâ. And, of course still possible, keywords may be indices or intellectual coordinates to some agreed-upon âmapâ of HCI knowledge (although one might ask by which textbook those coordinates are even constructed). The point is that the practical purposes of keyword deployment get lost in the co-word analysis because all keywords are treated in the same way.
None of the above necessarily diminishes Kostakosâs notion of âThe Big Hole in HCIâ but it certainly exposes some problematic methods by which the claim is substantiated in the first place. Nevertheless, in order to provide a grounding for such a claim, Liu et al. must also conceptualise HCI as a discipline, which is what I turn to next.
On disciplines and disciplinarity
Both Liu et al.âs CHI paper and Kostakosâs interactions article refer to HCI on a number of occasions as a discipline. The main argument refers to a âlifecycleâ for motor themes and their role in disciplines. Disciplinary architecture here is described by various quadrants (see image below), tracking themes as they are born (âQuadrant III: Emerging or declining themesâ), begin to stabilise (âQuadrant IV: Basic and transversal themesâ), go mainstream (âQuadrant I: Motor themesâ) and then die off (back to Quadrant III) or perhaps decline (âQuadrant II: Developed but isolated themesâ). Themes may never reach Quadrant I or may go straight to Quadrant II, or perhaps get stuck in Quadrant IV or never make it past Quadrant III. But the basic idea is of the lifecycle and notions of a healthy movement of themes across the graph.
This description, of course, assumes HCIâs classification as a discipline and offers remedial advice for establishing its stability (see the discussion on implications for design below). Largely this assertion passes without comment; in fact this reference to HCI as a discipline is necessary in order to make sure it becomes comparable with other disciplinary objects that Liu et al. hold as reference points based on results of other researchersâ co-word analyses. (The comparison disciplines used in Liu et al. are psychology, consumer behaviour, software engineering and stem cell research.) These reference points can then be used to show the absence of âQuadrant Iâ keyword clusters in HCI compared to other disciplines and thus the disciplinary deficiencies of HCI.
Even if we take HCI as a discipline, the corresponding implication of Liu et al. that disciplines are somehow âcomparableâ is itself contentious, I would argue. For instance, it is hard to see how, say, the activities of stem cell researchers have any bearing on the activities of HCI researchers, and it is not clear whether it is reasonable to assume their paper-writing practices, let alone their everyday research work practices, are similar. Or, perhaps, those of psychologists and software engineers. Instead I would suggest that each works with phenomena particular to them, and have methods of reasoning and research practices particular to them. What counts as relevant research questions in one has nothing necessarily to do with what counts in another. Further, it is also unclear with this disciplinary assumption in place why it might be that specialisms like stem cell research should be compared with all of psychologyâa broad church to say the leastâwhy not social psychology or cognitive psychology? Instead, co-word analysis may be just analysing how keywords (of whatever extraction method) get used, and it may be just that HCIâs use of keywords is different rather than deficient.
The very idea that HCI is a discipline at all is also itself certainly contentious. I think Yvonne Rogers is correct when she suggests that HCI is an âinterdisciplineâ. The implication (intended or not by her use of this term, I donât know) is that in being an interdiscipline, HCI should indeed have âThe Big Holeâ Kostakos identifies, because the very nature of an interdiscipline would be an absence of a disciplinary core. If there were some essential disciplinary core to HCI it would struggle in its role as broker between disciplines (as pointed out by Alan Blackwell recently (Blackwell, 2015)). Even the earliest moments of HCI commenced as a meeting place between cognitive psychologists, software engineers and, to some extent, designers. In other words âwe have never have been disciplinaryâ.
At its most basic the notion of a discipline is an attempt at finding a way of ordering knowledge (Weingart, 2010). It is not a ânatural factâ and we cannot treat âthe disciplineâ as transcendent features of a âhidden orderâ. âA disciplineâ is (Iâd argue) an epiphenomenon of the particular community of research practice. And itâs precisely because of this that the arguments made about the application of concepts borrowed from (broad brush) âscienceâ become difficult to handle. Onto which topic I turn next.
Accumulation, replication, generalisation: On âscienceâ and âthe scientificâ in HCI
One of the key assertions in the interactions article is that âa lack of motor themes should be a very worrying prospect for a scientific communityâ. Kostakos suggests that remedies should be pursued â[if] we want to claim that CHI is a scientific conferenceâ. I interpret this to mean that HCI has the potential for a scientific disciplinarity that may be established through the development of motor themes. Accordingly, a set of signature scientific procedures or âscientific qualitiesâ, as Iâll label them, are described by Kostakos so as to achieve this; these are mentioned as 1. accumulation (scienceâs work is that of cumulative progress), 2. replication (scienceâs work gains rigour from replicability), and 3. generalisation (scienceâs cumulative work involves expansivity). Kostakos describes how ânew initiatives have sprung up our field to make it more scientific in the sense of repeating studies, incremental research, and reusable findingsâ, which I take as reference to the replication (Wilson et al., 2011) and âinteraction scienceâ (Howes et al., 2014) agendas I describe above.
Yet making HCI âmore scientificâ is not really a new drive in HCI. HCIâs initial development was oriented strongly by many self-described scientists (going by Liu et al.âs scheme of labelling sciences) from psychology and cognitive science, both of which have often been at pains to demonstrate their scientific credentials through adherence methods presumed to be drawn from the natural sciences. So one could argue that the cultural foundations for HCIâs desire to be âscientificâ have always been present. In addition, attempts to reorder HCI back into accord with the âscientific qualitiesâ outlined by Kostakos have also been suggested before, such as notions from Whittaker et al. (2000) to develop standardised âreference tasksâ in order to establish generalisation, and therefore âscientificâ legitimacy. These attempts have faltered, however.
The problem, I think, is firstly that can be very problematic to engage in deployments of âscienceâ as a concept. Secondly I think it is mistaken at least to imply (or not guard against an implication even if unintended) that these qualities are properties of âscience itselfâ.
On the first point, âscienceâ is a linguistic chimera for HCI because the term is so diversely and nebulously applied, not only in Liu et al. and by Kostakos, but also in discourse within HCI more broadly. It is unhelpful for us because âscienceâ is often used to do very different things that we may well wish to avoid. For instance, this may be in establishing a kind of epistemic and / or moral authority, or an attempt to gain peer esteem for a research community in poor academic standing, or internally as a method for legitimising certain kinds of work and delegitimising othersâ (i.e., categorisation between âscienceâ and ânot scienceâ) in the course of cultural wars. âScienceâ then becomes problematic because such (rhetorical) uses can tend to be deployed in place of adequate assessments of research rigour on its own terms (what âown termsâ might mean is explored below).
This leads to my second point, the idea of the accumulation, replication and generalisation of findings as being intrinsic properties of âscienceâ rather than methodical practices conducted by a community of researchers (see Crabtree et al. (2013) and also Rooksby (2014) on this point). This latter view suggests that the standards of âwhat countsâ as a generalisation, âwhat isâ a relevant process of accumulation (which I take as the establishing within researchersâ discourse of particular motor themes), and âwhat motivatesâ the conduct of replications, should be decided upon as a matter of agreement between researchers. It cannot be determined through adherence to an external and nebulous set of âscientific standardsâ that are adopted from a notion of âscience in generalâ (e.g., what we might call âtextbookâ understandings developed from formal descriptions of the natural sciences)âfor no such thing really exists. Instead, if by âscienceâ we mean âdemonstrating a rigour agreed upon by practitioners of the relevant and particular genre of reasoning the work pertains toâ then I might consider it a useful term. But it seems unlikely this is what is being meant (itâs definitely very unwieldy!).
This all said, I have a great deal of sympathy for the desire of Liu et al., Kostakos, Wilson et al., Howes et al., and others who seek to increase the rigour of the HCI communityâsuch a motivation for the critique can only be encouraged. Yet, to reiterate, this cannot come at the expense of specifying singular-yet-nebulous approaches like making HCI âmore scientificâ, particularly when the model of âmore scientificâ is based on classic tropes of what a mythical âscienceâ is said to be, rather than as a matter of how researchers engage in the various shared practices to establish agreement and disagreement over findings.
Instead, I think if we take the âinterdisciplineâ challenge seriously we should be looking for two things of particular HCI contributions. Firstly, we should expect a rigour commensurate with the researchâs own disciplinary wellsprings, whether this is (cognitive, social, etc.) psychology, anthropology, software engineering or, more recently, the designerly disciplines. Rare examples of such âinternal rigourâ being taken to task is found in the âdamaged merchandiseâ (Gray and Salzman, 1998), âusability evaluation considered harmfulâ (Greenberg and Buxton, 2008) or âethnography considered harmfulâ (Crabtree et al., 2009) debates (although in HCI they feel like more like âscandalsââwhich perhaps says something about the level of debate in HCI more than anything else). What this means is that the adoption of materials, approaches, perspectives, etc. from disciplines âexternalâ to HCI (and remember in this view, there is only âthe externalâ) should not result in lax implementations of such imported concepts, approaches, etc. within the HCI community. The âmagpie-ismâ of HCI research is a double-edged sword: increasing vigour and research creativity, yet often resulting in violence being done to the origins of imported approaches, concepts, etc. And without specialist attention, weak strains are sustained / incubated within HCI; the controversies outlined above are manifestations of this problem. Secondly, we should expect a rigour in the HCI research contributionâs engagement with the notion of being an âinterdisciplineâ. This is what âimplications for designâ is all about (albeit quite a deficient form as pointed out by Kostakos and others); that is, an attempt to meet others at the interface of disciplines. But more on this next.
The curse of the interdiscipline: Implications for design
The interactions article builds upon Liu et al. by arguing that âthe reason our discipline lacks mainstream themes, overarching or competing theories, and accumulated knowledge is the culprit known as implications for designâ. The absence of HCIâs engagement with proper âscientific qualitiesâ like generalisation and accumulation is thus pinned to the perceived need to write âimplications for designâ sections in CHI papers in order to get them past peer review even when the rest of a paper is presenting a high quality of research work. Moreover, within the interdisciplinary community of HCI, it really can never be enough, as Kostakos rightly points out, just to vaguely target ârelevant practitionersâ.
While I have sympathy for this argument, I also think a reassessment has to be made as to why the âimplications for designâ discussion has emerged in the first place, which I have hinted at above. We can use similar questions to those posed over keywords: what is âbeing doneâ in the writing of âimplications for designâ? (Helpfully, Sas et al. (2014) have recently published a categorisation of the different kinds of uses âimplications of designâ is put to.)
I would argue that âimplications for designâ can be read as a gesture towards being an âinterdisciplineâ. They are typically an effort to answer the question âwhy should I (the reader) care about this work?â, a question that is in no way unique to HCI. It would be a mistake to assume that we need not be accountable to the âinterdisciplinary otherâ in HCI. And yes, often the gesture is poorly performed and poorly labelled.
Instead we should perhaps start considering âimplications for HCIâ rather than âimplications for designâ as a better sign of taking work at the interface of disciplines seriously.
Update (22/04/15)
Erik Stolterman has expressed similar concerns about HCIâs âcoreâ, see his blog post.
Jeff Bardzell has responded to this in a blog post, arguing that it may be better to conceptualise HCI as a set of relations (i.e., it has a relational identity) rather than having a core.
References
Blackwell, A. F. (2015). HCI as an inter-discipline. To appear in Proc. CHI 2015 (alt.chi).
Carroll, J. M. and Campbell, R. L. (1986). Softening up Hard Science: reply to Newell and Card. HumanâComputer Interaction, 2(3):227-249, Taylor and Francis, 1986.
Crabtree, A., Rodden, T., Tolmie, P., and Button, G. (2009). Ethnography considered harmful. In Proc. CHI 2009.
Crabtree, A., Tolmie, P. and Rouncefield, M. (2013). âHow many bloody examples do you want?â - fieldwork and generalisation. In Proc ECSCW 2013.
Gray, W. D. and Salzman, M. C. (1998). Damaged merchandise? a review of experiments that compare usability evaluation methods. Hum.-Comput. Interact., 13, 3 (September 1998), 203-261.
Greenberg, S. and Buxton, W. (2008). Usability evaluation considered harmful (some of the time). In Proc. CHI 2008.Â
Howes, A., Cowan, B. R., Payne, S. J., Cairns, P., Janssen, C. P., Cox, A. L., Hornof, A. J., and Pirolli, P. (2014). Interaction Science Spotlight. CHI 2014.
Kostakos, V. (2015). The big hole in HCI research. interactions 22, 2 (February 2015), pp. 48-51.
Liu, Y., Goncalves, J., Ferreira, D., Xiao, B., et al. (2014). CHI 1994â2013: Mapping two decades of intellectual progress through co-word analysis. In Proc. CHI 2014.
Newell, A. and Card, S. K. (1985). The prospects for psychological science in human-computer interaction. Hum.-Comput. Interact. 1, 3 (September 1985), pp. 209-242.Â
Rogers, Y. (2012). HCI Theory: Classical, Modern, and Contemporary. Morgan & Claypool, May 2012.
Rooksby, J. (2014). Can Plans and Situated Actions Be Replicated? In Proc. CSCW 2014.
Sas, C., Whittaker, S., Dow, S., Forlizzi, J., and Zimmerman, J. (2014). Generating implications for design through design research. In Proc. CHI 2014.
Weingart, P. (2010). A short history of knowledge formations. In Thompson, J. Klein and Mitcham, C. (eds.), The Oxford Handbook of Interdisciplinarity. OUP Oxford, pp. 3-14.
Whittaker, S., Terveen, L., and Nardi, B. A. (2000). Letâs stop pushing the envelope and start addressing it: a reference task agenda for HCI. Hum.-Comput. Interact. 15, 2 (September 2000), pp. 75-106.
Wilson, M. L., Mackay, W. E., Chi, E. H., Bernstein, M. S., Russell, D., Thimbleby, H. W. (2011). RepliCHIâCHI should be replicating and validating results more: discuss. CHI Extended Abstracts 2011: pp. 463-466.
Implications of the UX-HCI survey: Some clarifications for HCI academics
Having seen some academic responses to the UX-HCI survey I performed, I thought a number of clarifications might help to unpack the assumptions and purposes of the survey that I had while constructing it, or have realised afterwards.
Doing a survey as a reflective tool
My research practices don't involve doing surveys. Surveys are one of the most abused research instruments; they entail an enormous range of dangers. As such, I'm not really interested in the survey-as-a-survey for the purposes of traditional modes of 'data collection'. For me the purpose of the UX-HCI survey was about making some attempt at an initial engagement with UX professionals (via social media, etc.) in ways that were very low cost.Â
The way the survey was written was naturally oriented towards what I thought their attitudes might be. Of course, the survey is read by some in UX professionals terms of a set of easily-discerned 'academic' attitudes: that it would produce such a response is useful for reflecting upon how I oriented to 'what UX professionals are like'.
The survey necessarily glosses lots of pertinent issues: What is a 'practitioner'? What 'counts' as 'academic'? The survey necessarily sets up a particular and very much assumed relationship between the nature of academic work and the work of professionals. It also pragmatically creates a division between these worlds.
Hence, the work of setting up a survey and conducting it therefore becomes a revealing activity in and of itself: its work is about making things practically visible for us as academics (particularly in deconstructing my own ignorance). This practical 'visibility' is achieved (for me) through doing the 'routine' work of survey construction and administration. By reflecting on this I can then better understand what the various glosses, assumptions and elisons might even be.
At the same time, academics HCI researchers themselves responding to the survey and its provocations also make certain topics about this area visible for me. These might be reflections on how HCI itself has been configured (e.g., with its promises to practice and practitioners), or how HCI researchers themselves conceptualise their work's relationship to the Fellowship's topics (e.g., being challenged by it or defining their work as 'doing something else').
Why assume academia should serve industry?
It is interesting and notable that the very idea of investigating relations between UX (and allied practices) and HCI would suggest to some a position of instrumental reasoning being applied to that relationship. In other words, the idea of the survey can provoke the question: Why assume academia should serve industry?
The answer to this is that this is about taking HCI rhetoric at its word. Personally I'm not really interested in the 'should' of the question; I'm more interested in how HCI has already been configured. At the moment it seems that there is a lot of talk at academic HCI venues like CHI around 'the practitioners' and offering things for them. 'The practitioners' as they are conceived are a bit like 'the users' of times before HCI existed: they are present in our discussions but we don't give weight to considering the relevance of their perspectives. Thus the obvious investigation to do is start filling out that picture of 'the practitioners' with real people and real practices. We might find that actually 'the practitioners' as we mythologise them are not the people HCI is actually interested in serving and uncover very good reasons why not (and therefore what HCI 'should' be doing). Or we might find we are indeed 'serving' them but in ways we didn't understand. And so on...