Understanding Digital Research

Measuring my sleep cycle - now that was interesting. Although I have friends who do that on a regular basis, and I have to admit, I have thought about it myself, I was never too much into the quantified-self, tracking-my-every-move craze. Let alone post about it.

But here I am. Things change, I guess.

The last three nights somebody slept very close to me: my phone.

I tested the app "sleep cycle" to measure the depth, length and quality of my sleep.

It's very easy to download the app, you position your phone next to yourself on top of the mattress, possibly somewhere where you don't roll over during the night.

Though I am not very happy with the amount of data you can export from that app, its still very interesting what the statistics tell you in the morning:

(unfortunately this is in German, sorry, but I translated a few words):

You can see: I was in bed from 00:39 until 07:39 and had a fitful sleep.

Another disadvantage this app has: it only measures the time in bed and not the hours of "real sleep". The sleeping quality of this night on Monday was 63%: you can see the sleep quality when you export your data, because it needs at least data from 5 nights to show the sleep quality directly in the app.

Low sleep quality, not enough hours of sleep and awake at 3 in the morning.

Explanation for that: Assignment dues and living in an apartment with walls out of paper together with a Spanish flatmate who cooks at 3 in the morning in the kitchen next to my room on a regular basis.

What is interesting:

The sleep quality does not seem to be directly related to the length of sleep:

Though I had less sleep from Tuesday to Moday, the quality of this nights sleep is higher (76%).

Last night I had a very deep sleep, which could be related to the fact, that my flatmate didn't come home last night; But she did this morning: Guess when? 6 in the morning - you can see it in the statistics.

I tried to import the data in tableaux, but unfortunately not much was possible to do there with the data I could export through the app.

Maybe there is another way to export more data - would be interesting.

I think the whole quantified self movement has two sides: 

Yes, it is very useful for medical reasons and also on a personal basis to improve the fitness of your body or something like that.

I also read about apps which measure the emotions of your relationship and then tell you what to do or not to do.

In my opinion one has to be careful with that. We are human beings and  normally we should have intuitive feelings what is good for ourselves and what is not.

I decided to use the free app Sleep Time by Azumio for Iphone. I chose this app, because it seemed like I could get the raw data out of the app and send it to myself via email. However, after receiving a zip file full of data, I had no clue what to do with it, let alone know how to clean the data and use it in Tableau.

The data itself is quite detailed, but does not tell me much about the quality of my sleep itself. Here is a screenshot of the zip file containing the data:

I then opened the highlighted data set in Excel (I tried them all), which is displayed below:

As you can see, this data gives me information on what the app itself was doing while I was sleeping. Which is seen in ‘alarm is triggered..’ and ‘startClockupdates’ for example. While this information is pretty interesting, I am not able to use it in Tableau.

The app itself produced some visualizations as well, so I decided to just share those. The first one is the classic visualization of a sleep cycle including both light sleep, deep sleep and times I was awake. And yes.. I went to bed pretty late both times using this app. This week has been stressful!

I really liked this second visualization provided by the app as well, as it looks like something I would have been able to do in Tableau as well:

I do not know if I would use the app a lot after this experiment. I do quite like using self-tracking data like this and don’t really see a problem with it or find it creepy like I have read people say on Reddit. I really liked seeing the visualizations of my sleep cycle and thus being able to see what the quality of my sleep was like. But I do not know how accurate these findings would be coming from a phone lying close to me in my bed. I am sure this has been tested, but I still question the outcome. It will for sure give you a good indication of what your sleep cycle is like though.

A thing I didn’t like was the so called ‘wake-up phase’. This allows you to set a time frame in which you want to be waked up. This means that if you pick a time frame of 20 minutes and you set your alarm at 08:00, you can be woken up ranging from 7:40 to 8:20. In this way the app can wake you during your light sleep, which would ensure you to be less tired when waking up. When I tested it I set my alarm for 06:45 and got waken up at 06:32. Now I have to admit I did notice that I woke up fairly easily, but those fifteen minutes are still too precious for me to let go off. Not a morning person! So if I were going to use the app again and I would have to wake up early, I would get rid of the ‘wake-up phase’.

Like for many others, for me too, Facebook has become the place to keep in touch with friends, but perhaps even more so, to connect with colleagues and stay informed. A previous visualization of my network yielded exactly such results.

As part of my job as a journalist, I've been using the network as one of my main news feeds, thus following a number of national and international newspapers, magazines, and freelance reporters. As a result, my news feed is usually a mix of articles from different publications. It is not often that any of my friends and/or colleagues would start a political discussion themselves, but if they are interested in one, they'd repost a link that would then encourage a conversation. 

In my first observation I could figure out two different things:

Firstly, I liked many Facebook pages of newspapers and magazines, which means I see current articles about political happenings in my newsfeed and hence can read the comments under this article or the content of the article.

Secondly, I have a lot of friends who are politically active or socially committed and also let their Facebook community know about this and their opinion.

  I scrolled down my Facebook newsfeed for the last 17 days and could notice:

Especially 4 of my Facebook friends are posting about the current happenings and ongoing discussions in the world everyday. They share links and respond to the comments under the shared links.

What is interesting is the fact that mostly people were commenting with the same or similar political view. They were discussing what could be improved or shared more information about a topic. Sometimes they had a different kind of view on a particular topic, but on a minimal level.

So no big political debates in my Facebook network?

I asked all 4 of them if they share their opinion openly, which means making their newsfeed accessible to all Facebook users or if it is just accessible to their friends or friends of friends.

As they are all having their private settings on “just visible for friends” it explains the findings above easily.

Most of their Facebook friends probably have the same political attitude or at least not a complete different view of things and therefore there are no or not many comments which contain an "argumentation".

It would be interesting if there are more findings in academic research to this finding, I am sure there are.

Let us go back to the Facebook pages of newspapers and magazines. Of course the comments show different point of views in the comments and of course people have different opinions about things (would be sad if it would not be that way).

  But this is what I observed:

(The following is put in very easy words and I know that I am throwing around with stereotypes, but there is a point)

The way of commenting or arguing with others, depends a lot on the level of quality and credibility of the (online-) newspaper or (online-) magazine.

If it is a “heavy” quality paper, the comments and argumentations between the commentators are of a higher quality than the comments under postings from a tabloid newspaper.

The latter has comments under its postings, which include politically extreme standpoints, swearwords and sometimes not a clear argumentation in their critical comments.

Whereas the quality paper has comments, which are more structured. The commentators are using more arguments, cite other papers to proof their point and tend to stay more objective.

“You are not a product” (Ello Manifesto)

In its manifesto, Ello describes itself as an anti-Facebook social network, which does not sell ads and doesn’t sell data to third parties - hence doesn’t see users as a product.

Herewith Ello comes into play exactly at that moment when the criticism around Facebook and its use of “our” data is getting louder than ever before. For example Facebook will soon, according to their website, launch a new ad program that can read users’ browser history to deliver targeted ads specific to their interests (Kosoff, 2014).

Of all the tasks thus far, visualizing my LIWC dataset in Tableau has been the most challenging one. In a nutshell, it involved editing text files, writing Excel formulas (which should not be a big deal, but for a total Excel non-expert like me, it was somewhat of a battle), creating different sheets and merging those. It was all good until I imported my newly created Excel file into Tableau, and all my data disappeared. After going over the instructions time and time again, I figured out which step (in TextWrangler) I had missed along the way. All things considered, I was eventually able to make my visualizations. 

First, I was curious to see what combinations #digitaljournalism was used in, in a positive context. 

I tried to get a visualization of my data set for more than 3 hours- without success.

I could not combine the datasets in Tableau, though I tried everything what my classmates suggested on reddit.

I removed .txt and saved it.

I entered the formula and "1" appeared. I don't know how exactly it should look like, but it looked correct to me according to Erickas description.

--> 

I imported the dats sets "tweets" and "users"

But when it came to Tableau, it would not work out: 

As there will be guest speakers in class on Friday talking about how digital data is being used in “real business life”, I asked myself how companies are actually using big data or data visualization tools.

Since I have been working in television for two years, I know that for example the viewing-figures/ratings are visualized with tools comparable to Tableau.

But besides that, what could be the use in media related companies? I did a little bit of research and found an article/interview on “How Time Warner Cable Uses Big Data to Optimize the Viewers Experience”.

As you probably know Time Warner Cable (TWC) is an American telecommunications company, and not a small one - they have over 14 million costumers.

Hence Times Warner Cables has access to a lot of data and yes, they use big data tools in order to adjust their infrastructure to the changing wishes of their customers.

The data provides an insight in what their customers are looking for and they can easily create different profiles out of that.

Personalized Advertising

Joan Gillman, President of Time Warner Cable Media, explained that they combine public data sets such as real estate records, demographics or voter registration records with local viewing habits.This allows them to create advertising campaigns that are highly targeted.

TWC also does multi-channel advertising: Big data techniques are being used to measure the engagement of users on each individual platform and can adjust an ad campaign on each platform if necessary.

Detailed metrics lead to detailed information

For cable companies like Time Warner Cable, aggregated user data is very important. With data they can optimize their network and programming.

Normally TWC knows – thanks to big data - how often costumers use what service (OTT services, interactive TV or mobile TV); So it is easier for them to interpret information about peaks in their network demand.

Time Warner Cable uses Alteryx to help them understand how their viewers watch their programming as well as their advertising clients performed.

Thanks to their data analysis in the last years, Time Warner Cable was able to perform cross-platform analysis in order to predict which homes would be interested in what movies via their Movie on Demand platform. Thereby they can increase their sales.

This company understands that data is inevitable in the process and has successfully used to find new revenue streams, improve their marketing strategies as well as their network infrastructure.

   More interesting case studies can be found here: Case studies

I decided to use hashtagEdges because that was one of my focus points in the Tableau visualizations as well. However, I’m not sure if I made the right choice and I will explain why.

This is what my entire hashtag edges data looks like in Gephi..

As you can see it is very spread out. Almost to the point where you can barely see the nodes and edges when trying to get everything in view. This has to do with the fact that there is no one hashtag that is used very often. This is why it might be a better choice to look at UserEdges or rtEdges.

 I zoomed in on a couple of nodes to get a better view. I started with the biggest and most connected option: 

This is the most retweeted tweet and the hashtags are thus most often used. Sadly, the tweet is not associated with any positive or negative emotion. The purple indicates that there is none: two zeros in both PosB and NegB.

But there were of course hashtags that did get linked to positive or negative emotions.

This tweet by Jackhowson96 shows how he uses the hashtags #FlatWhite and #Forever in a positive way. These hashtags were featured in my tableau visualizations as well.

This tweet by hayleypatrick shows negative emotion regarding the hashtags #CostaSwanley, #dairyfree, #glutenfree and #cherrybakewells. Again, this tweet is seen in my Tableau visualization as well.

To get this graph I had to remove the biggest ‘bubble’ out of them all: the tweets containing no hashtags. This one consisted of 47 tweets with 30 of them positive. However, I wanted to look at the hashtags alone, so I decided to remove this bubble. This left me with this visualization:

Since I removed the bubble with records without hashtags, the hashtags that are left only come from 1 tweet. The effect of this is that there is no real distinction between which tweet is more popular than the other. However, I can look at what hashtags are related to positive emotions. Hashtags such as #relaxing, #treat, #lovecoffee are all used in a positive manner. Which makes sense. However, I feel like one hashtag does not belong in this category, and that is the #sortitout hashtag. This is not used in a positive manner at all, but is categorized in this way. I guess this shows the downsides of sentiment analysis where it is hard to take context into account for example. 

I then proceeded to do the same for negative emotions connected to the different hashtags. I once again removed the bubble with records without hashtags, which consisted of 47 tweets with 5 of them classified as negative:

This once again caused less variation in my outcome, but we can see that negative emotions are used way less than positive emotions. Only two tweets are associated with negative emotions. I once again question whether #WimpyKid should be classified as negative.

The next thing I looked at was positive emotions relating the location of the user tweeting Costa Coffee. This can be important, as Costa Coffee might be more popular in certain areas of the United Kingdom than others.

When looking at this graph it is clear that most positive emotions are connected to the locations England and London. England has 17 records and 9 positive tweets, which is quite a lot. However, the fact that the location England or United Kingdom is used, does not give much insight in which areas of the UK Costa Coffee is perceived more positively.

I did the same thing for negative emotions:

This shows that again, there are not many negative emotions. Only both England and London had 1 tweet associated with negative emotions.

Another simple bar graph shows this distribution of negative and positive emotions. Out of 79 records, 44 are associated with positive emotions and only 7 with negative emotions:

Since both my personal and professional interests have a lot to do with new technologies, journalism and innovation, I decided to work more closely with the Twitter dataset for #digitaljournalism. In particular, I went on to visualize the relationships between #digitaljournalism" and other search terms.

This post was originally published here. 

I’ve had a Twitter account since November 2009 but until last year, that was all it was. An account. Two tweets in total, six proud followers (who were kindly forced to follow me), succeeded by plenty of inactivity. Yet, last year, I made myself a promise to change that. In fact I documented it in a blog post and swore to “power through the #- and @-madness and to join the never-ending conversation.” (You can read the rest of that oath here).

Now let me be very honest here: I am not the most avid Twitter user, and one look at my account would most likely already reveal that. I’ve posted all in all 278 tweets, I’m following 253 people and organizations, and in return, I’m being followed by a mere 162. I know. I’m a journalist, I need to get on top of my tweeting game. I will.

This week’s assignment asked us to do an experiment on one of your social network sites. Since I already had the Costa Coffee theme for my twitter data set, I decided to just stick with this coffee theme! For two days straight I liked everything coffee related I saw on my Facebook feed.

First of all I made sure to like the Facebook pages of two coffee places I myself quite like: Starbucks and Costa Coffee.

After liking the pages, I immediately saw older posts and recent posts by these companies appearing on my newsfeed:

Lisa is living in Munich and is 25 years old. Besides her studies she is working in the media agency Textbau as a media consultant/ journalist.

Lisa, what were the reasons that made you „quit“ Facebook? 

Realizing that most of the content people share on Facebook is not necessarily content they themselves like, but content they think others will like (or that will "generate" likes via the like-button, i.e. the social beer game of whatever it was called), which makes Facebook a bit like high school, with all its peer pressure groups. I was starting to feel too old for that, which was one of the reasons I quit.

It's no accident that "to quit smoking" and "to quit facebook" reveal similar structures of addiction. Whether you like it of not, you become addicted to the neverending stream of information Facebook provides. Once I realized that, I decided it was time to quit.

Or take friendships as an example: it was becoming increasingly difficult for me to figure out who the important people were in real life. I wanted to see if the people that were very actively engaging with me online would still be in my life once I went offline.

What did you miss during your Facebook-abstinence and did you have positive experiences?

I didn't miss anything. And that was probably the most positive experience: as long as you're still on facebook, you imagine you will miss out on so many things, but you really don't. Cutting off the constant stream of information not only improved my ability to concentrate a lot, it also forced me to only stay in touch with people that really matter, and to spend more quality time with them.

What happened that you decided to „come back“?

I guess everyone who has signed up to a social network is acting and reacting with it and automatically has a social networking behaviour. But sometimes I think I am REALLY backward regarding social networks.

I am using Facebook.

And checking my mails with Googlemail.

And...that was it. Well, reddit and tumblr came into my life 3 weeks ago.

I am not using Instagram, Twitter or something else. I actually can’t name the reason for that. Maybe I just don’t want to be available on different platforms because this automatically means that you spend more time on that and I am not willing to give more time. 

The second reason is, that with the social network services I am using, I actually have everything I need. At least I thought that, until I read Nienke's post here - now I have to think this over.

I am not highly active on Facebook: I post photos and status updates once or twice a month, comment on photos approximately two to three times a week and like posts, photos or comments approximately every second day.

The first time I actually realized that Facebook seems to use algorithms to invade my newsfeed was when I changed my relationship status. All of a sudden I got invitations to wedding fairs in Munich, bridal shops (not even engaged btw!) or special offers on engagement rings.

As a sacrifice for DRM I misused my Facebook account to like almost every post or photo my active Facebook friends were posting. I did not inform them about this experiment before intentionally.

I could classify two reaction patterns:

Out of the reason that I am normally not liking or posting that much on Facebook, my Facebook friends got very confused. I got a lot of messages asking if I am ok or informing me that somebody probably hacked my facebook profile and that I should take care of that.

Facebook friends whose posts or photos I liked a lot, started to like back. They took old posts or pictures on my wall and started to give likes on them.

This shows:

1)  that everybody has a specific behavioural pattern on Facebook and if you break out of this, some of your Facebook friends may notice.

2) that likes can be a form of „payment“ on Facebook. I give you a like, you give me a like, you just have to look at #likeforlike on instagram for example. Not for nothing there are dubious companies selling likes for companies' Facebook sides or content of private profiles.

Quitting Facebook. Is it even possible? I could not do it myself (it is the only social network I am using- at least I will keep that).

The set of data I used for this visualization consists of 100 tweets with the search term #greenpeace.

The first visualization shows the time zones where #greenpeace was used the most. There are many tweets that are marked with none, which means that these posts are not connected with information about the particular location.

As you can see in the visualization the most tweets are posted in the area of US & Canada (Pacific Time and Eastern Time), but also from Brisbane, London and Tokyo.  

The following visualization shows which apps or service people tweeting about #greenpeace are using.

Twitter for Android and Twitter Web client are the most popular.

The last visualization shows the hashtags most used in the combination with #greenpeace.

For example:

#Japan #SaveTheWhales #Sharks #bees #birds