#gdelt

Last year the GDELT project published a paper on how crunching through the big data of history can help us spot patterns and work out where the world is heading next. The GDELT database monitors news (broadcast, print, web) from across the world in over 100 languages and uses complex computer algorithms to codify what’s happening throughout the globe. Certainly our ability to project the future will improve, as we apply more sophisticated ArtificialIntelligence to draw such predictions about the future

Earlier this week however, Vint Cerf, one of pioneers of Internet, in his recent conversation at the American Association of Advancement of Science expressed a different concern.  He said that the humanity could be headed towards a digital black hole as the digital objects are becoming unreadable as the technology evolves.  He said that we be putting misplaced confidence in the longevity of digital information.  The information GDELT and similar such efforts are trying to preserve is tip of the iceberg. The volume of data that is getting generated continues to grow exponentially, much of which (currently) of no obvious value. Much of this data is being stored with little thought on how it will be retrieved by historians of the future generations. We are at the risk of losing a lot of this information, compromising the ability of future historians to look backwards.

A few years back, I was chatting with a friend and client in the Aerospace industry responsible for archiving engineering data for the airplanes they designed. I was intrigued to learn that the company kept all their designs on printed paper, and not in digital format. My friend told me that that the ever-evolving digital world meant there was no safe way to store this information in digitized manner.

Data Mining Reveals How News Coverage Varies Around the World

Last year, the news media reported on 195,000 disasters around the world. The ones you heard about depend crucially on your location.

One interesting question about the nature of news is how well it reflects the pattern of real events around the world. It’s natural to assume that people living in a certain part of the world are more likely to read, see and hear about news from their own region. But what of the international news they get—how does that compare to the international news that people in other parts of the world receive?

Today, we get an answer to these questions thanks to the work Haewoon Kwak and Jisun An at the Qatar Computing Research Institute in Qatar. These guys have analyzed the news agendas in different parts of the world to see how the coverage reflects actual events in other parts of the world. And to visualize the different news agendas, they’ve created cartograms to reflect the coverage. These are maps in which the land area of a country is distorted by the amount of news coverage it receives in a given region (the image above shows how international news is viewed in North America).

Kwak and An begin with a database of 195,000 disasters that occurred between April 2013 and July 2014 and which were reported by more than 10,000 news outlets around the world. They noted the country in which each news outlet was based and then counted the published stories from other parts of the world. Finally, for various regions, they created a map of the world showing where the news was from.

The maps make for interesting viewing. They clearly show how the news agenda differs across the planet. Unsurprisingly, people in south Asia consume far more news about disasters in that region than people in North America, for example. And people in Latin America consume far more news from Argentina than Europe.

More interesting are the anomalies. For example, people everywhere consumed relatively large amounts of news from Egypt and Syria, mainly about the unrest in these countries and the accompanying humanitarian crises.

Kwak and An go on to investigate the factors that determine why people in one part of the world view disaster news from another. They found, for example, that population size is significant. People in all regions are more likely to see disaster news from other large countries, probably because there are more likely to be immigrants from those large countries who provide demand for that kind of coverage.

But by far the biggest factor that determines news coverage is whether an international news agency, such as Reuters, or Associated Press, covers the disaster. That’s unsurprising given that most news outlets have subscriptions to one or more agencies and are therefore able to use their stories easily. This is the primary mechanism behind the way news stories sometime snowball around the world.

Interesting work that reveals the way patterns of news coverage change around the globe.

Ref: http://arxiv.org/abs/1410.3710: Understanding News Geography and Major Determinants of Global News Coverage of Disasters

via http://www.technologyreview.com/view/532036/data-mining-reveals-how-news-coverage-varies-around-the-world/

Abstract: In this work, we reveal the structure of global news coverage of disasters and its determinants by using a large-scale news coverage dataset collected by the GDELT (Global Data on Events, Location, and Tone) project that monitors news media in over 100 languages from the whole world. Significant variables in our hierarchical (mixed-effect) regression model, such as the number of population, the political stability, the damage, and more, are well aligned with a series of previous research. Yet, strong regionalism we found in news geography highlights the necessity of the comprehensive dataset for the study of global news coverage.

Winner of the Google GDelt Data Challenge and awarded the Google Service Prize. The project will be posted on Google's official blog in the coming weeks.

Winner of the nameCheap best domain name. 

Winner of Nod Ring

Summary:

Bordering.me is a 3D data visualization of worldwide marterial conflict and material cooperation between cities and how connected those relationships are to you. Material conflict includes things like the US refusing the continue to import natural gas from Canada through the Keystone XL pipeline while material cooperation might be apple opening up a new warehouse with Foxconn in China. The visualization is set up with a holographic projection while the user interacts with the globe using a gesture sensing ring. From the start the user inputs their location to show all the cities with which their area has had direct material cooperation with in the past month. With a swipe up they can then see cities with 1 degree of separation. With another swipe they see that after 2 degrees of separation, the material negotiations relevant to their location are connected to part of the entire planet. 

Story:

Last weekend I was flown out to Boston to participate in HackMIT. I didn't have a team when I got there so on the morning of, I hunted down a couple of genius freshmen, Ian Macalinao and Akhilesh Yarabarla, saying they were looking into doing 3D data visualization. At the last second we picked up a senior math major named Feyman Liang.

We rigged up this holographic display with a spare monitor, some sheets of plastic, and a hand-full of Bloomberg stickers. 

at first we thought we would mess around with some fit bit health data.

However, after listening to a talk by the Senior Engineering Lead from Google Ideas we decided we wanted to use data that could actually change people's world view. 

We ended up using Google bigQuery on the GDelt database to pull a list of all events of material cooperation and material conflict between cities in the past month. Material cooperation might be Apple(Cupertino) opening a new factory in Beijing. Material conflict might be DC blocking flights from Nigeria due to the threat of Ebola. 

Each of these events what associated with a location and an edge width(<1 being a conflict | >1 being a cooperation). We mapped this data onto a 3D globe in order by degrees of separation from the user interacting with the map. 

To further experiment with the idea of immersive 3d experience with data we mapped the interactions to nod ring gestures. Here is a badly filmed demo: