#socrata

Overview

Our nation needs new approaches to pressing social concerns and this post is about using Open Data in solving such problems. Here I will highlight some of the work being done in this area across public sector and civil society (non-profits, research & academic institutions, start-ups, private entities etc.). A specific example of such approach, to eradicate hunger in the United States by improving efficacy of the US Department of Agriculture's "Supplemental Nutritional Assistance Program", is presented in the end.

The Open Data approach to social good

Government agencies have long focused on making "evidence-based decisions" to administer programs that solve society’s most pressing problems. Typically, such decisions and the underlying hypotheses are based on tradition, instinct, or politics that in turn derived from data merely from public surveys or operational silos. Rather, government agencies need to also tap into datasets across key social concerns (and attributes) such as demographics, income, employment, education, safety, security, health, justice etc., to get a more complete picture that in turn can produce better outcomes. Most of these datasets go outside the purview of any single government agency and effective solutions will need to integrate new datasets shared by governments, businesses, and other organizations across the country or around the world.

For data to provide the impact and changes necessary to benefit society, the government must open this data (obviously, in a secure way that protects privacy) so that it can be leveraged by public-private partnerships to innovate and build new solutions across the Open Data Value Chain. This value chain must have three defining characteristics to deliver benefits to society at large -

Unleash Open Data from across multiple Government Agencies that is cleansed, of high quality and easy to use. Good news is that the Federal Government has been taking quite a few steps in this area such as opening up data sets at Data.gov and the White House publishing its first Open Data Policy of making open and machine readable formats as the new default for government information. Further, The Digital Accountability and Transparency Act of 2014 (DATA Act) is the nation's first Open Data law that directs the Federal Government to transform all spending information into open data. To know more about DATA Act you can read my earlier blog on Driving Federal Performance

Promote Community Based Approach to generating actionable insights from Open Data to improve outcomes. Unleashing Open Data provides a foundation upon which actionable insights can be generated and applied towards intended social outcomes. However, no single entity including any Federal Agency, has the expertise, resources, and tools to mine Open Data exclusively for comprehensive insights. As the old adage goes, "it takes a village to raise a child", so does the approach to mining insights from Open Data. For example, Public-Private Partnerships across government/non-profits/NGOs/start-ups are required to bring multiple perspectives in solving problems collaboratively. Crowdsourced Ecosystems such as the Data Science Bowl that convene Data Science & Analytics communities, solve challenges that no single entity, market, or industry can manage alone. Finally, collaborative technology tool sets such as Sailfish further enable data scientists to curate data, collaborate and analyze such data and discover answers to the most challenging problems.

Drive Citizen Centric Digital Experiences from Open Data to serve beneficiaries.  These insights generated from Open Data will be more valuable when delivered to Citizens in a timely / cost effective manner over the web and mobile devices, all within their individual social contexts. You can find an use case of how a Chat Bot could build a healthy nation by bringing more people to our national parks.

In essence, to elevate and accelerate the Federal Government's mission, agencies need to proactively open data sets, facilitate public-private partnerships in leveraging data analytics to generate insights that shape outcomes and deliver intuitive digital experiences to drive those intended outcomes.

Open Data Value Chain - A Case Study to Eradicate Hunger

One such example of how the USDA, US Census Bureau, Booz Allen Hamilton, and mRelief have approached the problem to address "hunger in the united states", is shared below. 

Hunger is a serious issue in the United States -

87% of Government's Supplemental Nutritional Assistance Program (SNAP) recipients live in households with children, seniors or people having disabilities

1 in 4 children, 1 in 5 adults and 900K+ veterans receive SNAP assistance yearly

Roughly 1 in 2 people between the ages of 18 and 65 will receive SNAP assistance at some point of time in their lives and 1 in 4 households with children do not have enough money to buy food

With such staggering numbers how can the American Dream become a reality for the most vulnerable sections of our society? The USDA, through its SNAP program, brings support to such low-income families for their basic nutritional needs. However, only 75% of eligible citizens participate in the program. So how can we find and fight hunger?

Open Data can eradicate hunger

Various entities across the Public sector, non-profits, and private entities are coming together to derive value out of Open Data to address nation's hunger.

Government agencies such as USDA and US Census Bureau are publishing data sets such as SNAP Data (on attributes of SNAP beneficiaries) and American Community Survey (on income, demographics and other social factors of communities) as Open Data, making it easy for the civil society to use. While in the past most analysis relied on data within an agency's purview, it is now possible for a new data model to easily link these disparate data sets to generate new insights to address efficacy of the SNAP program, previously impossible.

Private entities such as Booz Allen Hamilton, have been taking a community oriented approach to solving problems pertaining to hunger by leveraging (i) their own in-house pool of data scientists (and Summer Games program), (ii) extending with experience gained for working with crowd sourced communities such as Kaggle and partners such as  Child Nutrition Access and Accountability through Technology (CNAAT); and (iii) using collaborative tools such as Sailfish, to come up with specific recommendations for finding and fighting hunger. Using advanced analytic techniques such as clustering analysis and machine learning techniques such as random forests Booz Allen was able to understand driving factors for variations in SNAP participation rates across different states within a cluster and across clusters, to come up with specific targeted recommendations.

Non-profit partner, mRelief, leveraged Open Data to build web and mobile applications that bring SNAP eligibility information to citizens through an easy to use digital experience.

So as a government agency are you ready to solve society's next big problem? It does not matter where you are within the maturity spectrum of Open Data Value Chain. If you are looking to transform the way your agency deals with Open Data, you can email me at "paleru_rk at bah dot com".

For more information on Open Data, you can visit the The Center for Open Data Enterprise. USDA, US Census Bureau, Booz Allen Hamilton and mRelief have been invited to showcase these solutions at the first White House Open Data Innovation Summit. If you are planning to attend, please stop by to meet with us at Room 146C. 

Socrata Financial Transparency Suite

Socrata’s suite of financial transparency apps combine a familiar, intuitive interface with powerful search and multi-faceted visualization capabilities. Built on the Socrata platform and delivered as