#gov20

This is a great example of how we may be able to use passive smartphone data to help people be aware of challenges they may be experiencing when they may not be paying conscious attention to them. Or, such data could be sent to a personal assistant or caseworker with the individual’s permission. 

Oakland has a new School Superintendent, I like him, partly because of the following statement he dropped at a meeting of the Youth Ventures Joint Powers Authority recently- all the city and county heavy-hitters were there, discussing the possibility of hiring an out-of-state firm to do a data report on Oakland. There was much debate about the need to do this, the need for non-local data folks, the quality of local data, but Anton Wilson wonderfully cut through that: “Since I’ve come to Oakland I’ve seen a huge stream of data come across my desk. We don’t need more data, we’re not data poor- we’re analysis poor.”

I could have high five him for that statement. But that would have been awkward from across the room.

His point is one that I’ve harped on about for some time in Oaktown. We have troves of data, but barely a person doing thoughtful analysis of it to inform decision making, policy, evaluation (with the exception of some bigger programs that do get evaluated heavily).  A similar incident highlights this even more starkly. In front of the County board of supervisors, a department chief was utterly stumped when asked a seemingly simple, core metric about their department, after an injection of $75M dollars in the past couple of years for a new program.  I know that agency has tons of data, but I’m also aware of failed efforts to replace huge parts of their technology base and a stagnating effort to build a data team, so while I share some of the pain, ultimately it’s up to all senior leaders to take seriously and invest in people and systems to help make modern government agencies data smart if not fully data driven.

Part of our current problem is that the understanding of technology and data is very poor at the executive level and this often results in unwise mashing of technology and data folks with little thought to those being the right people with the right skills to actually understand your operations. I’ve talked often of the need to integrate data analysts and researchers into regular agency strategy and planning to they can respond as needs arise, but this is also a higher level problem- started when those responsible for departments do not themselves have enough data savvy or technology awareness to make good initial decisions.

If you’re one of the data geeks or tech folks in government, a good way for you to both increase your value and to help grow your organization is to add a layer of analysis or context when asked for simple data products. Instead of just giving the numbers of what you’re asked about, give some context to how that has changed, ways that measuring that thing have changed, gaps in your data that make that data fuzzy, or even better, ask those annoying questions like “What is this being used for? What decisions are you trying to make? Can I help you when it comes to digesting this information at a planning meeting?” You’ll be stunned at the number of exec level meetings with people saying ‘I don’t really know what these data mean” or “I wish we knew some context around these data”, but never bother to pass those issues down to you. Suggest you can both produce better products and also help with analysis if you are part of the process.

My former colleague Mark Headd, the first Chief Data Officer of Philadelphia, tweeted out Monday that it's a good time to check out the flu shot app developed by Chicago, made openly reusable for any city. That got me thinking about how we might be able to repurpose it for Los Angeles -- mostly as an experiment in how easily we could take local data and make a useful app.

The timing made sense as well, as we are entering flu season, and on top of that , the Los Angeles County has a widespread and helpful program to enable Angelenos to get a flu shot for free. 

It took only a few hours (maybe 2), thanks to the data being available and the code being out there. Here's what we prototyped, datala.github.io/flushots:

The point was really to illustrate how to build a simple but useful app atop government data. So then the question is: how did we do it?

Step 1: Get the data

The County publishes the locations of its free immunization program locations on its website as a pdf. To turn that pdf into more readable data, we used a simple, free tool called PDFTables from ScraperWiki.  All you have to do is upload your PDF, and it'll dynamically and quickly transform it into a excel spreadsheet you can work with. 

Once you've got that data, you're well on your way to deploying the application, because all you need to do is format the data according to the data standard and upload it to Google Fusion Tables. Put simply, just rearrange some of the columns in excel and upload it -- you're set. Takes about 5 minutes.

Step 2: Stand up the app

The Flu Shot App was initially developed in Chicago by Tom Kompare. Since then, not only have they made multiple iterations to make the app better, many other cities have redeployed it, such as Philadelphia. Hence the opportunity for a fully reusable tool and a common data standard. Since the code (the technology) behind the flu shot app is open source and freely available on github, we were able to grab it and customize it for LA. Additionally, since the software is mostly just javascript and html (and a bit of php) we can host it on github pages at no cost. (All you have to do is make a branch called gh-pages.)

Step 3: Make it LA

Just a little bit of customization is needed to customize for your city. In the main.js file you just need to plug in your Google Fusion Tables link and change to your city as the starting point. (Takes about 30 seconds.) After that, we opted to make some style changes with "LA blue" ;) and then added in some additional information for context about the app and why we built it. (Took about 10 minutes.)

That's all; that's how we were able to build out datala.github.io/flushots in about 2 hours (or less, really).

***

This application is built as a prototype to show what's possible with government data. Simple, fast, and easy. If you have questions about the flu shot program itself, please contact the Department of Public Health at the County, and note that this is not an official app from the County or the City -- just a prototype.

OpenGovData_White by nik_g on Flickr.

Not everything can be Big Data. Not everything should be either. But some data do need a kick in the pants, so to speak. Are the data you produce or use real time, coming down the pipe as a feed everyday, or are you stuck with years old data for your planning and analysis purposes? If you are in the latter, don’t feel bad — you’re not alone.

For those tracking Ebola outbreaks in West Africa, the stream of data is steady but not real time, yet decisions that impact people’s lives are being made every day about resourcing and responding to this crisis.  In the USA there are similarly important data needed — many infections or diseases are notifiable — requiring direct notification of the Centers for Disease Control and Prevention. However regular hospital visits, treatments and surgeries go through a very big, very slow pipeline from local clinics and hospitals up to the state agency level and after processing, refining and some magical treatment, these data flow back to local public health and research agencies some years later. Traditionally this timeline was “all we could do” because of technology limitations and other reasons, but as we rely more and more on access to near real-time data for so many decisions, health data often stands out as a slouch in the race for data driven decisions.

In a different vein, campaign finance data for political donations is sometimes surprisingly fast. In California all donations to campaigns require the filing of a Form 460 declaring who gave the funds, their employer and zipcode. Campaigns are supposed to file these promptly, but this does not always happen until certain filing deadlines. Nevertheless, these data contain valuable insights for voters and for campaigns alike. These data get submitted as a flow, but they then end up in a complex format not accessible to average people — until someone changes that. A volunteer team at OpenOakland created a very powerful automation process that takes these data and reformats them in a way that makes them accessible and understandable to everyone at http://opendisclosure.io. Yet even this system of automated data processing and visualization suffers from a lack of perfectly updated data on a daily basis- the numbers shown each day only reflect the data filed to date, so big donations or changes in patterns do not show up until those are filed — often at a somewhat arbitrary deadline.

Unfortunately not all data are filed frequently and do not come with an easy to use API connection to allow developers and researchers to connect to them directly. Take crime data. Very important information with a high demand for all sorts of decisions at local levels. Your police force may publish good crime data each day or maybe just each month which is useful for real estate people and maybe good for analysts and crime investigations, but how do we know if our local efforts have successfully impacted crime? We go to national data. The Federal Bureau of Investigations (FBI) collects data from most law enforcement agencies in the country and publishes it at as the Uniform Crime Reports (UCR). Unfortunately, these data are published years after the fact. There is a convoluted process for local agencies to format and filter their reports, but then these data take years to get published.

We recently created a violent crime fact sheet using the latest (and recently published) available UCR data — for 2012. This lag in data means that county supervisors and other officials are trying to evaluate the impact of crime prevention efforts but can’t even compare their outcomes with other cities due to the lag in this data – we have to wait for two more years to see if these data indicators  changed in other comparable cities, or if our interventions did have a measurable impact.  This sort of time lag means that no local officials have good comparable data in a reasonable time frame- a poor system for modern policy makers to rely on. The FBI is working to slowly implement a newer system, but it is not clear that the lag will improve.

Every agency responsible for collecting data for operational purposes MUST start thinking about how it can make these data safely available to decision makers and to the public on an expedited process.  The technology is now very accessible to support this, and if necessary we should be considering bifurcated approaches — the old, slow feed to state and federal agencies and a new, agile feed for local use. Privacy standards and quality are simply things that guide how we can do this, they are not actual barriers unless we choose to let them be.

Government is a business, albeit one with a monopoly on services it provides — and it’s not cool for government to be making decisions using years old data when the private sector is increasingly data driven and real time. We can do this!