Business & Data Analytics

Overview

Our nation needs new approaches to pressing social concerns and this post is about using Open Data in solving such problems. Here I will highlight some of the work being done in this area across public sector and civil society (non-profits, research & academic institutions, start-ups, private entities etc.). A specific example of such approach, to eradicate hunger in the United States by improving efficacy of the US Department of Agriculture's "Supplemental Nutritional Assistance Program", is presented in the end.

The Open Data approach to social good

Government agencies have long focused on making "evidence-based decisions" to administer programs that solve society’s most pressing problems. Typically, such decisions and the underlying hypotheses are based on tradition, instinct, or politics that in turn derived from data merely from public surveys or operational silos. Rather, government agencies need to also tap into datasets across key social concerns (and attributes) such as demographics, income, employment, education, safety, security, health, justice etc., to get a more complete picture that in turn can produce better outcomes. Most of these datasets go outside the purview of any single government agency and effective solutions will need to integrate new datasets shared by governments, businesses, and other organizations across the country or around the world.

For data to provide the impact and changes necessary to benefit society, the government must open this data (obviously, in a secure way that protects privacy) so that it can be leveraged by public-private partnerships to innovate and build new solutions across the Open Data Value Chain. This value chain must have three defining characteristics to deliver benefits to society at large -

Unleash Open Data from across multiple Government Agencies that is cleansed, of high quality and easy to use. Good news is that the Federal Government has been taking quite a few steps in this area such as opening up data sets at Data.gov and the White House publishing its first Open Data Policy of making open and machine readable formats as the new default for government information. Further, The Digital Accountability and Transparency Act of 2014 (DATA Act) is the nation's first Open Data law that directs the Federal Government to transform all spending information into open data. To know more about DATA Act you can read my earlier blog on Driving Federal Performance

Promote Community Based Approach to generating actionable insights from Open Data to improve outcomes. Unleashing Open Data provides a foundation upon which actionable insights can be generated and applied towards intended social outcomes. However, no single entity including any Federal Agency, has the expertise, resources, and tools to mine Open Data exclusively for comprehensive insights. As the old adage goes, "it takes a village to raise a child", so does the approach to mining insights from Open Data. For example, Public-Private Partnerships across government/non-profits/NGOs/start-ups are required to bring multiple perspectives in solving problems collaboratively. Crowdsourced Ecosystems such as the Data Science Bowl that convene Data Science & Analytics communities, solve challenges that no single entity, market, or industry can manage alone. Finally, collaborative technology tool sets such as Sailfish further enable data scientists to curate data, collaborate and analyze such data and discover answers to the most challenging problems.

Drive Citizen Centric Digital Experiences from Open Data to serve beneficiaries.  These insights generated from Open Data will be more valuable when delivered to Citizens in a timely / cost effective manner over the web and mobile devices, all within their individual social contexts. You can find an use case of how a Chat Bot could build a healthy nation by bringing more people to our national parks.

In essence, to elevate and accelerate the Federal Government's mission, agencies need to proactively open data sets, facilitate public-private partnerships in leveraging data analytics to generate insights that shape outcomes and deliver intuitive digital experiences to drive those intended outcomes.

Open Data Value Chain - A Case Study to Eradicate Hunger

One such example of how the USDA, US Census Bureau, Booz Allen Hamilton, and mRelief have approached the problem to address "hunger in the united states", is shared below. 

Hunger is a serious issue in the United States -

87% of Government's Supplemental Nutritional Assistance Program (SNAP) recipients live in households with children, seniors or people having disabilities

1 in 4 children, 1 in 5 adults and 900K+ veterans receive SNAP assistance yearly

Roughly 1 in 2 people between the ages of 18 and 65 will receive SNAP assistance at some point of time in their lives and 1 in 4 households with children do not have enough money to buy food

With such staggering numbers how can the American Dream become a reality for the most vulnerable sections of our society? The USDA, through its SNAP program, brings support to such low-income families for their basic nutritional needs. However, only 75% of eligible citizens participate in the program. So how can we find and fight hunger?

Open Data can eradicate hunger

Various entities across the Public sector, non-profits, and private entities are coming together to derive value out of Open Data to address nation's hunger.

Government agencies such as USDA and US Census Bureau are publishing data sets such as SNAP Data (on attributes of SNAP beneficiaries) and American Community Survey (on income, demographics and other social factors of communities) as Open Data, making it easy for the civil society to use. While in the past most analysis relied on data within an agency's purview, it is now possible for a new data model to easily link these disparate data sets to generate new insights to address efficacy of the SNAP program, previously impossible.

Private entities such as Booz Allen Hamilton, have been taking a community oriented approach to solving problems pertaining to hunger by leveraging (i) their own in-house pool of data scientists (and Summer Games program), (ii) extending with experience gained for working with crowd sourced communities such as Kaggle and partners such as  Child Nutrition Access and Accountability through Technology (CNAAT); and (iii) using collaborative tools such as Sailfish, to come up with specific recommendations for finding and fighting hunger. Using advanced analytic techniques such as clustering analysis and machine learning techniques such as random forests Booz Allen was able to understand driving factors for variations in SNAP participation rates across different states within a cluster and across clusters, to come up with specific targeted recommendations.

Non-profit partner, mRelief, leveraged Open Data to build web and mobile applications that bring SNAP eligibility information to citizens through an easy to use digital experience.

So as a government agency are you ready to solve society's next big problem? It does not matter where you are within the maturity spectrum of Open Data Value Chain. If you are looking to transform the way your agency deals with Open Data, you can email me at "paleru_rk at bah dot com".

For more information on Open Data, you can visit the The Center for Open Data Enterprise. USDA, US Census Bureau, Booz Allen Hamilton and mRelief have been invited to showcase these solutions at the first White House Open Data Innovation Summit. If you are planning to attend, please stop by to meet with us at Room 146C. 

Law mandates public companies to report their financial and operating results every quarter (10Q) and year (10K). Such reports are analyzed to understand how these companies are performing and capital markets allocate funds accordingly. Since information is eventually made available to investors, public companies proactively manage their internal capital allocation to projects that drive most effective financial and operational performance.

However, as a taxpayer, the average citizen does not have similar information on how the Federal Government is performing and how efficiently his or her tax dollars are being used. Should the Citizen not have access to such data so that there is more democratic participation in our Federal spending? Should Federal Agencies not have such information readily available to proactively manage their own performance? If you want to know more about why this is an important topic for government and civil society, please read one of my earlier blogs titled "Why the government shutdown is actually good?".  

The fact that Federal Government actually fulfills a mission, versus merely managing towards a commercial objective, only makes it that much more difficult to assess, quantify and determine performance. For example, as a tax payer, it is not easy to understand how a specific Federal Grant towards research in science and technology from the National Science Foundation is yielding commercial value to society, or how public health through the Supplemental Nutritional Assistance Program prevents a strain on our healthcare system down the line, or how a particular Federal Loans program through the Small Business Administration is generating economic output and jobs.

Like many corporations that are dealing with the explosion in "volume", "velocity", "variety" and (most recently) "veracity" of data; most Federal government agencies are struggling to get their hands around what is most likely to happen in the context of the public services they deliver and what to do about it? Imagine if the following is possible -

US Citizenship and Immigration Services is able to forecast influx of refugees and its effect on public services such as primary education and healthcare;

US Department of Treasury is able to simulate its "debt to GDP" ratio by predicting economic output and funding needs using drivers such as population growth, long term inflation, tax receipts, real interest rates, employment, fiscal spending etc.;

Center for Medicare and Medicaid Services within the Department of Health & Human Services is able to improve healthcare and get "better bang for the buck" based on Population Health initiatives or Accountable Care Organizations. If you want to know more about this topic, please read one of my earlier blogs on How to transform Healthcare Performance through Data Analytics?); and

General Services Administration is able to look at the efficacy in utilizations of Federal Fleet and Federal Real Estate

Many Federal Government agencies are sitting atop a treasure trove of data. Unleashing this data can help them gain insights on their financial and operational performance, as well as take actions to influence mission outcomes while promoting transparency, collaboration, and accountability.

In May 2014, President Obama signed the Digital Accountability and Transparency Act of 2014 (DATA Act) (P.L. 113-101) into law. Implementation of the DATA Act will not only make it easier to understand how the Federal government spends taxpayer dollars but will also serve as a tool for better oversight, data-centric decision-making, and innovation both inside and outside of government. The U.S. Department of the Treasury (Treasury) and the Office of Management and Budget (OMB) are leading the government-wide implementation of the DATA Act.

By May 2017, most Federal Agencies will be mandated by law to comply with the DATA Act which means all these agencies have to submit certain core information around appropriations, obligations, payments to the US Treasury / OMB in machine readable formats. Such information will further be delivered in visually intuitive ways at "USASpending.gov". There is bound to be lot of interesting questions that will arise once civil society gains access to such information. Should Federal Agencies, particularly Chief Financial Officers (CFOs) / Chief Information Officers (CIOs) / Chief Data Officers (CDOs), not be prepared in advance to analyze and proactively manage their respective performance? Should they also not be able to prepare and tell their own stories around performance instead of society interpreting it in other ways? These are all important questions that need to be answered by every agency across the Federal Government.  To that extent, Federal Agencies have to move beyond compliance and look at this as an opportunity to drive their agency's financial and operational performance.

To move beyond compliance and depending on their respective maturities, Federal Agencies have to take one or more of the following three steps -

UNLEASH AGENCY DATA - Federal agencies need to publish financial and other operational information such as payments, procurement, grants, loans etc. using standardized reporting formats for which they need to have a current state assessment of their data / systems and a clear strategy prior to implementation

ANALYZE PROGRAMS & PROACTIVELY INTERVENE - Once Federal Agencies are exchanging data in a standardized way in compliance with the DATA Act schema, they need to apply diagnostic, predictive and prescriptive analytic techniques to extract insights from program level data that will fuel mission centric decisions

DISCOVER AND VISUALIZE PROGRAM INFORMATION - Use advanced data discovery and visualization techniques, that deliver strategic insights through executive dashboards and highly contextual operational information to front line staff to drive agency performance

So are you ready to move beyond compliance and use DATA Act to transform the way your agency performs? If so, you can email me at "paleru_rk at bah dot com". For more information on DATA Act, you can visit the Data Coalition website.

Data Integrity - The Holy Grail

Data Integrity is the biggest of challenges in analytical value chains. If you were to ask any data scientist, the answer will be clear i.e., 80% of his or her time is spent on data preparation / management before any analytical business value can be extracted from such data. Traditionally, organizations have resorted to extensive investments in IT platforms and processes but have not seen any significant ROI. In my opinion, there are creative alternatives to this problem but only applicable when the business context of data is well understood. Based on my experience working with highly sophisticated data scientists / business systems analysts / data geeks / technology professionals, I have learned quite a few such techniques, three of which are listed below.

Process Hype

Having worked in the data management space for a significant portion of my career, am always skeptical when I come across one more of those heavy IT centric process / platforms that claims to solve data integrity problems - I mean a whole industry that is spinning up around process related tools, technology and skills for data governance, data quality, master data management, single version of the truth etc. The unfortunate part is that substantial resources are invested in these processes and tools with little clarity on how it is done, resulting in anemic ROI. 

Having supported CxOs around systems and data analytics initiatives, particularly CFOs who primarily live in the world of numbers, I can confidently say that data accuracy and associated quality should be seen in the context of the business problem being solved. 

For example, in the case of General Ledger data, mere three cents not reconciled can hold up on monthly closing of books of accounts. Whereas, in the case of social or open data sets used within other functions (say marketing), “good enough” versus “accurate” data will help support actionable decisions. 

So how do we determine the right approach to take? Creative data management may have the answer.

“Creative” Data Management

Having tackled data integrity problems, I have learned the need to approach data problems from a creative solution perspective. Listed below are 3 such specific examples - from simple to complex (out of a variety of solutions we deployed) -

Data Validation (Simple): A variety of business applications source “address” as an input from customers, vendors, partners etc. Some of address fields end up being erroneous or inconsistent - e.g., wrong zip code associated with a city & state or a # does not actually exist on a specific street. Organizations have deployed large master data management platforms with associated data quality processes to correct this at the backend. A simpler strategy for managing this upfront is to validate the address information right at capture time by calling a  web-service (interface program)  USPS Address Management & Verification Services. This service not only validates the address but also suggests what might be the right address. Apart from such simple services, there are a variety of value added services that maybe available with USPS e.g., cost of shipping to these addresses. Such services from USPS can remove the need for storing and managing redundant – essentially USPS becomes a disconnected system of record  in the cloud - away from the core enterprise information.

Computations and Business Rules (Slightly Complex): In any analytical value chain (the process of extracting insight from information), there is a need to tap into data from multiple systems (ERPs, CRMs, Data warehouses etc.). When extracting such information and marrying them together, the problem of “semantic inconsistency” between these data sets arise. For example, in banking, when marrying customer information with outstanding loan balances, sometimes the “accrued” interest is not readily available in analytical systems. Basically, a “semantic inconsistency” is induced in the analytical value chain - i.e., “what the customer really owes the bank on that particular day does not reconcile with the outstanding loan balance that extracted from the transactional system (due to ‘accrued interest’ not being included)”. Organizations have typically resorted to extensive IT investments to capture all such information from transactional systems of record. The problem with this approach is that expensive IT resources are used to address unique and rare business needs. In this example, accrued interest in typically required in the customer service function and that too when the customer wishes to close his or her account).  An alternative, especially when the need is specific within a department or a function, is to actually apply the business rules and computations to recalculate the accrued interest at the last mile of the business need - which involves starting with the customer’s outstanding balance, interest rates, calculations based on policies etc. to arrive at the accrued interest that tallies with the system of record. The advantage with this process is the accrued interest computation is dynamic, can be customized for different date ranges and usually very inexpensive when used within last mile analytical tools including  datamarts or dashboards built using tools such as Tableau or Qlik.

Pre-Load Reconciliation (Complex): In analytical applications / data-warehouses, particularly finance or accounting centric, the need for reconciliation is paramount. If the assets and liabilities don’t tally (even by a couple cents), the books of accounts cannot be closed. Most times, when there is a reconciliation issue, the first instance at which it surfaces is post facto i.e., at reporting. Expensive business resources such as CPAs spend their time reconciling different account balances within a GL with their respective Sub Ledgers (SL) - instead of extracting insights from data to support business decisions. Such effort is complex, manual, expensive and above all is undertaken at a time sensitive juncture in the financial closing cycle. A solution to this data problem is to “reconcile at run-time”. Essentially this means, when transactions from within an ERP is loaded into an analytical layer (say a data-warehouse), a “pre-load reconciliation” program / routine runs to make sure that all account balances are reconciled between the General Ledger (GL) and the Sub Ledger (SL). If there is a mismatch within any one account (out of 10s of thousands of accounts), the entire process is reversed and the particular mismatch is flagged for Controller’s intervention. The advantage with this approach is that it is automated, pre-facto, timely and most importantly shows where the problem lies rather than searching for a “needle in the haystack”

Conclusion

News from Google just hit the wires. Google in its new avataar will become “Alphabet”, a spin on “Alpha-bet” and the erstwhile “Google” itself becoming one of the Alpha-bets.

This got me thinking Financial Economics 101, and here I am churning the “Capital Asset Pricing Model” in my mind.  I am not sure if I agree with Larry Page's financial economics spin on "Alpha-bet".

“We also like that it (”Alphabet”) means alpha‑bet (Alpha is investment return above benchmark)” -  Larry Page (Alphabet CEO)

As far as I know, in "Capital Asset Pricing Model", the linear regression equation is of the form 

"Expected Rate of Return" = "Alpha" + "Beta" * ("Market Return" - "Alpha")

Essentially "Alpha" is "Risk Free Rate of Return" and “Beta” is really the risk premium or multiple that one expects relative to risk, for bets in product-markets. So, in my humble opinion, if Google says it will make "Alpha-bets", are they placing bets on products that are derivatives on “risk free rate of return” i.e., treasury / bond bets? I thought that was what Wall Street bond traders do. 

I think Alphabet’s true opportunity lies in "Beta-bets", i.e, how can the Beta (or "slope of the line")....call it "innovation", whatever, really drive the “incremental return” over and above the “risk free rate of return”. This is where exponential thinking and technologies come to play. 

We liked the name Alphabet because it means a collection of letters that represent language, one of humanity's most important innovations, and is the core of how we index with Google search!

- Larry Page & Sergey Brin (Google Founders)

Guess "Beta-bet" didn't sound great so they choose "Alpha-bet". Maybe their right brain fascination to language and its effect on humanity (not to mention its influence on lateral thinking), is finally ruling over left brain analytical thinking?  As Larry said in another article "we don’t want to be “geeked” over “increasing cost per clicks”. 

Rise of Cloud Computing: If you follow my writings, I am sure you know by now that I believe in the fundamental economic value of any technology. I cut through the IT jargon and get to real world implications of technology to consumers, businesses, governments, and other organizations. In my opinion, most disruptive technologies have one thing in common and its not about technology itself, rather the capability to change the way we derive “economic value add” across society as a whole. Nowhere else has this fact been better established than in the rise of “cloud computing”.

Cloud is one of the most disruptive technologies that has evolved in the last 10-15 years and its rise predominantly associated with compelling business models, shaking the very foundation of the IT ecosystem - from technology vendors, systems integrators / service providers to IT departments. If you have not read, I would highly recommend reading “The State of the Cloud” (or its takeaway from this Business Insider article) by Byron Deeter, Partner at Bessemer Venture Partners. Take a look at the change in market capitalization of public cloud companies…..

.....and the Compounded Annual Growth Rates (CAGR) in revenues of all cloud companies.

Salesforce, as a company makes as much revenue today (in 2015) as much as that of the entire industry in 2008.

So what’s driving Cloud adoption?: While we all are well aware of factors driving the trend in adoption of cloud computing (listed below and is by no means comprehensive), I will try to add some color behind these factors. My intent in to steer your thinking prior to cloud transformation efforts you may undertake -

Brining agility into IT: Cloud companies have proven that hosting and scaling applications in the cloud is a more agile approach to solving business problems using technology, without the need to rely on IT organizations who have their hands full in managing legacy IT infrastructure. It is in this context that the concept of DevOps (a portmanteau of Software Development & IT Operations) is becoming very popular.

DevOps is a software development method that emphasizes communication, collaboration, integration, automation, and measurement of cooperation between software developers and other IT professionals. The method acknowledges the interdependence of software development, quality assurance, and IT operations, and aims to help an organization rapidly produce software products and services and to improve operations performance. 

There are host of skills, best practices and tools that are evolving in the market that is clearly increasing the agility of Cloud companies and their ability to release application versions, almost seamlessly.

Innovation and skills optimization: IT organizations are finding it increasingly difficult to attract and retain the right talent. Given the compulsions to juggle support for multiple legacy systems, even the skills available within such IT organizations are fairly generic. For example, ERP Business Analysts who maybe focused on “core finance processes / systems” are being asked to diversify support into “operational or customer related business processes”. While this approach may work for around core systems of record, as innovations start coming up at the peripheries it will difficult for IT organizations to continue innovating. Having run analytics / business IT functions, in my opinion, this “jack of all trades” approach being pursued by IT organizations will not work. Even if Line of Business (LOB) is willing to fund specific innovation projects, it is almost impossible for IT to execute, given generic skills and outdated infrastructure - a reason we see business functions increasingly flock to nimble cloud vendors to solve specific problems. Cloud computing companies such as  Concur for expense management solutions orWorkday for core HR and finance processes are  becoming popular. Companies such as Athena build software platforms for Healthcare providers (physician offices) that are also being embraced by newer business models in healthcare delivery -please see one of my earlier blog posts on emerging business models in Healthcarein which I mention about Privia. Privia uses Athena’s technology in the cloud to help optimize clinical performance.

How can IT departments compete on innovation and agility using generic skills available internally when Venture Capital companies (VCs) are fueling innovation in the cloud. Business functions just have to look outside and they have a plethora of solutions for every conceivable problem and that too at a very micro-optimized level.

No wonder there is a rise in the “long tail” of innovation for speciality solutions to business problems in the cloud.

In such as scenario, it is better for IT to move from a “build” to "orchestrate” approach, where they are seen as trusted business partners in evolving innovation roadmaps that bring the “best of breed” applications in the cloud together with “on-premise” infrastructure for seamless business transformation. 

Opex versus Capex: As business models change, at a pace hitherto unseen, organizations (particularly CFOs) prefer the pay-as-you-go plans of cloud computing. Rise of public, private and hybrid cloud options are structuring software services and costs based on business needs for outcomes.

Lets look at it this way, if I kept moving from one city to another every other year - would I buy or rent? Obviously, rent! So why would organization challenged with changing business models and processes lock into on-premise software when they can easily rent from a specialized SaaS provider? This is particularly applicable for new business problems that are morphing by the day.

Economies of scale and efficiencies: This is probably one of the most popular reasons for push towards cloud. The years following the economic crisis had put a strain on corporate budgets for IT. CIOs are being left with very little budgets for spend outside of maintenance upkeep (or lets just say, keeping the lights on). This is forcing organizations to move away from multi-year (multi million $) business transformation programs using on-premise software that was tied to major upfront licensing costs, extensive implementations costs, cost of maintenance / unnecessary upgrades and extensive hardware infrastructure costs.

On the other hand cloud companies are - Embracing Open Source software stacks or OEM models - cutting down massive licensing costs; Using standard frameworks on commoditized hardware - e.g., with emerging frameworks such as Hadoop to cut storage / processing costs (read how this is being done in one of my earlier blog posts on “big-data”);

Building customized solutions, not mere applications - given their niches and deep domain expertise;

Minimizing expensive upgrade cycles - preventing IT resource from being committed to endless upgrade cycles that are driven by on-premise software vendor priorities, even if the features from such upgrades provide little business value.

Challenges in Cloud adoption?: The purpose of this blog is not to get behind the strategic overview or execution approaches in adopting cloud computing. I will dedicate a separate blog in future for that purpose. Having said that, let me at least list some of those challenges in adopting the Cloud -

Security and Privacy - How to trust information in the hands of cloud vendors? Do they have the necessary IT security certifications / audit in place? How to address liability and reputation risks in case of data breaches?  

Systems Integration - How to cross pollinate core systems and innovation at the edges i.e., how to integrate business processes / data across the core systems of record that are managed internally and innovations in the cloud.

Support & Operations - How to support applications that are being released in a rapid delivery cycle by these cloud vendors and how to ensure all the added functionality is end-to-end tested (including regressing testing). Basically, there is a n! (n Factorial) increase in complexity of test cases for each new release or added functionality (n). There is also the challenge in testing configuration changes across multiple cloud services i.e., the almost impossible combinations of testing configurations.  

So what is the Future for cloud computing?: In my opinion, there are various factors driving cloud adoption / challenges as listed above. I still believe that this is just a tip of the iceberg and start of a disruptive revolution. In the case of certain mature applications like CRM, cloud revenue has already displaced on-premise software revenue by taking more than 51% of the spend. In my opinion, this trend will continue and start affecting other categories of software applications (please see picture below). 

In my opinion, cloud computing is going up the value stack…. What started as…. “Infrastructure-as-a-Service (IaaS)” with companies such as Rackspace;

has evolved to….

“Platform-as-a-Service (PaaS)” with companies such asInformatica that provide ETL capabilities in the cloud;  and

“Software-as-a-Service (SaaS)” with companies such as Salesforce that provides CRM capabilities or Netsuite that provides ERP capabilities in the cloud;   but the most disruptive of these models will evolve when companies embrace…..

“Information-as-a-Service (IaaS)” with companies like IMS that provide anonymized prescription information as a service to Life Sciences industry or Dun & Bradstreet that provide credit information or vendor status (minority or women owned etc.) as a service for compliance reporting. While all this maybe in early stages, I was pleasantly surprised recently when listening to Aneesh Chopra (first / former Federal CTO of the United States in The White House under President Barack Obama) quoteJoyce Hunter, CIO of US Department of Agriculture. Joyce is actively considering opening up non-confidential / open data from the USDA for monetization by consumers / corporations. For e.g., information about Tickets for entry into National Parks (managed by Parks & Recreational Association within USDA) when opened up, has the potential of being used by commercial providers to wrap and sell value added services as part of their offerings - e.g., why can’t an Expedia or Orbitz source such open tickets data in the cloud and sell value added services of promoting tour / vacation packages? Insights-as-a-Service (InaaS) when companies can mash different sets of data within an enterprise (say CRM data) and / or outside (say census data) to deliver business insights contextually to consumers, businesses, citizens, governments, and society at large. Please read this post in the WSJabout how all these sophisticated capabilities are being applied - “How Real Estate uses big data (including obituary and lifestyle data) to track clients”

Conclusion: As the value of the cloud stack goes up the chain from “Infrastructure-as-a-Service” to “Insights-as-a-Service”, the valuations of these cloud companies, disruptive force of these technologies and eventually the transformative capabilities of the businesses they serve will explode exponentially. Fasten your seat belts and enjoy the thrill ride.

This post is about the approaches to “big-data” technology. However, I start with a little detour and analogy on innovation. The purpose of this analogy is to trigger “Enterprise IT Organizations” to think outside convention before embarking on a journey to adopt or build “big-data”, “data management”, “data integration” and related services. Start with the “Why” (not the “What”): If you have not already, I would highly recommend watching Simon Sinek’s famous TED Talk on “Start with the Why (also embedded below)”. In this talk Simon synthesis how successful and inspirational leaders / brands, such as Dr. Martin Luther King Jr., Apple Inc., and The Wright Brothers, drew strategic intent from their respective missions / purpose and how they brought together all that thinking for seamless execution.

Whether we build products, sell services, look for a job, support a social cause, whatever……most of us start with “what we do”, go on to explain “how we do it” and finally try to get to “why we do it”. While this inside out approach looks logical to each of us - going from our zones of comfort of what we know best (i.e., “what” we do) to the more fuzzy and relatively unknown areas (i.e., “why” we do something) - it does not matter to our customers / citizens / followers / audience etc. To clarify, success / rewards / money etc. is not a “why” but a “result” from things we do. 

Simon Sinek summarizes - ”People buy into WHY we do something, not WHAT we do”. Its not about us, its about them. So we have to reverse that order and draft our strategic intent  to Why -> How -> What.

So what does this have to do with “Big-Data Services” and “Enterprise IT Organizations”?: Over the last couple weeks, I have had three different conversations on the topic of big-data technology with - (i) a senior IT executive of a Tier 1 Bank, (iii) a senior business executives / CFO and (iii) Ray Wang from Constellation Research - futurist /  big-data strategist / digital disruption thought leader / industry analyst. All three conversations, directly or indirectly alluded to “how enterprise should think about “data services” (including big-data) and what should their strategy be?”. Questions that we all noodled were -

What will be the next killer technology - Hadoop with MapReduce Vs NoSQL Datbases, or Graph Databases Vs Columnar Databases etc.?

Will Hadoop (or a variant such as Apache Spark) replace Enterprise Data Warehouses (EDW) (e.g., Teradata)?

Will data stack vendors with integrated in-memory analytical processing engines (e.g., SAP HANA or OBIEE etc.) be disrupted?

Will self-service driven enterprise data-marts / data discovery & visualization tools (e.g.. Tableau or Qlik) / business user driven ETL tools (e.g., Paxata) replace traditional BI  / ETL?

The single factor that came up in all three conversations - IT organizations are struggling to understand how to bring all these technologies together - driven by the rapid pace of evolution (and marketing hype from technology vendors).

While we have all heard about the macros trends or 3Vs of Volume, Variety, Velocity of Big-Data, little has been debated, beyond technical communities and silicon valley think thanks, on the many technical factors that need to be evaluated before defining a “big-data” technology services strategy. Here are some key technical factors driving data architectural discussions (and by no means this list is comprehensive) -

Analytics is going to Data (not the other way around): With storage becoming cheap, data volumes exploding and analytical processing over large data sets becoming performance intensive - data is no longer required to be brought into an analytical engine. Its the other way around, with analytics now taken to data lakes. (e.g., MapReduce takes analytical processing into Hadoop).

Data stores reflecting real world semantics: Some special purpose databases are reflecting the semantic structure of data and their inherent relationships and not storing it in relational or object constructs (e.g., Graph Databases reflect Customer -> Orders -> Products Purchased as a graphical construct i.e., how our minds think not how we store data in rows and columns)

Capture versus analyze: Is the big-data coming into tan enterprise being captured for future analysis or analyzed as it is captured? (e.g., Hadoop Distributed File Systems versus Complex Event Processing)

Optimizing analytical queries: While SQL based querying has become a standard over the last couple decades, newer optimization techniques are becoming popular requiring different data stores (e.g., columnar databases using key-value pairs versus SQL result sets)

Self service, Data Agility and Late Binding: Business use of last mile analytical processing / self service layers is becoming very popular versus pre-prepared data (i.e., data that has gone through ETL and loaded into warehouses). In fact, business is no longer willing to wait for on turnaround times from IT and demanding data agility. Consequently, “late binding”where the transformation of as part of the ETL process is getting closer to analytical querying - ETL is increasingly becoming ELT. (e.g., Data discovery & visualization tools such as Tableau or business user centric tools such as Paxata or Trifacta are becoming popular. If you are interested in the topic, please read one of my earlier blog post - Business is winning the BI battle, but should it be a battle?).  

So how do IT organizations decide which big-data technology to use?:  As a “business analytics solutions” professional, I have always focused on “business outcomes based on insights from big-data”. Please see my earlier blog on “Data = Opportunity, but is your company monetizing information?” that gives an overview of outcomes driven business analytics or “How to transform Healthcare Performance through Data Analytics?” if you want to know more about specifics of my views within a particular industry. 

If we agree that choice of big-data technology depends on the specific use cases for business outcomes using better insights from data (the “why”), the choice of technology (the “what”) becomes fairly simple. In such as scenario, all we have to do is to follow a straight forward process to weigh the pros and cons of factors of a particular technology (as listed above) with respect to the specific use case for business insight. Here are some high level use cases for business insights from data, with an indicative choice of technology (again this list is by no means comprehensive and can change based on exact needs):

Integrated vendor specific analytics stacks: This is the best bet for purpose driven applications that require seamless reporting / analytics. For example, it is better for General Ledgers (GL) systems to have core analytics / reporting integrated within the applications stacks. Having supported Financial Comptrollers, I can say with a fair degree of certainty, that financial reporting within or close to an integrated System of Record (say a Oracle Financials or SAP Financial - GL Module)  is best achieved in an integrated analytical stack (Oracle’s OBIEE or SAP’s HANA respectively). Comptrollers are very uncomfortable about data integrity when a transaction leaves the System of Record - even if it is for reporting purposes (accuracy and reconciliation are of paramount concerns).

Datawarehouses: While I am not a big fan of datawarehouses from an agility standpoint, I would tend to disagree with some pundits who predict that the demise of Datawarehouses is around the corner. While I believe technologies such as Hadoop Distributed Fie System will co-exist from a data capture / storage standpoint, datawarehouses still have a role to play within the ecosystem particularly given their relative analytical strength (stemming from ETL). For example, datawarehouses as a cross systems “Single Version of the Truth” will continue to stay e.g., marrying GL with operational transactional systems or sub-ledgers. So if you are asking me, Teredata is here to stay! No that’s not a stock tip.

Data-marts / Last Mile OLAPs / Business Driven ETLs / Data Visualizations: As discussed earlier, there is a fast growing business user driven need for departmental analytics. In particular, finance and marketing users who are data savvy e.g., with financial performance management or digital marketing analytics, will use OLAP cubing style tools such as Oracle Essbase / IBM Cognos TM1 or even some of the next generation SaaS tools such as Anaplan (I am hearing a lot of good things in the market about this offering). Tableau and Qlik will emerge as data discovery and visualization tools of choice for the same user base. The issue with this particular approach is data governance challenges. However, my belief is, with sufficient governance and better processes these late binding capabilities / last mile analytics are here to stay.

Hadoop Distributed File Systems / Map Reduce (or Apache Spark): The strength of this approach is its ability to capture large volumes of data in a scalable fashion for processing later. Look at this approach becoming popular with emerging large data sets such as mobile, social, web analytics etc. For example, a company might capture its fan engagement on its Facebook page or track interactivity on its website or location details from mobile devices to analyze and tailor campaigns. 

Machine Learning / Artificial Intelligence: This is an emerging area and will require large volumes of data being pumped into analytical engines for finding patters, correlations, etc.  These platforms / frameworks are in early stages of their evolution cycle and will evolve based on needs.

Conclusion: There was a time when data technology lagged business needs. Today, that’s not the case. There are quite a few sophisticated offerings as discussed above and more in the offing. In my opinion, the choice of technology will follow an evolutionary path to support the disruptive business models being designed by companies. Disrupt or be disrupted, and for the ones that disrupt big-data is here to stay (and grow) by their side.

Thanks for visiting this blog post. Given the scope of topic, this post is longer than my other posts. I have, therefore, split this into three sections - strategic overview, execution approaches and conclusion. You can choose to read any or all sections based on specific interest and time available. I welcome all comments, questions, shares and likes.

SECTION I - STRATEGIC OVERVIEW

If I am asked to pick one industry in the US that is most starved for actionable insights from data, I would not hesitate to pick “Healthcare”. 

What is the problem?: Healthcare expenditure constitutes 17-18% of US GDP (growing 2.4% faster than GDP since 1970) and health outcomes, as defined by efficiency scores (34.3) and life expectancy (78.7 yrs), trail that of all other developed economies in the world. Our performance and outcome scores are much closer to that of third world countries while our healthcare related spending is the highest. Much of the debate around these data points is centered around the macro-economics (and politics) of who pays the bill i.e., socialized medicine versus capitalist medicine, but there is very little clarity or discussion on how to transform healthcare system as a whole to improving care quality and doing so efficiently. Prognosis:  My belief is that healthcare will undergo rapid transformation in the coming decade, hitherto unseen, driven by some macro trends -

demographic factors e.g., aging baby boomer population;

changing lifestyles e.g., millennials’ focus on disease prevention over cure or adoption of emerging health related technologies;

payer mix e.g., who will pay the bill - medicare / medicaid or private insurance or out-of-pocket?; and

delivery effectiveness e.g., delivering safe and quality care most efficiently.

How is the healthcare ecosystem responding?: The entire healthcare ecosystem - Providers (comprised of hospitals, physician practices, senior care providers, hospices etc.), Payers (comprised of insurance companies, government, individuals etc. and Others (comprised of pharma and life-science companies, researchers, equipment and technology providers, diagnostic & nutritionists etc.) -  is monitoring these macro trends to evolve their respective strategic vision and transformation roadmaps. Every entity is starting to think about care delivery and associated financial models towards “better health outcomes for patients”, away from the “pay as you go” model of charging fees for procedures or services delivered. A classic example of this trend is seen in the rising number of Accountable Care Organizations (ACOs), in which providers come together to optimally manage care delivery, negotiate reimbursements and target incentives tied to better health outcomes, all within the framework prescribed by Center for Medicare & Medicaid Services (CMS) that ties reimbursements and incentives to better care for individual patients, better population health and lower growth in expenditures. Strategic trends in healthcare transformation: Driven by all this, in my opinion, there are three main strategic trends evolving -

1. Clinical Informatics - The clinical side of healthcare across the prevention-cure-recovery cycle is adopting information and technology to be effective. and in the process moving healthcare “from one that is episodic to a continuum”.

Precision Medicine - Personalized therapy based on individual genetic signatures; Cognitive Health Computing - Helping doctors narrow down diagnosis using machine learning and artificial intelligence; Smart Devices, Wearables, Ingestibles - Fitness tracking for lifestyle management or health monitoring for proactive intervention. Further, these devices also offer newer ways of leveraging information as defined by Internet of Things (IOT) Nutrition Apps - Managing calorific intake Open Health Data - Using published data sets from the government for understanding health service quality and / or improving access to services. As an active member of the Federal Big Data Working Group in Washington DC, I have had an opportunity to be exposed to some great innovations that is happening in the Open Data space, including health care data that is being leveraged in public-private partnership models  If you want to know more about my views on how information is transforming many other industries, please read one of my earlier blog posts on “Data = Opportunity”, but is your company monetizing information? 

2. Emerging Care Delivery Models - New business models are emerging that are focussed on streamlining care delivery and in the process creating revenue opportunities. These companies partner with healthcare providers to streamline delivery and do so by (i) working with a technology vendors and process consultants, (ii) building a network for better care co-ordination, and (iii) work with payers, suppliers etc. to negotiate better terms. Example of these type of emerging companies include Privia or Evolent. These companies are growing very fast and have the backing of private equity firms / investment banks such as Goldman Sachs.

3. Provider Operational Transformation - Providers are also focusing on improving operational efficiencies across care delivery, care quality, proactive intervention and resource optimization for enhanced performance.

The limited focus of this blog post is to raise some thoughts on the strategic transformation of healthcare from the context of 2 and 3 above i.e., emerging care delivery models and provider operational transformation. This post WILL NOT to delve into clinical informatics (1 above). I will dedicate separate posts in future for data analytics related innovations in these areas.  

What’s are the critical challenges and opportunities for healthcare transformation?: In the backdrop of these strategic headwinds and opportunistic tail winds, healthcare ecosystem is trying to  -

Manage and grow revenue by delivering better health outcomes: Identify, analyze and proactively manage revenues by delivering better health outcomes;

Improve care quality while managing cost of care delivery: Benchmark, analyze and improve care quality and operational efficiencies while managing cost of care delivery;

Proactively predict and intervene to manage health risks and hazards: Predict patient safety and proactively manage risks such as “Never Events” and high-cost health care cases;

Optimize profitability: Understand revenue and cost structures across various dimensions of health care - hospitals, locations, practice lines, procedures, patient sets etc. to optimize profitability  

How can data analytics help?: To operate effectively, healthcare ecosystem needs to -

Track disparate data sets to know “how are they performing” with respect to care delivery, patient satisfaction and financial performance;

Analyze such data sets to better understand “why are they performing the way they are”;

Predict and act upon such analysis to decide “what to do to enhance performance”.

As the industry evolves to adopt these sophisticated practices, Healthcare ecosystems and business practices that embrace “Data Analytics” strategically and execute meticulously, will not merely survive but thrive in the emerging environment. 

SECTION II - EXECUTION APPROACHES

So how should the healthcare ecosystem go about this Data Analytics led transformation? As seen earlier, there are 4 specific challenges. Lets look at some specific ideas for each of those challenges, backed my recommendations on using data analytics.  

1. Manage and grow revenue by delivering better health outcomes: Care providers are increasingly looking to grow and manage revenues while improving health outcomes. Unlike other industries where there is a possibility to go after new revenue sources, care providers have to focus on getting more from existing revenue rather than going after new sources - i.e., earning incentives based on care outcomes delivered or getting more out of their existing billing. For example, providers can mine data from Electronic Health Records (EHR) to understand and manage measures in the context of patient experience, care coordination, patient safety, preventive health, and at-risk population to target better incentives as part of the ACO. Also, they struggle to understand factors contributing to revenue leakage and variance between gross and net revenue can be staggering.

How can Data Analytics help: In the case revenue leakage example, while billing systems may provide gross and net revenues at aggregate levels and actual patient billing at the granular levels, forward looking care providers should be able to extract / marry data from across billing and collection systems into analytical data marts (or data warehouses) such as MicroStrategy and use highly intuitive reporting tools such Tableau, to look at such differences in revenue across multiple dimensions - healthcare facility, periods (day, week, month, quarter, year etc.), specific procedures (specific CPT codes) etc. Ability to visually see such information and being able to drill to understand the root cause for denials helps manage revenue leakage.

2. Improve care quality while managing cost of care delivery: If at all there is one big area ripe for transformation within healthcare ecosystem, it is about improving operational efficiencies while managing cost of care delivery. There is lot of waste and abuse in the way healthcare dollars are spent on unnecessary diagnostics, procedures etc., most times having very little impact on outcomes. Consulting companies such as The Advisory Board have dedicated practices for healthcare to benchmark costs across the industry. Armed with such information, providers may need to apply (i) spend analytics and best practices around supply management (i.e., buying from preferred suppliers, ordering in economic order quantities and avail timely payment discounts), (ii) improve admission-to-discharge process efficiency and increase throughput to get more out of existing assets, (iii) reduce length of stay without compromising discharge readiness, (iv) reduce unnecessary testing and diagnostics and (v) reduce billing errors with associated personnel costs.

How can Data Analytics help: Today’s expense management and financial General Ledger (GL) systems have evolved to tag / track various types of expenses at the transaction level i.e., expenses at this lowest level of granularity. The transactions can be loaded into datawarehouses or OLAP  platforms or data cubing engines such as Oracle Essbase or IBM Cognos TM1 for further analysis depending upon the specific need. Again these tools integrate easily with visually intuitive dashboards or self service business intelligence or data discovery tools for exceptional end user analytical experience.

3. Proactively predict and intervene while managing health risks:  High cost of healthcare is usually associated with certain catastrophic events and high cost healthcare cases. As an example, “Never Events” are errors in medical care, clearly identifiable and measurable, such that the risk of occurrence is significantly influenced by the policies and procedures of the health care organization. A "never event" includes falls, infections, erroneous amputations and other small-to-large-scale disasters. Reducing these events is, of course, best for patients, but there's also financial pressure to reduce never events - medicare and most private insurers do not pay for never events. Another example, patients with chronic illness in their last two years of life (also known as end of life care) account for about 32% of total Medicare spending, much of it going toward physician and hospital fees associated with repeated hospitalizations. While providers cannot chose patients or diseases, they can definitely predict potential high health risk cases and focus early in the treatment cycle to prevent deterioration. Also providers look at data trends in realtime to optimally manage patients being discharged. Providers need to predict and pro-actively manage such risks to manage performance.

How can Data Analytics help: Care providers could use a variety of data sources such as patient EHR (for aggressive care intervention), real time streaming data of hospital air quality (for infection outbreaks), discharge reports (for evaluating re-admittance risks) etc. These types of disparate and emerging data sets can be aggregated into big-data platforms (e.g., Hadoop) to mine patterns and predict risks. Programming models and tools such as Map Reduce, IBM’s SPSS or SAS are very good at such predictions.

4. Optimize profitability: Understand revenue and cost structures across various dimensions of health care - hospitals, locations, practice lines, procedures, patient sets etc. to optimize profitability. This is a fairly deep topic and of immense interest to CFOs and CIOs worldwide. In case of further interest, I would recommend reading a research paper I published (when I worked at IBM) as a joint research with the CFO Magazine. The paper can be found here => http://businessfinancemag.com/site-files/businessfinancemag.com/files/archive/businessfinancemag.com/files/misc_file/IBM-unlocking-profitability.pdf

SECTION III - CONCLUSION Are you part of the healthcare ecosystem and looking to transform? If so, here are some recommendations (based on the maturity of your initiative) (i) Envision Strategy - start by envisioning a strategy - what is your organizational vision around transformation particularly in using information and data analytics, (ii) Obtain Sponsorship - obtain investor, board & c-suite sponsorship as applicable, 

(iii) Understand Culture - understand the culture of your organization before articulating a vision,

In 2001, Gartner analyst Doug Laney, coined the term “big-data”, to articulate the impending explosion in Volume, Variety, and Velocity (“the 3 Vs”) of data in society. Much has been talked since about big-data. Today the term big-data is primarily used to describe “how” technology can handle these large and disparate sets of data, not “what” can be done with big-data. 

Infonomics - In the late 1990s, Doug also coined another important but lesser known term, “Infonomics”, to assert the “economic significance of information”. Infonomics refers to a framework by which businesses could value, manage and most importantly wield information as a revenue generating asset. 

Infonomics signifies the business value of information and if that’s the case should companies not embrace the concept to enhance customer experience, generate new revenue, innovate product & service offerings, and optimize margins?

While companies have traditionally leveraged information to manage expenses, drive operational efficiencies, mitigate risks etc., the purpose of this post is not to delve into those  use cases.

Rise of the info.-disruptors - While companies such as Amazon, Google, LinkedIn, Netflix are all well known to monetize information, and in the process have displaced many retailers, newspapers, recruiting firms, media companies respectively; many recent startups such as Uber, AirBnB, and Alibaba are disrupting many other industries. 

Think about it -  Uber - Worlds largest taxi company, owns no cars AirBnb - World’s largest hospitality company, owns no properties Alibaba - World’s most valuable retailer, owns no inventory

So how are these info.-disruptors able to compete with traditional incumbents and capture marketshare? 

Essentially, these nimble startups have leveraged information generated in the marketplace to better serve customers and in the process generate revenue and gain marketshare. The fact that these startups had no information to start with and had to create a marketplace for such information to be freely shared by consumers, speaks volumes to their innovation potential. They create compelling value propositions in the marketplace for a latent or unmet need, so much so, that consumers willingly adopt and share information to satiate their needs.

Traditional incumbents in these markets i.e., taxi operators (Uber’s competition), hospitality companies (AirBnB’s competition), retailers (Alibaba’s competition) and others have not proactively seized the opportunity. The sad fact is that most of these incumbents also sat on rich sets of information about their customers, product / service consumption patterns, etc. and did very little to better serve their customers and protect or enhance market shares.

Are traditional brick and mortar companies ready for the massive disruption that’s ahead? In my opinion, it may not be long before we see many traditional products and services dissolve as information that is delivered over software in the network ether. In such a scenario, are traditional brick and mortar incumbents prepared to handle the disruption? 

Here are some of the examples of what is already happening in some industries. Higher Education - Massive Open Online Courses (MOOCs) is a rising trend with universities throwing open their course content (almost free). Companies such as Coursera and Edx are offering software platforms that facilitate this mechanism and in the process are generating large sets of data that can be used in enhancing the learning experience and a potential opportunity for monetization. 

Payments - Growth in multiple payment channels that facilitate monetary exchange in the context of customer preferences and their unique transactional needs have changed the thinking from a “bank as the center of the universe” paradigm - e.g., ACH debit or Wire Transfer. Companies such as Square, PayPal, Apple Pay, Google Wallet etc. are redefining such payment mechanisms and with it the information and monetization opportunities.  Banking - Financial disintermediation, a process by which, deposits are directly channeled to fund loans, bypassing the banking system, is becoming an interesting business model. Companies like Lending Club make peer to peer lending possible. Other examples such as Wealth Front disrupting wealth management firms or eToro using social data for investment advice and trading are abound. If you want to read more on eToro and social information being used in stock trading, please see my earlier post here http://informationanalytics.tumblr.com/post/65281414830/dont-believe-in-society-or-yourself-when-it-comes

Other Examples -

Philanthropy - Kiva dis-intermediating non-profits in charity Conferences - Meetup as a free alternative to groups sharing knowledge  Healthcare - Wearables and Ingestibles promoting healthy lifestyles Home Automation -  Google acquisition, Nest, manages home experience  Transportation - Driverless cars disrupting automobile manufacturers Shipping & Military - Drones shipping packages and fighting wars

....and these are just a few in an endless list of examples.

So is this all “doom and gloom” for traditional companies?  I personally don’t think so. Everything depends on how traditional companies respond to this challenge. Will they treat it as an opportunity and invest; or will they perceive it as a threat and hunker down? Such decisions will result in outcomes and their eventual existence. 

In my opinion, as products and services turn into information and software, the lines of distinction between traditional companies and info.-disruptors will disappear. It will be a free market with little entry barriers. Not only will traditional assets on a balance sheet such as real estate, fixed assets, inventory etc. cease to be entry barriers, but when unmanaged could become an overhead and a drag on the incumbents. Traditional companies that see the opportunity in leveraging information and building competitive business models will thrive, and in the process become info-disruptors themselves. There are some early examples of such companies e.g., Walmart, USAA, and Capital One. I will dedicate a future blog post about the innovation in these companies. 

Is your traditional brick and mortar company and industry ready to monetize information? If not, start thinking about a strategy and build the leadership and teams to be put in place? No, and you will not find the skills in your industry. Your best bet is to look for data analytics professionals from the software industry with proven solutions experience.

Last week I was inducted to serve on the Board of Directors of Artomatic, one of the large organizations in the metropolitan Washington DC area promoting arts and building a community. Artomatic’s mission is to create community, build audience and expand economic development by transforming available space into a playground for artistic expression. 

Artomatic is a non-profit, volunteer-run 501(c)3 organization that organizes and hosts large arts festival occurring every 12-18 months in the DC area, usually in a commercial location slated for demolition. The event is unjuried (first-come-first-served) and showcases creative work in visual art, music, film, performance, and fashion. The 2012 Artomatic attracted about 1700 artists / performers worldwide and over 70,000 visitors. Artomatic is funded in part by the D.C. Commission on the Arts and Humanities, an agency supported by the National Endowment for the Arts. It’s a honor and privilege to have been selected as the Asst. Treasurer of Artomatic, but also a tremendous responsibility, and one that I thought long and hard about before taking on. It’s a commitment, requiring the same level of attention and care as on any private company board. I wanted to share some of my thinking, around the decision, with my network. I also want to ask for all of your support and help as I seek to contribute to the mission of this organization. I believe this opportunity gives me back more than what I will be able to give. Here is why I decided on taking up this opportunity - Art has a very big role to play in society and I want to contribute - Art promotes peace and harmony within communities while opening minds to alternative perspectives. It can change the way we view the world, and in fact, life sometimes imitates art rather than the other way around. To give an example - in the movie “Minority Report”, released in 2002 but set in the future year 2054, Steven Spielberg showcased bio-metric scanners located at retail store entrances to mine customer data for making promotional offers. Here we are in 2012, already seeing early concepts in retail marketing campaigns where location data from our mobile devices is being used to make campaign offers. Déjà vu!

The organization and role complements my professional life - The right brain of “art” complements my left brain of “analytics”. I strongly believe that one without the other is not complete. No amount of business, data and digital analytics can influence behavior without elegance of user experience. In my professional life, I am an extensive user of a software product called Tableau for data discovery and visualization and it is just a classic example of where my right brain dates my left brain. Further, the Asst. Treasurer role complements what I do in my day job within the CFO’s office at the George Washington University where the Treasurer is a customer I serve. The role at Artomatic will help me walk in my customer’s shoes apart from making me eat my own dog food when it comes to tailoring Business Analytics solutions.

My way of giving back to the community - Art has always fascinated me. I have always related to artists and their subliminal messaging. Being able to help artists and in the process build a community sounded very appealing. I was not born with artistic abilities, but have always appreciated all forms of art. I enjoy street musicians in New Orleans as much as classical artists in Washington DC. Artomatic’s approach to showcasing unjuried arts resonated at a very philosophical level in me i.e., ultimately nothing in life should be judged prematurely.

What I learned from Gartner's 2015 predictions on BI & Analytics market? 

Every year I look to Gartner’s predictions on the Business Intelligence and Analytics market, not only because I want to know what’s right and where the market is headed but also to learn what maybe wrong and what can practitioners like me learn from it.

Here are 3 trends and my takeaways from Gartner’s predictions for 2015 -

Trend 1: Balance of power continues to shifts from IT to Business, driven by Data Discovery and Visualization tools - Gartner’s 2015 prediction is in line with what they has been saying for quite some time now, i.e., traditional business intelligence (BI) and analytic models are being disrupted as balance of power shifts from IT to Business. Nothing surprising here, looking at how Business centric data discovery and visualization tools - e.g., Tableau, QlikView, have systematically disrupted incumbent IT-centric BI platforms - e.g., IBM Cognos.

Takeaway - While IT centric BI platforms thrived in days of data aggregation and mass distribution of pre-prepared data and reports, Business centric BI tools have started tilting the equation in favor of agile self-service in solving business problems with data analytics. It’s putting the business data expert and power user at the forefront while changing the BI landscape from “reports for the masses”, to “analytical insights for the classes”.

Trend 2: Data preparation capabilities are emerging that will provide business users and analysts the ability to extend the scope of self-service - One of the biggest challenges faced by any enterprise seeking business insights from data is the need to depend on a central IT organization. IT traditionally uses Extraction, Transformation and Loading (ETL) tools and routines for bringing data from across the enterprise into a data warehouse, using tools such as Informatica, IBM InfoSphere DataStage. The problem with this approach has been (i) business has to get into an IT queue and wait, (ii) clunky tools that requires IT skills for even basic usage; and (iii) data transformation (if applied during the ETL process) becomes outdated quickly as business conditions change. Typically the process of Data Preparation achieves two objectives, (i) aggregates enterprise data from across different source systems into a data warehouse for ease of end-use and (ii) attempts to create a single version of the truth within a data warehouse. Emerging self-service Data Preparation tools such as Trifacta, Paxata have made both these processes redundant i.e., (i) there is no need to physically aggregate data as long a logical layer that has the potential to marry the semantics of data from across the enterprise, just-in-time, for business user self-service i.e., let the data live where it is and integrate it at the end point of the business solution needs, obviously under proper governance; and (ii) are driven on end user demand versus IT supply principle i.e., business user needs driven versus IT capacity and understanding driven. If you want to know more about what’s happening in the Data Preparation market, please see the recent study byDresner Advisory.

Take away - Self-service data preparation will do to traditional IT-centric data integration what data discovery platforms have done to traditional IT-centric BI. It’s coming!

Trend 3: End user thirst for data coupled with nimble self-service tools has created an analytical sprawl across the enterprise and making governance a challenge - End-user clamor for access to business data, combined with IT’s inability to satisfy this need, has manifested in self-service BI initiatives in many organizations. The need for just-in-time and contextual insight, driven by business compulsions has contributed to this trend. Vendors have responded with mass consumable, broadly deployable, easy-to-use and, often, cloud-based technologies for basic query, analysis and reporting. Often, these solutions are implemented by business units that have circumvented IT and as a result, they are disposed to analytic sprawl — an inconsistent or incomplete use of data. While this may solve problems in the interim, this is not a sustainable approach in the long run from a Data Governance standpoint.

Takeaway - While I agree with Gartner on the need to better govern the analytical sprawl, I disagree to the extent that IT has the capacity to start defining the semantics for such data using Data Governance. IT is just custodian of data and rarely understands the semantics. IT has to increasingly create an environment where Business users can manage their business analytics needs in a governed way.

It’s Valentines Day - time to celebrate love and friendship. Now into my middle age, well past the adrenaline rush of high school friendships and teen infatuations, I wonder what is it that I have learned about love from the deepest, happiest and most fulfilling of personal relationships. My intent here is to see if I can apply any of that learning towards improving the most challenging of relationships i.e., between “Business and IT”, in the enterprise digital era?

It is well know, across most organizations, that IT’s agility to solve problems at the pace of market realities is diminishing. At the same time, given information security risks, Business adoption of technology needs to be better governed. While these competing objectives have deep rooted philosophical differences, the unfortunate situation across most organizations is that it has resulted in passive aggressive or sometimes even overt push backs between these two organizations in working together in meeting business objectives. What was once the most romantic of relationships in the mainframe era has suddenly turned sour with FUD (Fear, Uncertainty and Distrust).  

Can the spark be reignited? Can the best be brought about in this relationship within the constraints of environment realities? Can this be made a win-win partnership to solve business problems?

Yes, I believe this is a definite possibility and can be learned from other successful relationships. Here are 3 things that Business and IT leadership can do to rekindle the fire -

Acceptance - First and foremost, we all know, love is about acceptance. We all are different and we usually come together by accepting each others respective strengths and weaknesses. However, sometimes completely driven by factors beyond our control, our capacities and capabilities may change over time. It is at this time, we need get back to the basics that built relationships in the first place - Acceptance. For example, 30 years ago when IT was more about hardware / software infrastructure services, centralized control and services delivery made sense. However, as more and more of Gen Ys  (Internet era babies) or Millennials enter the consumer market place and workforce, market realities have changed. Its about highly sophisticated IT services “on-demand”. In this era, it takes both sides - Business and IT - to understand and “accept” respective perspectives. Without acceptance of this reality, there is not even a remote possibility of being able to work together.

Trust - Beyond acceptance, building trust is crucial in any relationship. We all know that there is an inherent sense of trust in our deepest of personal relationships. Trust is first seeded in transparency and later nurtured in sacrifice. Trust is as much about actions as about words. I believe leadership across both Business and IT organizations have to set the tone and follow through specific actions leading to a positive culture that eventually becomes an epidemic across Business and IT organizations.

Engagement - Finally, we all know even after trust is built, there is still hard work that is left. Last night, I watched Gillian Flyn’s movie, “Gone Girl”, reminding me of the fact that “marriage is always about hard work”. Similarly, there is a need for Business and IT to be constantly engaged and working hard at solving problems creatively. Its about trying new tools, technology and processes - maybe its helping enable business oriented self-service tools or Do It Yourself (DIY) technologies or Bring Your Own Device (BYOD), maybe its creating risk free sand box for businesses environments to play within or maybe its about practical governance versus impractical control.

While I have no silver bullet to solve this problem, at a minimum I believe there is a need for acceptance, trust and engagement when it comes to balancing business agility with IT governance. On this Valentines day, I am very hopeful that the winds of change with reignite the spark in this relationship.

I recently came across another thoughtful “LinkedIn Pulse” feed titled “The management narrative that makes room for Finance” by Jack Sweeney, author, blogger and pod-caster at “CFO Thought Leader”.

In his post Jack argues that when it comes to managing a business corporation, despite Peter Drucker (renowned 20th century management guru) changing his early thinking from a “command and control structure” to an “information based organization”, he (Drucker) had largely ignored the ability of Finance Organizations to influence and support that change.

Back in business school, I distinctly remember reading a chapter written by Drucker on "Controls, Control and Management", in which he said "Controls" is means and "Control" is the end. In my early 20s, and with little experience, I had “no clue” what that really meant. Now that I am older (not very much wiser), am starting to interpret, understand and reason Drucker’s thinking.

By "Controls and Control" Peter Drucker probably envisioned centralized management in a relatively stable environment where organizations could be managed through “command and control” structures. With that line of thinking, I can understand whereJack is coming from, when he says Peter Drucker probably never appreciated the strength of Finance in supporting a “collaborative and change” oriented management structure. If you want to know more on a “command and control” versus a “collaborate and change” management structure, I highly recommend watching this short 2 minute video by CFO Thought Leader. 

However, if I take a more liberal interpretation of “Controls” as “measurement of information” about the performance of an organization and “Control” as the “eventual end state” the businesses needs to get to, I see an “information based organization” view in the definition – something that more closely aligns to Peter Drucker’s changed thinking. It is about how to understand what has happened or is happening to a business and what should be done to move the organization to where its leadership wants to take it.

I completely agree with Jack on the need for “collaborative change management” and “finance being best positioned to effect that change”, and would even stretch it to say that eventually CFOs need to collaborate based on “information (and analytics around such information)” to help their business partners with decision that facilitates change. This fits a more contemporary interpretation and something that Peter Drucker would probably agree.

Unfortunately, compared to what we have today, the information foundation and associated processes / technology in the 50s (when Drucker wrote about this topic) were relatively immature. I understand why Drucker started where he started, but am sure he will change his thinking if he looked at today's intrepid Finance Leaders and their ability to influence change by collaborating with their business partner using information / analytics (also called Financial Analytics - a topic about which I write / speak regularly).

I believe “Financial Analytics” is the “Holy Grail” that will help CFOs and their Finance Organizations facilitate collaboration and change across the enterprise.

Thanks to Samuel Dergel (CFO thought leader -  speaker, executive coach and author of Guide to CFO Success) for sharing a recent article published by Rob O'Regan in Computerworld on the rise of the “Chief Analytics Officer” => http://www.computerworld.com/article/2688352/chief-analytics-officer-the-ultimate-big-data-job.html. 

I cannot agree more with the article i.e., the Chief Analytics Officer (CAO) role is not a flash in the pan, but is slowly / surely emerging and here to stay. I however disagree with CAO being categorized as the ultimate “Big Data” job. IMHO, inter-alia, Analytics is not merely about “Big Data” per my views below.  To start with, the issue is where will this CAO role eventually sit within an organization, which (again in my humble opinion) will make or break its eventual effectiveness and success. While it maybe tempting for me to start with Finance as a natural home (given my pre-disposition / affinity to Financial Analytics), let’s take a step back to see “where it should not be” and “what attributes may influence on eventually where it should be”. 

1. First of all as Jack Phillips, CEO of the International Institute for Analytics clearly nails it - CAO’s should not report to CIOs. CIOs largely have data infrastructure capabilities / responsibilities and usually lack the understanding of “how business decisions are made and actions are taken about product markets based upon insights from data”. Please read my earlier blog Never Put the Data (Cart) in front of the Business Problem (Horse) Being Solved if you want to understand my point of view on this topic.

2. CAOs are different from a variety of data related roles that are emerging. In my humble opinion, here are a few that the article alludes to

Chief Data Officer (CDO) is essentially the role within IT organizations that is struggling to bring business process / data thinking to the forefront relative to typical IT mindset of networks / computing hardware / managing packaged applications & tools / security / telecom / help desk etc. While not completely impossible, this transformative breed is rarely found inside IT departments. When it sometimes does exist, it is the form of business savvy Software Solution Architects such as Paddy Hannon, at Edmunds.com who rose up from within the IT organization due to deep business acumen. 

Chief Digital Officers (CDOs) are typically aligned functionally within marketing and deal with emerging digital footprints from new trends such as social / mobile data. These CDOs are particularly critical to consumer oriented businesses trying to harness social / mobile data in Customer Acquisition / Retention / Churn Management (typical examples include Telco Industry - Wireless Subscribers, Banking - Credit Cards) or Customer Experience (Retail - Store Merchandising and Operations) etc. The recent study from Ernst & Young reinforces this trend - 69% of companies say customer experience was vital to their growth strategies, but just 12% said they take full advantage of analytics to extract customer insights and deliver better customer service. I am sure Chris Mazzie, CAO at Ernst & Young knows this better than anyone else. 

3. CAO role has to move beyond an efficiency play of merely consolidating analytics groups across the enterprise (as the article puts it) to one that will eventually be effective in bringing business value through insight (from data using analytics) and actions (influencing change in organizational thinking / decision making / business processes). CAO is as much a translator, a mediator, a business professional, a business solutions architect, a software tools / technologist, a statistician, a infographic artist / story teller etc. It’s a very complex role with both hard and soft skills to be brought to the table. It’s also about embracing a culture of “data oriented questioning of business decisions” and building “creative smart” (a term coined by Eric Schmidt, Google’s Executive Chairman and Jonathan Rosenberg, Google’s former Products Head in their recent book “How Google Works”) teams that excel in solving critical business problems. 

4. Finally, IMHO, CAOs are most likely to emerge from within those functions within the enterprise that are strategically aligned and tactically capable of steering the organization with insights from data. Examples include

Companies with “customer intimacy business models” such as IBM will most likely have its analytics chief emerge from within marketing / sales.

Companies with “product superiority and excellence business models” such as Google or Apple will most likely have the CAO emerge from within products / marketing / design / engineering. 

Companies with “operational efficiency / effectiveness business models” such as a Walmart, FedEx will most likely have its analytics head from within Finance or Operations. This maybe also be true for industries such as Healthcare, Higher Education / Non Profit, Public Sector etc. where expense management and cost effective services delivery are overarching concerns. 

To sum it up, CAOs are a desperate need and companies looking to create / fill this role will have to strategically conceive and meticulously execute in their respective contexts. Ultimately, it is also about clearly defining “what it is should not be” as much as “what it should be”.

While the purpose of Business Analytics is to optimize outcomes, particular care should be taken in randomly applying analytical techniques while missing the very fundamentals of any business - “in serving it’s customers”.

As the sad day of terror ended in Sydney, a bad case of Uber’s analytical approach to pricing came to light - “algorithm based price surge”. Uber’s algorithm driven price surge started overcharging people fleeing the Central Business District (CBD) at Sydney following the terror attack => http://www.wsj.com/articles/BL-DGB-39428

While I am a strong proponent of analytic techniques such as pricing optimization (based on dynamic demand-supply), it is sad that Uber’s algorithm failed to recognize the rare event as an emergency and in the process “human spirit lost to algorithms” (albeit momentarily). While Uber claims its price surge algorithm did attract more drivers to a high traffic zone, my take is that the algorithm failed to interpret / differentiate the type of traffic pattern while the price increase kicked in. Basically, the algorithm treated the surge in traffic from people fleeing the CBD during the terror attack similar to how it would have treated crowd leaving any stadium after a cricket match.

While Uber did offer free rides as compensation, the damage to reputation was done. Such unintended consequences of analytical approaches can damage the very purpose and reputation of any business i.e., serving customers while building a brand.

In my opinion, there are three things to take away from Uber’s experience -

1. Pricing optimization is not always right.

2. Pricing algorithms should be smart enough to anticipate and tweak behavior / outcomes under special situations.

3. No business (irrespective of its smarts) can build a brand against human spirit and compassion.

I have always had this hypothesis, based on my empirical observations and discussions with senior thought leaders, that Finance Organizations have been slow adopters of advanced financial analytical techniques - e.g., statistical techniques for cash forecasts or data mining / regressions for bad debt predictions.  

Came across this blog on Proformative by Rob Kugel of Ventana Research that validates my hypothesis. In a survey conducted with Financial Analytics users, 58% (or almost 3 in 5) say that “significant or major changes are required” in their analytical capabilities / processes / technologies. 

Four main reasons seem to be driving this inertia - (i) too slow to implement; (ii) isn’t adaptable to change; (iii) there aren’t enough skilled people to do this work; and (iv) data used in it is inaccessible or too difficult to integrate.  Most organizations have rightly focused resources on the data challenges. However, as data access and integration has matured with Business Intelligence, Master Data Management and Data Warehouses platforms, the focus should essentially shift to -

hiring the right financial analytics geeks (typically FP&A users with data science skills),

equip them with nimble data discovery & visualization / OLAP / predictive analytics style financial analytical tooling such as Tableau / IBM Cognos TM1 / Oracle Essbase / SAS / R etc., that help in modeling and analysis (versus mere data access and integration); and

help them manage their own modeling environments, co-exiting in a flexible ecosystem that grows in an extendable and governed way. 

The survey clearly points to 71% of Financial Analytics users relying on spreadsheets, but 67% find that these very spreadsheets have been cause of the problem - neither finance analytics users have had an opportunity to upgrade their skills, use the right tools or build in a flexible modeling environment.    

If you have not already, please go to CFO Thought Leader when you have some time to spare for some good insights on what currently is on the minds of transformational CFOs. 

I had a chance to listen to Jeff Beckwith, CFO, Merchant Capital & Cash (MCC) - a small business consumer finance company that thrives on its analytical capability to differentiate in the small business financing marketplace.

MCC leverages information to not just monitor / manage collections but also price its assets. Ability to get daily information on assets with delayed payments not only helps in collections but also in better asset management (thru securitization). It uses analytical models to forecast liquidity positions and funding requirements.

Jeff leverages information at MCC with a primary goal is financial performance management across two key dimensions - increase revenue and decrease loss rates. MCC leverages information on assets and applies statistical regression models to look at loan losses on a static pool basis i.e., assets originated during a given period. Such losses are further broken down by month of origination, product type, underwriter, broker used to bring in the business etc. Further, analysis of trends in losses and delinquencies etc. are undertaken. All this analytics not only help in managing the business better but leads to a better bottom line management. 

Tesla CEO Elon Musk shares his rule about interviewing job seekers at his company.

http://www.businessinsider.com/elon-musk-job-interview-rule-2013-12

As hiring managers, we often come across resumes suited for positions we try to fill. However, this is just the starting point in a process to get to the right person. In dynamic product-market conditions, diverse skills and experiences are required for any person to be successful in his or her job. Therefore, as hiring managers, we have to stretch beyond conventional resumes to build an optimal candidate pool.

In my case, from within the candidate pool, I try to understand “how” each candidate approaches problem solving (at various levels) versus merely “what” problems he / she has solved”. 

In my opinion, ultimately anyone can claim anything on a resume (as long as they have the years / job breaks to back them). However, a sure shot at determining a person’s credibility / capability is to understand, at granular levels, how he / she solved problems.Bingo, the right versus wrong hire.