#epistemy

A while back I had written a post "Decision Making Is a Science" in which I urged people to stop making pseudo-scientific decisions based on instinct. This is an important companion to that post. Lately, it's become even more important to make rapid decisions in the absence of applicable data, without making a lot of mistakes. I’m thankful to several people at HP that trained me well to deal with ever-new situations, embracing an experience and intuition driven, action-oriented approach to complex, emergent situations.

When dealing with national security, cyber security, massive scale technology operations, civil unrest, invasions, biological weapons, viral outbreaks, early stage technology investing and other areas where "new" is the norm, it is important to very quickly pick a decisive course of action. Waiting and watching, gathering more data, and episteme will result in more damage, which then requires stronger than initially necessary medicine, which in turn usually bears its own vile consequences. [Two quick examples: a proposal to solve the pre-cyber Monday infrastructure issue by introducing a new, untested piece of hardware in all our datacenters, that would have tremendous expense and potentially other catastrophic short and long term consequences. Second: arming the rebels in Iraq that have now metastasized into ISIS]

Use your instinct, intuition and cross-domain knowledge to rapidly find and apply a course of action. Here’s the tricky part: Don’t be wedded to your initial view as the situation develops. Don’t suffer from confirmation bias, where you interpret each new piece of evidence according to your fancy. Worse, if new data diverges from your initial hypothesis, don’t keep waiting for the next set of data points to arrive and prove you right, prepare a new response, and begin executing its lighter components. Course correction does not mean wildly veering from one solution to another. It’s just an exhortation to recognize data correctly as it streams in. Think of a counter that you set initially to a certain value, say 1,000 or 10,000 and each incoming data point is a +1, -1, +10, -10, +100, or -100. Don’t be afraid to adjust or even reverse course when it becomes necessary.

Many brilliant people let their egos get in the way when it comes to discarding their strongly held views. Often, they are afraid to contradict themselves publicly. This is a common reason why management fails in modern corporations. Instead of a continuous stream of experiments and business adjustments there are grand plans and grander prognostications, with great shame and loss of prestige attached to correcting course. Sure, employees dislike whiplash and stress from course correcting all the time, but an agile organization should be able to absorb a fair amount of change, provided it is complemented with trust and great communication, instead of the autocratic style found at most large companies.

The brightest people I know have strong opinions, weakly held. This concept was cemented in my mind at an Alliance of CEO’s meeting where Bob Johansen, Distinguished Fellow at the Institute for the Future quoted work from Paul Satto. Then, there is the classic story of how George Soros changes his mind about things when he gets new data.

There are situations where continuous application of force at a point of attack yields results, e.g. in a military campaign or a user acquisition campaign. But even within those umbrella campaigns are a series of micro or large adjustments based on incoming data. Big data makes the data gathering, and information extraction process more powerful, allowing even more strongly opinions, weakly held.