Product manager's dilemma: data or intuition?
Do I follow my gut? Or the data?
Every product manager (and entrepreneur) seems to struggle with this question. And unfortunately, there is no hard-and-fast ârightâ answer about how to best design new products. As a result, product managers often take an âit dependsâ approach, varying the balance between intuition and data situation-by-situation.
This raises more fundamental questions about product development. When is data more reliable than intuition? How does a data- or intuition-driven culture perform in the long term? What approach leads to the most âsuccessfulâ, âdelightfulâ products?
Notwithstanding any empirical data (yes, a proverbial sin in HBSâs âDigital Innovation & Transformationâ class), itâs instructive to consider a few examples of data- and intuition- driven product development.
Letâs take Google to start.
So the story goes, Marissa Mayer famously tested over 40 shades of blue to maximize click-through-rates. This is but one example of the rigorously analytical culture Google is known for, which is hardly surprising given its engineering-centric ethos. It also seems to fit with their product (at least search) where small percentage increases in key metrics drive millions of dollars in return.
Apple on the other hand was guided by Steve Jobâs mantra that âItâs really hard to design products by focus groups⌠a lot of times, people donât know what they want until you show it to them.â  This mantra was underscored by a perspective that Apple products are designed for what Apple employees wanted for themselves. This also fits with their products: ones where it seems hard for consumers to truly articulate their needs, wants and preferences.
However, both approaches have their downfalls. Google has been much maligned for having poorly designed, emotionless user experiences contrived from a âby the numbersâ approach. Whereas Apple has missed opportunities as a result of its wall-garden approach and has struggled to introduce a meaningful innovation since the iPadâs launch in 2010.
Neither of these examples solves our early questions, though. Thatâs why I personally keep coming back to: âit dependsâ.
It depends what youâre trying to accomplish. It depends what types of products youâre building. It depends what fits with your existing culture.
Google highlighted an approach that can be wildly successful for incremental product improvements, especially for web-based products that can be rapidly tested in real-time. Apple highlighted an approach that can be wildly successful for transformative, new-to-the-world products, especially where hardware and industrial design are critical elements.
Really, the major takeaway is that a balanced âdata vs. intuitionâ product development culture should be built within your startup, company or organization.
In many ways, the New Zealand yachting team in the 1995 Americaâs Cup struck the perfect balance. Their engineering team used complex data models to inform design decisions, but ultimately deferred to the âcrewâ on how yacht configurations âfeltâ in the water. If the crew didnât like a configuration, the engineering team never even looked at the data.
With all of that, I think there are three additional takeaways:
Always start product design from genuine insight: whether from observation or feedback
Create a product feedback-loop that integrates data, intuition and clear metrics
Make the tough call: itâs what the product manager is paid to do!
⌠and always remember, you canât A/B test your way to a great product!