Product Analytics Guide: Four questions that unlock disruptive innovation

Use data to build disruptive products. Say NO to low-value features.

Photo by NASA on Unsplash

Beyond AB tests, product analytics helps pinpoint the most valuable features to build before we sink any time into it. It improves our odds of finding and tackling problems that will make a big impact.

This guide outlines four analyses to run at different stages of design: From problem discovery to solution deployment. So product, engineering, and analytics teams are data-aware and focus on building high value features.

The four questions guiding our analyses are:

Key questions during problem definition and solution design

1. At first, “is the problem important?”: The goal is to compare the potential upside of fixing a problem relative to other problems. For example, what’s the $ dollars opportunity between building a new recommendation engine vs. implementing a new search engine?
2. “What’s the root cause?” during Discovery and Define: Use data to validate if and why we believe a problem is happening.
3. “What’s the best idea?” during Development: Cost/benefit analyses help us compare the upside potential and resource needs of different solutions.
4. “Is the solution working?” as we Deliver: AB tests and other counterfactual analyses help assess if the changes are indeed meeting expectation.

Let’s dive deeper now that we have an analysis plan.

1. Is the problem important?

A data-aware designer will first clarify the context behind a problem by asking four questions:

Key questions to help gather context and assumptions behind a problem

These details allow us to start Exploratory and Valuation analyses to see if a problem is worth solving.

Exploratory analysis helps validate assumptions by asking:

• What are the key metrics we want to improve and measure (e.g. checkout…