The simple path to insight

Three steps to reliably generate insights

Many problem solvers — design researchers, consultants, especially early in their careers find it hard to generate insights reliably. It’s not that they don’t know how to do it: they have seen and created insights before. The fear can sometimes be of being unable to get to insights reliably.

This is because we tend to think of insights as sudden realizations that happen magically when we think really hard about a problem.

This is not the case: there are reliable ways to generate insights. I will illustrate below how, by thinking about a problem in spatial dimensions, we can easily picture where insights lie. Armed with this as a tool, a problem solver can approach any situation with more confidence they can create value.

What are insights?

A quick note before we dive in: what is an insight?

An insight is an “aha!” moment. It often occurs when you suddenly view a problem from a new perspective, allowing you to make connections or identify patterns that were previously unnoticed.

I like to say that insight is simply something interesting — but I have a precise definition of interesting.

Every problem has a knowledge baseline, what is known at the start. There are also often preconceived ideas on how new knowledge can be created or where the solution space will be, what I call the project’s common sense. Something interesting is something that either extends our knowledge baseline, or challenges common sense. If you are a consultant, a good insight is 80% of what you get paid for.

Insights are a problem solver’s bread and butter, but in the business world, they are still underrated. A few years ago, a McKinsey survey found that “only 35% of 2,135 global executives believed their strategies rested on unique and powerful insights”, and “only 14 percent of surveyed executives placed novel insights among the top three strategic influencers of financial performance”. The likely explanation is that executives look around and assume that “everyone is looking at the same data, we know what everyone knows”. This suggests that insights in the business arena, far from being commonplace, are still an underrated opportunity.

The problem-solving cube

Let’s visualize our problem as a cube. On the vertical dimensions, we see the core elements of problem-solving: frameworks, models, variables and data. I discuss these in depth in my Modeling in Problem Solving series of articles, but the gist is the following:

• Models are at the core of problem solving. A model is a simplified representation of an observed or imagined reality driven by a goal — in our case, solving the problem at hand. Any model is composed of an abstract, a-priori part — the framework, and an empirical part, data, organized into variables.
• Frameworks, also known as mental models, are the result of our previously accrued knowledge about the world. They are generic templates that help us thinking about a class of problems — eg. breakdowns, curves, matrixes, etc. Examples include the BCG matrix, the normal distribution curve, and the service design blueprint.
• Variables represent selected elements of reality in our model. In the case of the BCG matrix, for example, the variables are market growth and relative market share.
• Data, which in this context can be quantitative or qualitative, is the closest part of the cube to the actual, physical world. They are the values that our variables take in the specific instance under observation.

To clarify with an example, I can choose to model a target population using the framework of personas. The variables required include demographics, behaviors, etc., and the data I plug in will be the result of my actual observation of real-world people. This whole ensemble constitutes a model of that particular population.

As explained in the series of articles mentioned above, the core of problem-solving is the iterative loop of applying frameworks to data and reinterpreting frameworks in light of new data. We can visualize our work as problem solvers as starting by applying a framework to a problem, collecting the data necessary to validate or falsify it, adjust the framework to the new evidence, and so on, until we converge on a model that is both accurate enough and novel enough to yield a good solution.

It goes without saying that developing insights is a core part of this process.

On the z axis of the cube we have a simple representation of domains. Our core domain, loosely defined, is the area, vertical, sector or subject within which we are operating. For example, in developing solutions to help a bank retain its clients, we can define our domain as financial services, digital products for financial services or loyalty-enhancing solutions.

I like to imagine this side of the cube to actually be infinite: all possible knowledge lies on some continuum, and our skills as creative problem solvers is based in large part on our ability to move seamlessly on this spectrum.

So now that we have visualized the problem-solving space, let’s see how we develop insights.

The three steps to insight

To generate these captivating insights, we can follow a three-step process:

Preparatory step: understanding the knowledge baseline

The first step towards generating valuable insights is to thoroughly assess the existing knowledge of a project. This involves identifying assumptions, uncovering hidden biases, and understanding the relationships between various data points. For example, when conducting market research for a new product, we must evaluate the target audience’s preferences, industry trends, and competitors’ offerings. This initial assessment forms the foundation upon which we can build our insights and uncover areas for innovation.

Importantly, for consultants this means getting a clear understanding of what your client knows. In most cases you won’t be able to accumulate in the timespan of a project the knowledge your client has built up in years.

That’s fine and normally, expected.

It is, however, important to understand:

• What are the client’s known and unknown unknowns. Known unknowns are typically the ones they are paying you for; real value though often lies in uncovering unkown unknowns. The best way to do this is by reframing the problem statement — a whole interesting subject which I’ve written about here.
• What is project common sense — where does the client expect that solutions will come from. Delivering a good solution will often involve challenging such common sense — but not so much as to cause outright rejection.

Step 2: Analysis

With a comprehensive understanding of the baseline knowledge, we can delve deeper into the data through three primary avenues:

1.Exploring data: zooming in

Getting to insights — approach 1: diving into the data

This is the most obvious approach, but it’s important. It involves collecting new information or examining existing datasets more closely. For example, company looking to improve customer satisfaction might conduct surveys and interviews to gather additional feedback or analyze existing customer support logs to identify recurring issues. Consultants will routinely start a project by “crunching some data” in their dataset, or doing exploratory data analysis. The type of things we’ll be looking for change from project t project, but some of the key questions to ask could be:

• What are average values in our dataset?
• How widely spread is our data?
• Why are there outliers? What do edge cases have in common?
• What are the trends in a time series? What do they depend on?
• Is there any sign of Simpson’s paradox?

In our cubic project space, we are looking at a vertical downward movement: unpacking data to uncover insights.

2. Applying different frameworks: Moving up and down

Getting to insights — approach 2: using different frameworks

A potential game changer is viewing data through different lenses, such as taxonomies, explanatory models, or predictive tools, can help us gain fresh insights. In the context of the retail project, a taxonomy could be used to categorize customer complaints by type, an explanatory model might illustrate the relationship between staff performance and customer satisfaction, and a predictive tool could forecast the impact of proposed solutions on future customer satisfaction levels. In a manufacturing setting, a consultant might apply a “lean manufacturing” framework to identify areas of waste and inefficiency; but also, they could think of getting rid of waste completely through circular manufacturing.
Here they key is to challenge the project’s common sense by applying frameworks that are unexpected.

In our cube, here we are looking at a vertical up-and-down movement: we have a dataset, which is by default, sliced according to an existing framework. We move up two levels and ask ourselves which other frameworks could make sense of our data.

Naturally, to facilitate this step, it helps to know a lot of frameworks. This is something that can is learned in time, but there are cheats. Check for example this collection of 500+ frameworks I put together.

3. Drawing parallels: Moving sideways

Getting to insights — approach 3: finding similarities with different domains.

Uncovering connections between seemingly unrelated contexts or sectors can also yield valuable insights. For example, the retail team could examine customer satisfaction strategies employed by successful companies in other industries, such as hospitality or e-commerce, and adapt these strategies to suit the retail environment. A healthcare provider aiming to improve patient experience could draw inspiration from the hospitality industry’s focus on customer service and adapt those principles to their own context.

Here we need to thread carefully: depending on the client, we could be at risk of confusing them. But here, we also have the highest potential for innovation.

Interestingly, McKinsey at some points, came up with a set of standardized “flexons”, lenses to look at problems through frameworks from different domains.

On our cube, we are starting from our framework and moving sideways into extended domains of the problem.

Step 3: Insights elaboration

So let’s say that we apply our analysis methods one by one. How do we know when we are onto something? The simple answer is, when we say “aha!” (or when we think our client will say so).
It is essential that the insight be:

• Non-obvious
• Generalizable — besides rare exceptions, the insight needs to be telling me something beyond itself
• Actionable — I need to be able to do something about it

So when fleshing out an insight, I need to make sure I’m adding a “so what” — why is this important and why it affects the project.

Here are a few additional tips on where to look for insights within the problem-solving space:

• Look for edge cases. Averages are typically well-known. Outliers and edge cases, on the other hand, are often surprising. The big question of course is how important they are as a signal.
• Look for contrast. Many macro-trends that are well-known contain counter-trends, that are sometimes the result of some maverick doing things differently, but that can also be an early sign of a pendulum swing.
• Look for anecdotes. While statistical relevance is essential, even anecdotal or non-generalizable findings can offer valuable insights. For instance, an innovative solution adopted by a single company in a different industry might inspire a groundbreaking new approach in another field. The key is to remember that innovation often stems from what is not average or mainstream and that providing compelling examples to support our insights is crucial for their effectiveness.

Insights generation as a teachable skill

The ability to generate insights is indeed a skill that can be cultivated by anyone with the right approach. By systematically evaluating the knowledge baseline, analyzing data from various perspectives, working on frameworks and elaborating on tensions and gaps, we can unlock the potential for innovation and empower individuals to contribute meaningfully to their fields. With this framework in hand, we can more confidently embark on innovation projects in full knowledge that insights are somewhere out there, and they will come out.

Interested in learning more about the spatial approach to problem solving? Check out my Modeling in Problem Solving series!

An edited version of this article appeared on CEO Magazine and on Designit’s blog