Running product reviews people actually look forward to

It’s a gloomy morning as I walk through a light drizzle under a gray sky on my way to work. As I enter the office and say hello to others on the team, they seem abnormally unnerved. Their mood reminds me of the knot I’ve had in my own stomach for the past 24 hours. The cause of all this anxiety? It’s product review day. Questions like the following reverberate through each team member’s head as they uncomfortably prepare for another journey into the unknown that is their next product review:

“When will this be shipped? Why is it taking so long?”

“Will the initial version of the product do <x> too?”

“I just spoke with <Customer X> yesterday and they want the product but need it to do <x>.”

“Will this product work for <non-target customer segment>?”

“What about <feature that’s out of scope for the initial version>? When will we be able to do that?”

Every product review, held on a regular basis with the founders, or sometimes the entire executive team, is different. One week the team is fending off tactical product ideas and solution suggestions which undermine their confidence in their ability to figure out the “how” for a particular customer problem. The next week, they’re wrestling with unpredictable questions which send the meeting careening down a path the team isn’t prepared to provide a confidence-inspiring answer. After each product review, the team feels less confidence in their work, less clear on the direction they should be pursuing, and less autonomy to drive alignment across the team on the contradictory directives and requests they’ve received. In the worst case scenario, the inconsistent “guidance” that comes out of these meetings causes the team to repeatedly change direction and ship something that fails to deliver value to customers. Over time, trust between the team and the founders and executives erodes. Unable to control what is debated in the meeting, the team optimizes for avoiding all debate and resorts to a “safe” agenda which consists of rehashing already circulated updates and uncontroversial discussion topics.

Every Product Manager, Designer, or Engineer has experienced product reviews that have gone something like what I’ve described above. Despite the oft-stated objectives of “empowering the team” and “increasing decision-making velocity,” most product reviews remain highly ineffective at achieving these goals. There are limited resources on this topic, many of which speak to less-frequent, strategy, priority, and roadmap reviews. Having seen this play out at a few different companies, including product teams I myself have led, I felt it would be worth offering my own perspective on how to run product reviews that support the development of a customer-centric, creative, and innovative product culture with empowered product teams that move quickly.

First Principles: What is a “product review” and why is it important?

There is no one single definition of a “product review” but in practice, I define it as being a regular meeting to discuss validated customer insights and the impact of those insights have on current and future product decisions. The product review is an opportunity to review how effectively a product is delivering customer value and to discuss open strategic questions to enable the product team to make better decisions and bigger bets. Even if a company calls this meeting something different than a “product review,” most have a mechanism that’s designed with this purpose in mind.

Product development is an inherently uncertain and risky undertaking. There is uncertainty around the customer need and product solutions that address it. There is often uncertainty around technical feasibility and execution to deliver a solution that works. Given this uncertainty, product reviews are important because they drive alignment across the team on key customer insights and the level confidence in those insights such that over time, the team has a clearer understanding of the customer and how they derive value from the product. Product reviews are analogous to pressing pause in the middle of a hard fought battle to step back as a team, look at the big picture, and make updates to your plans based on what you just learned in the field. The most effective product reviews serve the needs of the team to share and debate new insights and any changes in product direction that should flow from them.

Running an effective product review

When it comes to running a product review, team culture is equally important as getting the execution right. It starts with ensuring the environment in which you’re operating exhibits some baseline characteristics. Without these, attempts to implement effective product reviews, let alone an empowered product culture, are fighting an uphill battle.

• Trusted and empowered teams. Without trust and respect between the product teams and the founders and executives, nothing else matters.
• Psychological safety where people can be creative and debate facts and ideas objectively, which stand on their own merit (vs. the individual they’re offered by).
• A deep understanding of customers, who the target customer segment is, and their needs.
• Continuous discovery of what delivers customer delight. Teams should demonstrate deep curiosity and be willing to question even their most deeply held assumptions.

These cultural factors lay the foundation for effective execution. Well-run product reviews share a few key characteristics:

Limited meeting size

Product reviews with 15–20+ people in attendance are not conducive to deep discussion and debate. Limit product reviews to the core team plus a few key executive-level stakeholders. As a good rule of thumb: if someone is attending the meeting with the intent of just listening (vs. participating), they should be excluded. Those individuals can read the notes or slides that are circulated after the meeting to get a recap.

Set consistent context

The best way to control the focus of the meeting is to consistently and repeatedly remind the group of a few key items:

1. Why we’re doing this
2. Who we’re solving for
3. How we’re measuring success
4. Biggest risks we’re focused on
5. Input/help needed from the group

Laying out this context up front can help remind people that the product review is for the team’s benefit. If the team walks out of the room at the end of the meeting having not received helpful input on the items outlined, the meeting was a failure.

Focus agenda on customer insights

Focus the agenda on new customer insights since the last product review and the implications of those insights on the team’s understanding of what makes the product valuable and delightful, and how they can build on that value over time. While it’s tempting to include a status update or milestones slide, or have the design team demo prototypes, these can often derail the meeting and take time away from more important topics. Every minute spent talking about some specific interaction on a particular screen in the design is a minute not spent discussing what the team has learned from actual customers. In the context of explaining new customer insights, it might be helpful to show designs to help people better understand the feedback. Leading with the customer insights and using the designs as a tool to bring those to light can help keep the discussion grounded in customer insights and avoid derailing into an ad-hoc design review.

*While product reviews are a great place to surface critical decisions facing the team, they should not be a forum where decisions are made. Product reviews should focus on debating the options associated with key decisions and collecting input from others. The decision itself should be left to the team outside the product review. Once product reviews become decision-making meetings, others will demand that they are included in the meeting so they can participate in the decision-making process. This causes the attendee list to balloon. When you leave decision-making to the team outside of the product review meeting, you empower them follow an appropriate decision-making process so stakeholders who weren’t in the room are included. This allows the product review to remain focused on its primary purpose of discussing customer insights and the impact of those insights on product decisions.

Choose the right cadence

Keeping product reviews focused on customer insights is a great forcing function for setting the appropriate cadence of product reviews. If you’re in the early stages of product development and collecting new feedback weekly, a weekly review may work well. If you’re in the middle of shipping the next major feature and won’t have new insights until it’s in the hands of customers, dialing back the product review cadence makes sense.

What this means for Product Managers

It’s the Product Manager’s job to control the meeting. That means setting the appropriate context, ensuring the room is clear on what guidance and input the team is looking for from the product review, and then keeping the discussion on track. This means you may need to politely defer questions that are not in scope or redirect the conversation if it gets off track.

What this means for founders and executives

As a founder or executive participating in a product review, your job is to:

• Approach the customer insights objectively: Your opinions carry more weight with the team and modeling an objective approach will encourage the team to do the same. While you should not hesitate to share your opinions, ensuring you’re doing so in a way that demonstrates a willingness to be proven wrong by customer insights affords the team lattitude to disagree when they identify insights that contradict opinions you hold.
• Keep comments in scope: Manage your temptation to comment on things that are out of scope of the product review. It’s easy to let a question slip that threatens to derail the meeting. Demonstrate support for the team when they politely defer your comment to keep the meeting on track.
• Frame questions and comments to reinforce the autonomy of the team: When providing feedback, it’s helpful to ask if the team has learned anything from customers to support any personal observations or comments you make. This framing helps keep the discussion focused on validated insights from customers while positioning the team, the ones closest to customer, as the experts on the customer needs. It also creates an opening for the team to disagree with assertions you may make based on their own evidence from the field.

What success looks like

For the product team

They leave the meeting with more confidence, focus, and clarity on their direction forward, feeling full ownership over their ability to achieve the desired outcome.

For the founders and executive team

They leave the meeting feeling confident the team is being thorough when collecting customer insights and interpreting them thoughtfully to drive effective decisions on the product’s path forward.

Measuring Success

I’ve found a simple survey after each product review can help the team measure success and track progress over time. The survey consists of the following questions:

• After our last product review, on a scale from 1–5, rate how well we understand the needs of our customers (collectively as a team)
• On a scale from 1–5, how useful did YOU personally find our last product review?
• For non-core team members: On a scale from strongly disagree to strongly agree, indicate your level of agreement with the following based on the last product review: “I feel confident in the team’s decision-making and direction”
• For core team members: On a scale from strongly disagree to strongly agree, indicate your level of agreement with the following based on the last product review: “I feel we have more focus and full-team alignment on what we need to do to make this product valuable to customers”
• What team are you on? (this question is for segmenting purposes)

Above all, remember that conducting effective product reviews is an organizational muscle that must be developed. You won’t achieve perfection overnight and even when you feel like you’re in a groove, you’ll have one where you regress to old habits. The most important thing is to ensure the entire team is committed to assessing performance and continually improving so you’re eventually running product reviews that people actually look forward to.

— —

Have other strategies for running effective product reviews to create an empowered product culture? Send them my way at dslate47@gmail.com!

Thanks to Sarah Jacobson for her comments and input on this post!