Finding value at the bookends
Improving the design of solutions by conducting foundational pre-design research and post-launch analyses.
The UX research field is experiencing a significant transformation as it becomes increasingly prevalent and expands beyond dedicated user researchers. Both user researchers and “people who do research” are engaging in more user interviews, surveys, and other forms of data collection, recognizing its value in informing product development decisions. This shift in the landscape of UX research brings diverse perspectives and expertise, driving collaboration and cross-functional understanding within organizations. It also allows for more straightforward data collection by product team members, freeing time for dedicated user researchers to handle more complex research.
At the same time, as budgets tighten and companies face increased pressure to demonstrate the impact of their investments, the demand for understanding how research affects the bottom line is becoming more critical. The challenges of resource constraints and financial accountability necessitate a deeper understanding of research's value to product development and overall business outcomes. It is no longer sufficient for research that is a nice-to-have; it must be recognized as an essential component in driving success.
To address these challenges and ensure research’s essential role, a collective shift in perception is required from all stakeholders involved in product development. Research must be positioned as a strategic driver of innovation, market differentiation, and customer satisfaction. Building research as an integral part of the product development process, organizations can tap into its potential to inform and guide decision-making, minimize risks, and drive positive business outcomes. I believe that earning that seat at the table begins with conducting research at the bookends of the product development cycle.
Creating value at the bookends
Recent conversations within the UX research community have shed light on a potential imbalance in research focus, with an overemphasis on the middle ground of product development. This emphasis often leads to spending significant time on fine-tuning solutions through interviews and usability studies. While this approach ensures functional and user-designed products, they may still not achieve product-market fit or create the desired impact for users. Therefore, there is growing recognition that strategically shifting applied research to the bookends of the product development process, both before the design phase and after the product’s launch, will generate insights that earn UX a seat at the strategy table.
By conducting research earlier in the process, teams can gain a deeper understanding of customers, their environments, and the context of their actions before generating solutions. This upfront knowledge created by foundational research allows for the development of solutions that address real user problems, align with user priorities, and have a higher likelihood of success. With a well-defined starting point, the speed of solution creation can be increased, and the need for rework is reduced as potential obstacles and constraints are identified and addressed earlier in the process.
Following through on the other side of product development, measuring the impact that research and design generate is another way to ensure research is creating business impact. Researchers are commonly pulled in many ways, and engagements conclude with a readout or a workshop to sign off on feature updates. Truly closing the loop means conducting post-deployment research that creates an understanding of a study's impact on business metrics (sales, costs, etc.) and engagement metrics (CSAT, usage, etc.). These measures are beneficial to quantify value and replicate the most successful projects.
Starting early: the value of foundational research
Foundational research conducted to understand the customer better before solutioning is pivotal in driving value creation at the early stages of product development. Leveraging this type of research, organizations can enhance roadmaps and solution design by defining customers, their environments, and the context of their actions. This deep understanding allows for creating solutions that effectively address real user problems, ensuring that the focus remains on developing solutions that truly matter to users.
Developing foundational knowledge about customers and the context they are acting within involves examining various aspects, including demographics, psychographics, behaviors, motivations, and goals. By gathering this information, organizations can gain insights into user needs and preferences, enabling them to prioritize the most critical user problems to solve. An upfront understanding allows for more targeted and effective solution design, reducing the likelihood of developing solutions that miss the mark. All this means foundational research expedites effective solution creation. Overall, by aligning solutions with well-defined customer requirements from the outset, teams can save time, effort, and resources by avoiding unnecessary iterations and rework.
“Foundational research expedites effective solution creation”
Ideally, this research will utilize behavioral and attitudinal data to understand users’ actions and perceptions comprehensively. Behavioral data provides insights into users' interactions with products or services, such as usage patterns, feature adoption rates, and engagement levels. This data-driven approach helps uncover user behavior trends and patterns, informing the design of solutions that cater to user preferences and usage habits.
Attitudinal data then adds a human dimension to foundational research. By combining product analytics with user sentiment, organizations can gain insights into why a user took a certain action witnessed in the data. This deeper understanding of user perspectives allows for identifying pain points, unmet needs, and opportunities for innovation. Collecting attitudinal data alongside behavioral data provides a well-rounded view of users, enriching decision-making and solution design with a more holistic understanding of their motivations and desires.
Common methods for collecting attitudinal data include in-product surveys, ethnographic methodologies, and qualitative interviews. These approaches allow researchers to gather direct insights from users, exploring their thoughts, preferences, and opinions about the environment a solution is being designed within and current attempts to rectify the user problems. By incorporating attitudinal data into foundational research, organizations can better empathize with users, ensuring that solutions align with their expectations and preferences. This human-centered approach leads to improved user experiences, increased user satisfaction, and a greater likelihood of achieving product-market fit.
At its core, foundational research conducted at the early stages of product development is essential for creating value and driving successful outcomes. By defining customers, their environments, and the context of their actions, organizations can develop solutions that address real user problems and align with user priorities. Leveraging behavioral and attitudinal data enables teams to understand users comprehensively, inform decision-making, and design products and services that resonate with their needs and preferences. By incorporating foundational research into the product development process, organizations can set the stage for success, saving time, effort, and resources while delivering user-centric solutions that are more likely to achieve their intended impact.
Assessing Feature Impact: The Power of Post-Deployment Research
Post-deployment research is crucial in understanding whether a feature has achieved its intended impact. Evaluating the effectiveness of deployed features to determine their success and inform future decision-making provides a significant service to product teams. By assessing feature impact, companies can identify which features work well and understand the factors contributing to their success. Likewise, for underperforming features, it is necessary to investigate why they are not meeting expectations and make informed decisions about fixing or decommissioning them. Calculating feature success enables continuous learning and improvement, ensuring product development focuses on creating value and delivering a positive user experience.
Measuring impact requires the collection and analysis of behavioral data. By examining user behavior patterns, engagement metrics, and other quantitative data, researchers can gain insights into how users interact with the features and assess their overall impact. Behavioral data alone provides valuable insights, which can be further enhanced by incorporating attitudinal data. Attitudinal data helps researchers understand the full picture by adding the user’s subjective experience associated with the feature. This deeper understanding provides a more comprehensive view of how users are affected by the feature, why they make certain decisions, and what specific behaviors they are changing.
“A statistical test can tell you that outcomes changes, only users can share what that meant to them.”
Quantitative research methods such as basic t-tests and Propensity Score Matching are commonly employed when assessing feature impact, given their ability to provide statistical significance. These methods help researchers determine whether the observed changes in user behavior are valid and can be attributed to the deployed feature. However, while these statistical tests explain what happened, they may not capture the full picture of user sentiment and the reasons behind behavioral changes.
This is why attitudinal data is important, providing valuable insights into user beliefs, motivations, and attitudes toward the feature. A statistical test can tell you that outcomes changes; only users can share what that meant to them. Complementary attitudinal data can be collected through various feedback methods but are most effective when positioned as in-the-flow data collection techniques such as in-product surveys. These approaches allow researchers to capture user feedback, opinions, and satisfaction levels, providing valuable qualitative insights to supplement the quantitative data.
Assessing feature impact through post-deployment research empowers companies to iterate and improve their products based on real-world user feedback. Employing behavioral and attitudinal data, businesses can identify successful features to replicate their positive impact and uncover insights from underperforming features to drive improvements. This iterative approach drives innovation, ensures product development efforts align with user needs, and ultimately leads to improved business outcomes.
Expanding your research efforts
Beginning new projects on the bookends of the development process begins with an audit of your current research efforts, plotting each project based on four stages: foundational, generative (solution creation), evaluative (solution evaluation), and post-development. If you find your projects centered on generative and evaluative, aim to expand outside these in the next quarter. With that intention in place, discovering your first “bookend” project can be challenging, but it should be if you follow one of these two paths.
The first path is the quickest way to a post-deployment study. Start by discovering a feature that is about to be launched. Ideally, it is something you have worked on or are familiar with. From there, work with the product team to determine how they would define a successful launch. Distill metrics to track those conversations, ideally describing user actions and how they impact their lives. With the research plan in place, ensure the metrics will be tracked and wait a few weeks after launch to begin the analysis.
The second path is the easiest way to charter a foundational study. Start by asking your stakeholders what high-level questions they have about customers. Ideally, you will gather 3 to 5 questions that customers could answer. From there, determine the themes among these research questions; for example, if a PM asks, “What are the primary distractions during meetings,” the theme is characteristic of a good meeting. With themes, you can build research questions focusing on the customer’s needs and environment to share with the team and prioritize. This approach leverages the foot-in-the-door technique as you can report findings to the team first to answer their specific questions, followed by relevant additional insights.
Regardless if you take one of these approaches or another, throughout the entire scoping of the research, be sure to think through how you will measure the impact of this research. For foundational, this can be done by measuring the time it takes to generate a solution and comparing it to what your team thinks is average. If that isn’t feasible, you can, at a minimum, point towards the features created in part of the findings. For post-deployment, on top of having solid insight on whether a feature is working, you can measure how many teams review the feature and the study to assist in developing their solutions. Or, if you have the capability, you can always survey your stakeholders to determine how they feel about the impact of the foundational or post-deployment survey and compare it to a generative or evaluative project.
Although likely apparent throughout this discussion, these studies can be a greater challenge because they require close collaboration with other teams. For foundational research to be impactful, you must have a working relationship with product, design, and possibly marketing. Post-deployment research will require access to data that product analytics and data science have and may even require engineering. If you have a strong relationship with one of these groups, that may take precedence when deciding where to create your next project.
Looking forward
As the field of applied research continues to evolve, there is a growing recognition that its focus should expand beyond refining solutions and also encompass ensuring that the right solutions are developed in the first place. This shift emphasizes the importance of research at the bookends of the product development process.
Businesses can make informed decisions that drive improved business outcomes by maximizing research value through a comprehensive approach encompassing the early stages of foundational research and post-deployment assessments. This approach ensures that research is not merely an afterthought but an essential component throughout the product development lifecycle.
By shifting the perception of research from a nice extra to an essential component, companies can harness the power of applied research to inform decision-making, mitigate risks, and create products that resonate with users. Embracing research at the bookends of product development ensures a more user-centric approach and improves the overall effectiveness and impact of products and services.
