UX Longitudinal Studies

Photo by Ruediger Theiselmann on Unsplash

Continuos user research has become default in many UX teams. We have the opportunity to test and learn about our clients in every observation and/or interaction we conduct and participate in. In a time where Agile teams are searching for the next “lean research technique” we can tend to loose sight of the opportunities we might be missing with methods that don’t necessarily fit in our sprint timeframe.

An interesting method to consider in UX research toolbox is a longitudinal study, where researchers conduct several observations of the same subjects over a period of time, sometimes lasting years.

“The benefit of a longitudinal study is that researchers are able to detect development or changes in the characteristics of the target population at both the group and the individual level. The key here is that longitudinal studies extend beyond a single moment in time. as a result, they can establish sequences of events.”

This is not new to the scientific community. Longitudinal studies in psychology, genetics, biology, and even citizen science projects (lasting over 116 years!) have been used for almost 200 years.

Understanding how people develop habits and acquire new/different needs over time is music to UX designers’ ears. Longitudinal studies can give you insight for a long term relationship with clients.

Tips for beginners:

• Start small: longitudinal studies usually have a considerable amount of investment (time, cost and risk). Therefor pre-test the methodology by working with a smaller panel in a limited time frame (think lean!). This will allow you to learn more about gaps, opportunities, and reduce risk.
• Carefully choose your panelists: we are used to recruiting people who “allegedly” fit the profile we need, but didn’t quite hit the mark (maybe they weren’t 100% honest during the recruitment process?). In a study where you can substitute these people, this is just business as usual. However, in a longitudinal study your sample is sacred. The definition of the methodology is to study the same subjects and observation techniques over a period of time. Thus the panelists need to be highly engaged in order to keep the study going. FYI: it is absolutely normal to consider a margin of loss in the panel (participants can give up, move to a different country, become ill, cancel the service/product the study was about, etc). It might be a good idea to check the profile regularly (depending on how long your study will go).
• The metric: the methodology is very appealing, I know. Avoid using this as a vanity metric (I’m guilty as charged on this one). Knowledge gaps in customer journeys’ and business goals (retention, adoption, engagement, etc) are a good start to figuring out how longitudinal studies fit your teams’ needs. You can try using the CE11, Google Heart framework or even the Pyramid of Value by Bain (ex: does time influence on clients values towards your business? do they acquire more value? do they acquire more emotional and less functional values?) as your longitudinal metrics. It all depends on the overall goal.
• Themes: here are a couple of themes that can help spark insights:

Discovery studies: Understanding user needs and motivations.

First-time user experience studies: How first-time experiences impact adoption and retention.

Omnichannel studies: Understanding the intersection between the physical and digital worlds.

Consider going qual

Sure, a lot of what you discover using anyone of these metrics can be acquired through analytics. But what analytics won’t give you are the WHYS. So be sure to add in-depth interviews throughout the study to give more color to your data.

This will be an opportunity to explore participants’ perceptions, especially if you are attempting to detect interaction issues, which usually people have a hard time pinpointing what the enjoyed or what bothered them during an experience.

A team of believers

At times longitudinal studies will seem somewhat un-interesting, especially if you don’t see any shift in metrics over a long period of time. Keep your core team weekly engaged by brainstorming on hypothesis to explore during the next months:

• Did something happen locally that had an effect on the panels results (elections, holiday season, natural disasters, etc)?
• Is the economy taking a toll on the panels’ behavior (unemployment, exchange rate)?
• Are there any features in your product/service the panel will have access to in a near future (mesure the impact post implementation)?
• Are the results different among profiles (generation, gender, monthly/year income, etc)?
• Could these results be used for building new personas?

Longitudinal Studies are quite a challenge, but very rewarding. I would love to hear from you if you any experience with this methodology within the UX field!