Opening our minds to AI-moderated research
How to confidently find yourself in the new research method.
I was talking to a former colleague about AI moderation for research sessions — where an AI moderator conducts qualitative research with a human respondent.
Her reaction was equal parts disgusted and demoralized: “Uggggggh, crap.”
I recognize that reaction — it’s the unsettling feeling of AI threatening to take away something you do, something you love.
For qualitative researchers, this hits especially hard. We believe in our work, and we take pride in it. If you look inside any qualitative researcher, you’ll find a trophy cabinet of moments when they cracked the mysterious human code and uncovered an insight no one else saw.
AI moderation seems to devalue or jeopardize all of that. But I’d like to offer a different perspective.
Finding craft in the new method
I started experimenting with AI moderation about a year ago — at first, to prove it wrong. But eventually, I found myself building my own AI moderator, DeepNeed, to prove it right.
I’ve trained many practitioners over the years. I even taught qualitative methods at MIT for two semesters. My thinking was: how would we feel about this technology if it respected our “rules for the dance”?
And in doing so, something in me shifted. I moved from a place of self-protection and defensiveness to one of gratitude — for what this method affords us.
Why? Because I’ve seen it open doors: more opportunities for qual, more credibility, and more strategic impact.
Perhaps more importantly, I found that it still felt like our work. The craft was still there. And I still liked my part in it.
The methodological barriers
The first step in opening our minds is to address the three most common critiques of AI moderation:
- AI moderators can’t probe to uncover the deep “why.”
- AI moderators miss emotional subtext and environmental context.
- AI moderators can’t build rapport.
Critique 1: AI moderators can’t probe to uncover the deep “why”
There’s something almost magical about a seasoned interviewer, fluidly probing to uncover deeper motivations. When I was first learning, I remember watching veteran interviewers and thinking they had superpowers.
The general consensus is that AI can’t do this well.
As a recent Ipsos report put it:
“An AI moderator bot often behaves like a novice moderator that is constantly looking down at the discussion guide and, as a result, takes its eyes off the prize. It misses out on fertile opportunities to probe.”
Of all the critiques, this is the one I disagree with most.
If an AI moderator fails to probe effectively, it’s a design flaw — not a fundamental limitation of the technology. Like a human, AI has to be trained to move beyond the discussion guide and recognize rich moments worth exploring.
At DeepNeed, we use an agentic workflow, where a coding agent and an interview agent work together to determine where to probe. Here’s an example from a recent interview:
It won’t win a Pulitzer. But does it uncover the deeper why? Yes.
And more often than not, I find myself thinking: That’s exactly what I would have asked.
Critique 2: AI moderators miss emotional subtext and environmental context
The second critique is that AI moderators lack the ability to interpret body language and use subtle cues to probe deeper.
As Ipsos states:
“Seasoned moderators read between the lines — noticing hesitations, excitement, or discomfort and following up on those signals. Without those cues, there’s a risk the research team misses subtext.”
I agree that AI can miss these latent signals — but I don’t see it as a dealbreaker.
When n = 10, picking up on these cues is critical to maintaining data integrity. But when n = 400, individual moments of hesitation or nuance tend to average out, revealing broader patterns at scale.
On top of that, many AI moderation platforms, such as Listen Labs, already incorporate video functionality, allowing human researchers to analyze subtext. Technologies like Affectiva’s Emotion AI claim to detect complex emotional states, further bridging the gap.
The more significant limitation, in my view, is the lack of environmental context. Research interviews are often a blend of conversation and observation — something AI still struggles to replicate.
That said, the pandemic proved that in-person research isn’t always necessary to capture context. Platforms like Dscout and WatchMeThink have pioneered virtual observation, allowing researchers to gather rich, real-world data remotely.
And this is just the beginning. AI companies are actively developing vision functionality — Ethan Mollick’s demo of OpenAI’s Live Mode suggests that LLMs will grow their ability to analyze real-time video, giving AI moderators the ability to interpret context visually.
If you’re looking for a foolproof reason why AI moderation will never work, I wouldn’t bet on this one.
Critique 3: AI moderators can’t build rapport
The final argument is that AI can’t create the trust needed for respondents to fully open up.
As researchers, we take pride in our ability to mirror emotions, validate experiences, and create a safe space for honest reflection.
I don’t disagree. In fact, at DeepNeed, we’ve worked hard to design ways for AI to validate responses throughout an interview.
But I also question the premise.
We may be underestimating how intimidating or invasive human-led interviews can feel — especially when compensation is involved.
Early research suggests that respondents often prefer AI moderators precisely because they perceive AI as non-judgmental.
A London School of Economics study on French voters found that:
- 50% preferred an AI interviewer
- Only 15% preferred a human interviewer
- 35% were indifferent
According to the study, participants felt that:
“The AI is a non-judgmental entity… they could freely share their thoughts without fear of being judged.”
This aligns with established psychology research showing that people disclose more honest and sensitive information to computer-based agents when they believe no human is observing them.
The very thing we assume is AI’s weakness — its lack of human warmth — may actually be a strength when it comes to encouraging candid responses.
The value barrier
I don’t think the biggest barriers to AI moderation stem from these methodological critiques. The real issue is that we don’t yet fully grasp its value — and how could we? The technology is still in its infancy.
As practitioners, we often treat traditional qualitative research like a Michelin-star dining experience — deep, intentional, and crafted with care.
“Table 3? You should have seen how their faces lit up at the ceviche!”
At the other end of the spectrum, quantitative research is like Chipotle — quick, efficient, and mass-produced. It serves a purpose, but no one’s walking away talking about how life-changing their burrito was.
I believe the real value of AI moderation lies somewhere in between — perhaps like a local café or deli.
Like a local café, AI moderation serves a unique purpose and has a reason to exist. It’s not just about “fast, fast, fast!” — it’s about leveraging scale and speed to create a distinct method in its own right, not just a poor imitation of traditional approaches.
Speed provides strategic bandwidth
Qualitative research is incredibly resource-intensive. It’s easy to get so caught up in coding, debriefing, and insight formation that we lose sight of the core question we’re meant to answer.
Metaphorically, our stakeholders are asking for a fully cooked meal, but we deliver an organized report of perfectly chopped carrots.
On a recent project, we had just three weeks to deliver results before a board presentation — an impossibly tight turnaround for a traditional study. But with AI, we collected the data in two days, giving us ample time to think, refine, and iterate.
We ended up reworking the core framework/story three times before delivering it, ensuring a much better outcome.
Speed provides more opportunities for research
This one is perhaps the most obvious: when research becomes faster (and more affordable), it becomes more accessible.
Instead of being seen as a slow, resource-intensive process, stakeholders start to view qualitative research as a nimble tool they can deploy more frequently and more strategically.
We can try to force stakeholders to dine at our Michelin-star restaurant — but we need to realize they’re increasingly choosing to skip the restaurant altogether.
Aaron Cannon, co-founder of Outset.ai, makes this point by drawing a parallel to the cost of data storage. As the price of computer memory and storage plummeted over the last 50 years, it didn’t just make existing computing cheaper — it enabled entirely new innovations, like smartphones.
Similarly, as the cost and time investment of qualitative research decreases, it doesn’t just make research easier — it expands what’s possible.
Scale provides a fuller picture
There’s a common perception that AI moderation produces worse data — but that’s not necessarily true.
Scale doesn’t just accelerate research; it uncovers insights and opportunities we might have missed in a smaller sample or overlooked because they didn’t immediately resonate with us.
For instance, in a recent study, we conducted 400 interviews with patients. We captured 300 hours of audio, and we identified 3,580 deep customer needs. We clustered those needs into 29 aggregate categories, organized by frequency.
At DeepNeed, we use all this data to build a foundational mapping of the full customer needs landscape.
A single AI-moderated interview might not feel as nuanced. But across hundreds of interviews, the sheer scale builds a richer, more complete picture.
Scale provides credibility.
I don’t know about you, but I’m tired of fighting the sample size war.
Every company seems to have key stakeholders who don’t trust or believe in qualitative work. But something shifts when we can back up deep insights with quantitative-level scale.
There’s a big difference between presenting an insight that emerged from four conversations versus 165.
The sheer volume doesn’t just strengthen credibility — it also allows us to capture the nuances within broader patterns, giving us a richer, more defensible story.
The Last Barrier: Ourselves
The final challenge isn’t technological — it’s personal.
This work is deeply meaningful to us. We dedicate our creative talents to understanding others, people entirely different from ourselves.
Research isn’t just a job; it shapes our identities. We dedicate our creative talents to understanding others — people entirely different from us — and their stories stay with us.
Something clicked when I did my first AI-moderated study. I found myself:
✅ Excitedly digging into what people said, hunting for golden nugget
✅ Noticing interesting word choices that summed up key insights
✅ Building a story I knew would help debunk the team’s biggest myths
And I realized: this is still the work I love.
AI moderation isn’t just a tool; it’s a new method. And like any method, it still needs human expertise at the helm.
The key is to design our place within these new models, with confidence. So I encourage you: try it. Experience the value firsthand. And confidently find yourself in the new method.
References
- Ipsos. (2024). “AI Moderation in Qualitative Research: Opportunities and Limitations.” [Referenced for critique of AI moderation capabilities]
- Ratcliff, M. (2023). “The State of AI in Qualitative Research.” Murmur Research. [Referenced regarding AI qualitative research being “open-ended quant”]
- London School of Economics. (2023). “AI vs. Human Interviewers: Respondent Preferences in Political Research.” [Study finding 50% of respondents preferred AI interviews]
- Mollick, E. (2024). “Demonstration of OpenAI’s Live Mode.” [Referenced regarding multimodal vision capabilities]
