AI UX Research: Do we still need users?

Unless your product has a raving cult of fans, user research sucks: You’re dangling Amazon gift cards in front of unmotivated users to get a 5 question Google Form filled out (and half of those don’t submit). Booking an interview with a user can feel as hard as getting one at Apple. Even after a multimedia assault via email, LinkedIn, and carrier pigeon, you might end up with 27 responses that trickle in over 7 weeks.

And when you comb through the responses, most answers are obvious, some completely useless, and just a few deliver insights.

Doing your own user research feels masochistic. But so does the price tag on agencies promising to do it for you.

It feels like user research is just one of those hard things that are worth it. Until now.

Can AI save us product people from the yoke of hard, manual user research? Technology has simplified many a complex process. And now, it could transform how we do and interact with user research.

For this article, I used AI in two types of user research:

1. User interviews
2. Observing the target market

User research without the (human) users—just as good?

If you’ve played around with ChatGPT, you’ve probably instructed it to respond like a type of person. Maybe you told it to act as an editor for your writing. Maybe a career crisis led to it acting as a career coach. Maybe boredom inspired a prompt about responding with rap lyrics about washing machines (guilty).

Because it feasted on so many viewpoints, GPT can emulate multiple personalities, professions, and states of mind. While most of us used this for entertainment or useful-ish professional use cases, some thought deeper about this:

Could AI be a 24/7 available user panel, ready to give us feedback at the push of a button?

That seems to be what the startup Synthetic Users was wondering. Their answer: A resounding yes. And various research institutions back them up.

Their product simulates a broad spectrum of your target market: You can define broad demographics such as age, location and profession and drill down into specifics like disabilities and personality traits.

Their AI tool then creates AI personas matching your audience and product. How it does that seems to be a trade secret, but Synthetic Users describes their subjects as “composite creatures put together in the bowels of a neural network.”

After inputting the problems you solve and the solution you’ve built/want to build for it, Synthetic Users reports on what segments of the synthetic users think.

I thought this concept was fascinating. So I interviewed Hugo Alves, the co-founder and Chief Product Officer at Synthetic Users:

Do you think human user research will become redundant? If so, why?

Hugo (H): The short answer? I don't know. However, as AI models evolve in both capacity and complexity, they're inching ever closer to emulating not just human reasoning but also our emotions. These models are beginning to predict how specific demographics might respond to a given situation. The implications are profound. It's not merely about generating coherent text; it's about mirroring human faculties like rationality and sentiment.

Yet, one aspect that keeps me circumspect about AI entirely substituting human user research is our inherent physiological cues—whether we're cold, hungry, or experiencing any other physical sensations. These tactile experiences profoundly shape our responses, and it seems challenging for a model to encapsulate such nuances.

However, given the astonishing capabilities I've observed in recent AI developments, I wouldn't rule out the possibility of AI one day replacing humans in certain research areas. But for now, we're not there yet.

What would you say to a PM who thinks AI-generated research is just “fake”?

H: To the PM who labels AI-generated research as "fake," I'd urge them to reconsider. This isn't some whimsical computer trickery. AI models today, especially the large-scale ones, have processed and understood almost the entirety of the web's text. That's a vast expanse of human knowledge, emotion, intention, and behavior they've taken in.

When these models generate insights, they're essentially mirroring attitudes and behaviors of countless agents—people with real-world intentions, goals, and needs. It's not about fabricating emotions but emulating them based on a staggering amount of information. So, while it's not a direct human emotion, it's a highly informed reflection of it.

To label this profound capability as merely "fake" is to overlook the science, the data, and the potential behind it. What we have here isn't a replacement for human intuition but rather an unparalleled tool to simulate human reactions across a broad spectrum of scenarios. And in the hands of a forward-thinking PM, that's pure gold.

Will Synthetic User Research change how often/intensely product teams do user research?

H: Will Synthetic User Research reshape the frequency and intensity of user research within product teams? Undoubtedly. Let's consider today's landscape. A significant portion of products being developed and launched have little to no user research behind them. This isn't because product teams undervalue insights but because traditional user research is time-consuming, often expensive, and logistically intricate.

Introduce Synthetic User Research to this equation. Suddenly, user research isn't an episodic, massive undertaking but an on-tap resource. Imagine a world where product managers, designers, and developers can ping their target audience with questions 24/7, receiving insights almost instantaneously and at a fraction of the traditional cost. This constant feedback loop could become a natural part of the development cycle rather than a segmented phase.

So, yes, the advent of Synthetic User Research will likely boost the frequency of research because it drastically reduces barriers. Product teams can integrate these insights seamlessly into their workflows, making research an ongoing dialogue rather than a periodic consultation. The outcome? A profound uptick in the quality of products, as they're perpetually refined and realigned with user needs. That's the endgame every product team aims for, and Synthetic User Research might just be the tool that accelerates us toward that reality.

When should product people NOT use synthetic user research?

H: When shouldn't product teams turn to synthetic user research? First and foremost, for high-risk decisions. If there's significant uncertainty paired with potential severe consequences, relying solely on synthetic insights is precarious. As always, in such scenarios, triangulating data from diverse sources and methodologies is paramount.

Secondly, remember that while synthetic research can be incredibly insightful, it shouldn't operate in a vacuum. Human judgment, a cornerstone of traditional research, remains indispensable. It's essential that product people exercise discernment and supervision, ensuring that the insights gleaned align with broader context and human intuition. After all, technology augments our capabilities; it doesn't replace our judgment.

What else do you want to say that we didn’t ask you about?

H: We're really pushing to build a solid evaluation framework for our synthetic user research. Why? Because we genuinely want everyone to understand how these synthetic interviews stack up against the real-deal human ones. And the coolest part? We're designing this framework so it's not just for us. Anyone, even our competitors, can use it to check their products.

To me, this is about more than just our product—it's about giving the whole industry a trust boost. We're not just pulling this out of thin air; there's some real-deal academic research backing up our approach. It's not just about making a splash; it's about doing things right and setting a gold standard.

Yet, one aspect that keeps me circumspect about AI entirely substituting human user research is our inherent physiological cues—whether we're cold, hungry, or experiencing any other physical sensations. These tactile experiences profoundly shape our responses, and it seems challenging for a model to encapsulate such nuances.

However, given the astonishing capabilities I've observed in recent AI developments, I wouldn't rule out the possibility of AI one day replacing humans in certain research areas. But for now, we're not there yet.

If you want to play around with Synthetic Users, you can see a sample study here. Thank you to Hugo Alves for the interview. Shoutout to his Co-founder Kwame. And check out Synthetic Users here!

This was fascinating. Even if tools like this don’t replace human user research, I wonder what kind of solutions instantly available user research might lead to.

In technology, whenever something becomes way faster, easier or cheaper to access, it usually finds its way into

Something like this maybe? A Slack bot that automatically responds to this:

With this?

I’m excited to see how much easier this becomes and the use cases this enables.

AI: The future of user research?

Whether it’s writing, product management, or illustrations… whenever you try a new generative AI use case, you generally draw the same conclusion: The outputs don’t suck. They’re actually pretty good. But they’re also not outstanding.

Many dismiss AI use cases because of this: They bring up an example of excellent human work/insight and state, “AI can’t do this!”. That’s correct, but it’s wrong to then dismiss AI.

There are two fallacies here:

First, AI detractors anchor to excellence.

Most human work is good enough, not excellent. AI might not paint the next Guernica (excellence), but it may might generate our blog header images (good enough). This is especially true for user research. AI research might not give you the input that unlocks a market-dominating product (excellence). But it may well help you find objections, challenges and considerations for your next feature launch (good enough).

Second, AI detractors neglect operational considerations.

User research is messy, takes a long time and consumes resources. And sure, there’s agencies for user research—if you have $10k burning a hole in your pocket.

AI user research is cheap/free, good enough and instant. Combining AI tools with a free market research tool enhances the value by providing quick, comprehensive insights. Does that mean you don’t need to talk to human users anymore? No!

To get exceptional input, you’ll likely still need humans, the same way you still need a human for exceptional illustrations.

But whenever a technology gets far more accessible (because it’s faster or cheaper to access), new solutions blossom which make use of this.

But AI user research simplifies ideation and gives even tiny startups access to a broad spectrum of ideas. I believe AI plays a big role in the future of user research—the outputs are good, even when you manually prompt. And specialized tools like Synthetic Users are likely to improve outcomes further.