A Synthetic User Can Sound Right— and Still Lead Your Product Wrong

What is a synthetic user in UX research?

A synthetic user is an AI-generated representation intended to respond like a particular person or user group. Some are built from a prompt or proto-persona. Others are informed by actual interviews, surveys or other research. A digital twin attempts to model an identifiable real participant from information about that person.

These are meaningfully different setups, as MeasuringU's taxonomy of synthetic users explains. But the common boundary is easy to miss: a generated answer is a model's output. It is not a newly observed account from a customer who has independently encountered your product.

Can synthetic users replace real UX research? Not as a general substitute. They can help teams prepare, propose hypotheses and, in particular settings, make predictions worth validating. Whether any particular prediction can support a product decision has to be demonstrated against relevant human evidence—not inferred from how convincing it sounds.

How an assumption returns wearing a research badge

Consider a hypothetical learning platform reviewing its course-pricing page. The team already believes that new learners find three plan options confusing.

It builds an AI persona around that belief and asks how the persona would choose a plan. The response mentions decision fatigue, unclear differences and wanting a simpler choice. Another synthetic interview reaches a similar conclusion. The report now contains a theme: learners are overwhelmed by pricing.

What changed?

The team has generated language around its assumption. It has not yet established that real learners experience this problem, how often it occurs, whether it affects purchase decisions or whether removing a plan would solve it.

GOODSCOOL JOURNAL / FIG. 02

Read full figure transcript

FIG. 02 ILLUSTRATIVE UI RECONSTRUCTION · NOT RESEARCH DATA How a hunch becomes a “finding” A hypothetical learning team puts its own assumption inside a synthetic persona—and gets that assumption back in polished language. Course plans / conceptual interface UI STUDY Pick the plan that fits your next step. Illustrative plan choices · no actual pricing shown 01 / EXPLORE Starter Foundations and guided practice Compare plan 02 / BUILD Studio Projects and feedback loops Compare plan 03 / GROW Complete Structured end-to-end learning Compare plan INPUT / TEAM ASSUMPTION “Three options confuse new learners.” Not yet tested with customers. ↓ OUTPUT / SYNTHETIC PERSONA “I feel overwhelmed choosing between them.” Generated from an assumption-containing prompt. ↓ UNJUSTIFIED PROMOTION “Our users find the pricing confusing.” No independent participant observation has occurred. Read the failure mode: the interface and dialogue are hypothetical. Repetition of an assumption by a simulation is not corroboration of that assumption.

The distinction matters because each evidence source answers a different question:

• Simulated dialogue can suggest what to investigate.
• Interviews with real people can reveal experiences, language, motivations and contradictions, within the limits of the study.
• Observed product use can show where people hesitate or fail at a task.
• Analytics and support records can reveal patterns in recorded behavior or complaints, but require checks for instrumentation, selection bias and context.

These sources can complement one another. They are not interchangeable. A handful of interviews cannot establish a population-wide percentage; a funnel drop-off cannot, on its own, explain the reason people leave; and a fluent synthetic answer cannot independently establish either fact.

This is why provenance—the origin of a claim—belongs in the main research readout, not in its footnotes.

A small score difference can become a very different decision

In September 2026, Lucas Plabst, Jeff Sauro and Jim Lewis published a MeasuringU experiment comparing human usability ratings with ratings predicted by AI digital twins.

Their starting point was a retrospective survey of 420 US participants about four AI chat products: ChatGPT, Claude, Gemini and Grok. The researchers generated two twins for each participant. One received a summarized persona; the other received that summary plus nearly all the participant's underlying survey responses. The real participants' System Usability Scale (SUS) answers were withheld from both twins.

The resulting average SUS score across the four products was 79.5 for humans, 83.9 for summary twins and 86.2 for full twins. The synthetic means were consistently higher. More participant information did not bring the full-twin means closer to the human means in this experiment.

The practical consequence becomes clearer at the product level. Grok received a human mean of 78.4 (B+) but a full-twin mean of 86.2 (A+) on the study's published SUS grading scale. These are observed study results, not illustrative numbers.

That is not a trivial difference in a planning meeting. One result says good, with reasons to investigate further. The other risks being read as excellent, little to worry about.

The researchers also found less variability among twin responses than among human responses. The twins were not merely a little optimistic on average; their responses were more tightly clustered. Their individual-level correlations with the human scores were moderate, not perfect (0.56 for summary twins; 0.59 for full twins).

There is an important limit to this evidence: the study examined one retrospective survey context involving four already well-rated products. It does not establish the accuracy of every synthetic-user method on every UX task. It does establish that seemingly respectable average agreement can conceal an interpretation-changing bias.

And the often-repeated description of these outputs as roughly “93% accurate” is not a general-purpose guarantee. In this case it is a way of expressing the mean difference on a 100-point SUS scale, not proof that 93% of individual people or future product decisions were predicted correctly.

What happens when you try to model an individual?

A different, broader experiment asks whether a twin actually tracks the person it represents.

In the 2026 Science Advances paper Digital twins are funhouse mirrors: Five systematic distortions, Tianyi Peng and colleagues evaluated twins across 19 preregistered studies and 164 outcomes. Each twin was informed by the corresponding person's answers to more than 500 prior questions.

Across those diverse outcomes, the reported average correlation between twin and human responses was 0.20—a weak individual-level relationship. The researchers identified distortions including insufficient individuation, stereotyping, representation bias and overly rationalized responses. A publicly accessible author-paper version provides the study abstract and methods context.

This work covers many attitudes and intentions, not a direct replacement study for every usability method. It would be a mistake to turn its correlation into a universal failure rate for all AI-assisted research. But it challenges an appealing shortcut: give the model enough information about someone and it will eventually become a reliable substitute for asking them.

More detail is not the same as demonstrated predictive validity.

None of this means AI-assisted predictions are inherently worthless. A 2025 Nielsen Norman Group review describes studies in which tailored simulations captured some population-level patterns or helped fill missing survey responses. Those are specific prediction tasks under specific conditions—not a license to turn an untested persona interview into a customer finding. The question is always which claim has been validated, for which people, against which real outcomes?

Rather than declaring synthetic users either revolutionary or useless, use three checks. This is a GoodScool editorial decision framework—not a scientifically validated scoring instrument.

1. Provenance: where did this claim come from?

Label the source where readers first encounter the conclusion: AI-generated simulation, human-reported experience, observed behavior, or instrumented product data. If a response was produced from a proto-persona, say what assumptions were used to construct it. If real participant data informed it, distinguish that source data from the model's newly generated answer.

A synthesized response may contain ideas worth investigating. It does not become an additional human participant just because it has a name and a transcript.

2. Validity: what has actually been checked?

If someone wants to use synthetic responses to estimate a real-user outcome, ask for a comparison with held-out human data on the same kind of task and audience. Check not just the average, but also error patterns, variation, subgroup performance and whether differences would change the interpretation.

Without that comparison, call the output a hypothesis. With it, state the exact context where the model performed acceptably. Do not generalize beyond the validation.

3. Consequence: what will we do if the answer is wrong?

Using a simulation to discover an awkward interview question is inexpensive to reverse. Removing a pricing option, deprioritizing accessibility work or changing a sensitive eligibility flow may not be.

The stronger and less reversible the commitment, the stronger the evidence required. That principle does not apply only to AI: rushed interviews, unrepresentative samples and misread dashboards also deserve scrutiny.

GOODSCOOL JOURNAL / FIG. 03

Read full figure transcript

FIELD TOOL 01 GoodScool EDITORIAL FRAMEWORK · ILLUSTRATIVE Give every research claim a passport. Before a claim enters a product decision, record its source, its limits and what would verify it. RESEARCH CLAIM / EXAMPLE New learners may confuse the differences between plans. HYPOTHESIS Origin AI-generated rehearsal Independent validation Not yet established Check next Observe appropriate learners choosing a plan Decision permission Improve the research plan; don't remove a plan yet Original GoodScool operating aid, not a validated research scale. The hypothetical claim has not been observed in actual learners.

↗ Revisit Fig. 01: three evidence permissions

Put synthetic users to work before the interview

There is a genuinely useful role for AI-generated participants, provided the team names it correctly: research rehearsal.

Give a model a clearly labeled hypothetical user context and test your interview guide. Ask it to challenge an ambiguous question, surface unfamiliar domain terms or produce contrasting possible reactions. Let the exercise expose what the team still needs to learn.

Then go to people who actually encounter the problem. If you're investigating plan selection, observe appropriate participants making that choice. Ask why a label confused them. Compare their behavior with recorded support questions and relevant analytics when available. Treat each method according to what it can—and cannot—show.

Nielsen Norman Group's evaluation of synthetic users similarly identifies preparation and hypothesis generation as useful applications while warning against replacing real-user findings with simulated ones.

A simple decision note can prevent a practice exercise from being promoted into evidence:

Claim to investigate: New learners might misunderstand the differences between plans.
Source: AI-generated rehearsal based on an unvalidated team assumption.
Evidence status: Hypothesis, not a finding.
Next research: Observe relevant learners choosing between plans; investigate misunderstandings and alternative explanations.
Decision today: Improve the research plan, not the pricing architecture.

GOODSCOOL JOURNAL / FIG. 04

Read full figure transcript

FIELD NOTE 02 RESEARCH INTERVIEW DESIGN · ILLUSTRATIVE Challenge the premise in the question. LEADING PROMPT “Which part of these plans confuses you?” Presupposes confusion; it can invite the answer you expect. OPEN TASK PROMPT “How would you choose the plan that meets your needs?” Observe the choice first; ask about hesitation and reasoning afterwards. A question rewrite is a starting technique, not a guarantee of unbiased interviewing. Test it in the context of the real research task.

Nothing in that note requires the simulation to be perfect. It only requires the team to be precise about what it learned.

The real danger is an unmarked promotion

Synthetic users will keep improving. Some future methods may prove reliable for particular predictions, audiences or research tasks. They should be evaluated on that evidence, not dismissed by category or accepted by default.

For product teams, the urgent problem is simpler. A generated answer can travel from a prompt to a slide, from a slide to a finding, and from a finding to a roadmap commitment—without ever crossing the threshold of independent customer evidence.

Use synthetic users to sharpen the questions. Let real-world evidence earn the answers.

That isn't resistance to AI. It's the difference between producing persuasive research artifacts and making responsible product decisions.