> ## Documentation Index
> Fetch the complete documentation index at: https://runrehearsals.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Interpreting Results

> How to read replica results honestly, and what to check when a simulation surprises you

## Look at the Spread of Responses

A cohort's value is in how the responses split across people. Say 30% of replicas react one way and
60% another: that split is the finding. Reporting only an average would blur those two groups into
one middling number and hide the disagreement, which is the real range of opinion you are trying to
see. You read this split in the project report once a Rehearsal finishes (see
[Running & Monitoring a Rehearsal](/docs/platform/running-a-rehearsal)).

<Note>
  Stated intent from a cohort is directional, not definitive. "72% would consider buying" is best
  read as which option wins and by roughly how much. It is a way to rank and compare, not a literal
  forecast of real-world sales.
</Note>

## When a Simulation Surprises You

An unexpected result is not automatically a wrong result. Before you discount it, work through this
checklist in order.

<Steps>
  <Step title="Did you interview the right people?" icon="filter">
    Check the screening. A surprising number often means the cohort included people the study is not
    really about. Predicting churn is meaningless if half the cohort never subscribed.
  </Step>

  <Step title="Was the question leading or biased?" icon="scale-unbalanced">
    Re-read every question as if you were trying to prove the opposite. Loaded framing, an anchored
    price, or a presumed behavior can manufacture a result. Try the
    [rephrase check](/docs/methodology/asking-good-questions#asking-the-same-thing-more-than-once).
  </Step>

  <Step title="Is it a stated-versus-observed gap?" icon="eye">
    People (and replicas) overstate intent on discretionary things ("I would cancel over \$3") and
    understate inertia. If the surprise runs in that direction, the result may be more honest than
    your expectation.
  </Step>

  <Step title="Does the reasoning hold up?" icon="comments">
    Read the transcripts. If a counterintuitive result is backed by consistent, grounded reasoning
    across replicas, it is likely real and worth understanding rather than dismissing.
  </Step>
</Steps>

## A Unanimous No Is Also a Signal

When a full cohort rejects something, that is information. In one validation, every replica
declined a status-priced product that later underperformed in the real market. A 0% result from a
representative cohort is not a bug. Run before launch, it is an early warning you would otherwise
pay for in ad spend and opportunity cost.

## Handling Results With Care

Sometimes a simulation contradicts a strongly held expectation, such as a creative someone loves or
a price someone is sure about. Approach these the way a good researcher would.

<CardGroup cols={2}>
  <Card title="Suspect the question first" icon="pen-to-square">
    More often than not, a jarring result traces back to how the question was framed rather than the
    replicas being wrong. Fix the framing and rerun before concluding anything.
  </Card>

  <Card title="Back it with the transcripts" icon="quote-left">
    When a result is surprising, read the actual reasons replicas gave in the transcripts, and
    compare against a real-world benchmark if you have one. The number is far more convincing once
    you can see the thinking behind it.
  </Card>
</CardGroup>

<Info>
  Creatives are genuinely hard to judge. Even experienced teams often cannot articulate why an ad
  fails. In the Mera NYC case study, the founder had spent weeks and real budget on a creative she
  was convinced would win. Replicas flagged it as the weaker option, matching the real ROAS
  outcome. The lesson is not to trust the machine blindly. It is that a grounded replica panel is a
  cheap check before you commit real money.
</Info>

## What to Do Next

* Rerun with neutral framing if the question checklist surfaced anything.
* Increase the cohort size for divisive topics where you need the full spread.
* Split by intent to see whether the result depends on the mindset you are testing.
* Read the transcripts. The reasoning behind the spread is usually the actual insight.

<Card title="Back to Overview" icon="house" href="/docs/methodology/overview" horizontal>
  Revisit how replicas differ from personas and a general AI.
</Card>
