Testing a campaign on customers who don’t exist.

Our founder also runs a fashion marketplace. We built its Agentic OS the way we build one for any client, inside its own CRM: 90 agents in eight departments, each team checked by a judge. There, a campaign to the whole base is the kind of action we keep with a person. Once it is out, it is out. A tone that lands badly or an offer that confuses reaches everyone at once, and the unsubscribes do not come back.

So before a send, we test it on customers who don’t exist. These are notes on how the synthetic customers are built, what they are good for, and where we have learned not to trust them.

Why simulate a send

A/B tests are the honest way to compare messages, but they test on real people. Half your audience gets the weaker version, and you learn after the fact. For a weekly campaign in four languages, across email, SMS and push, there are more variants than there is audience to test them on.

A simulation is a cheap first filter. It does not replace the real result. It helps decide which variants deserve to reach real people at all, and it catches the obvious mistakes before anyone sees them.

How a synthetic customer is built

We do not invent personas from a marketing brief. Each one is built from real buyers.

1. Split the base into strata. Groups of buyers that behave alike: how often they buy, what they buy, how they respond to discounts, which channels they use, which language they read.
2. Describe each stratum from its data. Order history, categories, return behaviour, past campaign responses. The description is written from what these buyers did, not from what we imagine they want.
3. Give each persona a voice. A persona model reads the description and answers as that kind of buyer would, when shown a subject line, a message and an offer.

Every stratum gets its own persona, so the simulation reflects the mix of your actual audience, not an average customer who does not exist either.

Two models, one prediction

Each persona is played by two different persona models, working as an ensemble. For every variant of a campaign, both predict whether that buyer would open, click, convert or unsubscribe, and each gives the objection it would have: “the discount isn’t worth the shipping”, “I bought this last week”, “this doesn’t sound like you”.

Using two models matters for the same reason judges run on a different model family from the agents they check. When the two agree, the signal is stronger. When they disagree, the disagreement is itself useful: it usually points at a message that could be read two ways.

The predictions are calibrated against real results, stratum by stratum: what the personas predicted is compared with what actual buyers did, and the simulation is adjusted. A simulation that is never checked against reality drifts into fiction.

A synthetic customer is a hypothesis about real ones. It has to be tested against them.

What they get right, and wrong

After enough sends, the pattern is clear.

Ranking is what they are built for: which variant is likely to do better, not by how much.

The absolute rates are the weak point. Models are not good at saying exactly how many people will open something, and we do not use them that way. Novelty is another: a type of campaign the audience has never seen has no history for the personas to draw on. And price sensitivity is hard. What a buyer says about a discount and what they do on a Friday night are not the same.

The rule that blocks a launch

The simulation is part of the launch. A newsletter or SMS launch without a recent simulation behind it is flagged, and it can be blocked until a new one has run. If the content has changed since the last run, the simulation is out of date.

The personas do not decide whether a campaign goes out. A person does, with the predictions, the objections and the disagreements on one screen. The synthetic customers only make sure that nobody sends a campaign to real ones without asking the question first.