Run an A/B test

Steps

1. Set the goal. Open "Insights", then "Goals", then "New goal". Choose "Page reach" and the page that counts as success. The "Live backtest preview" shows how often it would have matched in the past. Then choose "Preview change" and "Approve & apply".
2. Create the experiment. Open "Optimization", then "Experiments", then "New experiment". Give it a name and a key (the key cannot change later), choose your goal as its "Primary goal", and set the split under "Traffic variations"; the first is the control. Choose "Preview draft", then "Save draft".
3. Attach it to a page. In "Configuration & QA", under "Rendered bindings", bind the treatment to your page.
4. Write version B. Open the page in the Composer, choose version B under "Editing for", and edit it in place.
5. Plan and lock. Choose "Plan sample & duration", pick the metric, choose "Calculate estimate", then "Review & lock plan" and "Approve & lock permanently".
6. Go live. Choose "Activate". The card asks "Put this experiment LIVE?"; choose "Approve & go live".

Good to know

• "Activate" opens the plan before it is locked, and locking is permanent.
• You can lock the plan on a brand-new site's first day. The experiment builds its own history: an estimate appears after two days of traffic, and the test runs for the days you chose, 14 by default.
• Results come from a nightly snapshot. Before the planned length, the screen marks any look as interim; the result is read once, at the planned length.
• If the split itself looks wrong, a guard holds back every verdict: "Results are not trustworthy while this persists."
• No winner is ever chosen automatically. "Promote the winner?" stops the test and copies the winner into the page's draft; publishing it is a second approval.
• Only visitors who allow personalization take part. Browsers that send Global Privacy Control, and visitors in opt-in countries who have not said yes, see the original page.
• Each visitor keeps their version every time they return.
• Putting an experiment live needs your approval. An edit to a live experiment's version waits for your approval too.

In a sentence

Ask the copilot "How long must it run?". Locking the plan, starting the test and promoting a winner always pause on an approval card.