A small difference in observed results does not automatically establish a better creative. Check the measurement definition, test design, and uncertainty before declaring a winner. If the comparison cannot answer the original question, record what prevented a conclusion.
Start with the denominator.
| Fictional illustration | Creative A | Creative B |
|---|---|---|
| Landing-page visits | 200 | 200 |
| Recorded purchases | 8 | 10 |
| Observed purchase rate | 4% | 5% |
The observed difference is one percentage point, or 25% relative to A’s observed rate. That arithmetic does not establish a reliable improvement. These invented counts are not Aitoma results and are not a statistical significance test.
Confirm that the events, attribution window, destinations, and counting rules match. Missing tracking, different traffic, or a changed offer can make the apparent creative comparison answer a different question.
Check whether the experiment stayed intact.
Record what changed during the run: audience, budget allocation, landing page, offer, placement, or creative itself. If several important factors changed together, avoid assigning the outcome to one headline or image.
Google’s campaign experiment guidance describes experiments as a way to compare changes with a control. Use the current experiment reporting and appropriate analysis for the platform and metric; there is no universal minimum number of clicks that makes every test conclusive.
Choose the next action deliberately.
| Observation | Reasonable next action |
|---|---|
| Measurement is broken | Repair and validate measurement before another comparison. |
| Several factors changed | Redesign the test around a clearer question. |
| Uncertainty remains large | Use the planned analysis and practical effect threshold to decide whether more evidence is worthwhile. |
| The difference is too small to matter operationally | Consider production effort and other relevant constraints without claiming a performance winner. |
Avoid repeatedly changing the stopping rule to chase a favorable result. Decide the intended duration and analysis approach before the run, and document deviations.
Write a decision record.
Record the question, tested change, dates, data definition, outcome, uncertainty, and next action. “No reliable distinction established under these conditions” is more useful than an unsupported winner label.
Keep exploratory observations as hypotheses for the next brief. Do not convert them into a revenue promise or assume organic engagement establishes paid advertising profitability.