A product packaging A/B test compares two defined versions against a question you can measure. It should help you choose between specific changes, such as where the variant name appears. A handful of compliments or a busier market day cannot establish that a package increased sales.
01
Write a hypothesis that can be wrong
A useful hypothesis links a change to a task: moving the tea blend name above the illustration will help shoppers find the correct pouch. Define success as correct identification, time to identify or another observable result. More premium is too broad unless you explain how it will be assessed.
Record the current version as A and the proposed version as B. Keep the product, quantity, price and supporting copy stable when those are not the variables being tested. If the new package also introduces a different photograph and offer, you cannot isolate the effect of the label layout.
02
Choose between a task comparison and a sales experiment
For a usability comparison, give participants a realistic task without naming the design feature. Ask them to find a particular scent among several packages and note what happens. Avoid teaching the layout first. Presentation order can affect performance, so vary it and document your method.
For an online conversion experiment, use a system that can assign visitors consistently and record exposures and outcomes. Decide the duration and evidence threshold before looking for a winner. Repeatedly stopping when one version happens to lead can produce an unreliable conclusion.
| Question | Suitable evidence |
|---|---|
| Can people identify the scent? | Observed selection task |
| Do people understand the contents? | Unaided explanation |
| Does the page convert differently? | Controlled visitor experiment with enough data |
03
Keep a record of conditions and exclusions
For an illustrative task study, record version, participant context, correct choice and the point of confusion. Eight observations can show a wording problem worth fixing, but should not be presented as a precise estimate of the whole market. Save the package images used in the session.
Market stalls introduce extra variation through weather, footfall, stock and the seller’s explanation. Comparing one Saturday with another is not the same as random assignment. Such observations may suggest a question, but label them as observational rather than attributing a sales difference to the package.
04
Use the result to make a bounded decision
Review the evidence against the original hypothesis, including unexpected harms. A larger scent name may help recognition while crowding another important field. If the result is inconclusive, say so and consider a simpler change or a different research method instead of inventing a winning percentage.
Approve the selected package through the normal print-proof process before a full run. Keep the experiment record separate from the production file so the team knows both why the decision was made and which artwork is current. Revisit the question when the container or product range changes.
BEFORE YOUR NEXT STEP
Your practical checklist
- Define the hypothesis and outcome in advance.
- Keep unrelated product and offer details stable.
- Document assignment, conditions and observation limits.
- Report an inconclusive result when the evidence is insufficient.
Questions & answers
Is a preference poll an A/B conversion test?+
No. It measures stated preferences among respondents. A conversion experiment measures behavior under a controlled comparison and needs enough data.
Can I test several design changes together?+
You can compare whole concepts, but you cannot then attribute the result to one element. State that limitation and avoid a narrow causal claim.
Sources & further reading
Official references for the technical points in this guide. Check current specifications before production.
