Free tool
A/B test sample size calculator
Enter your current conversion rate, the smallest lift worth detecting and the daily visitors to the page you are testing. The calculator returns the total visitors the test needs and how many days that takes, for a classic 50/50 split or a holdout where only a share of visitors is kept as control.
Visitors needed in total
42,218
21,109 of them in the control group. At your traffic that is about 22 days, so run 4 full weeks.
How to read the result
The figure is the total number of visitors across both groups needed to detect the lift you entered, at the confidence and power you chose. Run the test until that total is reached, in whole weeks, and read the result once at the end. Stopping early because one version looks ahead produces false winners.
If the number of days is longer than you can wait, test a bigger change, measure a step with a higher rate such as add to cart, or decide from customer research instead. Our guide on how much traffic an A/B test needs has a table of common cases, and testing on a low-traffic website covers what to do with less.
Why the control share matters
A classic A/B test splits traffic evenly, which reaches a verdict fastest. Always on personalisation usually keeps a smaller holdout, such as 20 percent, so fewer visitors miss changes that are probably helping. The trade is time: the smaller group fills slowly and sets the pace of the comparison. IntellQ keeps 20 percent as control by default and will not promote a change until each group has at least 200 visitors and the probability it beats control reaches 95 percent. Read more in what is a holdout group.
Questions
How is the sample size calculated?
With the standard two proportion formula, two sided, at the confidence and power you choose, for the share of traffic kept as control. The total covers both groups. Everything runs in your browser.
Why does a holdout need more visitors than a 50/50 test?
Because the smaller group limits the comparison. With 20 percent in control, roughly 60 percent more visitors are needed in total for the same precision as an even split.
What lift should I enter?
The smallest relative improvement that would be worth acting on. Halving it roughly quadruples the visitors needed, so be realistic: 20 to 30 percent suits most small and mid sized sites.
Which daily visitors should I use?
Visitors who reach the page being tested, not site wide traffic. A product page change only reaches people who view product pages.