Is your A/B test result real or luck?
You changed a headline, price, email subject or button, and version B did better. But was it a real improvement or random chance? This calculator tells you in plain English.
- Conversions: Two-proportion z-test with p-value, confidence range for the difference, relative lift and the probability that B is really better.
- Averages: Welch's t-test for average order value, time on page or any measurement, from summary figures or raw numbers.
- Sample size: How many visitors you need before starting, and how many days that takes.
Results are checked against the Python SciPy and statsmodels statistics libraries.