Split Tester
Channel A
Winner
Channel A Conv. Rate0.00%
Channel B Conv. Rate0.00%
Difference0.00%
Z-Score0.000
P-Value0.000
Statistically Significant?No
Recommended ActionGather more data
Get Your Results Emailed
Enter your email and we'll send you a summary of your calculator calculation. No spam, unsubscribe anytime.
How to Use
Name your two marketing channels and enter the number of visitors and conversions for each. The calculator uses a two-proportion z-test to compare the conversion rates. Select your desired confidence level — 95% is standard for most marketing decisions.
Statistical Significance Explained
pA = ConversionsA / VisitorsA
pB = ConversionsB / VisitorsB
pPooled = (ConversionsA + ConversionsB) / (VisitorsA + VisitorsB)
SE = √(pPooled × (1 - pPooled) × (1/VisitorsA + 1/VisitorsB))
Z = (pA - pB) / SE
If the absolute z-score exceeds the critical value (1.645 for 90%, 1.96 for 95%, 2.576 for 99%), the difference is statistically significant.
Frequently Asked Questions
Statistical significance tells you whether the difference in conversion rates between two channels is likely due to a real performance difference rather than random chance. A 95% confidence level means there is only a 5% probability that the observed difference is due to luck.
95% is the standard confidence level for most marketing tests. Use 90% for exploratory testing where you want to identify potential winners quickly. Use 99% when the stakes are high, such as making a large budget reallocation.
For reliable results, aim for at least 100 conversions per channel. The more traffic and conversions you have, the smaller the difference you can detect. Small sample sizes can lead to false positives or missed opportunities.
If there is no statistically significant winner, continue running both channels and gather more data. You can also try improving your ad creatives, targeting, or landing pages before testing again.