MIINDIGITAL

Optimize for AI. Win visibility. Drive results.

Free A/B Test Significance Calculator

Enter the number of visitors and conversions for your control (A) and variant (B) to instantly check whether your A/B test result is statistically significant, or just noise. Get conversion rates, relative uplift, z-score, and p-value in one click. No signup, no spreadsheet, nothing sent to a server.

Test Data
Variant A (Control)
Variant B (Challenger)
95% confidence is the standard threshold used in most A/B testing tools.
Results
Enter visitors and conversions for both variants to test significance
—
Rate A
—
Rate B
—
Relative Uplift
—
Z-score
—
P-value
    How to read this: a lower p-value means the difference between A and B is less likely to be random chance. Most marketers require p < 0.05 (95% confidence) before declaring a winner.

    Need help with your digital marketing?

    MiinDigital works with small businesses on SEO, GEO, websites and email marketing.

    Get in touch

    How to Use the A/B Test Significance Calculator

    1. Enter Variant A’s visitors and conversions: this is usually your control, the original page, email, or ad.
    2. Enter Variant B’s visitors and conversions: the new version you’re testing against the control.
    3. Choose a confidence level: 95% is the standard used by most A/B testing tools and is a safe default.
    4. Read the banner and insights to see whether the result is statistically significant, which variant is winning, and by how much.
    5. Copy the results with one click to share in a report, deck, or team chat.

    What the Numbers Mean

    • Conversion Rate: the percentage of visitors who converted for each variant. Calculated as conversions ÷ visitors × 100.
    • Relative Uplift: how much better (or worse) Variant B performed compared to Variant A, as a percentage change.
    • Z-score: how many standard deviations apart the two conversion rates are. Larger (in either direction) means a bigger, more reliable difference.
    • P-value: the probability of seeing a difference this large (or larger) if there were actually no real difference between A and B. Lower is stronger evidence of a real effect.
    • Statistical Significance: whether the p-value is low enough, given your chosen confidence level, to trust that the difference is real rather than random noise.

    Why Statistical Significance Matters in A/B Testing

    It’s tempting to call a winner the moment one variant pulls ahead, but small samples are noisy: a version that looks 20% better after 50 visitors can easily flip after 500. This calculator runs a two-proportion z-test to tell you whether the gap is big enough, for your sample size, to trust. Decide your sample size before you start, and don’t stop the test the moment it crosses the line. Evan Miller’s analysis shows that checking results over and over and stopping when they look good can push a nominal 5% false-positive rate above 25%.

    Frequently asked questions

    What confidence level should I use for A/B testing?

    95% is the industry standard and a safe default for most marketing tests. Use 99% for high-stakes decisions where a false positive would be costly, or 90% for early-stage, low-risk experiments where you want faster signal.

    What sample size do I need for a reliable A/B test?

    There’s no single number, it depends on your baseline conversion rate and the size of the effect you’re trying to detect. As a rough guideline, aim for at least a few hundred conversions per variant; results below 100 visitors per variant should be treated as directional, not conclusive.

    What does “p-value” actually mean?

    The p-value is the probability of seeing a difference at least this large if A and B actually performed the same. A p-value of 0.03 means that, with no real difference, a gap this big would show up about 3% of the time. It is not the chance that your result is wrong. At 95% confidence, anything below 0.05 counts as significant.

    My test isn’t significant yet: what should I do?

    Keep running it. “Not significant” doesn’t mean “no difference,” it often means you don’t have enough data yet to detect the difference reliably. Avoid making changes mid-test, and resist the urge to stop early just because the numbers dip in your favor for a day.

    Is this tool free and does it store my data?

    Yes, completely free with no account required. All calculations happen in your browser, your test data is never sent to any server.

    Sources

    1. Evan Miller, “How Not To Run an A/B Test“, accessed September 2026.

    Related free tools