Which metric should be monitored when troubleshooting discrepancies in A/B Test reporting?

Question

Options

• ABounce rate
• BSample size
• CPage views
• DConversion rate

• A
  4% (1)
• B
  93% (25)
• D
  4% (1)

Explanation

Sample size is the critical metric to monitor when troubleshooting A/B test reporting discrepancies because an insufficient or imbalanced sample size introduces statistical noise, skews results, and can make random variation appear as a meaningful difference between variants. Many reporting anomalies trace back to tests that ended too early, had unequal traffic splits, or lacked the statistical power needed to detect real effects.

Why the distractors are wrong:

• Bounce rate (A) is an outcome metric - a result of the test, not a diagnostic tool for reporting discrepancies.
• Page views (C) are a raw traffic count and don't reveal whether the test is statistically valid or whether groups are comparable.
• Conversion rate (D) is the primary goal metric of most A/B tests, not a troubleshooting lever - discrepancies in conversion rate are the symptom, not the cause.

Memory tip: Think of sample size as the foundation of an A/B test - if the foundation is cracked (too small, unbalanced, or still growing), everything built on top (conversion rates, bounce rates, page views) will look wrong. When something seems off, always check the foundation first.

No community discussion yet for this question.