Which factors determine the sample size for an A/B test? (Select two.)

Question

Options

• ATraffic volume
• BLift percentage
• CPage load time
• DTest developer experience

• A
  95% (52)
• C
  2% (1)
• D
  4% (2)

Explanation

Traffic volume (A) and lift percentage (B) are the two core statistical inputs for calculating A/B test sample size. Traffic volume determines how long you need to run the test to collect enough observations - low traffic means you need more time to reach the required sample. Lift percentage (the minimum detectable effect) has an inverse relationship with sample size: smaller expected improvements require far larger samples to detect reliably, while larger expected lifts need fewer users.

Page load time (C) is a metric you might measure in an A/B test, not a factor that determines how many users you need. Developer experience (D) has no bearing on statistical power or sample size calculations whatsoever.

Memory tip: Think of it as "How many people do you need, and why?" - you need enough traffic to run the test in a reasonable timeframe, and the smaller the lift you're trying to detect, the more people you need. Everything else (load time, who built it) is irrelevant to the math.

No community discussion yet for this question.