What factors should be considered when determining the winning variation in an A/B Test? (Select two.)

Question

Options

• ALift percentage
• BConfidence level
• CPage views per visitor
• DTotal campaign duration

• A
  80% (28)
• C
  9% (3)
• D
  11% (4)

Explanation

Determining a winning variation in an A/B test requires both lift percentage (A) and confidence level (B) because lift tells you how much better one variation performed (the magnitude of improvement), while confidence level tells you how certain you can be that the result isn't due to random chance - typically needing 95%+ confidence before declaring a winner. Without both metrics, you risk either acting on a statistically meaningless result or ignoring a meaningful one.

Why C is wrong: Page views per visitor is an engagement metric, not a determinant of which variation won - it doesn't indicate whether a variation outperformed the control or whether the result is statistically valid.

Why D is wrong: Total campaign duration is a planning input (you run the test long enough to collect sufficient data), not a factor used to evaluate which variation wins after the data is collected.

Memory tip: Think "LC = Legit Conclusion" - Lift (the what) + Confidence (the how sure) are the two pillars needed to legitimately declare a winner in any A/B test.

No community discussion yet for this question.