What does a flat trendline in an A/B Test report typically indicate?

Question

Options

• AConsistent performance across variations
• BThe winning variation has been identified
• CSignificant performance discrepancies
• DInsufficient sample size to draw conclusions

• A
  17% (3)
• B
  6% (1)
• C
  6% (1)
• D
  72% (13)

Explanation

A flat trendline in an A/B test report signals that performance metrics are not changing over time - which happens when there isn't enough data yet for meaningful differences to emerge, making D the correct answer. Without sufficient sample size, statistical noise dominates and the test cannot reliably detect true differences between variations, producing that flat, inconclusive pattern.

Why the distractors are wrong:

• A is tempting but wrong - consistent performance would show stable, separated lines, not necessarily a flat one; "consistent" implies a conclusion has been reached.
• B is incorrect because a flat trendline means no variation is winning - the opposite of identification.
• C contradicts the premise entirely; significant discrepancies would appear as diverging or separated lines, not a flat one.

Memory tip: Think of a flat trendline as the test "shrugging" - it literally doesn't have enough data to lean either way, so the graph can't go anywhere. Flat = not enough data to react.

No community discussion yet for this question.