Comparing Results Across Survey Waves: A Methodology Guide¶

"NPS is up 6 points from last quarter" is only a meaningful statement if the two measurements are actually comparable - same question, similar audience, similar conditions. A change in the number can just as easily reflect a change in how the wave was run as a real shift in opinion.

Keep the Question Wording Identical¶

Even a small wording change between waves can shift how respondents answer, independent of any real change in opinion. If you're tracking a metric over time, lock the question wording once you start the series and resist the urge to "improve" it mid-tracking-study - a wording change is effectively starting a new series, not continuing the old one.

Keep the Audience Comparable¶

If Wave 1 was distributed to your existing customer list and Wave 2 leaned more heavily on Panel Recruitment, a shift in the numbers could reflect the audience difference rather than a real change in sentiment. Keeping your distribution approach consistent across waves - or at minimum, being aware of what changed - is important context for interpreting any movement.

Watch for Sample Size Differences Between Waves¶

If one wave collected substantially more or fewer responses than another, the two waves carry different amounts of statistical noise (see When Your Sample Size Is Too Small to Trust a Result) - a change between a large-sample wave and a small-sample wave deserves more scrutiny than a change between two similarly sized waves.

Keep Text Analytics Categories Consistent¶

If you're tracking themes from an open-ended question across waves, independently regenerating categories each wave risks producing category sets that don't line up cleanly - see Comparing Category Sets Across Two Survey Waves for the specific fix (build once, reuse the definitions).

Bringing Waves Together in a Report¶

Once you've kept the above consistent, Combined Survey lets you build comparison widgets across waves in one report rather than manually reconciling separate exports - see Cross-Tabs and Combined Survey Analysis in Reporting.

Small Movements Are Often Noise¶

A 2-3 point shift in a tracked metric between waves is frequently within the range of normal sample variation rather than a real change - see When Your Sample Size Is Too Small to Trust a Result. Before reacting to a small movement, consider whether it's a consistent trend across multiple waves or a one-off blip.

FAQ¶

How often should I run waves for a tracking study?
This depends on how quickly you expect the thing you're tracking to actually change - too frequent, and you're mostly measuring noise between waves rather than real movement.

Is it okay to change screening criteria between waves?
If you need to, document the change and interpret results with that context in mind - a screening change can shift who's eligible to respond, which affects comparability.

Can Combined Survey handle more than two waves at once?
Yes - there's no fixed cap on how many surveys you bring into a Combined Survey comparison.

What if I need to change the question wording partway through a tracking series?
Consider it the start of a new series for comparison purposes, and be transparent that a direct wave-over-wave comparison across the wording change isn't reliable.

For more on keeping Text Analytics results specifically aligned across waves, see Comparing Category Sets Across Two Survey Waves.