How Many Panel Respondents Do You Need for a Reliable Result¶

Sizing a Panel Recruitment wave involves two separate questions that are easy to conflate: how many completed responses does your analysis actually need, and how many people do you need to recruit to net that many completions, given that a real share of the people who start won't end up qualifying. Get the first one wrong and your results are statistically shaky. Get the second one wrong and you fund a wave for fewer usable responses than you actually meant to collect.

The General Sample-Size Question¶

This part isn't specific to panel recruitment - it's the same statistical reasoning behind sizing any survey. A commonly cited rule of thumb for a large population is that roughly 385 completed responses gets you to a 95% confidence level with a 5% margin of error on a single proportion - tighter margins or higher confidence levels need meaningfully more. Segment-level analysis (comparing results across age groups, or across two conditions) needs that same minimum within each segment you plan to analyze separately, not just in the total - a survey with 400 total responses split evenly across four segments only has about 100 per segment, which is a real constraint worth planning around before you launch rather than discovering after your target's already been hit. We've written a fuller guide on this general question, independent of panel recruitment specifically, if you want the deeper version: How Many Survey Responses Do You Need.

The Panel-Specific Part: Screen-Outs Don't Count Toward Your Target¶

A Panel Recruitment wave's target is a count of approved responses - people who both qualified past your screening criteria and completed the survey. Anyone who's recruited but doesn't qualify is screened out, and a screened-out submission doesn't count toward that target at all. If your screening criteria are narrow - a specific age band, a specific employment status, a specific product usage history - a real share of recruited participants can be expected not to qualify, and that share needs to factor into how you think about a wave's target, not just the raw number you'd want if everyone who showed up happened to qualify.

There's no universal screen-out rate to plan around - it depends entirely on how narrow your criteria are relative to the general population. Broad criteria (a general age range, a general region) screen out relatively few people; narrow, specific criteria (a specific job title, a specific rare condition) screen out far more. If you don't have a strong sense of how narrow your criteria are in practice, it's worth erring toward funding a slightly larger wave than your bare statistical minimum, rather than finding out partway through that you're recruiting far more screen-outs than approved responses.

Putting Both Together¶

Start from your analysis's real minimum (the general sample-size question above, including any segment-level minimums), then size your wave's target upward based on how narrow your screening is - broad criteria need little to no adjustment; narrow criteria may need a meaningfully larger target to net the same number of usable, approved responses. If a wave underperforms because screen-outs are eating into it faster than expected, launching a second wave is the direct fix - see our guide on launching and funding a wave for how that works.

FAQ¶

Does a screened-out submission cost the same as an approved one?
No - screened-out submissions are paid at a smaller, fixed rate, separate from your survey's main per-response reward. See our Panel Recruitment FAQ for the specifics.

How do I know my likely screen-out rate before launching?
There's no built-in estimate for this - it depends on how your specific criteria compare to the general population, which you're generally better positioned to judge than a generic tool would be. If you're unsure, a smaller initial wave can help you calibrate before committing to a much larger one.

Should I just always fund a larger wave to be safe?
Not necessarily - overfunding a wave with very broad criteria wastes budget on a buffer you didn't need. Match the buffer to how narrow your actual screening is, not to a flat rule applied regardless of criteria.

See Launching and Funding a Panel Recruitment Wave for the full setup walkthrough.