How Many Survey Responses Are Enough?
The answer depends on the size of the population, but not the way you'd think. For a 500-person organization and an 80-million-person country, the number you need is not that different.
The question "how many people is enough?" has a precise answer, but the answer is counterintuitive.
The counterintuitive part: population size barely matters
To reach a 5% margin of error at 95% confidence:
| Population size | Sample needed |
|---|---|
| 500 people | 218 |
| 5,000 people | 357 |
| 50,000 people | 381 |
| 80 million | 385 |
Yes: to survey an entire country and to survey a city of 50,000, you need roughly the same number of people. What matters isn't the size of the population; it's the randomness of the sample.
Small populations: you need a high percentage
In a 50-person organization, 218 people don't exist. There you need a high response rate, at least 80%, or the 10 people who answered aren't representative of the rest.
A bigger error than sample size: selection bias
1,000 responses from people who volunteered are worse than 200 from a random sample. The people who answer surveys are usually either very satisfied or very dissatisfied; the indifferent ones don't respond. That means your distribution is more polarized than reality.
How to reduce it: send the survey to a defined sample and follow up, rather than posting the link publicly and waiting.
When you segment, the number you need multiplies
This is the most important practical point. If you have 400 responses and want to compare five departments, each department has about 80 responses, and each department's margin of error is now around 11%, not 5%.
So if department A has 72% satisfaction and department B has 68%, that difference is not significant. Before you make a decision based on a difference like that, look at the number of responses in each group.
Rules of thumb to start with
- Under 30 responses in a group: read them qualitatively only; don't compute percentages.
- 30 to 100: the overall trend is reliable; differences under 15% are not.
- Over 300: 5% differences start to mean something.