“You surveyed only N = 1000 respondents. How can these results be representative of the entire country with a population of millions?”
If you’ve wondered the same, you are not alone. We often receive these queries from readers and clients who are afraid to place their trust in polls that survey only a fraction of people to derive insights for the entire target audience.
So, is a sample size of N = 1000 sufficient?
Yes and no.
Let’s tackle the yes first
A common misconception when it comes to polls is that you need to survey a really large chunk of the population to get reliable results. This is simply not true.
A sample size of N=1000 can provide a reasonably accurate representation of the population with a margin of error of approximately +/- 3%, if the sample is randomly selected, is representative of the population, and if the survey is well-designed
An important factor to think about is the margin or error. So, what is MoE all about?
Well, it is often humanly not possible to survey every person from the target audience (e.g., the entire country or all smokers). You are going to be able to survey only a subset of them. The margin of error tells you how far away from the “true” value of the population is your survey result. So for instance, if your survey results reveal “30% of Singaporeans love the color yellow at an MoE of +/- 5% “, this means that the “true” value of the actual population could be anywhere between 25%-35%. Ideally, you would want your margin of error to be low so that you get a precise estimate.
Now the margin of error largely depends on the sample size of your survey, and they have an inverse relationship. The larger the sample size, the smaller the margin of error. ±3% is usually an acceptable level of margin of error for surveys targeted at the general population.
The table below shows that if you are aiming for a ±3% MoE, you need only N = 345 samples if your target population is N = 500. Interestingly, a sample size of N = 1000 is giving you the same level of accuracy for all population sizes above N = 10,000. What this means is that if you want to get survey results that reflect the opinions of for instance 6 million Singaporeans, you are good with a sample size of N = 1000.
This table lists the sample sizes you need to aim for depending on the target population size and desired margin of error.
Now, coming to the part where N = 1000 may not be good enough
Let’s say you are surveying a group of N = 1000 respondents to draw conclusions about the general population. It is important to ensure that the respondents are randomly selected, and that the sample is representative of the population in key demographic criteria in order to make accurate conclusions. It is often difficult to control for a host of demographic variables, and so it is common to control it for those few that may lead to biggest skews for the topic you are polling (e.g., age, gender, location and household income).
This is typically done by a method known as quota sampling. Quota sampling is a non-probability sampling method in which a sampling plan is created before fieldwork. For example, you can set quotas to ensure your sample consists of an equal distribution of men and women (N = 500 each, instead of a skewed sample of n = 800 women and N = 200 men for instance). This way you can ensure that your survey results closely mirror that of the target population.
It also depends on the context and the purpose of the survey. With a sample size of n=1000, you can get a general sense of the population’s opinions or characteristics if the sample is representative of the population and the survey is well-designed. However, larger sample sizes may be needed if you want to make inferences with more precision or if your population is very diverse, or if you want to drill into sub-segments within the population that may be niche and hard to reach (e.g., minority ethnic groups).
