How to Determine the Sample Size You Need for Customer Analytics

By Jeff Sauro

It would be good to know what customers think or what product features they like so you can build better products. How do you get this information? By asking your customers to take a poll.

But the problem is you rarely can ask all your customers to take your poll. That would be too time intensive and expensive, and you’d be swimming in data. Instead you collect data from a sample of customers. Your sample size can be small — 5 to 10 customers, or very large — 10,000+ customers. The data you collect from your sample represents all your customers.

Surveying a sample of customers is a tried and true method. Polling agencies that measure electoral votes take the same approach: They ask only a subset of the voting population to predict how the entire electorate will vote.

You can achieve a rather low margin of error after just sampling a small fraction of the entire population. There is a diminishing return from increasing sample sizes beyond a certain point. While you want your estimate to be accurate, you have to determine the sample size you can afford based on your budget and company resources.

The table shows a table of confidence intervals around different sample sizes. You can see how the margin of error and width of the confidence interval decrease as the sample increases. As a general approximation, you need to quadruple your sample size in order to cut your margin of error in half.

Confidence Intervals (CI)
Sample Size Margin of Error (+/–) 95% CI Low 95% CI High
13 24% 26% 74%
21 20% 30% 70%
39 15% 35% 65%
46 14% 36% 64%
53 13% 37% 63%
63 12% 38% 62%
76 11% 39% 61%
93 10% 40% 60%
115 9% 41% 59%
147 8% 42% 58%
193 7% 43% 57%
263 6% 44% 56%
381 5% 45% 55%
597 4% 46% 54%
1,064 3% 47% 53%
2,398 2% 48% 52%

For example, at a sample size of 381, your sample estimate will have a margin of error of approximately plus or minus 5%. That means if a survey to customers indicates that 50% say they would repurchase their subscription in the following year, you can be 95% confident that between 45% and 55% of all customers actually will. The sample size needed when making comparisons (say, between different product designs) will differ.