Sample Size Calculations
It used to be that statisticians made their living by analyzing other people’s data. Unfortunately, the personal computer revolution has encroached on our livelihood. Anyway, we statisticians are an adaptive species, so we’ve found a new career—calculating sample sizes. Nowadays, no grant proposal gets to square one without a sample size calculation.
To be honest, sample size calculations are the most tedious part of the job. As we will see, they are based on some heroic assumptions about the likely differences you will encounter at the end of the study. In fact, the only time you are really in a good position to do a defensible sample size calculation is after the study is over because only then will you have decent estimates of the required parameters.
Failing that, it’s possible to get about any size of sample you could ever want. As we will show you, every sample size calculation involves separate estimates of four different quantities in order to arrive at the fifth, the sample size. The real talent that statisticians bring to the situation is the ability to fiddle all those numbers through several iterations so that the calculated sample size precisely equals the number of patients you wanted to use in the first place.
We start with the two normal curves of Fig.1130. A moment’s reflection reveals that the shape of those curves is determined by the following four quantities:
-
The distance between the two mean values, which we’ll call d.
-
The width of the two curves (assumed to be the same). This equals the SD, s, divided by the square root of the sample size, n.
-
The distance from the left-hand curve (the null hypothesis) to the critical value (in Fig.1130, it’s 4.92). This is in turn related to the choice of the alpha level (0.05, 0.10, or whatever).
-
The distance from the right-hand curve (the alternate hypothesis) to the critical value (in Fig.1130, it’s 5.08), and this is related to the choice of the beta level.
If we know any three of the four numbers, we can determine the fourth analytically. So if we fix the difference between the two means, the SD, the sample size, and the alpha level, then we can determine the beta level. That’s what we did in the previous section, but it also works the other way. If we fix the alpha level and beta level, the difference between the two means, and the SD, then this determines the sample size.
That’s how sample size calculations proceed. We take guesses (occasionally educated, more commonly wild) at the difference between means and the SD, fix the alpha and beta levels at some arbitrary figures, and then crank out the sample size. The dialogue between the statistician and a client may go something like this:
Statistician: So, Dr. Doolittle, how big a treatment effect do you think you’ll get? Dr. Doolittle: I dunno. That’s why I’m doing the study. S: Well, how do 10 points sound? D: Sure, why not? S: And what is the standard deviation of your patients? D: Excuse me. I’m an internist, not a shrink. None of my patients are deviant. S: Oh well, let’s say 15 points. What alpha level would you like? D: Beats me. S: 0.05 as always. Now please tell me about your beta? D: I bought a VHS machine years ago. S: Well, we’ll say point 2. (Furiously punches calculator.) You need a sample size of 36. D: Oh, that’s much too big! I only see five of those a year. S: Well, let’s see. We could make the difference bigger. Or the standard deviation smaller. Or the alpha level higher. Or the beta level higher. Where do you want to start fudging?
The actual formula is a bit hairy, but we can guess what it might look like from the picture. The bigger the difference, the smaller the required sample size, so the difference between groups must be in the denominator. The bigger the SD, the larger the required sample size, so the SD must be in the numerator. You really don’t have much to say about the alpha level; unless you are really desperate, that has to be 0.05. Finally, if you are willing to forego the opportunity to mess around with the beta level and accept a standard beta level of 0.2 (a power of 80%), which is as low as you should dare go anyway, then the whole mess reduces to the following easy formula5497 :
Figure 1352 –
Some figures may not display clearly when rendered as a PDF or printed.
So, taking the above example, with a difference (d) of 10, an SD (s) of 15, and alpha and beta as required, the sample size is as follows:
Figure 1353 –
Some figures may not display clearly when rendered as a PDF or printed.
Amazing how much they will pay us to do this little trick.
Regrettably, the world is a bit more complicated than that because this formula only works when you have two groups and a continuous DV, so we have to devise some alternatives for other more complicated cases. To do so, we will be using terms such as analysis of variance (ANOVA) and regression, which you won’t encounter until later, so you may have to take some of this on faith.
The following are strategies for some other situations:
-
Difference between proportions (see Chapter 9). Often, people like to do studies where they count bodies at the end and see whether the latest (and most expensive) drug can save lives. They end up comparing the proportion of people in the drug group (Pd) with the proportion in the placebo group (Pp). The sample size equation uses the difference between the two (Pp – Pd) in the denominator. However, the SD is calculated directly from the average of the proportions—(Pp + Pd)/2, which we will call p, using some arcane theory of numbers—resulting in the following modified formula:
n = 16[(p(1-p)]/[(Pp - Pd)2]
-
Difference among many means (see Chapter 5). Although there are exact methods to calculate sample sizes when you have many means and are using ANOVA methods, these involve additional assumptions about the way the means are distributed and are even less defensible (if that’s possible). What we do is pick the one comparison between the two means that we care the most about, and then use the original formula.
-
Relation between two continuous variables (see Chapter 6). Calculating sample size for a regression coefficient involves knowledge of both the coefficient and its SE, at which no mortal can hazard a guess. A better way is to do the calculation on the correlation coefficient, r. As it turns out, the SD of a correlation coefficient is approximately equal to 1/√(n–2), so to test whether this differs from zero (the usual situation), we can again manipulate the original formula, and it becomes strangely different:
n = 4 + (8 / r)
-
Relation among many variables (see Chapters 6, 7, 8, and 13 to 19). At this point, we throw up our hands and invoke an old rule of thumb—the sample size should be 5 to 10 times the number of variables.
One final caveat, though: sample size calculations should be used to tell you the order of magnitude you need—whether you need 10, 100, or 1,000 people, not whether you need 22 or 26.5343
Content on this page was last changed on March 19, 2009.
© 2002 BC Decker Inc. Show Disclaimer
| 5343. | Norman GR, Streiner DL. PDQ Statistics . 3rd ed. Hamilton, Ontario: BC Decker Inc.; 2003. |
| 5497. | Lehr R. Sixteen S-squared over D-squared: a relation for crude sample size estimates. Stat Med 1992;11:1099–102. |
Next Page: Statistical Versus Clinical Signifigance »