Finding the Power of a Hypothesis Test

By Deborah J. Rumsey, David Unger

When you make a decision in a hypothesis test, there’s never a 100 percent guarantee you’re right. You must be cautious of Type I errors (rejecting a true claim) and Type II errors (failing to reject a false claim). Instead, you hope that your procedures and data are good enough to properly reject a false claim.

The probability of correctly rejecting H0 when it is false is known as the power of the test. The larger it is, the better.

Suppose you want to calculate the power of a hypothesis test on a population mean when the standard deviation is known. Before calculating the power of a test, you need the following:

  • The previously claimed value of

    image0.png

    in the null hypothesis,

    image1.png

  • The one-sided inequality of the alternative hypothesis (either < or >), for example,

    image2.png

  • The mean of the observed values

    image3.png

  • The population standard deviation

    image4.png

  • The sample size (denoted n)

  • The level of significance

    image5.png

To calculate power, you basically work two problems back-to-back. First, find a percentile assuming that H0 is true. Then, turn it around and find the probability that you’d get that value assuming H0 is false (and instead Ha is true).

  1. Assume that H0 is true, and

    image6.png

  2. Find the percentile value corresponding to

    image7.png

    sitting in the tail(s) corresponding to Ha. That is, if

    image8.png

    then find b where

    image9.png

    If

    image10.png

    then find b where

    image11.png

  3. Assume that H0 is false, and instead Ha is true. Since

    image12.png

    under this assumption, then let

    image13.png

    in the next step.

  4. Find the power by calculating the probability of getting a value more extreme than b from Step 2 in the direction of Ha. This process is similar to finding the p-value in a test of a single population mean, but instead of using

    image14.png

    you use

    image15.png

Suppose a child psychologist says that the average time that working mothers spend talking to their children is 11 minutes per day. You want to test

image16.png

versus

image17.png

You conduct a random sample of 100 working mothers and find they spend an average of 11.5 minutes per day talking with their children. Assume prior research suggests the population standard deviation is 2.3 minutes.

When conducting this hypothesis test for a population mean, you find that the p-value = 0.015, and with a level of significance of

image18.png

you reject the null hypothesis. But there are a lot of different values of

image19.png

(not just 11.5) that would lead you to reject H0. So how strong is this specific test? Find the power.

  1. Assume that H0 is true, and

    image20.png

  2. Find the percentile value corresponding to

    image21.png

    sitting in the upper tail. If p(Z > zb) = 0.05, then zb = 1.645. Further,

    image22.png

  3. Assume that H0 is false, and instead

    image23.png

  4. Find the power by calculating the probability of getting a value more extreme than b from Step 2 in the direction of Ha. Here, you need to find p(Z > z) where

    image24.png

    Using the Z-table, you find that

    image25.png

Hopefully, you were already feeling good about your decision to reject the null hypothesis since the p-value of 0.015 was significant at an

image26.png

of 0.05. Further, you found that Power = 0.6985, meaning that there was nearly a 70 percent chance of correctly rejecting a false null hypothesis.

image27.jpg

This is just one power calculation based on a single sample generating a mean of 11.5. Statisticians often calculate a “power curve” based on many likely alternative values. Also, there are some unique considerations to take into account if

image28.png

but this gives you the gist of things.