The Language of Hypothesis Testing
The theory of statistical hypothesis testing was developed in the early 20^{th} century and has been the mainstay of practical statistics ever since. It was designed to apply the scientific method to situations involving data with random fluctuations (and almost all realworld data has random fluctuations). Following are a few terms commonly used in hypothesis testing.

Null hypothesis (abbreviated H_{0}): The assertion that any apparent effect you see in your data does not reflect any real effect in the population, but is merely the result of random fluctuations in your sample.

Alternate hypothesis (abbreviated H_{1} or H_{Alt}): The assertion that there really is some real effect in your data, over and above whatever is attributable to random fluctuations.

Significance test: A calculation designed to determine whether H_{0} can reasonably explain what you see in your data.

Significance: The conclusion that random fluctuations alone can’t account for the size of the effect you observe in your data, so H_{0} must be false, and you accept H_{Alt}.

Statistic: A number that you obtain or calculate from your data.

Test statistic: A number, calculated from your data, usually for the purpose of testing H_{0}. It’s often — but not always — calculated as the ratio of a number that measures the size of the effect (the signal) divided by a number that measures the size of the random fluctuations (the noise).

p value: The probability that random fluctuations alone in the absence of any real effect (in the population) could have produced an observed effect at least as large as what you observe in your sample. The p value is the probability of random fluctuations making the test statistic at least as large as what you calculate from your data (or, more precisely, at least as far away from H_{0} in the direction of H_{Alt}).

Type I error: Getting a significant result when, in fact, no real effect is present, only random fluctuations.

Alpha: The probability of making a Type I error.

Type II error: Failing to get a significant result when, in fact, some effect really is present.

Beta: The probability of making a Type II error.

Power: The probability of getting a significant result when some effect is really present.