How to Compare Sets of Matched Numbers
The unpaired (independent-sample) t tests, one-way ANOVA, ANCOVA, and their nonparametric counterparts deal with comparisons between two or more groups of independent samples of data, such as different groups of subjects, where there’s no logical connection between a specific subject in one group and a specific subject in another group.
But you often want to compare sets of data where precisely this kind of pairing exists. Matched-pair data comes up in several situations (illustrated here for two sets of data, but applicable to any number of sets):
The values come from the same subject, but at two or more different times, such as before and after some kind of treatment, intervention, or event.
The values come from a crossover clinical trial, in which the same subject receives two or more treatments at two or more consecutive phases of the trial.
The values come from two or more different individuals who have been paired, or matched, in some way. They may be twins or they may be matched on the basis of having similar characteristics (such as age, gender, and so on).
Comparing matched pairs
Paired comparisons are usually handled by the paired student t test. If your data isn’t normally distributed, you can use the nonparametric Wilcoxon Signed-Ranks test instead.
The paired Student t test and the one-group Student t test are really the same test. When running a paired t test, you (or the software) first calculate the difference between each pair of numbers (for example, subtract the pretreatment value from the post-treatment value), and then test those differences against the hypothesized value 0 using a one-group test.
Comparing three or more matched numbers
When you have three or more matched numbers, you can use repeated-measures analysis of variance (RM-ANOVA). The RM-ANOVA can also be used when you have only two groups; then it gives exactly the same p value as the classic paired Student t test.
If the data is non-normally distributed, you can use the nonparametric Friedman test. (Be careful — there are several different Friedman tests, and this isn’t the same one that’s used in place of a two-way ANOVA!)
Another problem to be aware of with RM-ANOVA and more than two levels is the issue of sphericity — an extension of the idea of equal variance to three or more sets of paired values. Sphericity refers to whether the paired differences have the same variance for all possible pairs of levels.
Sphericity is assessed by the Mauchly test, and if the data is significantly nonspherical, special adjustments are applied to the RM-ANOVA by the software.