在用 ANOVA 時，會得到一個 F-test，這個 test 只能告訴你整個 model 是否顯著 (i.e., 總體平均數不相同或不完全相同)，並不能告訴你特定的組與組之間是否有差異。有很多種方法可以比較組間的平均值是否有差異，這些方式就稱作多重比較 (multiple comparisons)

如果要作 post hoc comparisons (有人叫 posteriori test) ，有許多選擇：1) Fisher’s Least Significant Difference (又稱為 LSD)；2) Tukey’s Test；3) The Ryan Procedure (REGWQ)；4) The Scheffé Test；5) Dunnett’s test for comparing all treatments with a control。

會產生這麼多方法，主要是比較的方式不同。有的方法是與均值作比較 (i.e., 所有組別的平均皆相同)，有的方法是配對比較 (pairwise comparison) (i.e., 與控制組比較、與最佳組比較)，有的是 contrast (特定兩組比較)。

比較表 (出自 Statistical methods for Psychology by David C. Howell，第六版，頁375)： 

Note:
^a: Error rate per comparison.
^b: Family error rate (FW). 
FW^*: against complete null hypothesis.
t^** : modified t test.
F^*: modified.
Pairwise***: Tukey HSD can be used for all contrasts, but is poor for this purpose.

----------------------

After you have run ANOVA and got a significant F-statistic (i.e. you have rejected the null hypothesis that the means are the same), then you run Sheffe’s test to find out which pairs of means are significant. The Scheffe test corrects alpha for simple and complex mean comparisons. Complex mean comparisons involve comparing more than one pair of means simultaneously.

Out of the three mean comparisons test you can run (the other two are Fisher’s LSD and Tukey’s HSD). The Scheffe test is the most flexible, but it is also the test with the lowest statistical power. Deciding which test to run largely depends on what comparisons you’re interested in:

If you only want to make pairwise comparisons, run the Tukey procedure because it will have a narrower confidence interval.
If you want to compare all possible simple and complex pairs of means, run the Scheffe test as it will have a narrower confidence interval.

Understanding Scheffé's Test

While Scheffé's test has the advantage of giving the experimenter the flexibility to test any comparisons that appear interesting, the drawback of this flexibility is that the test has relatively lower statistical power than tests that are designed for pre-planned comparisons.

The Scheffé test is one of the oldest multiple comparison procedures in use today. It is important to recognize that it is a frequently misused procedure and that it is also a valuable test when used as Henry Scheffé intended it. Unlike competitors such as Tukey's Honestly Significant Difference test, the Scheffé test is specifically designed for the situation in which post hoc comparisons involve more than pairwise differences. For example, it could be used to compare the mean of two groups to the mean of two other groups on the basis of interesting differences that appeared after the data had been collected. 

Scheffé Test (雪費檢定；同步檢定法) post-hoc test

同時測試所有可能的 contrast，並視是否其中之一為 0。

臨界值最大，最不容易顯著 (也表示較不容易犯 Type II error)。

可用於各組人數不同、非常態分配上。

Generally, Tukey and Scheffé tests are more conservative. They find it harder to see differences and generally give the same result. In relation to the differences:
- In pairwise comparisons, Tukey test is based on studentized range distribution while Scheffe is based in F distribution.
- Tukey's test is very rigorous, controlling the type I error very well, but favors the type II error.
- The Scheffe test allows comparing any contrast between means and allows different number of observations per treatment.

OR

We would posit that there are no major differences.
Huberty, C. J, & Morris, J. D. (1988). A single contrast test procedure. Educational and Psychological Measurement, 48, 567-578.

【出處】

https://researcher20.com/2010/05/27/%E5%A4%9A%E9%87%8D%E6%AF%94%E8%BC%83-multiple-comparisons/

https://www.statisticshowto.com/scheffe-test/

https://www.investopedia.com/terms/s/scheffes-test.asp

https://www.researchgate.net/post/What_are_the_major_differences_between_scheffe_and_tukey_post_hoc_tests