# ANOVA *post hoc* analysis¶

*Post hoc* analyses are currently in development.

The simple approaches described below will approximate appropriate *post hoc* results.

As a rule-of-thumb, **NEVER** report *post hoc* results which disagree with the main test results. If the results disagree, the *post hoc* procedures are too simple.

If you report *post hoc* results for **spm1d** analyses please explicitly highlight the limitations described in this document.

Warning

Current *post hoc* procedures in **spm1d** are likely too simple.

The example belowe uses a simple Bonferroni correction, but this assumes that the *post hoc* tests are independent, which is usually not the case.

Danger

Current *post hoc* procedures in **spm1d** are generally not valid.

They are not valid because they involve separate smoothness assessments for each *post hoc* test.

Although the resulting errors are expected to be small, we cannot guarantee validity.

Users are encouraged to explore other *post hoc* correction methods.

## One-way ANOVA¶

This examples revisits the one-way example.

Since the one-way ANOVA results reached significance, we may justifiably conduct *post hoc* analyses.

For one-way ANOVA, *post hoc* analyses consist of two-sample t tests conducted on all group pairs.

Since there are three groups in our one-way ANOVA example, there are three pairs to test (1v2, 1v3, and 2v3).

A simple way to correct for multiple tests across these three group pairs is to use the Bonferroni correction:

```
>>> alpha = 0.05
>>> nTests = 3
>>> p_critical = spm1d.util.p_critical_bonf(alpha, nTests)
```

The resulting critical *p* value is **0.016952**.

*Post hoc* tests will be deemed significant if they reach the critical *t* value implied by the p = 0.016952.

Test statistics may be computed for the three group pairs as follows:

Note

The *equal_var* option should be the same as was selected for ANOVA.

```
>>> t12 = spm1d.stats.ttest2(Y1, Y2, equal_var=False)
>>> t13 = spm1d.stats.ttest2(Y1, Y3, equal_var=False)
>>> t23 = spm1d.stats.ttest2(Y2, Y3, equal_var=False)
```

After computing the test statistics we can conduct inference using the corrected critical *p* value:

Warning

The *two_tailed* option must always be *True* in *post hoc* analyses; one-tailed inference is inconsistent with the null-hypothesis implied by ANOVA.

```
>>> t12i = t12.inference(alpha=p_critical, two_tailed=True)
>>> t13i = t13.inference(alpha=p_critical, two_tailed=True)
>>> t23i = t23.inference(alpha=p_critical, two_tailed=True)
```

In this case all tests reach significance.

These results may be interpreted exactly as in the case of a single t test: if there were truly no group differences, then smooth, random 1D continua would produce SPM{t}s that reaches the critical *t* threshold in fewer than *alpha* = 5% of many repeated experiments.

Warning

p values for *post hoc* tests.

If desired, you may report exact *p* values for each suprathreshold cluster, but in this case the p values must be adjusted for multiple comparisons.

To compute appropriate *p* values, use the inverse to the Bonferroni correction through **spm1d.util.p_corrected_bonf**, as described below.

Let’s consider the following result from one of the *post hoc* tests conducted above:

```
SPM{T} inference field
SPM.z : (1x101) raw test stat field
SPM.df : (1, 33.641)
SPM.fwhm : 4.82031
SPM.resels : (1, 20.74555)
Inference:
SPM.alpha : 0.017
SPM.zstar : 4.05216
SPM.p : (<0.001, <0.001, <0.001, 0.008, <0.001)
```

Note

When multiple *p* values exist, they are listed in the following order: (1) upper threshold left-to-right, (2) lower threshold left-to-right. In the “t23i” case above (rightmost panel of the figure) there are five *p* values. The first corresponds to the single upper-threshold cluster. The next four correspond to the four lower-threshold clusters.

The five *p* values listed in the results above correspond to this test’s five supra-threshold clusters.

Corrected *p* values may be computed as follows:

```
>>> p_value = 0.008
>>> nTests = 3
>>> p_value_corrected = spm1d.util.p_corrected_bonf(p_value, nTests)
```

**The corrected p value is 0.028**. If this cluster had precisely touched the threshold, its *p* value would have been *alpha*.