Applied Multivariate Statistical Analysis Chapter 6 - Comparisons of Several Multivariate Means

Introduction

The comparison of several multivariate means is comparing multivariate means from different populations. Comparing, here, means whether there is a statistically signficant difference between the effects of the two populations or if they are negligible due to random variation. A lot of these techniques use the same logic of confidence regions and confidence intervals to compare between two or means. Like the univariate case, for two population means, it is better to use a t-test for p-values and for more than 2 population means, to use Analysis of Variance (ANOVA). Although, it sounds like ANOVA tests just variance, it actually compares variances across the means of different groups. This chapter goes through different experimental designs for comparing different amounts of population means, starting from simple paired comparisons to multivariate means from more than 2 population means consisting of two or more factors (not to be confused with factor analysis).

Paired Comparisons and a Repeated Measures Design

Paired comparisons can be expressed as two treatments such as before or after a treatment (rather than as two populations). Hence, $X_{1j1}$ is variable 1 under treatment 1 and $X_{2jp}$ is variable p under treatment 2. Treatments just signify a change in experimental environment. To compare these two sample means, we calculate the differences and use the $T^2$-statistic to test if it is greater than some F-distribution to reject the null hypothesis $H_0: \delta = 0$ (the differences are 0). Using the differences for each $\delta_i$ (the differences between each treatment for the same variable), we can also calculate confidence regions, simultaneous confidence intervals, and Bonferonni simultaneous confidence intervals.

Repeated measures design is when all treatments are administered to each unit. Here, we now have more than two treatments and use contrast matrices to algebraically convey differences and put this as our $T^2 = n(C\bar{x})(CSC^T)^{-1}(C\bar{x})$. We compare this with an F-distribution and can create a confidence region and simultaneous confidence intervals.

Comparing Mean Vectors from Two Populations.

In this section, we are also using $T^2$-statistic to reject the null hypothesis here. This time, there are assumptions concerning the structure of the data:

1. The sample $X_{11}, \ldots, X_{1n_1}$ is a random sample of size $n_1$, from a p-variate population with mean vector $\mu_1$ and covariance matrix $\Sigma_1$. (same for population 2)
2. $X_{11}, \ldots, X_{1n_1}$ is independent of $X_{21}, \ldots, X_{2n_2}$

When $n_1$ and $n_2$ are small, we must further assume that the populations are multivariate normal and they have the same covariance matrix $\Sigma_1 = \Sigma_2$. With this assumption, we can pool the covariance matrices to estimate a common covariance matrix. Use the $T^2 = (\bar{x}1-\bar{x}_2-\delta_0)^T[(\frac{1}{n_1} + \frac{1}{n_2})S{pooled}]^{-1}(\bar{x}1-\bar{x}_2-\delta_0) > c^2$ where $c^2$ is expressed as the F-distribution. This rejects the null hypothesis $H_0: \mu_1 = \mu_2$ or $\mu_1 - \mu_2 = 0$. Again, you can make a confidence ellipse and if it doesn’t contain the origin then this cn also reject the null hypothesis and can use the location of the ellipse in relation to the axes to make inferences between the two population means. The simultaneous confidence intervals use $S{pooled}$ but still uses the same logic. There is also a way of expressing the quadratic form distance if the covariance matrices are unequal and another for when sample sizes are not large.

Comparing Several Multivariate Population Means

This is used for comparing more than two populations. The assumptions are

1. each random sample is from some population with a mean and that random samples from different populations are independent
2. all populations have the same covariance matrix
3. each population is multivariate normal

The univariate and multivariate versions of ANOVA are nearly identical in the fact their formula is of the same structure and terms but replaces the squared terms with the corresponding outer products. \(\text{structure:} \\ X_{\ell j} = \mu + \tau_\ell + e_{\ell j} \\ X_{\ell j} = \text{(overall mean) + (treatment effect) + (random error)} \\ x_{\ell j} = \bar{x} + (\bar{x}_\ell - \bar{x}) + (x_{\ell j} - \bar{x}_\ell) \\ \text{(observation) = (overall sample mean) + (estimated treatmenet effect) + (residual)}\)

This method uses the sum of squares and cross products to equal the treatment Between sum of squares and cross products and the residual Within sum of squares and cross products to rewrite the MANOVA table in terms of the sources of variations being the treatment and residual(error). Instead of using the $T^2$-statistic, to reject the null hypothesis $H_0: \tau_1 = \ldots = \tau_g = 0$ ($\tau$ is the treatment effect), we use ratio of generalised variances and Wilks’ lambda $\Lambda^* = \frac{

W

}{

B+W

}$

Simultaneous Confidence Intervals for Treatment Effects

This uses the t-distribution to measure the simultaneous confidence intervals for some $\hat{\tau}{ik} - \hat{\tau}{\ell i}$.

Testing for Equality of Covariance Matrices

To test for equality of covariance matrices, we use Box’s M-test (which uses the $\chi_p^2$-distribution)to reject the null hypothesis $H_0: \Sigma_1 = \ldots = \Sigma_g = \Sigma$. We must do this if we want to be able to pool the sample covariances in the previous methods, since this is done only by assuming equality of covariance matrices.

Two-Way Multivariate Analysis of Variance

The MANOVA used before was one-way since it only measures each population mean having one factor or set of experimental conditions. First, we test the interaction effects between factors then the individual factors themselves. We use Wilks’ lambda here as the likelihood ratio test to reject the null hypothesis $H_0: \gamma_{11} = \gamma_{12} = \ldots = \gamma_{gb} = 0$ by $\Lambda^* = \frac{|SSP_{res}|}{SSP_{int}+SSP_{res}}$ (for interaction effects). For individual factors, the null hypothesis of treatment effects $H_0:\tau_1 = \tau_2 = \ldots = \tau_g = 0$ for factor 1 and $H_0: \beta_1 = \beta_2 = \ldots = \beta_b = 0$ for factor 2, for instance. We use the formula described before (Wilks’ lambda) to test these hypotheses too. The structure of this equation is now: \(E(X_{\ell k r}) = \mu+ \tau_\ell + \beta_k + \gamma_{\ell k} \\ \text{(mean response) = (overall level) + (effect of factor 1) + (effect of factor 2) + (factor 1-factor 2 interaction)}\)

Profile Analysis

Profile analysis is used when $p$ treatments are administered to two or more groups of subjects. There are three null hypotheses:

1. Are the profiles parallel? $H_{01}: \mu_{1i} - \mu_{1i-1} = \mu_{2i} - \mu_{2i-1}$. This just uses the $T^2$-distribution with a contrast matrix against and F-distribution.
2. Assuming the profiles are parallel, are the profile coincident? $H_{02}: \mu_{1i} = \mu_{2i}$. This uses the $\textbf{1}$ matrix and is the $T^2$-distribution against the F-distribution.
3. Assuming the profiles are coincident, are the profiles level? $H_{03}: \mu_11 = \mu_12 = \ldots = \mu_{1p} = \mu_{21} = \mu_{22} = \ldots = \mu_{2p}$. This uses a contrast matrix in quadratic form against an F-distribution.

Repeated Measures Designs and Growth Curves

Nothing here.

Perspectives and a Strategy for Analysing Multivariate Models

Instead of using Wilks’ lambda statistic for MANOVA between the matrices “Between” $B$ and “Within” $W$, there are also Lawley-Hotelling trace, Pillai trace, Roy’s largest root. A strategy for multivariate comparison of treatments are

1. Try to identify outliers. Check data group and residual vectors for outliers to calculate with and without them.
2. Perform a multivariate test of hypothesis. The choice they give is to use likelihood ratio test which is equivalent to Wilks’ lambda test.
3. Calculate the Bonferonni simultaneous confidence intervals. If the multivariate test reveals a difference, then calculate the Bonferonni confidence intervals for all pairs of groups/treatments, and all characteristics. If no differences significant, look at Bonferonni confidence intervals for larger set of response, including differences and sums of pairs of responses. Note: differences may only appear in one of many characteristics, so do not include too many other variables not expected to show differences among treatments.