# Multi-Way Mixed-Effect Model

### Turbine Example

The pressure drop experienced across an expansion valve in a turbine is influenced by gas temperature on the inlet side, turbine speed, and inlet gas pressure. These three factors are arranged in a factorial design, with temperature fixed, and pressure and speed random. The coded data (from Montgomery) are given.

#### Program Listing

```OPTIONS LS=76 NONUMBER NODATE NOCENTER;

DATA turbine (DROP=rep);
DO temp = 60 TO 90 BY 15;        /* 3 fixed gas temperature levels */
DO pressure = 50, 75, 85;             /* 3 random pressure levels */
DO speed = 150, 200, 225, 300;  /* 4 random turbine speed levels */
DO rep = 1 TO 2;   /* 2 replicates */
INPUT drop @@;
OUTPUT;
END;
END;
END;
END;
DATALINES;
-2 -3  0 -9 -1 -8  4  4 -6  4 -5 -1 -8 -2 -3 -7 -1 -2 -4 -8  0 -7
-2  4 14 14  6  0  1  2 -7  6 22 24  8  6  6  2 -5  2 20 16  2  0
3  0 -5 -1 -8 -8 -2 20 -1 -2 -2  1 -8  3  1 -7 -9 -8 -8  3 -2 -1
-1 -2 -4 -7  1  3
;

TITLE1 'Multi-Way Mixed Effects Model';
TITLE2 'Turbine Example';
PROC GLM DATA=turbine;
CLASS temp pressure speed;
TITLE3 'ANOVA Table (fixed effect model)';
MODEL drop = temp | pressure | speed;
RUN;
TITLE3 'Random Effects with Correct Tests';
RANDOM  pressure|speed
temp*pressure temp*speed temp*pressure*speed
/ TEST;
RUN;
TITLE3 'Tests by User-Specified Error Term';
TEST H=temp*pressure temp*speed pressure*speed
E=temp*pressure*speed;
RUN;
QUIT;

TITLE3 'Using PROC VARCOMP';
PROC VARCOMP DATA=turbine;
CLASS temp pressure speed;
MODEL drop = temp | pressure | speed / FIXED=1;
RUN;

TITLE3 'Using PROC MIXED';
PROC MIXED DATA=turbine CL METHOD=REML;
CLASS temp pressure speed;
MODEL drop = temp / CL    /* Confidence Limits for fixed effects */
DDFM=SATTERTH;   * Satterthwaite method for
Denominator Degrees of Freedom;
RANDOM  temp*pressure temp*speed pressure|speed;
RUN;
```

Note that the RANDOM statement option TEST provides correct F tests including Satterthwaite approximate F tests. The TEST statement requests tests with user-specified error term (can only specify one error term).

#### Output Listing

##### from Proc GLM
```Multi-Way Mixed Effects Model
Turbine Example
ANOVA Table (fixed effect model)

The GLM Procedure

Class Level Information

Class         Levels    Values
temp               3    60 75 90
pressure           3    50 75 85
speed              4    150 200 225 300

Number of Observations Used          72
Multi-Way Mixed Effects Model
Turbine Example
ANOVA Table (fixed effect model)

The GLM Procedure

Dependent Variable: drop

Sum of
Source            DF       Squares  Mean Square  F Value  Pr > F
Model             35   3179.819444    90.851984     4.24  <.0001
Error             36    770.500000    21.402778
Corrected Total   71   3950.319444

R-Square     Coeff Var      Root MSE     drop Mean
0.804952      1332.378      4.626314      0.347222

Source                DF    Type I SS  Mean Square  F Value  Pr > F
temp                   2  1023.361111   511.680556    23.91  <.0001
pressure               2     7.194444     3.597222     0.17  0.8460
temp*pressure          4   137.888889    34.472222     1.61  0.1927
speed                  3   423.819444   141.273148     6.60  0.0011
temp*speed             6  1211.972222   201.995370     9.44  <.0001
pressure*speed         6   209.472222    34.912037     1.63  0.1669
temp*pressure*speed   12   166.111111    13.842593     0.65  0.7882

Source                DF  Type III SS  Mean Square  F Value  Pr > F
temp                   2  1023.361111   511.680556    23.91  <.0001
pressure               2     7.194444     3.597222     0.17  0.8460
temp*pressure          4   137.888889    34.472222     1.61  0.1927
speed                  3   423.819444   141.273148     6.60  0.0011
temp*speed             6  1211.972222   201.995370     9.44  <.0001
pressure*speed         6   209.472222    34.912037     1.63  0.1669
temp*pressure*speed   12   166.111111    13.842593     0.65  0.7882

Multi-Way Mixed Effects Model
Turbine Example
Random Effects with Correct Tests

The GLM Procedure

Source               Type III Expected Mean Square
temp                 Var(Error) + 2 Var(temp*pressure*speed) + 6
Var(temp*speed) + 8 Var(temp*pressure) + Q(temp)
pressure             Var(Error) + 2 Var(temp*pressure*speed) + 6
Var(pressure*speed) + 8 Var(temp*pressure) + 24
Var(pressure)
temp*pressure        Var(Error) + 2 Var(temp*pressure*speed) + 8
Var(temp*pressure)
speed                Var(Error) + 2 Var(temp*pressure*speed) + 6
Var(pressure*speed) + 6 Var(temp*speed) + 18
Var(speed)
temp*speed           Var(Error) + 2 Var(temp*pressure*speed) + 6
Var(temp*speed)
pressure*speed       Var(Error) + 2 Var(temp*pressure*speed) + 6
Var(pressure*speed)
temp*pressure*speed  Var(Error) + 2 Var(temp*pressure*speed)

Multi-Way Mixed Effects Model
Turbine Example
Random Effects with Correct Tests

The GLM Procedure
Tests of Hypotheses for Mixed Model Analysis of Variance

Dependent Variable: drop

Source          DF   Type III SS   Mean Square  F Value  Pr > F
temp             2   1023.361111    511.680556     2.30  0.1712
Error       6.9674   1551.115597    222.625000
Error: MS(temp*pressure) + MS(temp*speed) -
MS(temp*pressure*speed) - 21E-16*MS(Error)

Source          DF   Type III SS   Mean Square  F Value  Pr > F
pressure         2      7.194444      3.597222     0.06  0.9379
Error       5.9762    331.928320     55.541667
Error: MS(temp*pressure) + MS(pressure*speed)
- MS(temp*pressure*speed) - 36E-16*MS(Error)

Source            DF   Type III SS   Mean Square  F Value  Pr > F
temp*pressure      4    137.888889     34.472222     2.49  0.0991
temp*speed         6   1211.972222    201.995370    14.59  <.0001
pressure*speed     6    209.472222     34.912037     2.52  0.0814
Error             12    166.111111     13.842593
Error: MS(temp*pressure*speed)

Source          DF   Type III SS   Mean Square  F Value  Pr > F
speed            3    423.819444    141.273148     0.63  0.6164
Error       7.0886   1581.208780    223.064815
Error: MS(temp*speed) + MS(pressure*speed) -
MS(temp*pressure*speed) - 29E-16*MS(Error)

Source                DF  Type III SS  Mean Square  F Value  Pr > F
temp*pressure*speed   12   166.111111    13.842593     0.65  0.7882
Error: MS(Error)      36   770.500000    21.402778

Multi-Way Mixed Effects Model
Turbine Example
Tests by User-Specified Error Term

Dependent Variable: drop

Tests of Hypotheses Using the Type III MS
for temp*pressure*speed as an Error Term

Source            DF   Type III SS  Mean Square  F Value  Pr > F
temp*pressure      4    137.888889    34.472222     2.49  0.0991
temp*speed         6   1211.972222   201.995370    14.59  <.0001
pressure*speed     6    209.472222    34.912037     2.52  0.0814
```
##### from Proc VARCOMP
```Multi-Way Mixed Effects Model
Turbine Example
Using PROC VARCOMP

Variance Components Estimation Procedure

Class Level Information

Class         Levels    Values
temp               3    60 75 90
pressure           3    50 75 85
speed              4    150 200 225 300

Number of Observations Used          72

MIVQUE(0) SSQ Matrix

Source                      pressure    temp*pressure
pressure                      1152.0        384.00000
temp*pressure              384.00000        384.00000
speed                     1.7821E-31       1.5236E-30
temp*speed                8.0749E-30       1.2603E-29
pressure*speed             288.00000         96.00000
temp*pressure*speed         96.00000         96.00000
Error                       48.00000         48.00000

MIVQUE(0) SSQ Matrix

Source                         speed       temp*speed
pressure                  1.7821E-31       8.0749E-30
temp*pressure             1.5236E-30       1.2603E-29
speed                      972.00000        324.00000
temp*speed                 324.00000        324.00000
pressure*speed             324.00000        108.00000
temp*pressure*speed        108.00000        108.00000
Error                       54.00000         54.00000

MIVQUE(0) SSQ Matrix

Source                pressure*speed  temp*pressure*speed

pressure                   288.00000             96.00000
temp*pressure               96.00000             96.00000
speed                      324.00000            108.00000
temp*speed                 108.00000            108.00000
Multi-Way Mixed Effects Model
Turbine Example
Using PROC VARCOMP

Variance Components Estimation Procedure

MIVQUE(0) SSQ Matrix

Source                pressure*speed  temp*pressure*speed
pressure*speed             396.00000            132.00000
temp*pressure*speed        132.00000            132.00000
Error                       66.00000             66.00000

MIVQUE(0) SSQ Matrix

Source                         Error             drop
pressure                    48.00000        172.66667
temp*pressure               48.00000           1160.7
speed                       54.00000           7628.8
temp*speed                  54.00000           9814.8
pressure*speed              66.00000           3842.9
temp*pressure*speed         66.00000           4312.9
Error                       69.00000           2927.0

MIVQUE(0) Estimates

Variance Component                drop
Var(pressure)                 -2.16435
Var(temp*pressure)             2.57870
Var(speed)                    -4.54398
Var(temp*speed)               31.35880
Var(pressure*speed)            3.51157
Var(temp*pressure*speed)      -3.78009
Var(Error)                    21.40278
```

Note that some variance component estimates are negative, so they are effectively zero. Proc MIXED will set them zeros and adjust estimates of nonzero variance components.

##### from Proc MIXED
```Multi-Way Mixed Effects Model
Turbine Example
Using PROC MIXED

The Mixed Procedure

Model Information

Data Set                     WORK.TURBINE
Dependent Variable           drop
Covariance Structure         Variance Components
Estimation Method            REML
Residual Variance Method     Profile
Fixed Effects SE Method      Model-Based
Degrees of Freedom Method    Satterthwaite

Class Level Information

Class       Levels    Values
temp             3    60 75 90
pressure         3    50 75 85
speed            4    150 200 225 300

Dimensions

Covariance Parameters             6
Columns in X                      4
Columns in Z                     40
Subjects                          1
Max Obs Per Subject              72

Number of Observations

Number of Observations Used              72
Number of Observations Not Used           0

Iteration History

Iteration    Evaluations    -2 Res Log Like       Criterion
0              1       463.93294480
1              3       435.93675347      0.00024970
2              2       435.90198225      0.00000446
3              1       435.90128526      0.00000000

Multi-Way Mixed Effects Model
Turbine Example
Using PROC MIXED

The Mixed Procedure

Convergence criteria met.

Covariance Parameter Estimates

Cov Parm           Estimate     Alpha       Lower       Upper
temp*pressure        0.7068      0.05     0.07785    9.542E11
temp*speed          26.7428      0.05     11.6681      111.41
pressure                  0         .           .           .
speed                     0         .           .           .
pressure*speed       1.2668      0.05      0.1904      458255
Residual            19.9934      0.05     13.7525     31.7258

Fit Statistics

-2 Res Log Likelihood           435.9
AIC (smaller is better)         443.9
AICC (smaller is better)        444.5
BIC (smaller is better)         444.7

Solution for Fixed Effects

Standard
Effect      temp  Estimate    Error    DF  t Value  Pr > |t|   Alpha
Intercept          -2.0000   2.8036  9.28    -0.71    0.4931    0.05
temp        60     -0.6250   3.9381  9.08    -0.16    0.8774    0.05
temp        75      7.6667   3.9381  9.08     1.95    0.0831    0.05
temp        90           0        .     .      .       .           .

Solution for Fixed Effects

Effect      temp      Lower       Upper
Intercept           -8.3125      4.3125
temp        60      -9.5213      8.2713
temp        75      -1.2297     16.5630
temp        90            .           .

Multi-Way Mixed Effects Model
Turbine Example
Using PROC MIXED

The Mixed Procedure

Type 3 Tests of Fixed Effects

Num     Den
Effect         DF      DF    F Value    Pr > F
temp            2    9.08       2.75    0.1165
```