Welch t Tests

Gastric Example

A clinical trial was conducted to compare a new pain relief drug compound to the standard and to a combination containing 50%-50% mixture of the two. Fifteen subjects were randomly selected and five were randomly assigned to each of the three treatment groups. Each subject was given a fixed amount of drug per day for each of the seven days. At the end of the seven day period gastric (G) and duodenal (D) irritation were both determined by endoscopic techniques by a gastroenterologist using a five point scale 0 to 4 with 0 = normal.

Program Listing

OPTION LS=76 NODATE NONUMBER NOCENTER;

DATA gastric;
 DO patient = 1 TO 15;
   INPUT group gastric duodenal @@;
   OUTPUT;
 END;
 DATALINES;
 1 2 3   1 2 5   1 1 1   1 3 4   1 2 2   2 0 1   2 1 1
 2 1 1   2 0 0   2 2 0   3 0 0   3 0 0   3 1 1   3 1 1   3 1 0
;

PROC FORMAT;   /* Create values for variable `group' */
 VALUE drug 1='New Drug'  2='Standard'  3='Combination';

TITLE1 'One-Way ANOVA';
TITLE2 'Gastric Example';
PROC GLM DATA=gastric;
 FORMAT group drug.;
 CLASS group;
 MODEL gastric duodenal = group;
 RUN;
 TITLE3 'Brown-Forsythe Homogeneous Variance Test';
 MEANS group/HOVTEST=BF;   /* homogeneous variance test */
 RUN;
 TITLE3 'Least Square Means';
 LSMEANS group/PDIFF;   /* tij using MSE */
 RUN;
RUN;
QUIT;

%MACRO welch(data=_LAST_, response=, factor=, first=1, second=2,
             alpha=0.05, fmt=);
 PROC TTEST DATA=&data ALPHA=α
   TITLE1 "Welch t Test for &data Data";
   TITLE2 "for mu[&first] - mu[&second] = 0";
   CLASS &factor;
   WHERE &factor IN (&first,&second);
   VAR &response;
   FORMAT &factor &fmt.;
 RUN;
%MEND;

%welch(factor=group, response=duodenal, fmt=drug., alpha=0.10)
%welch(data=gastric,response=duodenal,factor=group,second=3,
       fmt=drug.,alpha=0.10)
%welch(data=gastric,response=duodenal,factor=group,first=2,second=3,
       fmt=drug.,alpha=0.10)

Output Listing

One-Way ANOVA
Gastric Example
The GLM Procedure
 
Dependent Variable: gastric

                                                                 
                            Sum of
Source            DF       Squares   Mean Square  F Value  Pr > F
Model              2    5.73333333    2.86666667     5.73  0.0179 *
Error             12    6.00000000    0.50000000                 
Corrected Total   14   11.73333333                               


R-Square     Coeff Var      Root MSE    gastric Mean
0.488636      62.39177      0.707107        1.133333


Source        DF     Type I SS   Mean Square  F Value  Pr > F
group          2    5.73333333    2.86666667     5.73  0.0179

Source        DF   Type III SS   Mean Square  F Value  Pr > F
group          2    5.73333333    2.86666667     5.73  0.0179


One-Way ANOVA
Gastric Example
The GLM Procedure
 
Dependent Variable: duodenal

                                                                 
                            Sum of
Source            DF       Squares   Mean Square  F Value  Pr > F
Model              2   20.93333333   10.46666667    10.13  0.0027 **
Error             12   12.40000000    1.03333333                 
Corrected Total   14   33.33333333                               

R-Square     Coeff Var      Root MSE    duodenal Mean
0.628000      76.23975      1.016530         1.333333

Source        DF     Type I SS   Mean Square  F Value  Pr > F
group          2   20.93333333   10.46666667    10.13  0.0027

Source        DF   Type III SS   Mean Square  F Value  Pr > F
group          2   20.93333333   10.46666667    10.13  0.0027


One-Way ANOVA
Gastric Example
Brown-Forsythe Homogeneous Variance Test
The GLM Procedure

Brown and Forsythe's Test for Homogeneity of gastric Variance
       ANOVA of Absolute Deviations from Group Medians
                                                             
                      Sum of        Mean
Source        DF     Squares      Square    F Value    Pr > F
group          2      0.1333      0.0667       0.22    0.8040
Error         12      3.6000      0.3000                     

Brown and Forsythe's Test for Homogeneity of duodenal Variance
       ANOVA of Absolute Deviations from Group Medians
                                                             
                      Sum of        Mean
Source        DF     Squares      Square    F Value    Pr > F
group          2      2.1333      1.0667       2.46    0.1271 *(25%)
Error         12      5.2000      0.4333                     


One-Way ANOVA
Gastric Example
Brown-Forsythe Homogeneous Variance Test

The GLM Procedure

                                                                      
Level of         --------gastric----------   --------duodenal---------
group        N         Mean        Std Dev         Mean        Std Dev
Combination  5   0.60000000     0.54772256   0.40000000     0.54772256
New Drug     5   2.00000000     0.70710678   3.00000000     1.58113883
Standard     5   0.80000000     0.83666003   0.60000000     0.54772256


One-Way ANOVA
Gastric Example
Least Square Means

The GLM Procedure
Least Squares Means
                    gastric      LSMEAN
group                LSMEAN      Number
Combination      0.60000000           1
New Drug         2.00000000           2
Standard         0.80000000           3


     Least Squares Means for effect group
     Pr > |t| for H0: LSMean(i)=LSMean(j)
                       
         Dependent Variable: gastric
i/j              1             2             3
   1                      0.0087        0.6627
   2        0.0087                      0.0199
   3        0.6627        0.0199              

                   duodenal      LSMEAN
group                LSMEAN      Number
Combination      0.40000000           1
New Drug         3.00000000           2
Standard         0.60000000           3

     Least Squares Means for effect group
     Pr > |t| for H0: LSMean(i)=LSMean(j)
                       
         Dependent Variable: duodenal
i/j              1             2             3
   1                      0.0016        0.7611
   2        0.0016                      0.0029
   3        0.7611        0.0029              

NOTE: To ensure overall protection level, only probabilities associated 
      with pre-planned comparisons should be used.


Welch t Test for _LAST_ Data
for mu[1] - mu[2] = 0

The TTEST Procedure
                           Statistics
                         Lower CL          Upper CL  Lower CL
Variable  group       N      Mean    Mean      Mean   Std Dev  Std Dev
duodenal  New Drug    5    1.4926       3    4.5074    1.0266   1.5811
duodenal  Standard    5    0.0778     0.6    1.1222    0.3556   0.5477
duodenal  Diff (1-2)       1.0084     2.4    3.7916    0.8498   1.1832

                           Statistics
                       Upper CL
Variable  group         Std Dev    Std Err    Minimum    Maximum
duodenal  New Drug        3.751     0.7071          1          5
duodenal  Standard       1.2994     0.2449          0          1
duodenal  Diff (1-2)     2.0245     0.7483

                               T-Tests
Variable    Method           Variances      DF    t Value    Pr > |t|
duodenal    Pooled           Equal           8       3.21      0.0125
duodenal    Satterthwaite    Unequal      4.95       3.21      0.0242

                    Equality of Variances
Variable    Method      Num DF    Den DF    F Value    Pr > F
duodenal    Folded F         4         4       8.33    0.0640 *(25%)


Welch t Test for gastric Data
for mu[1] - mu[3] = 0

The TTEST Procedure
                           Statistics
                         Lower CL          Upper CL  Lower CL
Variable  group       N      Mean    Mean      Mean   Std Dev  Std Dev
duodenal  New Drug    5    1.4926       3    4.5074    1.0266   1.5811
duodenal  Combination 5    -0.122     0.4    0.9222    0.3556   0.5477
duodenal  Diff (1-2)       1.2084     2.6    3.9916    0.8498   1.1832

                           Statistics
                       Upper CL
Variable  group         Std Dev    Std Err    Minimum    Maximum
duodenal  New Drug        3.751     0.7071          1          5
duodenal  Combination    1.2994     0.2449          0          1
duodenal  Diff (1-2)     2.0245     0.7483

                               T-Tests
Variable    Method           Variances      DF    t Value    Pr > |t|
duodenal    Pooled           Equal           8       3.47      0.0084
duodenal    Satterthwaite    Unequal      4.95       3.47      0.0181

                    Equality of Variances
Variable    Method      Num DF    Den DF    F Value    Pr > F
duodenal    Folded F         4         4       8.33    0.0640 *(25%)


Welch t Test for gastric Data
for mu[2] - mu[3] = 0

The TTEST Procedure
                          Statistics
                         Lower CL          Upper CL  Lower CL
Variable  group       N      Mean    Mean      Mean   Std Dev  Std Dev
duodenal  Standard    5    0.0778     0.6    1.1222    0.3556   0.5477
duodenal  Combination 5    -0.122     0.4    0.9222    0.3556   0.5477
duodenal  Diff (1-2)       -0.444     0.2    0.8442    0.3934   0.5477

                           Statistics
                       Upper CL
Variable  group         Std Dev    Std Err    Minimum    Maximum
duodenal  Standard       1.2994     0.2449          0          1
duodenal  Combination    1.2994     0.2449          0          1
duodenal  Diff (1-2)     0.9372     0.3464

                               T-Tests
Variable    Method           Variances      DF    t Value    Pr > |t|
duodenal    Pooled           Equal           8       0.58      0.5796
duodenal    Satterthwaite    Unequal         8       0.58      0.5796

                    Equality of Variances
Variable    Method      Num DF    Den DF    F Value    Pr > F
duodenal    Folded F         4         4       1.00    1.0000

The variances for response variable duodenal seem to be heterogeneous. Hence the pairwise comparisons of least square means above are inappropriate. The above pairwise Welch (Satterthwaite) t are used for pairwise comparisons. However, the reported P values for Welch t tests are individual c.i.'s. To produce Bonferroni-Welch t test results, one can do one the following two ways (level α = 0.10 tests assumed)

  1. Compare each P value with α/3 = 0.0333. With 1=New Drug, 2=Standard, 3=Combination, the comparisons μ12 (Welch P value = 0.0242) and μ13 (Welch P value = 0.0181) are significant (at family error rate of α = 0.10).
  2. Produce adjusted P values by multiplying each P value by the number of comparisons (3 here). This is called the Bonferroni adjustment. Then compare each adjusted P value with α. The same conclusions (at family error rate of α = 0.10) are reached because the adjusted P values for comparisons μ12 (Bonferroni-Welch P value = 3×0.0242) and μ13 (Bonferroni-Welch P value = 3×0.0181) are, respectively, 0.0726 and 0.0543.
Note also that SAS uses non-integer degrees of freedom to compute P values. The confidence intervals reported SAS output from TTEST procedures are 90% individual Welch confidence intervals. To get 90% Bonferroni-Welch confidence intervals, do adjustment accordingly:

B-W c.i. = est. ± t.05/3,df/t.05,df ×(half-width of W c.i.)