22 Categorical Predictors

Used to include predictors (independent variables) measured on a nominal scale, such as a treatment group, a type of school, or a region, in a regression model. A regression multiplies each predictor by a slope, which requires numbers. A categorical predictor must therefore first be turned into one or more numerical variables, a process called coding. A predictor with k categories is always represented by k - 1 coded variables. One category is left without its own variable, because its information is already contained in the others11 An employee who does not work in the office or in a hybrid arrangement works remotely. A third variable for remote work would add no information and would make the model impossible to estimate..

The way the categories are coded does not change how well the model fits the data: R^2, the F test of the model, and the predicted values are the same for any coding. What changes is the meaning of the intercept and of the slopes. Choosing the coding is therefore choosing which questions the coefficients answer.

22.1 Coding Schemes

The three most common coding schemes are shown in Table 22.1 for a predictor with three categories, A, B, and C:

• Dummy coding (also called treatment coding) gives a value of 1 to the cases in one category and 0 to all others. One category, the reference category, is coded 0 on all variables. The intercept is the mean of the reference category, and each slope is the difference between the mean of one category and the mean of the reference category22 It is the default in most statistical software.;

  2 It is the default in most statistical software.

• Effect coding (also called sum-to-zero coding) works like dummy coding, but the reference category is coded -1 on all variables. The intercept is the grand mean, the mean of the category means, and each slope is the difference between the mean of one category and the grand mean;
• Contrast coding uses weights chosen by the researcher to test specific comparisons, following the rules for planned contrasts: the weights of each contrast add up to zero, and the contrasts are orthogonal to each other.

22.2 The Study

A mid-size company introduced flexible work arrangements and, a year later, surveyed its employees. Ninety employees, 30 in each arrangement, reported their job satisfaction on a scale from 0 to 100. Some worked full time in the office (Office), some worked from home two or three days a week (Hybrid), and some worked from home full time (Remote). The survey also asked whether they had children living at home and how many years they had worked for the company. The questions asked are: Does job satisfaction differ between the work arrangements? And does it differ once the other characteristics of the employees are taken into account? The data set33 Column labels are: employee, arrangement, children, years, and satisfaction. was made up for this example. The first few rows of the data set are shown in Table 22.2.

  Office   Hybrid   Remote 
60.03333 71.26667 68.53333 

      No      Yes 
69.72917 63.04762 

The office employees are the least satisfied (M = 60.03), and the hybrid employees the most (M = 71.27). Employees with children at home report lower satisfaction than those without (M = 63.05 vs. M = 69.73).

Of note is the fact that the employees chose their work arrangement. The analyses below show how satisfaction differs between the arrangements, not that an arrangement makes employees more or less satisfied44 Employees who chose to work remotely may differ from the others in many ways, for example in the length of their commute or in their family situation. See A Few Words About Causality..

22.3 A Predictor With Two Categories

Let’s start with the simplest case, a predictor with two categories: having children at home or not. With dummy coding, the employees without children (No) are coded 0 and form the reference category, and those with children (Yes) are coded 1:

\widehat{satisfaction} = b_0 + b_1 \times Children_{Yes}

             Estimate Std. Error   t value     Pr(>|t|)
(Intercept) 69.729167   1.505780 46.307675 1.406897e-63
childrenYes -6.681548   2.204236 -3.031231 3.198661e-03

The equation can be solved for each category:

• For employees without children, Children_{Yes} = 0, and the predicted satisfaction is the intercept, 69.73, the mean of the No group;
• For employees with children, Children_{Yes} = 1, and the predicted satisfaction is 69.73 + (-6.68) = 63.05, the mean of the Yes group.

The slope, -6.68, is the difference between the two means, and its t-Test is the test of that difference. It is the independent samples t-Test in its equal-variance version (Student’s)55 Like the regression, it pools the variance of the two groups. Welch’s version, used by default in the t-Test chapter, does not, and gives slightly different results.: an independent samples t-Test on the same data, with equal variances assumed, gives t(88) = 3.03, p = .003, the same value as the slope, with the opposite sign66 The sign depends only on which group is subtracted from which..

With effect coding, the No group is coded -1 and the Yes group 1:

             Estimate Std. Error   t value     Pr(>|t|)
(Intercept) 66.388393   1.102118 60.237099 2.540069e-73
children.e  -3.340774   1.102118 -3.031231 3.198661e-03

The intercept, 66.39, is now the grand mean, halfway between the two group means, and the slope, -3.34, is the distance from the grand mean to the mean of the Yes group, half of the difference between the groups. The t value and the p-value of the slope are exactly the same as with dummy coding. Only the meaning of the coefficients has changed.

22.4 A Predictor With Three Categories

22.4.1 Dummy Coding

The work arrangement has three categories, so it is represented by two dummy variables, with the first category, Office, as the reference category77 The dummy variables are created automatically by the software.:

Call:
lm(formula = satisfaction ~ arrangement, data = wa)

Residuals:
     Min       1Q   Median       3Q      Max 
-21.5333  -7.0833   0.9667   5.8417  27.4667 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)         60.033      1.807  33.230  < 2e-16 ***
arrangementHybrid   11.233      2.555   4.397 3.09e-05 ***
arrangementRemote    8.500      2.555   3.327  0.00129 ** 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 9.895 on 87 degrees of freedom
Multiple R-squared:  0.1947,    Adjusted R-squared:  0.1762 
F-statistic: 10.52 on 2 and 87 DF,  p-value: 8.126e-05

• The intercept, 60.03, is the mean of the reference category, Office;
• arrangementHybrid, 11.23, is the difference between the hybrid and the office employees, 71.27 - 60.03. It is significant (p < .001);
• arrangementRemote, 8.50, is the difference between the remote and the office employees, 68.53 - 60.03. It is also significant (p = .001).

Dummy coding is the natural choice when one category is a natural baseline, such as a control group or, here, the traditional arrangement. It does not compare the other categories with each other. To compare the hybrid and remote employees, the reference category has to be changed:

                                             Estimate Std. Error   t value
(Intercept)                                 71.266667   1.806576 39.448476
relevel(arrangement, ref = "Hybrid")Office -11.233333   2.554884 -4.396807
relevel(arrangement, ref = "Hybrid")Remote  -2.733333   2.554884 -1.069846
                                               Pr(>|t|)
(Intercept)                                2.686769e-57
relevel(arrangement, ref = "Hybrid")Office 3.094575e-05
relevel(arrangement, ref = "Hybrid")Remote 2.876477e-01

The hybrid employees are now the reference. The difference between the remote and the hybrid employees, -2.73, is not significant. Of note is the fact that comparing all pairs of categories this way runs several tests, and the familywise error rate should be controlled, as with the post hoc tests that follow an ANOVA.

22.4.2 Testing the Predictor as a Whole

The t-Tests of the coefficients test single differences between categories. The question does satisfaction differ between the work arrangements? is about the predictor as a whole, and is answered by comparing a model without the predictor with a model that includes it.

Hypotheses. H₀: In the population, the work arrangement explains none of the variance in job satisfaction; the slopes of both coded variables are zero. H₁: At least one of the slopes is different from zero.

Decision criteria. \alpha = .05. The two models are compared with an F test that has as many degrees of freedom as the coded variables added, k - 1 = 2. H₀ is rejected if the p-value is smaller than \alpha. The effect size is the increase in R^2.

Analysis of Variance Table

Model 1: satisfaction ~ 1
Model 2: satisfaction ~ arrangement
  Res.Df     RSS Df Sum of Sq      F    Pr(>F)    
1     89 10577.4                                  
2     87  8518.3  2    2059.1 10.515 8.126e-05 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Decision. The work arrangement explains a significant part of the variance in job satisfaction, F(2, 87) = 10.52, p < .001. This is the F test reported at the bottom of the regression output, and it is also the one-way ANOVA of satisfaction by arrangement.

Effect size. R^2 = 0.19. The work arrangement accounts for 19% of the variance in job satisfaction, a large effect.

22.4.3 Effect Coding

       [,1] [,2]
Office    1    0
Hybrid    0    1
Remote   -1   -1

              Estimate Std. Error   t value     Pr(>|t|)
(Intercept)  66.611111   1.043027 63.863261 7.061638e-75
arrangement1 -6.577778   1.475063 -4.459320 2.443386e-05
arrangement2  4.655556   1.475063  3.156174 2.196415e-03

With effect coding, the last category, Remote, is coded -1 on both variables. The coefficients read:

• The intercept, 66.61, is the grand mean, the mean of the three category means88 With groups of different sizes, the intercept is the unweighted mean of the category means, which is not the same as the mean of all employees.;

  8 With groups of different sizes, the intercept is the unweighted mean of the category means, which is not the same as the mean of all employees.

• arrangement1, -6.58, is the difference between the office employees and the grand mean: they are 6.58 points below it;
• arrangement2, 4.66, is the difference between the hybrid employees and the grand mean: they are 4.66 points above it.

The remote employees have no coefficient of their own. Because the deviations from the grand mean add up to zero, theirs is minus the sum of the other two: -(-6.58 + 4.66) = 1.92. Effect coding answers the question which categories are above or below the average?99 It is also required for the Type III sums of squares used in factorial designs.

22.4.4 Contrast Coding

Before the survey, the company had two specific questions. Are the employees who work from home, at least part of the time, more satisfied than those who work in the office? And among them, does it matter whether they work from home part time or full time? Each question is a planned contrast, and the two contrasts are orthogonal.

                           Estimate Std. Error   t value     Pr(>|t|)
(Intercept)               66.611111  1.0430271 63.863261 7.061638e-75
arrangementFlexVsOffice    3.288889  0.7375315  4.459320 2.443386e-05
arrangementHybridVsRemote  1.366667  1.2774421  1.069846 2.876477e-01

With contrast coding, the intercept is again the grand mean. The slopes depend on the scale of the weights, so they are easiest to read through their tests and through the differences they describe:

• FlexVsOffice compares the average of the hybrid and the remote employees (69.90) with the office employees (60.03). The flexible arrangements are associated with higher satisfaction, by 9.87 points, t(87) = 4.46, p < .001;
• HybridVsRemote compares the hybrid and the remote employees. The difference of 2.73 points is not significant, t(87) = 1.07, p = .288.

The two contrasts answer exactly the company’s questions, with two tests planned in advance and no correction of \alpha1010 Each contrast still has its own risk of a Type I error, so the chance of at least one in the pair is close to .10. A stricter analysis would test each one at \alpha = .025.. Of note is the fact that the F test of the model and R^2 are the same as with dummy and effect coding. All three models fit the data equally well; they only describe it differently.

22.5 Categorical and Continuous Predictors Together

Categorical predictors can be combined with each other and with continuous predictors, as in any multiple regression. Let’s return to the dummy coding, with the office employees as the reference category, and add the years with the company and having children at home.

Call:
lm(formula = satisfaction ~ years + children + arrangement, data = wa)

Residuals:
     Min       1Q   Median       3Q      Max 
-18.0316  -7.5806   0.1096   5.9769  28.7596 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)        58.8430     2.5943  22.682  < 2e-16 ***
years               0.6174     0.2299   2.685 0.008710 ** 
childrenYes        -4.8860     2.0172  -2.422 0.017550 *  
arrangementHybrid  10.5889     2.4774   4.274 4.99e-05 ***
arrangementRemote   8.9001     2.4522   3.629 0.000484 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 9.317 on 85 degrees of freedom
Multiple R-squared:  0.3025,    Adjusted R-squared:  0.2696 
F-statistic: 9.214 on 4 and 85 DF,  p-value: 3.114e-06

In a model with several predictors, each coefficient is interpreted holding the other predictors constant:

• years, 0.62: each additional year with the company is associated with 0.62 more points of satisfaction, for employees with the same family situation and work arrangement;
• childrenYes, -4.89: employees with children at home are 4.89 points less satisfied than those without, for the same years and work arrangement;
• arrangementHybrid, 10.59, and arrangementRemote, 8.90: the differences between the hybrid and the remote employees and the office employees, for the same years and family situation;
• The intercept, 58.84, is the predicted satisfaction of an office employee without children who has just joined the company.

Is the work arrangement still related to satisfaction once the other two predictors are taken into account? As before, the predictor is tested as a whole, by comparing the model without it with the model that includes it:

Analysis of Variance Table

Model 1: satisfaction ~ years + children
Model 2: satisfaction ~ years + children + arrangement
  Res.Df    RSS Df Sum of Sq      F    Pr(>F)    
1     87 9203.7                                  
2     85 7378.2  2    1825.5 10.516 8.306e-05 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Decision. The work arrangement explains a significant part of the variance in job satisfaction over and above the years with the company and having children at home, F(2, 85) = 10.52, p < .001.

Effect size. Adding the work arrangement increases R^2 by 0.173, from 0.13 to 0.30.

22.5.1 Reporting the Results

A multiple regression analysis examined the job satisfaction of 90 employees as a function of their years with the company, having children at home, and their work arrangement (dummy coded, with office work as the reference category). The model explained 30% of the variance in job satisfaction, F(4, 85) = 9.21, p < .001. The work arrangement added significantly to the prediction over and above the other predictors, ΔR² = .173, F(2, 85) = 10.52, p < .001. Compared with employees working in the office, hybrid employees reported higher satisfaction, b = 10.59, t(85) = 4.27, p < .001, and so did remote employees, b = 8.90, t(85) = 3.63, p < .001. Satisfaction was also higher for employees with more years in the company, b = 0.62, p = .009, and lower for employees with children at home, b = -4.89, p = .018.

22.6 Summary

A categorical predictor with k categories enters a regression as k - 1 coded variables. The coding does not change the fit of the model, but it decides what the coefficients mean. With dummy coding, the intercept is the mean of the reference category and the slopes are differences from it. With effect coding, the intercept is the grand mean and the slopes are deviations from it. With contrast coding, the slopes test the comparisons chosen by the researcher.

Of note is the fact that the familiar tests of group differences are special cases of the regression. A dummy-coded predictor with two categories is the independent samples t-Test with equal variances, and a predictor with three or more categories, tested as a whole, is the one-way ANOVA. Adding other predictors to the model compares the groups while holding these predictors constant, which is the idea behind the analysis of covariance (ANCOVA).