23 Multiple Logistic Regression

Logistic regression analysis responds to the needs in many domains to predict a categorical binary response11 A binary response takes values 0 or 1. For example, between the right or wrong answer, between survival and death, or between to buy and not to buy. based on two or more predictors. For example, for a response (criterion) variable with two possible values (e.g., Yes or No), logistic regression offers the possibility to attach probability values to the responses given a set of predictors. That is, logistic regression helps understand how multiple predictor variables, together, predict the response or criterion variable membership in one or the other of the two categories of the response (criterion) variable.

The dichotomous nature of the response variable prevents the calculation of a numerical value, as it is the case with regular regression tests. Instead, it uses the binomial probability theory, with only two values to predict, and the maximum likelihood method to generate a best fitting equation that is used to classify the data to the appropriate category based on the regression coefficients.

The basic formula for Logistic Regression is similar to the one used in Linear Regression:

logit(P) = a + b \cdot X

For multiple predictors, the formula changes to:

logit(p) = \beta_0 + \beta_1 \cdot X_1 + \beta_2 \cdot X_2 + ... + \beta_n \cdot X_n

In the equation above, p is the probability the outcome or characteristic of interest is attained, X_i represent the predictors, and \beta_i represent the relative contribution of these factors.

The response (criterion) variable in Logistic Regression has only two values, 0 and 1. The model does not predict it directly. It predicts the logit of the probability that the response is 1, the natural logarithm of the odds:

logit(p) = log(odds) = ln(\frac{P}{1-P})

where P is the probability of predicting a 1 (attain the outcome of interest).

In the end, what we are interested in is the probability (P) that the desired outcome occurs. For that, we need to do a little bit of simple algebra22 While they may look scary at first sight, the computations are simple and will be explained in context later in this chapter.:

\begin{array}{c} ln(\frac{P}{1-P}) = a + bX\\ \frac{P}{1-P} = e^{a+bX}\\ P = \frac{e^{a+bX}}{1+e^{a+bX}} \end{array}

23.1 The Study

To understand a bit better where and how logistic regression is useful, let’s look at an example study designed to investigate the effects of self-explanation33 Self explanation is an explanation a learner generates on his or her own, as opposed to explanations provided by an external source (e.g., book, instructor, peer). on learners’ performance on causal reasoning tasks. Specifically, this study was designed as a completely randomized44 The participants were randomly assigned to one or the other of the two experimental groups., two group (control and treatment), between-subjects55 Because each participant is member in only one of the two experiment groups: control or treatment. experiment that used self-explanation to elicit causal mechanism explanations when reasoning about causally linked events. The target field was medicine, a domain that relies heavily on the understanding and use of extensive and complex causal processes. The overarching research question was:

In the medical field, when learners are reasoning causally, does using self-explanation to elicit an explanation of the causal mechanism(s) improve, on average, learners’ performance on tasks involving such reasoning processes?

Based on the existing literature, the study used prompts to train learners to self-explain before answering a question. That is, to think about and attempt to formalize in writing the principle(s) involved in solving practice problems presented to them. For the purpose of this experiment, the participants were randomly assigned to one of the two groups, control or treatment. The participants in both groups were asked to answer the same multiple choice questions in the training stage, with the difference that those participants in the treatment group were asked, in addition to answering the question and before choosing an answer, to formally explain the causal mechanism behind the problem posed in the question. The participants in the control group were only prompted to choose an answer, without being prompted to explain the mechanism first. It was hypothesized that the participants that had a chance to practice self-explanation (those in the treatment group, who were asked to explain before responding) would perform better, on average, than the control group on a subsequent similar problem, for which the prompt was removed and all participants, in both groups, performed the same task.

Learners’ performance was measured using a single multiple-choice question with only one correct answer for which learners first selected an answer they believed to be correct and then explained the mechanism that supported their choice. An opportunity to change the answer was offered to the participants after they submitted their explanation of the phenomena with a request to explain why the new answer is better than the previous one. The instrument also assessed the participants’ prior knowledge related to the topic used in testing, both as self-assessment and, more objectively, through a set of multiple choice questions. In addition, age group, gender, income group, undergraduate major, and intended medical speciality were collected as demographic variables.

About 350 first- and second-year medical students were invited to participate in the study and processes were set in motion to convince enough students to participate to at least meet the minimum sample size of about 10066 A basic sample size of 88 was computed based on recommendations from Keppel (1991, p. 74) of at least 44 participants per group for medium 0.6 effect size, a power of 0.8, and an alpha level of 0.05. Recommendations from other authors ranged from 40 to 60 per group. Therefore, a choice was made to consider 50 participants per group an acceptable value, which makes 100 participants the minimum sample size for the experiment.. In the end, the recruitment efforts generated a sample of 117 valid responses.

This example will cover only a subsection of the full study that was answered using logistic regression and will use a curated data set. Data manipulation and transformation procedures used to generate the data set used in this example are not covered. This example will focus on the following research question:

Does the practice of self-explanation as causal mechanism elicitation technique affects, on average, learners’ performance on causal reasoning tasks?

The variables included in the model are:

Categorical Performance Score (criterion/response, nominal scale) - calculated assigning a value of 0 to a wrong answer choice or a value of 1 to the correct answer choice.

Experiment Group (predictor, nominal scale) - determined by the group (control or treatment) to which the participant was randomly assigned to.

Year of Study (covariate) - introduced as covariate to control for potential differences in performance due to where the student is situated on the progression timeline in medical school (first or second year medical students). This attempts to account for additional knowledge, experience, and other skills that may help their performance on causal reasoning tasks.

Hypotheses.

• Null hypothesis (H₀): Holding the year of study constant, practicing self-explanation does not change the odds of answering correctly. The coefficient for group is zero (\beta_{1} = 0), so its odds ratio is 1.
• Alternative hypothesis (H₁): Holding the year of study constant, practicing self-explanation changes the odds of answering correctly (\beta_{1} \neq 0).

Decision criteria. The significance level is set at \alpha = .05. Each coefficient in the model is tested with a two-tailed Wald z test, reported in the z value and Pr(>|z|) columns of the output. H₀ is rejected if the p-value for group is smaller than \alpha.

23.2 Assumptions

As with all other statistical analysis tests, Logistic Regression has some requirements to be met:

• The response (criterion) variable has to be dichotomous (has only two values). For this example, the response variable is dichotomous by design. Therefore, this assumption is verified.
• The groups (categories) are mutually exclusive, meaning that one case can only be in one of the groups. The random assignment of participants to one of the treatment groups, either control or treatment, and only one group, verifies this assumption.
• Independent observations, each participant contributing a single case
  • Usually ensured by the design of the study;
  • Violated when the same participants are measured more than once, or when they influence each other’s responses.
• A linear relationship between each continuous predictor and the log odds of the response77 The relationship between a predictor and the probability of the outcome is S-shaped, not linear. It is the log odds, the scale on which the model is estimated, that should change at a constant rate.

  7 The relationship between a predictor and the probability of the outcome is S-shaped, not linear. It is the log odds, the scale on which the model is estimated, that should change at a constant rate.

  • Detected by adding to the model the product of the predictor and its logarithm, which should not be significant, or by plotting the log odds against the predictor;
  • Violated when the log odds change faster in one part of the range than in another. In this study both predictors are categorical, so the assumption does not apply.
• Enough cases in the smaller category of the response
  • A common rule of thumb is at least 10 cases in the smaller category for each coefficient in the model, not counting the intercept88 With fewer cases the coefficients are unstable, and with very few the model may not be estimated at all.. It is a minimum, not a guarantee: categories with very few cases, a predictor that separates the two outcomes perfectly, and the precision needed for the odds ratios also matter;

    8 With fewer cases the coefficients are unstable, and with very few the model may not be estimated at all.

  • Violated when one outcome is rare. This is checked below, once the data is loaded.

23.3 Analysis

The data file has been prepared beforehand to include, from the more than 60 variables in the raw data set, only variables that may be relevant to this analysis. So, first, let’s familiarize with the data. Table 23.1 shows the first few data rows.

The variables of interest are group, score, and year. A first step in the analysis is to convert the variables into factors99 Both are categories coded with numbers. As factors, they are treated as categories rather than as quantities.. For this purpose two new variables, groupf and yearf will be added to the data set, representing group and year as factors. Once this step has been performed, it is time to define the model. The research question being investigated here asks if the treatment (practice of self-explanation) affects, on average, performance on causal reasoning tasks. Therefore, the model will look at how performance (criterion) represented by the variable score is related to the predictor group. The variable year is introduced to account for the potential effect of the year in medical school1010 First or second year students.. A summary shows the count of records (frequencies) for each category for each of the predictors converted to factors.

Group (1=Control Group, 2=Treatment Group)

 1  2 
61 56 

Year of Study (1=First Year Students, 2=Second Year Students)

 1  2 
44 73 

Of the 117 students, 49 are in the smaller category of the response. The largest model below has three terms (group, year, and their interaction), which needs at least 30 cases by the rule of thumb, so the sample is large enough.

For the analysis, the predictors (factors) are converted to values of 0 and 11111 See Dummy Coding in the Categorical Predictors chapter.. For the current analysis and data set, the recoding (or dummy coding1212 Is the process of recoding a categorical variable with 2 or more levels into a binary variable (categorical variable with only 2 levels), with values 0 and 1, variable known as dummy variable. as it is sometimes known) is performed as shown in Table 23.2.

The logistic regression equation and the model we start this analysis with is:

logit(p)=\beta_{0}+\beta_{1}\cdot group+\beta_{2}\cdot year+\beta_{3}\cdot group \times year

In this model group and year are the main effects1313 The effect of each variable taken individually on the response (criterion) variable. while the term group \times year represents the interaction effect1414 The combined, simultaneous, effect of the two variables taken together.. Let’s look at how a summary of this model looks like.

Call:
glm(formula = score ~ groupf + yearf + groupf * yearf, family = binomial, 
    data = logReg)

Coefficients:
               Estimate Std. Error z value Pr(>|z|)  
(Intercept)    -0.94446    0.44544  -2.120    0.034 *
groupf2        -0.08516    0.68545  -0.124    0.901  
yearf2          0.49248    0.56151   0.877    0.380  
groupf2:yearf2  1.03358    0.83763   1.234    0.217  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 159.10  on 116  degrees of freedom
Residual deviance: 148.74  on 113  degrees of freedom
AIC: 156.74

Number of Fisher Scoring iterations: 4

Looking at the output above, it can be noted that the name of the variables in the left column is accompanied by a value. That is because R1515 Other statistical analysis software application perform the same conversion. converts the predictors in a process known as dummy or treatment coding. It creates a set of dichotomous1616 Dichotomous variables have only two values variables (see Table 23.2 as well) where each level of the predictor is contrasted to a predefined reference level chosen to be as one of the values of the respective predictor variable. In this analysis the variables have only two levels, therefore the process consists of choosing a reference level as one of the two values. In this case the value 1 (representing the control group) is selected as the reference level. The value 21717 In the output R uses the values of original variables 1 and 2 and not the internal values of 0 and 1 it uses for analysis. This is because while the analysis uses numbers, the output can use strings to provide more information, if the data was collected and entered with string labels or values. in the output indicates that the treatment group (represented by the value 2 in the original dataset) is contrasted to the reference level, the control group. If the predictor has more than two levels, one of the levels will be chosen as reference level and two or more dichotomous variables will be generated for the remaining levels, that contrast each of them with the reference level. Each of these levels will be entered as a separate factor in the output.

The analysis of the full model shows that there may be no significant main effects or interaction effects. A simpler model, with fewer terms, may describe the data as well, and a stepwise logistic regression can be conducted to explore this. Of note is the fact that the purpose of the search is a simpler description of the data, not significance. A model selected from the data is exploratory: its p-values are too small, because they do not take the search into account, and it should be confirmed on new data.

Similar to multiple linear regression, the stepwise analysis can be conducted either forward or backward. The forward approach starts with a blank model and enters each term one at a time, computes the model, and compares it against the previous one. The process will continue as long as the more complex model1818 Has more variables than the previous one. has a lower AIC than its predecessor1919 AIC is not a significance test. The model with the lower value is preferred, and there is no p-value for the difference.. Once adding a term no longer lowers the AIC, the process stops. The backward approach looks at things in reverse. It starts with the full model and starts removing variables in decreasing order of their contribution to total variance.

For this specific case, considering that we started with the full model2020 Known as enter model, in which case all terms of the model are entered at the beginning. we’ll use the backward approach. The output of this model suggests that a more parsimonious model exists. It includes only the main effects and shows significance for year and the model’s constant2121 The (Intercept) line of the output.. An ANOVA analysis conducted between the competing models shows which factor(s) were eliminated.

Start:  AIC=156.74
score ~ groupf + yearf + groupf * yearf

               Df Deviance    AIC
- groupf:yearf  1   150.28 156.28
<none>              148.74 156.74

Step:  AIC=156.28
score ~ groupf + yearf

         Df Deviance    AIC
<none>        150.28 156.28
- groupf  1   152.75 156.75
- yearf   1   156.18 160.18

Call:
glm(formula = score ~ groupf + yearf, family = binomial, data = logReg)

Coefficients:
            Estimate Std. Error z value Pr(>|z|)   
(Intercept)  -1.2640     0.3930  -3.217   0.0013 **
groupf2       0.6084     0.3893   1.563   0.1181   
yearf2        0.9836     0.4157   2.366   0.0180 * 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 159.10  on 116  degrees of freedom
Residual deviance: 150.28  on 114  degrees of freedom
AIC: 156.28

Number of Fisher Scoring iterations: 4

So let’s look at what the ANOVA output tells us.

            Step Df Deviance Resid. Df Resid. Dev      AIC
1                NA       NA       113   148.7438 156.7438
2 - groupf:yearf  1 1.539142       114   150.2830 156.2830

The ANOVA analysis shows that when the interaction factor was eliminated, the model’s AIC improved slightly. The simpler model describes the data about as well as the full model, with one term fewer. To interpret the results we need to look again at the generic equation of the logistic regression:

logit(p)=\beta_{0}+\beta_{1}\cdot group+\beta_{2}\cdot year+\beta_{3}\cdot group \times year

From the logistic regression output, the \beta coefficients in the logistic regression equation are found in the Estimate column. Therefore, with values, the equation becomes:

logit(p)=-1.264 +0.6084\cdot group +0.9836\cdot year The odds ratio computed for each of the parameters are indicators of the odds of performing better (answering correctly) versus the odds of performing worse (answering incorrectly) is increased or decreased by a factor indicated by the value of the odds ratio. The direction is provided by the sign of the raw estimated \beta coefficient. If the coefficient is negative, the odds are decreased by the computed value while if the coefficient is positive, the odds are increased by the value of the odds ratio.

The odds ratio for group:

group\,odds\,ratio=e^{0.6084}=1.84 Therefore, holding the year of study constant, being in the treatment group2222 Indicated by the number 2 at the end of the variable name in the logistic regression output. increases the odds of preforming better rather than worse by a factor of 1.84. That is, being in the treatment group2323 Participants in the treatment group were prompted to use self-explanation to help improve score. increases by 84% the odds of a better performance score than by being in the control group. Of note is the fact that this effect is not significant: the data are also compatible with no effect of the treatment, so the odds ratio describes the sample rather than an established effect.

The odds ratio for year:

year\,odds\,ratio=e^{0.9836}=2.67

This suggests that, holding the treatment constant, being a second year2424 Indicated by the number 2 at the end of the variable name in the logistic regression output. medical student increases the odds of performing better rather than worse by a factor of 2.67, meaning second year medical students see a 167% increase in odds for a better performance score.

Decision. In the final model, the coefficient for group has z = 1.56, p = .118. The p-value is larger than \alpha = .05, so H₀ cannot be rejected: the data do not show that practicing self-explanation changed the odds of answering correctly. The increase in odds estimated above for the treatment group is not significant. Year of study, on the other hand, is significant (z = 2.37, p = .018).

Effect size. In logistic regression the odds ratios are the effect sizes, and their 95% confidence intervals show how precisely they are estimated. For group, the odds ratio is 1.84, but its confidence interval runs from 0.86 to 3.98. Because the interval includes 1 (no change in odds), the effect could be absent, which agrees with the decision above. For year, the odds ratio is 2.67, with a confidence interval from 1.21 to 6.21 that lies entirely above 1.

This concludes the logistic regression analysis test. Nevertheless, every statistical test is run in the context of a study and once the results are know, they should be interpreted in that context. Next section, while not relevant to the application of the Logistic Regression analysis, is intended to offer insights in how the results of the analysis may be interpreted in the context of the study.

23.4 Model Fit and Model Comparison

The stepwise procedure chose the final model by its AIC. Let’s look more closely at how logistic regression models are evaluated and compared, since the tools differ from those of linear regression.

23.4.1 Deviance and the Likelihood Ratio Test

Logistic regression models are estimated by maximum likelihood: the coefficients are the values that make the observed outcomes most probable. The log-likelihood (LL) of a model sums, over all cases, the logarithm of the probability the model gives to the outcome actually observed:

LL = \sum\left[Y_i \ln(P_i) + (1 - Y_i)\ln(1 - P_i)\right]

where P_i is the predicted probability that case i has the outcome. The log-likelihood is always negative, and the closer it is to zero, the better the model fits. Multiplied by -2, it becomes the deviance, which plays the role of the residual sum of squares of linear regression: the smaller the deviance, the better the fit.

The deviance of a model is compared with the deviance of the null model, which has only the intercept and predicts the same probability, the overall proportion of correct answers, for every participant. The difference between the two deviances follows a chi-square distribution, with as many degrees of freedom as the predictors added. This likelihood ratio test is the logistic regression equivalent of the F test of a linear regression model. It can compare any two nested models, the larger of which includes all the predictors of the smaller one.

Analysis of Deviance Table

Model 1: score ~ 1
Model 2: score ~ groupf
Model 3: score ~ groupf + yearf
Model 4: score ~ groupf * yearf
  Resid. Df Resid. Dev Df Deviance Pr(>Chi)  
1       116     159.10                       
2       115     156.18  1   2.9185  0.08757 .
3       114     150.28  1   5.8958  0.01518 *
4       113     148.74  1   1.5391  0.21475  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The models are built one step at a time. The null model has a deviance of 159.10. Adding the experimental group lowers it by 2.92, a change that is not significant, \chi^{2}(1) = 2.92, p = .088. Adding the year of study lowers it by another 5.90, a significant improvement, \chi^{2}(1) = 5.90, p = .015. Adding the interaction does not improve the model further, \chi^{2}(1) = 1.54, p = .215. Taken together, the two main effects improve the fit significantly over the null model, \chi^{2}(2) = 8.81, p = .012. This is the model chi-square, the test of the model as a whole.

23.4.2 Information Criteria

The likelihood ratio test can only compare nested models. The Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC) can compare any models fitted on the same data. Both add to the deviance a penalty for the number of parameters, k, so that a more complex model is preferred only if it fits sufficiently better:

AIC = -2LL + 2k \qquad BIC = -2LL + k\ln(n)

The model with the smallest value is preferred.2525 The BIC penalizes complexity more than the AIC whenever the sample has more than seven cases, and tends to choose simpler models.

The AIC agrees with the stepwise procedure and with the likelihood ratio tests: the model with the two main effects has the smallest value (Table 23.3). The BIC, with its larger penalty, slightly prefers the null model, 163.86 against 164.57. Differences of less than about two points are usually considered too small to choose between models, which shows that the evidence for the year of study, while significant, is not strong.

23.4.3 Pseudo R-Squared

Logistic regression has no R^2 in the sense of a proportion of explained variance. Several pseudo R^2 measures imitate it, by comparing the log-likelihood of the model, LL_M, with that of the null model, LL_0:

• McFadden’s R^2, 1 - LL_M / LL_0, the proportional improvement in the log-likelihood;
• Cox and Snell’s R^2, 1 - e^{2(LL_0 - LL_M)/n}, which cannot reach 1 even for a perfect model;
• Nagelkerke’s R^2, the Cox and Snell R^2 divided by its maximum possible value, so that it can range from 0 to 1.

  McFadden   CoxSnell Nagelkerke 
     0.055      0.073      0.098 

The three measures (McFadden 0.055, Cox and Snell 0.073, Nagelkerke 0.098) agree that the model explains only a small part of the differences in performance. They are not proportions of variance and have no agreed guidelines, so they are best used to compare models fitted to the same data2626 McFadden values between .2 and .4 are often described as an excellent fit, which puts this model far below..

23.4.4 Classification Table

A more concrete way to judge a logistic regression model is to ask how well it classifies the participants. Each participant is predicted to answer correctly if the model gives them a probability of .5 or more, and incorrectly otherwise, and the predictions are compared with the actual answers:

        Predicted
Observed  0  1
       0 54 14
       1 26 23

The model classifies 66% of the participants correctly2727 This is its accuracy on the same data used to fit it. On new participants the accuracy is usually lower.. This number has to be compared with what can be achieved without any predictors: predicting that every participant answers incorrectly, the more common outcome, would already be right for 58% of them. The model improves on this by only 8 percentage points. It identifies 79% of the participants who answered incorrectly (its specificity), but only 47% of those who answered correctly (its sensitivity).

23.4.5 Reporting the Results

A logistic regression analysis examined whether practicing self-explanation (treatment vs. control group) predicted a correct answer on a causal reasoning task, controlling for the year of study (N = 117). The model with both predictors fit the data significantly better than the null model, χ²(2) = 8.81, p = .012, Nagelkerke R² = .10, and classified 66% of the participants correctly. Second-year students were more likely to answer correctly than first-year students, OR = 2.67, 95% CI [1.21, 6.21]. The effect of the treatment was not significant, OR = 1.84, 95% CI [0.86, 3.98]. Adding the interaction between group and year did not improve the model, χ²(1) = 1.54, p = .215.

23.5 Additional Analysis - TL/DR

The results of the analysis so far are mixed, showing that the treatment itself, while still included in the equation, does not show a significant main effect in the overall sample. Let’s look at some of the elements that may have impacted the results, additional information about the study’s design, and how these affect data analysis.

First, given the population of students at the medical school was relatively small and considering the expected percentage of respondents, second year medical students offered an insufficient participant pool. Therefore, based on the timeline of the study and the curricula at the medical school, which ensured that the participants had sufficient knowledge of relevant domains, a decision was made to include first year medical students as well.

Second, the literature and prior pilot studies suggested that prior knowledge in domains relevant to the practice and test questions matters. Therefore, the study included both a subjective measure of prior knowledge, as a self-evaluation assessment reported by the participants, and a more objective, though brief, evaluation of the participants’ prior knowledge using multiple choice questions. Including first year medical students offered a chance to better understand the effects of prior knowledge as it is expected this prior knowledge to be less extensive than that of second year medical students.

With this new knowledge, the model that has been presented thus far can be extended to account for the effects of prior knowledge while controlling for the year of study. By introducing prior knowledge in the regression equation, the model selected included an interaction between prior knowledge and treatment group which suggests that the treatment works differently for different levels of prior knowledge.

The existence of an interaction term is relevant because the interpretation can no longer be conducted for each individual predictor while holding the others constant as explained for an interpretation of a logistic regression showing only main effects. In this case, the interpretation covers multiple simple regression equations, for each level of the predictors that are part of the interaction term. For example, consider the following regression equation2828 pk = prior knowledge:

logit(p)=\beta_{0}+\beta_{1}\cdot group+\beta_{2}\cdot pk+\beta_{3}\cdot group\times pk+\beta_{4}\cdot year

The interaction term is represented by group \times pk. For this equation, analysis can be conducted for the two levels of the group (0 = control group, 1 = treatment group), by entering the values 0 or 1 into the equation. This will produce the following two regression equations, which only include main effects and can be interpreted as described before.

For group = 0 (control group):

logit(p)=\beta_{0}+\beta_{2}\cdot pk+\beta_{4}\cdot year

The interpretation will now discuss the odds ratio of prior knowledge to affect the response variable for the participants in the control group only.

For group = 1 (treatment group):

logit(p)=\beta_{0}+\beta_{1}+\beta_{2}\cdot pk+\beta_{3}\cdot pk+\beta_{4}\cdot year

Which can be further reduced to:

logit(p)=(\beta_{0}+\beta_{1})+(\beta_{2}+\beta_{3})\cdot pk+\beta_{4}\cdot year

The resulting equation is interpreted only in the context of the treatment group and can look a the odds ratio for the levels of the prior knowledge predictor to influence the response variable.