13 Regression with Qualitative Dependent Variables
Suppose I want to build a model of voting. I decide to use the 2016 American National Election Studies1 survey results to try to understand how race is associated with voting. Respondents in the 2016 survey were asked about who they voted for in 2012, and I’m going to focus on their 2012 voting patterns for now. Using the statistical software package Stata to conduct my analysis, I find the following distributions for my two main variables of interest:
. tab vote
PRE: RECALL OF LAST (2012) PRESIDENTAL |
VOTE CHOICE | Freq. Percent Cum.
----------------------------------------+-----------------------------------
1. Barack Obama | 1,728 56.58 56.58
2. Mitt Romney | 1,268 41.52 98.10
5. Other SPECIFY | 58 1.90 100.00
----------------------------------------+-----------------------------------
Total | 3,054 100.00
. tab race
PRE: SUMMARY - R SELF-IDENTIFIED RACE | Freq. Percent Cum.
----------------------------------------+-----------------------------------
1. White, non-Hispanic | 3,038 71.68 71.68
2. Black, non-Hispanic | 398 9.39 81.08
3. Asian, native Hawaiian or other Paci | 148 3.49 84.57
4. Native American or Alaska Native, no | 27 0.64 85.21
5. Hispanic | 450 10.62 95.82
6. Other non-Hispanic incl multiple rac | 177 4.18 100.00
----------------------------------------+-----------------------------------
Total | 4,238 100.00Notice that my dependent variable (vote) is qualitative. It can take on three possible values: voted for Obama, voted for Romney, or voted for other. I can build a simple set of regression models to see how race predicts vote choice. The key is to first convert each of the three categories for my dependent variable into its own dummy (or binary) variable—meaning a variable that is always equal to either 0 or 1. I can accomplish this in Stata with the following code:
tab vote, gen(vote_)I now have several new variables in my dataset that have names starting with “vote_”:
. tab vote_1
vote==1. |
Barack |
Obama | Freq. Percent Cum.
------------+-----------------------------------
0 | 1,326 43.42 43.42
1 | 1,728 56.58 100.00
------------+-----------------------------------
Total | 3,054 100.00
. tab vote_2
vote==2. |
Mitt Romney | Freq. Percent Cum.
------------+-----------------------------------
0 | 1,786 58.48 58.48
1 | 1,268 41.52 100.00
------------+-----------------------------------
Total | 3,054 100.00
. tab vote_3
vote==5. |
Other |
SPECIFY | Freq. Percent Cum.
------------+-----------------------------------
0 | 2,996 98.10 98.10
1 | 58 1.90 100.00
------------+-----------------------------------
Total | 3,054 100.00I also convert my race variable into a set of dummy variables by running:
tab race, gen(race_)I can then run three regressions, one for each value of my dependent variables. I will use regular linear regression (least squares) for this example, although there are arguably better and more precise models for qualitative dependent variables (e.g., various types of probit and logit regression). Nonetheless, we can get by with linear regression. When using linear regression with a binary dependent variable, we call the model a linear probability model.
Let’s start by analyzing voting for Obama (vote_1) as the dependent variable:
. reg vote_1 race_2 race_3 race_4 race_5 race_6
Source | SS df MS Number of obs = 3,036
-------------+---------------------------------- F(5, 3030) = 76.29
Model | 83.3981974 5 16.6796395 Prob > F = 0.0000
Residual | 662.426572 3,030 .218622631 R-squared = 0.1118
-------------+---------------------------------- Adj R-squared = 0.1104
Total | 745.824769 3,035 .245741275 Root MSE = .46757
------------------------------------------------------------------------------
vote_1 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
race_2 | .4972868 .0281049 17.69 0.000 .4421802 .5523934
race_3 | .2078207 .0541766 3.84 0.000 .1015941 .3140472
race_4 | .1028423 .1353307 0.76 0.447 -.162507 .3681916
race_5 | .3135004 .032158 9.75 0.000 .2504466 .3765542
race_6 | .1042547 .0441427 2.36 0.018 .017702 .1908075
_cons | .480491 .0097901 49.08 0.000 .4612952 .4996868Since our independent variable is qualitative, we have an omitted category. In this case, we’ve left category 1 (race_1) out of our regression, which indicates non-Hispanic White respondents. Our constant (or y-intercept) indicates the predicted value of the dependent variable when all independent variables are equal to zero. We can see this by writing out the regression equation:
\[ \widehat{vote\_1}=.48+.50race\_2+.21race\_3 +.10race\_4+.31race\_5+.10race\_6 \tag{13.1}\]
For non-Hispanic White respondents, race_1 equals one and all other race dummy variables equal zero, so we get:
\[ \widehat{vote\_1}=.48+.50(0)+.21(0)+.10(0)+.31(0)+.10(0)= .48 \]
Remember, vote_1 is equal to zero if the respondent didn’t vote for Obama, and it is equal to one if the respondent did vote for Obama. Our predicted value is neither zero nor one; instead, we get .48. This can be interpreted as indicating the probability of a one. In other words, a non-Hispanic White respondent has a .48 probability of voting for Obama. We can also convert this probability to a percentage by moving the decimal place two spots to the right: a non-Hispanic White person is estimated to have a 48% chance of voting for Obama, according to this model.
Now, let’s look at the slope coefficients. The coefficient for White (race_2) equals .50. Thus, a one-unit increase in race_2 is associated with a .50-unit increase in vote_1. Let’s break that down a bit to see if we can create a clearer interpretation. Since race_2 is a dummy variable and non-Hispanic White is the omitted category, a one-unit increase in race_2 correspondents to having a White respondent instead of a White respondent. And since our dependent variable is binary, we should think in terms of probabilities, which can be converted to percentages: a .50-unit increase in vote_1 means a 50 percentage-point increase in the probability of voting for Obama. So putting this altogether, we’d say: (non-Hispanic) White voters are 50 percentage points more likely to vote for Obama than (non-Hispanic) White voters, according to this model.
Similarly, Asian voters are 21 percentage points more likely to vote for Obama than (non-Hispanic) White voters. Native American voters are 10 percentage points more likely to vote for Obama than (non-Hispanic) White voters. Hispanic voters are 31 percentage points more likely to vote for Obama than non-Hispanic White voters. And voters identifying as multiracial or other race are 10 percentage points more likely to vote for Obama than (non-Hispanic) White voters. All of these differences are statistically significant, except for Native American versus White voters (probably because there are only 27 Native Americans in the sample, making the estimate of this difference very imprecise).
Let’s move onto running a regression for the second category of our dependent variable:
. reg vote_2 race_2 race_3 race_4 race_5 race_6
Source | SS df MS Number of obs = 3,036
-------------+---------------------------------- F(5, 3030) = 72.35
Model | 78.6117037 5 15.7223407 Prob > F = 0.0000
Residual | 658.463395 3,030 .217314652 R-squared = 0.1067
-------------+---------------------------------- Adj R-squared = 0.1052
Total | 737.075099 3,035 .242858352 Root MSE = .46617
------------------------------------------------------------------------------
vote_2 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
race_2 | -.483031 .0280207 -17.24 0.000 -.5379725 -.4280895
race_3 | -.2002027 .0540143 -3.71 0.000 -.306111 -.0942944
race_4 | -.0822373 .1349253 -0.61 0.542 -.3467917 .182317
race_5 | -.3014791 .0320617 -9.40 0.000 -.364344 -.2386142
race_6 | -.1344972 .0440105 -3.06 0.002 -.2207906 -.0482038
_cons | .498904 .0097607 51.11 0.000 .4797657 .5180423Now we’re looking at predictions of voting for Mitt Romney. Our constant is .50, indicating that a non-Hispanic White voter has a 50% chance of voting for Mitt Romney. The coefficient of -.48 for race_2 indicates that (non-Hispanic) White voters are 48 percentage points less likely to vote for Mitt Romney than (non-Hispanic) White voters. I won’t go on to interpret the rest of the coefficients, but they follow the same pattern.
Finally, let’s look at a regression with vote_3 as the dependent variable:
. reg vote_3 race_2 race_3 race_4 race_5 race_6
Source | SS df MS Number of obs = 3,036
-------------+---------------------------------- F(5, 3030) = 2.23
Model | .20833556 5 .041667112 Prob > F = 0.0490
Residual | 56.6836275 3,030 .018707468 R-squared = 0.0037
-------------+---------------------------------- Adj R-squared = 0.0020
Total | 56.8919631 3,035 .018745293 Root MSE = .13678
------------------------------------------------------------------------------
vote_3 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
race_2 | -.0142558 .0082213 -1.73 0.083 -.0303757 .0018642
race_3 | -.007618 .0158479 -0.48 0.631 -.0386917 .0234557
race_4 | -.020605 .0395873 -0.52 0.603 -.0982258 .0570158
race_5 | -.0120213 .009407 -1.28 0.201 -.030466 .0064234
race_6 | .0302425 .0129128 2.34 0.019 .0049238 .0555611
_cons | .020605 .0028638 7.19 0.000 .0149898 .0262202This regression provides some insights into who supported third-party candidates in the 2012 election. First, our constant indicates that a non-Hispanic White voter has a 2% chance of voting third-party. (Non-Hispanic) White voters are one percentage point less likely to vote third-party than White voters, although this difference is only significant at the .10 level. The only other significant slope coefficient is for race_6, where we see that people who identify as multiracial or other race are estimated to be three percentage points more likely to vote third-party than (non-Hispanic) White respondents.
One final thing I want to show you is that our results will be in a slightly different format but will be in one sense equivalent if we decide to use a different category as our omitted category when using a qualitative independent variable. Let’s say we want to make White (race_2) our reference category. Compare the following results to the previous regression:
. reg vote_3 race_1 race_3 race_4 race_5 race_6
Source | SS df MS Number of obs = 3,036
-------------+---------------------------------- F(5, 3030) = 2.23
Model | .20833556 5 .041667112 Prob > F = 0.0490
Residual | 56.6836275 3,030 .018707468 R-squared = 0.0037
-------------+---------------------------------- Adj R-squared = 0.0020
Total | 56.8919631 3,035 .018745293 Root MSE = .13678
------------------------------------------------------------------------------
vote_3 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
race_1 | .0142558 .0082213 1.73 0.083 -.0018642 .0303757
race_3 | .0066378 .017388 0.38 0.703 -.0274557 .0407313
race_4 | -.0063492 .0402287 -0.16 0.875 -.0852274 .072529
race_5 | .0022345 .0118186 0.19 0.850 -.0209387 .0254077
race_6 | .0444983 .0147623 3.01 0.003 .015553 .0734435
_cons | .0063492 .0077064 0.82 0.410 -.0087611 .0214595Now, our constant tells us that a White voter has a .6% chance of voting third-party. This is the same prediction we would get from our prior model where race_1 was the omitted category: to find our prediction for White voters from the prior results, we would have added the coefficient for race_2 (-.014) to the constant (.021), yielding .006 or .6% (or .007 if we use the rounded numbers shown in parentheses).
The coefficient for race_1 tells us about how White voters differ from White voters. Notice that the p-value is exactly the same as what we saw in the prior table for race_2, and the coefficient for race_1 in this table is the same as the coefficient for race_2 in the prior table, except the sign has changed. That’s because comparing White to White is the same as comparing White to White, except that we’re going in the opposite direction.
If you download the data yourself and have access to statistical software, you can go on to play around with these two sets of results more on your own if you’d like. Both regression equations will yield the same prediction for a voter of any given race. The difference lies only in the starting point, as represented by the constant. However, the p-values will usually differ because they are describing a different comparison (e.g., comparing Asian to White in this table versus comparing Asian to White in the prior table). Thus, it doesn’t really matter which category you pick as your omitted category, except that you may care more about some comparisons than others. You can also run the same regression multiple times but with different omitted categories so that you can get the p-values for a full set of comparisons across groups.