The Nuanced Effects of Medicare for All
By Kevin Reuning
A recent analysis of the 2018 Congressional election found that Medicare for All support hurt Democrats. The model used to show this was simple, including only the 2016 Presidential results, the total funding for each candidate and if there was a Republican incumbent or not, in addition to whether the Democrat supported Medicare for All. If we want to make decisions for what candidates should do in 2020 based on 2018 results we need to be certain that we’ve found a real relationship. Predicting what you should do based on observational data is a tricky business. It becomes even more tricky when you are dealing with strategic actors. We might have a few questions: Who endorsed M4A? What sort of districts were they running in? What sort of media coverage was there on M4A during the race? All of these could affect the relationship between supporting M4A and winning an election.
To test this we collected data on the two-way 2018 returns, the 2016 Presidential two-way vote share, funding for each candidate, spending by outside groups in the district, whether a candidate supported M4A, whether an incumbent was running, and if there were any candidate scandals during the race. In addition, we obtained data on media coverage of M4A in each district from Deck[1]. You can find the code and data for everything here on github. Here, we use the National Nurses Union coding, to replicate the analysis that Abramowitz originally published. However, our analysis of these data suggests that it does not accurately reflect Medicare for All support (for example, the NNU list includes Kim Schrier as an M4A supporter, but a contemporaneous fact check suggests she did not support M4A, a contemporaneous news article includes her endorsing a public option and she did not co-sponsor M4A when she was elected). Future blogs will discuss our findings there.
We first estimated a model without any M4A support measures. We included funding differences between each candidate (log(Dem Funding) - log(Rep Funding)), the total funding for the race logged (the assumption that races with a lot of money being spent were likely to lean Democratic), the difference in outside spending on either side, two indicators if there were scandals for the Democratic or Republican candidate and indicators if there were a Democratic incumbent or Republican incumbent. Finally, we include the 2016 Presidential return results interacted with whether an incumbent was running (this accounts for the fact that the 2016 results will be less informative for incumbents).
Our first question was whether including a candidate’s stance on M4A would improve model fit: could we better predict the results in 2018 if we include this information? To test this we estimated Leave-One-Out cross-validation using Pareto-smoothed importance sampling (PSIS). The LOO metric allows us to measure how well a model fits the data while limiting the potential for overfitting. When modeling relationships we always run a risk of overfitting models, which means that our estimates reflect the random noise in our data and so will not predict new data well. Given that we are concerned with predicting what will happen in 2020 we want to make sure we are not capturing noise.
The below figure shows the LOO Information Criterion (LOOic) for our base model and for a model that includes M4A support interacted with race type (Open, Dem Incumbent, Rep Incumbent) and a model with the effect conditioned on district lean, which we will discuss later on. Including the M4A data actually reduces model fit according to LOOic. In fact, between all three models, the difference in model fit is not statistically significant. Given that our baseline model performs just as well as a model that includes M4A stances, we should probably stop here and conclude that M4A stances do not have any strong predictive effects on who won in 2018. This is a story that will probably not satisfy M4A proponents or opponents: by itself, M4A didn’t hurt or hinder.
But what is the point of doing data analysis if you aren’t going to flog a dead horse to see what pops out? We can start by examining the effects of M4A stances on the three types of races in our data: Rep Incumbent, Dem Incumbent, and Open. Here we find a mixed story. M4A stances have no statistically significant effects for candidates running in open seats and for Democratic incumbents but might hurt candidates challenging Republican incumbents. The effect is around -1 points with 95% CI that runs from almost 0 to -2. It is possible though that this effect is heterogeneous across races with Republican incumbents. It is entirely plausible that Democratic challengers that endorsed M4A did so in more conservative districts and so were unlikely to win because of this and not because of their M4A support.
To test this we estimated another model where we interacted support for M4A with the 2016 Presidential vote share. We found an interesting pattern. First, the only place where the interaction really seemed to matter was for Democratic incumbents. Here endorsements of M4A increased their vote share but only in districts that were heavily Democratic. Across the other types of races, the interaction was not substantively interesting, although for open seats it followed a similar pattern as it did for Democratic incumbents.
Another obvious question is whether M4A media coverage had an effect on this relationship. Here we used data gathered by Deck on media coverage within the district and measured the sentiment of the media coverage along with the number of articles. We created a measure then that was the overall sentiment of media coverage times the log of the number of articles. Negative numbers indicate a lot of negative coverage, positive numbers indicate a lot of positive coverage. Just under a quarter of races had no M4A coverage, although this was not seemingly related to whether a candidate had endorsed M4A. Thirty-eight percent of races had M4A media coverage where the Democrat endorsed M4A and 36 percent had M4A media coverage where the Democrat did not endorse M4A.
Modeling how this interacts with the effect of M4A finds potential relationships for open and Republican seats but no relationship for Democratic incumbent seats. In both open and Republican seats more negative media coverage increases the likelihood of there being a negative effect for M4A endorsement. This would indicate that perhaps one of the things driving the effect of M4A on races is the media’s coverage of it. This is not something necessarily within the control of a candidate running in 2020 but is something worth thinking about.
As a final examination of what may be driving M4A effects on vote share, we looked at what leads to candidates being perceived to be more liberal. The ongoing media coverage of M4A argues that candidates who embrace it are taking up leftwing positions that are out of step. We can test this by using data from the CCES on the perceived ideology of candidates. Previously we used this to estimate ideological scores for candidates that are comparable across races. For this model, we just looked at Democratic non-incumbents as incumbents are likely to not have a single issue position change voter perceptions of them. We included the Presidential vote share in 2016 along with a few demographic characteristics of the candidate that we might expect to color perceptions of the candidate.
What we find is that supporting M4A does not lead to candidates being perceived as more liberal. The coefficient is substantively small and 95% CI bars cross the 0 lines. This results even holds when dropping the 2016 vote share variable. It does not appear then that M4A endorsement is leading to candidates being perceived as leftwing socialists. We find the same result both qualitatively and quantitatively when using an alternative modeling approach that considers individual-level and district-level covariates to predict the likelihood that CCES respondents would perceive their Democratic house candidate as “Somewhat Liberal”, “Liberal”, or “Very Liberal”.
Analyzing campaigns is difficult. Although cross-sectional analysis like the one presented here can help inform strategy, it has its limits. One major problem is that we cannot be certain that other candidates or district attributes are not driving our findings. By including other variables we can limit the chances of this. If we want to eliminate them entirely then we likely have to leave the realm of observational studies and look at experiments. Experiments though have their own drawbacks, often not capturing reality well.
If we feel relatively comfortable with the variables we’ve included in our analysis then we still need to think about other, often untested, assumptions within our model. Here we showed that the effect of M4A is conditional on other attributes of the race.
Kevin Reuning (@KevinReuning) is an assistant professor of political science at Miami University.
Editor’s note: This is the first in a series of pieces analyzing the impact of Medicare for All. In order to replicate the initial analysis we examine, we use the National Nurses Union Medicare for All list. However, a future blog will analyze whether the National Nurses Union coding correctly identifies Medicare for All candidates and how that impacts Abramowitz’s analysis.
[1] Deck licenses news articles published online and national/local TV broadcasts (the database currently has over 110 million articles and over 630 million minutes of TV coverage). Deck then use a variety of methods to identify mentions of specific candidates. Deck uses a mix of vendor-provided sentiment analysis data and in-house natural language processing tools to categorize what those mentions are about and whether the sentiment toward the candidate is positive, negative, or neutral.