How Well Does DFP’s Progressive Issue Model Reflect Vote Choice?
By Alexander Agadjanian (@A_agadjanian)
In a recent GQ magazine article, Way to Win co-founder Tory Gavito and DFP co-founder Sean McElwee presented compelling new evidence on the question of whether Democrats should persuade swing voters or galvanize their base to have the best chances of retaking the White House. One graphic in particular gained significant attention:
The number of progressives--those scoring 70 or higher on DfP’s Progressive Issue Model--who turned out in 2012 but not in 2016 far eclipsed Trump’s 2016 margin of victory in four key states. Had they been mobilized in 2016, these individuals perhaps could have changed the outcome of the election. The key assumption underlying this claim is that DfP’s model reflects actual vote choice well, though this is not readily known--what if some of those scored 70+ on the progressive model are in fact Republican voters? This would undermine some of the claims in the article.
The turnout numbers in the above analysis are based on the voter file, but a big limitation with this data is that it does not contain vote preference. Analysts often turn to creating model scores for how Democratic or Republican an individual on the voter file might be (see Catalist’s partisanship score as a prominent example). Comparing this model against actual reported vote choice, as expressed on a survey, is a worthwhile exercise, and in the case of the GQ article, can clarify how well the model accomplishes what it intends to do--capture Democratic voters.
To look into this, I obtained the DfP April omnibus survey (ran through YouGov Blue) that was matched to the voter file and thus could be connected to the Progressive Issue Model scores for the survey-takers. The DfP progressive issue score goes from 0 to 100; among the people who took this survey, it ranged from 10.04 to 88.47. I grouped individuals in deciles covering this range, going from those with a score of 10-20 (least progressive, and in this context, least likely to vote Democratic) to those with a score of 80-90 (most progressive/most likely Democratic voters). In the below graph, I plot the two-party Democratic vote percentage at each model score decile, along with 95 percent confidence intervals.
The gradually increasing vote share is a positive sign, as this confirms that the progressive issue model effectively captures Democratic vote propensity across its distribution. At the lowest level, only 3.9 percent of individuals vote Democrat, while at the highest 95.9 percent do. Results are similar when only focusing on survey-takers who are known to have voted in the 2016 election or subsetting to individuals in states with party registration on the voter file (which might factor into the model’s accuracy).
In order to best speak to DfP’s earlier analysis that appeared in the GQ article, I also calculated Democratic vote share among survey-takers scoring 70 or higher on the model score. From the omnibus survey, 92.9 percent of these individuals voted Democratic in 2016. So even when applying this percentage to the totals shown in the GQ graphic, the number of 2012 voters who sat out in 2016 still exceeds Trump’s margins. (A limitation here is that the validation uses a national survey. It’s unclear how much the survey-based Democratic percentage among those scoring 70+ varies by state--such as the key swing ones in the GQ graphic--but it’s unlikely to be enough to account for the non-voters/margin gap.)
In sum, the brief analysis here shows that DfP Progressive Issue Model does well in reflecting Democratic vote propensity for those on the voter file, and thus lends further credence to the claims made in the original piece.
Alexander Agadjanian (@A_agadjanian) is a research associate at the MIT Election and Data Science Lab.