Exceptionally Low Turnout Can Account For Polling Errors
At The American Prospect: Tweet In the home stretch, I wrote that midterm polling is far less accurate than in Presidential years. Today, in The...
Errors Are Inevitable – But Who Will Benefit?
Tweet
Despite the certainty of pundits, we actually don’t know who will win the Senate! In The New Yorker, I explain.
From 2004 to 2012, only thirteen Senate races have had margins of less than three percentage points in the week before the election. Of these, four were won by the trailing candidate. One more, the Florida 2004 race, was tied in the polls, and was eventually won by the Republican, Mel Martinez, by 2 percentage points. Scoring that one as half correct, the overall rate of wins by a front-runner is 65%, a bit better than chance.
In light of that, the probability that all six close Senate races (AK, CO, IA, KS, NH, and NC) will be won by the candidate in the lead is only 7%. A wrong call is almost inevitable. We should not be surprised to see one to three races to be won by the candidate who trails this morning. This allows us to hazard a guess as to the most probable path to Democratic retention of the Senate (which PEC currently has at 35%).
Last night, I gave poll-based probabilities for Senate and governorships. Republicans are favored to take over, but what is the likeliest route to Democrats retaining control? To estimate that, I will use the margin/standard-error-of-the-mean (margin/SEM) ratio (see table at the end of this post) as a measure of which margins are flakiest. From this, upsets seem likely in Iowa (margin/SEM=1.0/0.8=1.3), followed by Alaska (margin/SEM=1.0/1.7=0.6). If they flip, that gives
Democrats+existing Independents: 45 safe seats plus NH, NC, IA, and AK. Total: 49 seats.
Republicans: 45 safe seats plus AR, KY, LA (runoff), GA (maybe runoff). and CO. Total: 50 seats.
Kansas: Orman, who would make the 50th vote for the Democrats+Independents – or facilitate a power-sharing arrangement.
I note that the margin/SEM ratio in New Hampshire is 3.1, suggesting I might escape having to eat a bug.
I’ll have more to say about errors in The New Yorker later today. In the meantime, where do you see underdogs winning?
Topics:
I’d say the most likely Dem good news surprises would be Iowa and Colorado, plus if somehow Nunn pulls it out in Georgia. The most likely GOP unpredicted wins are in NH and NC, obviously. I do think that McConnel will stay South of 50% and that Grimes’ weasely refusal to answer whether or not she voted for Obama blew her a winnable race. 8% of Kentucky is Af-Am, and the black community is fiercely loyal to him. Having the Dem Grimes dis him like that probably cost her 2-4% in total vote between Blacks and disgusted liberals. It certainly didn’t get her more votes from those who despise Obama.
The Grimes lesson: If you know you’re going to get a tough question eventually, run your responses internally over and over until your answer isn’t ridiculous. She shoild have been prepped with a response that felt sufficient, and attacked both McConnell and partisan DC politics.
There are certainly polling errors, which can and most likely will change a few of these very close races. The issue about those concerned about control of the Senate is that Rs come in to this discussion with 49 seats and Ds with 45.
There may be errors in the polling for Kansas, NC, and NH to the Democrats benefit and it will still only get them to 48 seats. Ds need to take 5 of 6 to get to 50-50.
My 2 predictions in the previous thread were Kansas to Rs and Alaska to Ds with is an even trade.
Colorado seems to be just as safe or more safe than New Hampshire. So a polling error that could reverse CO may also reverse NH.
Ds need to hold the 3 States they lead in AND take 2 of 3 of the states where they trail. The odds of that specific combination are long.
These things don’t happen entirely in a vacuum – – – as Sam has written about in the past if the polling is off it can be off in one direction nationwide – so if Dems would perform better in both Colorado and New Hampshire.
Also; Colorado polling has a history of significantly underestimating Democratic support each of the last three races. That’s no guarantee it’ll happen again but it’s certainly something that gives me hope that Udall will indeed pull out Colorado.
Of all the “ground game will save us” narratives that dems are offering, by far I think the most plausible is AK. The native populations that the Begich campaign has invested so fiercely in reaching out to are practically a billboard unto themselves of a voter who’d be discarded by a likely voter screen. I can’t remember where I saw it, but democrats actually LEAD in early ballots returned so far. In Alaska. Like, for reals.
Between Begich’s outreach, the notorious misses in Alaskan polling and the fusion ticket for governor, that is the Senate seat I can most easily see the prognosticators eating a plate of crow over.
Agreed, Alaska is the most likely surprise on the D side. I think Brown is the most likely one on the R side, more so than Hagan due to the really strong Dem and African American early voting in NC.
You can all but put Georgia into the GOP column. While it’s possible that Nunn might erase the 3% lead, a Democrat has never won a runoff in Georgia.
With all due respect to your polling averages, RCP (which tosses partisan polls) has Sullivan up 2.4% and he has led in 11 of the past 12 polls… three times up by 6, twice up by 5, and twice up by 4. Still under 3% – But a lot of pollsters would have to be awfully wrong for Alaska to go Blue.
I’d be more inclined to question NC where Hagan is sitting at 44% and there is a Libertarian candidate polling at 5% … Third Party candidates almost always underperform their polls, and if those Libertarians bleed to Tillis… it makes it at best an even race… fighting for the 7% so that are undecided… Also, Hagan has led in only 7 of the past 12 polls…
I would be very surprised if Nunn won Georgia and I am quite concerned about Hagan holding on in NC. I think the thing is – these races are probably icing in either direction. If Hagan loses then the Democrats have no hope of 50. If Nunn wins then it’s likely that Democrats have outperformed expectations across the country and I will be a very happy camper. I’ve got my fingers crossed for that second scenario but I realize it’s very unlikely.
“…a Democrat has never won a runoff in Georgia.?”
Here’s why I don’t count it out. First, this is a “throw the bums out” election. The Georgia Senate race doesn’t have any bums to throw out. Ergo, the anti-incumbent fervor can only play indirectly. Will people blame the do-nothing leadership on the President or on the Senate and House? My money’s on the latter. I’d watch carefully to see if the President’s job approval numbers continue to improve. If so, I think Ms Nunn has a good chance to break that tradition.
How a state has voted in the past is not necessarily predictive of how that state will vote in the future, or even at the present. If it were, California, which voted staunchly Republican almost throughout the 20th century, would still be voting staunchly Republican now. Times–and demographics–do change.
If Perdue wins in GA, he is going to have to fight tooth and nail for it in the runoff in Jan. It is far from a foregone conclusion.
The polls in GA are garbage this cycle and I am at a loss why everyone is not treating them with more skepticism. Every public poll in GA predicted that Perdue’s primary challenger, Jack Kingston, was going to win the primary, and we know how that turned out.
I don’t think that poll misfire was a random fluke. It turned out Perdue did well in metro Atlanta, among young voters, and minorities so pollsters are having problems tracking these groups. These are groups that Nunn is relying on for a win so it would not be surprising to me if the polls were wrong again.
I agree that Georgia is traditionally very red, but the demographics have been changing rapidly. Early voting stats suggest that African-American turnout is higher than expected, which may drive the election to a run-off. If that happens, with Senate control in the balance, the national parties will pour unprecedented energy and resources into the runoff. It will be all about GOTV and appeal to folks at the margins, some of whom may not care for Mr. Perdue’s past as an outsourcer. We’ll see.
I agree with Jonny, C.H. On the street in GA, in the rural area I live in, I hear more from Republicans and Libertarians fed up with things the way they are. My family is both large and largely Republican. Very few of them will be voting Republican this time. Part of that is backlash against Deal, who has been a horrible Governor, and a backlash against local officials, problems that are making voters sick with the whole lot of incumbents, whether that be party or individuals. I think Sam has called it best – we will not know until it’s over. It’s that close here in Georgia, and in the country as a whole.
you could well be right. However, the turnout of Af-Ams in Georgia makes up a full 1/3 of the electorate, far higher than in 2010, and they are overwhelmingly voting for Nunn. Assume she gets 90% of that vote. That means she only needs 31% of the white vote to get to 50%. I admit, a tall order, but not beyond plausible for the daughter of Sam Nunn…
I am selfish: I want Iowa, Colorado,& Alaska for Democrats (along with holding NH&NC of course.) then Orman can be the 52nd Democratic vote.)
oops, I guess Orman would be only the 51st Democratic vote. Either which way Democrats have to win 5 of these 6 tight races (I’ll take all 6, thank you) … the odds look long when looking at it that way.
(then there’s Georgia, Lousiana, Arkansas, and Kentucky …. unfortunately I have very little hope that the Dems have any significant chance of pulling out those states but I would love to be shocked – especially in Kentucky.)
I don’t agree odds look long. Any state which has a large minority group should be looked at as available to the Dems. All the negativity against the president and the attacks against a immigration plan may lead a larger vote then anticipated by the pollsters..
Great chart.
I don’t understand why there is an automatic assumption that either Nunn or Landrieu need to get over 50 today as they would lose in a runoff. I understand low voter turnout for local or even state legislature elections but I hardly think that the DSCC would not work like crazy on those two elections in December if it made the difference of holding the Senate.
It’s not automatic but the Democrats have dropped the ball before on this sort of thing. In 2008 the Georgia runoff for a Senate seat was pretty darn important,* and the Democrats didn’t get the job done. Saxby Chambliss led in the general election 49.8-46.8, and cruised to victory in the runoff without much trouble (57.5-42.5).
*Another Democratic senator would have meant that the Dems still had 60 seats after Kennedy was replaced by Scott Brown, which would have made revising and passing the ACA so much easier.
yes but the GOP would be working just as hard. it would be a pretty amazing spectacle (and it actually looks quite possible) for control of the Senate to come down to Georgia & Lousiana. … I personally don’t like the Democrats odds in such a scenario … but if the Dems have 49 seats and those are both going to run-off … that would be very interesting.
I like that the outcomes have been far to the left end of the bars for the last several elections. But the 2004 one doesn’t do much to make me confident that we can count on this one being to the left…
2004 was a huge bummer. I still sometimes can’t believe this country re-elected George W. Bush.
I tell ya folks, the post-election dust may get thick. For example, Sen. Manchin (WV-D) is up for grabs, as well as Orman, if he wins. Two runoffs, two wall-flowers at the prom, and, possibly, a recount or two makes me want to stock-up on antacids.
I find it interesting that no one is mentioning the Manchin-ian Candidate. I believe if the Senate flips, a la the PEC models most likely outcome (i.e., 52 R vs. 48 D+I), it could easily end up 54 R vs. 46 D, with Manchin & Orman being 53 & 54.
Oh, how a small number of votes can have a cascade effect.
Bad news for the Ds tonight could result in even more horrendous news in the new year.
that’s a very good point. I certainly don’t trust Joe Manchin (him turning GOP would probably secure his own future in West Virginia — and I have a feeling Joe Manchin is his biggest concern) and Orman has already said he’ll caucus with whoever has control so… yeah things could get out of hand fast.
54 Rs would make taking the Senate back in 2016 less likely too (I think if the Rs win just 52 their odds of holding on in 2016 are very slim… even at 54 they wouldn’t be great – Dems are in great position in 2016)
personally I prefer the Democrats pull the shocker and not only all 6 of these close races but Arkansas & Kentucky too. and Georgia. yeah. and Georgia. 54 D 46 R. I’ll take that. !
If it’s a 49-50 scenario and Orman goes to the dems, imagine what the R’s would offer Manchin to switch.
Interesting that in all the four elections 2006,2008,2010 and 2012, the arrow ends up well on the left side of the “plank”…
That is an interesting visual. Have we heard a single commenter mention a 4-cycle utter Dem Senate winning streak? No…they seem to have passed on that. Which is odd. I would think GOP advantage versus winning streak would sell papers.
Democrats are going to win all of those. Polls are biased toward Republicans by over 3% this time. That’s my prediction. Orman caucuses with the Dems.
I’m going with one conventional view: if the Republicans take the Senate this time, at least this may benefit Democrats immensely in 2016 when voters see how much worse the Rs can really be when they control both houses. I’m not necessarily a believer in the idea that things have to hit rock bottom before they can get better, but in this case, it may be true.
CNN just called for McConnell in KY– that was quick
With respect for the analysis and hope that I’m wrong, I would suggest that the chart does not tell us reality is on the blue side of the spread most of the time … it tells us that reality is on the winning side of the spread all of the time. In other words, “Whoever wins, wins bigger than we thought.”
Surely I wish I were able to offer a more optimistic analysis, but in the interest of advancing actual understanding, I suggest this is a plausible explanation of all the data points.
P.S. Sigh.
P.P.S. What this really tells us, is that if you think your ‘side’ is going to lose, beware that feeling: that’s /all the more reason/ to get out and vote, and help others do the same.
Starting to look to me like the polling errors this time will prove to have been all over the map, rather than consistent in direction.
So are you going to hold yourself accountable for the obvious anti-Republican bias that you showed throughout this process? Let’s go back and see where you were a month ago, while most web sites and traditional pundits had already called the Senate for the GOP. What good are predictions when you aren’t right until the day of the election?
Try not to let your blue show so loudly. 🙂
If you look a month ago, at the start of October, we were showing conditions favoring a GOP takeover – as did all sites that relied on polls.
Sam –
I think the point trying to be made is that you had the Democrats favored for much of Sept and October… and even up till the last day or so… showed the GOP chances at 51% – I joked on my website that every time I refreshed your site down the stretch the GOP chances went up even as there was no changes in any polling.
At the end of the day… most everyone got the same races correct…. with the only exceptions being a split on Kansas… and the upset in N.C. (which I actually predicted).
http://coldheartedtruth.blogspot.com/2014/11/senate-projection-consensus.html
Normally I moderate incorrect statements…but in other domains, you do good work, So this gets through. But I do want to offer a correction.
You might not have read it if you didn’t follow the site. But PEC had a November probability of >50% for Republicans for the last 2-3 weeks of the campaign. And as you can see in the Meta-Margin and seat-estimator graphs, the snapshot went over to the GOP in late September. The November prediction blended a D-leaning long-term projection and an R-leaning snapshot is just the way it goes with predictions. Like the other sites, it converged toward a recent-polls-centric view. And yes, it moved in the absence of new data because of the passage of time. It’s like weather: as the clouds gets closer, rain is more likely.
On Halloween I posted these numbers on my blog:
New York Times – 71%
538 (Nate Silver) – 69%
Daily Kos – 69%
Huffington Post – 67%
Princeton Election Consortium (Sam Wang) – 55%
Predictwise – 75%
Washington Post – 95%
So… yes, I realize that your model showed a GOP victory down the stretch just as most models did… but at a substantially reduced percentage in comparison. You also had a difference between the percentage “if the election was held today” (paraphrasing) and a “day of the election” percentage that showed a significantly higher chance of Democratic control.
I appreciate the hard work that goes into this… I was just confused by some of the calculations. They didn’t seem to make sense to me.
Once upon a time my coldheartedtruth domain was a dedicated projection site every election. Complete with weighted polling averages, spreadsheets, and percentages. I even used to track polls on the sidebar and the whole works. I have a math/stats minor from college so it was something I enjoyed. In elections from 2004 thru 2010 I missed like three statewide races (I did Presidential and Senate while another poster did the Governor and house races).
This was back in the days when it was just Scott Elliot and Gerry Daly, Sean Trende (before he joined RCP) and a couple of other projection sites… we all linked each other, emailed, commented on each other’s site, etc… Sean was even a posting member at one time on CHT.
I simply don’t have the time or the resources to compete with guys like Nate Silver and the big networks. Everyone is in the game now.
I think it’s great that you take the time to do this outside of the big media outlets… and I love the fact that you have garnered the attention of Mr Silver as well!