What Are The Odds

As the presidential election enters the home stretch, one sentiment that unites the nation is skepticism about the polls. Don’t trust them, Democrats say: Hillary was ahead in 2016 and lost. Don’t trust them, Republicans echo: They said we were behind, and Trump pulled it off.

David Byler ’14, a political columnist for The Washington Post, is one of many data analysts who will tell you that the polls were pretty accurate in 2016. The final average at the website Real Clear Politics, where Byler worked at the time, predicted that Hillary Clinton would win the national popular vote by 3.2 points; she won by 2.1. Because she narrowly lost several battleground states, where polls were less accurate than those conducted nationally, she lost the Electoral College. The national polls in fact were further off the mark in 2012, when they predicted a 0.7-point Obama victory in the popular vote and he won by 3.9 points, but because they only misjudged the margin and not the ultimate winner, no one remembers.

In a normal universe, Byler would have spent the last half of August in Milwaukee and Charlotte at the Democratic and Republican conventions. Generations of political columnists made their reputations in the crowded arenas and smoke-filled rooms. Byler, a relative novice, thinks they would have been fun, but even when there isn’t a pandemic he works remotely from his home in Los Angeles. As a data journalist, shoe leather is less important than internet access and a phone. 

Data analysis is one of the few fields in journalism that is growing. Byler and his colleagues apply big data — the reams of statistical information such as polling results, economic reports, or census files — to help understand the news. Or, to predict it, in the case of elections. Where a traditional journalist might try to discern what voters are thinking by visiting a diner in Ohio, for example, a data journalist could skip the diner altogether in favor of polling or government data from similar areas across the country, to paint a bigger picture. To put it another way, Byler does for politics what bookmakers do for football games and meteorologists do for hurricanes. 

Along with his professional peers, such as Nate Silver of the website FiveThirtyEight, Nate Cohn of The New York Times, and Harry Enten at CNN, Byler tries to make sense of the polls, explaining what is significant and what isn’t. Millions of Americans turn to these experts daily — or for some obsessives, hourly — to get the latest on the horse race or whether the margin is narrowing in, say, Michigan. They are also the people millions of others curse for “blowing it” in 2016.

BYLER’S FIRST POLITICAL MEMORY is from election night in 2000. He grew up in Parkersburg, West Virginia, where his parents, both doctors, were staunch Bush supporters. Young David followed the returns, coloring each state on a map as red or blue as the results were called. 

“I noticed that the Republican states were also the Confederate states,” he recalls, “and I had learned in school that the Confederates were the bad guys. So I had this little existential crisis. I remember going to my parents and asking, ‘What’s happening here?’ I think that was the first elections question I ever asked.” Regrettably, he does not recall his parents’ response.

“Being aware of the limits of the discipline is a huge part of not being overconfident,” says David Byler ’14.

Many political-data journalists come to the field from other areas. Silver, for example, first honed his love of probabilities as an online poker player, while Enten was an amateur meteorologist. Byler has always loved pure mathematics and statistics. At Princeton, he majored in operations research and financial engineering. He also edited the Princeton Tory and participated in campus ministry. As a conservative, evangelical Christian from West Virginia, he felt able to observe his classmates as something of an outsider. Byler sees his undergraduate years as a period of generational transition between relative student apathy and activism. From the sidelines, he was fascinated.

“It felt as if a lot of the people [then] were adopting an ideology that would easily fit into the left establishment,” he says. “To me, that was a bit boring. I almost wished people were willing to stake out stances that were more passionate.” 

Byler’s columns for the Tory reveal him to be more of an analyst than an advocate. In 2012, for example, he wrote a long analysis of Occupy Princeton, a campus offshoot of the left-wing Occupy Wall Street movement. It was a topic that seemed teed up for a conservative diatribe, but rather than attack the group, Byler spent nearly 2,000 words analyzing its structure and its goals. His conclusion: “While the Tory does not endorse the positions of Occupy Princeton, the reasons for its popularity are certainly thought-provoking.” 

Though Byler has written for other right-leaning publications, including The Weekly Standard (now defunct), it is hard to find any ideological biases in his work. Indeed, he prefers not to discuss his views on politics, believing that they are irrelevant and that doing so is counterproductive. “People will distrust you if they disagree with you,” he says.

Like most ORFE majors, Byler was headed toward a career in finance or consulting. He had a few lucrative job offers, but with deadlines to accept them approaching, he sent a blind email to the political website Real Clear Politics. It was forwarded to Tom Bevan ’91, who founded the site with John McIntyre ’91 in 2000. Bevan, in fact, still has the email, which politely expressed Byler’s love of politics and asked if there were any job openings. Impressed that Byler would forgo a much higher salary to do something he loved, Bevan decided to take him on.

For a little over two politically rich years, from the 2014 midterms through the 2016 election, Byler assisted Sean Trende, one of the most respected data journalists in the business. Trende began by giving the new Princeton graduate a reading list that included more than five years of Trende’s back columns as well as staples of modern political science literature, such as David Mayhew’s Electoral Realignments and John Judis’ The Emerging Democratic Majority. He was impressed that Byler devoured everything he was given and, later, by his clear-headedness. Early on election night in 2016, Trende tweeted that exit polls pointed toward a clear Clinton victory. He soon received an annoyed email from Byler reminding him that there was yet far too little information to justify such a call. “David was a good reality check for me,” Trende says. 

Byler moved on to The Weekly Standard as its chief elections analyst in October 2017, remaining through the 2018 midterms. It was there that he made his first forays into election modeling. His Senate forecasting model, called Swing Seat, crunched current economic data as well as polling data going back to 1992 to generate its projections. Swing Seat predicted that the Republicans would win 52 seats. They won 53.

As gratifying as it was to nearly nail the results, Byler was even more pleased that his model helped tease out how public opinion and public events interact. Both parties believed, for example, that the contentious confirmation hearings for Supreme Court nominee Brett Kavanaugh would help their side in the upcoming midterms, but Byler thinks his model illustrated that the sexual-assault allegations against Kavanaugh worked to the GOP’s advantage, firing up its base and helping to secure a few tenuous seats that enabled it to retain the Senate.

“You could have gotten to this conclusion without a model,” he wrote in a post-mortem for the Washington Examiner, “but the model helped by drawing a clear trendline, using pre-established rules (which is a good safeguard against unconsciously cherry-picking data to strengthen your point) and giving us a quantifiable estimate of how much the story moved the needle. Put simply, good model-driven analyses can help us be less speculative on questions about what does and doesn’t move public opinion and how our institutions and elections respond to shifts in the electorate.” 

Poll numbers describe current conditions, Sam Wang emphasizes. They do not predict the future.

Photo: Sameer A. Khan

The Washington Post, eager to beef up its data analytics before the presidential race, hired Byler in January 2019, a month after The Weekly Standard folded. His most significant contribution to the paper’s coverage was designing the Post Opinions Simulator, which tried to predict the early primaries (with roughly equal numbers of hits and misses). The Simulator also included a feature that allowed readers to manipulate the model by changing variables, exploring, for example, how the New Hampshire primary might turn turn out if Amy Klobuchar doubled her fundraising or Andrew Yang improved his polling. 

The horse race, though, has only been part of Byler’s focus. In March, he wrote a column assessing how Democratic moderates broke their prisoner’s dilemma before Super Tuesday to coalesce behind Joe Biden and thwart Bernie Sanders. Throughout the spring and summer, Byler assessed more than a dozen of Biden’s possible running mates and what they could bring to the Democratic ticket. Although he did not predict whom Biden would choose, and made the case pro and con for each, his assessment of Kamala Harris now looks prescient. “Joe Biden may be the presumptive Democratic nominee for president, but he is not the future of his party,” he wrote in a column on April 16. “So he should choose a running mate who represents the future and can lead on policy after the Biden era. That person is Sen. Kamala D. Harris (D-Calif.).”

(In mid-August, following Harris’ selection, Byler landed in social-media hot water for a day after describing her as a “small-c conservative, party-friendly” pick. Byler explained that he did not mean Harris herself was conservative, but rather that she was a relatively low-risk selection.)

BACK WHEN BYLER JOINED Real Clear Politics, Trende also gave him a few nonpolitical books to read. Knowing Byler’s background in math and statistics, Trende assigned Thomas Kuhn’s 1962 classic, The Structure of Scientific Revolutions, and Paul Feyerabend’s Against Method. Kuhn, in particular, maintained that science is a social activity as well as a rational one, and that human biases always warp the search for truth. Scientists work within a particular paradigm (such as Newtonian physics) until a new paradigm (relativity or quantum physics) comes along and upends the old certainties. Trust your models, in other words, but remain skeptical that they can explain everything. 

“Every 22-year-old probably has too much faith in methods and is too wide-eyed about what we can really know,” Byler says. “Being aware of the limits of the discipline is a huge part of not being overconfident.”

While he cautions against paying too much attention to the horse race, Byler acknowledges that it consumes public attention, driving clicks and page views. At their best, electoral models can help illustrate which factors drive opinion and which don’t, as well as detecting late shifts in support. Part of any data journalist’s job is helping readers understand concepts such as probability — that saying a candidate has an 80 percent chance of winning, for example, doesn’t mean that it’s a sure thing. 

Furthermore, polls are a snapshot in time and rely on assumptions the pollster makes about the composition of the likely electorate. If those assumptions are wrong, the poll will miss the outcome. In fact, there is likely to be even more uncertainty this year, since no one has experience modeling an election held largely by mail during a pandemic. That is why Byler and others advise not reading too much into any single poll, good or bad, but taking the average instead. 

Nevertheless, many of us treat polls like a crystal ball or a security blanket, craving an expert in a nerve-wracking election season who can hold our hand and assure us that everything is going to be all right. No one knows this better than Sam Wang, a Princeton professor of neuroscience and founder of the Princeton Election Consortium website. When people point an accusing finger at analysts for blowing the 2016 election forecast, Wang is the man many of them have in mind.

After predicting the 2012 presidential race with almost perfect accuracy, Wang announced in the weeks leading up to the 2016 election that Hillary Clinton had a 99 percent chance of winning the Electoral College (he shaved that to 93 percent by Election Day) and promised to “eat a bug” if he was wrong. The Sunday after the election, Wang dutifully swallowed a cricket live on CNN.

In an article for the Summer 2020 issue of the Columbia Journalism Review (CJR), “Our Polling Trauma,” Wang revisited his famous mistake. The problem, he wrote, was converting polling margins into probabilities. The polls, in short, never showed a race that justified Wang assigning Clinton a 99 percent probability of winning. “I overestimated the precision of my model,” he confesses. 

What he should have done was emphasize that even a two-point difference between the final polls and the outcome in key states could enable Trump to squeak through. Accordingly, Wang will not give probabilities on the Princeton Election Consortium homepage this year. He will focus instead on predicting the raw number of electoral votes and Senate seats Biden and the Democrats are likely to win, as well as the party’s chance of holding the House. 

Those predictions may occupy the sexy banner spot atop the PEC’s homepage, but there is much more data and analysis throughout the site. Wang still trusts the numbers; he worries, in fact, that we continue to misunderstand what happened four years ago. “If we become afraid of polls, I think we are learning a false lesson,” he wrote in his CJR article. “It isn’t that polls were inaccurate during the last presidential election. ... The problem is that our brains may have turned an emotional experience with polling into a lasting trauma.”

Byler says much the same thing. “There is a psychological tendency that a lot of people have to either chuck the polls or believe in them religiously,” he observes. “It’s my job to find a middle path between those outlooks.” 

Wang, nevertheless, deprecates people (we know who we are) who obsessively check FiveThirtyEight during election season and focus on swings of tenths of a percentage point in its model’s results, changes that are essentially meaningless. He illustrates this with a hypothetical: If there are two bullets in a six-chamber gun, you’d have a 67 percent chance of surviving a game of Russian roulette, yet no sane person would consider those safe odds. Remove a bullet and your chances rise to 84 percent. To paraphrase Clint Eastwood: Do you feel lucky today? Poll numbers describe current conditions, Wang emphasizes. They do not predict the future.

Used incorrectly, they can also drive passivity. Wang is trying to use the data on his website as a vehicle to direct an engaged citizenry. In a sense, he remarks, pollsters are indeed like oddsmakers predicting a football game, though with one critical difference. You, the voter, can affect the outcome.

To encourage that, the Princeton Election Consortium is hyping “moneyball” districts, a term derived from Michael Lewis ’82’s book about undervalued baseball statistics, to identify down-ballot races where voters have the most leverage. He focuses particularly on state legislative races that could flip control of a chamber and thus influence political reapportionment for the next decade (Wang also directs the Princeton Gerrymandering Project). Voters can use that information to decide where to direct their campaign contributions or where they might volunteer to make phone calls or write postcards. For the sake of our democracy — and your sanity — Wang pleads, stop refreshing polling websites every half hour, and get involved instead. 

“I don’t think it’s bad for people to be really interested in elections,” Byler says, laughing. “As hobbies go, that’s a comparatively good one. But I also want people to have healthy blood pressure. I hope we are doing a better job explaining what polls and probabilities mean.”

Come election night, he and much of the rest of the country will be glued to the internet and the TV as results trickle in, possibly over many days. By that point, of course, the game will be over, so to speak. Until then, whatever your political leanings may be, there is still time to go down on the field and try to influence the result. For a good citizen, that might be a better way to spend the next few weeks than sitting in the stands, staring at the scoreboard.  

Mark F. Bernstein ’83 is PAW’s senior writer.