Reflecting on the 2022 Election

There was an election! I have some feelings! Mostly excitement and pride in a job well done. Though I’m sad that Democrats have lost the House, keeping the Senate and Republicans having such a narrow majority limits the damage that can be done.

I want to write a little about it. All of this may be unnecessary, but I want to get these thoughts and feelings off my chest, especially since my alternatives are either packing for Monday’s move or processing my feelings about my favorite human in Formula One possibly having his career saved by my least favorite team and both of those seem terrible. Some of it is going to be slightly nerdy analysis of polling and forecasting’s accuracy in this election, and some of it is going to be personal reflections on my last election cycle in the business. I’ll put section headers so that if you don’t care about one or the other, you can skip. Let’s go!

Pretty good, actually! Well, a lot of them. Not all of them.

Not all pollsters are created equal. Independent pollsters like news organizations, universities, and the like release polls fairly regularly, though not as often as they used to due to the high expense of polling in a time where people refuse to answer calls from unknown numbers. (There are, of course, other methods of reaching voters.) These polls tend to be high-quality and more reliable, and they were this year.

One good example comes from the New York Times polling four House races. They chose KS-03 (poll Davids +14, actual Davids +12), NV-01 (poll tied, actual Titus +6), NM-02 (poll Vasquez +1, actual Vasquez +1), and PA-08 (poll Cartwright +6, actual Cartwright +2). Not bad at all! Especially given that NV-01 covers parts of Las Vegas and its suburbs, which is a difficult place to poll - the population is transient, a lot of people work nights while most pollsters are dialing, and traditionally strong turnout operations in the state can drag reluctant or uninterested Democratic voters to the polls.

There are partisan pollsters on both sides that regularly release polls, though, and some of those didn’t fare as well. Trafalgar Group, a Republican polling outfit, released a lot of polls fairly close to the election, many of which turned out not to be accurate. A couple of particularly brutal misses were in the Colorado Senate race (poll O’Dea +1, actual Bennet +15) and Vermont (poll Welch +7, actual Welch +41!). Another Republican pollster, Remington Research Group, also released polling that ended up being too rosy for Republicans, particularly in the key races of Pennsylvania (poll Oz +3, actual Fetterman +5), Georgia (poll Walker +4, actual Warnock +1), and Arizona (poll Kelly +1, actual Kelly +5), though to their credit they did well in Ohio (poll Vance +5, actual Vance +7).

Mixing these lower-quality partisan polls in with higher-quality independent polls can hurt the accuracy of a polling average. Let’s look at Pennsylvania’s Senate race as an example. Marist College, which has a long history of polling and a good reputation, was one of the last pollsters in the field and showed Fetterman with a 6-point lead, very close to his final result of +5. Slightly older polls from Suffolk University/USA Today and Fox News (yes, Fox News actually does good polling) showed Fetterman with 2- and 3-point leads, respectively. But polls from the aforementioned Trafalgar and Remington showed Dr. Oz with narrow leads.

I do want to address a theory that floated around in Democratic Twitter circles, that the Republican polling released was a deliberate attempt to “flood the zone” with low-quality polling that looked good for Republicans. I have no idea if this was the motivation and don’t feel comfortable speculating on that. There are a lot of reasons to release a poll that have nothing to do with trying to move polling averages (good press coverage, fundraising, motivating supporters). I think it’s quite possible that these pollsters really believed they were doing more accurate polling than independent pollsters who have traditionally done well but struggled in 2016 and 2020. Regardless, they were quite wrong on this one, and that should affect how their work is covered and weighted in averages moving forward.

And one other thing before we move on: polls will often be wrong, but trying to predict which direction they will be wrong in is a fool’s errand. In 2012, there was an infamous attempt to “unskew” polling by some Republicans, believing party ID was too favorable to Democrats. In fact, polling slightly underestimated President Obama that year, though it was mostly accurate. In 2016, many people believed that even as good as the polling looked for Hillary Clinton, it wasn’t reaching enough people of color, who would surely vote against Donald Trump in droves. They did, of course, but the people actually not being reached were white voters without a college degree…and we all know how that turned out, let’s move on before my blood pressure spikes. This year, a lot of pundits and others assumed that polls would underestimate Republicans, as they did in 2020. Nope!

There are two big ones: FiveThirtyEight and RealClearPolitics. One of them came off a lot better than the other, though I think both have room for improvement.

FiveThirtyEight releases three versions of their forecast: a “Lite” version, which uses polling as the only input, a “Classic” version that adds in other relevant data points like candidate fundraising and past vote patterns, and the “Deluxe” version that includes ratings from traditional experts like Cook Political Report, Crystal Ball, and InsideElections. (These experts will look at polling and other quantitative factors, but their final rating is a subjective one rather than using a model to make predictions.)

The Lite version fared best out of the three this year and was pretty accurate. Overall, it projected an exact 50/50 shot for each party to take the Senate and a 1-in-4 chance Democrats would hold the House. Seems pretty accurate! Score one for polling averages! The problem is that adding other inputs in moved the forecast toward Republicans, which didn’t hold up. The Classic was nearly identical on the Senate, but put Republicans’ odds of retaking the House at 82%, making predictions in some House races that didn’t hold up as well. The Deluxe had Republicans with a 59% chance of retaking the Senate, including showing Walker as a slight favorite in Georgia, and an 84% chance of regaining the House. (FiveThirtyEight has acknowledged a small error in the Deluxe forecast, but says it was not enough to affect overall predictions.)

I think that should be cause for reflection among the FiveThirtyEight team. Are they adding value with extra factors, or is it bells and whistles that distract from what we see in polling? Regardless, I’ve read and used their work since 2008 and will continue to do so.

RealClearPolitics, on the other hand, did quite poorly this year. Let’s look at Arizona Senate forecasts as an example (FiveThirtyEight here, RealClearPolitics here). FiveThirtyEight underestimated Mark Kelly’s margin a little, estimating it as +2 (he won by 5), but showed him as a favorite. However, RealClearPolitics projected the race as a Republican pickup. Whoops! In another example, Washington state’s Senate race, RealClearPolitics projected just a 3-point win for Patty Murray, while FiveThirtyEight showed Murray as a heavy favorite with a projected 9-point win. Murray actually won by 15 - though neither projection was their best, FiveThirtyEight had a better handle on it than RealClearPolitics.

And here’s the thing: RealClearPolitics is run by Republicans. (A quick look at their news aggregation on the main page will confirm this, though I don’t recommend it. Yikes.) They have a reputation for being trustworthy independent analysts despite the personal views of Sean Trende and their other analysts. Frankly, I’m not sure this reputation is deserved. FiveThirtyEight’s method of deciding which polls to include is to include everything, but weight polls by pollster quality and recency. RealClearPolitics includes only a handful of the most recent polls. I’m not opposed to this approach in theory - it can potentially pick up late movement better than letting older polls fade over time - but they aren’t transparent or consistent about how they decide which polls to include and exclude in averages. I don’t think they deserve the benefit of the doubt after this year’s forecasts that clearly and inaccurately favored their preferred political party. If you have an interest in election forecasts, FiveThirtyEight would be my strong preference over RealClearPolitics.

Pretty damn good! Also terrified of the major life change I’m about to make. But mostly good.

I got the chance to personally work on some key races this year, most critically helping elect Tina Kotek as Governor of Oregon in a tough three-way race. Oregon is facing serious challenges on homelessness, but I truly believe Tina is the best candidate to tackle the issue with both compassion and determination, and she will protect the rights of all Oregonians. I also helped elect some great candidates to Oregon and Washington state legislatures and enshrine protections for reproductive freedom in Vermont’s state constitution. I didn’t get every race exactly right, but I was quite accurate in my weighting and, I think, provided the right strategic guidance to target races appropriately and win the close ones.

My firm, GBAO, did a ton of great work in other races too. My amazing coworkers helped re-elect Governors Tony Evers and Laura Kelly, flipped Pennsylvania’s Senate seat Democratic with John Fetterman, and guided Raphael Warnock to a lead in the initial race - we can never take anything for granted, but I believe Warnock will win the runoff.

I joined GBAO six years ago after asking several people I knew from the DCCC and elsewhere their thoughts about them. Everyone I asked told me some version of “they’re really good pollsters, but more importantly, they’re really good people.” Both have been proven correct. It’s been an honor to work with them and I genuinely don’t think I would have made it as long as I did in polling anywhere else. I’m so grateful I got to work here and looking forward to another 6-7 months of doing my part to leave it in a good place.

Before this election, I was really struggling with feelings of guilt over leaving politics behind. One thing I don’t talk about often is that I was seriously thinking about leaving after 2016. I was working brutally long hours, not being paid fairly for my job, and struggling with having to focus day in and day out on just how horrible Donald Trump was (and is). Sometimes in September and October 2016, I sustained myself with the thought that I just had to get to November and then once Hillary Clinton won I could go take a market research job for better pay and way fewer hours.

Oops.

Once Trump won, I felt like I couldn’t leave politics. I had to make things better. I felt compelled to help figure out what had gone so wrong in the polling and to elect candidates who would fight back against Trump’s horrible racist, sexist policies and actions. I sure as hell wanted a better situation within politics, and I found myself one, but I wasn’t going to leave. I made it through 2018 and 2020, doing good work but also struggling to keep my mental health in a good place with Trump in the Oval Office, all the horrible shit happening in the world, and the pressures of working in politics. (Worth noting, though, that among GBAO’s sterling qualities as a workplace is all the support my bosses and coworkers offered me when I was dealing with the worst of it.)

By this election cycle, I knew I was burned out, tired of living so far away from family, and in need of a career change. I decided to stay through 2022 and then pursue the career I really wanted. (Part of this was that I was deeply unenthusiastic about market research jobs when I got to the actually-applying part and so much more excited about law school and labor law, but selling out was also harder than I realized. A lot of hiring managers were skeptical of me.) But the guilt remained. Roe v. Wade had been overturned! Republicans might take the House and Senate and fuck up catastrophically, preventing Biden from any bit of progress. And what if polling had the same problems in 2020? What the hell kind of person was I to walk away when so much bad shit was happening or could happen?

If any of these thoughts seem irrational or not actually my responsibility to you, you are correct, and also I’ve already had that discussion with my therapist so you can keep that to yourself, please and thank you.

But Election Night came and went, and it helped me feel okay about leaving. Democratic campaigns and polling did good work this cycle and prevented the worst outcome. Some things even got better. That’s all you can ask for in this business. There are still problems to fix, but if I resolved to stay until all the problems were fixed I’d have to be carried out in my urn. It’s okay for me to leave, because there’s a whole lot of good people ready to pick up where I left off, doing good work and fighting the good fight.

I’m still nervous about a lot of things. It’s my nature, and I’m taking a lot of big risks - moving several hundred miles away, making a career change, committing myself to three years out of the workforce. I’m going to need to face my fears around those. But after November 8, I’m a little less worried about what I’m leaving behind. Thank God.