Some mid-term polls have been spot on — but figuring out which pollsters and poll aggregators to believe can be a challenge

by W Joseph Campbell, American University School of Communication [This article first appeared in The Conversation, republished with permission]

Pollsters indulged in airy self-adulation after the 2022 midterm elections. They explained that pre-election polls did well overall, signaling the results of high-profile US Senate and governorship elections.

Alluding to the stunning failures of the polls in 2016 and 2020, when the 2022 results were released, Joshua Dyck, director of UMass Lowell’s polling center, claimed: “The death of the polls has been grossly exaggerated.”

Nate Silver, a well-known data journalist and election forecaster, went to twitter to announce that the 2022 midterm elections were “one of the most accurate election years ever”.

Still, a sense of doubt lingered: while they didn’t repeat their failures in the recent national election, the 2022 polls have been patchy in accuracy rather than spectacular, and performance ratings often depended on which poll was consulted. Or maybe more specifically, which survey aggregation site was consulted. Aggregators typically compile and analyze results reported by a variety of pollsters. They often adjust the composite data to emphasize the results of recently completed surveys or to minimize the impact of unusual or “outlier” surveys.

Missing, near and far

As compiled by popular website RealClearPolitics, polls in the key 2022 Senate elections in Arizona, Colorado, Florida, New Hampshire, Pennsylvania and Washington collectively missed winning margins by more than 4 percentage points.

Differences between poll averages and results were particularly striking in Colorado, Florida, New Hampshire and Washington, where incumbents won easily. For gubernatorial elections, results in Arizona, Colorado, Florida, Michigan, Pennsylvania and Wisconsin differed from the poll average by 4 percentage points or more.

Predictions published on FiveThirtyEight.com by Silver differed slightly less significantly from the results than those by RealClearPolitics – but still expected closer Senate races than Colorado, New Hampshire and Pennsylvania.

Expectations that Republicans would undoubtedly win resounding victories were bolstered by the predictions by RealClearPolitics. It projected that the GOP would win three Senate seats and control the upper house 53 seats to 47 – a result that turned out to be illusory.

Although hedged, the final, so-called “deluxe” forecast posted by Silver on FiveThirtyEight.com and updated on Election Day did little to dampen expectations of a GOP surge. The forecast says Republicans have a 59% chance of gaining control of the Senate.

“To be honest,” Silver wrote, “59 percent is a sufficient advantage that if you offered to make me an even money bet on Republicans, I would take it.”

elections and disagreements

To say poll performance was patchy in 2022 doesn’t mean the election polls were all off target.

Far from it.

For example, the latest Siena College/New York Times polls accurately pointed the direction of the Senate races in Arizona, Georgia and Pennsylvania.

Still, as I noted in my book Lost in a Gallup: Polling Failure in US Presidential Elections, “It’s a rare election that doesn’t raise any controversy of opinion.” And that’s not so surprising, since polls from a variety of public bodies, some of which have partisan orientations.

This time the controversy revolved around Republican-leaning pollsters like the Trafalgar Group and the inclusion of those polls in the averages compiled by RealClearPolitics. Critics claimed the inclusion of such data led RealClearPolitics to exaggerate Republican prospects. RealClearPolitics senior election analyst Sean Trende disputed such an interpretation as “a theory that doesn’t work well.”

Trafalgar, which had been rated A-minus for accuracy by FiveThirtyEight.com in 2021, saw its polls fail conspicuously in 2022. In the race for the US Senate in New Hampshire, for example, the latest Trafalgar primary poll showed Republican Don Bolduc had a narrow lead. Bolduc lost by 9 percentage points to incumbent Maggie Hassan.

Trafalgar also estimated that Republican Tudor Dixon ended the campaign by a slim margin over Gov. Gretchen Whitmer in Michigan. Whitmer won by 10.5 points.

These were no small mistakes, and Trafalgar’s inaccuracies drew criticism from even friendly sources. “They were not reliable indicators of what was to come,” Scott Johnson wrote on the Republican-leaning Powerline blog. Trafalgar did not respond to an email seeking comments on its polling performance in 2022.

Election failures, however, tended to be bipartisan. Data for Progress, a Democrat-leaning pollster rated “B” by FiveThirtyEight in 2021, estimated closer Senate races than what turned out to be in Colorado and New Hampshire, and signaled the wrong winners in Arizona and Nevada.

Data for Progress nonetheless seemed eager to claim the success of its polls, posting what appeared to be online and saying it “beat the poll averages and was more accurate than any other pollster” at midterm. The draft contained several placeholders marked with “xx” that indicate where data points should be inserted.

Advertisement reading
In 1940, Gallup praised the accuracy of its polls in an advertisement in the newspaper industry’s Editor & Publisher magazine. Screenshot, publisher and publisher, 11/9/1940

Pollsters are not afraid to congratulate themselves

So what can we take away from the surveys for the midterms of 2022?

The results reaffirmed that election polling is an uneven and risky endeavor, especially at a time when some pollsters are experimenting with new methods of reaching potential respondents, while others still rely on traditional, telephone-based techniques.

The 2022 results also confirmed a self-gratifying impulse that’s never very far away for practitioners in a field that has seen many failures and disappointments.

Pollsters are not necessarily shy about boasting when their estimates are reasonably close to the election results. This trend has been observed intermittently for more than 80 years since George Gallup placed double-page spread advertisements in Editor & Publisher magazine in 1940 and 1944 to proclaim the accuracy of his polls in the presidential elections of those years.

The midterms also confirmed the news media’s insatiable appetite for poll results. New polling data — much of it produced or commissioned by the news outlets themselves — seemed inevitable in the closing days of the 2022 campaign. As is usual in national elections, polls shaped expectations that in some cases faded as the votes were counted.

W. Joseph Campbell, Professor of Communication Studies, American University School of Communication

This article was republished by The Conversation under a Creative Commons license. Read the original article.

Source