[NEWSCASTER] “A new poll has Kamala Harris and Donald Trump locked in a dead heat.” [NEWSCASTER] “Latest Wall Street Journal poll...” [NEWSCASTER] “New Washington Post poll out this morning.” [NEWSCASTER] “A new exclusive poll.” New polls have been released what feels like every day this election cycle. [NEWSCASTER] “Trump ahead by two.” [NEWSCASTER] “Harris up by four.” [NEWSCASTER] “Deadlocked race.” [NEWSCASTER] “A tied race.” And you might be staring at ones like these, hoping it'll answer the question of who will win this presidential election. But there's a lot more to these numbers than meets the eye. And there's a few reasons they'll disappoint you if you rely on them to predict the future. [NEWSCASTER] “It is still anybody's race.” These numbers are from the Times Siena poll on October 25th,

which suggests that 48% of American voters planned to vote for Kamala Harris, while 48% planned to vote for Donald Trump. But of course, pollsters didn't survey every American. They surveyed a sample of them. Today, pollsters find their sample of people to survey through a mix of phone calls, text messages, online surveys, or by mail. And if you're thinking, who even answers those kinds of things? The answer is... not a lot of people. The response rate for polling can be as low as 1 to 2%, which means pollsters have to reach out to as many as 100,000 people to pull 1,000 people, which is a typical sample size for a national poll, or 50,000 people to reach the roughly 500 people typical for a state poll. Which sounds small, but that amount of people can still be accurate at measuring the opinion of a population if you read the polls with a few caveats in mind.

The first thing to know about these numbers is there is a margin of error, which means this number is best thought of as the center point in a range of possible outcomes. On average for a sample size of 1,000, any number that is 3% different in either direction could still technically be accurate. And that's simply because any time you depend on a sample to represent a larger population, there's a lack of precision. A general rule of thumb is that the bigger the sample size, the smaller the margin of error. But there's only so much pollsters can do. Polling more people is a lot more costly and time-consuming, which matters because big news events can alter people's opinions. And ultimately, increasing the sample size can have diminishing returns on accuracy. [COURTNEY KENNEDY] “And it really starts to level off around 1,000... 1,500...2,000 interviews in terms of if you keep putting in more effort,

are you going to be more accurate? The answer becomes no pretty quickly. And part of it is because as a pollster, you're either doing a phone poll or you're doing an online poll. And if you keep interviewing thousands and thousands of more people through that mechanism, the next 10,000 are going to be just like the 2,000 people you already spoke with. This margin of error doesn't account for other errors in the polling process. Things that are harder to measure, like excluding certain demographic groups, low response rates among certain groups, and people misunderstanding the survey questions or misreporting their opinions. Which means this 3% margin is actually a low estimate. [KENNEDY] The rule of thumb is that you should actually double it. Before a poll is released, pollsters have to do some work to make sure the sample size looks just like the entire population. Based on sources like the US Census and voting files. It's called weighting.

For one, some groups are more likely to be reached by pollsters and respond to them. [KENNEDY] “You have, usually too many college graduates, often too many women, and maybe too many white adults. If you look at 2016 and 2020, Trump supporters were more difficult to reach in surveys. After data collection, there's ways in which your sample now departs from being random and representative. Now, weighting is how we pollster fix that.” So if each individual in a poll starts off with a value of one, weighting is when people who are underrepresented compared to the larger population get a value higher than one. While groups who are overrepresented get a smaller value. So if our imaginary sample here was made up of 60% women and 40% men, pollsters would make the women count less, and the men count more to match the fact that about 52% of US voters are women, while 48% are men.

They repeat this for factors like age, race, and geographic location, too, until each individual is weighted in a way that makes the sample a good stand-in for the larger population it represents. Recently, pollsters started consistently weighting for education, too. In the 2016 election, some state polls underestimated Donald Trump's support among non-college-educated people, an oversight that resulted in an election outcome few saw coming. Weighting respondents based on how their education level matches the general population is intended to help solve for that. In the 2024 election, pollsters are increasingly weighting for things like how people voted in past elections and party affiliation, too, to avoid any other ways it could be over- or underrepresenting a candidate's supporters. But the thing is, weighting a sample to match the general US adult population is pretty simple. Through sources like the US census, pollsters know how many people are in each age group, or gender, or racial group, or whether someone went to college.

But for a poll to be a precise prediction of an election outcome, pollsters are attempting to weight against an unknown population: the people who will actually vote in the future, on election day. There's no guarantee that anyone surveyed ahead of an election will actually vote. Some polls make educated guesses, counting people more who say they are likely to vote, or who are registered to vote, or who have voted in the past. But at most, only two-thirds of Americans of voting age turn out to vote. And pollsters can only make informed guesses on who those people will be. Polls are good for a lot of things – at capturing where the population stands on big issues. Or figuring out which states are really going to be decisive in the election outcome. But when the polls on voting choice are as close as they are in this election, the main thing they tell us is – simply that the race is close. And the only way we'll know who will win is to pay less attention to the polling news cycle,

and wait until the votes are counted. Before you go, I want to tell you about Vox's new membership program. We put this video together as quickly as we could when we realized there was a lot of curiosity, confusion, and obsession over the election polls. This kind of journalism takes a lot of work: interviewing experts, writing, editing, and animating. Our membership program is the best way to support this kind of work – both here on our YouTube channel but also on Vox's news site. If you join, you'll get access to more articles from Vox's newsroom and other member-only benefits. I hope you'll consider becoming a member by going to the link on the screen or the one in the video description below.

What polls can actually tell us

Vox helps you understand our world.

Vox video is Bridgett Henwood, Cath Spangler, Adam Freelander, Joey Sendaydiego, Coleman Lowndes, Christophe Haubursin, Kim Mas, Laura Bult, Ed Vega, Rajaa Elidrissi, and with research support from Melissa Hirsch. 

Become a member today: https://www.vox.com/support-now

Check out our FAQ on permissions and licensing of our videos: https://www.voxmedia.com/pages/licensing

✉️ Write us: voxvideo@vox.com


What polls can actually tell us

Scripts