Opinion polls play a central role in every election campaign.
They give us a glimpse into the mindset of Canadian voters and how they react to the campaign.
While polls capture sentiment on a wide variety of questions, most people (including the media) focus on the “horse race” question to answer who’s winning in popular support, and to project who’s likely to win on election day.
Caution must be used when interpreting opinion polls. They’re not all created equal. Dismissing them outright is as foolish as trusting them blindly.
But with healthy scrutiny, they can be a useful source of information for voters who are weighing their options: whether to vote strategically, support the underdog or jump on the bandwagon.
So here’s what you need to know to make sense of polls in this election campaign.
When did the polling happen?
A poll is a snapshot in time and is not necessarily predictive of future behaviour.
Polls that are in the field (surveying Canadians) for only a day or two are more highly influenced by a single news cycle, such as a new tariff announcement by U.S. President Donald Trump or a policy announcement by a party. Polls that are in the field for too long tend to be lagging indicators – public opinion may have substantially shifted over the field window, such that the results are no longer up to date.
How did the pollster collect their data?
The second area to note about polling is the methodology, or how polling agencies collect their data. Each methodology has its idiosyncrasies, with varying response rates and degrees of coverage error – that is, who would be excluded from participating in the poll. For example, someone without internet access would not be able to participate in an online poll.
Three main methodologies are employed.
Get breaking National news
The first is online polling, which relies mostly on pre-recruited opt-in panels of people who have agreed to participate in surveys from time to time. Obviously, those who are not registered to be on these panels would be excluded from participating. Quotas are used to ensure that the poll has enough men, young people or British Columbians, for example. And when the demographic profile of those responding to the survey doesn’t reflect the demographic makeup of Canada, weighting is used to ensure that the sample of respondents matches the Canadian population according to the census.
But there’s a problem: given the opt-in nature, the sample from these panels tends to skew more activist, on both sides of the political spectrum, but particularly on the progressive side. NDP support tends to be overstated in these polls.
The second methodology employed by some pollsters is IVR – interactive voice recording – robo polls. While technically random-sampling, which is a good thing, the response rate is dreadfully low, effectively excluding a large swath of the population – mostly younger Canadians – who would never entertain a phone call from a voice recording.
The sample gathered tends to skew very old, so the data needs to be manipulated, or weighted, to reduce the influence of those oversampled and increase the influence of those under sampled.
The final methodology used by some pollsters is live-operator telephone interviewing, which incorporates real humans dialling Canadians on both their cellphones and their landlines. For decades, this was considered the gold standard for polling in Canada (and still is by many) because it excludes the fewest people from being eligible to participate, since almost everyone has a phone. But as response rates decline and with fewer households owning a home phone, even this methodology poses reliability challenges.
In this election campaign, there have been polls published within a week of each other that show two competing narratives: a double-digit lead for the Liberals and a statistical tie with the Conservatives. Clearly both can’t be right, but the variation might at least be partially explained by the field dates or the methodology employed.
What is the sample size of the poll?
Sample size is another key consideration when evaluating polling. In this case, larger isn’t necessarily better.
A large sample that does not accurately reflect the voting population (due to some bias or coverage error) is more likely to be wrong than a smaller sample size that does accurately reflect the voting population.
And even with the most sophisticated random sampling methods, every 20th poll will be a miss – with the truth falling outside of the margin of error of the poll. If you see a rogue poll – an outlier – wait for another poll to confirm or deny it before jumping to conclusions.
Look for the trends.
Winning the popular vote doesn’t always mean winning the election
Finally, the national popular vote does not always predict who will win the most seats.
In both 2019 and 2021, the Conservatives won more votes, but the Liberals won more seats. That’s because the Conservative vote is inefficient: the party wins by wide margins in Saskatchewan and Alberta but loses by narrow margins in Ontario. Lots of votes translating into fewer seats.
Look deep into the data tables published by the pollsters to find the results in key regions like the 905 area surrounding Toronto, southwestern Ontario and the Quebec City area. Since this is where a lot of competitive seats are, what happens in these regions will determine the outcome of the election more than the national popular vote tallies.
Voter turnout matters too
Perhaps the biggest caveat with polling is it surveys the general population, but about 60 to 70 per cent will show up. However, most Canadians will tell pollsters that they intend to vote. In order to deal with this, some pollsters employ turnout models to predict who will show up to vote and how that subset of voters will impact results – in U.S. elections, it’s absolutely critical, but in Canada these models haven’t proven necessary.
Federal polling in Canada is quite accurate. But that doesn’t mean you should let your guard down. Scrutinize the polls and kick the tires. Look at the timing of the polls, methodology, sample size and weighting. Be skeptical, but don’t let the results alone sway your opinion of the poll itself. And if you’re satisfied with the science behind the numbers, digest the results and take comfort in the fact that you know more about what’s going on with Canadians than you did before.
Sean Simpson is a senior vice-president with Ipsos Public Affairs.
Comments