TORONTO – Veteran pollsters have a word of advice for Ontario voters getting seasick from riding the shifting tides of the province’s election survey results: Ignore them, they’re probably wrong anyway.
At least 14 polls have been released since Premier Kathleen Wynne dissolved the legislature at the beginning of the month, and there’s no consensus yet on what the result might be on June 12.
One day a pollster reports Tim Hudak’s Conservatives are poised to form a majority government. The next day, a different pollster is out with numbers suggesting the Liberals may be re-elected.
Seasoned pundits observing the campaign from the sidelines say the numbers flooding the headlines are inherently flawed and of limited value to the discerning voter.
Polls commissioned by mainstream media outlets share few traits in common, they said, adding firms use different methodologies, questions and sample sizes to obtain their results.
The one quality the polls do share, however, is a focus on surface questions that don’t tell the full story of what drives voters to the ballot box and don’t offer an accurate picture of what will happen once they get there, the pundits said.
Indeed, unreliable polls stole headlines away from election victors in Quebec and British Columbia during their most recent campaigns, as the final vote counts differed sharply from survey projections.
Veteran pollster Allan Gregg, who currently serves as Principal at Earnscliffe Strategy Group, said the general questions around voter intentions don’t mean much, since most people make their final decisions in the waning days of the campaign.
Polling firms working for the media also don’t have the financial resources to develop sound survey practices, he said, adding the political parties are the ones with the funds to invest in developing solid numbers and tracking more meaningful trends.
“It’s an entirely different beast, what the political parties are doing versus what the media is doing,” Gregg said in a telephone interview.
Daniel Cohn, professor of Public Policy and Administration at York University, said the problem runs even deeper than lack of finances.
In Depth: Ontario Election 2014
Accurate research takes both money and time to amass, he said, adding the demands of today’s political climate make quick, cheap and inaccurate data collection the only option.
“To do a decent public opinion poll that’s going to be representative of public opinion because it’s going to be as close to random as you can get, that’s going to take you a couple of weeks,” he said. “And in a four-week election campaign and a 24-7 media cycle, that’s just not acceptable.”
Mainstream polls collect their data through two primary methods, both of which experts say have fundamental flaws.
Traditional online polls survey a pool of participants who have volunteered to offer their opinions. As a result, pollsters say the findings are not random and are therefore not necessarily representative of the whole population.
In Photos: Seat projections and polls as of May 27
This view is shared by the Marketing Research and Intelligence Association, the polling industry’s professional body, which says online surveys cannot be assigned a margin of error due to their lack of random sampling.
Experts say lack of randomness also plagues the cheapest and most common form of telephone polls, known as Interactive Voice Response (IVR) or automated polls.
Surveys that rely on one member of a household answering general questions with the press of a button automatically lose out on other views in the same household. Worse, they say, is the fact that telephone polls by their nature tend to favour older voters who are more likely to both own a landline and pick it up when it rings.
Cohn said technology has compromised this method even further by shrinking the size of the respondent pool. People are increasingly ditching their landlines in favour of unlisted mobile phones and use common features like call display to screen out pollsters and their questions, he said.
Some research companies have tried to address this by revamping their IVR systems to keep up with the times.
Frank Graves, president of Ekos Research, says his firm has included cellphones in the polls conducted for the Ontario election.
His team also favours an approach of calling participants more than once and drafting multiple versions of the same questions to limit bias.
Both exercises cost time and money, he concedes, adding the gap between survey results and election-day figures usually has more to do with poor voter turnout than faulty research.
“That doesn’t mean it’s just ‘a pox on all your houses.’ No. You can do good, sound polling,” he said.
“It would be great to devote more resources to do it better, but we don’t put out any poll that we don’t think is right.”
Some experts also say that telephone polling has its place when conducted the old-fashioned way.
Dimitri Pantazopoulos relied on live phone interviews while collecting polling data for B.C. Premier Christy Clark during the province’s last election.
While nearly all public surveys forecasted a win for the NDP, Pantazopoulos masterminded the only poll to accurately predict that Clark’s Liberals would carry the day.
Pantazopoulos said his focus wasn’t on soliciting responses from the largest swath of the population, adding sample sizes become less relevant after the first few hundred interviews.
He said his approach of running a smaller rolling poll throughout the campaign, rather than concentrating on specific field survey dates, allowed him to address what he feels is the key shortcoming of most mainstream surveys.
“They’re focused on the ballot number. Nobody’s sitting there and asking people what’s motivating people to vote, what’s going to change the undecided, what will move weak voters over to the other side,” Pantazopoulos said of media polls.
Pollsters then compound their problems by trying to explain their findings using factors they haven’t analyzed.
“They say, ‘oh, these guys have gone down. It must be because of the 100,000 public service cuts,”‘ Pantazopoulos said in reference to one of Hudak’s most controversial campaign pledges. “Well, maybe, but it may be because people don’t like the leader’s hair cut or something. We don’t know. Why go through all the effort of being scientific and then guess as to what the reason is?”
Gregg’s advice for voters musing on ballot-box outcomes?
“Wait till election day. You’ll find out sooner or later.”
Comments