National Household Survey: Here’s what we know about what we don’t know
ABOVE: The Canadian government’s decision to scrap the long-form census for a voluntary survey is facing renewed scrutiny. Mike Le Couteur reports.
Who filled out the National Household Survey?
If you didn’t, chances are you’re an anglophone living in rural Canada, especially southern Ontario, western Manitoba or Interior B.C. You’re more likely to be single than married. And chances are you voted Tory.
A Global News analysis reveals more information on what we don’t know about Canada’s population – what groups were underreported in what used to be the gold standard for Canadian population data.
Turns out that by quashing the mandatory long-form census, the federal Conservatives appear to have cannibalized data-gathering on their own political base: Ridings that voted Tory in the 2011 election reported the country’s highest non-response rates to the 2011 National Household Survey. The greater Tory support, the less likely they were to fill out the survey.
In depth: Statscan’s National Household Survey
The very poor and the very rich were also less likely to fill out the survey; ridings with the most anglophones were less likely to complete it; married couples were moderately more likely to fill out the survey, and Canadians in rural areas - with the significant exception of the remotest areas of the North - were less so.
The federal government ended the mandatory long-form census in 2010 on the grounds that it’s an unnecessary invasion of privacy. Now local governments, epidemiologists and service providers across Canada – in Tory heartlands, most of all – are trying to figure out whether they trust this data enough to use it – and what to do if they don’t.
“It’s obviously disappointing that I am now using some of the support dollars for various programs to actually overcome the previous long-form census decision”
<a href="http://www.fraserhealth.ca/" target="_blank">Fraser Health</a>, which covers a huge swath of British Columbia's Lower Mainland, may have to cut funds from health programming to collect data it would normally get from the census. The Abbotsford and Mission areas, both represented by Conservative MPs, had especially low response rates.
“It’s obviously disappointing that I am now using some of the support dollars for various programs to actually overcome the previous long-form census decision. And there are many groups in the same boat,” said Chief Medical Officer Paul Van Buynder.
Peterborough, Ont., epidemiologist Andrew Kurc sat down Friday to draft recommendations on what the local health unit should do with National Household Survey stats. Short answer? He doesn’t know yet.
“We’re still evaluating other options,” he said in an interview. “One of our options is to continue to use the old census data, which is a lot more reliable, it looks like, at this point.”
As Global News reported last week, Peterborough has the dubious honour of the highest non-response rate of any Canadian city. But nearby Haliburton-Kawartha-Brock has the highest of any riding, at 43.7 per cent.
“Surveys can be adjusted”
MP Barry Devolin isn’t concerned.
“I don’t share the concern that this information, at the end of the day, will be skewed or less accurate than it might have been if everybody filled it out,” he said in an interview.
“Something is lost, but I guess the flip side of it is, given what I would say are very legitimate privacy concerns that people have, how far is it appropriate for the government to go to gather this information, and what’s not appropriate? … The picture that we have of a population will not be as clear as it would have been if we had compelled them. I think that’s very much a difference of degree, as opposed to, say, people who jump to the conclusion of ‘Before it was accurate; now it’s not.’”
(All that being said, when it comes to why his constituents tended, more than anyone else, not to fill out the survey, he says, “as an MP, I’m as curious about that as anybody.”)
"Surveys can be adjusted. … There’s a lot of existing information including other ridings to compare these things to.”
“If the people who are policy-makers don’t have the data, how do you make good decisions?”
Some statisticians are less sanguine. Former Statistics Canada chief Munir Sheikh, who resigned over the switch to a voluntary survey, has called this version “largely useless.”
“It’d be kind of ironic if one of the losers from the abolition of the long-form census was the Conservative Party, because they’re not going to have information about their base,” she said.
At the same time, this makes the Conservative Party’s own data collection all the more important – and underscores the new prominence of private data collection – through pollsters, loyalty programs or analytics companies – in a post-long-form census environment.
Tory spokesperson Cory Hann did not respond to a request for comment. Tony Clement, who was Industry Minister when the long-form census was quashed and whose Parry Sound-Muskoka had the second-highest non-response rate in the country, did not respond to requests for comment Tuesday.
Industry Minister James Moore hasn’t yet said whether he’ll go with a mandatory long-form census or voluntary survey for 2016. Statistics Canada is in the midst of making recommendations to his office, but won’t say what it’s recommending.
“The census is an incredibly important part of the infrastructure of this country. You can’t govern effectively and you can’t make private sector and local government decisions without reliable, effective data,” Moore said in an interview last week. “We do believe we have that data in the census that we have before us now but certainly if there are concerns and things need to be improved going forward into the 2016 census we will do certainly our assessment of this past year’s census and make adjustments going forward.”
Interactive: Explore Canada’s ridings and track National Household Survey refuseniks
(All maps measure non-response rates: The darker the colour, the fewer people sending back their surveys)
Canada’s most populous province boasts the country’s census refusenik capitals: Haliburton-Kawartha-Brock had the highest non-response rate, followed by Parry Sound Muskoka. By contrast, there was significantly higher participation in the Ottawa area, especially around Ottawa-Orleans.
Direct link to map
British Columbia’s Southern Interior also had especially high non-response rates; so did Abbotsford-Mission, which worries the region’s chief medical officer.
Direct link to map
The western Manitoba region around Dauphin was also one of the ridings with the lowest rate of response. There were plenty of census refuseniks in Lethbridge, AB, as well.
Direct link to map
Little Prince Edward Island has some of the country’s highest non-response rates to the National Household Survey. But Western Nova and southwest New Brunswick are up there, as well.
Direct link to map
What does choosing to fill out (or not) a voluntary survey say about you? “It’s like any other kind of volunteering,” says Carleton’s Frances Woolley. “Culture, values, personality, a whole lot of other things influence that.” So she says it’s no surprise that, in some cases, ridings where a high percentage of residents have a mother tongue other than English or French also had high survey response rates. But the more anglophones a riding has, the less likely it was to fill out the household survey – perhaps because linguistically homogeneous areas also tend to be more rural.
We found something interesting when we looked at how a neighbourhood’s density’s linked to its National Household Survey participation: For the most part, the denser an area is, the more likely its residents are to respond. But that changes for super-dense downtown ridings, many of which would also have mixed incomes (and we know particularly poor people tend to participate less).