National Household Survey: Here’s what we know about what we don’t know

ABOVE: The Canadian government’s decision to scrap the long-form census for a voluntary survey is facing renewed scrutiny. Mike Le Couteur reports.

Who filled out the National Household Survey?

If you didn’t, chances are you’re an anglophone living in rural Canada, especially southern Ontario, western Manitoba or Interior B.C. You’re more likely to be single than married. And chances are you voted Tory.

A Global News analysis reveals more information on what we don’t know about Canada’s population – what groups were underreported in what used to be the gold standard for Canadian population data.

In depth: Statscan’s National Household Survey

The federal government ended the mandatory long-form census in 2010 on the grounds that it’s an unnecessary invasion of privacy. Now local governments, epidemiologists and service providers across Canada – in Tory heartlands, most of all – are trying to figure out whether they trust this data enough to use it – and what to do if they don’t.

“It’s obviously disappointing that I am now using some of the support dollars for various programs to actually overcome the previous long-form census decision”

Tweet This Click to share quote on Twitter: "It's obviously disappointing that I am now using some of the support dollars for various programs to actually overcome the previous long-form census decision"

“It’s obviously disappointing that I am now using some of the support dollars for various programs to actually overcome the previous long-form census decision. And there are many groups in the same boat,” said Chief Medical Officer Paul Van Buynder.

Peterborough, Ont., epidemiologist Andrew Kurc sat down Friday to draft recommendations on what the local health unit should do with National Household Survey stats. Short answer? He doesn’t know yet.

“We’re still evaluating other options,” he said in an interview. “One of our options is to continue to use the old census data, which is a lot more reliable, it looks like, at this point.”

“Surveys can be adjusted”

Tweet This Click to share quote on Twitter: "Surveys can be adjusted"

MP Barry Devolin isn’t concerned.

“I don’t share the concern that this information, at the end of the day, will be skewed or less accurate than it might have been if everybody filled it out,” he said in an interview.

“Something is lost, but I guess the flip side of it is, given what I would say are very legitimate privacy concerns that people have, how far is it appropriate for the government to go to gather this information, and what’s not appropriate? … The picture that we have of a population will not be as clear as it would have been if we had compelled them. I think that’s very much a difference of degree, as opposed to, say, people who jump to the conclusion of ‘Before it was accurate; now it’s not.’”

(All that being said, when it comes to why his constituents tended, more than anyone else, not to fill out the survey, he says, “as an MP, I’m as curious about that as anybody.”)

“If the people who are policy-makers don’t have the data, how do you make good decisions?”

Tweet This Click to share quote on Twitter: "If the people who are policy-makers don't have the data, how do you make good decisions?"

Some statisticians are less sanguine. Former Statistics Canada chief Munir Sheikh, who resigned over the switch to a voluntary survey, has called this version “largely useless.”

“It’d be kind of ironic if one of the losers from the abolition of the long-form census was the Conservative Party, because they’re not going to have information about their base,” she said.

At the same time, this makes the Conservative Party’s own data collection all the more important – and underscores the new prominence of private data collection – through pollsters, loyalty programs or analytics companies – in a post-long-form census environment.

Tory spokesperson Cory Hann did not respond to a request for comment. Tony Clement, who was Industry Minister when the long-form census was quashed and whose Parry Sound-Muskoka had the second-highest non-response rate in the country, did not respond to requests for comment Tuesday.

Industry Minister James Moore hasn’t yet said whether he’ll go with a mandatory long-form census or voluntary survey for 2016. Statistics Canada is in the midst of making recommendations to his office, but won’t say what it’s recommending.

“The census is an incredibly important part of the infrastructure of this country. You can’t govern effectively and you can’t make private sector and local government decisions without reliable, effective data,” Moore said in an interview last week. “We do believe we have that data in the census that we have before us now but certainly if there are concerns and things need to be improved going forward into the 2016 census we will do certainly our assessment of this past year’s census and make adjustments going forward.”

Interactive: Explore Canada’s ridings and track National Household Survey refuseniks

(All maps measure non-response rates: The darker the colour, the fewer people sending back their surveys)

Ontario

Canada’s most populous province boasts the country’s census refusenik capitals: Haliburton-Kawartha-Brock had the highest non-response rate, followed by Parry Sound Muskoka. By contrast, there was significantly higher participation in the Ottawa area, especially around Ottawa-Orleans.
Direct link to map

Maritimes

Little Prince Edward Island has some of the country’s highest non-response rates to the National Household Survey. But Western Nova and southwest New Brunswick are up there, as well.
Direct link to map

What does choosing to fill out (or not) a voluntary survey say about you? “It’s like any other kind of volunteering,” says Carleton’s Frances Woolley. “Culture, values, personality, a whole lot of other things influence that.” So she says it’s no surprise that, in some cases, ridings where a high percentage of residents have a mother tongue other than English or French also had high survey response rates. But the more anglophones a riding has, the less likely it was to fill out the household survey – perhaps because linguistically homogeneous areas also tend to be more rural.

We found something interesting when we looked at how a neighbourhood’s density’s linked to its National Household Survey participation: For the most part, the denser an area is, the more likely its residents are to respond. But that changes for super-dense downtown ridings, many of which would also have mixed incomes (and we know particularly poor people tend to participate less).