Oct. 3: Updated with a response from Statistics Canada
Canadians were less likely to fill out Statistics Canada’s National Household Survey if they were very poor. Or very rich. Or lived in less dense neighbourhoods.
Global News analyzed results of the voluntary survey that replaced the country’s most reliable compendium of population data and found it underrepresents people on both income extremes, as well as those most likely to need government services that rely on this data.
The voluntary National Household Survey has come under fire for being less accurate than its predecessor, the mandatory long-form census.
Response rates for the voluntary survey were sharply lower than 2006: over 26% of surveys weren’t returned, compared to 6.5% with the compulsory long-form census five years ago.
Because of that, Statistics Canada sharply lowered its standards: The point at which a neighbourhood’s results are suppressed due to a high non-response rate is 50% this year, compared to a 25% cutoff in 2006.
“The NHS, being a voluntary survey, had larger non-response rates than would be the case for a mandatory survey,” StatCan spokesperson Edelweiss D’Andrea said in an email. “For this reason, Statistics Canada needed to set a threshold which would reflect this change in methodology.”
Even big cities such as Toronto and Vancouver would have huge swaths of neighbourhoods suppressed were the old standards applied. (See the maps below.)
In Toronto, census tracts with higher numbers of non-English-speakers were more likely to fill out the survey.
The federal Conservatives scrapped the long-form census in 2010, replacing it with a voluntary National Household Survey. Industry Minister James Moore will decide how to proceed for the 2016 census cycle.
Bad data from this survey could have real implications for planning and service allocation. If you use city roads and plumbing, energy infrastructure, schools, immigrant settlement services, public housing or a multitude of other services, you rely on information from Statistics Canada’s census-turned-survey.
“What’s affected the most is our ability to allocate resources well,” Centre for Policy Alternatives senior economist Armine Yalnizyan said in an interview earlier this month. “We’ll be making a lot more mistakes. We’ll be wasting a lot of money.”
“The data from the NHS are very useful,” responds Statistics Canada’s Gabrielle Beaudoin. “In terms of overall population, NHS estimates are published for 97% of the population. At national, provincial and census metropolitan area levels, the accuracy of the data is good. There is more volatility for some smaller areas such as census tracts. In those instances, users are encouraged to use data at higher geographic levels.
“The example cited in this story is one case where a group is underrepresented in the respondents to the survey in some areas. There is a risk of bias in the estimates when respondents differ from non-respondents. While our procedures mitigate the risk of bias, this risk is not eliminated. This applies to all voluntary surveys.”
“As always, we continue to look at ways to improve the next census cycle while protecting Canadians’ privacy,” said Sebastien Gariepy, a spokesperson in Industry Minister James Moore’s office.
Ontario’s still deciding whether to use household survey data (among other things, the Education Ministry wanted data for “several school-aged variables”). Toronto won’t use the data for trendlines at all.
A Vancouver spokesperson told Global News the city has “concerns” the quality of the data “may affect the City’s ability to identify changing socio-economic trends and target services where they are needed most.”
This effect was predicted by critics of the survey, which replaced the compulsory long-form census last used in 2006.
Compliance was also lower at the top of the income scale: Response rates fell for tracts with a median household income of $130,000 and higher.
Toronto
Almost all Toronto-area census tracts pass muster if you hold them to Statscan’s 2011 standards. But under big chunks of North York, west-end Toronto and areas around Pickering, Durham and Peel had response rates low enough to exclude them under 2006 standards.
Alberta
Under the old standards, data for much of Edmonton, Lethbridge and Medicine Hat – as well as Airdrie and Bragg Creek – would’ve been too unreliable to use.
Vancouver
Results in Langley, Aldergrove and Mission were deemed good enough for Statscan’s new standards – but they wouldn’t have been five years ago.
Comments