Statistics Canada’s National Household Survey is too unreliable to compare with previous long-form censuses, Toronto has decided.
Canada’s biggest city won’t use the new survey to inform historic trendlines unless Statistics Canada provides better technical information that puts planners more at ease.
Toronto, like other cities and a multitude of government and private organizations, relies on data from the long-form census to get a sense of where the community’s going.
At least, it used to.
Now, every city department – from planning to transit, traffic and social services – will need to find another way to figure out how Toronto’s communities are changing.
“The city of Toronto is not doing any type of historical comparisons between the 2006 census and the equivalent 2011 National Household Survey variables,” Harvey Low, head of the city’s Social Research and Analysis Unit, said in an interview.
The federal Conservatives scrapped the mandatory long-form census in 2010, replacing it with a voluntary survey, which asks largely the same questions but whose accuracy has come into question: These surveys get a much lower response rate, among some populations more than others.Ultimately, critics argue, the only thing a voluntary survey tells you is who likes to fill out surveys.
Ottawa contends the new survey’s still usable; Statistics Canada, for its part, has released data with caveats that high non-response rates in some areas could make the information less reliable.
Meanwhile, governments and agencies around the country are rethinking the way they use information and casting about for alternatives.
Will ditching census trendlines pose a problem for Toronto’s decision-making?
“Only in the sense that the true indication of a neighbourhood outcome is how things change over time,”Low said. “The city needs that consistency in data over time if we are to make better informed decisions around place-based planning.
“I wouldn’t say that we can’t do our jobs. It’s that we have one less toolset to do it, that’s all. We’re looking at alternatives.”
One thing that worries Low and his colleagues is that they don’t know what they don’t know: They still can’t be sure how to gauge the survey’s accuracy.
“The whole issue of non-response bias, certain groups not responding to the survey remains an issue. But we haven’t received any detailed, any technical documentation,” he said.
This becomes especially tricky when you’re trying to drill down into super-specific subsets of the population.
“The thing that was fantastic about the census … is you could really drill down into specific groups,” Carleton University economist Frances Woolley said in an interview. “I you wanted to look at the new Canadian parents of children with disabilities in Vancouver, you could do that.”
Others have the same reservations: “The Ontario government continues to assess data from the National Household Survey as it is released,” Finance Ministry spokesman Scott Blodgett said in an email last week, when StatsCan released its second tranche of data.
“This data potentially gives helpful information for analysis,” Blodgett said. “However, the government will use additional prudence in using data from the National Household Survey because of issues related to quality and comparability.”
Low said the city may change its mind “once we get further clarification from Statscan that the numbers are robust enough that we can do that.”
Until then, the survey’s just a snapshot of the city, he said.
“We feel that it still gives us a good estimate in terms of the diversity of a particular population. It won’t allow us any historical comparison.”
Treasury Board President Tony Clement, who was Industry Minister when the long-form census was scrapped, declined to comment Wednesday. So did current Industry Minister Christian Paradis; his office forwarded questions to Statistics Canada.