TORONTO – The highest-profile test used to rank students from different countries is called the Programme for International Student Assessment (PISA) and was developed by the Organisation for Economic Co-operation and Development (OECD).
PISA tests reading, math and science, and has emphasized one of the three domains each time it has been administered (every three years, starting in 2000).
Statistics Canada randomly selects schools from areas that are willing to participate according to OECD criteria that include provincial representation, geographic representation, smaller and larger schools. Fifteen-year-old students within those schools are then selected. This is referred to as a ‘double random selection.’
All students across the globe hear a standard script from the officials (StatsCan in Canada) giving out the test, and it’s made clear to students that it doesn’t count towards their grade.
The test is designed to provide policy-oriented indicators of student skills that can be applied internationally, and highlight factors that contribute to successful students, schools and education systems.
Andrew Parkin, director of the Council of Ministers of Education Canada (CMEC), says the first publication of the PISA results caused a “flurry of exchange visits” as educators from countries who did less well went to visit countries that scored better.
But what can Canada learn from the PISA tests that would make our system better? Answers are sometimes elusive.
Pierre Brochu is an assessment coordinator at CMEC and co-authored a report analyzing Canadian PISA results. Brochu says PISA is so complex with so many countries participating, you can “mine PISA data and have it say what you want.”
Pasi Sahlberg, director general of the Centre for International Mobility (CIMO) in Finland, believes PISA is a high-quality tool that measures knowledge and skills, but also highlights some limitations.
“PISA doesn’t include any aspects of ‘how students work on problems’, their team skills, or attitudes to particular problems,” Sahlberg says in an email to Globalnews.ca. “PISA also have only a limited access to measure real creative skills of students given the nature of the test situation.”
University of British Columbia professor Charles Ungerleider is the director of research and managing partner at Directions Evidence and Policy Research Group and previously worked on a government team analyzing PISA results. He thinks international comparisons are less useful than studying Canada’s own education system over time.
Ungerleider highlights only two valuable findings from PISA: that countries don’t have to choose between social equity and results, and that after controlling for the socioeconomic background of parents, Canadian students in public schools perform as well as those in private schools.
“PISA gets a lot of profile, which is great. Because it has that international comparison, it lends itself so much to dinner table conversations, newspaper articles, policy research,” says Parkin. “But you just gotta keep in mind that underneath that in Canada are 13 jurisdictions who are running their own assessment programs…that don’t add up to a big international report, but are driving thinking about education all the time.”
Brochu adds that PISA is the only test available in some countries, but is only one piece of the puzzle in Canada.
CMEC points to the upcoming publication of results from the Progress in International Reading Literacy Study (PIRLS), which is a test assessing reading-literacy achievement in grade four students. This will include results from a national Canadian sample for the first time in December.
Comments