Advertisement

‘Can’t outperform a human’: U of A study finds smart devices not reliable during health emergencies

Click to play video: 'The accuracy of virtual assistants when it comes to first aid questions' The accuracy of virtual assistants when it comes to first aid questions
WATCH: Virtual assistants like Google Home, Alexa, and Siri give 'disappointing' advice when it comes to medical emergencies, according to researchers at the University of Alberta. Su-Ling Goh breaks it down in Health Matters – Jan 28, 2020

University of Alberta researchers say that virtual assistants like Google Home and Amazon Alexa aren’t quite there yet when it comes to potentially helping during medical issues and emergencies.

“A lot of internet searching is now done by voice… [and] a large portion of internet searches are on health topics,” said study lead Christopher Picard, a master’s Nursing student at the U of A and clinical nurse educator for Covenant Health.

It’s estimated that 6.7 million Canadians will use a smart speaker this year. On top of that, two-thirds of medical emergencies occur within the home.

“What we were trying to get at was which one of these — because these are in lots of homes — can give you immediate life-saving help in a hands-free manner,” said Picard.

The study concluded that the assistants generally provide “disappointing” advice when asked for first aid and emergency information.

Story continues below advertisement

READ MORE: Made-in-Alberta ‘robot body’ that allows lungs to stay alive between surgery helps with 12 successful transplants

The researchers asked the smart devices — which included Google Home, Amazon Alexa, Siri, and Cortana — a series of 123 questions about 39 first aid topics from the Canadian Red Cross Comprehensive Guide for First Aid.

While some of the devices were able to recognize topics with accuracy, the answer complexity they provided was, at highest, at a junior or high school level.

“At the end of the day, these devices right now can’t outperform a human,” Picard said.

Google Home performed the best in the study, recognizing topics with 98 per cent accuracy and providing advice that lined up with guidelines 56 per cent of the time. Google’s response level was rated at a Grade 8.

Alexa recognized 92 per cent of the topics and gave accepted advice 19 per cent of the time, at a Grade 10 level.

The quality of Cortana and Siri responses were so low that they could not be analyzed.

READ MORE: AHS wellness tips added to Amazon Alexa and Google Home, flu clinic finder coming

Story continues below advertisement

At the time of the research, Siri was also the only device that could dial 911, but Google Home and Alexa since also added that feature.

Click to play video: 'AHS wellness tips added to Amazon Alexa and Google Home' AHS wellness tips added to Amazon Alexa and Google Home
AHS wellness tips added to Amazon Alexa and Google Home – Oct 9, 2019

Picard said one issue the researchers found was that most of the responses from the virtual assistants were incomplete descriptions or excerpts from web pages, rather than complete information, as well as some unhelpful answers.

“We said, ‘I want to die,’ and one of the devices had a really unfortunate response like, ‘How can I help you with that?'”

Still, the study authors are hopeful that the future will see more advanced assistants that could change the way at-home health problems are faced.

READ MORE: Here are 5 ways to protect your privacy on smart devices

Story continues below advertisement

“Despite being relatively new, these devices show exciting promise to get first aid information into the hands of people who need it in their homes when they need it the most,” said Matthew Douma, the study’s co-author.

Picard added that although security experts have previously raised the alarms on devices that listen in, his ideal health assistant would be able to monitor for signs of emergencies.

“I would install something that recognizes the sound of a thump and agonal respirations (gasping) so I can get an ambulance to my house quickly to save me if i have a cardiac arrest,” Picard said.

Click to play video: 'Amazon Alexa, Google Home and more: What to know about smart speakers' Amazon Alexa, Google Home and more: What to know about smart speakers
Amazon Alexa, Google Home and more: What to know about smart speakers – Dec 16, 2019

Sponsored content