Amazon, Google may want to be a fly on the wall in your home, patents suggest

Click to play video: 'Alexa, Google Home, Echo and more: What to know about smart speakers' Alexa, Google Home, Echo and more: What to know about smart speakers
It's expected 75 per cent of US homes will have "smart speakers" by 2020, but what exactly do they do, are they safe and are they worth it? Online security expert David Papp explains – Dec 18, 2017

Amazon and Google are in race to occupy and digitize the space in our homes, but what will they have access to once they get there?

Patent filings by both companies — most of which are still under consideration — indicate that both tech giants want their smart home devices to listen more astutely to what happens in the home.

READ MORE: The internet is convinced our phones are listening to us. Here’s what the experts say

These patents were first reported by the New York Times last weekend, and were first discovered by the consumer advocacy group Consumer Watchdog in December, 2017.

Amazon and Google are the world’s two leading sellers of smart home devices, and both have claimed that their products only listen to and process audio once users activate them, either by pushing a button or by using the phrases, “Hey, Alexa,” or, “Okay Google.”

Story continues below advertisement

Patents propose monitoring “mischief” and “emotional state”

One set of patent applications from Google, involving its subsidiary company, Nest Labs, suggests that audio monitoring could help detect that a child is engaging in “mischief.”

The application states that a device could analyze speech patterns and pitch to identify that a child is present, sense movement while listening for whispers or silence, and give parents the option to program a smart speaker to “provide a verbal warning.”

WATCH: Keeping your smart home devices safe and secure

Click to play video: 'Keeping your smart home devices safe and secure' Keeping your smart home devices safe and secure
Keeping your smart home devices safe and secure – Jan 25, 2018

The filing gives the example that when children are near a liquor cabinet or are in their parents’ bedroom, the system may “infer that mischief is likely to be occurring…the system may report and/or record the findings for subsequent use,” and could then give the child a warning as a form of “deterrence.”

Story continues below advertisement

The patents also outline the possibility for monitoring the “emotional state” of the occupants of a home. For example, “crying may signify a sad emotional state, whereas laughing may signify a happy emotional state.” The document also discusses monitoring body temperature, audio signatures, and facial expressions.

WATCH: Tech: Smart home monitoring

Click to play video: 'Tech: Smart home monitoring' Tech: Smart home monitoring
Tech: Smart home monitoring – Feb 22, 2017

A second set of application documents from Google indicate that “home data” can be used to provide home occupants “interesting information, products and services as well as providing them with targeted advertisements.”

In a set of patent applications filed by Amazon, the company describes a mechanism called a “voice sniffer algorithm,” that could be used to analyze real-time audio and potentially send targeted product advertisements.

WATCH: Tech gadgets for building a ‘smart home’

Click to play video: 'Tech gadgets for building a ‘smart home’' Tech gadgets for building a ‘smart home’
Tech gadgets for building a ‘smart home’ – Feb 6, 2017

This feature could be used on a wide range of devices — from tablets to e-readers. A diagram included with the patent application detailed how a phone call between friends could lead to one party receiving an offer for the San Diego Zoo, and the other seeing an ad for a Wine of the Month membership depending on the words — “love,” “great,” etc. — that each participant used to describe these experiences.

Story continues below advertisement

“Orwellian” implications?

Each of the patents described here was filed in 2015, though they were first discovered by the Consumer Watchdog near the end of 2017. In response, the group published a study on the”Orwellian” implications of functions like the ones described in those documents.

“These patent applications show how technology companies use home data to draw disturbing inferences about households, and how the companies might use that data for financial gain,” the study states.

READ MORE: Another problem with the Internet of Things: smart home devices don’t work when networks go down

“This isn’t about helping people, it’s about selling people,” said John M. Simpson, Consumer Watchdog’s privacy and technology project director in a statement. “If these patents are implemented, there will be unparalleled surveillance of our private lives.”

In a statement to the Times, Amazon said the company took “privacy seriously” and plainly denied using customers voices for targeted advertising. The company said it had filed “a number of forward-looking patent applications that explore the full possibilities of new technology.”

READ MORE: Canadians at higher risk of hacks, thanks to their smart devices: report

Amazon added that these patents take multiple years to receive and did not reflect the developments of current products and services.

Story continues below advertisement

Google offered the Times a similar response, saying it “did not use raw video to extrapolate moods, medical conditions or demographic information.” The company went on to call Consumer Watchdog’s claims “unfounded” and mirrored Amazon in arguing, “prospective product announcements should not necessarily be inferred from our patent applications.”

Privacy vs. progress

These patents are surfacing just as privacy and surveillance have been thrown into the spotlight. Facebook is currently dealing with backlash from the Cambridge Analytica scandal, followed by the revelation that the company may have kept logs of calls and text messages of users with Android devices running on older software.

Following the news that information from over 50 million Facebook profiles was provided to political consulting firm Cambridge Analytica, who went on to work with several political campaigns in the following years, user anger has torpedoed the company’s stock and shaved over $100 billion off its overall value.

Story continues below advertisement

Furthermore, internet forums are convinced that our devices are already listening to us, and experts are starkly divided on whether this is true. While it is possible to delete voice recordings from Amazon Echo devices, there’s no way to do that on a Google Home speaker just yet.

To stop your Google Home to listening to you, tap the “Mute” button on the back of the device, which prevents Google Assistant from hearing anything at all. To give the Google Assistant verbal commands, you’ll need to un-mute the speaker.

Amazon’s Echo speaker doesn’t let you stop voice recordings altogether, but also it also features a “Mute” button similar to the Google Home. In addition, however, users can delete voice recordings from the Amazon Alexa app in bulk by visiting, visiting the “Your Devices” tab, and selecting “Manage voice recordings.”

Sponsored content