Watch what you say: Apple opens the door to a nightmare world of always-listening tech

Apple this week announced plans to roll out "always-listening" features for newer Apple Watches. One of the features, Live Rewind, lets you "go back in time" and replay the most recent 15 seconds of ambient audio as a text transcript. The other feature, Siri Recap, listens throughout the day and summarizes your conversations as high-level notes into a written report on your iPhone.

Apple claims these features are helpful to "jog" your memory. The massive trade-off is that your Apple Watch is always listening to you and anyone else within earshot while the feature is enabled. 

Here's the relevant section from Apple's live event:

Given these features we're bound to be controversial, it's clear that Apple put thought into the security and privacy guardrails before shipping these features (which may still change). The tech giant says on its support page that the always-listening tech doesn't record or store audio, and that the recaps are end-to-end encrypted so that any summarized data is inaccessible to anyone else, not even Apple. And, its security documentation [PDF] detailing how the features work is worth reading as well.

What is more concerning to me, to underscore on a point I made in a Bluesky post, is that Apple is opening the door to a world where other companies, device makers, and app developers try to recreate or rival Apple's technology and do a far shoddier job, all the while creating a worse surveillance monster and introducing new security and privacy risks along the way.

So far there have been a few companies and startups that made always-listening devices, like pendants, often including AI. Apple by comparison is one of the largest technology companies in the world and its global reach means millions of people will get these features and switch them on. Apple's market dominance, coupled with its decision to make always-listening tech, sets a precedent that signals it's OK for other companies to follow suit and normalize this intrusive tech. It also implies that there is money to be made in developing this technology, making it a space ripe for disruption (or exploitation) by profiteers.

Bloomberg ($) reports lawyers are already finding that Apple's always-listening tech is at odds with the law in several U.S. states. Some states require all parties' consent before being recorded, setting up the feature for potential legal battles down the line. This comes amid the rise of smart camera-enabled "pervert glasses" like those made by Meta and Snap, which have supercharged abuse and harassment against people by recording them without consent.

Privacy for me, but not for thee?

While Apple has emphasized security and privacy features for its own users, it hasn't explained how people opt out of its customers' recordings.

I chatted this week with Em, a digital privacy and security technologist, activist, and author of the Control Alt Delete blog, and whose work you have no doubt seen while she wrote for Privacy Guides. Em told me that these new Apple features follow a "recent trend of gadgets that are constantly collecting information from surroundings, including from people who aren't wearing or buying these gadgets themselves," like smart glasses and speakers.

Em said that while anyone can make notes of every conversation they have, this kind of technology "normalizes it happening at all time and in every context."

"Even if Apple seems to have put some thought into protecting the privacy and consent of the people using the Apple Watch, this technology doesn't do much to protect the privacy and consent of everyone interacting with people using it," Em told me.

I also asked Thorin Klosowski, a senior security and privacy activist at Electronic Frontier Foundation, for his thoughts and he said similarly that Apple's approach is largely about protecting its customers' data but does little for anyone else.

"Increasingly we’re seeing tech companies frame privacy this way — as personal responsibility — but it has never been just about that, it’s a collective effort that relies as much on personal choice as it does social contracts," said Klosowski. He added that we should all be free to converse without fear of tech documenting everything that's being said. "Especially when combined with all the other similar devices we’re seeing, whether that’s pendants or glasses or rings, there is absolutely a risk of normalization here."

EFF's privacy litigation director Adam Schwartz added that since there's no practical means for people to consent or decline recording, "we suggest people should think twice before using this technology, out of respect to the conversational privacy of others."

For its part, Apple says on its support page that users should "consider those around you where conversations may be private or sensitive." The broader message Apple seems to be sending is: Prepare for a world where everyday devices are always listening in.

Thank you so much for reading and subscribing to ~this week in security~! I hope you enjoyed and found this article helpful. If you like it, please share a link on your social media! Please reach out with any feedback, questions, or comments about this article: this@weekinsecurity.com.