Facebook
Twitter
Pocket
Reddit
Email
The voice is the music of the soul. Since the time of Plato and Aristotle, it’s been linked to a person’s character and inner life. And it often betrays things we try to hide: fatigue, fear, attraction, scorn.
But the sound of your voice may reveal even more than you realize—for instance, details of your physical and mental health, your cultural and education backgrounds, even your political preferences.
Scientists are increasingly learning how to use computers to track these vocal inflections and fingerprints, but some scholars are sounding the alarm. Today, speech-recording devices are everywhere, in smart speakers and phones like Siri and Alexa and in the voice control systems in TVs, cars, and appliances. Some of these systems have “always listening” protocols that monitor audio continuously in the environment for trigger phases to activate.
Given the intimate personal information contained in your voice, these technologies present major privacy risks. The recordings of your voice that they collect could one day be used against you, warns Tom Bäckström, professor of speech and language technology at Aalto University in Finland: to increase your insurance premiums; tailor ads to your interests, needs, or emotional states; screen you for certain jobs; and worst of all, harass, stalk, or extort you.
“When someone talks, a lot of information about their health, cultural background, education level, and so on is embedded in the speech signal. That information gets transmitted with the speech, even though people don’t realize it,” Bäckström explained in a statement.
Read more: “What Makes an Opera Star Stand Out?”
To deter abuse before it happens, Bäckström and his colleagues are working to develop protective measures and technologies. They recently published a paper in the Proceedings of IEEE that reviews the potential risks posed by speech-recording technologies, and the measures that could be taken to protect against those risks.
Some of the risks include voice assistants accidentally recording private conversations, companies using voice data for profiling or surveillance, stalkers or abusers using voice data to track people, and shared devices leaking one person’s private information later on.
The scientists built a metric that they say can be used to tell how much information about a person a particular voice clip reveals, based on the pitch of the voice or other features, such as the actual words it contains. The new metric is the first, they claim, that can capture how much is revealed in any one audio recording of a voice. Bäckström says the technology could be used specifically to help people understand how much of their privacy is at risk when they use certain speech-recording technologies.
The team of scientists is also working to build technologies that can compress, distort, anonymize, or re-synthesize speech, so that only the essential information gets transmitted when a person uses a speech-recording technology. And they’re developing “adversarial training” protocols that would train systems to complete an intended task without tracking private personal information contained in a voice.
“To ensure privacy, you decide that only a certain amount of information is allowed to leak, and then you build a tool which guarantees that,” he explained. “But with speech, we don’t really know how much information there is. It’s really hard to build tools when you don’t know what you’re protecting, so the first thing is to measure that information.”
Another privacy-saving approach would be to disentangle the components of speech when companies record it, the authors write—separating words from tone, identity, and emotional content, though this is one of the more challenging projects. Encryption is yet another tool that might help protect voice privacy, though it’s slow and expensive. Then there’s sound engineering, which could be used to alter a vocal recording in such a way that only the intended listener could hear it clearly. One thing individuals can do to protect their privacy in advance is to avoid sending any voice data to the cloud, the scientists suggest, though this can be tricky with shared services.
The speech technology privacy field is in its infancy, and many open questions for study remain: how to handle consent in real-time voice interactions, how privacy degrades over longer recordings, how to protect against future attacks, how to deal with multi-person conversations, how to design systems that people trust. Perceived privacy is in some ways as important as actual privacy, the study authors argue, given the strong psychological ownership people tend to feel about their voices. People will abandon systems they don’t feel they can trust, they point out.
Ultimately, much of the burden will lie with the companies that make speech-activated technologies. The ethical thing to do is to build systems that minimize what they hear, record, remember, and infer.
Because the machines are learning to listen—in ways that a human never could. 
Enjoying Nautilus? Subscribe to our free newsletter.
Lead image: Anton Vierietin / Shutterstock
