California’s AB-1395 Highlights the Challenges of Regulating Voice Recognition

Under the radar of ongoing debates over the California Consumer Privacy Act (CCPA), the California Senate Judiciary Committee will also soon be considering, at a July 9th hearing, an unusual sectoral privacy bill regulating “smart speakers.” AB-1395 would amend California’s existing laws to add new restrictions for “smart speaker devices,” defined as standalone devices “with an integrated virtual assistant connected to a cloud computing storage service that uses hands-free verbal activation.” Physical devices like the Amazon Echo, Google Home, Apple HomePod, and others (e.g. smart TVs or speakers produced by Sonos or JBL that have integrated Alexa or Google Assistant), would be included, although the bill exempts the same cloud-based voice services when they are integrated into cell phones, tablets, or connected vehicles. 

Although AB-1395 seeks to address legitimate consumer privacy concerns, its core provisions likely contain pitfalls. Nonetheless, it raises important questions about the best ways to regulate privacy in the context of “listening” devices.

First, it’s clear that speech-to-text recognition has made incredible strides in the past decade, due in large part to companies being able to train machine learning models on very large datasets of human speech. These models are not perfect–they are continuing to work on heavy accents, unusual speech patterns, and non-English speech–but they have improved dramatically in recent years. Only a few years after the first voice assistants hit the market, speech recognition has now become a common way of interacting with computers, and a game-changer for accessibility.

Notwithstanding these ground-breaking benefits, most people are justifiably wary of devices that seem to “listen,” “spy,” or retain or use data in unexpected ways. FPF explored these concerns in a 2016 White Paper, Always On: Privacy Implications of Microphone-Enabled Devices. We have also explored uses of voice recognition in Smart TVs. Sometimes privacy concerns are based on misunderstandings of how voice-activated technology works–for example, we distinguished in an Infographic on Microphones in Internet of Things (IoT) Devices, between “always on,” “voice-activated,” and “manually activated” devices, which operate and collect data differently. Other concerns are totally valid, for example those raised by consumer privacy advocates regarding data retention defaults, design of user choices, or concerns about possible future uses of data in unexpected ways.

These issues can and should be addressed through comprehensive privacy legislation. FPF supports a non-sectoral, comprehensive federal privacy law, and in its absence has written in support of the California Consumer Privacy Act (CCPA), which creates baseline protections for Californians that apply across sectors and types of technology, including smart speakers. For example, many companies provide options for data deletion, and this will soon be mandated as a consumer right under the CCPA. Enshrining these and other privacy rights into law, if bolstered by ongoing rule-making and effective enforcement, allows the law to set clear limits across sectors and technologies, while remaining flexible enough to adapt to evolving technology in the future. So-called “smart speakers” are a great example of this: five years ago they did not exist. Five years from now, it may already be an antiquated concept, as cloud-based voice recognition transcends the physical boundaries of standalone devices, and becomes increasingly integrated as a core feature of almost all new technology, e.g. connected cars, wearables, and outdoor smart city kiosks.

If California decides to address the narrow slice of “smart speakers,” we recommend that they take a close look at two core aspects of AB-1395 (as revised 06/26/2019) that could cause unintended consequences, or not be as effective at addressing consumer privacy concerns as intended:

• Sharing Data with Third Parties. Section 22948.20(b) appears to prohibit a company from sharing transcript data with third parties, even if a user affirmatively consents and requests such sharing. This might be a drafting error and thus an easy fix, but as currently written it would outlaw many common and beneficial features of smart speakers. Many household smart speakers or “voice assistants” (e.g. Amazon Echo, Google Home, and many others) serve as a “hub” or “portal” for connecting to a user’s other devices or services. For example, a user might use a voice assistant to: turn on or off the lights, adjust the air conditioning, add something to their calendar, order take-out food, or order a taxi or shared ride. All of these examples require sharing identifiable data (an interpretation of the user’s request, e.g. “turn on the lights”). In many circumstances, owners of these devices expect this kind of data sharing to occur at their request, and on their behalf (in other words, with meaningful consent).
• Retention. Section 22948.21(a) requires separate, opt-in consent for retention of voice or transcript data, and that manufacturers provide a “basic” retention-free version to customers who don’t opt in. In the context of voice recognition, access to large amounts of data has driven the rapid advancement of voice recognition in the last decade, and continues to drive product improvement–for example, as discussed above, for learning to recognize heavy accents, speech disorders, or non-English speech. However, consumer advocates are justified in their concerns about indefinite data retention as a “default,” particularly when users have limited ability to delete their data. One way to address this is through consumer deletion rights, which many leading companies provide and are mandated by the California Consumer Privacy Act (CCPA). An even better, more nuanced approach, might be to require or encourage companies to create meaningful, easier-to-use choices, such as automatic recurring deletion options (as Google recently introduced). Another common-sense privacy protection would be to require that it be possible to request data deletion through a voice request. Unfortunately, AB-1395 does not take any of these approaches, but instead creates an “all or nothing” framework for data retention. Most consumers probably want something in between–the ability to get the benefits of voice personalization (for example, if they themselves have a strong accent or unusual speech pattern), and perhaps support product improvement, but with easier, better, or more meaningful deletion options.

We hope consumer privacy will continue to be a core legislative priority in 2019 and 2020, as the United States draws closer to drafting and passing a baseline comprehensive privacy law. States that address these issues in the meantime should do so thoughtfully and with an eye towards effective regulation to address real privacy concerns while supporting the benefits of emerging technologies.