FPF Urges Federal Trade Commission to Craft Practical Privacy Rules

FPF Comments Regarding FTC ANPR Urge the Commission to Provide Individuals with  Strong, Enforceable Rights and Companies with Greater Clarity about their Obligations under Section 5 of the FTC Act.

The Future of Privacy Forum filed comments regarding the Federal Trade Commission’s Advance Notice of Proposed Rulemaking, recommending that the Commission prioritize practical rules that clearly define individuals’ rights and companies’ responsibilities. 

The Commission has spent decades enforcing prohibitions against unfair and deceptive data practices regarding a wide range of established and emerging technologies. Those privacy and security enforcement actions have been based on the FTC’s statutory authority, which provides flexibility to address consumer harms arising from novel technologies and business practices, but which does not articulate granular rights for consumers or requirements for businesses. Clear, practical rules can more specifically define what data practices the Commission considers unfair or deceptive. The current FTC rulemaking is an opportunity to provide individuals with strong, enforceable rights and companies with greater clarity about their obligations under Section 5 of the FTC Act.

FPF’s comments urge the Commission to:

• Codify its “common law” privacy and security norms. FTC enforcement actions are often viewed by practitioners as precedent or guidance. But settlements and consent decrees do not provide explicit, comprehensive rules that companies must follow and upon which consumers can rely. The Commission should codify key aspects of its deception and unfairness settlements while also incorporating lessons from FTC staff reports, workshops, privacy laws, self-regulatory regimes, and commercial best practices. Specifically, the FTC should:
  • require businesses to provide material, clear, and prominently accessible data use policies;
  • require businesses to implement reasonable security measures;
  • require businesses to comply with the representations they make about privacy and security, including self-regulatory commitments; 
  • prohibit companies from circumventing individuals’ clearly expressed privacy preferences without clear, explicit, superseding consent from the individual; and
  • articulate the circumstances in which the FTC considers discriminatory algorithmic decision-making to be an unfair trade practice, the factors the Commission considers when weighing that determination, and the degree to which the Commission’s analysis relates to other anti-discrimination regimes.
• Go beyond its common law privacy and security norms to mitigate important privacy risks and establish increased clarity regarding companies’ responsibilities. When crafting these sorts of rules, the FTC should be guided by three principles:
  • data exists on a spectrum of identifiability, rather than in binary categories of “personal information” or “not personal information,” and privacy enhancing technologies can reduce the identifiability of data and otherwise mitigate risks;
  • standards for evaluating the fairness of “secondary uses” of data are needed to define the boundaries of what secondary uses are compatible, based on a careful evaluation of context, expectations, harms, and benefits of processing, including competition; 
  • It is especially important to consider the harms that sensitive data use can create, the manner in which those harms impact marginalized communities, and the heightened protections that may be appropriate to mitigate those harms. At the same time, sensitive data is essential to a wide range of activities, including detecting and addressing disparate outcomes.

As a practical matter, the FTC acts as the primary U.S. privacy enforcement agency. Although FPF views a new, pragmatic, comprehensive federal privacy law as the ideal mechanism for grappling with complex technologies and data flows, clear and practical FTC rules defining unfair and deceptive practices would benefit individuals and businesses.

Understanding Extended Reality Technology & Data Flows: Privacy and Data Protection Risks and Mitigation Strategies

This post is the second in a two-part series. Click here for FPF’s XR infographic. The first post in this series focuses on the key functions that XR devices may feature, and analyzes the kinds of sensors, data types, data processing, and transfers to other parties that power these functions. 

I. Introduction

Today’s virtual (VR), mixed (MR), and augmented (AR) reality environments, collectively known as extended reality (XR), are powered by the interplay of multiple sensors, large volumes and varieties of data, and various algorithms and automated systems, such as machine learning (ML). These complex relationships enable functions like gesture-based controls and eye tracking, without which XR experiences would be less immersive or unable to function at all. However, these technologies often depend on sensitive personal information, and the collection, processing, and transfer of this data to other parties may pose privacy and data protection risks to both users and bystanders. 

This post examines the XR data flows that are featured in FPF’s infographic, and analyzes some of the data protection, privacy, and equity issues raised by the data that is processed by these devices, as well as strategies for mitigating these risks.

Key risks include:

• Sensitive inferences: XR devices collect, process, and share large quantities of data about users’ bodies and environments. This data could be used to make inferences—whether accurate or not—about sensitive aspects of peoples’ lives, such as their sexual orientation or health conditions.
• Digital fingerprinting: Tracking users’ and bystanders’ bodies could allow for digital fingerprinting and the loss of anonymity in XR environments.
• Bystander data collection: Non-users in proximity to XR technology may be unaware that it is collecting and processing data about them, as well as for what purposes and with whom the technology is sharing this information.

Key mitigation strategies include:

• On-device processing and storage: Processing and storing data on a user’s device, as opposed to remotely on a processor’s server, to ensure that the data remains in the user’s hands and not accessible to others.

• Purpose limitation and data minimization: Limiting data collection, storage, and/or usage, including third-party use, to particular, specified purposes, and requiring data controllers to provide notice to or obtain consent from users if they plan to use this data for a different purpose. 
• Privacy-enhancing technologies (PETs): Certain technological innovations can also be useful tools for managing privacy risks. For example, advances in encryption and differential privacy can allow for privacy-preserving data analysis and sharing, and the use of synthetic data sets can alleviate concerns about data sharing or secondary data use.
• Bystander protections: Designing XR devices so that they ensure bystanders’ data is not unduly collected. This could include automatically blurring bystanders’ faces, or using a system of lights on a head-mounted display to signal to non-users the device is on and potentially collecting data.

II. Processing Large Volumes and Varieties of Sensitive Personal Data

XR technologies raise traditional privacy and data protection risks, but also implicates larger questions around surveillance, social engineering, and freedom of expression. As noted in the first blog post in this series, XR technologies require large volumes and varieties of data about the user’s body and their environment. Certain collection and use limitations may therefore be challenging or impossible to implement, since some of XR’s core functions require extensive data collection and processing. Now and in the future, XR technologies may also transfer data to other users and third parties, such as software companies, hardware manufacturers, and advertisers. While devices generally process raw sensor data on device, they may transmit raw or processed sensor data to an application and other parties for further processing to improve representations of virtual content or enable shared experiences. While these transmissions of data may improve a user’s XR experiences, they can also create new privacy and data protection risks for users and bystanders.

Eye tracking underpins many current and future-facing use cases, such as enhanced graphics, expressive avatars, and personalized content, but it may pose privacy and data protection risks to users. This is due to eye tracking data’s sensitive nature, its potential role in significant decisions affecting users, and the unconscious nature of the behaviors from which some of this data is derived. Organizations could use data related to pupil dilation and gaze to potentially infer information—whether accurate or not—about the user, such as their sexual orientation, age, gender, race, and more. Organizations may also use this data to attempt to diagnose medical conditions, such as ADHD, autism, and schizophrenia. Despite the sensitive nature of this data, users often lack the capacity to meaningfully control its collection or use. Without proper controls, this information may be further shared with third parties. This raises the likelihood of organizations using this data to inform major decisions about a user, which could have real-world impacts on XR users.

Sensors that track a user’s bodily motions may also cause harm due to their potential to undermine anonymity. The first post in this blog series analyzed how tracking a user’s position can enable functions like mapping the user’s space to help place virtual content. But this tracking could also be a means to digitally fingerprint users and individuals, including bystanders, especially given the volume and variety of data that XR devices gather and process. At the same time, this tracking data raises the same de-identification and anonymization concerns that exist regarding similarly granular non-XR data types, such as behavioral biometrics, historical geolocation, and genetic information. Digital fingerprinting may therefore undermine individuals’ ability to maintain anonymity in XR environments. This may discourage users from fully expressing themselves or participating in certain activities due to worries about retaliation.

III. Statutory Obligations 

It is unclear how well current legal protections mitigate the privacy risks posed by certain processing activities in the XR context. Whether or not bodily information like gaze and gait are covered by existing biometric regulations may depend on these laws’ definitions of biometric data. For example, under the EU’s comprehensive privacy law, the General Data Protection Regulation (GDPR), this type of data qualifies as “personal data” if it relates to an identified or identifiable person, such as a user or bystander. Thus, an organization that records, collects, assesses or uses this data in any other manner would be subject to GDPR obligations such as transparency, fairness, data minimization or storage limitation. 

Pursuant to the GDPR, “biometric data” includes personal data resulting from the specific technical processing of a person’s physical, psychological, or behavioral characteristics, and which allows for identification. Organizations are subject to heightened obligations under the Regulation depending on the purpose for which they process biometric data. Specifically, the GDPR prohibits organizations from processing such personal data, unless one of the permissible grounds strictly defined by the law applies. The Regulation defines biometric data to include only that which an organization could use for identification purposes. As described in FPF’s prior blog post, however, an organization may process eye and other bodily information for non-identification purposes, such as to debug applications or improve products . This raises questions as to whether the GDPR’s protections for sensitive data categories would always apply to these XR functions. Notably, even if this eye and other bodily information does not meet the “sensitive data” criteria, the rest of the Regulation would still apply to this data. Furthermore, European ePrivacy rules may apply to a user’s system that connects to or pairs with XR equipment.

Similar lack of certainty exists in U.S. law. For example, the Illinois Biometric Information Privacy Act (BIPA) applies to information based on “scans” of hand or face geometry, retinas or irises, and voiceprints. This definition of “biometric identifiers” does not explicitly cover the collection of behavioral characteristics or eye tracking. Whereas the GDPR may still apply to an organization that processes eye and other bodily information if it is personal data or qualifies as other sensitive data categories, BIPA may not apply at all. This highlights how existing laws’ protections for biometric data may not extend to every situation involving XR technologies. However, protections may apply to other special categories of data, given XR data’s potential to draw sensitive inferences about individuals.

IV. Bystander and Environmental Data

Bystanders’ privacy can also be impacted when XR devices and third parties collect and process sensor data. Some of the privacy and data protection issues affecting bystanders mirror the  privacy risks to XR users. However, unique notice challenges arise with respect to bystanders. Non-users in proximity to an XR user may be unaware that the device is collecting and processing data about them, as well as for what purposes and with whom the device is sharing this information. Like users, bystanders also cannot control the unconscious behaviors that provide the sensor data inputs for XR experiences. Even if a bystander generally understands that a device is collecting data about them, the unconscious nature of some behaviors means that bystanders may neither be aware of the behaviors nor specifically understand that a device is processing data about these behaviors. 

Bystander data could facilitate both use cases that are detrimental to a non-user’s privacy and decisions that negatively affect them. Future XR technologies will likely incorporate facial characterization or analysis technologies that can allegedly sense cognitive states or infer emotions—whether accurate or not—based on sensor data. Insights from these technologies could help organizations construct a portrait of the locations a non-user frequents, their interests, and medical conditions. 

IV. Strategies for Mitigating Risks

Organizations that provide XR technologies can implement a number of strategies to address the risks raised by XR data collection, use, and sharing. While no single intervention by itself mitigates all of these risks, some combination of strategies is likely to decrease risks and help minimize harms that may result. For instance, processing and storing data on a user’s device, as opposed to remotely on a processor’s server, helps ensure that the data remains in the user’s hands and not accessible to others. Organizations can also work to limit data collection, storage, and/or usage, including third-party use, to particular, specified purposes, and provide notice to or obtain consent from users if they plan to use this data for a different purpose. Companies should set policies and guidelines for third-party developers’ data practices, and monitor to ensure compliance with said policies.

Certain privacy-enhancing technologies (PETs) are useful tools for managing privacy risks. For example, advances in encryption and differential privacy can enable privacy-preserving data analysis and sharing, and the use of synthetic data sets can address concerns about data sharing or secondary data use. Another option is to provide greater user controls, allowing users to control the kinds of data collected about them—particularly sensitive data like eye tracking and facial expressions data—and with whom this data is shared. 

Some organizations have chosen to design XR devices so that they ensure bystanders’ data is not unduly collected, for instance by automatically blurring bystanders’ faces, or using a system of lights on a head-mounted display to signal to non-users the device is on and potentially collecting data.

Organizations using XR should be transparent about how they use and plan to use XR data, and publicly commit to guidelines and/or ethical principles. This could also include something akin to an institutional review board (IRB) to ensure compliance with these principles. Finally, organizations can build privacy into an organization’s culture and processes and create bodies like oversight boards to ensure privacy protections endure beyond other changes in mission and values.

V. Conclusion

The complex web of data, sensors, algorithms and automated systems, and parties that enable important and sometimes central XR functions also can raise privacy and data protection concerns. Devices and ML models may collect and process large volumes and varieties of sensitive personal data, over which users and bystanders may lack meaningful controls, and that other parties could use to make important decisions affecting these individuals. The disclosure of this data may also undermine user anonymity, which could discourage users from freely expressing themselves due to fears of retaliation. Providing bystanders with notice that communicates that a device is collecting data about them, let alone for what purpose and to whom the data is transmitted, is challenging and may not be possible. This creates difficulties related to obtaining affirmative express consent to data processing activities in XR, where consent is predicated on the individual being informed. There is also uncertainty about how existing laws interact with XR technologies, such as how body-based data fits within existing legal definitions of biometrics. The risks to users and bystanders outlined in this post underscore the importance—and, sometimes, challenge—of ensuring appropriate safeguards exist at the technical, policy, and legal level to mitigate against harms that may arise in this space.

Brussels Privacy Convening Focuses on Empowering Vulnerable and Marginalized People, Launches New Project

The Future of Privacy Forum (FPF), a global non-profit focused on data protection and privacy, and the Brussels Privacy Hub of Vrije Universiteit Brussel (VUB) will jointly present the sixth edition of the Brussels Privacy Symposium on November 15, 2022. The in-person event will convene in Brussels, bringing together policymakers, academic researchers, civil society, and industry representatives to discuss privacy research and scholarship. 

In line with this year’s topic, “Vulnerable People, Marginalization, and Data Protection,” participants will explore the extent to which data protection and privacy law — including GDPR and other modern data protection laws like Brazil’s LGPD — safeguard and empower vulnerable and marginalized people. They will also debate how to balance the right to privacy with the need to process sensitive personal information to uncover and prevent bias and marginalization. Stakeholders will discuss whether prohibiting the processing of personal data related to vulnerable people serves as a protection mechanism. 

The event marks the launch of “VULNERA,” the International Observatory on Vulnerable People in Data Protection, led by the Brussels Privacy Hub and supported by the Future of Privacy Forum. The observatory aims to promote a mature debate on the multifaceted connotations surrounding the notions of human “vulnerability” and “marginalization” existing in the data protection and privacy domains.

“I’m excited to begin the groundbreaking and much-needed work we have ahead of us,” Gabriela Zanfir-Fortuna, FPF’s Vice President for Global Privacy, said. Zanfir-Fortuna is also a member of VULNERA’s executive team as a Scientific Coordinator, which is joined by more than 30 members of a broad scientific network. “This initiative will focus on understanding how data protection and privacy law puts safeguards in place to protect the rights of vulnerable and marginalized people in societies increasingly underpinned by digital data flows.”

Professor Gianclaudio Malgieri, Co-Director of Brussels Privacy Hub of Vrije Universiteit Brussel added: “The VULNERA International Observatory will explore theories of vulnerability, marginalization, and intersectionality, examining how data protection law and policy apply to people in certain contexts that may be vulnerable or marginalized, such as women, children, people on a low or zero income, racialized communities, and people of color, ethnic and religious groups, migrants, LGBTQIA+ and non-binary people, the elderly, and persons with disabilities,” 

Representatives from the European Network Against Racism, Dutch Human Rights Council, European Commission, Irish DPC, European Digital Rights (EDRi), European Data Protection Supervisor, and other relevant organizations will share their expertise during the Brussels Privacy Symposium. 

“As we think about the next iteration of the digital age, it’s important that we have a more global consensus on how to protect those who have been historically marginalized,” said Rob van Eijk, FPF’s Managing Director for Europe. “The timing for the launch of VULNERA, and this symposium at-large, could not have been at a more critical juncture.” 

For more information about the event, the agenda, and speakers, visit the FPF site. To learn more about the VULNERA, visit the Brussels Privacy Hub site. 

Event Report: FPF Side Event and Workshop on Privacy Enhancing Technologies (PETs) at the 2022 Global Privacy Assembly (GPA)

The 2022 Global Privacy Assembly (GPA) – which brings together most global data protection authorities (DPAs) every year since 1979, to share knowledge and establish common priorities among regulators – took place between October 25 and 28, in Istanbul (Türkiye). The Future of Privacy Forum (FPF) was invited by the organizers of the GPA (the Turkish DPA) to host a two-part side event during the GPA’s Open Session (on October 25 and 26), in addition to a capacity building workshop for regulators during the Closed Session (on October 28).

These sessions covered the topic of Privacy Enhancing Technologies from three different approaches:

• The regulators’ take: PETs are promising, but no silver bullet. The first part of the FPF Side Event offered the regulators’ perspective, and was titled ‘Regulatory Views on the Role and Effectiveness of PETs’. The session was moderated by Limor Schmerling Magazanik, Director of the FPF-affiliated Israel Tech Policy Institute (ITPI), and counted on the contributions of Rebecca Kelly Slaughter (Commissioner at the US Federal Trade Commission, or ‘FTC’), Tobias Judin (Head of the International Department of the Norwegian DPA, the ‘Datatilsynet’), Gilad Semama (Privacy Commissioner of Israel), and Vitelio Ruiz Bernal (Director of Supervision at the Mexican DPA, the ‘INAI’).
• The view of practitioners: a call for regulatory clarity and predictability. The second part of the Side Event was entitled ‘Lessons Learned from Implementing PETs’, and saw various privacy leaders from the industry share their experiences of leveraging PETs in their compliance efforts. The panel was moderated by FPF’s CEO Jules Polonetsky and had as panelists Anna Zeiter (Chief Privacy Officer at eBay), Emerald De Leeuw-Goggin (Global Head of Privacy at Logitech), Barbara Cosgrove (CPO at Workday), and Geff Brown (Associate General Counsel at Microsoft).
• FPF’s capacity building workshop. The FPF workshop during the GPA’s Closed Session was conducted by FPF’s Vice President for Global Privacy, Dr. Gabriela Zanfir-Fortuna, and Managing Director for Europe, Dr. Rob van Eijk. The session covered the legal qualification of PETs under the EU’s data protection framework – as well as how they could be leveraged to attain compliance with it -, as well as a primer on three PETs, i.e., Differential Privacy, Synthetic Data, and Homomorphic Encryption. This workshop was a condensed version of the Masterclass that FPF hosted at the 2022 Computers, Privacy and Data Protection (CPDP) Conference last May (recorded).

Below we summarize the discussions in the two FPF Side Events with regulators and privacy leaders and highlight key takeaways.

The regulators’ take: PETs are promising, but no silver bullet

Moderator Limor Schmerling Magazanik opened the first discussion by observing that regulators have a dual role regarding PETs: issuing guidance to clarify when and how PETs should be deployed in different scenarios to ensure compliance with privacy laws; and providing tailored advice to lawmakers that wish to promote the use of PETs for the pursuit of public interest tasks and the responsible use of data. 

On this note, Gilad Semama noted that PETs seem to present solutions for combining innovation in the tech sector with the protection of privacy as a Constitutional right in Israel. Semama highlighted that companies have expressed their need for certainty on how they can use PETs to achieve compliance with the privacy framework. The speaker added that it is challenging to find a one-size-fits-all solution in this respect, but that the Privacy Commissioner is trying to pass flexible guidance and answer the public’s queries on PETs for the benefit of businesses and DPOs, by referring to accountability and helping them choose the most appropriate PET for specific use cases. According to Semama, PETs should be complemented with other data security solutions to provide meaningful protections. On the other hand, he noted that companies that are developing PETs in Israel need access to funding and that a recent joint project from the regulator and the Innovation Authority of Israel may be of help. 

Next up, Rebecca Kelly Slaughter stressed the potential that PETs might offer in promoting competition and consumer protection, as they can represent innovation and a positive metric of competition. However, some applications of PETs can be misleading and competition-inhibiting. This means that, according to Slaughter, the value of PETs should be assessed against their concrete effects. The Commissioner stated that the FTC should mainly focus on providing guidance to assist businesses developing and implementing PETs through FTC rulemaking, instead of strict enforcement. However, the FTC will not approve broad safe harbor provisions for the use of specific PETs, as their effectiveness is generally context-specific.

Tobias Judin followed up on Slaughter’s comments, by saying that, when it comes to greenlighting PETs, DPAs should explain that companies do not need to choose between data collection and privacy, or between innovation and data protection. Judin used health research as an example, outlining that often researchers need to collect data about rare diseases across jurisdictions to make the dataset more representative, even knowing that the level of data protection is not equivalent in all targeted countries. In that context, PETs such as homomorphic encryption or differential privacy may provide reassurance to research subjects. Judin also stressed that confidential computing can mitigate security vulnerabilities that often exist when research data is stored on premises and not in the cloud.

Lastly, Vitelio Ruiz Bernal stressed the importance of helping businesses achieve security standards that can help them comply with data protection law. In this respect, he mentioned the INAI’s data protection laboratory, which is dedicated to analyzing apps and web-based applications that are subject to a black-box. The INAI has found that processors which assist controllers in those contexts are often under-resourced and reluctant to use PETs due to their perceived high costs. Bernal revealed that the INAI is currently looking for public-private collaborations to develop accessible PETs and to issue guidelines on specific PETs (e.g., encryption), also inspired by the work of the Berlin Group on the matter. Given Mexico’s specific legal requirements in terms of cloud service security, Bernal mentioned that PETs could potentiate the uptake of cloud services by increasing trust among stakeholders.

The view of practitioners: a call for regulatory clarity and predictability

To frame the second panel of the Side Event, Jules Polonetsky reflected on the privacy community’s eagerness to learn about how industry privacy leaders are integrating PETs into their compliance strategies, their successful and less successful stories. On the other hand, Jules queried the panelists about the actions they would like to see from regulators and policymakers in this space to promote the uptake of PETs. 

Anna Zeiter revealed that eBay has had meetings with its lead DPA in Germany about how PETs could help them comply with the Court of Justice of the European Union (CJEU)’s Schrems II ruling on international data transfers, in particular on the implementation of supplemental measures in accordance with the European Data Protection Board (EDPB)’s guidance. In that context, the DPA focused on measures such as tokenization and encryption (in transit and at rest).

Emerald De Leeuw-Goggin mentioned Logitech’s offerings of PETs as a service for its internal teams of software developers. According to the speaker, this involved making PETs more accessible and scalable within the wider decentralised organisation, the development of privacy engineering capabilities, and the buy-in of Chief Technology Officers. De Leeuw-Goggin noted that PETs are still not mainstream enough for an SME owner to feel confident investing and implementing them, also due to the existing skills gap in the field. As PETs become mainstream, they will also become more understandable and usable across sectors and company sizes.

Barbara Cosgrove stated that B2B companies like Workday tend to receive questions from their customers on how to best implement PETs into their software solutions. This includes masking or pseudonymizing data, or limiting employee access to data. Sometimes, more sophisticated measures – like differential privacy – could be adequate, but companies are reluctant in investing resources in the absence of regulatory clarity, particularly on de-identification. Cosgrove agreed that businesses and regulators need to put their brains together in developing use cases and standards that would increase legal certainty around the effective use of PETs. Co-regulatory solutions like Codes of Conduct could facilitate demonstrations that PETs are used in a compliant manner. 

Finally, Geff Brown highlighted how differential privacy has become usable in multiple apps, allowing providers to process aggregated telemetry data at scale for analytics. Microsoft is using the technique to improve its Natural Language Processing models, including text and speech prediction. In that context, differential privacy allows companies to demonstrate the accuracy of the model without compromising individuals’ privacy. Brown argued that tech savvy companies need to better explain PETs to consumers and corporate customers, but that standardization efforts and favorable DPA positions can also help. In this context, Geff wished for an EDPB update to the 2014 guidance on anonymization, and to have regulators carry out PET testing and share the results with the public, thereby increasing knowledge and trust in the technologies.

Further reading: 

• A Visual Guide to Practical Data De-Identification (FPF, 2016)
• Privacy 2020: 10 Privacy Risks and 10 Privacy Enhancing Technologies to Watch in the Next Decade (FPF, 2020)
• FPF Files Comments on White House Office of Science and Technology Policy Actions to Advance Privacy-Enhancing Technologies (FPF, 2022)

Call for Nominations Open: FPF’s Award for Research Data Stewardship

When companies share data with researchers in a way that protects data, the collaboration can unlock new scientific insights and drive progress in medicine, public health, education, social science, and many other fields. 

FPF is thrilled to announce the open nomination period for FPF’s 3rd Annual Award for Research Data Stewardship. The Award recognizes partnerships between companies and research institutions where a company shares data it holds in a privacy-protective manner with a researcher or research team for scholarly publication. 

An example of an extraordinary award-winning partnership between researchers and a company to advance scientific and medical progress to benefit society through privacy-protective research data sharing is Stanford Medicine researchers and medical wearable and digital biomarker company Empatica. This award-winning collaboration studied whether data collected by Empatica’s researcher-friendly E4 device, which measures skin temperature, heart rate, and other biomarkers, could detect COVID-19 infections before the onset of symptoms. 

The award is presented to the company and its academic partner based on several factors, including the adherence to privacy protection in the sharing process, the quality of the data handling process, and the company’s commitment to supporting academic research. 

Learn more about past award winners.

Save the Date: FPF’s Award for Data Stewardship Virtual Award Presentation (May 10, 2023)

The Award is a part of FPF’s “Corporate Data Sharing for Research: Next Steps in a Changing Legal and Policy Landscape” project to accelerate the safe and responsible sharing of administrative data between companies and academic researchers. This project is supported by the Alfred P. Sloan Foundation, a not-for-profit grantmaking institution whose mission is to enhance the welfare of all through the advancement of scientific knowledge.

GDPR and the AI Act interplay: Lessons from FPF’s ADM Case-Law Report

In May 2022, the Future of Privacy Forum (FPF) launched a comprehensive Report analyzing case-law under the General Data Protection Regulation (GDPR) applied to real-life cases involving Automated Decision-Making (ADM). Our research highlighted that the GDPR’s protections for individuals against forms of ADM and profiling go significantly beyond Article 22 – which provides for the right of individuals not to be subject to decisions based solely on automated processing that produces legal effects or significantly impacts them, and are currently being applied by courts and Data Protection Authorities (DPAs) alike. These range from detailed transparency obligations to applying the fairness principle to avoid situations of discrimination and strict conditions for valid consent in ADM cases. 

As EU lawmakers are now discussing the amendments they would like to include in the European Commission (EC)’s Artificial Intelligence (AI) Act Proposal, what lessons can be drawn from GDPR enforcement precedents–as outlined in the Report–when deciding on the scope and obligations of the Act? 

This blog will explore: the link between the GDPR’s provisions as relevant for ADM and the AI Act Proposal (1); how the AI Act’s concepts of providers and users fare compared to the GDPR’s controllers and processors (2); how the AI Act facilitates GDPR compliance for the deployers of AI systems (3); the opportunities to enhance or clarify obligations under the AI Act through the lens of ADM jurisprudence (4); the overlaps between GDPR enforcement precedents and the AI Act’s prohibited practices or high-risk use cases (5); the issue of redress under the GDPR and the AI Act (6); and a compilation of lessons learned from the FPF Report in the context of the debates around the AI Act (7). 

Note: when referring to case numbers in this blog, the author is using the numbering of cases in the FPF Report. 

1. Both the GDPR and the proposed AI Act are grounded on Article 16 TFEU for the protection of personal data

One of the two legal bases used by the EC to justify the AI Act Proposal is Article 16 of the Treaty on the Functioning of the European Union (TFEU), which mandates the EU to lay down the rules relating to the protection of individuals with regard to the processing of personal data. This means that, at least to some extent, the AI Act’s rules would complement the protections afforded to data subjects under the GDPR, which is also based on Article 16 TFEU. In fact, in their 2021 Joint Opinion on the AI Act, the European Data Protection Supervisor (EDPS) and the European Data Protection Board (EDPB) have suggested making GDPR compliance a precondition for allowing an AI system to enter the European market as a CE marked product under the AI Act. 

AI systems that would be regulated under the initial Proposal of the AI Act are the ones that rely on the techniques and approaches mentioned in Annex I of the Proposal (such as machine learning and logic-based approaches). Such techniques and approaches could constitute or enable ADM schemes to be implemented by controllers covered by the GDPR. However, the AI Act is generally agnostic regarding the decision-making scheme vis-à-vis individuals when the deployment of an AI system is at stake. This means that its scope is broader than ADM falling in or outside of the Article 22 GDPR prohibition (i.e., ‘qualifying’ or ‘non-qualifying’ ADM).

• As an illustration, Article 3(1) AI Act mentions that software that can produce “predictions and recommendations” – which courts and DPAs have generally not considered to be fully automated decision-making thus far – may constitute AI systems covered by the AI Act if they use one or more of the techniques and approaches mentioned in Annex I. 
• Moreover, the AI Act’s rules under Title III, Chapter 2 and 3 on high-risk AI systems (i.e., the ones that are intended to be used as a safety component of a product, or are themselves products, covered by the Union harmonization legislation listed in Annex II, or that fall under the Annex III list) apply to the systems’ providers, users, importers or distributors even if the final decision that affects a natural person should be taken by a human for the user on the basis of the suggestions, signals, prompts or recommendations provided by the AI system. 

However, even where the AI Act does not provide for specific protections for the rights of individuals where AI systems are underpinned by, or result in solely ADM having a legal or similarly significant effect on them, the safeguards provided by Article 22 GDPR will nevertheless kick in. On the other hand, and as mentioned in the Report, even in cases where Article 22 GDPR does not apply to a particular AI system, the rest of the GDPR applies to non-qualifying ADM and generally to the processing of personal data via AI systems. This could include the AI system’s training, validation and input data, as well as the AI system’s outputs, if they qualify as ‘personal data’ under the GDPR, regardless of whether the data are processed by the system’s providers or users. It is also noteworthy that there may be instances of ADM covered by the GDPR that do not involve any use of AI systems, but rather other forms of automating decisions. 

2. The AI Act’s users are generally the GDPR’s controllers in the AI system’s deployment phase

Instead of focusing on the role of the parties in the AI value chain with regard to the processing of personal data, the AI Act Proposal focuses on the entities which develop and market or that use AI systems for commercial purposes, i.e. ‘providers’ and ‘users’ respectively. Each is assigned specific obligations, but most of the regulatory burden is placed on providers, in particular when it comes to high-risk AI systems (HRAIS) and their conformity assessments. 

This way of defining the main actors subject to obligations under the future AI Act may create inconsistency with the GDPR’s definitions, roles, and responsibilities for covered entities. It has also led the EDPB and the EDPS to ask EU policymakers to ensure the AI Act’s obligations are consistent with the roles of controller and processor when personal data processing is concerned. Indeed, ‘providers’ under the AI Act will likely not be considered the data controllers under the GDPR during their AI systems’ deployment phase. 

Under the GDPR, the bulk of obligations, liability and accountability for how personal data are processed are assigned to ‘controllers’. It is the AI Act’s ‘users’ who will rather be the ‘controllers’ under the GDPR in the deployment phase of AI systems. Therefore, even if they have a very limited set of duties under the Act (e.g., Article 29), they maintain liability and accountability about how personal data is used by, or resulting from the use of AI systems, under the GDPR. It is more likely that providers would be qualified as “processors” under the GDPR, processing personal data on behalf of users, notably if they provide support or maintenance services for AI systems involving the processing of personal data on users’ behalf and under their instructions. 

The situation is different in the development phase of AI systems, where ‘providers’ will likely be considered “controllers” under the GDPR whenever they build AI systems relying on collection, analysis, or any other processing of personal data. The same will be the case for the testing phase of AI systems (e.g. bias monitoring) and for post-market monitoring purposes, which may be legally mandatory under Articles 10 and 61 of the AI Act. In these cases, the status of the provider as controller would derive from the law that imposes data processing duties (i.e., the AI Act), as mentioned in EDPB guidance on the concept of controller (para. 24).

Complex questions further arise in relation to potential joint controllership situations under the GDPR, between “users” and “providers” under the AI Act, in the deployment phase of AI systems. For instance, does the legal obligation that providers have under the AI Act to determine the AI system’s intended purpose and technical data collection lead to the qualification of providers as joint controllers with their customers (users), even if they do not obtain access to the AI system’s input or output data, especially after the Court of Justice of the European Union (CJEU)’s Jehovan todistajat ruling? 

3. The AI Act facilitates to a certain extent GDPR compliance for ‘users’ of AI systems 

References to GDPR compliance in the AI Act proposal are scarce. An example is the authorization granted to providers to process special categories of data covered by Articles 9(1) and 10 GDPR when conducting bias monitoring, detection, and correction in relation to HRAIS (under Article 10(5) of the Act). Another is the obligation for users to use the information received from providers’ HRAIS instructions of use when carrying out Data Protection Impact Assessments (DPIAs) under the GDPR, as per Article 29(6) AI Act. However, as the crux of GDPR obligations for controllers in the commercial deployment phase of AI systems will arguably rest with the systems’ users, it is worth exploring whether the AI Act’s obligations imposed on providers of HRAIS may put users in a better position to comply with the GDPR. 

Under Article 13 AI Act, providers have extensive transparency obligations towards their customers (i.e., ‘users’ and, most likely, GDPR ‘controllers’ as explained above), with a view to enable the latter ‘to interpret the system’s output and use it appropriately.’ This transparency comes in the form of instructions of use that should specify, inter alia, the HRAIS’s intended purpose, level of accuracy, performance, specifications for input data, and implemented human oversight measures (as detailed in Article 14 AI Act). Additionally, the HRAIS’s technical documentation that the provider is required to draw up under Article 11 AI Act – and whose elements are listed under Annex IV AI Act – will provide insight about the HRAIS’s general logic, key design choices, main classification choices, relevance of the different parameters, training data sets, and potentially discriminatory impacts, among other features. 

Regardless of the wording under Article 29(6) AI Act, users may use the information obtained from providers under Article 13 AI Act and the HRAIS’s technical documentation not only to comply with their duty to carry out a DPIA but to ensure broader alignment with the GDPR and its transparency imperatives. Such information may also prove useful to comply with other GDPR obligations, such as providing notice to data subjects about profiling and ADM and to complete their records of AI-powered data processing activities under Article 30 GDPR. 

However, it should be noted that, under the AI Act Proposal, duties for providers only exist with regards to HRAIS, whereas the users/controllers’ above mentioned obligations under the GDPR may apply even where the underlying AI systems are not qualified as such. For example, under the EDPB’s criteria, controllers could still be obliged to carry out a DPIA on an AI system that is not included in the Annex III AI Act list of HRAIS, such as a social media recommender system or an AI system used in the context of online behavioral advertising. 

Specifically, with regards to controllers’ notice obligations on profiling and ADM, the FPF Report shows that DPAs and courts in Europe agree that Articles 13(2)(f) and 14(2)(g) GDPR provide for an ex-ante obligation to proactively inform data subjects about the system’s underlying logic, significance and envisaged consequences, and not about the concrete automated decisions that affect them. On the other hand, an obligation to provide decision-level explanations exists where the data subject exercises his or her data access right under Article 15(1)(h) GDPR, as is illustrated by two Austrian DPA decisions (cases 14 and 21 in the Report) and an Icelandic DPA decision (case 38 in the Report). In such instances, DPAs ordered controllers to disclose specific elements of information regarding automated credit or marketing scores attributed to data subjects, notably the algorithm’s parameters or input variables, their effect on the score, and an explanation of why the data subject was assigned a particular score. 

Thus, when complying with the GDPR’s transparency obligations, controllers who qualify as users under the AI Act would find immense value in leveraging the sort of information that Articles 11 and 13 of the AI Act mandate providers to make available with regards to their HRAIS. Recent GDPR case law on ADM could make a case for extending providers’ transparency duties beyond HRAIS, and for ensuring the standard of intelligibility for the information AI providers should make available to users is one that enables the latter to comply with their GDPR obligations. 

A possible avenue to be considered through the legislative process  would be to create a general duty under the AI Act for providers to assist users in their GDPR compliance efforts in relation to the AI systems they sell, even in cases where providers would not act as  data processors for the users. Some impetus for this approach may be found in Article 9(4) AI Act, which mandates providers to inform users about the risks that may emerge from the use of the AI system, and to provide them with appropriate training. 

4. The GDPR’s ADM case law calls for further development or clarification of obligations under the AI Act

Some of the decisions analyzed in the FPF Report may provide indications that the AI Act’s obligations for providers and users – at least when HRAIS are at stake – need further development or clarifications. 

Accuracy and transparency: In a landmark ruling, the Slovak Constitutional Court (case 4 in the Report) established that local law should require additional measures to protect individuals when automated assessments are carried out by State agencies. According to the Court, such measures could include: (i) checking the AI system’s quality, including its error rate; (ii) ensuring that the criteria, models, or underlying databases are up-to-date, reliable, and non-discriminatory; and (iii) making individuals aware of the existence, scope and impacts of automated assessments affecting them. 

• Measures (i) and (ii) seem to be very close to the data quality, accuracy, robustness, and cybersecurity requirements proposed under Articles 10 and 15 AI Act. However, these obligations are geared towards HRAIS’s providers, and not users/controllers. In its decisions against Deliveroo and Foodinho (cases 3 and 6 in the Report), the Italian DPA fined the controllers for not verifying the accuracy and correctness of their automated rider-management decisions and underlying datasets, although these are not explicit requirements under the GDPR’s Article 22(3). Therefore, and at least for HRAIS, the EU legislator could eliminate legal uncertainty by incorporating data quality and accuracy requirements into Article 29, which sets out users’ obligations in this  context. Such a requirement could go beyond merely checking whether the HRAIS’s input data is relevant in view of the intended purpose, as Article 29(3) AI Act currently requires.
• With regards to measure (iii), it should be noted that making individuals aware of the scope and impact of an automated assessment that falls outside of Article 22 GDPR goes beyond Articles 13(2)(f) and 14(2)(g) GDPR. As a rule, DPAs and courts have agreed with the EDPB by stating that the detailed transparency requirements under said provisions only apply to ‘qualifying’ ADM (see Chapter 1.6.c of the Report). Additionally, the Slovak Constitutional Court’s requirement goes further than Article 52 AI Act, which contains disclosure duties for users who deploy certain AI systems that the Proposal considers to be ‘low-risk’, such as emotion recognition systems, biometric categorization systems, and ‘deepfakes’. In the initial text of the AI Act, there are no transparency requirements towards affected persons when HRAIS are at stake, and there are no such obligations for non-high-risk systems other than the ones set out in Article 52 AI Act. Incorporating transparency rights for affected persons in the AI Act, even if only in HRAIS use cases, can reduce information asymmetries between individuals and organizations when decisions are not fully automated (e.g., an AI system whose recommendations merely support human decision-making). 

Lawful grounds for data processing: the AI Act sporadically mentions the interplay with the GDPR’s rules on lawful grounds and exemptions from the prohibition on processing special categories of data. Most notably, Article 54 AI Act creates stringent conditions for further processing of personal data in the context of AI regulatory sandboxes, which do not have an obvious connection to the purpose compatibility test in Article 6(4). Additionally, Article 10(5) AI Act authorizes providers of HRAIS to tackle potential biases through the processing of special categories of data, as long as appropriate safeguards are in place. However, it fails to elaborate on the conditions for the collection of personal data from publicly available sources for mandatory training, validation, and testing of HRAIS. The narrow interpretation of ‘manifestly making data public’ assumed by the EDPB and (more recently) by Advocate General Rantos of the CJEU, together with the enforcement actions against Clearview AI (cases 10 to 13 in the Report), may significantly hinder the possibilities for AI providers to scrape data from the web to test their AI models against bias. Obtaining consent from data subjects for those purposes is often unfeasible, and the legitimate interests lawful ground often plays a limited role when sensitive data are at stake.

Article 10(5) of the AI Act could also potentially facilitate compliance with Article 9(2)(g) GDPR, which allows for the processing of special categories of personal data where their processing is necessary for reasons of substantial public interest, as long as it is based on Union or Member State law. Provided that countering bias would be qualified as “substantial public interest”, the Union law specifically providing for an obligation to process sensitive data – which in this case would be the AI Act, needs to provide for suitable measures to safeguard fundamental rights.

This complexity could offer an opportunity for the EU legislator to set boundaries and clear rules on the collection and use of personal data for training, validation, and testing of AI systems, at least for HRAIS. 

AI risk management through the lens of ADM: Article 9 AI Act forces providers to establish and maintain a risk management system in relation to their HRAIS, including the ‘identification and analysis of [their] known and foreseeable risks.’ National court decisions on ‘legal or similarly significant effects’ of ADM under Article 22 GDPR may provide useful criteria that providers should consider when conducting risk analysis of HRAIS that could affect natural persons. In its Uber and Ola rulings (cases 17 to 19 in the Report), the District Court of Amsterdam analyzed the impact on drivers of the algorithms the companies were relying on for the functioning of their respective mobile applications providing ride-hailing services. The Court looked into: (i) the sensitivity of the data sets or inferences at stake; (ii) the temporary or definitive nature of the effects on data subjects (or their immediacy); (iii) the effects they would have on the drivers’ conduct or choices; and (iv) the seriousness of the financial impacts potentially involved for individuals. Factors such as these could be codified into Article 9 AI Act as valuable guidance for HRAIS providers’ risk management exercises. 

Incorporating human oversight does not rule out the Article 22 GDPR prohibition: a question arises about whether the Article 14 AI Act requirements for the provider to set up human oversight tools for HRAIS would bring such systems outside of Article 22 GDPR. The answer is ‘not necessarily.’ While the prohibition in Article 22 GDPR may apply to AI systems that are not considered ‘high-risk’ under the AI Act, when HRAIS are indeed at stake, Article 14 AI Act only requires providers to incorporate features that enable human oversight, but not to ensure human oversight as a default. 

That will be on the user of HRAIS (i.e., most likely the controller under the GDPR) to ensure via organizational arrangements. If the latter does not, then it may be in breach of Article 22 GDPR, if its ADM scheme is covered by the prohibition. Moreover, we have learned from the decision of the Portuguese DPA against a university that used proctoring software to monitor its students during exams (case 26 in the Report), and the court cases involving Dutch gun applicants and Austrian jobseekers (cases 8 and 9 in the Report), that merely having a human in the loop with the power and competence to make final decisions does not necessarily mean that the decision will not be considered ‘solely’ automated, and thus, that Article 22 GDPR does not apply. For that, human decision-makers need to receive clear instructions or training about why and when they should follow the AI system’s recommendations or not. 

In that respect, the EU legislator could envision that users have an obligation to inform their human decision-makers about the elements listed under Article 14(4) AI Act, so that the latter are able to make informed decisions based on the HRAIS’s output, and avoid so-called ‘automation bias’. 

5. Prohibited AI practices and HRAIS overlap with GDPR enforcement precedents on ADM

In general, the AI Act’s Annex III list of HRAIS seems to be based on litigated uses of AI systems by private and public bodies, including some that were analyzed by courts and DPAs under the GDPR and thereunder deemed to be unlawful for a variety of reasons, as the FPF Report shows. Some examples include:

• Cases of biometric identification and categorisation, where DPAs have often found the underlying collection and storage of data, plus the training of the AI system to be in breach of the GDPR’s rules on lawful grounds (e.g., Clearview AI enforcement cases);
• Systems that select students for university admissions (case 25 in the Report), assess or test them (cases 7 and 26);
• Some uses of AI systems in recruitment processes, which may be justified under the exception in Article 22(2)(a) GDPR (case 3 in the Report);
• AI systems used for worker management or for managing ride sharing apps and the service provided by gig workers were the focus of the Italian DPA (cases 3 and 6), and Dutch courts (cases 17 to 19 in the Report);
• Usages of AI to manage public services and benefits (cases 9, 20 27, and 32 in the Report), where DPAs agree that strong data quality and bias monitoring requirements are essential;
• Automated creditworthiness checks through AI (cases 14, 15, 22, 23, and 36 to 39 in the Report), although Annex III excludes from its scope AI systems that are developed by small scale providers for their own use.

Despite that significant overlap, some AI use cases that were investigated by European DPAs and courts have not been included in the Annex III list, including recommender and content moderation systems (case 24 in the Report), online behavioral advertising systems, and systems used by tax authorities to detect potential fraud (cases 4 and 27 in the Report). Likewise, the EC did not include commercial emotion recognition systems in the HRAIS list, in spite of the recent Hungarian DPA decision against a bank that used an AI system to detect the emotions of the customers who contacted its help center, and the fact that such systems are included in paragraph 6(b) of the Annex when used for law enforcement purposes. These enforcement actions could be an early indicator of the future potential enlargement of HRAIS use cases outlined in Annex III.

Some of these use cases may already be prohibited under the AI Act’s Article 5(1), notably where they would rely on ‘subliminal techniques beyond a person’s consciousness in order to materially distort a person’s behavior’ and may lead to harm, or where they would constitute “social scoring”. The latter could be the case of the SyRi algorithm, which the Dutch Government used when trying to detect instances of benefits fraud in neighborhoods hosting poor or minority groups, and that both the District Court of The Hague and the Dutch DPA deemed to be unlawful (case 27 in the Report). In any case, some of the concepts under Article 5 AI Act (like ‘detrimental or unfavorable treatment’) could be clarified to make the prohibition more certain for AI providers and users, lest the latter will struggle as much as with Article 22 GDPR’s definition of ‘legal or similarly significant effects.’ 

6. The issue of ensuring redress where AI systems not underlined by personal data may significantly affect individuals and communities

It is undeniable that the GDPR provides meaningful rights to individuals who are subject to or affected by AI systems underpinned by processing of personal data. For example, they can access a detailed description of how they were profiled through Article 15, as well as obtain human intervention, express their point of view, and file objections when ‘qualifying ADM’ is at stake. Additionally, providers and users of AI systems have a priori obligations when they act as ‘controllers’ under the GDPR, to implement data minimization, storage limitation, purpose limitation, confidentiality and, most relevant, fairness requirements in how they collect and use personal data, just to name a few of their obligations. 

However, when ‘qualifying ADM’ is not at stake, ‘controllers’ do not have an obligation to ensure human intervention or the possibility to contest an AI-powered decision. This is a gap that could potentially be tackled through the AI Act.

 On this note, although the AI Act is largely not rights-oriented, Article 52 AI Act on disclosure duties for certain AI systems creates a precedent for enshrining judiciable rights for individuals who are exposed to or targeted by AI systems. 

On the other hand, the Slovak Constitutional Court ruling (case 4 in the Report) required the local legislature to enshrine redress rights for individuals to effectively defend themselves against errors of the automated system at issue. Such actions could be possible under the broad effective judicial redress provided by Article 82 GDPR, but only where the underlying processing of personal data would be conducted in breach of GDPR rules such as transparency, fairness, and purpose limitation. For those cases where processing of personal data is not involved, but where AI systems may still significantly impact individuals, the AI Act could potentially fill a redress gap.  

7. Lessons from the ADM GDPR case law in the AI Act context

Some of our findings in the ADM GDPR Case-Law Report are useful for the debate around the AI Act, as shown above. Putting them in the context of the European Commission’s Proposal to regulate AI and the ensuing legislative process, we found that:

1. The AI Act and the GDPR are bound to work in tandem – they are both grounded on Article 16 TFEU and they have many areas where they complement each other, as well as areas where they could be better coordinated so that both their goals are achieved. For instance, the obligations for ‘users’ and ‘providers’ under the AI Act could be further attuned to the GDPR, by enhancing and clarifying transparency requirements of ‘providers’ towards users or by including a general reference to best efforts of providers to facilitate GDPR compliance where ‘users’ are deemed as ‘controllers’. 
2. Data accuracy and data quality requirements under the GDPR could be strengthened with cross-references in the AI Act for HRAIS, such as in Article 29.
3. Further safeguards for the processing of sensitive personal data to counter bias in AI systems could be laid out more clearly for the purpose of protecting fundamental rights.
4. Obligations to train ‘humans in the loop’ on what criteria they should take into account to rely on or to overturn a decision resulting from the application of a HRAIS could complement the protections the GDPR affords through Article 22. 
5. The existing and future GDPR case-law on ADM may be a source for the future enhancement of the HRAIS list provided in Annex III of the AI Act, considering that most of the existing HRAIS list overlaps with ADM systems that were already subject to GDPR enforcement.
6. While the GDPR right to an effective judicial redress can potentially cover most situations where AI systems relying on or resulting in personal data infringe principles like fairness, transparency, and purpose limitation, there is room to consider options for further redress, such as in situations where AI systems rely on non-personal data and significantly affect the rights of individuals or communities. 

Understanding Extended Reality Technology & Data Flows: XR Functions

This post is the first in a two-part series on extended reality (XR) technology, providing an overview of the technology and associated privacy and data protection risks.

Click here for FPF’s infographic, “Understanding Extended Reality Technology & Data Flows.” 

I. Introduction

Today’s virtual (VR), mixed (MR), and augmented (AR) reality environments, collectively known as extended reality (XR), are powered by the interplay of multiple sensors, large volumes and varieties of data, and various algorithms and automated systems, such as machine learning (ML). These complex relationships enable functions like gesture-based controls and eye tracking, without which XR experiences would be less immersive or unable to function at all. However, these experiences often depend on sensitive personal information, and the collection, processing, and transfer of this data to other parties may pose privacy and data protection risks to both users and bystanders.

This blog post analyzes the XR data flows that are featured in FPF’s infographic, “Understanding Extended Reality Technology & Data Flows.” This post focuses on some of the functions that XR devices support today and may support in the near future, analyzing the kinds of sensors, data types, data processing, and transfers to other parties that enable these functions. The next blog post will identify some of the privacy and data protection issues that XR technologies raise.

II. XR Functions

XR devices use several sensors to gather personal data about users and their surroundings. Devices may also log other types of data: data about a person’s location when the device connects to GPS, cell towers, or other connected devices around it; data about the device’s hardware and software; and usage and telemetry data. Devices utilize this data and may further transfer it to enable a variety of functions, which are the technologies that power use cases. For example, eye tracking is a function that enables the use case of optimized graphics.  

A. Mapping and Understanding the User’s Environment

Sensors on XR devices may work in tandem to collect various kinds of data—such as surrounding audio, the device’s acceleration, orientation, and environment depth data—to map and find objects in the user’s physical environment. Mapping the space entails constructing three-dimensional representations of the user’s environment in order to accurately situate users and content within a virtual space. Understanding the user’s environment involves identifying physical objects or surfaces in the user’s space to help place virtual content. These functions may enable shared experiences and other use cases.

To map and identify objects in the user’s environment, XR devices collect data across a number of sensors, such as microphones, cameras, depth sensors, and inertial measurement units (IMUs), which measure movement and orientation. When a sensor is experiencing a performance problem or certain sensor data is not available, the device may utilize other sensors, which may implicate a less accurate data proxy or fallback. For example, if photons from a depth sensor fail to indicate a user’s position, the device may use an AI system to fill in the sensory gap using pixels closest to the area where the depth sensor directed the photons. 

Once the device has gathered data through its sensors, the device and XR applications may need to further process this data to map and identify objects in the user’s physical space. The kind of processing activity that occurs depends on the features a developer wants its application to have. A processing activity that often occurs after a device collects sensor data is sensor fusion, in which algorithms combine data from various sensors to improve the accuracy of simultaneous localization and mapping (SLAM) and concurrent odometry and mapping (COM) algorithms. SLAM and COM map users’ surrounding areas, including the placement of landmarks or map points, and help determine where the user should be placed in the virtual environment. Some types of XR, including certain MR applications, leverage computer vision AI systems to identify and place specific objects within an environment. These applications may also use ML models to help determine where to place “dynamic” virtual content—virtual objects that respond to changes in the environment caused by adjustments to the user’s perspective. These mapping and object identification functions may also allow for shared experiences. For example, in a theoretical pet simulation, multiple users could toss a virtual ball against a building wall for a virtual puppy to catch. 

While XR devices generally do not send mapping and environmental sensor data to other parties, including other users, there are a few exceptions. For example, raw sensor data may be transmitted to XR device manufacturers to improve existing device functions, such as the placement and responsiveness of virtual content that users interact with. An XR device may also process and relay users’ location information, such as approximate or precise geolocation data, to enable shared experiences within the same physical space. For instance, two individuals in a public park could interact with each other’s virtual pets in an AR app, with each player using their own devices that recognize the placement of both the virtual pets and the other player in the park. In other situations, certain parties can observe processed sensor and other data, such as an application developer or an entity controlling the external server that enables an application’s multiplayer functionality. Therefore, the nature of the data, device and application features may affect who can access XR data.

B. Controller and Gesture-Based Interactions with the Environment

Some XR technologies gather and process sensor data to enable controller- and gesture-based interactions with physical and virtual content, including other users. Gesture-based controls allow users to interact with and manipulate virtual objects in ways that are more reflective of real-world interactions. Most devices use IMUs and outward-facing cameras combined with infrared (IR) or LED light systems to gather data about the controller’s position, such as the controller’s linear acceleration and rotational velocity, as well as optical data about the user’s environment. Some manufacturers are introducing new data collection systems that overcome other methods’ deficiencies, such as when the controllers are outside of the cameras’ view. When visual information about a controller’s position becomes unavailable, IMU data may act as a fallback or proxy for determining controller location. For gesture-based controls, devices gather data about the user’s hands through outward-facing cameras. 

XR technologies use algorithms and ML models to provide controller- and gesture-based controls. In controller-based systems, algorithms use data about the controller’s position to detect and measure how far away the controllers are from the user’s head-mounted display (HMD). This allows the user’s “hands” to interact with virtual content. For example, MR or VR maintenance training could allow a mechanic to practice repairing a virtual car engine before performing these actions in the real world. Gesture-based controls utilize ML models, specifically deep learning, to construct 3D copies of the user’s hands by processing images of their physical-world hands and determining the location of their joints. The 3D copies may be sent to developers to enable users to manipulate and interact with virtual and physical objects in applications through pointing, pinching, and other gestures. 

C. Eye Tracking and Authentication

Eye tracking and, to a lesser extent, authentication power a variety of XR use cases, such as user authentication, optimized graphics, and expressive avatars. An XR device can use data from the user’s eye to authenticate the person using the device, ensuring that the right user profile, with its unique settings, applies during a session. Devices may use inward-facing IR cameras to gather information about the user’s eyes, such as retina or iris data, to this end. ML models can then use the collected eye data to determine whether the person is who they claim to be. 

Now and in the future, XR devices will increasingly feature eye tracking to optimize graphics. Graphics quality can affect a user’s sense of immersion, presence, and embodiment in XR environments. One technology that can enhance graphics in XR environments is dynamic foveated rendering, or eye-tracked foveated rendering (ETFR), which tracks a user’s eyes to reduce the resolution that appears in the peripherals of the HMD’s display. This allows the device to display the user’s focal point in high resolution, reduce processing burdens on the device, and lower the chance of motion sickness by addressing a cause of latency. XR devices may also facilitate better graphics by measuring the distance between a user’s pupils or interpupillary distance (IPD), which affects the crispness of the images on the headset display. For example, in a virtual car showroom, a device may utilize the above technologies to enhance the detail of the car feature that the user is looking at and ensure that objects appear at the correct scale. 

To determine the parts of the HMD display that should be blurred and help focus the lenses, a device may use inward-facing IR cameras to gather data about the user’s eyes. Some XR devices may use ML models, such as deep learning, to process eye data to predict where a user is looking. These conclusions inform what parts of the display the model blurs. In the future, XR devices may use algorithms to more accurately measure the distance between a user’s pupils, further improving the crispness of the graphics that appear on the HMD’s display. 

Eye tracking is a subset of a broader category of XR data collection—body tracking. Body tracking captures eye movements (described above), facial expressions, and other body movements, which can help create avatars that reflect a user’s reactions to content and expressions in real-time. Avatars are the person’s representative in a virtual or other artificial environment. Avatars are already featured in popular shared VR experiences, such as VRChat and Horizon Worlds, but several factors limit their realism. Today’s avatars typically do not reflect all of a user’s nonverbal responses and may lack certain appendages, like legs. Going forward, an avatar may mirror a user’s reactions and expressions in real-time. This will enable more realistic social, professional, and personal interactions. For example, in a VR comedy club, a realistic avatar may display facial and other body movements to more effectively deliver or react to a punchline. 

To depict a user’s reactions and expressions on their avatar, XR technologies need data about the eyes, face, and other parts of the user’s body. A device may use IMUs and internal- and outward-facing cameras to capture information about the user’s head and body position, gaze, and facial movements. XR devices may also use microphones to capture audio corresponding with certain facial movements, known as visemes, as a proxy for visuals of the user’s mouth when the latter is unavailable. For instance, the sound of laughter may cause an avatar to show behavior associated with the act of laughing. 

As with the other functions, body-based insights on XR devices may need to transmit data to other parties, which may use algorithms to process collected data to create expressive avatars. XR devices may transmit data about a user’s avatar, such as gaze and facial expression information to app developers. Developers may use ML models, including deep learning, to process information about the user’s eyes and face to detect the face and make conclusions about where a user is looking and the expression they are making. For facial movements, a deep learning model may analyze each video frame featuring the user’s face to determine with which expressions their facial movements correspond. These expressions are then displayed on the avatar. Devices may then share the avatar with central servers so that other users can view and interact with the avatar.

In addition to avatar creation and operation, future XR technologies may monitor gaze and pupil dilation, motion data, and other information derived from the user’s body to generate behavioral insights. XR tech may be capable of using sensor data that is generated in response to stimuli and interactions with content to make inferences about user interests, as well as their physical, mental, and emotional conditions. When combined with information processed by other sensors, such as brain-computer interfaces (BCIs), these body-derived data points could contribute to the creation of more granular individual profiles and insights into the user’s health. In a medical XR application, for example, doctors could use gaze tracking to diagnose certain medical conditions. However, other parties may use the functions to learn about other highly sensitive information, such as a user’s sexual orientation, which could harm individuals. 

III. Conclusion

XR technologies often rely on large volumes and varieties of data to power device and application functions. Devices often feature several sensors to gather this data, such as outward-facing cameras and IMUs. To enable different kinds of XR use cases, including shared experiences, devices may utilize ML models that process data about the user and their environment and transmit this data to other parties. While the collection, processing, and transmission of this data may be integral to immersive XR experiences, it can also create privacy and data protection risks for users and bystanders. The next blog post analyzes some of these risks. 

Read the next blog post in the series: Privacy and Data Protection Risks to Users and Bystanders

Federal Court deems university’s use of room scans within the home unconstitutional

I. Summary

A federal court recently ruled that a public university’s use of room-scanning technology during a remotely proctored exam violated a student’s Fourth Amendment right to privacy. The decision in Ogletree v. CSU is the clearest indication to date of how courts will treat Fourth Amendment challenges to public higher education institutions’ use of video room scans within students’ homes. Schools, test administrators, and professional licensure boards often use proctoring technologies in an effort to dissuade cheating by remote test takers. These technologies take a variety of forms and may involve live proctors observing test takers via webcam, eye-tracking technology, artificial intelligence, recording via webcam and microphone, plug-ins that disable a test taker’s computer from accessing third-party websites or stored materials, and room scans. At issue in this case was a room scan of a student’s bedroom workspace. 

Since the start of the COVID-19 Pandemic, more schools have incorporated remote proctoring software into testing procedures. The increased use of such technology in both K-12 and higher education settings has led to widespread discussion about the resulting privacy implications, including whether remote proctoring practices violate students’ privacy rights. In August, the US District Court for the Northern District of Ohio offered some clarity–as well as new questions–when it granted summary judgment to college student Aaron Ogletree (“the student”), in his Fourth Amendment lawsuit against Cleveland State University (“CSU,” or “the university”). The Court determined that the room scan amounted to a Fourth Amendment search because: (1) CSU is a public institution and thus a state actor; (2) the student had an intuitive expectation of privacy within the bedroom of his home; and (3) the student’s expectation of privacy was reasonable and one generally accepted by society. The Court further found that CSU’s Fourth Amendment search was unreasonable by weighing four factors: (1) the student’s privacy interest; (2) the nature of the search; (3) the government concern; and (4) efficacy. Finding only one factor (the government concern) weighed in favor of CSU, the Court deemed the search unreasonable and thus unconstitutional.

While the Court’s decision is not dispositive of many interesting issues, it offers clarity on some and poses new questions about others. Some of the takeaways from the decision include:

• Going forward, courts are likely to be skeptical of public higher education institutions conducting room scans within a student’s home absent a warrant.

• Although nonpublic actors are not directly implicated, the Court’s decision may lead to broader critiques of room scans. These critiques may influence new norms for institutions such as private universities and nonpublic professional licensure boards. 

• This decision calls into question the lawfulness of public institutions’ use of other proctoring features beyond room scans. 

• Several important questions about what this decision will mean for remote proctoring and student privacy remain unanswered.

II. Analysis

The student in this case sued his university after he was asked to complete a room scan of his bedroom workspace before a remote exam, alleging that the practice violated his Fourth Amendment rights. The Fourth Amendment of the United States Constitution states:

The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no warrants shall issue, but upon probable cause, supported by oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.

The Court’s opinion first determines that CSU’s room scan amounted to a Fourth Amendment search, and second, rules that the search was unreasonable. While Fourth Amendment decisions are highly fact-specific, the Court’s analyses of these two factors indicate how other courts may evaluate similar cases in the future.

The Room Scan was a Fourth Amendment Search

In the first stage of its analysis, the Court determined that the room scan did in fact amount to a Fourth Amendment search. Fourth Amendment searches involve government action that violates “a subjective expectation of privacy that society recognizes as reasonable.” Because CSU is a public institution, it is a government actor. As such, the Court had to determine whether the student possessed a subjective expectation of privacy when he took the remote test, and if so, whether his expectation of privacy was one reasonably recognized by society. The Court determined both of these elements were met. 

Much of the Court’s analysis hinges on the location where the room scan took place: the student’s bedroom within his home. The Court acknowledged that privacy within the home is a cornerstone of the Fourth Amendment jurisprudence and that there is little question as to whether the student had a subjective expectation of privacy in his bedroom. The Court further found this expectation to be one reasonably understood by society as there is widespread agreement that privacy interests exist within one’s home. 

In arriving at its conclusion that a Fourth Amendment search occurred, the Court rejected multiple arguments from CSU. For example, since proctoring services commonly include room scans and students routinely use them, the university argued that the student’s expectation of privacy was unreasonable. The Court was not persuaded by this argument, noting that a practice can be commonly used but still implicate a privacy interest. A lack of opposition from other students does not invalidate the inherent expectation of privacy that exists in the home. The Court explained: 

Though schools may routinely employ remote technology to peer into houses without objection from some, most, or nearly all students, it does not follow that others might not object to the virtual intrusion into their homes or that the routine use of a practice such as room scans does not violate a privacy interest that society recognizes as reasonable, both factually and legally.

CSU also cited case law regarding routineness and plain view. Here, the university attempted to equate room scans to activities that courts have declined to characterize as Fourth Amendment searches. Specifically, these precedents involve routine practices that employ modern technologies to observe what is openly visible. Once again, the Ogletree Court rejected the university’s arguments, holding: “[r]oom scans go where people otherwise would not, at least not without a warrant or invitation.” As such, the routine use of room scans for remote exams does not make the student’s expectation of privacy in his bedroom unreasonable. 

The Court also rejected the notion that a room scan is not a Fourth Amendment search simply because the technology that room scans use is publicly available. Here, the opinion notes:

While cameras might be generally available and now commonly used, members of the public cannot use them to see into an office, house, or other place not publicly visible without the owner’s consent.

Moreover, the Court was not persuaded by the university’s invocation of Quon or Wyman – two Supreme Court cases that involved arguably similar searches. While the Ontario v. Quon decision involved similar facts and found a government search to be lawful, the case centered around employee monitoring software, not remote proctoring room scans. The Ogletree Court declined to apply Quon and subsequent case law beyond an employment context. The Court also engaged in a lengthy discussion about the precedent from Wyman v. James. Wyman is a Supreme Court case finding that mandatory home inspections to qualify for government benefits are not Fourth Amendment searches. The Court examined many factors in its analysis of Wyman’s applicability to the case at hand. Ultimately, however, the Court determined that the circumstances in the Wyman case were fundamentally different from the Ogletree facts and could not be equated to support the university’s argument.

The Fourth Amendment Search was Unreasonable

Having determined that the room scan was a Fourth Amendment search, the Court then moved to its second analysis, the reasonableness of the search. This analysis employs a balancing test whereby a court weighs the “intrusion on the individual’s Fourth Amendment interests against its promotion of legitimate governmental interests.” 

The room scans at issue in this case can be characterized as suspicionless searches, in that they were not conducted because of specific suspicion of a single student. While suspicionless Fourth Amendment searches are generally unconstitutional, the Ogletree Court acknowledged the exception for searches in which the government has “special needs beyond the normal need for law enforcement.” The test for this exception–and the test the Ogletree Court used in its analysis–considers four factors:

1. The nature of the privacy interest affected; 
2. The character of the intrusion; 
3. The nature and immediacy of the government concern; and 
4. The efficacy of this means of addressing the concern

Factor 1: The Privacy Interest

For the first factor, the Court reiterated the well-understood privacy interest that existed within the student’s bedroom. The high value that society and Fourth Amendment jurisprudence place on privacy within the home worked in favor of the Plaintiff in this case. CSU argued that students have different Fourth Amendment rights in a school setting given the unique custodial relationship that exists. Relying on case law involving public K-12 institutions, CSU suggested that the student had lesser privacy interests at the time of the room scan. The Court rejected this assertion, however, pointing out that CSU’s argument rested on precedent involving minor students whose school attendance is required. Here, the student was an adult who voluntarily enrolled in a higher education institution. The Court made this distinction and explained:

Mr. Ogletree was an adult at the time of the search at issue and enrolled at Cleveland State by choice. Although this setting might affect the nature of the privacy interest at stake to some degree, it is difficult to see how enrollment in a higher educational institution would limit the core protections of the home under the Fourth Amendment on the facts and circumstances of this case.

Factor 2: The Intrusion

For the “character of the intrusion” factor, the Court relied heavily on factors from the case record. The Court discussed the lack of alternatives the student had when taking the test given the COVID-19 Pandemic and his inability to take the exam anywhere other than his bedroom. The Court noted that even before the Pandemic, it would be difficult for students to weigh their privacy interests when deciding on a testing location given the university’s policy of leaving remote testing decisions to the discretion of individual instructors. Here, the Court acknowledged:

In normal times, a student might be able to choose another college or among classes with different options for tests and assessments. A student who valued privacy more might opt for courses with in-person tests, while another who prefers convenience might tolerate an intrusion of the sort at issue here. Cleveland State’s policies and practices make such choices and tradeoffs opaque, at best. Faculty members have discretion on how to implement remote testing.

The Court further pointed out that the student only had two hours of notice of the room scan because of policy changes.

The Court’s analysis for this part of the four-part test also included factors that favored CSU, including the fact that the room scan was minimally invasive in that it lasted less than a minute and was in the student’s control. Moreover, the Court acknowledged that some privacy interests might be traded away for the exchange of a good or service, such as an education or degree. However, the Court maintained that regardless of any tradeoffs, the student kept his constitutional rights. When weighing the factors that favored the student against the factors that favored the university, the Court ultimately concluded that “the core protection afforded to the home, the lack of options, inconsistency in the application of the policy, and short notice of the scan weighed in Plaintiff’s favor.”

Factor 3: The Government Concern

The third factor, “nature and immediacy of the government concern” weighed in the favor of CSU as the parties and the Court agreed that the university had a legitimate interest in preventing academic dishonesty. The room scans were ultimately employed to help meet this interest.

Factor 4: Efficacy

The Court then turned to the fourth and final prong of the test: the efficacy of the school’s means to address its concern. For this factor, in particular, it is important to remember that this case deals with one form of proctoring: room scans. Here, the Court considered the alternative options that existed to achieve the university’s goal of deterring academic dishonesty. 

In his argument against the effectiveness of room scans, the student pointed out that the school has many proctoring methods at its disposal, including technology that prevents a test taker from accessing the internet or saved documents during an exam, hiring proctors to monitor students during the duration of a test, and AI detection for plagiarism.  In support of his argument about the various alternatives that existed, the student pointed out that the university’s policy left proctoring methods to the discretion of individual educators. The student also argued against the efficacy of room scans by discussing different ways a student who is required to complete a room scan before an exam could still access prohibited materials during the testing period.

In contrast, the university argued that a room scan is an effective method to achieve the university’s interests in preventing academic dishonesty. To support its argument that other proctoring features do not offer the same detection and deterrent functions that room scans do, the university suggested that such programs “are not effective at achieving these functions and that sometimes they are inappropriate for students with disabilities.” Here, the university’s argument seemed to hinge on the ineffectiveness of other methods of remote proctoring. 

The Court was ultimately persuaded by the student’s arguments against efficacy and concluded that “a record or sporadic and discretionary use of room scans does not permit a finding that room scans are truly, and uniquely, effective at preserving test integrity.” Not only did other safeguarding methods exist, but the Court also pointed out the existence of alternative evaluation methods–such as a final project or term paper–that do not require remote proctoring at all. This section of the Court’s analysis is especially interesting given the efficacy critiques that often arise in the public discourse surrounding remote proctoring technology. 

As three of the four factors (nature of the privacy interest affected, character of the intrusion, and efficacy of means) weighed in favor of the student, and only one of the factors (nature and immediacy of the government concern) weighed in favor of the university, the Court concluded that the Fourth Amendment search was not reasonable. 

Having determined that the room scan amounted to a Fourth Amendment search and that the search was unreasonable, the court found the Plaintiff’s Fourth Amendment rights had been violated.

Procedurally, it is notable that the District Court granted the student’s motion for summary judgment. Summary judgment requires a party to show that there is “no genuine dispute as to any material fact and the movant is entitled to judgment as a matter of law.” Moreover, a grant of summary judgment requires a court to consider the evidence “in the light most favorable to the party opposing the motion.” That means the Court in Ogletree not only determined that the student was entitled to judgment as a matter of law, but it arrived at this determination by viewing the evidence in a light that favored the university.

Impact on Remote Proctoring

The Court’s decision to grant Ogletree’s motion for summary judgment is the clearest indication to date of how federal courts may treat Fourth Amendment cases involving the use of room scans in remote proctoring software. Nonetheless, it is too soon to tell whether other courts will follow suit or what this decision will mean for remote proctoring generally. Regardless, schools that use remote proctoring software, and more specifically, deploy room scan features, should be mindful of the decision in this case and the Court’s reasoning.  

Fourth Amendment cases are especially fact dependent. As such, it is very possible that the case could have had a different result had the circumstances been even slightly different. For example, because of the unique relationship between schools and K-12 pupils, there remains ambiguity as to whether the Court would have arrived at the same result in a case about an elementary or high school student. Moreover, this case focused specifically on room scans; it is unclear whether other forms of remote proctoring, such as ongoing monitoring when a student takes a remote test in their home, would amount to a Fourth Amendment violation under this Court’s efficacy and reasonableness analyses. Nonetheless, this case is a win for student privacy and an indicator of how other courts may rule in future cases. 

Regardless of the questions that still exist, interested parties–including schools, students, boards of licensure, and proctoring companies–should be aware of this decision. Entities that employ proctoring software should be mindful of the Court’s reasoning and consider potential legal risks and privacy implications before employing proctoring technologies or requiring room scans within the home.

New Infographic Highlights XR Technology Data Flows and Privacy Risks

As businesses increasingly develop and adopt extended reality (XR) technologies, including virtual (VR), mixed (MR), and augmented (AR) reality, the urgency to consider potential privacy and data protection risks to users and bystanders grows. Lawmakers, regulators, and other experts are increasingly interested in how XR technologies work, what data protection risks they pose, and what steps can be taken to mitigate these risks.

Today, the Future of Privacy Forum (FPF), a global non-profit focused on privacy and data protection, released an infographic visualizing how XR data flows work by exploring several use cases that XR technologies may support. The infographic highlights the kinds of sensors, data types, data processing, and transfers that can enable these use cases.

XR technologies are powered by the interplay of multiple sensors, large volumes and varieties of data, and various algorithms and automated systems, such as machine learning (ML). These highly technical relationships enable use cases like shared experiences and expressive avatars. However, these use cases often depend on information that may qualify as sensitive personal data, and the collection, processing, and transfer of this data may pose privacy and data protection risks to both users and bystanders.

“XR tech often requires information about pupil dilation and gaze in order to function, but organizations could use this info to draw conclusions—whether accurate or not—about the user, such as their sexual orientation, age, gender, race, and health,” said Daniel Berrick, a Policy Counsel at FPF and co-author of the infographic. These data points can inform decisions about the user that can negatively impact their lives, underscoring the importance of use limitations to mitigate risks.

FPF’s analysis shows that sensors that track bodily motions may also undermine user anonymity. While tracking these motions can help map a user’s physical environment, it can also enable digital fingerprinting. This makes it easier for parties to identify users and bystanders while raising de-identification and anonymization concerns. These risks may discourage individuals from fully expressing themselves and participating in certain activities in XR environments due to their concerns about retaliation.

Moreover, FPF found that legal protections for bodily data may depend on privacy regulations’ definitions of biometric data. It is uncertain whether US biometric laws, such as the Illinois Biometric Information Privacy Act (BIPA), apply to XR technologies’ collection of data. “BIPA applies to information based on ‘scans’ of hand or face geometry, retinas or irises, and voiceprints, and does not explicitly cover the collection of behavioral characteristics or eye tracking,” said Jameson Spivack, Senior Policy Analyst, Immersive Technologies at FPF. Spivack was also a co-author of the infographic.

This highlights how existing laws’ protections for biometric data may not extend to every situation involving XR technologies. However, protections may apply to other special categories of data, given XR data’s potential to draw sensitive inferences about individuals.

Meet David Sallay, FPF’s new Youth & Education Privacy Director

FPF is thrilled to announce the new Director of our Youth & Education Privacy Program, David Sallay. David comes to FPF from the Utah State Board of Education, where he previously served as the Chief Privacy Officer and the Student Privacy Auditor at the Utah State Board of Education, where he worked with schools and districts on implementing Utah’s state student privacy law. 

Before focusing on privacy, he worked in education as a teacher of English as a Foreign Language at Qatar University and high schools in Hungary. He holds a Master’s in Public Policy from the University of Utah and a Master’s in Education from the University of Pittsburgh. 

Learn more about David in the Q&A below.

1. You started out as a teacher–how did you get involved in student privacy?

I was in the right place at the right time. I worked for the Utah State Board of Education as the assessment data specialist when Utah passed its student privacy law, which created several new positions. It seemed like an exciting new area to work in, so I applied and got the job.

2. What’s one thing that you wish more teachers understood about student privacy?

I wish more teachers knew that student privacy doesn’t have to be a zero-sum game. We shouldn’t have to make trade-offs where you can’t teach as well or use helpful tools in the classroom to achieve security or privacy. Still, since privacy is unfortunately rarely the default setting, we’ll want to work together to ensure our use of technology and data in the classroom is a true win-win where students learn better while having their data protected.

3. A lot is going on in youth & education privacy today…What’s one thing that you are optimistic or encouraged about?

I am encouraged by the move to age-appropriate design codes in some jurisdictions since it should lead to more products mapping better to the expectations of parents, educators, and children.

4. What’s one thing that you are worried or concerned about?

On the flip side, I am also worried about age-appropriate design codes if they don’t strike the right balance, and it can be really hard to find that Goldilocks zone of just right.

5. What’s something that you think is flying under the radar?

Many of the most damaging privacy harms are small in terms of the number of students they impact and likely won’t appear on the front page of a newspaper. If you see a lot of the privacy complaints that go to the US Department of Education or that we investigated in Utah, they were very often one individual at the school doing something with a record that harmed one or two students. So I think it’s important to look at both big-picture issues as well as the smaller ones.

6. Utah’s approach to student privacy has been held up as a model for other states to follow (including by FPF!). What advice do (or would) you give another state looking to replicate your approach?

One of the things I think our legislature and state board did well was gathering stakeholders together to study the issue instead of adopting a one-size-fits-all approach. The other thing is to put someone in charge full-time (e.g., a chief privacy officer) to provide adequate support.

7. What are you reading/listening to lately for work (related to youth privacy)?

I’ve been reading Privacy’s Blueprint by Woodrow Hartzog since it focuses a lot on design as a way to build trust and will hopefully help me better wrap my brain around the new design codes being proposed.

8. What are you reading/listening to lately for fun?

I’m really drawn to anything about other Hungarian-Americans. Since it’s October, I’m reading a biography of Béla Lugosi that is interesting (there was a lot more to him than just playing Dracula). Also, every October, I try to listen to every Oingo Boingo album since that gets me in a proper Halloween mood.

9. Do you remember the first time you heard about FPF? What made you want to join the team officially? What is your top priority?

I first heard about FPF when I started working on privacy in Utah. We didn’t know how to build our program and quickly discovered FPF as a resource. At that time, the FPF youth and education team was just one person, so it’s been really neat to see the program grow. A lot of the appeal in joining was being able to work on privacy problems beyond the borders of Utah and beyond education (i.e., in youth privacy). So far, it’s been really good to meet everyone on the team. My top priority right now is to get to know the team better and understand their interests and goals so I can figure out the best way to support them so that collectively we can provide the most value for our stakeholders and ultimately make a real impact for students and youth.

Interested in learning more about FPF’s Youth & Education Privacy work? Visit Student Privacy Compass to learn more.