The aim of this blog post is to provide further clarity on the Crosswalk and what it means for organizations developing and deploying AI systems. The blog post is structured into four parts.
First, the blog post will briefly summarize AI Verify (a fuller summary on which can be found in FPF’s blog post here).
Second, the blog post will summarize the NIST AI RMF, what it aims to achieve, and how it works.
Third, the blog post will explain the “Crosswalk”, and how users can expect to use the document.
Fourth, the blog post will conclude with observations on what the Crosswalk means for international cooperation in the area of AI governance.
AI Verify – Singapore’s AI governance testing framework and toolkit
AI Verify is an AI governance testing framework and toolkit launched by the IMDA and the Personal Data Protection Commission of Singapore (PDPC). First announced in May 2022, AI Verify enables organizations to conduct a voluntary self-assessment of their AI systems through a combination of technical tests and process-based checks. In turn, this allows companies who use AI Verify to objectively and verifiably demonstrate to stakeholders their responsible and trustworthy deployment of AI systems.
At the outset, there are several key characteristics of AI Verify that users should be mindful of.
First, while AI Verify is presently in a “Minimum Viable Product” phase, it is designed to provide one-stop AI testing and will therefore evolve going forward.
Second, rather than attempting to define ethical or governance standards or thresholds (and providing AI systems with a “pass” or “fail” mark), AI Verify merely provides an objective measure to verify the performance of an AI system. By open-sourcing AI Verify, the IMDA and the PDPC hope that the global AI testing community can gravitate towards workable standards in various industries and use cases.
Third, to promote user trust, AI Verify permits organizations to conduct self-testing on premises, and also allows for third-party testing. The AI Verify Foundation, which oversees the development and adoption of AI Verify, also welcomes users to contribute codes and plug-ins to enhance the toolkit.
Fourth, and crucially, AI Verify has presently been developed to support supervised learning AI models (e.g. binary classification, multiclass classification, regression models). It has not been designed for generative AI systems, though this is something that the AI Verify Foundation is working on separately through initiatives such as its Generative AI Evaluation Sandbox and the LLM Evaluation Catalogue.
AI Verify comprises two parts: (1) a Testing Framework, which references 11 internationally-accepted AI ethics and governance principles grouped into 5 pillars; and (2) a Toolkit that organizations can use to execute technical tests and to record process checks from the Testing Framework. The 5 pillars and 11 principles under the Testing Framework are:
Transparency on the use of AI and AI systems
Principle 1 – Transparency: Providing appropriate information to individuals impacted by AI systems
Understanding how an AI model reaches a decision
Principle 2 – Explainability: Understanding and interpreting the decisions and output of an AI system
Principle 3 – Repeatability/reproducibility: Ensuring consistency in AI output by being able to replicate an AI system, either internally or through a third party
Ensuring safety and resilience of the AI system
Principle 4 – Safety: Ensuring safety by conducting impact/risk assessments, and ensuring that known risks have been identified / mitigated
Principle 5 – Security: Ensuring the cyber-security of AI systems
Principle 6 – Robustness: Ensuring that the AI system can still function despite unexpected input
Ensuring Fairness
Principle 7 – Fairness: Avoiding unintended bias, ensuring that the AI system makes the same decision even if a certain attribute is changed, and ensuring that the data used to train the model is representative
Principle 8 – Data governance: Ensuring the source and quality of data by adopting good data governance practices when training AI models
Ensuring proper (human) management and oversight of the AI system
Principle 9 – Accountability: Ensuring proper management oversight during AI system development
Principle 10 – Human agency and oversight: Ensuring that the AI system is designed in a way that will not diminish the ability of humans to make decisions
Principle 11 – Inclusive growth, societal and environmental well-being: Ensuring beneficial outcomes for people and the planet.
As mentioned earlier, FPF’s previous blog post on AI Verify provides more detail on the objectives and mechanics of AI Verify’s Testing Framework and Toolkit. This summary merely sets the context for readers to better appreciate how the Crosswalk document should be understood.
AI Risk Management Framework – U.S. NIST’s industry-agnostic voluntary guidance on managing AI risks
The AI RMF underscores the perspective that responsible AI risk management tools can assist organizations in cultivating public trust in AI technologies. Intended to be sector-agnostic, the AI RMF is voluntary, flexible, structured (in that it provides taxonomies of risks), measurable and “rights-focused”. The AI RMF outlines mechanisms and processes for measuring and managing AI systems and provides guidance on measuring accuracy.
The AI RMF itself is broken into two parts. The first part outlines various risks presented by AI. The second part provides a framework for considering and managing those risks, with a particular focus on stakeholders involved in the testing, evaluation, verification and validation processes throughout the lifecycle of an AI system.
The AI RMF outlines several AI-related risks
The AI RMF outlines the following risks presented by AI: (1) Harm to people – e.g. harm to an individual’s civil liberties, rights, physical or psychological safety or economic opportunity; (2) Harm to organizations – e.g. harm to an organization’s reputation and business operations; and (3) Harm to an ecosystem – e.g. harm to the global financial system or supply chain. It also notes that AI risk management presents unique challenges for organizations, including system transparency, lack of uniform methods or benchmarks, varying levels of risk tolerance and prioritization, and integration of risk management into organizational policies and procedures.
The AI RMF also provides a framework for considering and managing AI-related risks
The “core” of the AI RMF contains a framework for considering and managing these risks. It comprises four functions: “Govern”, “Map”, “Measure”, and “Manage.” These provide organizations and individuals with specific recommended actions and outcomes to manage AI risks.
Governing – This relates to how AI is managed in an organization, such as creating a culture of risk management, outlining processes, documents and organizational schemes that anticipate, identify and manage AI risks, and providing a structure to align with overall organizational principles, policies and strategic priorities. Categories within the “Govern” prong include:
Creating and effectively implementing transparent policies, processes and practices across the organization related to the mapping, measuring and managing of AI risks;
Maintaining policies and procedures to address AI risks and benefits arising from using third-party software and data; and
Mapping – This establishes the context in which an organization can identify and frame the risks of an AI system (such as who the users will be and what their expectations are). Once mapped, organizations should have sufficient contextual knowledge on the impact of the AI system to decide whether to design, develop or deploy that system. Outcomes from this function should form the basis for the measuring and managing functions. Specific categories within this function include:
Assessing AI capabilities, targeted usage, goals and expected benefits and costs;
Mapping risks and benefits for all components of the AI system, including third-party software and data; and
Determining the impact of the system on individuals, groups, communities, organizations and society.
Measuring – This function is about using information gathered from the mapping process as well as other tools and techniques to analyze and monitor AI risks. This function may be addressed by implementing software testing and performance assessment methodologies. The “Measure” prong also includes tracking metrics for trustworthy characteristics and impacts of the AI system. It should provide an organization’s management with a basis for making decisions when trade-offs arise. Specific categories include:
Identifying and applying appropriate methodologies and metrics;
Evaluating AI systems for trustworthiness;
Maintaining mechanisms for tracking AI risks; and
Gathering feedback about the efficacy of measurements being used.
Managing – This function involves allocating resources to address risks identified through the functions above, and on a regular basis. Organizations should use information generated from the Governing and Mapping functions to manage and decrease the risk of AI system failures by identifying and controlling for risks early. Organizations can implement this function by regularly monitoring and prioritizing AI risks based on assessments from the Mapping and Measuring functions. Specific categories within this function include:
Planning, preparing, implementing and documenting strategies to maximize AI benefits and minimize negative impacts, including input from relevant AI actors in the design of these strategies; and
Ensuring that risks that arise are documented and monitored regularly.
The Crosswalk is a mapping document that guides users on how adopting one framework can be used to meet the criteria of the other
To observers familiar with AI governance documentation, it should be apparent that there is complementarity between both frameworks. For instance, the AI Verify framework contains processes that would overlap with the RMF framework for managing AI risks. Both frameworks also adopt risk-based approaches and aim to strike a pragmatic balance between promoting innovation and managing risks.
Similar to other crosswalk initiatives that NIST has already done with other frameworks, this Crosswalk is aimed at harmonizing international AI governance frameworks to reduce fragmentation, facilitate ease of adoption, and reduce industry costs in meeting multiple requirements. Insiders have noted that at the time when the AI Verify framework was released in 2022, NIST was in the midst of organizing public workgroups for the development of the RMF. From there, the IMDA and NIST began to work together, with a common goal of jointly developing the Crosswalk to meet different industry requirements.
Understanding the methodology of the Crosswalk
Under the Crosswalk, AI Verify’s testable criteria and processes are mapped to the AI RMF’s categories within the Govern, Map, Measure and Manage functions. Specifically, the Crosswalk first lists the individual categories and subcategories under the aforementioned four functions. As these 4 core functions address individual governance/trustworthiness characteristics (such as safety, accountability and transparency, explainability and fairness) collectively, the second column of the Crosswalk – which denotes the AI Verify Testing Framework – sets out the individual principle, testable criteria, and process and/or technical test that correlates to the relevant core function under the AI RMF.
A point worth noting is that the mapping is not “one-to-one”; each NIST AI RMF category may have multiple equivalents. Thus, for instance, AI Verify’s Process 9.1.1 for Accountability (indicated in the Crosswalk as “Accountability 9.1.1”) appears for both “Govern 4” and “Govern 5” under the AI RMF. This is to reflect the differences in nature of both documents – while the AI RMF is a risk management framework for the development and use of AI, AI Verify is a testing framework to assess the performance of an AI system and the practices associated with the development and use of this system. To achieve this mapping, the IMDA and NIST have had to compare both frameworks at a granular level – down to individual elements within the AI Verify Testing Framework – to achieve alignment. This can be seen from the Annex below, which sets out for comparison the “crosswalked” elements, as well as identifies the individual testable criteria and processes in the AI Verify Testing Framework.
Other aspects of understanding the Crosswalk document are set out below (in a Q&A format):
Does the Crosswalk only cover process-based checks or are technical tests within AI Verify (e.g. on fairness and robustness) also covered?
No. These have technically not been mapped. Technical tests on areas like explainability and robustness are performed by the testing algorithm within the AI Verify Testing Toolkit. Given that the AI RMF does not have a technical testing toolkit, there is technically no equivalent in the AI RMF to map to. However, technical tests provided by AI Verify may still be used to help organizations meet outcomes under the AI RMF. For instance, under AI RMF Measure 2.9, a suggested action is to “Explain systems using a variety of methods, e.g. visualizations, model extraction, feature importance, and others.” In this regard, explainability testing under AI Verify provides information on feature performance. To use another example, under AI RMF Measure 2.11, one of the suggested actions is to “Use context-specific fairness metrics to examine how system performance varies across groups, within groups, and/or for intersecting groups. Metrics may include statistical parity, error-rate equality, statistical parity difference, equal opportunity difference, average absolute odds difference, standardized mean difference, percentage point differences.” Accordingly, AI Verify’s fairness testing allows system deployers to test the fairness of their models according to these fairness metrics.
Across the Crosswalk, references are made to criteria under AI Verify (e.g. 1.1, 8.3, etc.), while in others, references are made to specific processes (e.g. 3.2.1, 4.3.1, etc.). What is the difference?
Certain testable criteria under AI Verify only contain one testing process, whereas, in others, testable criteria contain multiple testing processes. In the latter case, where only one of these processes has been mapped to the AI RMF, the Crosswalk indicates and maps only that specific testing process.
Where the Crosswalk refers to entire sections of testable criteria (e.g. “7.1 – 7.4”), are organizations expected to adopt all processes under the criteria?
No. Where the Crosswalk refers to entire sections of AI Verify’s Testing Framework, this indicates that all corresponding testing processes in those sections are relevant to achieving the relevant outcome under the AI RMF.
The category of “organizational considerations” (Criteria 12) currently cannot be found in the AI Verify. What does this mean?
This is currently being incorporated into the latest version of the AI Verify testing tool.
The Crosswalk shows that practical international cooperation in AI governance and regulation is possible
The global picture on AI regulation and governance is shifting rapidly. Since the burst of activity around the development of AI ethical principles and frameworks in the late 2010s, the landscape is becoming increasingly complex.
It is now defined within the broad strokes of the development of AI-specific regulation (in the form of legislation, such as the proposed EU AI Act, Canada’s AI and Data Act or Brazil’s AI Bill), the negotiation of an international Treaty on AI under the aegis of the Council of Europe, executive action putting the onus on government bodies when contracting AI systems (with President’s Biden Executive Order as chief example), the provision of AI-specific governance frameworks as self-regulation, and guidance by regulators (such as Data Protection Authorities issuing guidance on how providers and deployers of AI systems can rely on personal data respecting data protection laws). This varied landscape leaves little room for a coherent global approach to govern a quintessentially borderless technology.
In this context, the Crosswalk as a government-to-government effort shows that it is possible to find a common language between prima facie different self-regulatory AI governance frameworks, paving the way to interoperability or a cross-border interchangeable use of frameworks. Its practical relevance for organizations active both in the US and Singapore cannot be overstated.
The Crosswalk also provides a model for future crosswalks or similar mapping initiatives that will support a more coherent approach to AI governance across borders, potentially opening the path for more instances of meaningful and practical international cooperation in this space.
As an increasing number of organizations utilize artificial intelligence (“AI”) in their patient-facing services, health organizations are seizing the opportunity to take advantage of the new wave of AI-powered tools. Policymakers, from United States (“U.S.”) government agencies to the WhiteHouse, have taken heed of this trend, leading to a flurry of agency actions impacting the intersection of health and AI, from enforcement actions and binding rules to advisory options and other, less formal guidance. The result has been a rapidly changing regulatory environment for health organizations deploying artificial intelligence. Below are five key lessons from these actions for organizations, advocates, and other stakeholders seeking to ensure that AI-driven health services are developed and deployed in a lawful and trustworthy manner.
Lesson 1: AI potential in healthcare has evolved exponentially
While AI has been a part of healthcare conversations for decades, recent technological developments have seen exponential growth in potential applications across healthcare professionals and specialties requiring response and regulation of use and application of AI in healthcare.
In 2023, OCR stated that HIPAA-regulated entities should analyze AI tools as they do other novel technologies; organizations should “determine the potential risks and vulnerabilities to electronic protected health information before adding any new technology into their organization.” While not a broad endorsement of health AI, OCR’s statement suggests that AI has a place in the regulated healthcare sector.
The Food and Drug Administration (“FDA”) has taken an even more optimistic approach toward the use of AI. Also an agency within HHS, the FDA is responsible for ensuring the safety, efficacy, and quality of various pharmacological and medical products used in clinical health treatments and monitoring. In 2023, the FDA published a discussion paper intended to facilitate discussion with stakeholders on the use of AI in drug development. Drug discovery is the complex process of identifying and developing new medications or drugs to treat medical conditions and diseases. Before drugs can be marketed to the public for patient use, they must go through multiple stages of research, testing, and development. This entire process can take around 10 to 15 years, or sometimes longer. According to the discussion paper, the FDA strives to “facilitate innovation while safeguarding public health” and plans to develop a “flexible risk-based regulatory framework that promotes innovation and protects patient safety.”
Lesson 2: Different uses of data may implicate different regulatory structures
While there can be uncertainty regarding whether particular data, such IP address data collected by a consumer-facing website, is covered by HIPAA, HHS and the Federal Trade Commission (“FTC”) have made clear that they are working together to ensure organizations protect sensitive health information. In particular, failure to establish proper agreements or safeguards between covered entities and AI vendors can constitute a violation of the HIPAA Privacy Rule when patient health information is shared without patient consent for purposes other than treatment, payment, and healthcare operations.
However, some data collected by HIPAA-covered entities may not be classified as protected health information (“PHI”) and could be permissibly shared outside HIPAA’s regulatory scope. Examples include data collected by healthcare scheduling apps, wearables devices, and health IoT devices. In these circumstances, the FTC could exercise oversight. The FTC is increasingly focused on enforcement actions involving health privacy and potential bias and has historically enforced laws prohibiting bias and discrimination, including the Fair Credit Reporting Act (“FCRA”) and the Equal Credit Opportunity Act (“ECOA”). In 2021, the FTC underscored the importance of ensuring that AI tools avoid discrimination and called for AI to be used “truthfully, fairly, and equitably,” recommending that AI should do “more good than harm” to avoid violating the FTC’s “unfairness” prong of Section 5 of the FTC Act.
Lesson 3: What’s (guidance in the) past is prologue (to enforcement)
While guidance may not always be a precursor to enforcement, it is a good indicator of an agency’s priorities. For instance, in late 2021, the FTC issued a statement on the Health Breach Notification Rule, followed by two posts in January 2022 (1, 2). The FTC then applied the Health Breach Notification Rule (HBNR) for the first and second time in 2023 enforcement actions.
The FTC has recently honed in on both the health industry and AI. Agency officials published ten blog posts covering AI topics in 2023 alone, including an article instructing businesses to ensure the accuracy and verifiability of advertising around AI in products. In April 2023, the FTC issued a joint statement with the Department of Justice (DOJ), the Consumer Financial Protection Bureau (CFPB), and the Equal Employment Opportunity Commission (EEOC) expressing its intent to prioritize enforcement against discrimination and bias in automated decision-making systems.
The agency has separately been working on enforcement in the health sector, applying the unfairness prong of its authority to cases where the Commission has found that a company’s privacy practices substantially injured consumers in a manner that did not outweigh the countervailing benefits. This focus resulted in major settlements against health companies, including GoodRx and BetterHelp, where the combined total fine neared $10 million. In July, the FTC published a blog post summarizing lessons from its recent enforcement actions in the health sector, underscoring that “health privacy is a top priority” for the agency.
Lesson 4: Responsibility is the name of the game
Responsible use has been the key concept for policymakers looking to be proactive in establishing positive norms for the use of AI in the healthcare arena. In 2022, the White House Office of Science and Technology Policy (OSTP) published the Blueprint for an AI Bill of Rights (“Blueprint”) to support the development of policies and practices that protect and promote civil rights in the development, deployment, and governance of automated systems. In highlighting AI in the health sector, the Blueprint hopes to set up federal agencies and offices to serve as responsible stewards of AI use for the nation. In 2023, the OSTP also updated the National AI Research and Development (R&D) Plan to advance the deployment of responsible AI, which is likely to influence health research. The Plan is intended to facilitate the study and development of AI while also maintaining privacy and security and preventing inequity.
Expanding on the Blueprint, on October 30, 2023, the Biden Administration released its Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence (“EO”). The EO aims to establish new standards for the responsible use, development, and procurement of AI systems across the federal government. Among other directives, the EO directs the Secretary of HHS to establish an “HHS AI Taskforce” in order to create a strategic plan for the responsible use and deployment of AI in the healthcare context. The EO specifies that this strategic plan must establish principles to guide the use of AI as part of the delivery of healthcare, assess the safety and performance of AI systems in the healthcare context, and integrate equity principles and privacy, security and safety standards into the development of healthcare AI systems.
The EO also directs the HHS Secretary to create an AI Safety program to centrally track, catalog, and analyze clinical errors produced by the use of AI in healthcare environments; create and circulate informal guidance to advise on how to avoid these harms from recurring; and develop a strategy for regulating the use of AI and AI-tools for drug-development. The Fact Sheet circulated prior to the release of the EO emphasizes that, “irresponsible uses of AI can lead to and deepen discrimination, bias, and other abuses in justice, healthcare, and housing” and discusses expanded grants for AI research in “vital areas,” including healthcare.
On November 1, 2023, the Office of Management and Budget (“OBM”) released for public comment a draft policy on “Advancing Governance, Innovation, and Risk Management for Agency Use of Artificial Intelligence,” intended to help implement the AI EO. The OMB guidance, which would govern federal agencies as well as their contractors, would create special requirements for what it deems “rights-impacting” AI, a designation that would encompass AI that “control[s] or meaningfully influence[s]” the outcomes of health and health insurance-related decision-making. These include the requirements for AI impact assessments, testing against real-world conditions, independent evaluation, ongoing monitoring, human training “human in the loop” decision-making, and notice and documentation.
Finally, the National Institute of Standards and Technology (“NIST”) also focused on responsible AI in 2023 with the release of the Artificial Intelligence Risk Management Framework (“AI RMF”). The AI RMF is meant to serve as a “resource to the organizations designing, developing, deploying, or using AI systems to help manage the many risks of AI and promote trustworthy and responsible development and use of AI systems.” The AI RMF provides concrete examples on how to frame risks in various contexts, such as potential harm to people, organizations, or an ecosystem. In addition, prior NIST risk management frameworks have provided the basis for legislative and regulatory models, meaning it may have increased importance for regulated entities in the future.
Lesson 5: Focus and keep eyes on the road ahead
AI regulation is a moving target with significant developments expected in the coming years. For instance, OSTP’s Blueprint for an AI Bill of Rights has already been used to inform state policymakers, with legislators both highlighting and incorporating its requirements into legislative proposals. The Blueprints’ five outlined principles aim to: (i) ensure safety and effectiveness; (ii) safeguard against discrimination; (iii) uphold data privacy; (iv) provide notice and explanation; and (v) enable human review or control. These principles are likely to continue to appear and to inform future health-related AI legislation.
In 2022, the FDA’s Center for Devices and Radiological Health (CDRH) released “Clinical Decision Support Software Guidance for Industry and Food and Drug Administration Staff,” which recommends that certain AI tools be regulated by the FDA under its authority to oversee clinical decision support software. Elsewhere, the FDA has noted that its traditional pathways for medical device regulations were not designed to be applied to AI and that the agency is looking to update its current processes. In 2021, CDRH issued a draft “Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan”, which introduces a framework to manage risks to patients in a controlled manner. The Action Plan includes specific instruction on data management, including a commitment to transparency on how AI technologies interact with people, ongoing performance monitoring, and updates to the FDA on any changes made to the software as a medical device. Manufacturers of medical devices can expect the FDA to play a vital role in the regulation of AI in certain medical devices and drug discovery.
Conclusion
The legislative and regulatory environment governing AI in the U.S. is actively evolving, with the regulation of the healthcare industry emerging as a key priority for regulators across the federal government. Although the implementation and development of AI into healthcare activities may provide significant benefits, organizations must recognize and mitigate privacy, discrimination, and other risks associated with its use. AI developers are calling for the regulation of AI to reduce existential risks and prevent significant global harm, which may help create clearer standards and expectations for AI developers and developers navigating the resources coming from federal agencies. By prioritizing the development and deployment of safe and trustworthy AI systems, as well as following federal guidance and standards for privacy and security, the healthcare industry can harness the power of AI to ethically and responsibly improve patient care, outcomes, and overall well-being.
Regu(AI)ting Health: Lessons for Navigating the Complex Code of AI and Healthcare Regulations
As an increasing number of organizations utilize artificial intelligence (“AI”) in their patient-facing services, health organizations are seizing the opportunity to take advantage of the new wave of AI-powered tools. Policymakers, from United States (“U.S.”) government agencies to the WhiteHouse, have taken heed of this trend, leading to a flurry of agency actions impacting the intersection of health and AI, from enforcement actions and binding rules to advisory options and other, less formal guidance. The result has been a rapidly changing regulatory environment for health organizations deploying artificial intelligence. Below are five key lessons from these actions for organizations, advocates, and other stakeholders seeking to ensure that AI-driven health services are developed and deployed in a lawful and trustworthy manner.
Lesson 1: AI potential in healthcare has evolved exponentially
While AI has been a part of healthcare conversations for decades, recent technological developments have seen exponential growth in potential applications across healthcare professionals and specialties requiring response and regulation of use and application of AI in healthcare.
In 2023, OCR stated that HIPAA-regulated entities should analyze AI tools as they do other novel technologies; organizations should “determine the potential risks and vulnerabilities to electronic protected health information before adding any new technology into their organization.” While not a broad endorsement of health AI, OCR’s statement suggests that AI has a place in the regulated healthcare sector.
The Food and Drug Administration (“FDA”) has taken an even more optimistic approach toward the use of AI. Also an agency within HHS, the FDA is responsible for ensuring the safety, efficacy, and quality of various pharmacological and medical products used in clinical health treatments and monitoring. In 2023, the FDA published a discussion paper intended to facilitate discussion with stakeholders on the use of AI in drug development. Drug discovery is the complex process of identifying and developing new medications or drugs to treat medical conditions and diseases. Before drugs can be marketed to the public for patient use, they must go through multiple stages of research, testing, and development. This entire process can take around 10 to 15 years, or sometimes longer. According to the discussion paper, the FDA strives to “facilitate innovation while safeguarding public health” and plans to develop a “flexible risk-based regulatory framework that promotes innovation and protects patient safety.”
Lesson 2: Different uses of data may implicate different regulatory structures
While there can be uncertainty regarding whether particular data, such IP address data collected by a consumer-facing website, is covered by HIPAA, HHS and the Federal Trade Commission (“FTC”) have made clear that they are working together to ensure organizations protect sensitive health information. In particular, failure to establish proper agreements or safeguards between covered entities and AI vendors can constitute a violation of the HIPAA Privacy Rule when patient health information is shared without patient consent for purposes other than treatment, payment, and healthcare operations.
However, some data collected by HIPAA-covered entities may not be classified as protected health information (“PHI”) and could be permissibly shared outside HIPAA’s regulatory scope. Examples include data collected by healthcare scheduling apps, wearables devices, and health IoT devices. In these circumstances, the FTC could exercise oversight. The FTC is increasingly focused on enforcement actions involving health privacy and potential bias and has historically enforced laws prohibiting bias and discrimination, including the Fair Credit Reporting Act (“FCRA”) and the Equal Credit Opportunity Act (“ECOA”). In 2021, the FTC underscored the importance of ensuring that AI tools avoid discrimination and called for AI to be used “truthfully, fairly, and equitably,” recommending that AI should do “more good than harm” to avoid violating the FTC’s “unfairness” prong of Section 5 of the FTC Act.
Lesson 3: What’s (guidance in the) past is prologue (to enforcement)
While guidance may not always be a precursor to enforcement, it is a good indicator of an agency’s priorities. For instance, in late 2021, the FTC issued a statement on the Health Breach Notification Rule, followed by two posts in January 2022 (1, 2). The FTC then applied the Health Breach Notification Rule (HBNR) for the first and second time in 2023 enforcement actions.
The FTC has recently honed in on both the health industry and AI. Agency officials published ten blog posts covering AI topics in 2023 alone, including an article instructing businesses to ensure the accuracy and verifiability of advertising around AI in products. In April 2023, the FTC issued a joint statement with the Department of Justice (DOJ), the Consumer Financial Protection Bureau (CFPB), and the Equal Employment Opportunity Commission (EEOC) expressing its intent to prioritize enforcement against discrimination and bias in automated decision-making systems.
The agency has separately been working on enforcement in the health sector, applying the unfairness prong of its authority to cases where the Commission has found that a company’s privacy practices substantially injured consumers in a manner that did not outweigh the countervailing benefits. This focus resulted in major settlements against health companies, including GoodRx and BetterHelp, where the combined total fine neared $10 million. In July, the FTC published a blog post summarizing lessons from its recent enforcement actions in the health sector, underscoring that “health privacy is a top priority” for the agency.
Lesson 4: Responsibility is the name of the game
Responsible use has been the key concept for policymakers looking to be proactive in establishing positive norms for the use of AI in the healthcare arena. In 2022, the White House Office of Science and Technology Policy (OSTP) published the Blueprint for an AI Bill of Rights (“Blueprint”) to support the development of policies and practices that protect and promote civil rights in the development, deployment, and governance of automated systems. In highlighting AI in the health sector, the Blueprint hopes to set up federal agencies and offices to serve as responsible stewards of AI use for the nation. In 2023, the OSTP also updated the National AI Research and Development (R&D) Plan to advance the deployment of responsible AI, which is likely to influence health research. The Plan is intended to facilitate the study and development of AI while also maintaining privacy and security and preventing inequity.
Expanding on the Blueprint, on October 30, 2023, the Biden Administration released its Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence (“EO”). The EO aims to establish new standards for the responsible use, development, and procurement of AI systems across the federal government. Among other directives, the EO directs the Secretary of HHS to establish an “HHS AI Taskforce” in order to create a strategic plan for the responsible use and deployment of AI in the healthcare context. The EO specifies that this strategic plan must establish principles to guide the use of AI as part of the delivery of healthcare, assess the safety and performance of AI systems in the healthcare context, and integrate equity principles and privacy, security and safety standards into the development of healthcare AI systems.
The EO also directs the HHS Secretary to create an AI Safety program to centrally track, catalog, and analyze clinical errors produced by the use of AI in healthcare environments; create and circulate informal guidance to advise on how to avoid these harms from recurring; and develop a strategy for regulating the use of AI and AI-tools for drug-development. The Fact Sheet circulated prior to the release of the EO emphasizes that, “irresponsible uses of AI can lead to and deepen discrimination, bias, and other abuses in justice, healthcare, and housing” and discusses expanded grants for AI research in “vital areas,” including healthcare.
On November 1, 2023, the Office of Management and Budget (“OBM”) released for public comment a draft policy on “Advancing Governance, Innovation, and Risk Management for Agency Use of Artificial Intelligence,” intended to help implement the AI EO. The OMB guidance, which would govern federal agencies as well as their contractors, would create special requirements for what it deems “rights-impacting” AI, a designation that would encompass AI that “control[s] or meaningfully influence[s]” the outcomes of health and health insurance-related decision-making. These include the requirements for AI impact assessments, testing against real-world conditions, independent evaluation, ongoing monitoring, human training “human in the loop” decision-making, and notice and documentation.
Finally, the National Institute of Standards and Technology (“NIST”) also focused on responsible AI in 2023 with the release of the Artificial Intelligence Risk Management Framework (“AI RMF”). The AI RMF is meant to serve as a “resource to the organizations designing, developing, deploying, or using AI systems to help manage the many risks of AI and promote trustworthy and responsible development and use of AI systems.” The AI RMF provides concrete examples on how to frame risks in various contexts, such as potential harm to people, organizations, or an ecosystem. In addition, prior NIST risk management frameworks have provided the basis for legislative and regulatory models, meaning it may have increased importance for regulated entities in the future.
Lesson 5: Focus and keep eyes on the road ahead
AI regulation is a moving target with significant developments expected in the coming years. For instance, OSTP’s Blueprint for an AI Bill of Rights has already been used to inform state policymakers, with legislators both highlighting and incorporating its requirements into legislative proposals. The Blueprints’ five outlined principles aim to: (i) ensure safety and effectiveness; (ii) safeguard against discrimination; (iii) uphold data privacy; (iv) provide notice and explanation; and (v) enable human review or control. These principles are likely to continue to appear and to inform future health-related AI legislation.
In 2022, the FDA’s Center for Devices and Radiological Health (CDRH) released “Clinical Decision Support Software Guidance for Industry and Food and Drug Administration Staff,” which recommends that certain AI tools be regulated by the FDA under its authority to oversee clinical decision support software. Elsewhere, the FDA has noted that its traditional pathways for medical device regulations were not designed to be applied to AI and that the agency is looking to update its current processes. In 2021, CDRH issued a draft “Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan”, which introduces a framework to manage risks to patients in a controlled manner. The Action Plan includes specific instruction on data management, including a commitment to transparency on how AI technologies interact with people, ongoing performance monitoring, and updates to the FDA on any changes made to the software as a medical device. Manufacturers of medical devices can expect the FDA to play a vital role in the regulation of AI in certain medical devices and drug discovery.
Conclusion
The legislative and regulatory environment governing AI in the U.S. is actively evolving, with the regulation of the healthcare industry emerging as a key priority for regulators across the federal government. Although the implementation and development of AI into healthcare activities may provide significant benefits, organizations must recognize and mitigate privacy, discrimination, and other risks associated with its use. AI developers are calling for the regulation of AI to reduce existential risks and prevent significant global harm, which may help create clearer standards and expectations for AI developers and developers navigating the resources coming from federal agencies. By prioritizing the development and deployment of safe and trustworthy AI systems, as well as following federal guidance and standards for privacy and security, the healthcare industry can harness the power of AI to ethically and responsibly improve patient care, outcomes, and overall well-being.
FPF Files Comments with the Consumer Financial Protection Bureau Regarding Personal Financial Data Rights
On December 21st, 2023, the Future of Privacy Forum filed comments with the Consumer Financial Protection Bureau (CFPB) in response to the notice of proposed rulemaking (NPRM) regarding personal financial data rights. FPF’s comments focus on promoting privacy as a core tenet in the U.S. open banking ecosystem in order to protect individuals’ personal information while enhancing user trust.
This NPRM is the latest milestone in the Bureau’s multi-year effort to create a regulatory framework for open banking in the U.S. using its Section 1033 authority. Section 1033 was passed as part of the Consumer Financial Protection Act (CFPA) of 2010 and it governs access to a person’s data held by a consumer financial services provider. The CFPB’s proposed rule requires data providers, such as banks, card issuers, and digital wallets, to share certain kinds of consumer financial data (e.g., transactions information and account balance) with authorized third parties at the consumer’s request. As the CFPB sets out, “[t]his proposed rule aims to . . . push for greater efficiency and reliability of data access across the industry to reduce industry costs, facilitate greater competition, and support the development of beneficial products and services.”1
In our submission, FPF provides several recommendations to the CFPB, including:
Encouraging the development of industry standards for third party privacy rules and data provider denials of access requests;
Supporting an opt-in standard and use of de-identified data, while providing an approach for high-risk uses;
Clarifying an approach to address ‘dark patterns’ to discourage consumer manipulation;
Strengthening the phase-out of and directly prohibiting third parties from engaging in screen scraping of data from online consumer accounts; and
Harmonizing various privacy rules that result in numerous and different notices and choices.
1Required Rulemaking on Personal Financial Data Rights, 88 Fed. Reg. 74796, 74843 (Oct. 31, 2023).
Understanding Body-Related Data Practices and Ensuring Legal Compliance in Immersive Technologies
Organizations are increasingly incorporating immersive technologies like extended reality (XR) and virtual worlds into their products and services, blurring the boundaries between the physical and digital worlds. Immersive technologies hold the potential to transform the way people learn, work, play, travel, and take care of their health, but may create new privacy risks as well. Many of these technologies rely on large amounts of data about individuals’ bodies, without which they would be less immersive, and in some cases couldn’t function at all.
Body-related data raises particular privacy risks, and leading organizations in the immersive technology space are adopting risk-based approaches for handling this type of data. Focusing on the risks—to the organization and to those impacted by the organization’s data practices—makes it easier not only to comply with the law but also to ensure more ethical data practices.
There are concrete steps organizations can take to ensure that body-related data is handled safely and responsibly. As part of their data protection strategies, organizations should:
Understand their data practices: mapping these practices, specifying their purposes, and identifying all relevant stakeholders.
Evaluate their legal obligations: analyzing existing legal obligations, as well as how they may change in the near future based on emerging trends.
Identify risks to individuals, communities, and society: cataloging the features of their data and data practices that create greater risks.
Implement best practices: operationalizing technical, organizational, and legal safeguards to prevent or mitigate the identified risks.
To guide organizations as they develop their body-related data practices, the Future of Privacy Forum created the Risk Framework for Body-Related Data in Immersive Technologies. This framework serves as a straightforward, practical guide for organizations to analyze the unique risks associated with body-related data, particularly in immersive environments, and to institute data practices that are capable of earning the public’s trust. Developed in consultation with privacy experts and grounded in the experiences of organizations working in the immersive technology space, the framework is also useful for organizations that handle body-related data in other contexts as well. This post will explore the first two stages of the risk framework: understanding an organization’s data practices, and evaluating legal obligations to ensure compliance.
I. Understanding how organizations handle personal data
The first step to handling body-related data is for organizations to understand how they handle personal data. Doing so will help them communicate these practices to their users, regulators, the general public, and other relevant stakeholders. Developing a comprehensive understanding of an organization’s data practices is also critical for identifying potential privacy risks and implementing best practices to mitigate them. Organizations should bring together experts from different teams to document how they collect, use, and onwardly transfer body-related data. The following steps help organizations conduct these processes effectively.
Create data maps of data practices, particularly in regard to body-related data
Data mapping is the process of creating an inventory of all the personal data an organization handles, including how it’s used, to whom it is transferred, and how long it is kept. While tools exist to assist organizations with data mapping, it is helpful to assign a designated person within an organization, such as a chief privacy officer or data protection officer, to be responsible for completing the data map. Data mapping also helps organizations in certain jurisdictions maintain compliance with legal obligations related to data practice documentation. Certain kinds of body-related data—such as data about people’s faces, hands, voices, and body movements—will be particularly relevant in immersive environments, and organizations operating in this space should pay special attention to them.
Document the purpose of each data practice
In order to determine which data practices are necessary, and which may be adjusted, organizations must be able to specify what goal or purpose each practice serves. Organizations might engage in a particular data practice for a variety of purposes: enabling relevant features or products, improving a product’s technical performance, facilitating targeted advertising, or customizing a user’s experience, to name a few. This documentation will help inform an organization’s evaluations of its privacy risks and legal obligations, and generate buy-in from business stakeholders within the organization by linking their interests to privacy compliance.
Identify all relevant stakeholders impacted by data practices
Evaluating an organization’s legal obligations and privacy risks requires key organizational leaders to understand which stakeholders are implicated—both as partners in data transfer agreements and as people impacted by the organization’s data practices. Organizations must understand the kinds of entities with whom they are transferring data, and who specifically within these third parties are handling the data. They should also understand who is impacted by their data practices, including data subjects or users as well as bystanders whose data may also be implicated. Special attention should be paid to individuals and communities whose data may raise additional legal or ethical considerations, such as children and teens, and people from historically marginalized or vulnerable communities.
II. Analyzing relevant legal frameworks and ensuring compliance
Once an organization has established a thorough understanding of its data practices, the next step in preparing to handle body-related data is to evaluate whether the enumerated data practices are in compliance with the law. Collecting, using, or transferring body-related data may implicate a number of issues under current U.S. privacy law. However, most existing regulations were not drafted with immersive technologies in mind. It can therefore sometimes be unclear how these rules apply to immersive technologies, and an organization’s obligations will depend on where it operates, what kind of data it handles and why, and the size and nature of the organization, among other factors.
To understand and comply with all existing obligations, organizations need to know the scope of data types covered by current laws, the requirements and rights that attach to them, and the unique considerations that may apply in immersive spaces and in regard to body-related data. Existing privacy laws in the U.S. apply, depending on jurisdiction, to body-related data involving personal, biometric, sensitive, health, and publicly available data, and organizations should pay special attention to the specific requirements under such laws.
Organizations dealing with these data types have certain legal obligations, including:
Granting users access, correction, and deletion rights
Providing opportunities to provide consent
Avoiding “dark patterns” and manipulative or deceptive design
Being transparent and providing notice to users
Minimizing data collection and retention when necessary
Conduct data protection impact assessments (DPIAs)
Institute protections for kids and teens
2023 proved to be a significant year for state privacy laws, and new legislation and regulations will continue to impact the data privacy legal landscape. Organizations should keep an eye on the major areas for emerging legislation such as youth privacy and safety, as well as consumer health data. They should also monitor how emerging litigation impacts current requirements through interpreting current legislative language.
For more information on what organizations can do to ensure they handle body-related data safely and responsibly, read for the next post in our series, focusing on identifying risks and implementing best practices. For a comprehensive guide to body-related data practices in immersive technologies, see FPF’s Risk Framework for Body-Related Data in Immersive Technologies.
FPF in 2023: A Year in Review
As 2023 comes to an end, we want to reflect on a year that saw the Future of Privacy Forum (FPF) continue to expand its presence globally and domestically while organizing engaging events, publishing thought-provoking analysis, providing the latest expert updates, and more. FPF continues to convene industry experts, academics, consumer advocates, and other experts to explore the challenging issues in the data protection and privacy field.
The AI Impact
2023 was the year of AI. We saw AI technologies catapulted into the mainstream with Generative AI tools such as ChatGPT, Google Bard, and others. AI continues to have countries worldwide working to regulate the technology and companies scrambling to figure out how to navigate AI amongst their employees and their products and services.
To respond to the demand for understanding in AI, FPF worked with stakeholders on best practices, provided in-depth training on AI-related topics, and discussed the evolving impact of this technology with many of you at roundtable discussions, expert panels, and more.
Here are some of FPF’s biggest AI moments of 2023:
Hosted our first-ever Japan Privacy Symposium where Data Protection and Privacy Commissioners of the G7 DPAs discussed their approaches to regulating AI.
Discussed alternative solutions for processing of (personal) data with Machine Learning at CPDP Brussels and generative AI systems in Asia-Pacific during Singapore’s PDP Week.
Participated in a Capitol Hill briefing hosted by the Wilson Center and Seed AI in conjunction with the Congressional Artificial Intelligence Caucus “AI Primer: AI in the Workplace,” highlighting FPF’s Best Practices.
Provided testimony on the responsible use and adoption of AI technologies in New York City classrooms.
Published insightful op-eds in WIRED discussing the intersection of AI and immersive technologies and The Hill on generative AI and elections.
Held stakeholder workshops on the current regulation of generative AI throughout the APAC region.
Organized a session at the Global Privacy Assembly on the use of public information for LLM training.
Relaunched the FPF Training program, providing in-depth expert sessions on topics such as the EU AI Act, the fundamentals of AI and machine learning, and more.
Continuing FPF’s Global Reach
In 2023, FPF closely followed and advised upon significant developments in Asia, the European Union, Africa, and Latin America. We also discussed privacy and data protection with many of you at key conferences and events across the globe, including in Washington, DC, Brussels, Tokyo, Singapore, Bermuda, and Tel Aviv.
As India’s Digital Personal Data Protection Act sprinted through its final stages in August after several years of debates, postponements, and negotiations, FPF provided an in-depth, comprehensive explainer of its important aspects and key provisions, as well as discussed its extraterritorial effects in a LinkedIn Live conversation. The Act also focused on protections for the processing of personal data of children and introduced the concept of “verifiably safe” measures and, FPF in partnership with The Dialogue released a Brief containing a Catalog of Measures for “Verifiably Safe” Processing of Children’s Personal Data Under India’s Digital Personal Data Protection Act (DPDPA) 2023. In partnership with NASSCOM, FPF also hosted a webinar series on the consent regime under India’s new Digital Personal Data Protection Act of 2023.
FPF saw its presence in Asia continue to grow as the FPF Asia-Pacific office entered its third year. FPF and S&K Brussels hosted the first-ever Japan Privacy Symposium in Tokyo, providing insight into the regulatory priorities of the G7 DPAs and global thought leadership on the interaction of data protection and privacy laws with AI. During Singapore’s PDP Week, our Asia-Pacific team held a roundtable on the governance implications of generative AI systems, spoke at the Asia Privacy Forum, and hosted an in-person training on the EU AI Act.
FPF remains consistently active in the European Union, with several engaging events bringing together the European data privacy community and numerous thought-provoking blogs, reports, and analyses published in 2023. FPF launched its in-depth report on enforcement of the EU’s GDPR Data Protection by Design and by Default obligations and hosted our 7th Annual Brussels Privacy Symposium with the Brussels Privacy Hub of Vrije Universiteit Brussel, which included opening remarks by European Commissioner for Justice Didier Reynders and European Data Protection Supervisor Wojciech Wiewiórowski. We also analyzed the regulatory strategies of European DPAs for 2023 and beyond in our continuing series.
In addition, our global experts provided analysis on privacy and data protection developments in Vietnam, Nigeria, Australia, Tanzania, and the African Union and published an overview comparing three regional model contractual frameworks for cross-border data transfers.
U.S. Legislative Activity
In 2023, FPF played a key role in informing regulatory agencies and state legislatures on privacy in various emerging technologies, such as AI. Our experts testified before state legislatures, provided informative analysis, submitted regulatory comments, and more.
We provided recommendations and filed comments with the:
U.S. Department of Health and Human Services Office for Civil Rightsregarding the Notice of Proposed Rulemaking on extending additional protections to reproductive health care data under the Health Insurance Portability and Accountability Act.
U.S. Federal Trade Commissionregarding the Notice of Proposed Rulemaking to clarify the scope and application of the Health Breach Notification Rule, and again regarding the use of “Privacy-Protective Facial Age Estimation” as a potential mechanism for verifiable parental consent under the Children’s Online Privacy Protection Act Rule.
California Privacy Protection Agency to inform the Agency’s rulemaking to implement the California Privacy Rights Act amendments to the California Consumer Privacy Act’s provisions on cybersecurity audits, risk assessments, and automated decision-making.
National Telecommunications and Information Administration in response to their request for comment on privacy, equity, and civil rights, and again in response to their request for comment on Kids Online Health and Safety as part of the Biden-Harris Administration’s Interagency Task Force on Kids Online Health & Safety.
Consumer Financial Protection Bureau in response to their request for comment regarding data portability for financial products and services, and again in response to their Request for Information (RFI) Regarding Data Brokers and Other Business Practices Involving the Collection and Sale of Consumer Information.
2023 also saw developments in various U.S. state commercial privacy laws. We found that the number of state laws increased from five to twelve (or, arguably, thirteen), and in response, provided timely analysis in Iowa, Indiana, Montana, Tennessee, Florida, Texas, Connecticut, Oregon, Utah, and Delaware. In addition, Washington and Nevada became the first to pass broad-based consumer health data privacy legislation. Earlier this month, our Director for U.S. Legislation Keir Lamont took a look ahead at the state privacy landscape in 2024.
For the 13th year, FPF recognized leading privacy research and analytical work with the Privacy Papers for Policymakers Award held on Capitol Hill. The winners spoke about their research in front of an audience of academic, industry, and policy professionals in the field. The event featured keynote speaker FTC Commissioner Alvaro Bedoya.
Youth & Education Privacy
Federal and state policymakers turned to the protection of children online, with President Biden notably mentioning it for a second year in a row during this year’s State of the Union address.
In partnership with LGBT Tech, we outlined recommendations for schools and districts to balance inclusion and student safety in technology use. Our analysis builds on thorough research, including interviews with recent high school graduates who identify as LGBTQ+, to gather firsthand accounts of how student monitoring impacted their feelings of privacy and safety at school.
Over the summer, we published one of our popular infographics examining age assurance technologies. The infographic’s authors unpacked the risks and potential harms associated with attempting to discern someone’s age online and potential mitigation tools in this LinkedIn Live conversation.
Privacy by design for kids and teens also expanded globally in 2023. As policymakers, advocates, and companies grapple with the ever-changing landscape of youth privacy regulation, we hosted a well-attended webinar with a wide range of global experts discussing the current state of kids’ and teens’ privacy policy.
The Rise of Emerging Technologies, Examining the Open Banking Ecosystem, & Analysis on Research Data Sharing
As stakeholders became increasingly interested in immersive technologies, notably AR/VR/MR, we responded by releasing the Risk Framework for Body-Related Data in Immersive Technologies, whichassists organizations in safely and responsibly handling body-related data. Our team also held a series of webinars exploring the intersection of immersive technology with topics like AI, advertising, education, and more.
In March, we published an infographic breaking down the complex U.S. open banking ecosystem, supported by over a year of meetings and outreach with leaders in banking, credit management, financial data aggregators, and solution providers to comprehensively understand the developing industry of open banking, with the infographic’s authors discussing its privacy implications in a LinkedIn Live conversation.
In 2023, we continued to examine privacy and research data sharing by producing Data Sharing for Research: A Compendium of Case Studies, Analysis, and Recommendations, demonstrating how, for many organizations, data-sharing partnerships are transitioning from being considered an experimental business activity to an expected business competency. We also held the 3rd Annual Award for Research Data Stewardship, honoring representatives from Optum and the Mayo Clinic for their outstanding corporate-academic research data-sharing partnership. During this virtual event, we opened with a keynote address by U.S. Congresswoman Lori Trahan.
Bringing Together Leaders in Privacy and Data Protection
On a different track, FPF also built out a wide range of peer-to-peer meetings and calls for the senior executives working on data protection compliance issues. We hosted virtual meetings on key topics of interest on an every other month basis, smaller meetings for specific sector leaders, and in-person meetings in multiple cities.
The “Current State of Global Opt-Out Technology” event in March provided members with insights into regulator discussions, along with a vendor showcase featuring solution demonstrations.
Hosted 50+ in-person and virtual peer-to-peer meetings across the globe for intimate discussions among privacy executives focused on their top-of-mind issues.
Launched Privacy Metrics 2.0 to help advance industry OKRs and the underlying metrics, provide privacy leaders with the tools they need to have effective conversations with their boards, and provide useful information for ESG reporting and investor communications.
This is by no means a comprehensive list of all of FPF’s important and engaging work in 2023, but we hope it gives you a sense of our work’s impact on the privacy community and society at large. We believe our success is due to deep engagement with privacy experts in industry, academia, civil society, and government and our belief that collaborating across sectors and disciplines is needed to advance practical safeguards needed for data uses that benefit society. Keep updated on FPF’s work by subscribing to our monthly briefing and following us on LinkedIn, Twitter/X, and Instagram.
On behalf of the FPF team, we wish you a very Happy New Year and look forward to celebrating 15 years of FPF in 2024!
FPF Publishes New Report: A Conversation on Privacy, Safety, and Security in Australia: Themes and Takeaways
Australia’s Online Safety Act of 2021 (“Online Safety Act”) mandates the development of industry codes or standards to provide appropriate community safeguards with respect to certain online content, including child sexual exploitation material, pro-terror material, crime and violence material, and drug-related material. Through September 2023, the eSafety has registered six industry codes that cover: Social Media Services, App Distribution Services, Hosting Services, Internet Carriage Services, Equipment, and Internet Search Engine Services. In May 2023, however, the Commissioner rejected proposed codes for relevant electronic services (“RES”) and designated internet services (“DIS”) on account that they “do[] not provide appropriate community safeguards.” Under the Online Safety Act, the rejection of the RES and DIS codes by the Office of the eSafety Commissioner initiated a process in which the Commissioner drafted industry standards for these sectors. A draft of the industry standards was published on November 20, 2023, and is open for public comment until December 21, 2023.
For purposes of the FPF and meeting, participants were asked to assume the existence of industry standards that satisfies the Online Safety Act’s statutory requirements. As such, the goal was not to solicit arguments about any specific approach, but rather to provide an opportunity for experts to discuss underlying opportunities and challenges in regard to the creation of industry standards, particularly in regard to partially or entirely end-to-end encrypted services. While meeting participants were not in full agreement in regard to any specific point, there were many themes that came up multiple times within the conversation as well as areas of consensus on certain points, including:
Participants agreed broadly on the goals of the e-Safety Act and the mission of the e-Safety Commissioner
Several participants found deficits in the length and scope of the public consultation available throughout the process
Participants identified several potential benefits of an industry code beyond its intended scope
Participants broadly opposed any approach that would require otherwise encrypted messaging services to utilize content hashing and/or client-side scanning
Many participants discussed the need for unique treatment for different types of content based on distinctions in context
Participants flagged previous cases of mission drift in regard to certain legal authorities and warned of similar evolution
Participants flagged an important role for greater education, both for individuals as well as enforcers
Participants supported a broad public dialogue on effective responses and solutions
Participants identified a large number of unanswered questions in regard to the creation, implementation, and enforcement of industry codes that left much uncertainty
Australia has played a leadership role globally on issues related to Online Safety and is likely to continue to do so
Risk Framework for Body-Related Data in Immersive Technologies
Organizations building immersive technologies like extended reality and virtual worlds often rely on large amounts of data about individuals’ bodies and behaviors. While body-related data allows for new, positive applications in health, education, entertainment, and more, it can also raise privacy and safety risks. FPF’s risk-based framework helps organizations seeking to develop safe, responsible immersive technologies, guiding them through the process of documenting how and why they handle body-related data, complying with applicable laws, evaluating their privacy and safety risks, and implementing best practices.
While the framework is most useful for organizations working on technologies with immersive elements, it is also useful for organizations that handle body-related data in other contexts.
Stage 1: Understanding How Organizations Handle Personal Data
Understanding your organization’s data practices is the first step toward identifying potential privacy risks, ensuring legal compliance, and implementing relevant best practices to improve privacy and safety. It can also allow organizations to better communicate about those practices. To this end, organizations should:
Create data maps of their data practices, particularly in regard to body-related data types.
Document the purpose of each data practice.
Identify all relevant stakeholders impacted by data practices, including third-party recipients of personal data and data subjects.
Stage 2: Analyzing Relevant Legal Frameworks and Ensuring Compliance
Collecting, using, or transferring body-related data may implicate a number of current and emerging U.S. privacy laws. As such, organizations should:
Understand the individual rights and business obligations that apply under existing comprehensive and sectoral privacy laws.
Analyze how emerging legislation and regulations will impact body-based data practices.
Stage 3: Identifying and Assessing Risks to Individuals, Communities, and Society
Privacy harms may stem from particular types of data being used or handled in particular ways, or transferred to particular parties. In that regard, legal compliance may not be enough to mitigate risks, and organizations should:
1. Proactively identify and minimize the risks their data practices could pose to individuals, communities, and society. Factors that impact the risk of a data practice include:
Identifiability
Use for critical decisions
Sensitivity
Partners and third parties
Potential for inferences
Data retention
Data accuracy and bias
User expectations and understanding
2. Assess how fair, ethical, and responsible the organization’s data practices are based on the identified risks.
Stage 4: Implementing Relevant Best Practices
There are a number of legal, technical, and policy safeguards that can help organizations maintain statutory and regulatory compliance, minimize privacy risks, and ensure that immersive technologies are used fairly, ethically, and responsibly. Organizations should:
1. Implement best practices intentionally—adopted with consideration of an organization’s data practices and associated risks; comprehensively—touching all parts of the data lifecycle and addressing all relevant risks; and collaboratively—developed in consultation with multidisciplinary teams within an organization including stakeholders from legal, product, engineering, privacy, and trust and safety. Such practices include:
Data minimization
Local and on-device processing and storage
Purpose specification and limitation
Third party management
Meaningful notice and consent
Data integrity
User controls
Privacy-enhancing technologies (PETs)
2. Evaluate best practices in regard to one another, as part of a coherent strategy.
3. Assess best practices on an ongoing basis to ensure they remain effective.
Five Big Questions (and Zero Predictions) for the U.S. State Privacy Landscape in 2024
Entering 2024, the United States now stands alone as the sole G20 nation without a comprehensive, national framework governing the collection and use of personal data. With bipartisan efforts to enact federal privacy legislation once again languishing in Congress, state-level activity on privacy dramatically accelerated in 2023. As the dust from this year settles, we find that the number of states with ‘comprehensive’ commercial privacy laws swelled from five to twelve (or, arguably, thirteen), a new family of health-specific privacy laws emerged in Democratic-led states while Republican-led states increasingly adopted controversial age verification and parental consent laws, and state lawmakers took the first steps towards comprehensively regulating the development and use of Artificial Intelligence technologies.
While stakeholders are eager to know whether and how these 2023 trends will carry over into next year’s state legislative cycle, it is too early to make predictions with any confidence. So instead, this post explores five big questions about the state privacy landscape that will shape how 2024 legislative developments will impact the protection of personal information in the United States.
1. Will Any State Buck the Consensus Framework for ‘Comprehensive’ Privacy Protections?
Following the adoption of the California Consumer Privacy Act (CCPA) in 2018, many stakeholders expressed concern that U.S. states were poised to enact a deluge of divergent and conflicting state privacy laws, confusing individuals and placing onerous burdens on businesses for compliance. To date, the worst case scenarios for this dreaded “patchwork” have largely not come to pass. Instead, lawmakers outside California have repeatedly rejected the convoluted and ever-shifting CCPA approach in preference of iterating around the edges of the more streamlined Washington Privacy Act-framework. Alternative approaches like the ULC model bill or frameworks rooted in the federal American Data Privacy and Protection Act proposal have failed to gain any serious traction. Will this trend hold, or is any state positioned to upend the bipartisan consensus on privacy legislation and adopt an alternative regulatory framework that creates novel individual rights, covered entity obligations, or enforcement provisions?
Despite the overarching trend of regulatory convergence there are still meaningful differences between the post-California comprehensive state privacy laws. Notable new wrinkles adopted in the 2023 legislative sessions include the Texas requirement that even small businesses obtain consent to sell sensitive personal data, Oregon creating a right-to-know the specific third parties who receive personal data from covered entities, and Delaware extending certain protections for adolescents up to the age of seventeen. However, for the most part, the new class of comprehensive commercial privacy laws adhere to the same overarching framework, definitions, and core concepts, enabling regulated entities to build out of one-size-fits-most compliance strategies.
Next year, states wishing to enact protections for personal data held by businesses will have a clear blueprint with a bipartisan track record of success for doing so. However, the emerging inter-state consensus for privacy protection is not without its critics. In particular, some privacy advocacy groups have argued that the current laws place too much of the onus for protecting privacy on individuals rather than the businesses and nonprofits that are engaged in the collection, processing, and transfer of user data and have supported various models that would take a different approach.
Based on the 2023 lawmaking sessions, two states stand out as potential candidates to buck the Washington Privacy Act-paradigm by virtue of having unique privacy proposals previously clear a chamber in their state legislature. First is the Kentucky Consumer Data Protection Act (SB 15) from Senator Westerfield which passed the State Senate by a 32-2 vote in 2023. This bill included a GDPR-style ‘lawful basis’ requirement for the collection of personal data. Second, in New York State, Senator Thomas (who is now running for Congress) shepherded the New York Privacy Act (S 365) through the State Senate. The proposal included numerous distinct privacy rights and protections, particularly with respect to first-party online advertising. Could 2024 be the year that one or both of these proposals cross the finish line?
2. What will California do on Artificial Intelligence?
Recent advancements and public attention to Artificial Intelligence (AI) systems, particularly those with generative capabilities, have placed AI high on the agenda for policymakers at all levels of government. To be sure, automated decision making and profiling technologies have been in use in various forms for many years and are regulated by existing legal regimes both within and outside the privacy context. Nevertheless, lawmakers appear keen to explore new governance models that will allow the U.S. to unlock the social and economic benefits promised by AI while minimizing risks to both individuals and communities. As has been the case with commercial privacy legislation, California once again appears poised to play an important role in establishing initial, generally applicable rules-of-the-road for business use of AI systems. However, this time there are two overlapping approaches that stakeholders must track.
Of the two efforts taking place in California, the first is with the California Privacy Protection Agency (“the Agency”). The CCPA charges the Agency with establishing rules “governing access and opt-out rights with respect to businesses’ use of automated decisionmaking technology” (ADMT). The Agency interprets this provision as an authorization to create standalone individual rights to opt-out of various automated processing technologies. Agency board member Alastair Mactaggart has gone so far as to call the Agency “probably the only realistic” AI regulator in the United States on the basis of this provision. To date, the Agency has proposed draft regulations that would create individual opt-out rights with respect to ADMT in six distinct circumstances that extend far beyond existing legal regimes. These include when ADMT is used to reach significant decisions about an individual, when ADMT is used to profile an employee or student, and when ADMT is used to profile an individual in a public place.
Second, California legislators have also taken an active interest in establishing broad protections and rights with respect to the use of AI systems. In 2023, Assemblymember Bauer-Kahan’s AB331 on automated decision tools made substantial legislative progress and appears likely to be reintroduced next year. The proposal is geared toward preventing algorithmic discrimination and imports a developer-deployer distinction from global frameworks for the allocation of risk management, rights, and transparency responsibilities. While the proposal was not enacted on its first attempt, AB331 has nevertheless already proven to be influential in shaping how policymakers in other states are considering AI systems.
Critically, these two emerging Californian approaches to regulating AI systems broadly overlap and are in tension on many key issues. For example, the CCPA’s draft regulations would include systems that so much as “facilitate” human decisions, while AB 331 is focused on systems that are the “controlling factor” for decisions. Separately, AB 331 is focused toward high-risk “consequential decisions,” while the CPPA is considering several applicability thresholds based on data collection and use in certain contexts that are unmoored from any objective standard of individual harm. The manner in which these diverging California processes advance, and questions about how they would operate in conjunction, is likely to play a major role in the emergence of standards for AI governance in the United States.
3. Will 2024 (Finally) be the Year of Privacy Enforcement Actions?
As the emerging state-driven approach to regulating individual privacy in the U.S. continues to mature, the contours of personal rights and business obligations will necessarily begin to be shaped not just by laws on the books, but also their interpretation, implementation and enforcement. While five ‘comprehensive’ state privacy laws will be in effect at the start of 2024, there remains a scarcity of regulator actions enforcing this new class of law. To date, the only known enforcement action that reached a financial penalty is the California Attorney General’s 2022 settlement with the French cosmetics retailer Sephora, which was based primarily on alleged failure to allow customers to opt-out of behavioral advertising. Following a quiet 2023, could 2024 be the year that the public first experiences widespread enforcement of their new privacy rights?
One structural reason for a lack of visible enforcement actions may be that Virginia, Colorado, Connecticut, and until recently, California all provide the ability for businesses to ‘cure’ many or all alleged violations of their privacy laws before a formal enforcement action can take place (this right to cure shall sunset in both Colorado and Connecticut in 2025). Therefore, initial enforcement activity in the first wave of state privacy laws may be happening largely out of the public eye, with businesses rapidly bringing their programs into compliance in response to notices of suspected noncompliance. Furthermore, while the CCPA’s right to cure has already sunset, the ability of its regulators to fully enforce the law has been thrown into doubt until next year due to missed rulemaking deadlines and a subsequent lawsuit from the California Chamber of Commerce.
Despite what may be perceived as initial slow going, there are several indicators of regulatory interest that may foreshadow forthcoming enforcement actions. For example, the Colorado Attorney General has announced the release of a series of enforcement letters focused on educating companies about their new obligations, particularly with respect to processing sensitive personal data. Furthermore, the California Attorney General’s Office and the California Privacy Protection Agency have launched separate inquiries with the Attorney General’s office seeking information about how businesses are applying the CCPA to employee data while the Agency is investigating the connected vehicle space. The fruits of these efforts may result in an upswing in public enforcement activity in 2024.
Separately, much of the Washington My Health, My Data Act (MHMD), the first major state privacy law to contain a broad private right of action since the adoption of the Illinois Biometric Information Privacy Act (BIPA) in 2008, will take effect in March 2024. MHMD is a far-reaching and novel commercial health data privacy framework that contains numerous ambiguous and inartfully drafted provisions which may generate both confusion and ripe grounds for litigation. In contrast to BIPA however, MHMD’s private right of action is tied to the state’s Consumer Protection Act, which lacks statutory damages and requires a showing of injury to ‘business or property’ to recover damages – a requirement that may temper the trial bar’s enthusiasm for lawsuits. The forthcoming litigation landscape around the MHMD and its perceived success or failure for advancing individual privacy protection may shape the state privacy enforcement landscape in 2023 and significantly influence whether private enforcement mechanisms are considered for inclusion in future privacy laws.
4. Which States will Tinker with their Existing Laws?
Despite the purported ‘comprehensiveness’ of the new state privacy laws, enacting a commercial privacy regime has been shown to often be just the start of a state’s legislative engagement on privacy matters. In 2023 alone, four of the initial five movers on state privacy took meaningful further steps on commercial privacy legislation. First, California lawmakers amended the CCPA to expand the definition of sensitive personal data and create protections for reproductive care information while also passing a first-of-its-kind law to establish a one-stop-shop mechanism to enable people to delete personal information held by data brokers. Second, before the Connecticut Data Privacy Act even took effect, its original sponsors successfully adopted amendments to dramatically expand its terms to include novel protections for health and child data. Third, Utah enacted new legislation creating far-reaching restrictions and age verification requirements for social media and adult content websites. Finally, Virginia came close to adopting a Governor-sponsored amendment to the landmark VCDPA which would have created verifiable parental consent requirements for the collection of personal information from children under age 18.
With a dozen comprehensive privacy laws now on the books that mostly share a similar framework, perhaps the question stakeholders should be asking is not ‘who is the next domino to fall’ but, ‘which existing law will be the first to be substantially revised?’
5. Is Any of this Constitutional Anyway?
Certain observers, particularly those more skeptical of government regulation, have long argued that wide reaching state privacy laws are Constitutionally suspect given the Dormant Commerce Clause and the First Amendment, particularly pursuant to Sorrell v IMS Health (2011) precedent. Such concerns and objections have been a long simmering feature of the conversation around the evolving state privacy landscape; however, they gained new life in September when an Obama-appointed federal judge enjoined California’s novel California Age Appropriate Design Code Act (AADC) from taking effect. What impact will this injunction and ongoing litigation involving the AADC have on the broader U.S. privacy landscape?
Adopted in 2022, the California Age-Appropriate Design Code Act was always an odd fit for the American legal context. The statute is directly rooted in a United Kingdom Code of Practice designed to implement aspects of the General Data Protection Regulation with respect to children. Certain non-privacy focused AADC business requirements – like conducting age estimation of users, limiting access to “potentially” harmful content, and granting the state Attorney General power to second guess whether organizations’ content moderation decisions conform with their posted policies – are in clear tension with longstanding U.S. precedent.
It was therefore expected when the trade association NetChoice initiated litigation against the AADC in December, 2022. However, in a surprise to many observers, the Court’s subsequent injunction systematically assessed and determined that essentially every affirmative obligation of the AADC is unlikely to survive commercial speech scrutiny, including privacy focused requirements for conducting data protection impact assessments (DPIAs), setting high default privacy settings, minimizing data collection and processing, and restrictions on so-called ‘dark patterns.’ Many of these provisions are common features (at least conceptually) of both comprehensive and sectoral U.S. commercial privacy laws. Should the full scope of District Court’s holding survive the state’s appeal intact, it will raise significant questions about the continued constitutional integrity of privacy laws across the country while providing a blueprint for subsequent legal challenges.
Conclusion
This commentary has noted several jurisdictions where impactful privacy legislation, regulation, enforcement, and litigation is a near certainty in the new year. However, the rate of state privacy activity has expanded each year since 2018, and observers should expect a new barrage of privacy proposals starting when state sessions formally start convening in January. There are many questions, but perhaps only one clear forecast: another turbulent and exciting year in the ongoing state-level efforts to advance and secure new privacy rights and protections for personal data is on the close horizon. Interested stakeholders can follow The Patchwork Dispatch for industry leading-updates and analysis tracking emerging trends and key developments throughout the year.
The PrivaSeer Project in 2023: Access to 1.4 million privacy policies in one searchable body of documents
In the summer of 2021, FPF announced our participation in a collaborative project with researchers from the Pennsylvania State University and the University of Michigan to develop and build a searchable database of privacy policies and other privacy-related documents, with the support of the National Science Foundation. This project, PrivaSeer, has since become an evolving, publicly available search engine of more than 1.4 million privacy policies.
PrivaSeer is designed to make privacy policies transparent, discoverable, and searchable, for use by researchers in the privacy field as well as privacy practitioners in the marketplace. PrivaSeer supports searches of a corpus of privacy policies collected from the web at distinct points in time – currently four time stamps. Search results can be filtered by a wide variety of parameters, including the date of the crawl, the publisher’s industry, use of particular tracking technologies, inclusion of relevant regulations, assessment on Flesch-Kincaid Reading Level, and more. The high level of customizable searchability is made possible via NLP techniques designed and implemented by researchers at the Pennsylvania State University and the University of Michigan. The project will continue to add new tranches of policies to the existing corpus on a periodic basis.
Two Project-Related Publications Received “Best Student Paper” Awards This Year
In addition to building the eponymous online tool, the PrivaSeer project grant has supported the publication of a number of papers by researchers involved in the privacy field. First, an effort to systematically identify and discuss issues within the privacy research community titled “Researchers’ Experiences in Analyzing Privacy Policies: Challenges and Opportunities” was presented at the 2023 Privacy Enhancing Technologies Symposium held in Lausanne, Switzerland by lead author Abraham Mhaidli, one of PrivaSeer’s graduate researchers from the University of Michigan. The paper was selected as one of the winners of the Symposium’s Andreas Pfitzmann Best Student Paper Award.
The paper was based on semi-structured interviews conducted with 26 researchers from a variety of academic disciplines working in the privacy space, and investigated what common research practices and pitfalls might exist in the privacy research space. The co-authors identified a lack of consistent, re-usable, well-maintained tools as one of the major obstacles to ongoing privacy research, resulting in significant duplication of effort among the research community, and noted the difficulty in fostering interdisciplinary collaboration.
A second paper, “Privacy Now or Never: Large-Scale Extraction and Analysis of Dates in Privacy Policy Text,” was accepted at the 23rd Symposium on Document Engineering (DocEng), hosted in Limerick, Ireland. This paper was presented by PrivaSeer graduate researcher and lead author Mukund Srinath from the Pennsylvania State University, and investigated the degree to which online privacy disclosures comply with annual update requirements across a set of large-scale web crawls containing several million distinct policies. Using a newly developed method for extracting dates from plain-text documents, the researchers discovered that under 40% of public privacy notices contain readable dates, and further, updates correlated heavily to major changes in the data protection legal landscape, with a significant percentage likely dating to 2018 without subsequent change. The paper’s conclusions point to the significant compliance problem of ensuring that privacy notices are actually kept up-to-date, and suggest that for many data controllers this is not the case, although more recent updates were associated with URLs that saw greater amount of online traffic.
A third paper, “Privacy Lost and Found: An Investigation at Scale of Web Privacy Policy Availability,” was also accepted at DocEng, and was further selected as the winner of the Best Student Paper Award. This paper presented a large-scale investigation of the availability of privacy policies, seeking to identify and analyze potential reasons for policy unavailability such as dead links, documents with empty content, documents that consist solely of placeholder text, and documents unavailable in the specific languages offered by their respective websites. The paper was also able to offer critical analysis and conclusions regarding privacy notices generally, based on a number of statistical methodologies. Overall, the researchers found that privacy policy URLs were only available in 34% of websites examined, and were able to estimate population parameters for both the total number of English-language privacy documents on the web and for their likely distribution across different commercial sectors. The study was able to further the privacy research community’s understanding of the overall status of English-language privacy policy policies worldwide, and provide valuable information about the rate and likelihood of users encountering various difficulties in accessing them.
2023 Stakeholders Workshop Provided Valuable Input Into Refining the PrivaSeer Search Engine and Tools
In addition to the publications associated with the PrivaSeer project, on July 25, 2023, the Future of Privacy Forum hosted an interdisciplinary workshop with key stakeholders to present the project to members of the privacy research community in industry and civil society.
July’s workshop featured presentations from FPF’s Vice President for Global Privacy Dr. Gabriela Zanfir Fortuna, as well as project co-leads Dr. Shomir Wilson, Assistant Professor in the College of Information Sciences and Technology at the Pennsylvania State University and Dr. Florian Schaub, Associate Professor of Information and of Electrical Engineering and Computer Science at the University of Michigan. Dr. Zanfir-Fortuna provided a practical demonstration of the PrivaSeer tool in action, while Professors Wilson and Schaub provided an overview of PrivaSeer’s development and current functionality.
Presentations by the project’s co-leads were followed by a discussion of how the tool may be used and improved as a future resource for researchers and industry professionals with various key FPF stakeholders. Discussants raised the prospect of using PrivaSeer to research the emergence of specific terms relating to the use of AI/ML technologies in privacy notices, conduct comparative studies of privacy policies presented in multiple languages, and examine how required disclosures related to cross-border data transfers may be changing over time. Participants also discussed how the tool might be useful in assessing privacy-adjacent disclosures such as cookie notices and terms of service, and provided the research team with a wide array of useful feedback as the project progresses into its third year.
PrivaSeer is now a functional, public-facing tool available to the privacy community, both for researchers and for privacy professionals working in public or private-sector compliance. FPF will continue to support the development of new functionality in the tool, and our team looks forward to contributing however we can to the scholarship in this area.
