5:30 pm –
5:40 pm ET

Welcome Remarks

Jordan Francis, Elise Berkower Fellow, Future of Privacy Forum

John Verdi, Senior Vice President for Policy, Future of Privacy Forum

5:40 pm –
6:00 pm ET

Opening Keynote Address

U.S. Senator Peter Welch, (D-VT)

6:00 pm –
6:15 pm ET

Less Discriminatory Algorithms

Entities that use algorithmic systems in traditional civil rights domains like housing, employment, and credit should have a duty to search for and implement less discriminatory algorithms (LDAs). Why? Work in computer science has established that, contrary to conventional wisdom, for a given prediction problem there are almost always multiple possible models with equivalent performance—a phenomenon termed model multiplicity. Critically for our purposes, different models of equivalent performance can produce different predictions for the same individual, and, in aggregate, exhibit different levels of impacts across demographic groups. As a result, when an algorithmic system displays a disparate impact, model multiplicity suggests that developers may be able to discover an alternative model that performs equally well, but has less discriminatory impact. But without dedicated exploration, it is unlikely developers will discover potential LDAs.

Model multiplicity has profound ramifications for the legal response to discriminatory algorithms. Under disparate impact doctrine, it makes little sense to say that a given algorithmic system used by an employer, creditor, or housing provider is either “justified” or “necessary” if an equally accurate model that exhibits less disparate effect is available and possible to discover with reasonable effort. As a result, the law should place a duty of a reasonable search for LDAs on entities that develop and deploy predictive models in covered civil rights domains. The law should recognize this duty in at least two specific ways. First, under disparate impact doctrine, a defendant’s burden of justifying a model with discriminatory effects should be recognized to include showing that it made a reasonable search for LDAs before implementing the model. Second, new regulatory frameworks for the governance of algorithms should include a requirement that entities search for and implement LDAs as part of the model building process.

Presenting Co-Authors

• Emily Black, Columbia University
• Logan Koepke, Upturn

Discussant

• Michael Akinwumi, National Fair Housing Alliance

6:15 pm –
6:30 pm ET

Do No Harm Guide: Applying Equity Awareness in Data Privacy Methods

Researchers and organizations can increase privacy in datasets through methods such as aggregating, suppressing, or substituting random values. But these means of protecting individuals’ information do not always equally affect the groups of people represented in the data. A published dataset might ensure the privacy of people who make up the majority of the dataset but fail to ensure the privacy of those in smaller groups. Or, after undergoing alterations, the data may be more useful for learning about some groups more than others. How entities protect data can have varying effects on marginalized and underrepresented groups of people.

To understand the current state of ideas, we completed a literature review of equity-focused work in statistical data privacy (SDP) and conducted interviews with nine experts on privacy-preserving methods and data sharing. These experts include researchers and practitioners from academia, government, and industry sectors with diverse technical backgrounds. We offer an illustrative example to highlight potential disparities that can result from applying SDP methods. We develop an equitable data privacy workflow that privacy practitioners and decision makers can utilize to explicitly make equity part of the standard data privacy process.

Presenting Co-Authors

• Claire McKay Bowen, Urban Institute
• Joshua Snoke, RAND Corporation

Discussant

• Miranda Bogen, Center for Democracy & Technology

6:30 pm –
6:45 pm ET

AI Audits: Who, When, How…Or Even If?

Artificial intelligence (AI) tools are increasingly being integrated into decision-making processes in high-risk settings, including employment, credit, health care, housing, and law enforcement. Given the harms that poorly designed systems can lead to, including matters of life and death, there is a growing sense that crafting policies for using AI responsibly must necessarily include, at a minimum, assurances about the technical accuracy and reliability of the model design.

Because AI auditing is still in its early stages, many questions remain about how to best conduct them. While many people are optimistic that valid and effective best practice standards and procedures will emerge, some civil rights advocates are skeptical of both the concept and the practical use of AI audits. These critics are reasonably concerned about audit-washing—bad actors gaming loopholes and ambiguities in audit requirements to demonstrate compliance without actually providing meaningful reviews.

This chapter aims to explain why AI audits often are regarded as essential tools within an overall responsible governance system and how they are evolving toward accepted standards and best practices. We will focus most of our analysis on these explanations, including recommendations for conducting high-quality AI audits. Nevertheless, we will also articulate the core ideas of the skeptical civil rights position. This intellectually and politically sound view should be taken seriously by the AI community. To be well-informed about AI audits is to comprehend their positive prospects and be prepared to address their most serious challenges.

Presenting Co-Authors

• Brenda Leong, Luminos.Law
• Albert Fox Cahn, Surveillance Technology Oversight Project (S.T.O.P.)

Discussant

• Edgar Rivas, U.S. Senator John Hickenlooper (D-CO)

6:45 pm –
6:55 pm ET

Data Is What Data Does: Regulating Based on Harm and Risk Instead of Sensitive Data

Heightened protection for sensitive data is trendy in privacy laws. Originating in EU data protection law, sensitive data singles out certain categories of personal data for extra protection. Commonly recognized special categories include racial or ethnic origin, political opinions, religious or philosophical beliefs, trade union membership, health, sexual orientation and sex life, and biometric and genetic data.

Although heightened protection for sensitive data appropriately recognizes that not all situations involving personal data should be protected uniformly, the sensitive data approach is a dead end. The sensitive data categories are arbitrary and lack any coherent theory for identifying them. The borderlines of many categories are so blurry that they are useless. Moreover, it is easy to use nonsensitive data as a proxy for certain types of sensitive data.

With Big Data and powerful machine learning algorithms, most nonsensitive data give rise to inferences about sensitive data. In many privacy laws, data giving rise to inferences about sensitive data is also protected as sensitive data. Arguably, then, nearly all personal data can be sensitive, and the sensitive data categories can swallow up everything. As a result, most organizations are currently processing a vast amount of data in violation of the laws.

This Article argues that the problems with sensitive data make it unworkable and counterproductive as well as expose a deeper flaw at the root of many privacy laws. These laws make a fundamental conceptual mistake—they embrace the idea that the nature of personal data is a sufficiently useful focal point. But nothing meaningful for regulation can be determined solely by looking at the data itself. Data is what data does.

To be effective, privacy law must focus on harm and risk rather than on the nature of personal data. Privacy protections should be proportionate to the harm and risk involved with the data collection, use, and transfer.

Presenting Author

• Daniel Solove, The George Washington University Law School

Discussant

• Didier Barjon, U.S. Senate Majority Leader, Charles E. Schumer

6:55 pm –
7:05 pm ET

The Prediction Society: Algorithms and the Problems of Forecasting the Future

Today’s predictions are produced by machine learning algorithms that analyze massive quantities of data, and increasingly, important decisions about people are being made based on these predictions.

Algorithmic predictions are a type of inference. Many laws struggle to account for inferences, and even when they do, the laws lump all inferences together. But predictions are different from other inferences and raise several unique problems. (1) Algorithmic predictions create a fossilization problem because they reinforce patterns in past data and can further solidify bias and inequality from the past. (2) Algorithmic predictions often raise an unfalsifiability problem. Predictions involve an assertion about future events. Until these events happen, predictions remain unverifiable, resulting in an inability for individuals to challenge them as false. (3) Algorithmic predictions can involve a preemptive intervention problem, where decisions or interventions render it impossible to determine whether the predictions would have come true. (4) Algorithmic predictions can lead to a self-fulfilling prophecy problem where they actively shape the future they aim to forecast.

More broadly, the rise of algorithmic predictions raises an overarching concern: Algorithmic predictions not only forecast the future but also have the power to create and control it. The increasing pervasiveness of decisions based on algorithmic predictions is leading to a prediction society where individuals’ ability to author their own future is diminished while the organizations developing and using predictive systems are gaining greater power to shape the future.

Data protection/privacy law do not adequately address these problems. Many laws lack a temporal dimension and do not distinguish between predictions about the future and inferences about the past or present. We argue that the use of algorithmic predictions is a distinct issue warranting different treatment from other types of inference.

Presenting Co-Authors

• Daniel Solove, The George Washington University Law School
• Hideyuki Matsumi, Vrije Universiteit Brussel

Discussant

• Didier Barjon, U.S. Senate Majority Leader, Charles E. Schumer

7:05 pm –
7:20 pm ET

Beyond Memorization: Violating Privacy Via Inference with Large Language Models

Current privacy research on large language models (LLMs) primarily focuses on the issue of extracting memorized training data. At the same time, models’ inference capabilities have increased drastically. This raises the key question of whether current LLMs could violate individuals’ privacy by inferring personal attributes from text given at inference time.

In this work, we present the first comprehensive study on the capabilities of pretrained LLMs to infer personal attributes from text. We construct a dataset consisting of real Reddit profiles, and show that current LLMs can infer a wide range of personal attributes (e.g., location, income, sex), achieving up to 85% top-1 and 95% top-3 accuracy at a fraction of the cost (100×) and time (240×) required by humans. As people increasingly interact with LLM-powered chatbots across all aspects of life, we also explore the emerging threat of privacy-invasive chatbots trying to extract personal information through seemingly benign questions.

Finally, we show that common mitigations, i.e., text anonymization and model alignment, are currently ineffective at protecting user privacy against LLM inference. Our findings highlight that current LLMs can infer personal data at a previously unattainable scale. In the absence of working defenses, we advocate for a broader discussion around LLM privacy implications beyond memorization, striving for a wider privacy protection.

Presenting Co-Authors

• Robin Staab, ETH Zurich SRI Lab
• Mislav Balunovic, ETH Zurich SRI Lab

Discussant

• Alicia Solow-Niederman, George Washington University Law School

7:20 pm –
7:25 pm ET

Closing Remarks

7:30 pm –
8:30 pm ET

Food & Wine Reception