Fairness, Ethics, & Privacy in Tech: A Discussion with Chanda Marlowe
After beginning her career as a high school English teacher, Chanda Marlowe’s career change led her to become FPF’s inaugural Christopher Wolf Diversity Law Fellow. She’s an expert on location and advertising technology, algorithmic fairness, and how vulnerable populations can be uniquely affected by privacy issues.
What led you to the Future of Privacy Forum?
I was a high school English teacher and I decided I wanted to be an advocate for student rights. I went back to school and earned a dual law degree and Master’s in Communications from the University of North Carolina at Chapel Hill. That’s where I found that I was really drawn to student privacy. I first came to FPF because I wanted to intern someplace that was leading in student privacy. I had cited FPF’s work in my papers and presentations. Also around that time, President Obama endorsed FPF’s Student Privacy Pledge, which was a big moment. I left FPF after my internship, and then about two years ago, I came back as the Christopher Wolf Diversity Law Fellow.
What attracts you to privacy issues?
There is a lot of room for thoughtful discussion among people who are very smart in the privacy space because there is so much grey area. It’s special to be part of these conversations when technology is emerging so rapidly.
What areas of privacy have attracted you?
I came to FPF intending to work primarily on education issues, and I ended up with opportunities to work on much more, including location and ad practices, the Internet of Things, and algorithmic fairness. So now I follow those issues closely and I really get into the granular details, like what does it mean to be compliant when the legal landscape is constantly changing with GDPR, state laws, and federal legislation.
I’m also drawn to how privacy impacts global populations. I did my Master’s thesis on surveillance of students, and I’ve been fortunate to work on the issue of algorithmic fairness.
Increasingly, everything is being automated and the decisions that were once made by humans are now being made by machines. This can lead to challenges – using machines doesn’t get rid of all bias. There could be bias in the data set, you could have problems with the machine itself, or the programmer could be biased. So those are three ways bias can be injected into a machine learning system.
I’ve been working very closely with Lauren Smith on a paper that maps out the potential harms that can arise from automated decision-making, and considers whether existing laws are adequate to address them.
I also enjoyed working on an FPF project that considers how the Internet of Things can have unique privacy impacts on people with disabilities. It was amazing to be are part of a project where we convened academics, consumer advocates (including disability organizations), and companies that create products for people with disabilities, ranging from startups to large platforms, and have a conversation about privacy concerns. We were able to work very closely with the American Association of People with Disabilities to draft a paper that not only explores the nuances of privacy considerations for people with disabilities using IoT devices and services, but also provides recommendations to address privacy considerations, which includes considering what mechanisms for informing people with disabilities are making their way into products. For example, the Amazon Echo added auditory cues so people who are blind aren’t expected to react to a light. We want to encourage more thoughtful approaches like that. It was an amazing experience to collaborate with so many people in the disability community.
What do you see as up-and-coming privacy issues?
I’m happy to see that there’s more talk about what privacy means for vulnerable populations. There has been increased attention to bringing in groups that have not previously been invited to the table to talk. Privacy isn’t just in one small bucket anymore, separate from broader conversations.
It’s really important that FPF is taking deliberate efforts to make sure the next generation of privacy professionals is diverse. The creation of the Christopher Wolf Diversity Law Fellowship embodies a commitment to valuing diversity in all fields that involve privacy.
Your fellowship is scheduled to end in a few months. What’s next for you?
This has already been a wonderful experience, and I have more to accomplish before it ends. Recently, I had the amazing experience of speaking before the Congressional Black Caucus Institute about privacy legislation. That’s something I never would have had the opportunity to do if I had stayed in North Carolina.
I’m interested in staying in the privacy law and policy space. I have learned so much and have been given so many opportunities that I cannot wait to launch the next steps of my privacy career.
FPF will host our next Privacy Book Club on April 24 at 2:00 PM EST. Join us to discuss Habeas Data: Privacy vs. the Rise of Surveillance Tech by Cyrus Farivar. Sign up for the book club here.
We hope you will join us at our 10th Anniversary Celebration on April 30. Buy your ticket here.
Future of Privacy Forum is Turning 10!
On April 30, 2019 from 6:00 PM – 8:00 PM, we will host a 10th Anniversary Celebration in Washington D.C. — and you’re invited! We are delighted to announce that at the 10th Anniversary Celebration we will present the following awards:
Helen Dixon
Data Protection Commissioner, Ireland
Distinguished Public Service
J. Trevor Hughes
President & Chief Executive Officer, IAPP (International Association of Privacy Professionals)
Community Builder
Dale Skivington
Privacy Consultant and Adjunct Professor of Law, University of Colorado Law School
Former Chief Privacy Officer, Dell Inc. and Eastman Kodak Company
Career Achievement
Peter Swire
Elizabeth and Tommy Holder Chair of Law and Ethics, Scheller College of Business
Georgia Institute of Technology
Outstanding Academic Scholarship
Additionally, FPF would like to thank the Leadership Sponsors who make this event possible:
And thank you to our Event Sponsor:
Date and Time
Tue, April 30, 2019
6:00 PM – 8:00 PM EDT
Location
The Line
1770 Euclid St NW
Washington, DC 20009
Schedule for the Evening
6:00 p.m. Eat, Drink and Socialize
7:00p.m. Short Program
Welcome
Christopher Wolf, President, Future of Privacy Forum Board of Directors
Andrea Jelinek, Chairwoman, European Data Protection Board, Director, Austrian Data Protection Authority
Award Ceremony
Introduction
Debra Berlyn, Treasurer, Future of Privacy Forum Board of Directors
Presentation
Rebecca Kelly Slaughter, Commissioner, Federal Trade Commission
Distinguished Public Service Award: Helen Dixon, Data Protection Commissioner, Republic of Ireland
Presentation
Sandra Hughes, Secretary, Future of Privacy Forum Board of Directors
Career Achievement Award: Dale Skivington, Chief Privacy Officer Dell Technologies 2011 – 2018 and Eastman Kodak Company 1988 – 2011
Introduction
Alan Raul, Future of Privacy Forum Board of Directors
Presentation
Abigail Slater, Special Assistant to the President, National Economic Council
Community Builder Award: J. Trevor Hughes, President and Chief Executive Officer, IAPP
Presentation
Danielle Keats Citron, Morton & Sophia Macht Professor of Law, University of Maryland Carey School of Law
Outstanding Academic Scholarship Award:
Peter Swire, Elizabeth and Tommy Holder Chair of Law and Ethics, Georgia Institute of Technology,
Future of Privacy Forum Senior Fellow
7:30 p.m. Thank You Toast and Celebrate
Jules Polonetsky, Chief Executive Officer, Future of Privacy Forum
Future of Privacy Forum Future Leaders
Interested in attending?
We are sold out, but would appreciate your support here. Donations benefit FPF’s Scholarship Fund, which supports the Elise Berkower Memorial Fellowship and the Christopher Wolf Diversity Fellowship.
Interested in sponsoring?
Sponsorship opportunities are available and may be found here. For additional sponsorship opportunities for the 10th Anniversary Celebration, contact Barbara Kelly, Leadership Director at [email protected].
The Future of Ad Tech: A Discussion with FPF's Stacey Gray
Almost everyone has had a similar experience: visiting a website to shop for a product and then having an advertisement for that product “follow” them around the internet. Most free content today, from social media to news, is funded by ads. In order to deliver those ads and measure their effectiveness, companies today rely heavily on data-driven technology (“ad tech”).
Stacey Gray is FPF’s lead Policy Counsel on ad tech and location services. Prior to joining FPF as a Policy Fellow in 2015, Stacey was at the Georgetown University Law Center, primarily focused on civil rights litigation and technology. In this week’s 10th Anniversary post, she traces the evolution of ad tech from the simple cookies of the past to the personalized billboards we might see in the future.
How has advertising technology changed over the past 10 years?
Probably the biggest change in the last decade has been the adoption of smartphones. It’s cliché, but it’s hard to overstate how everyone carrying a pocket supercomputer with 20+ sensors on it has changed the way advertisers think about ad campaigns. Conversations 10 years ago were limited to the online and desktop world—how websites placed “cookies” on browsers and whether ad networks should be able to track people across different websites. Now advertising technology is focused on reaching individuals in hyper-specific contexts (the right person at the right time and place) and measuring effectiveness by finding ways to link behavior across many devices and platforms.
Another way to describe it is that there’s been a blurring of online and offline worlds. This is enhanced by the rise of the so-called “Internet of Things,” meaning that we’re surrounded by connected devices—cars, appliances, toys, TVs, speakers—that collect and share information. Location-based marketing has also grown tremendously, with companies interested in geo-fencing content (targeting ads to a particular area) as well as using location data to measure things like whether people visit stores after seeing an advertisement. So there has been an explosion in the amount of data available about people and devices. In some ways though, the more things change, the more they stay the same. Advertisers still have the same basic goals—to serve ads to people and to measure whether the ads were effective in driving sales or visits. They are just doing it across far more channels and platforms than ever before.
How has FPF made sense of these changes, and what practices has it recommended?
We primarily convene privacy leaders who work on ad tech to level-set and figure out best practices. The FPF location & ad tech working group has 300+ participants, and a lot of good work happens behind the scenes. When we comment publicly, it’s usually to try to bridge the gaps in understanding between industry, advocates, and policymakers, or to identify privacy tensions in new technologies.
For example, we published a report in 2015 on cross-device tracking, explaining how and why companies are so interested in linking devices belonging to the same person. We’ve analyzed the state of privacy practices in smart TVs and published more narrow guidance on things like session replay scripts, Apple software updates, or understanding Bluetooth beacons. Advertising technology often raises issues that go beyond privacy, too. For example, advertisers have to ensure that they are not unfairly or harmfully discriminating against people on the basis of class, race, or gender. Advocates worry about “filter bubbles,” transparency in political advertising, and other potential harms related to algorithmic decision-making about online content.
What other developments in this space are currently being debated?
In contrast, in the last couple years, we’ve seen a massive proliferation in laws and regulations that specifically impact ad tech. Most importantly, European regulators are beginning to enforce the General Data Protection Regulation (GDPR) against ad tech companies and in some cases are being asked to evaluate the legality of the ecosystem as a whole. In the United States, the California Consumer Privacy Act (CCPA) that comes into effect next year will require companies to allow people to opt out of “sale” arrangements with their data, which will have a major impact on typical online and mobile advertising business models. Meanwhile, many other states are following California’s lead and introducing privacy legislation to be considered in 2019, not to mention the ongoing debate in Washington, DC, over a baseline, comprehensive national privacy law. There is a great deal ahead in 2019 and the opportunity (and challenge) for FPF is to combine industry knowledge with support for strong privacy safeguards to help shape effective regulation.
What changes in advertising technology do you expect to see in the next decade?
Advertising will continue to merge the online and offline worlds. I expect digital billboards using “intelligent video analytics” to be a major trend over the next 5 to 10 years. This is already beginning to happen. Last year in New York City, for example, there was a Fashion Week ad campaign using digital signs with video cameras that applied AI to analyze people’s outfits as they walked by and complement them for wearing particularly interesting ensembles. Walgreens has also started using facial characterization technology in their stores to offer ads based on whether the visitor is perceived to be male or female or a certain age. This kind of real-time video analytics may not always impact privacy, but may nonetheless change the way we think about public spaces.
I also think we will see a major shift in the way we understand revenue models for publishers. Ad-supported business models have generated a lot of free content, which can be good and often seen as “democratizing,” but in many ways it has come at the cost of data privacy. As the regulatory landscape shifts, I expect we will see new ideas and models for things like micro-payments or other new ways for people to support quality content. That said, we want to make sure that we don’t create an income divide in who has access to things like quality news and that we’re equally addressing issues of fairness and equal access to digital products and services.
Finally, it seems clear that we’re at the beginning of a major growth in academic and private support for public interest technology. I’m proud of my alma mater, Georgetown University Law Center, for recently joining with 20 other leading universities in forming a Public Interest Technology University Network to solve problems at the intersection of law and technology. The field of privacy-enhancing technologies (“PETs”) is also really promising. As we look ahead, I hope FPF’s unique knowledge and voice in the privacy space will continue to be helpful in supporting this kind of technical research and scholarship.
FPF will host our next Privacy Book Club on April 24 at 2:00 PM EST. Join us to discuss Habeas Data: Privacy vs. the Rise of Surveillance Tech by Cyrus Farivar. Sign up for the book club here.
We hope you will join us at our 10th Anniversary Celebration on April 30. Buy your ticket here.
FPF Welcomes Mark MacCarthy as Senior Fellow
The Future of Privacy Forum is pleased to announce the addition of Mark MacCarthy as a Senior Fellow. MacCarthy will work with FPF staff and members on data ethics, artificial intelligence and other issues.
“Mark has been a leading champion for responsible data practices and has worked to incorporate thinking from academia and civil society into policymaking and industry standards,” said FPF CEO Jules Polonetsky. “He was a key partner in working with SIAA and FPF to develop the Student Privacy Pledge, a commitment adopted by more than 350 companies. We and our cross-sector stakeholders look forward to his advice and scholarship on the challenging tech policy issues ahead.”
MacCarthy is an adjunct professor at Georgetown University, where he teaches courses in technology policy in the Communication, Culture, and Technology Program and courses on privacy and AI ethics in the Philosophy Department. He is also a Senior Fellow at the Institute for Technology Law and Policy at Georgetown Law and a Senior Policy Fellow at the Center for Business and Public Policy at Georgetown’s McDonough School of Business. Previously, he was Senior Vice President for Public Policy at the Software & Information Industry Association, where he directed initiatives and advised member companies on technology policy, privacy, AI ethics, content moderation and competition policy in tech.
FPF Comments on the California Consumer Privacy Act (CCPA)
On Friday, the Future of Privacy Forum submitted comments to the Office of the California Attorney General (AG), Xavier Becerra.
In FPF’s outreach to the AG, we commended the office for its multi-faceted solicitation of feedback from diverse stakeholders and the public in recent months, including through public forums, testimony before the California Assembly, and requests for comments.
Specifically, we wrote to:
1. Commend the State of California for addressing important data protection rights, including transparency, access, deletion, and reasonable security, for personal information. California has long been a leader in data privacy, and in the last year has served as a legislative model for other states as well as sparking a serious national conversation regarding a federal privacy law. While FPF supports a strong, comprehensive, baseline federal privacy law, we believe that states that do advance legislation should do so in ways that provide consumers with comprehensive protections that are in line with the Fair Information Practice Principles (FIPPs) and take into account interoperability with the EU General Data Protection Regulation (GDPR).
2. Recommend that rule-making efforts recognize that data exists on a spectrum of identifiability. While some data is firmly linked to an individual or provably non-linkable to a person, significant amounts of data exist in a gray area — obfuscated but potentially linkable to an individual under some circumstances. We recommend that the AG take account of this spectrum of identifiability and provide incentives for companies to de-identify data using technical, legal, and administrative measures.
3. Encourage further analysis of the impact of CCPA on socially beneficial research by non-HIPAA entities. Although CCPA excludes health data regulated by the Health Insurance Portability and Accountability Act (HIPAA) and related laws, its provisions govern private companies that may choose to conduct socially beneficial research using non-HIPAA data, including: consumer wearable manufacturers; health-related mobile apps; and genetic testing companies. While these companies should surely be subject to data privacy rules, we recommend that the AG take a close look at specific areas where beneficial research can be enabled or facilitated, or where restrictive requirements may pose particular challenges for researchers.
4. Encourage the AG to establish guidelines for data subject access requests (DSARs) that are secure, practical, and meaningful for consumers. The right to access one’s personal information is a fundamental tenet of the FIPPs, as well as a central feature of privacy laws in the United States and around the world. At the same time, there are inherent risks for some businesses in complying with data subject access request (DSARs), and often a direct tension between access rights and other important privacy safeguards. Ultimately, access requests should be secure, practical for businesses, and meaningful for consumers.
5. Recommend greater clarity on the intersection of CCPA and existing student privacy laws governing education technology vendors. For the benefit of schools, administrators, and education technology (“edtech”) vendors, the AG should clarify key points of CCPA that are applicable to education and student privacy, including: edtech vendors’ CCPA obligations (if any) when they act solely on behalf of public schools or districts; the circumstances under which edtech vendors may be considered “service providers” under the law; and alternately, how edtech vendors may navigate compliance obligations of CCPA in line with federal laws governing student records and California’s existing student privacy laws.
We also attached a list of relevant resources, including FPF publications on a variety of commercial privacy topics that may be of interest to the AG. We hope that our comments and the associated resources will be helpful to the important, ongoing discussion regarding consumer privacy in the State of California.
Privacy in Higher Education: A Conversation with Sara Collins
Innovation in higher education is increasingly fueled by data. From financial aid applications, to online classes, to student success initiatives, college students provide an extraordinary amount of data to schools, companies, and the government. This data provides unprecedented insights into student behavior, and colleges are using it to shape curricula, processes, and services to meet students’ needs.
In this week’s edition of FPF at 10, Sara Collins explains how data continues to transform the higher ed landscape, and why sound privacy practices are needed to ensure a safe, enriching academic experience.
Our 10th anniversary celebration will be on April 30. RSVP here.
Sara Collins is a Policy Counsel on FPF’s Education Privacy Project focused on higher education. While K-12 student privacy is the subject of much proposed legislation and public debate, less attention is focused on higher ed, which Collins hopes to change.
What drew you to the world of higher ed privacy? Which questions were you interested in exploring?
Prior to joining the Education Privacy Project, I was an attorney at the Department of Education’s Federal Student Aid in the Enforcement Office, and prior to that I was the Legal Director at Veterans Education Success. Both of my prior jobs involved analyzing how well colleges were serving their students, and that could only be done with comprehensive and accurate data. Our higher education system is fueled by data, and one persistent question that has shaped my career is, “How can we use this data to improve students’ education experience? “FPF recognizes that using data can change higher education and make it more beneficial for students, but also wants to answer the question: “How do we meaningfully protect student data?”
Why is it important to think of college students as a distinct population when it comes to data privacy?
Colleges and universities have massive amounts of data on their students. We don’t talk about this enough, but most modern college campuses have turned into miniature smart cities. Not only are schools increasingly using Massive Open Online Courses (MOOCs), which depend on data collection, often by third-party vendors, to run efficiently, but the infrastructure of campuses is full of new systems that collect data. When used correctly, this information can assist colleges in determining what capital improvements to make, managing traffic on campus, or improving graduation rates. But school administrators and third-party contractors must be aware of the challenge of collecting that data in a responsible manner and safeguarding it once it is collected. As we move forward, FPF hopes to put out more resources to help both schools and ed tech vendors manage these systems with an eye toward privacy and security.
How is FPF helping to provide these resources, and what privacy developments in higher ed will unfold in the next 10 years?
We regularly hold student privacy bootcamps for small and startup ed tech companies to provide training on privacy laws, best practices, and advocates’ concerns. We also run a higher education privacy working group to explore privacy concerns related to predictive analytics, big data, ethics, and data infrastructure in higher education. FERPA|SHERPA has other resources for higher ed officials, companies, and policymakers.
As far as the future is concerned, I imagine we’ll see major developments with regards to Title IX proceedings.
Title IX proceedings often involve highly sensitive data raising the question of how to balance due process, privacy, and transparency. And with the Department releasing new draft regulations, this conversation will only become more important.
Beyond Title IX, I think universities, companies, and the government will have to grapple with what privacy is and why it matters. For instance, federal initiatives like FSA Next Generation and the Pell Grant program mandate data collection. However, Federal Student Aid has not provided much transparency into its data collection, use, retention, and destruction practices. In April of 2017, FSA experienced a hack of its online FAFSA tool, which took them months to resolve. The lack of communication from FSA, as well as its cavalier attitude toward the data they collect, have forced students to make the choice of either getting money for their education or safeguarding their data.
As universities and colleges continue to grapple with how to best integrate technology in education, they will need to determine how to measure privacy harms, how to centralize administration of their sprawling data collection systems, and, above all, how to harness the potential of new technologies to enrich the lives of students under their care and instruction.
FPF Comments on the Washington Privacy Act, SB 5376
Today, the Future of Privacy Forum submitted comments to the Washington State Senate Ways & Means Committee on the proposed Washington Privacy Act, Senate Bill 5376. FPF takes a “neutral” position regarding the Bill, and makes a few important points.
FPF commends the Bill’s sponsors for addressing a broad set of individual data protection rights. While FPF supports a baseline federal privacy law, states that do advance legislation should do so in ways that provide consumers with comprehensive protections, in line with the Fair Information Practice Principles (FIPPs) and the General Data Protection Regulation (GDPR).
FPF observes that risk assessments can play an important role in protecting consumer privacy. Leading privacy frameworks include risk assessments as one important tool in setting organizations’ data protection priorities and safeguarding the most sensitive consumer information.
In its comments, FPF recommends specific expert resources on data de-identification. Most personal information exists on a continuum of identifiability. While some data is firmly linked to an individual or provably non-linkable to a person, significant amounts of data exist in a gray area – obfuscated but potentially linkable to an individual under some circumstances. Wise policies take account of this spectrum of identifiability and provide incentives for companies to de-identify data using technical, legal, and administrative measures.
FPF recommends that biometric technologies, including facial recognition, may be better served by a separate, future regulatory effort, in light of the complexity involved in crafting meaningful regulation of those technologies. In recent years, FPF has published resources on the distinctions between related technologies, including facial detection, facial characterization, and facial recognition – and offers further engagement on those issues.
FPF recently established an office in Seattle, which is the center for our Smart Communities project. This effort brings together privacy leaders at municipalities around the country who are implementing smart city projects in order to help them develop strong data protection frameworks.
NOTE: On March 21, 2019 FPF submitted comments on Senate Bill 5376 to the Washington State House Committee on Innovation, Technology, and Economic Development.
CPDP 2019 Panel: Understanding the limits and benefits of data portability
By Gabriela Zanfir-Fortuna and Sasha Hondagneu-Messner
The Future of Privacy Forum organized a panel at the 2019 Computers, Privacy and Data Protection Conference in Brussels to discuss the limits and benefits of the right to data portability as introduced by the GDPR. This panel was chaired by Thomas Zerdick (EDPS), moderated by Stacey Gray (FPF), and the speakers were Joris Van Hoboken (VUB-LSTS/UvA, Gabriela Zanfir-Fortuna (FPF), Babak Jahromi (Microsoft), and Olivier Micol (DG JUST).
The subject of the panel was prompted by several catalysts, including the discussions that held the front page last year over how Cambridge Analytica accessed personal data of Facebook’s users through an app, without disclosing to users the details and reasoning of the processing at the time they asked permission to install it. The Cambridge Analytica scandal shows the importance of platforms limiting access of third parties to users’ personal data. However, such limitations cannot be absolute. In fact, technical means to ensure interoperability between systems seem to be mandated by EU law.
As a matter of fact, one of the GDPR’s biggest innovations is the introduction of data portability as a new right of the data subject. Data portability presupposes that individuals should be able to transfer their personal data between players in the market, be they old or new, or even between players in different markets. This requires that companies make data users’ available for transfer in an interoperable format and subject to a user’s request.
In addition, Stacey Gray, FPF Policy Counsel, highlighted that the conversation on data portability is equally as relevant right now in the United States, given the debates on federal privacy legislation and the fact that one state law, the California Consumer Privacy Act, already included in its provisions portability as a by-product of the right of access.
How did portability appear in the competition/privacy discourse?
To set up the discussion, Gabriela Zanfir-Fortuna, FPF Policy Counsel, provided context of the right to data portability. She reminded the audience that the first instance of portability mandated by EU law was about portability of phone numbers, as a result of the Universal Services Directive. As shown by its recitals, that provision was introduced to promote competition among telephone service providers. Zanfir-Fortuna also mentioned that the debate on portability in relation to digital data was initiated in the US more than two decades ago and discussed a case from 2000, FTC v. ReverseAuction.com, which raised portability questions since it was caused precisely by a new service porting personal data of users from an established service, but by its own motion and without the prior consent of the users. However, data portability was first regulated in the EU, by the GDPR, which introduced it as a new right of the data subject and which gives the right to an individual either to ask an organization to port his or her personal data directly to another organization, or to receive that data in an interoperable format.
What are the limits of the right to portability in the GDPR?
Olivier Micol, the Head of the Data Protection Unit of DG JUST (European Commission), went into the details of what is the scope of data portability as provided by the GDPR, since the right to data portability only covers personal data that was “provided” by the person to an organization. He pointed out that the easy scenarios are when data is provided by an individual in the form of an upload, such as uploading photos or filling out a form. The EU Data Protection Authorities have issued guidelines on data portability which state that personal data that is provided includes data observed from use, for example the data collected about where an individual puts their cursor on a webpage.
What is not protected, is data that are the byproducts of services such as when a data controller uses an algorithm and processes data. Inferences from data would fall out of the scope of data portability, he said. Later on during the discussion in the panel Zanfir-Fortuna pointed out that, in contrast, data portability as provided by the CCPA also covers inferred data, since there are no limitations in the law in this regard, making its scope wider from this point of view than the scope of GDPR portability. The Commission official concluded his remarks saying that so far there have not been observed many applications of data portability in practice.
Can portability work in practice?
Babak Jahromi, IT Standards Architect at Microsoft, followed and presented the Data Transfer Project that Microsoft, Google, Twitter, and Facebook have all been participating in. Jahromi pointed out three difficulties with data portability in practice: Syntactic (is the data an integer, string, floating, or something else?); Semantic (for example, if data references a “jaguar” is it discussing a car or an animal); and Policy-related (how does it interact with existing regulations and contractual requirements for these companies). The Data Transfer Project was founded based on three principles: making the transfer of data technically feasible; implementing direct data transfer across potentially unlimited number of organizations; and ensuring that everything is open source so as to promote engagement. He emphasized the importance of such a system being open source and with broad appeal among a number of companies and data controllers.
What is the role of platforms in privacy governance?
Joris Van Hoboken, Professor of Law at Vrije Universiteit Brussels, began with a general remark that data portability is a way for individuals to get control over and value from their personal data. However, as he emphasized, a lot of data is very social in nature and many times involves other persons (such as photos or conversations), and there could be issues if someone ports their data to a platform that has weaker privacy protections. Van Hoboken discussed how platforms have become key players in privacy governance insofar as they are involved in: governing access to data; the design of relevant interfaces and privacy mechanisms; establishing policy and technical standards, such as requirements related to privacy policies or specific types of data; policing behavior of the platform’s users; coordinating responsibility for privacy issues between platform users and the platform; and direct and indirect enforcement of a platform’s data privacy standards.
Gray then asked the panel whether a data controller who has received a request has a right to object to the form or the process of the request. Zanfir-Fortuna stated that on first look, the controller could not refuse portability on the grounds of the receiving entity not having sufficient security protections. Micol stated that security is a key concern, but it should not be used as an excuse for porting data. Ideally, each data controller would have the GDPR already as a starting point of compliance, and as such would have applied all protective portions such as transparency, lawfulness, etc. The panel also discussed policy portability and the scenario of whether portability is technically possible or useful if it envisages pseudonymized data.
See the recording of the panel following this LINK.
Consumer Genetic Testing: A Q&A with Carson Martinez
Carson Martinez is FPF’s Health Policy Fellow. She works on privacy challenges surrounding health data, particularly where it is not covered by HIPAA, as is the case with consumer-facing genetics companies, wearables, mobile health and wellness applications, and connected medical devices. Carson also leads the FPF Genetics Working Group and Health Working Group.
How did you come to work on consumer genetic testing issues at FPF?
During my time as Master’s student studying Bioethics and Science Policy at Duke University, I focused on the ethical and policy challenges of technological innovations in healthcare. At Duke, I had the pleasure of taking an Information Privacy Law class with David Hoffman, Associate General Counsel and Global Privacy Officer at Intel Corporation, who introduced me to the pioneering discussions surrounding data privacy. I ended up writing my Master’s thesis at Intel on how government entities and cloud service providers can take steps to promote use, enhance trust, and foster innovation in cloud storage technologies for medical imaging data.
David, who is also on FPF’s Advisory Board, introduced me to Jules Polonetsky and John Verdi. FPF had already worked with industry to create best practices around wearables, and they wanted to expand FPF’s healthcare work.
As the only Policy Fellow at FPF without a law degree, I come at privacy from a unique perspective. My experience with bioethics gives me a good understanding of the research world and the important balance between making data available to advance scientific fields and protecting patient privacy. I work on challenges related to technologies that are outpacing our health privacy laws, like HIPAA and how best to protect this sensitive data without specific guidelines or regulations. That means working with stakeholders to develop best practices and help companies follow them.
What are some of the privacy challenges around consumer genetic tests?
As the price of consumer genetic tests continues to drop, they have become very popular purchases and gifts. Millions of people have used consumer genetic tests to learn about their heritage, identify risk for future medical conditions, and connect with family members. Unlike other personal data, genetic data may implicate future generations and have cultural significance for particular groups. This uniquely sensitive data deserves a high level of privacy protection.
Beginning in 2017, we led a process to develop privacy best practices for the consumer genetic testing industry. Stakeholders who participated in that process included the leading consumer genetic testing companies – some of whom originally approached FPF about the project – as well as experts on the science from the National Society of Genetic Counselors and the American Society of Human Genetics and advocates, from groups like Consumers Union.
What did the stakeholders agree should be in the best practices?
The best practices establish standards for genetic data generated in the consumer context that require:
Detailed transparency about how genetic data is collected, used, shared, and retained;
Educational resources about the basics, risks, benefits, and limitations of genetic and personal genomic testing;
Access Rights;
Valid legal process for the disclosure of genetic data to law enforcement and transparency reporting on at least an annual basis;
A ban on sharing genetic data with third parties (such as employers, insurance companies, educational institutions, and government agencies) without consent or as required by law;
Restrictions on marketing based on genetic data; and
Strong data security protections and privacy by design.
Recently, FamilyTreeDNA’s president apologized to customers for not disclosing an agreement with the FBI to allow agents to test DNA samples and access consumer genetic data without a warrant. That agreement is out of step with the best practices, and we have removed FamilyTreeDNA a supporter to them.
What new privacy issues could arise around consumer genetic tests?
The science of genetics is still evolving. Someday, we may have access to additional insights from genetic data that we can’t see today. We don’t yet know about many health conditions that may have a genetic component.
In the future, there will be more people taking consumer genetic tests and the tests will offer more extensive analytics. More companies will seek FDA approval to validate the efficacy and safety for identifying markers for health issues. With more people participating in testing, the ability to identify individuals who have not taken tests also will increase. All of that points to the need for a big push on consumer education.
What do you foresee as rising health privacy issues, beyond genetic data?
Looking beyond genetic information, to health data broadly, I expect to see a focus on the Internet of Health Things, fueled by tremendous growth in telehealth, including services tied to wearable or implantable monitoring devices. Those devices could transmit information to doctors, insurers, or employers. As more data is generated, privacy and security concerns may grow as well.
Another rising issue is the interoperability of data. If data is more portable, it can be more easily analyzed. Hopefully, consumer access and the development of third-party APIs to facilitate consumer-directed exchanges will empower people to take control of their own health and biological information and enhance interoperability.
In the medical world, there are more and more opportunities to opt-in to data sharing. Increasingly, I think we will see the development and application of strong privacy engineering solutions to protect sensitive health data and promote sharing for research, such as secure multi-party computation and differential privacy.
Many companies with health data are implementing ethical review processes for their research, which is a positive development. Consumer participation in research should be voluntary, informed, and follow established ethical standards.
FPF will be holding its 10th Anniversary Celebration on April 30th in Washington, DC. Join us to look back on the last decade of privacy and for a glimpse of what will be ahead.
Ticket and registration information for the 10th Anniversary Celebration can be found here.
Artificial Intelligence: Privacy Promise or Peril?
Advanced algorithms, machine learning (ML), and artificial intelligence (AI) are appearing across digital and technology sectors from healthcare to financial institutions, and in contexts ranging from voice-activated digital assistants, to traffic routing, identifying at-risk students, and getting purchase recommendations on various online platforms. Embedded in new technologies like autonomous cars and smart phones to enable cutting edge features, AI is equally being applied to established industries such as agriculture and telecomm to increase accuracy and efficiency. We see already that machine learning is becoming the foundation of many of the products and services in our daily lives, the underlying structure in much the same way that electricity faded from novelty to background during the industrialization of modern life 100 years ago.
Understanding AI and its underlying algorithmic processes presents new challenges for privacy officers and others responsible for data governance in companies ranging from retailers to cloud service providers. In the absence of targeted legal or regulatory obligations, AI poses new ethical and practical challenges for companies that strive to maximize consumer benefits while preventing potential harms.
Along with the benefits from the increased use of artificial intelligence and machine learning models underlying new technology, we also have seen public examples of the ways in which these algorithms can reflect some of the most glaring biases within society. From chatbots that “learn” to be racist, policing algorithms with questionable results, and cameras which do not recognize people of certain races, the past few years have shown that AI is not immune to problems of discrimination and bias. AI however, also has many potential benefits, including promising outlooks for the disability community and the increased accuracy of diagnosis and other applications to improve healthcare. The incredible potential of AI means that it is important to address concerns around its implementation in order to ensure consumer trust and safety. The problems of bias or fairness in ML systems are a key challenge in achieving that reliability. This issue is complex – fairness is not a fixed concept. What is “fair” by one measure might not be equitable in another. While many industry leaders have identified controlling bias as a goal in their published AI policies, there is no consensus on exactly how this can be achieved.
In one of the most notable cases of apparent AI bias, ProPublica published a report in which they claimed an algorithm, designed to predict the likelihood a defendant would reoffend, displayed racial bias. The algorithm assigned a score from 1 to 10, claiming to offer an assessment of the risk that a given defendant would go on to reoffend. This number was then often used as a factor in determining eligibility for bail. Notably, “race” was not amongst the various inputs which were used in determining the risk level. However, in their report, ProPublica found that among defendants who went on to not reoffend, black defendants were more than twice as likely as white defendants to have received a mid- or high-risk score. ProPublica correctly highlighted the unfairness of such disparate outcomes, but the issue of whether the scores were simply racially biased, it turns out, is more complicated.
The algorithm had been calibrated to ensure the risk level of reoffending “meant the same thing” from one defendant to another. Thus, of the defendants who were given a level 7, 60% of white defendants and 61% of black defendants went on to reoffend – a statistically similar outcome. However, in designing the program to achieve this level of equity (a “7” means ~60% chance of reoffending, across the board) means that the program forced distribution between low, mid, and high-risk categories in a way that resulted in more Black defendants receiving a higher score. There is no mathematical way to equalize both of these measures at the same time, within the same model. Data scientists have shown that multiple measures of “fairness” may be impossible to achieve simultaneously.
As importantly, the implementation of these scores by the humans within the system is impossible to quantify. There is no way to ensure that the score for one defendant will factored in by the judge in the same way as the score for another. Because of this tension, it is important that AI and ML designers and providers are transparent in their interpretation of fairness – what factors are considered, how they’re weighted, and how they interact – and that they sufficiently educate their customers in what their technology does or does not do. This is of special importance when operating in such sensitive fields as the criminal justice system, financial services, or other applications with legally significant impacts on individual customers.
However, even companies whose systems are outside such highly charged environments must remain cognizant of the potential for bias and discrimination. In 2016 the “first international beauty system judged by machines” premiered. The program was supposed to select a few faces which “most closely resembled human beauty” from a selection of over 6,000 entries. It overwhelmingly selected white faces. This is almost certainly because the training data or test data sets included more white faces than others. Or that the datasets more often had images of white faces associated with “beauty” or “beautiful” in some context. Thus, the algorithmic model “learned” that one of the factors contributing to the conclusion “beautiful” was “whiteness.”
Many types of Machine Learning, including deep learning, mean that the exact processing by which an algorithm makes a recommendation is ultimately unclear, even to its programmers. It is therefore all the more important to be able to evaluate outcomes objectively, testing for patterns or trends that demonstrate an undesirable bias. There is no such thing as a system without bias. Instead, a commitment to fairness means designing a system that can be evaluated for illegal, discriminatory, or simply undesirable outcomes. Algorithms trained on existing data from historically human systems will mirror some level of human bias – the goal should be to establish baseline practices for how to manage or mitigate this risk.
The most basic requirement is ensuring that the data sets the system is trained and tested on are appropriately representative. The chief science officer of the AI beauty contest mentioned above confirmed that one of the issues with the algorithm was that it was not trained with a sufficient sample size of non-white faces. In a training landscape where one specific race is more highly correlated with the idea of “beauty”, the algorithm will reflect this bias in its outputs. (For example, systems developed in Asia better distinguish Asian faces over white ones while the opposite is true for systems developed in the United States.)
Similarly, in law enforcement, training datasets are likely to reflect the historic disproportionate incarceration of non-white populations, and will reach outputs that reflect those systemic biases. However, identifying the potential flaws in datasets can be difficult – there are biases less obvious than those affiliated with race, gender, or other high-visibility factors. Unconscious or unintended bias can be present in less obvious ways, so AI/ML developers must have processes in place to preempt, prevent, or correct such occurrences.
Strategies include responding to research that shows that ensuring the humans behind the algorithm are sufficiently diverse can make a significant impact. Studies have shown that the racial and cultural diversity of the creators of facial recognition software influences the accuracy of the system. This implies that who trains the systems is an important consideration. By promoting diversity within their workforces, companies are also more likely to increase the accuracy and value of their systems.
Finally, there are statistical tools – additional mathematical models – that can be used to systematically evaluate program outputs as a way of measuring the validity of their recommendations. These auditing programs are a way of leveraging more math to evaluate the existing process in ways that exceed what human evaluators might be able to identify.
Companies – both those who develop these technologies, and their customers who implement them in different areas – have a responsibility to use all the tools in their power to address the issues of bias in their Machine Learning models. From policy requirements, to development guidance, hiring diversity and sufficient training, they must be able to assure their customers that the products and services based on ML models are sufficiently equitable for their particular application.
The unique features of AI and ML include not just big data’s defining characteristic of tremendous amounts of data, but the additional uses, and most importantly, the multi-layered processing models developed to harness and operationalize that data. AI-driven applications offer beneficial services and research opportunities, but pose potential harms to individuals and groups when not implemented with a clear focus on controlling for, and managing, bias. The scope of impact of these systems means it is critical that associated concerns are addressed early in the design cycle, as lock-in effects make it more difficult to later modify harmful design choices. The design must include the long-term monitoring and review functions, as these systems are literally built to morph and adapt over time. As AI and ML programs are applied across new and existing industries, platforms, and applications, policymakers and corporate privacy officers will want to ensure that the programs they design and implement provide the full benefits of this advancing technology, while controlling for, and avoiding, the negative impacts of unfair outputs, with the ultimate goal that all individuals are treated with respect and dignity.
Helen Dixon
J. Trevor Hughes
Dale Skivington
Peter Swire
How has advertising technology changed over the past 10 years?
By Gabriela Zanfir-Fortuna and Sasha Hondagneu-Messner
How did you come to work on consumer genetic testing issues at FPF?