Yesterday, the Future of Privacy Forum filed comments with the California Privacy Protection Agency on the initial rulemaking under the California Privacy Rights Act (CPRA). The CPRA, which comes into effect in 2023, provides protections for sensitive personal information, expands the California Consumer Privacy Act’s opt-out rights, and requires businesses to provide mechanisms for individuals to access, correct, and delete data.
FPF offered resources and recommendations regarding automated decisionmaking, sensitive personal information, global opt-out signals, and de-identification. Among our comments, we suggest that regulations under the CPRA should:
Establish guidelines for automated decisionmaking (ADM) that produces “legal or similarly significant effects.”
Provide that information about “automated decisionmaking” follow NIST interpretability guidelines, and be meaningful and reasonably understandable to the average consumer.
Clarify a range of potential use cases for health and wellness data, by providing a principled, exemplar list of categories that are in or out of scope. In many cases, such distinctions will be based on context and reasonable use.
Ensure opportunities for socially beneficial commercial research using sensitive personal information.
Clarify the role of global opt-out signals in the context of today’s labyrinth of existing permission frameworks, including in authenticated and non-authenticated platforms.
Establish an open process for authoritative approval of new global opt-out signals that meet the technical specifications of the Agency over time.
Seek further input from de-identification experts and researchers to clarify key implementation issues for “deidentified data,” including the role of technical, legal, and administrative controls, and Privacy Enhancing Technologies (PETs).
Event Report from DigitalxADB: Driving Digital Development across Asia and the Pacific
On October 27, the Future of Privacy Forum (FPF)’s Asia-Pacific office and the Asian Development Bank (ADB) co-hosted an online event titled, “Trade Offs or Synergies? Data Privacy and Protection as an Engine of Data Driven Innovation” in the context of DigitalxADB. This edition was the third in ADB’s series of annual knowledge-sharing events for representatives of ADB’s 68 member countries and external partners to learn about and take part in efforts to further integrate “digital” into ADB.
1. Background
By way of a background, ADB was conceived in the early 1960s as a financial institution that would be Asian in character and foster economic growth and cooperation in one of the poorest regions in the world. Despite the region’s many successes, it remains home to a large share of the world’s poor: 263 million living on less than US$1.90 a day and 1.1 billion on less than US$3.20 a day. ADB assists its members, and partners, by providing loans, technical assistance, grants, and equity investments to promote social and economic development. ADB maximizes the development impact of its assistance by facilitating policy dialogues, providing advisory services, and mobilizing financial resources through co-financing operations that tap official, commercial, and export credit sources.
For FPF, the co-organization of this digital policy dialogue with an international organization as important in the region as the ADB was an opportunity to manifest its intention to be useful to the data protection and privacy community in Asia through a large variety of means. FPF Asia-Pacific sees its role as a platform for cooperation that is both expert and neutral capable of supporting all kinds of actions that can contribute to the development of best practices in data protection and privacy, to help bridge the gaps between law and practice, and advance thought leadership and support coherent policy development in this area. Such cooperation must involve a wide variety of stakeholders, whether from the public or private sectors, national or regional, where appropriate in partnership with international organizations.
2. Key takeaways
This event consisted of two panel discussions.
The first, titled “Industry Expectations and Cooperation with Privacy Regulators in Asia,” was moderated by Yoonee Jeong (Senior Digital Specialist, ADB) and attended by panelists Marcus Bartley-Johns (Asia Regional Director, Government Affairs and Public Policy, Microsoft), Yen Vu (Principal and Country Manager, Rouse Vietnam), and Royce Wee (Director, Head of Global Public Policy, Alibaba Group).
The second, titled “To Be or to Become a Privacy Regulator in Asia in the 2020s: What Challenges, What Role for International Cooperation?” was moderated by Dr. Clarisse Girot (Director for Asia Pacific, FPF) and attended by panelists Michael McEvoy (Information and Privacy Commissioner, British Columbia, Canada, and Chair, Asia Pacific Privacy Authorities Forum – APPA), Zee Kin Yeong (Assistant Chief Executive, Infocomm Media Development Agency—IMDA, and Deputy Commissioner, Personal Data Protection Commission – PDPC, Singapore), and Prof Thitirat Thipsamritkul (Faculty of Law, Thammasat University, and Vice President of the Digital Council of Thailand).
This post summarizes the discussions in these two stellar panels and highlights key takeaways:
There is growing momentum for data protection and privacy in Asia. In 2020/21 alone,Singapore, Japan, South Korea, New Zealand, China and Thailand have upgraded or passed their data protection laws, while Brunei, India, Indonesia, Vietnam, and Sri Lanka among others move closer to adopting data protection frameworks of their own. Panellists Yen Vu and Thitirat Thipsamritkul shared first-hand experiences with development of data protection legislation in Vietnam and Thailand, respectively, while Zee Kin Yeong and Michael McEvoy shared their national and international experience as seasoned regulators in Singapore and British Columbia, Canada.
A key consideration for data protection law in Asia is finding the right balance between convergence with global standards and adaptation to local conditions. As more data protection laws in Asia tend to be developed with reference to frameworks and policies from outside Asia, policymakers in Asia must find a way to integrate data protection and privacy principles with Asia’s unique histories, cultures, and values to ensure that data protection laws win support from both businesses and citizens.
Data protection and privacy laws are most effective when made and implemented in partnership with businesses, industry associations, and civil society, as well as data protection regulators. Regulators and organisations can each learn important lessons from one another and, together with other key stakeholders, collaborate on tackling shared challenges and taking advantage of shared opportunities in the digital economy.
It is fundamental to support the development of the community of data protection regulators in Asia, whether through actions to support the development of national regulators, or regional cooperation networks such as they are developing, in this region as elsewhere. Based on experience, the top priority of regulators must be placed on education of businesses, government, and citizens, and equipping them with the right knowledge, tools and capabilities to ensure the effectiveness of the data protection law.
Trust, transparency, and accountability are key for businesses operating in Asia. Panellist Marcus Bartley-Johns related how Microsoft has come to recognize that Asian consumers, especially young people, are privacy-conscious and eager to understand how companies use their data. Similarly, panellist Royce Wee explained how trust is a key ingredient for a secure, inclusive, and sustainable digital economy, and increasing trust and transparency can create a win-win situation for consumers and businesses alike. In this regard, data protection laws play an important role to foster that trust.
What challenges to address, and what roles for ADB and FPF?
Thomas F. Abell (Advisor, SDCC and Chief of Digital Technology for Development, ADB) gave the introductory speech to the event and shared his insights into how the COVID-19 pandemic had accelerated the digital economy in Asia Pacific as the region increasingly relies on “digital.” 2020 was a record year in terms of member governments’ demand for ADB’s digital development programmes – roughly 20% of ADB’s projects in 2020 involved a significant digital component. Going forward, ABD is looking to increase support for its member governments in this area, from working on digital programs and security, to seeking thought leaders to drive digital development initiatives, to launching a new program in data analytics early next year.
Dr. Clarisse Girot(Director for Asia Pacific, FPF) explained how global activities have taken on an increasingly important dimension in FPF’s work, with the development of regional offices in Europe, Israel, and most recently, Asia with the recent launch of FPF’s Asia Pacific Office in Singapore. In Asia Pacific, an essential mode of action will be to forge partnerships, run joint events, and bring together businesses, citizens, and international organisations to support governments and regulators in their efforts to adopt laws and policies that address growing privacy expectations, raise the level of data protection, and ultimately, support economic growth and digitalisation in the region, especially in the wake of COVID-19.
From this point of view, the ambitions of FPF and ADB on these issues are completely complementary. This event is an opportunity to explore with the panelists what could be their priority actions in this area, if necessary joint actions.
Dr. Girot further highlighted the tension between Asia’s status as not only the most populous but also most economically dynamic region in the world and the fact that data protection laws, for historical more than for political reasons, tend to be developed with reference to instruments, frameworks, and policies that have been designed and developed elsewhere – the EU’s General Data Protection Regulation (GDPR) being a case in point. Dr. Girot stressed the need to ensure that national frameworks are compatible with global standards that are necessary in a world where data flows are ubiquitous and underlie the digital economy.
But more prosaically, there is also a need to address challenges that have blocked adoption of data protection and privacy laws in some jurisdictions where they have been announced as “imminent” for several years. Passing a data protection law is not easy, even less today than in the past. A major challenge in Asia is how to articulate data protection laws with the “geopolitically loaded” concept of “data sovereignty” – a concept which has taken root specifically in China and India and looks to spread elsewhere. Another blocking factor is the legitimate concern that data protection and privacy laws would impose administrative constraints and compliance costs for local businesses, thereby restricting innovation and blocking trade. As well, baseline data protection laws intersect with sectoral laws, so that a lot of finetuning is required. Defining the material scope of the law is not easy. Such fear also extends to the decision whether to institute a data protection and privacy regulator and provide it with powers and control over governments, among others.
To address these challenges, regional and international cooperation, and cooperation between the public and private sectors, academia and civil society, is essential. Events like DigitalxADB are thus an opportunity to demonstrate the wealth of resources that international cooperation brings. They also help to identify the multiple ways in which both public and private actors, including FPF and ADB, can contribute by providing support for governments and regulators in Asia to tackle these challenges—be it financial, material, or “intellectual”.
The two panel discussions were set up to approach these subjects from two complementary angles.
Panel 1: “Industry Expectations and Cooperation with Data Protection and Privacy Regulators in Asia”
This first panel moderated by Yoonee Jeong was comprised of industry representatives from different backgrounds, who share the same difficulties in complying with fluctuating and variable data protection rules in the region. During the conversation, each panelist was asked how they envision that ADB or FPF could usefully contribute to addressing these challenges.
Below is a synthesis of the main comments made by each panelist in the course of the conversation.
Marcus Bartley-Johns(Asia Regional Director, Government Affairs and Public Policy, Microsoft) opened his comments by lauding the efforts by ADB and FPF for coming together to convene this dialogue, and underlining the great value which lies in the combination of ADB’s unique convening power and ability to work with countries across the region on these issues, and FPF’ capacity to share expertise globally on what’s happening in privacy regulation and a lot of deep connections with the privacy community across Asia. He went on to share two key insights from Microsoft’s view of data protection and privacy issues around the Asia Pacific region.
The first is that privacy is essential for both organisations and individuals across Asia, and therefore, effective privacy regulation is central to growth of the digital economy across Asia. In this respect, Microsoft and research firm IDC conducted a surveyof the perceptions and expectations of trust in digital services of more than 6000 consumers in this region in 2019. 53% of those consumers reported feeling that their personal privacy had been compromised or that their trust had been breached when using digital services. A higher share of respondents who reported negative experiences were young people. This challenges the oft-held assumption that because young people – especially in Asia – are high consumers of digital services, they do not care about privacy. A further example is that of the 19 million unique visitors to Microsoft’s privacy dashboard in 2020, Australia, China, Japan, Korea, and India were all in the top 20 countries of visitors who came to view, export, or delete their data.
The second is that opportunities for collaboration on data protection and privacy abound. Organisations like FPF and ADB (among other stakeholders) can play a key role in developing privacy regulation through providing resources and technical assistance to countries that are thinking about privacy regulation and consultation to countries that are drafting new privacy regulations or amending their existing regulations. In particular, regulation needs to be technology-neutral as there is a temptation among regulators in Asia to look for an easy technical fix – such as contractual terms – to demonstrate privacy protection.
There are also opportunities for regional cooperation to counter the trend of countries working in “silos,” leading to a fragmented regulatory framework that will not support trade and investment and will increase costs for local companies – especially Small and Medium Enterprises (SMEs), which unlike large multinational companies (MNCs) cannot invest significant funds and employ hundreds of full-time engineers to transform their data management. In this regard, Singapore has been instrumental in driving greater regulatory coherence in ASEAN. More work on interoperability is needed to ensure that compliance will be as straightforward as possible for SMEs while still keeping a high bar for privacy protection cross the region’s regulatory landscape.
Yen Vu(Principal and Country Manager, Rouse Vietnam) shared the experiences of Vietnam as the country developed its first personal data protection decree, which she hopes will be passed and take effect by the end of this year.
Despite facing technological, economic, and societal challenges, Southeast Asia has an opportunity to become a digital economy hub for Asia. For example, even as large parts of Vietnam were under strict lockdown due to COVID-19, its Internet-based economy still reported growth in transportation, food, e-commerce, and fintech. The challenges come from an ever-shifting regulatory environment in both Vietnam and the region, as well as the need for training and awareness-building for both the public and private sectors.
In 2020, Vietnam became one of the first countries internationally to announce a programme for national digital transformation. Data protection will be key to this digital transformation programme, which aims to develop digital government, economy, and society and to equip Vietnamese digital businesses with global capacity in key areas – including healthcare, education, finance, banking, agriculture, transportation, energy, natural resources, the environment, and industrial protection – over the next decade.
However, the situation on the ground is one of regulatory fragmentation as Vietnam still lacks an omnibus law on data privacy. This has caused confusion and poses challenges for business across all sectors, which must often seek guidance from the government on how to comply with requirements under security laws, such as data localization. There are opportunities for international organisations like FPF and ADB to support Vietnam, especially through capacity-building activities for both the public and private sectors.
Royce Wee(Director, Head of Global Public Policy, Alibaba Group) highlighted that now is a very interesting time to be in Asia because more and more Asian countries are coming up with data protection laws. Thailand recently joined Singapore, the Philippines, and Malaysia as jurisdictions which already have data protection laws in place, and Brunei, Indonesia, Vietnam, and India move closer to adopting new data protection laws. China is also a major mover in this space, having passed a trio of data-related laws in a short time – the Cybersecurity Law, Data Security Law, and most recently, the Personal Information Protection Law (PIPL) which came into effect at the start of November 2021.
These data protection laws are not homogeneous but rather, reflect each country’s philosophies, outlooks, and values as well as its unique needs and circumstances. Data protection is not a solely European construct, and each country has to strike a balance between individual rights and control on the one hand and reasonable/legitimate business needs on the other hand.
This can create significant challenges for MNCs like Alibaba Group, whose compliance policies must be localized to meet each jurisdiction’s standards and requirements. In this respect, MNCs typically adopt a “high watermark” say set by the EU GDPR as a starting point and then make adjustments based on specificities in local data protection laws.
However, this is only a narrow view of data protection. Trust remains an overarching objective for these laws and is a key ingredient for a secure, inclusive, and sustainable digital economy. For organisations, trust helps to build long-term relationships with customers in which customers will be more willing to provide more and better-quality data, and organisations will be better placed to provide high-quality services and value-for-money products to meet their customers’ needs.
For regulators, trust in the digital economy allows for greater economic development and dynamism and can help to bridge the digital divide, opening the digital economy to greater participation from all segments of society while also creating better jobs with higher incomes by matching skills and demand and enabling better policy implementation.
The road to trust is one of constant, iterative improvement because – due to fast-paced changes in technology, business models, consumer expectations, and even societal values – the journey never really has an end in sight.
Regulators play an important role in pushing businesses to do more and to do better in a spirit of partnership and goodwill, rather than adversity. At the same time, businesses play an important role in uplifting data protection standards across the board. While MNCs have an important signalling effect, the real power to “move the needle” for data protection standards and processes comes from SMEs as they represent the vast majority of businesses in Asia. Regulators can do a lot to bring SMEs on board by issuing guidelines, providing clarity on their regulatory intent, and supplying tools and technological solutions. For example, in Singapore, the Infocomm Media Development Authority (IMDA) come up with “tech packs” containing solutions that SMEs can easily adopt and adapt to meet their business needs while ensuring at least a minimum baseline data.
Cooperative partnership between regulators and businesses is a prerequisite to develop the right culture of data accountability for organizations. Regulators should explain their regulatory objectives, concerns, and priorities but also understand the constraints and limitations in businesses’ daily operations. Similarly, businesses should understand these regulatory objectives, concerns, and priorities, but also provide feedback as part of the consultation process before new laws are passed, to ensure that the laws are practical and effective and that businesses can comply with them. For example, if left to their own devices, some regulators in Asia have a strong tendency to include data localisation into their laws. However, as the digital economy is essentially borderless, this can harm cross-border data flows necessary for e-commerce and the adoption of cloud solutions.
International organisations like FPF and ADB can, through their thought leadership and convening power, play an important role by contributing to the law-making process, especially through innovative projects such as sandboxing schemes, exploring different models for data processing, innovation, and even valuation, promoting harmonisation of baseline global principles and standards for data protection, to work with/across regulators and businesses to create mechanisms to allow/facilitate greater trusted and secure border data flows, and promoting discussion and agreement on an ethical framework for data processing that includes emerging technologies such as artificial intelligence, machine learning, and the Internet of Things.
By sharing resources and expertise, regulators and businesses can build trust and solve common problems and achieve common objectives – from improving the transparency of data processing, to putting in place adequate security standards and agreeing on common criteria/list of reasonable and legitimate uses of personal data, to reskilling and upskilling workers for new jobs in the digital economy.
Panel 2: “To Be or to Become a Privacy Regulator in Asia in the 2020s: What Challenges, What Role for International Cooperation?”
This second panel moderated by Dr. Clarisse Girot was comprised of two data protection regulators (Yeong Zee Kin and Michael McEvoy) and of an expert involved in the lawmaking process in Thailand (Prof Thitirat Thipsamritkul). Below is a synthesis of the main comments made by each panellist in the course of the conversation.
Professor Thitirat Thipsamritkul(Faculty of Law, Thammasat University, Vice President of the Digital Council of Thailand) shared her experience with the development of a draft personal data protection law in Thailand, which was ultimately passed in 2019.
Historically, data protection and privacy had been seen as a side issue which was not as essential to Thailand’s digital economy as, for example, cybercrime, cybersecurity, and intellectual property law. Little by little, privacy law became more central to the discussion with the emergence of the EU GDPR and efforts by the public sector, academia, and civil society to bring privacy into legislative discussions around the digital economy. By 2019, with the passage of the Cybersecurity Law, the zeitgeist was that if Thailand needed a cybersecurity law, then it also needed a data protection law.
The legislative process for the resultant Personal Data Protection Act (PDPA) was unique in that it involved extensive collaboration between the public and private sectors, academia, and civil society. In particular, academia was instrumental in shaping the PDPA as it had already created “shadow regulation” in the form of the Thailand Data Protection Guidelines (TDPG) to help Thai companies to comply with the EU GDPR and do business with Europe. The Guidelines were widely used by Thai businesses and drew not only on international standards but also input from local businesses and organisations on the practicality of data protection measures. Even after the PDPA was passed, the Guidelines remained influential for businesses designing their compliance schemes.
Thai society is now ready to comply with the PDPA but has been occupied with the response to COVID-19 for the last year. Due to resistance to the PDPA from certain sectors of the economy, the Thai government postponed the PDPA’s entry into effect twice. There is generally a fear that the PDPA gives too wide a discretion to regulators and the courts and that the courts’ interpretation would be uncertain as the PDPA introduces an entirely new framework into Thai law, also because due to stringent provisions on criminal liability for breach of the Act.
The postponements have sparked a debate in Thailand as to whether privacy laws should be strengthened or whether the compliance burden should be reduced as a result of the pandemic. However, at the same time, many businesses, including those in the financial and health insurance sectors, have been declaring new privacy protective measures and policies even before the PDPA takes effect.
On a broader note, many of the data practices in Asia differ significantly from those in Europe or America – for example, Asia has a lot of online shopping livestreams, which are much less common in Europe and Asia. This means that each region must adopt different methods for implementing data protection and privacy principles, even if these core principles remain the same around the world. However, a shared problem for regulators around the world is capacity-building – this is where international cooperation can be most effective.
Zee Kin Yeong(Deputy Commissioner, PDPC, Singapore) started with a word of encouragement for Thailand and explained that even Singapore’s journey to enacting data protection legislation started with a voluntary, industry-created model code, which was introduced in 2001-2002 – a decade before Singapore enacted its own PDPA in 2012. This was a necessary and helpful step to full legislation as local online businesses voluntarily adopted the code and began to prepare for full data protection legislation.
Yeong Zee Kin had three areas of advice for governments and policymakers who are data protection and privacy:
the necessity for convergence with global norms when designing laws;
equipping businesses and companies with practical tools to implement the principles within their organisations; and
valuing partnerships with the data protection community and data protection officers, who can act as champions to help to build the data protection ecosystem.
On convergence with global norms, he stressed that nowadays, data “can’t be kept in a bottle” as it flows everywhere – both within and between economies around the globe – especially as companies operate in multiple jurisdictions. Therefore, it is essential to design laws to adhere to accepted global principles to the greatest extent possible because such familiarity is important from the perspectives of both compliance and the expectation of consumers and data subjects. An example of such a global principle is the admonition against localisation of computing facilities. Other relevant global principles can be found in the OECD Privacy Guidelines, the APEC Privacy Principles, and for Southeast Asia, the ASEAN Principles for Data Protection, as well as free-trade agreements like the CPTPP and RCEP.
At the same time, it is also necessary to adapt laws to local conditions – society, culture, and history. The recent amendments to Singapore’s PDPA, which were passed a year ago, illustrate the importance of convergence as well as adaptation to local conditions. In the amendments, Singapore adopted the concept of “legitimate interests” because it had become common in multiple data protection regimes worldwide. However, Singapore also recognized that its local businesses wanted clarity and found a concept as broad and generous as legitimate interests difficult to work with. In implementing the concept, Singapore therefore took a slightly different approach to other regimes and listed out specific examples of legitimate interests in the Schedule to the PDPA. Singapore also took the unique step of creating a “business improvement exception” based on suggestions by local companies but still required express consent, rather than legitimate interests or business improvement, for direct marketing based on feedback from local consumers.
Between convergence with global norms and adaptation to local conditions, we will probably see more regional groupings in data protection laws as factors like geographical proximity heavily influence culture and history, which in turn influence expectations of and approaches to data protection. We should encourage these regional groupings and cooperation – if regulators and policymakers can come together and find a common level, then we might end up with three or four regional groupings, which could then start building bridges between regions to encourage global consistency and convergence.
On equipping businesses with practical tools, Yeon Zee Kin recommended that regulators place themselves in the shoes of local business owners and managers who would need to implement principles in legislation. Regulators can use the kinds of common business objectives that companies care about, such as inventory management, analysis of sales performance, and management of customer and HR records, as an entry point for discussing how data can be used to achieve those objectives while also embedding good data protection principles into the process. It is also important to recognize that businesses often need external help. To that end, Singapore’s PDPC curated a brief list of core data protection practices and provided a list of outsourced data-protection-as a-service providers who could help business owners and managers with compliance.
Michael McEvoy(Information and Privacy Commissioner, British Columbia, and Chair, APPA Forum) agreed that there are many examples in which jurisdictions go through a transition period from having voluntary standards, guidelines, and principles to having full data protection legislation but added that in some cases, legislation may be a result of pressure from civil society, a shift to a more reformist government, or even simply a fluke of circumstances. However, even where the legislation seems to come to fruition suddenly, it is usually the product of many years of work and efforts to educate legislators.
Voluntary industry efforts in British Columbia – such as data breach notification although there is not yet a legal obligation making notification mandatory in the province– can be the start of good practice as they create a culture and environment of compliance. In experience, businesses generally want to “do the right thing” but they might not be able to figure out how to do it. As well, it is also certain that there is a fear. Misplaced fear on the part of businesses about regulator enforcement powers and in general that regulators may not understand the nature of innovative businesses may delay the adoption of a complete regulatory framework in some jurisdictions. Enforcement is certainly important. But the solution to address such concerns is first and foremost for regulators to go out and educate businesses and in some cases, governments, on what the “right thing” is and provide guidance, education, and assistance.
As personal data follows the flow of trade, more and more countries are waking up to the need for effective, sustainable, and trustworthy regulation for an increasingly digital world. This idea underpins the work of the Asia Pacific Privacy Authorities (APPA) Forum over the past 30 years to nurture and promote data protection in the Asia Pacific region. While initially there was not a lot of interest in APPA’s work, there definitely is now: British Columbia, which is home to APPA’s Secretariat and does approximately $14 billion worth of export trade with Pacific Rim countries, has come to recognize the importance of data protection to digital trade, and its legislature now supports APPA financially.
APPA’s 19 members – all data protection regulators in Asia Pacific – assist one another and share information and techniques to enhance their regulatory expertise. APPA has also extended a hand to other jurisdictions outside this region, such as recently in the Cayman Islands. Countries that are now considering implementing new data protection laws, where none previously existed, are fortunate in that these countries can learn from the experiences of jurisdictions that have gone through this process, adapting what is useful and avoiding the regulatory missteps that unfortunately happen from time to time. No two countries’ data protection laws will ever be identical because country is informed by its own history and culture. However, countries across the globe share a commitment to have at least some commonality, especially in allowing data to flow more freely and securely. In this respect, the GDPR and the concept of adequacy have been very helpful in the search for common ground and convergence on principles for protecting citizens’ data while encouraging trade, innovation, and flow of data.
The session thus ended on a very encouraging note.
To conclude, ADB and FPF thanked the speakers and announced that they would consider joint actions to support positive data protection developments in the region, in the spirit of cooperation which animated the whole of this session.
This blog was written with the support of the Global Privacy team of the Future of Privacy Forum.
Data Sharing … By Any Other Name
Data Sharing … By Any Other Name(Would Still Be a Complex, Multi-stakeholder Situation)
“It is widely agreed that (data) should be shared, but deciding when and with whom raises questions that are sometimes difficult to answer.”[1]
Data sets held by commercial and government organizations are an increasingly necessary and valuable resource for researchers. Such data may become the evidence in evidence based policymaking[2] or the data used to train artificial intelligence.[3] Some large data sets are controlled by government agencies, non-governmental organizations or academic institutions, but many are being accumulated within the private sector. Academic researchers value access to this data as a way to measure any number of consumer, commercial, and scientific questions at a scale they are unable to reach using conventional research data gathering techniques alone. Such data allows researchers access to information that allows them to answer questions on topics ranging from bias in targeted advertising, to the influence of misinformation on election outcomes, to early diagnosis of diseases through use of health and physiological data collected by fitness and health apps.
Recent attention on platform data sharing for research is only one conversation in the cacophony of cross-talk on data sharing. There are many different uses of the term “data sharing” to describe a relationship between parties who share data from one organization to another organization for a new purpose. Some uses of the term data sharing are related to academic and scientific research purposes, and some are related to transfer of data for commercial or government purposes. In this moment, where various types of data sharing are a concern elevated even to the attention of the US Congress and the European Commission[5], it is imperative that we are more precise which forms of sharing we are referencing so that the interests of the parties are adequately considered, and the various risks and benefits are appropriately contextualized and managed. In the table at bottom, we outline a taxonomy for the multiplicity of data sharing relationships.
Ultimately, the relationships between these entities are complex. In many cases, the relationship is 1-to-many, with a single agency or corporation sharing data with multiple researchers and civil society organizations or, as in the case with data trusts or data donation platforms, potentially one person sharing data with many research or commercial organizations through a trusted, intermediate, steward.[6] Likewise, researchers and civil society organizations may concurrently pursue data from multiple corporate or government organizations, in many cases for the ability to address those challenges that require extremely large quantities of data (Big Data) or complex networks of related data. This data flow is never just along a single channel, nor does it often stop after a single transfer. Governments and corporations share data with researchers; researchers return that data, generate new data, and share analysis and new questions and outcomes back around.
Managing these complex relationships requires multi-layered contracts, defined procedures, accountability mechanisms, and other technical and policy controls. The terms for data sharing cover obligations that both parties have, including privacy, ethics, governance, and other good stewardship protocols. Changes in the legislative landscape around data protection, privacy, and security mean that these relationships must adjust periodically to meet legal compliance obligations, on the data sharing or data using side.
At the Future of Privacy Forum, we are working to add context, nuance, and a considered evaluation of the needs of these many players to create guidelines and best practices to support data sharing, particularly for conduct of scientific and evidence-based policy research. What data is shared, under what conditions, controls, contracts, and use environments all have important privacy and governance implications. We have been actively working in this area since 2015, and continue to engage with various interested organizations around the challenges in today’s digital environment. With respect to the sharing of data itself, FPF is focused on finding ways to incorporate proportionate precautions so that any sharing activities adequately protect privacy and are designed with the full understanding of potential harms to the people whose data is transferred or the communities of which they are a part.
Data Sharing Relationships
Data Sharing Organization Type
Data Using Organization Type
Outcome of Data Sharing
Terms to Describe Data Sharing Relationship
Government Agencies
Researchers and Research Institutions
Researchers conduct evidence based evaluations of public programs
Researchers whose work is sponsored by corporations or who have privileged access to corporate data assets return data gathered for future corporate research and, in many cases, retain copies of that data for future scientific work
Researchers whose work is sponsored by or conducted under a government contract return data gathered for future agency research and, in many cases, retain copies of that data for future scientific work
Citizen groups, journalists, and communities of interest (e.g., patient advocacy groups) can gain access to data about themselves gathered during the research process so that they can use it for future treatment, advocacy, or research participation
Return of research data and/or research results[18]
Researchers and Research Institutions
Researchers and Research Institutions
Researchers can reuse other researchers’ data or combine their primary and others’ secondary data to answer novel questions without having to put people at risk of research harms by conducting further research with them
Archives can collect the primary data from multiple researcher to streamline the process of acquiring data to answer novel questions by re-examining data and not putting people at risk for research related harms by conducting further research with them
Data Stewardship bodies, such as Data Trusts or Data Donation Platforms
Researchers and Research Institutions; Government Agencies, Private Companies or Corporations
Individuals and groups share their data with others according to their interests as specified to and protected by a trusted, fiduciary, actor.
Data Trusts, Data Donation
[1] HHS Office of Research Integrity, ORI Introduction to RCR. https://ori.hhs.gov/content/Chapter-6-Data-Management-Practices-Data-sharing
[2] H.R.4174 – 115th Congress (2017-2018): Foundations for Evidence-Based Policymaking Act of 2018. (2019, January 14). https://www.congress.gov/bill/115th-congress/house-bill/4174
[3] “The Biden Administration Launches the National Artificial Intelligence Research Resource Task Force”. https://www.whitehouse.gov/ostp/news-updates/2021/06/10/the-biden-administration-launches-the-national-artificial-intelligence-research-resource-task-force/
[4] Goroff, Daniel, Jules Polonetsky, and Omer Tene. (2018). Privacy Protective Research: Facilitating Ethically Responsible Access to Administrative Data. The Annals of Political and Social Science, Vol 675, Issue 1, pp. 46-66. https://doi.org/10.1177/0002716217742605.
Harris, Leslie and Chinmayi Sharma. (2017). Understanding Corporate Data Sharing Decisions: Practices, Challenges, And Opportunities for Sharing Corporate Data with Researchers. Future of Privacy Forum. https://fpf.org/wp-content/uploads/2017/11/FPF_Data_Sharing_Report_FINAL.pdf.
[5] European Commission. (2021). “A European Strategy for Data” https://digital-strategy.ec.europa.eu/en/policies/strategy-data
[6] Open Data Institute. (2020). “Data Trusts in 2020”. https://theodi.org/article/data-trusts-in-2020
Future of Privacy Forum Releases Student Monitoring Explainer
On October 27, FPF released a new infographic, “Understanding Student Monitoring,” depicting the variety of reasons why schools monitor student digital activities, what types of student data are being monitored, and how that data could be used. While student monitoring is not new, it has gained significant traction recently due to the shift to remote learning and the increase in school-managed devices being issued to students.
“Student monitoring has been happening for years, but too often families only learn about it after their child has been flagged or they’ve read something in the news. And that lack of transparency creates questions and confusion about how exactly it works, and what is – and is not – being monitored,” said Amelia Vance, FPF’s Vice President of Youth and Education Privacy. “We hope that this infographic will help parents, students, educators, policymakers, and other stakeholders understand generally how student monitoring works and what it aims to do, and ultimately become empowered to ask questions about the monitoring products being used in their own districts, as there is often considerable variation.”
The infographic depicts the main reasons why schools monitor student activity online—ensuring student safety, legal compliance, and addressing community concerns—and highlights two areas of frequent confusion: what types of student data are being monitored, and how that data could be used.
While school administrators work with their chosen service provider to set up a monitoring system that meets their school’s needs, student data can be collected in multiple ways, including from:
School-Issued Devices: any student data that travels through an internet connection, wired or wireless, on a school-managed device.
School-Managed Internet Connections: data from students’ online content or activities on school-managed internet connections, potentially including take-home internet hotspots.
School Apps & Accounts: student data from certain school-managed accounts, regardless of whether students access the accounts from personal devices or personal internet connections at home.
Monitoring systems analyze student data from these sources for potential concerning indicators, which are typically related to warning signs of self-harm, violence, bullying, vulgarity, pornography, or illegal behaviors. Some systems flag content for human review. From there, depending on the nature and severity of the flagged content and monitoring system in place, several actions could occur. The student could be sent a warning, the content could be blocked, or a designated school contact could be alerted. These actions are explored in further depth in FPF’s accompanying blog.
“Many school administrators, students, and families may be aware that monitoring systems seek to identify concerning indicators from students’ online activities, but there is often less understanding about what occurs once a system does flag concerning activity,” said Yasamin Sharifi, a Policy Fellow in FPF’s Youth and Education Privacy team. “FPF’s new infographic clarifies the analysis, actions and data retention that a monitoring system and school may perform. This understanding is crucial for any stakeholder seeking to comprehend the practical impacts of a student monitoring system.”
The Future is Open: The U.S. Turns to Open Banking
FPF is pleased to work with a broad set of stakeholders on concepts around privacy and open banking. For more information on our new Open Banking Working Group and related projects, please contact Jeremy Greenberg: [email protected].
Open banking is a concept that describes banks and other financial institutions, such as credit unions, providing rights to customers over their financial data, including the ability to access, share or port data to third parties for various services.
The inherent tensions found in open banking between privacy, competition, and data portability requirements mirror similar concerns across the spectrum of Big Data.
Current challenges to a widespread and healthy open banking ecosystem in the U.S. include a lack of harmonized rules and principles for maintaining strong privacy protections involving financial data and an absence of standardized technical architecture.
The Consumer Financial Protection Bureau (CFPB) will take the lead on facilitating open banking in the U.S. and crafting rules regarding data protection and security; the CFPB should consider lessons learned from international approaches.
Open banking proponents and policymakers should be mindful of the unique sensitivity of financial information and the complex data protection risks raised by increased sharing of banking data—even when sharing is directed by consumers.
Introduction
In July 2021, President Biden signed the Executive Order on Promoting Competition in the American Economy. The Executive Order takes a “whole of government approach” to enforcing antitrust laws across the economy, with clear implications for data protection and privacy. Notably, the order encourages the Consumer Financial Protection Bureau (CFPB) to consider crafting rules under section 1033 of the Dodd-Frank Act in support of open banking with the goal of making it easier for consumers to safely switch financial institutions and use novel and innovative financial products while maintaining privacy and security.
The Order’s callout signals that the Biden administration views open banking as an important initiative for promoting consumer choice, fostering competition, and protecting consumers’ privacy. The debate around open banking highlights tensions between privacy and competition along with a number of privacy flashpoints including: data portability, access, sharing, transparency, control, and interoperability.
Open Banking Can Provide New Rights and Benefits to Consumers and Help Spur Competition, But Technical and Privacy Challenges Remain
Open banking is a concept that describes banks and other financial institutions, such as credit unions, providing rights to customers over their financial data, including the ability to share data or permissions over their data with third parties for various services. These rights include the right to access their financial data, port their data and switch financial institutions, and grant permission to third parties to carry out transactions and provide financial services to best meet a customer’s needs. For example, individuals could grant access to their financial data to a third party to complete an automated payment or provide tailored financial planning advice based on a consumer’s individual finances or credit history. Proponents of open banking argue that another benefit is increased competition among financial institutions. Firms entering into the financial sector may offer novel services that spur competition across the industry.
One current challenge to a widespread and healthy open banking ecosystem in the U.S. is a lack of harmonized rules and principles for maintaining strong privacy protections involving financial data. As a result, some traditional banking institutions concerned with maintaining strong customer privacy might be hesitant to support an open banking ecosystem that lacks clear and strong privacy rules and principles that equal, or exceed, the current financial privacy and security protections afforded to consumers by regulations such as the Gramm-Leach-Bliley Act (GLBA) or the Fair Credit Reporting Act (FCRA).
Another roadblock to widespread and privacy-protective open banking is the need for standardized technical architecture—particularly interoperable APIs— to enable the safe portability of financial data. A standardized and interoperable API would allow third parties to carry out their services on behalf of customers without accessing certain personal information, such as various login credentials. In the absence of widely adopted secure APIs, third parties sometimes turn to screen scraping to perform services, while collecting customer login credentials and other personal information, leading to potential privacy and security risks such as exposing consumer information in the case of a data breach and consumer impersonation. While current industry efforts such as the Financial Data Exchange’s (FDX) API are underway, a lack of standardized rules and technical standards, such as machine readable file rules, can lead to less privacy-protective methods of third parties accessing data.
The CFPB Will Continue Taking the Lead on Facilitating Open Banking in the U.S, While Considering Lessons Learned from International Approaches
Prior to the Executive Order, the CFPB has taken some preliminary steps to promote safe open banking in the U.S. In 2017, the agency released a set of broad non-binding principlesintended “to help foster the development of innovative financial products and services, increase competition in financial markets, and empower consumers to take greater control over their financial lives.” Key areas of focus include: data access (enabling consumers to obtain financial information in a timely manner without being compelled to share account credentials with third parties); informed consent (in which consumers understand terms & conditions with the ability to readily revoke authorizations granted to third parties); payment authorization (in which third parties are required to obtain specific authorization for distinct activities); efficient and effective accountability mechanisms (incentivizing stakeholders to prevent, detect, and resolve unauthorized access, sharing, and payments); among several other areas.
The CFPB next weighed in on the issue in 2020 when it held the CFPB Symposium: Consumer Access to Financial Records where experts discussed many of the concepts highlighted in the agency’s principles. Following the symposium, in October 2020, the CFPB initiated an Advanced Notice of Proposed Rulemaking on consumer access to financial records and how the agency might develop rules for implementing section 1033 of the Dodd-Frank Act. This is the same rulemaking effort highlighted in the Executive Order. The agency sought comments on the costs and benefits of open banking, and how the agency might handle many of the data protection-related concepts outlined in its 2017 principles, including: access, control, privacy, security, and standard setting. The CFPB has not issued a final rule or concluded the rulemaking, but the agency recently listed data sharing in its current regulatory agenda. Other than the CFPB, The Federal Reserve, Federal Deposit Insurance Corporation (FDIC) and the Office of the Comptroller of the Currency (OCC), released a Proposed Interagency Guidance on Third Party Relationships: Risk Management, focusing on banks managing risks in their third-party relationships with fintech companies, vendors, and other affiliates. While other regulators are involved in this space, the CFPB appears poised to return to their rulemaking effort as a near-term priority for the agency.
While the U.S. is serious about responsibly regulating and setting standards for open banking, other international models are well down this path. In 2015, the EU released an updated Payment Services Directive (PSD2), which went into effect in 2018. PSD2 aims to promote competition, privacy, and data transfer between EU countries and institutions. However, some PSD2 requirements, such as rules around consent, can significantly differ from requirements found in the GDPR and other European laws, leading to a lack of harmonization and confusion for consumers, regulators, and financial institutions. Other leading open banking approaches include recent efforts in the UK, Australia, Brazil, Israel, India, Canada, Mexico, and others. The technical standards and requirements around open banking will likely have to be harmonized between different regimes to promote the international and cross-border nature of the global economy.
Open Banking Highlights Broader Questions about Data Portability, Competition, and Cross-Border Data Flows
While the Executive Order sends a trumpet blast to regulators, consumers, and financial stakeholders that open banking is a priority area for the current administration, many of the data protection themes at play are much broader than open banking and touch multiple industries. The inherent tensions found in open banking between privacy and competition—such as the need to keep data private and in trusted hands vs. new players obtaining access or control over data for various purposes–exists across the spectrum of Big Data. Further, open banking helps animate the current debate and recent interest around data portability requirements from agencies such as the FTC. Ultimately, the need for interoperable rules and technical measures are not only necessary for beneficial and safe open banking, but for other international and cross-border data exchanges.
Future of Privacy Forum Promotes Verdi, Zanfir-Fortuna & Vance
FPF has promoted three of its leaders to more senior roles at the growing international non-profit. John Verdi has been elevated to Senior Vice President of Policy, Dr. Gabriela Zanfir-Fortuna has been appointed Vice President of Global Privacy, and Amelia Vance is now Vice President of Youth and Education Privacy.
For more than five years, John Verdi has been integral to FPF’s success as a mentor to our staff and an advisor to privacy leaders in the public and private sectors. Our international, youth and education programs are respected resources for civil society, policymakers, and companies because of the leadership of Gabriela Zanfir-Fortuna and Amelia Vance. These three appointments reflect the growth of FPF as data protection issues impact organizations around the world.
Jules Polonetsky, FPF CEO
As Senior Vice President of Policy, John Verdi supervises FPF’s policy portfolio, which advances FPF’s agenda on a broad range of issues. Verdi came to FPF in 2016 after serving as the Director of Privacy Initiatives for the National Telecommunications and Information Administration, where he crafted policy recommendations for the U.S. Department of Commerce and the Obama Administration on technology and innovation. Verdi previously oversaw the Electronic Privacy Information Center’s litigation program as General Counsel.
In Gabriela Zanfir-Fortuna’s new role as Vice President for Global Privacy, she will lead FPF’s work on global privacy developments, advising on EU data protection law and policy and working with FPF’s offices in Europe and Asia Pacific, as well as partners around the world. Zanfir-Fortuna gained years of experience in EU and international privacy law while working for the European Data Protection Supervisor in Brussels, as well as the Article 29 Working Party.
As Vice President of Youth and Education Privacy, Amelia Vance advises policymakers, academics, companies, and schools on child and student privacy laws and best practices; oversees the Student Privacy Compass website; and convenes stakeholders to ensure the responsible use of child and student data. She is a regular speaker at privacy and education conferences in the U.S. and abroad, has testified before Congress, spoken on child and education privacy issues for the Federal Trade Commission and U.S. Department of Education, and is part of the group of experts reviewing the OECD revised recommendations on the protection of children online.
Over her five years at FPF, Vance has grown the youth and education privacy project to 12 full-time staff. She came to FPF after serving as the Director of Education Data and Technology at the National Association of State Boards of Education. Prior to that role, she was a legal fellow at the Institute of Museum and Library Services and the Family Equality Council, an intern at the White House, the State Department, and the Office of Congressman Sander Levin, and a Field Organizer for the 2008 Obama campaign.
Five Things Lawyers Need to Know About AI
By Aaina Agarwal, Patrick Hall, Sara Jordan, Brenda Leong
Note: This article is part of a larger series focused on managing the risks of artificial intelligence (AI) and analytics, tailored toward legal and privacy personnel. The series is a joint collaboration between bnh.ai, a boutique law firm specializing in AI and analytics, and the Future of Privacy Forum, a non-profit focusing on data governance for emerging technologies.
Behind all the hype, AI is an early-stage, high-risk technology that creates complex grounds for discrimination while also posing privacy, security, and other liability concerns. Given recent EU proposals and FTC guidance, AI is fast becoming a major topic of concern for lawyers. Because AI has the potential to transform industries and entire markets, those at the cutting edge of legal practice are naturally bullish about the opportunity to help their clients capture its economic value. Yet to act effectively as counsel, lawyers must also be vigilant of the very real challenges of AI. Lawyers are trained to respond to risks that threaten the market position or operating capital of their clients. However, when it comes to AI, it can be difficult for lawyers to provide the best guidance without some basic technical knowledge. This article shares some key insights from our shared experiences to help lawyers feel more at ease responding to AI questions when they arise.
I. AI Is Probabilistic, Complex, and Dynamic
There are many different types of AI, but over the past few decades, machine learning (ML) has become the dominant paradigm.[1] ML algorithms identify patterns in recorded data and apply those patterns to new data to try to make accurate decisions. This means that ML-based decisions are probabilistic in nature. Even if an ML system could be perfectly designed and implemented, it is statistically certain that at some point it will produce a wrong result. All ML systems incorporate probabilistic statistics, and those systems can make incorrect classifications, recommendations, or other outputs.
ML systems are also fantastically complex. Contemporary ML systems can learn billions or more rules from data and apply those rules on a myriad of interacting data inputs to arrive at an output recommendation. Embed that billion-rule ML system into an already-complex enterprise software application and even the most skilled engineers can lose track of precisely how the system works. To make matters worse, ML systems decay over time, losing their use-case fitness based on their initial training data. Most ML systems are trained on a snapshot of a dynamic world as represented by a static training dataset. When events in the real world drift, change, or crash (as in the case of COVID-19) away from the patterns reflected by that training dataset, ML systems are likely to become wrong more frequently and cause issues that require legal and technical attention. Even in the moment of the “snapshot,” there are other qualifiers for the reliability, effectiveness, and appropriateness of training data. How it’s collected, processed, and labeled all bear on whether it is sufficient to inform an AI system in a way fit for a given application or population.
While all this may sound intimidating, an existing regulatory framework addresses many of these basic performance risks. Large financial institutions have been deploying complex decision-making models for decades, and the Federal Reserve’s model risk management guidance (SR 11-7) lays out specific process and technical controls that are a useful starting point for handling the probabilistic, complex, and dynamic characteristics of AI systems. Most commercial AI projects would benefit from some aspect of model risk management, whether it’s being monitored by federal regulators or not. Lawyers at firms and in-house alike, who find themselves needing to consider AI-based systems, would do well to understand options and best practices for model risk management, starting with understanding and generalizing the guidance offered by SR 11-7.
II. Make Transparency an Actionable Priority
Immense complexity and unavoidable statistical probabilities in ML systems makes transparency a difficult task. Alas, parties deploying—and thereby profiting from—AI can nonetheless be held liable for issues relating to a lack of transparency. Governance frameworks should include steps to promote transparency, whether preemptively or as required by industry- or jurisdiction-specific regulations. For example, the Equal Credit Opportunity Act (ECOA) and the Fair Credit Reporting Act (FCRA) mandate customer-level explanations known as “adverse action notices” for automated decisions in the consumer finance space. These laws set an example for the content and timing of notifications relating to AI decisions that could adversely affect customers, as well as establish the terms of an appeals process against those decisions. Explanations that include a logical consumer recourse process dramatically decrease risks associated with AI-based products and help prepare organizations for future AI transparency requirements. New laws, like the California Privacy Rights Act (CPRA) and the proposed EU AI rules for high-risk AI systems, will likely require high levels of transparency, even for applications outside of financial services.
Some AI system decisions may be sufficiently interpretable to nontechnical stakeholders today, like the written adverse action notices mentioned above, in which reasons for certain decisions are spelled out in plain English to consumers. But oftentimes the more realistic goal for an AI system is to be explainable to its operators and direct overseers.[2]
The import of a system that’s not fully understood by its operators is that it is much harder to identify and sufficiently mitigate risks. One of the best strategies for promoting transparency, particularly in light of the challenges around “black-box” systems that are unfortunately common in the US today, is to rigorously pursue best practices with respect to AI system documentation. This is good news for lawyers who are adept in the skill and attention to detail that is required to institute and enforce such documentation practices. Standardized documentation of AI systems, with emphasis on development, measurement, and testing processes, is crucial to enable ongoing and effective governance of AI systems. Attorneys can help by creating templates for such documentation and by assuring that documented technology and development processes are legally defensible.
III. Bias is a Major Problem—But Not the Only Problem
Algorithmic bias can generally be thought of as outputs of an AI system that exhibits an unjustified differential treatment between two groups. AI systems learn from data, including its biases, and can perpetuate that bias on a massive scale. The racism, sexism, ageism, and other biases that permeate our culture also permeate the data collected about us and in turn the AI systems that are trained on that data.
On a conceptual level, it is important to note that although algorithmic bias often reflects unlawful discrimination, it does not constitute unlawful discrimination per se. Bias also includes the broader category of unfair or unexpected inequitable outcomes. While these may not amount to illegal discrimination of protected classes, they may still be problematic for organizations, leading to other types of liability or significant reputational damage. And unlawful algorithmic bias puts companies at risk of serious liability under cross-jurisdictional anti-discrimination laws.[3] This highlights the need for organizations to adopt methods that test for and mitigate bias on the basis of legal precedent.
Because today’s AI systems learn from data generated—in some way—by people and existing systems, there can be no unbiased AI system. If an organization is using AI systems to make decisions that could potentially be discriminatory under law, attorneys should be involved in the development process alongside data scientists. Those anti-discrimination laws, while imperfect, provide some of the clearest guidance available for AI bias problems. While data scientists might find the stipulations in those laws burdensome, the law offers some answers in a space where answers are very hard to find. Moreover, academic research and open-source software addressing algorithmic bias is often published without serious consideration of applicable laws. So, organizations should take care to ensure that their code and governance practices with respect to identifying and mitigating bias have a firm basis in applicable law.
Organizations are also at risk of over-indexing on bias while overlooking other important types of risk. Issues of data privacy, information security, product liability, and third-party risks, as well as the performance and transparency problems discussed in previous sections, are all critical risks that firms should, and eventually must, address in bringing robust AI systems to market. Is the system secure? Is the system using data without consent? Many organizations are operating AI systems without clear answers to these questions. Look for bias problems first, but don’t get outflanked by privacy and security concerns or an unscrupulous third party.
IV. There Is More to AI System Performance Than Accuracy
Over decades of academic research and countless hackathons and Kaggle competitions, demonstrating accuracy on public benchmark datasets became the gold standard by which a new AI algorithm’s quality is measured. ML performance contests such as the KDD Cup, Kaggle, and MLPerf have played an outsized role in setting the parameters for what constitutes “data science.”[4] These contests have undoubtedly contributed to the breakneck pace of innovation in the field. But they’ve also led to a doubling-down on accuracy as the yardstick by which all applied data science and AI projects are measured.
In the real world, however, using accuracy to measure all AI is like using a yardstick to measure the ocean. It is woefully inadequate to capture the broad risks associated with making impactful decisions quickly and at web-scale. The industry’s current conception of accuracy tells us nothing about a system’s transparency, fairness, privacy, or security, in addition to presenting a limited representation of what the construction of “accuracy” itself claims to measure. In a seemingly shocking admission, forty research scientists added their names to a paper demonstrating that accuracy on test data benchmarks often does not translate to accuracy on live data.
What does this mean for attorneys? Attorneys and data scientists need to work together to create more robust ways of benchmarking AI performance that focus on real-world performance and harm. While AI performance and legality will not always be the same, both professions can revise current thinking to imagine performance beyond high scores for accuracy on benchmark datasets.
V. The Hard Work Is Just Beginning
Unfortunately at this stage of industry and development, there are few professional standards for AI practitioners. Although AI has been the subject of academic research since at least the 1950s, and it has been used commercially for decades in financial services, telecommunications, and e-commerce, AI is still in its infancy throughout the broader economy. This too presents an opportunity for lawyers. Your organization probably needs AI documentation templates, policies that govern the development and use of AI, and ad hoc guidance to ensure different types of AI systems comply with existing and near-future regulations. If you’re not providing this counsel, technical practitioners are likely operating in the dark when it comes to their legal obligations.
Some researchers, practitioners, journalists, activists, and even attorneys have started the work of mitigating the risks and liabilities posed by today’s AI systems. Indeed, there are statistical tests to detect algorithmic discrimination and even hope for future technical wizardry to help mitigate against it. Businesses are beginning to define and implement AI principles and make serious attempts at diversity and inclusion for tech teams. And laws like ECOA, GDPR, CPRA, the proposed EU AI regulation, and others form the legal foundation for regulating AI. However, technical mitigation attempts still falter, many fledgling risk mitigations have proven ineffective, and the FTC and other regulatory agencies are still relying on general antitrust and unfair and deceptive practice (UDAP) standards to keep the worst AI offenders in line. As more organizations begin to entrust AI with high-stakes decisions, there is a reckoning on the horizon.
Author Information
Aaina Agarwal is Counsel at bnh.ai, where she works across the board on matters of business guidance and client representation. She began her career as a corporate lawyer for emerging companies at a boutique Silicon Valley law firm. She later trained in international law at NYU Law, to focus on global markets for data-driven technologies. She helped to build the AI policy team at the World Economic Forum and was a part of the founding team at the Algorithmic Justice League, which spearheads research on facial recognition technology.
Patrick Hall is the Principal Scientist and Co-Founder of bnh.ai, a DC-based law firm specializing at the intersection of AI and data analytics. Patrick also serves as visiting faculty at the George Washington University School of Business. Prior to co-founding bnh.ai, Patrick led responsible AI efforts at the high-profile machine learning software firm H2O.ai, where his work resulted in one of the world’s first commercial solutions for explainable and fair machine learning.
Sara Jordan is Senior Researcher of AI and Ethics at the Future of Privacy Forum. Her profile includes privacy implications of data sharing, data and AI review boards, privacy analysis of AI/ML technologies, and analysis of the ethics challenges of AI/ML. Sara is an active member of the IEEE Global Initiative on Ethics for Autonomous and Intelligent Systems. Prior to working at FPF, Sara was faculty in the Center for Public Administration and Policy at Virginia Tech and in the Department of Politics and Public Administration at the University of Hong Kong. She is a graduate of Texas A&M University and University of South Florida.
Brenda Leong is Senior Counsel and Director of AI and Ethics at the Future of Privacy Forum. She oversees development of privacy analysis of AI and ML technologies, and manages the FPF portfolio on biometrics and digital identity, particularly facial recognition and facial analysis. She on privacy and responsible data management by partnering with stakeholders and advocates to reach practical solutions for consumer and commercial data uses. Prior to working at FPF, Brenda served in the U.S. Air Force. She is a 2014 graduate of George Mason University School of Law.
Disclaimer: bnh.ai leverages a unique blend of legal and technical expertise to protect and advance clients’ data, analytics, and AI investments. Not all firm personnel, including named partners, are authorized to practice law.
[1] Commentators have often used the image of Russian nesting (Matryoshka) dolls to illustrate these relationships: AI includes machine learning, and machine learning, in turn, includes deep learning. Machine learning and deep learning have risen to the forefront of commercial adoption of AI in applications areas such as fraud detection, e-commerce, and computer vision. See, e.g., The Definitive Glossary of Higher Mathematical Jargon, MATH VAULT (last accessed Mar. 4, 2021), https://mathvault.ca/math-glossary/#algo; Eda Kavlakoglu, AI vs. Machine Learning vs. Deep Learning vs. Neural Networks: What’s the Difference?, IBM BLOG (May 27, 2020), https://www.ibm.com/cloud/blog/ai-vs-machine-learning-vs-deep-learning-vs-neural-networks.
[2] In recent work by the National Institute for Standards and Technology (NIST), interpretation is defined as a high-level, meaningful mental representation that contextualizes a stimulus and leverages human background knowledge. An interpretable AI system should provide users with a description of what a data point or model output means. An explanation is a low-level, detailed mental representation that seeks to describe some complex process. An AI system explanation is a description of how some system mechanism or output came to be. See David A. Broniatowski, Psychological Foundations of Explainability and Interpretability in Artificial Intelligence (2021), https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=931426.
[3] For example, The Equal Credit Opportunity Act (ECOA), The Fair Credit Reporting Act (FCRA), The Fair Housing Act (FHA), and regulatory guidance, such as the Interagency Guidance on Model Risk Management (Federal Reserve Board, SR Letter 11–7). The EU Consumer Credit Directive, Guidance on Annual Percentage Rates (APR), and General Data Protection Regulation (GDPR) serve to provide similar protections for European consumers.
[4] “Data science” tends to refer to the practice of using data to train ML algorithms, and the phrase has become common parlance for companies implementing AI. The term dates back to 1974 (or perhaps further), coined then by the prominent Danish computer scientist Peter Naur. Data science, despite the moniker, is yet to be fully established as a distinct academic discipline.
Event Report: From “Consent-Centric” Frameworks to Responsible Data Practices and Privacy Accountability in Asia Pacific
On September 16, the Asia-Pacific office of the Future of Privacy Forum (FPF) held its first event following its launch in August 2021. This event was hosted by the Personal Data Protection Commission (PDPC) of Singapore during the very popular “Personal Data Protection week” (PDP Week 2021).
The theme of the event was Exploring trends: From “consent-centric” frameworks to responsible data practices and privacy accountability in Asia Pacific, and it is that of a larger project, carried out jointly by FPF with the Asian Business Law Institute (ABLI) across 14 Asian jurisdictions. The event was also co-organized by ABLI and FPF in the context of a cooperation agreement which was signed by the two organisations in August 2021.
This post summarizes the discussions in the two stellar panels featuring regulators, thought leaders, and practitioners from across the region, and highlights key takeaways:
Consent requirements which apply to the collection and processing of personal data, “notice & choice”, exceptions and alternatives to those requirements, combined, form an area where regulatory coherence is most needed in Asia-Pacific (APAC). Over-reliance on consent has led to the development of a “tick-the-box approach” to data protection, consent fatigue, and unnecessary compliance costs due to contradictory requirements in Asia Pacific.
Modern data protection laws should shift the onus of data protection from users to organizations, by promoting an accountability-based approach to data protection over a “consent-centric” one. Different avenues may be used to rebalance consent and privacy accountability in APAC, including through concepts such as legitimate interests, compatible uses and equivalent notions.
Making consent meaningful again in APAC can happen in a variety of ways, which include winding back the range of circumstances in which consent is sought; requiring consent only where it can be given thoughtfully, sparingly and with understanding; supporting enhanced transparency and consent through UX and UI design, with due attention brought to the different needs and literacy levels of users.
Harmonization is illusory in the face of Asia’s extreme diversity, but a bottom-up approach to convergence can work in the context of regional cooperation.
1. Repositioning consent requirements in APAC’s fragmented data protection landscape
Dr. Clarisse Girot, Director of FPF Asia Pacific and ABLI Senior Fellow, opened the discussion by explaining that the issue of a comparative look at “consent” requirements across the region was chosen as a key topic following suggestions from a vast network of stakeholders. Feedback showed that consent requirements which apply to the collection and processing of personal information, the “Notice & Choice” principle, exceptions and alternatives to those requirements, combined, form an area where regulatory coherence is most needed in the Asia Pacific (APAC) region.
In practice, the cumulative application of consent requirements for data processing in the region has led to the development of a “tick-the-box approach” to data protection in many jurisdictions. However, in APAC as elsewhere, overreliance on consent as a lawful ground by organisations has led to a general consent fatigue and unnecessary compliance costs, due to contradictory requirements.
An agreement is therefore forming across Asian jurisdictions that modern data protection laws should shift the onus of data protection from users to organizations, by promoting an accountability-based approach to data protection over a “consent-centric” one. This triggers a need to relativize the role of consent and to bring it back to the place which had been initially assigned to it by the very first data protection frameworks — namely, as one among many elements in a regulatory ecosystem which generally seeks to balance the role and interests of individuals, the responsibility of organizations, and broader social and societal interests with regard to the processing of personal data.
The main goal of the workshop was therefore to identify similar discussions that are taking place in multiple jurisdictions in APAC and to explore the possibilities of convergence among them. The discussion will also feed a joint comparative study with recommendations for convergence on consent and related data protection requirements, which will be published jointly by FPF and ABLI before the end of the year.
Both panels were composed of data protection professionals from different APAC jurisdictions and disciplines. Each speaker contributed with an original and expert point of view that could help identify commonalities, pathways for interoperability between Asian data protection frameworks, and concrete solutions to provide meaningful data protection to individuals — with or without consent.
Such reflections and recommendations are particularly timely at a time when key jurisdictions in Asia, including India, Indonesia, Thailand, Vietnam, Hong Kong SAR, Malaysia, Australia, are adopting new data protection frameworks or amending their laws, and new laws or major amendments recently came into force in jurisdictions like Thailand, Korea, New Zealand, China, or Singapore.
2. Rebalancing consent and privacy accountability
The title of the first panel was “Switching from a consent-centric approach to privacy accountability: a comparative view of APAC data protection laws”.
The panel was moderated by Yeong Zee Kin, Assistant Chief Executive, Infocomm Media Development Authority (IMDA), and Deputy Commissioner, PDPC, Singapore, with the inputs of Peter Leonard, Principal and Director at Data Synergies, Sydney, Takeshige Sugimoto, Managing Director at S&K Brussels, Tokyo, Shinto Nugroho, Chief Public Policy and Government Relations at Gojek, Jakarta, and Marcus Bartley-Johns, Asia Regional Director, Government Affairs and Public Policy at Microsoft, Singapore.
The goal of this first panel was to identify commonalities and pathways for interoperability between Asian data protection frameworks with regard to balancing the protection of individuals, accountability, and broader social and societal interests. This includes the role of consent, lawful grounds to process personal data, and/or other privacy principles in jurisdictions which do not contain provisions on “lawfulness” of processing.
The most important points highlighted during the discussion were the following:
2.1 How to achieve convergence across APAC’s fragmented and diverse landscape?
As an introductory note, Yeong Zee Kin stressed that APAC jurisdictions take different approaches towards privacy and data protection, but also that their laws are in different stages of developments (e.g., Japan and South Korea have had privacy laws for a long time, while Singapore, Philippines and Malaysia are more recent players). One may add that data protection or privacy laws follow different structures and not all modelled on EU GDPR, hence some key provisions (e.g. on “lawfulness” of data processing) have no equivalent in other jurisdictions.
A challenge which is endemic in APAC is therefore to identify a common ground in order to achieve convergence, while respecting the different inspirations and the particular culture that are enshrined in each jurisdiction’s privacy laws.
This raises a key question for participants, which is whether APAC stakeholders should aim for harmonisation or for more targeted actions of convergence, for instance through the mutual recognition of specific legal standards.
2.2 Over-reliance on consent and need for alternatives
Speakers highlighted that APAC-based organisations tend to overly rely on consent, even in cases where another solution or legal basis would be available and more appropriate. The potential consequence of such a practice is the erosion of the value of consent.
A view expressed by Peter Leonard and shared across the panel was that consent, “informational self-determination”, or “citizen self-management” of privacy settings, remains important. However, anyone should only be expected to self-manage what is realistically manageable by them. The need is felt to address both the frequency of consent requests and reduce the level of noise in privacy policies and collection notices, as well as rethink the role of privacy policies and collection notices.
Among “noise reduction measures”, he specifically cited appropriately targeted exceptions, whether through legitimate interests, industry codes or standards, class exemptions by regulators, or new generic concepts such as “compatible data practices”, in such a way that the control of individuals over their personal data is not adversely affected. As a baseline, moving away from consent requires recognizing the importance of concepts like “reasonableness” or “fairness” to support the alternative requirements of data protection laws.
Unambiguous express consent should remain necessary for categories of processing that create higher risk of privacy harm to individuals, in particular for manifestly sensitive data, including data about children, processing which directly contradict individuals’ rights and interests, or cannot reasonably be expected by them. This may also tie in with the concept of “no-go zones” as it has been developing in Canada, and which has gained some popularity in Australia.
2.3 Varying approaches and interpretations in different jurisdictions: Japan, Indonesia, Vietnam
Another point raised by the moderator and panellists was that material differences in the protections awarded by legal systems in APAC countries may hinder the path towards harmonisation. There is therefore a need to better understand how each law works before proposing solutions for convergence, so that they can be meaningful for all.
Takeshige Sugimoto commented on the “consent by default” situation which currently prevails in Japan. He noted that the Japanese data protection law (APPI) does not have a “legitimate interest” legal basis, but that—contrary to a common belief—it does not take a consent-centric approach either. Rather it permits processing of personal data based on the “business necessity” ground, as long as the data subject may reasonably expect the intended further usage of his or her data. The boundaries of permissible processing under APPI are therefore similar to GDPR, even without “legitimate interests” as a legal basis. In its adequacy decision on Japan, the European Commission actually states that the Japanese system also ensures that personal data is processed lawfully and fairly through the purpose limitation principle.
Sugimoto also mentioned the Japanese Personal Information Protection Commission (PPC Japan) guidelines, which list limited cases where consent must be sought, while pointing to other areas which are open to other legal bases and authorisation from the PPC. In other words, in his view there would be no significant difference, in practice, between what GDPR considers legitimate interests-based processing, and what APPI considers lawful processing.
Shinto Nugroho presented the situation in ASEAN from the perspective of Gojek, Indonesia’s first decacorn and SuperApp, with operations in Indonesia, Vietnam, Singapore, Thailand, and Philippines. Nugroho’s particular focus was on the challenges of operationalizing consent in times of crisis, like in the current Covid-19 pandemic. She noted that in its current state Indonesia’s data protection legislation is quite consent-centric, but that the draft Data Protection Bill to be soon adopted by the Parliament of Indonesia mentions consent as only one of seven available lawful grounds for processing personal data (others including contract, performance of a legal obligation and legitimate interests).
Nugroho welcomed this development. She explained how consent as a legal ground is neither always practical for controllers nor protective for individuals, and in fact sometimes even harmful for citizens. For instance, in Indonesia, out of 170 million inhabitants roughly 160 millions are eligible for vaccination against Covid. Gojek has secured massive vaccination slots from the governments, namely for its drivers who are frontliners. However, the government requires that everyone be registered in the public vaccination system first, for which consent is required. But not everyone has access to the Internet or has the literacy required to get registered; moreover, the vaccination register itself is work in progress. Securing 100% opt-in consent from millions of drivers to be registered in the scheme is not only going to slow down the process, but the drivers are also going to miss the notification, or fail to complete their registration. In such cases, for Gojek the most adequate legal basis to get the drivers registered would be its “legitimate interests” as an employer, together with clear purposes, and adequate transparency over mere consent. The consideration that drivers are exposed to a high risk of contamination at a time when the epidemic is hitting the country should override the need to obtain consent.
Lastly, Nugroho mentioned the ongoing discussions on the future Data Protection Decree of Vietnam, to be adopted imminently. The Decree does not provide for a legitimate interests basis, but at least similarly allows controllers to collect and process data on grounds other than consent (such as security, when permitted under the law, and research). Discussions on convergence must therefore factor in the fact that APAC data protection laws can vary even between neighbouring countries which have commonly drawn inspiration from similar sources (primarily EU GDPR) to draft their future comprehensive data protection frameworks.
2.4 Transparency & choice as trust enablers
Marcus Bartley-Johns welcomed the fact that the discussions enabled to introduce nuances in the conversation, because “making consent meaningful again” is a journey and for that we must avoid binary approaches (“for, or against consent”). He also concurred with Takeshige Sugimoto that laws and regulations can go in one direction, but business practices and embedded behaviors can go in another, and these variations are a key part of the discussion around consent.
Bartley-Johns shared a few data points on what consent means in the region. In 2019, Microsoft ran a survey on 6300 consumers across Asia on consumer perception of trust; 53% of the persons surveyed said that they had had a negative trust experience related to privacy when using a digital service in the region. Younger people reported a higher share of negative experience, and more than half of those said they would switch services if their trust was breached. Bartley-Johns added that the fact that consumers have reasons to be wary should be acknowledged, one of those reasons being the excessive difficulty for individuals to find out and understand how their data is being collected and used.
Another data point is in relation to the use of the privacy dashboard which enables Microsoft users globally to see and control their activity data including location, search, browsing data across multiple services. 51 million unique visitors have used that dashboard since its launch in May 2018 (19 million people in 2020). Japan, China, Australia, India and Korea feature in the top 20 markets from where users have been using the dashboard. In other words, the speaker stated that Microsoft’s experience shows that consumers wish to know what personal data is collected about them and exercise their options and rights when they are given an opportunity to have their say.
Following up on this point, Peter Leonard added that transparency plays a double role: on the one hand, it allows individuals to know how their data are being used, while at the same time provides safeguards against deceptive and manipulative statements by organisations, where appropriate “do only what you say” laws are in place at a local level.
2.5 “Legitimate interests” in context
On the whole, all the speakers expressed their support for the development of the concept of legitimate interests or equivalent concepts in APAC laws. The adoption across more privacy laws of alternative grounds for processing personal data, notably legitimate interests, is one of the potential areas for strengthening privacy regulatory coherence in the region. Microsoft for instance has advocated for this necessity in a recent policy paper calling for strengthening privacy regulatory coherence in Asia.
Speakers noted that a problem in APAC of increased reliance upon legitimate interests as an alternative for consent is that lists of legitimate interests are varied and jurisdiction-specific. This means that entities operating across borders and seeking a common denominator in their privacy policies and requests for consent will continue to be incentivised to overly rely upon consent, unless they are given some certainty on where the lawmakers and regulators are likely to use this notion. Convergence can be strengthened by the adoption of regulatory guidelines on implementing this approach and information sharing on their implementation.
Peter Leonard contributed by stating that, to make the legitimate interests lawful ground work in APAC, there could be a need for a mutual recognition scheme in the region of the differing definitions and approaches to the legitimate interests. According to him, this will not lead to absolute convergence, but will allow reaching a compromise that takes stock of local legal systems and cultures in diverse Asia. Failing this, we will have data controllers who will keep using consent as a common denominator by default.
In the view of Takeshige Sugitomo, having a compilation of use cases that clarify whether legitimate interests or consent would be the most appropriate legal basis in each case, would help achieve a more holistic regional approach. This could lead to international consensus on specific use cases which would be more efficient than awaiting joint regulatory guidance which might take years to be issued.
Marcus Bartley-Johns suggested that it would be valuable to check if the consensus that emerged from this panel could emerge in the regional and global regulatory community. This is important as more regulations and guidance have emerged in the last months in Asia which tends to make transparency or consent requirements even more prescriptive. In this respect, there would be real value in obtaining practical guidance from regulators on these issues, like PDPC has done, with indicative examples, use cases and scenarios that will give a basis for a more holistic approach to balancing consent and other approaches in the region.
Seconding the comments by Sugitomo and Bartley-Johns, Yeong Zee Kinindicated that one of the sources of inspiration for drafting PDPC’s guidelines on legitimate interests in the recently amended PDPA was the FPF’s report on legitimate interests in the EU, which provides a compilation of guidance or decisions by regulators and court cases where the scope of the legitimate interests lawful ground was clarified. He suggested that the right way forward would probably be to identify real world examples and use cases where a regional or global consensus can be reached on situations where we do not need consent, and the next step will be for regulators to start contextualizing the end result depending on their respective legal systems (necessity, reasonableness, legitimate interests, contractual necessity, vital interests, etc.).
The moderator suggested that FPF and other stakeholders contribute to building this library of “legitimate interests”, and that regulators could do their part by going out to their local industry and looking for such use cases. Subscribing to a remark by Peter Leonard, however, he acknowledged that in the broad spectrum of different cultures and histories in Asia, complete harmonization is not realistic. In contrast, taking a practical bottom-up approach to convergence might get us somewhere and we should seek to build on consensus as and when we find them, for instance bilaterally, between like-minded partners, and maybe more slowly, on a regional level.
3. Making consent meaningful (again)
The title of the second panel was “Shaping choices in the digital world: how consent can become meaningful again”. The panel was moderated by Rajesh Sreenivasan, Head, Technology Media and Telecoms Law Practice, Rajah & Tann Singapore LLP. It further included interventions by Anna Johnston, Principal, Salinger Privacy (Sydney), Malavika Raghavan, Visiting Faculty, Daksha Fellowship and FPF Senior Fellow for India, Rob van Eijk, FPF Europe Managing Director and Edward Booty, Founder and CEO of reach52.
Rajesh Sreenivasan started by saying that the problem with consent lies not on the concept itself but on the way this legal ground has been used for processing personal data. Especially in APAC, where multiple jurisdictions have very different approaches, he mentioned that obtaining meaningful consent requires answering two questions first: 1) Meaningful consent for whom: for the data subject or for the organisation?, and 2) Meaningful how? Additionally, the moderator openly asked participants whether in their view it was more pressing to make consent meaningful or to build alternative models for fair data processing, as consent might have become redundant in today’s context, at the speed at which data is being used.
3.1 Are current online consent-seeking practices fair?
Anna Johnston kicked off by supporting a burden shift, away from individuals and onto organisations, when it comes to consent standards. According to her, consent has almost lost its true meaning because it has been so over-used as a promise — in her own words, it has become like “your cheque is in the mail”!
The situation in Australia as she sees it is that consent is over-relied on, but also under-enforced. There is guidance from the Australian Privacy Commissioner (the OAIC) and there is case-law to back up that guidance, that consent in Australian law is similar to the GDPR: it cannot be bundled up with other things, it cannot be included in mandatory Terms and Conditions, in a Privacy Policy, it cannot even be “opted out” – consent as a lawful basis on which to collect, use or disclose personal information has to be the customer’s clear “opt in” choice, made freely, separate from all other choices. However the law is under-enforced, and so it is still very common to see business practices which follow a model of “bury the customer in fine print and make them agree to something we know they won’t even read”, and then claim that the customer has consented to something.
Surveys conducted by the OAIC actually suggest that only 20% of Australians feel confident that they understand privacy policies when they are actually reading them. Recently the Australian consumer and competition regulator, the ACCC, has called out this kind of power imbalance and these kinds of behaviours from the Big Tech platforms, and recommended that the Privacy Act should be amended, to make the standard required for consent much clearer in the law.
3.2 The boundaries of consent’s role
Overall, speakers agreed that there is a need to “make consent meaningful again”, primarily by winding back the range of circumstances in which consent is sought by organisations. Consent should only be required, and sought, where it can be given thoughtfully, sparingly and with understanding. Consent is only real consent where an individual has a real choice [note: an increasing number of data protection laws in Asia recognize the concept of “free and unbundled consent”]. A discussion is needed about when requiring consent is sensible, and how to ensure that capabilities of individuals to control their privacy settings are not compromised by any changes in consent requirements.
Winding back such requirements, to improve data privacy, may sound both radical and counter-intuitive. However, over both sessions an agreement has been formed that processing without consent should only be recommended if the processing is aligned with the ordinary expectations or direct interests of data subjects, and without ever overriding a requirement for transparency.
Anna Johnston thus opined that there should be clear distinction between business activities that require or do not require consent. For example, activities that are outside customers’ expectations should require consent (eg. asking someone to join a research project), whereas the same would not apply to unobjectionable, fair and proportionate activities (such as including an individual in a customer database) nor to others with public interest backing. She concluded by adding that there should also be a list of activities that are prohibited even if consent is given, including cases of profiling children for marketing purposes.
In his presentation, FPF’s Manager for Europe Dr. Rob van Eijk concurred and added that a lot of the debate on the consequences of the datafication of society has been around limiting the collection of data but also on its further use. Consent is one of the ways in which to regulate these two “gateways”, and if we look at that there are multiple ways in which we can ensure that everyone is on board. In practice, however, much of the burden is on the users to read and comprehend what is being put forward. This aspect was the key focus of this year’s Dublin Privacy Symposium organized by FPF, entitled Designing for Trust: Enhancing Transparency & Preventing User Manipulation.
An important point made during the symposium is that organisations should be proactive in increasing transparency from a design perspective so as to present users with a real choice and encourage them to make deliberate decisions. Understandability, how people read through the information, for instance, can be tested through technology in the online space. Another important point is that organizations should ask themselves whether they should be collecting all the envisaged data in the first place (in line with the minimisation principle).
They must also take active steps to prevent user manipulation not only in designing consent solutions (for instance through cookie banners), but also when they process data through machine learning algorithms. Finally, the question of vulnerable groups should be factored in the design of UX/UI (“have we left any groups behind?”). A lot can be done to make things more understandable. And this of course leads to the question of the extent to which the expression of choice can be embedded in the technology.
3.3 Dealing with users with different needs and literacy levels in APAC
The Asia Pacific region is a region of contrasts, especially in terms of literacy levels, including financial and health literacy, due among other reasons to different educational levels and the wide linguistic variety that exist in some countries.
FPF’s Malavika Raghavan made comments and shared findings issued from her research and extensive field work done in India, to explore how the mental models of internet users in India impact these discussions on consent, with a particular focus on the financial sector (eg. loan applications). She underlined the importance of understanding the context of non-Western users, particularly new generations of users in Asia, before even aiming at the design of laws and practices for obtaining meaningful online consent.
For instance, Raghavan pointed to surveys that showed that many mainstream Indian users – i.e. modest-earning individuals from primarily rural areas – do not understand the differences between their mobile phones, the internet, online services and allied services like payment platforms, because they exclusively use them on their phones. Understanding this reality (how users have never used a computer, but only mobile phones with preloaded apps, free allowances, etc.) is key to start thinking about designing consent, or even policymaking around consent.
However, literacy is not necessarily a barrier, and it is not related to digital skills: highly proficient digital users might not be literate, and reciprocally. Moreover, a large number of Indian families often share their mobile devices, which means that consent in those scenarios should be considered given for a group of individuals rather than separate individuals: this mental model is very far from the mental models of a designer or policymaker. Asking for one-to-one consent in such circumstances might not make sense. But however disadvantaged, individuals still have strong ideas about how their data may be shared.
The limitations of consent have been analysed by Raghavan in particular in her work on the Data Empowerment and Protection Architecture (DEPA) and the Consent Layer developed by Indiastack, which seeks to enable secure and effective sharing of personal data with third party institutions in India by using the concept of “consent managers”. Raghavan highlighted in her work how cognitive limitations operate on individuals’ decision-making about their personal data and how the threat of denial of service can make “taking consent” a false choice. To be effective, therefore, such systems must be supported by strong accountability systems and access controls that operate independent of consent. Relying solely on consent is not a good idea, as a wealth of data protection and consumer protection thinking has shown that consent is necessary but not sufficient for data protection.
Moreover, the panellist concluded, coders and digital platform designers should consider users’ perceptions, literacy and context when setting up online services. The law alone cannot fix what has been broken by technology. This, according to Raghavan, is particularly important in a jurisdiction where the highest judicial instances have recognized privacy as a fundamental right (such as India) and where users have strong ideas and reasonable expectations about how digital data flows occur. In said exercise, unbundling ancillary consent-needing data processing from online services’ terms and conditions should be front and center.
Edward Bootythen shared his experience as CEO of Reach52, a social enterprise and a growth-focused start-up that provides accessible and affordable healthcare for the 52% of the world without access to health services, with 5 key markets in Cambodia, Philippines, India, Indonesia, and Kenya.
Reach52 uses technology and community outreach to widen access to health services while simultaneously lowering their costs. Booty explained that his company is still small, but has accumulated a lot of sensitive data in the multiple countries in which they operate. He specifically shared about his experience in collecting health data and profiling residents for providing better care in remote rural communities in Philippines and Cambodia, and uncovering data-driven insights to inform more targeted, effective access to healthcare solutions. Although it is sometimes disheartening that some users do not care, not having legitimate consent from users in a data-driven business model constituted a risk to his start-up. Furthermore, reach52 still believes that it must help the people who use their services understand their rights around data collection and use, regardless of their education and literacy levels. Booty explained how consent was sought from individuals who provided their data for such a purpose, using video, visuals, and progressive disclosure, paying attention to the way terms are explained, and consent gained, so as not to fall short for people with low literacy and education levels. For this, support was obtained from Facebook accelerator and IMDA.
A specific challenge explained by Booty is that local and national government authorities were then coming to reach52 to obtain access to the datasets for a variety of purposes, notably to manage different humanitarian crises. The speaker shared that, as pressure from those authorities mounted, the organisation started working on ways to get more meaningful and granular consent from individuals for each of the needs that their data could serve. This involved engaging designers to deliver simple flyers with information to individuals about what could happen to their data after its collection, as well as about their data-related rights. The process included testing with different age groups to make the message intelligible for a wide audience.
3.4 How UX and UI can support enhanced transparency and consent
During the session, several times the idea was brought up that designers, and the improvement of the user experience and user interface (UX/UI), have an essential role to play in improving the regulation of architectures of choice.
In recent years, more academics and data protection regulators have underlined the fundamental role which UX/UI design can play for user empowerment and that design and interfaces must now make part of the compliance analysis. Universally-accepted icons could be a possible solution to improve intelligibility, said Anna Johnston. In her presentation, she argued that web designers should try to think with the mind of a user, by considering useful evidence and guidance on how to better design privacy notices, such as the UK Government’s piece on better privacy notice design.
Various ideas for improving privacy notices are modelled on successful designs used in safety messaging (like traffic light indicators), and product labelling (such as star ratings and nutrition labels). But this form of notice still does not work at scale. Anna Johnston expressed the view that the most innovative idea she has seen in this space comes from Data61, which is an arm of the Australian Government that has proposed machine-readable icons, based on Creative Commons icons from copyright law – these are universally agreed, legally binding, clear and machine-readable.
This latter suggestion was echoed by the findings of FPF’s Dublin Privacy Symposium on manipulative design practices, which were outlined by Dr. Rob van Eijk during the session. According to him, the Symposium’s speakers explained that providers should encourage users to make deliberate decisions online by avoiding so-called “dark patterns”, consider the needs of vulnerable groups (such as visually impaired or colour-blind users) and the best way of informing users where data collection devices do not have visual or audio interfaces (eg. IoT). Van Eijk added that cookie walls as they are developing in Europe may be a radical solution, as they prevent users from accessing content unless they agree to pay a fee or accept online tracking.
Conclusion
Commissioner Raymund Liboro, National Privacy Commissioner of Philippines, delivered the concluding remarks of the workshop.
To support the work of the FPF and ABLI and the discussions of the day, Commissioner Liboro evoked a topical case in the Philippines. In late August, his office ordered the take-down of money lending apps from the Google Play Store to sanction the practice of some online lending platforms. Such platforms harvested excessive information from their users without legitimate purpose through the use of unreasonable and unnecessary apps permissions, including saving and storing their clients’ contact list and photo gallery ostensibly to evaluate their creditworthiness. Yet an applicant’s creditworthiness may be determined through other lawful and reasonable means. Moreover, these apps have also been the subject of more than 2000 complaints of unauthorized use of personal data that resulted in harassment and shaming of borrowers before persons in their mobile devices’ contact list to collect debts.
Such behaviors and practices cannot be considered acceptable because users have supposedly given their “legitimate consent” to them, which was the companies’ first line of defence. This, Commissioner Liboro said, combined with the privacy paradox, urges the data protection community to reconsider the current regulatory paradigm which operates in Asia and globally. As policymakers now regulate in hyperscale – with encompassing laws coming up in China, India, Indonesia, Thailand, and so many countries in ASEAN hopping on, impacting millions of data subjects –, the current dependence on consent and paper compliance should be replaced with accountability and added onus on organisations to ensure and demonstrate compliance. Privacy accountability is a compelling force, and accountable organisations foster trust and thrive, said the Commissioner.
The workshop set the scene and informed the discussion around consent and accountability in the APAC jurisdictions. All participants agreed on the need to reconsider the use of the consent legal ground in the region. The datification of society as well as the global dimensions of privacy and data protection promise to urge policy makers to aim for convergence, while respecting the legal culture and approach of each separate jurisdiction.
Commissioner Liboro concluded the event by expressing his appreciation to everyone who participated in the discussions, and reminded the participants that this conversation aims at setting the foundations of a collective response that will benefit the privacy ecosystem in the Asia-Pacific region.
The next steps of the FPF ABLI project will be announced soon.
Brain-Computer Interfaces: Privacy and Ethical Considerations for the Connected Mind
View a report by FPF and IBM report focusing on BCI privacy and ethics here.
Introduction
Brain-computer interfaces (BCIs) are a prime example of an emerging technology that is spawning new avenues of human-machine interaction. Communication interfaces have developed from the keyboard and mouse to touchscreens, voice commands, and gesture interactions. As computers become more integrated into the human experience, new ways of commanding computer systems and experiencing digital realities have trended in popularity, with novel uses ranging from gaming to education.
Defining BCIs and Neurodata
BCIs are computer-based systems that directly record, process, analyze, or modulate human brain activity in the form of neurodata that is then translated into an output command from human to machine. Neurodata is data generated by the nervous system, composed of the electrical activities between neurons or proxies of this activity. When neurodata is linked, or reasonably linkable, to an individual, it is personal neurodata.
BCI devices can be either invasive or non-invasive. Invasive BCIs are installed directly into—or on top of—the wearer’s brain through a surgical procedure. Today, invasive BCIs are mainly used in the health context. Non-invasive BCIs rely on external electrodes and other sensors or equipment connected to the external surface of the head or body, for collecting and modulating neural signals. Consumer-facing BCIs primarily use various non-invasive methods, including headbands.
Key Applications and Top-of-Mind Privacy and Ethical Challenges
Some BCI implementations raise few, if any, privacy issues. For example, individuals using BCIs to control computer cursors might not not reveal any more personal information than typical mouse users, provided BCI systems promptly discard cursor data. However, some uses of BCI technologies raise important questions about how laws, policies, and technical controls can safeguard inferences about individuals’ brain functions, intents, or emotional states. These questions are increasingly salient in light of the expanded use of BCIs in:
Gaming – where BCIs augment existing gaming platforms and offer players new ways to play using devices that record and interpret their neural signals.
Employment – where BCIs monitor workers’ engagement to improve safety during high-risk tasks, alert workers or supervisors of dangerous situations, modulate workers’ brain activity to improve performance, and provide tools to more efficiently complete tasks.
Education – where BCIs can track student attention, identify students’ unique needs, and alert teachers and parents of student learning progress.
Neuromarketing – where marketers incorporate the use of BCIs to intuit consumers’ moods, and to gauge product and service interest.
Military – where governments are researching the potential of BCIs to help rehabilitate soldiers’ injuries and enhance communication.
It is important for stakeholders in this space to delineate between the current and near future uses and the far-distant notions depicted by science fiction creators. The realistic view of capabilities is necessary to credibly identify urgent concerns and prioritize meaningful policy initiatives. While the potential uses of BCIs are numerous, BCIs cannot at present or in the near future “read a person’s complete thoughts,” serve as an accurate lie detector, or pump information directly into the brain.
As BCIs evolve and are more commercially available across numerous sectors, it is paramount to understand the real risks such technologies pose. BCIs raise many of the same risks posed by home assistants, medical devices, and wearables, but implicate new and heightened risks associated with privacy of thought, resulting from recording, using, and sharing a variety of neural signals. Risks include, but are not limited to:
Collecting, and potentially sharing, sensitive information related to individuals’ private emotions, psychology, or intent;
Combining neurodata with other personal information to build increasingly granular and sensitive profiles about users for invasive or exploitative uses, including behavioural advertising;
Making decisions that significantly impact patients, employees, or students based on information drawn from neurodata (with potential but distinct risks if the conclusions are accurately, or inaccurately drawn);
Security breaches compromising patient health and individual safety and privacy;
A lack of meaningful transparency and personal control over individuals’ neurodata; and
Surveilling individuals based on the collection of sensitive neurodata, especially from historically and heavily surveilled communities.
These technologies also raise important ethical questions around fairness, justice, human rights, autonomy, and personal dignity.
A Mix of Technical and Policy Solutions Is Best for Maximizing Benefits While Mitigating Risks
To promote privacy-protective and ethical uses of BCIs, stakeholders should adopt technical measures including but not limited to:
Providing hard on/off controls whenever possible;
Providing granular user controls on devices and in companion apps for managing the collection, use, and sharing of personal neurodata;
Operationalizing best practices for security and privacy when storing, sharing, and processing neurodata including:
Encrypting sensitive personal neurodata in transit and at rest; and
Embracing appropriate security measures to combat bad actors.
Stakeholders should also adopt policy safeguards including but not limited to:
Rethinking transparency, notice, terms of use, and consent frameworks to empower users with a baseline of BCI literacy around the collection, use, sharing, and retention of their neurodata;
Engaging IRBs, corporate review boards, ethical oversight, and other independent review mechanisms to identify and mitigate risks;
Facilitating participatory and inclusive community input prior to and during BCI development and rollout;
Creating dynamic technical, policy, and employee training standards to account for the gaps in current regulation; and
Promoting an open and inclusive research ecosystem by encouraging the adoption, where possible, of open standards for the collection and analysis of neurodata and the sharing of research data under open licenses and with appropriate safeguards in place.
Conclusion
Because the neurotechnology space is especially future-facing, developers, researchers, and policymakers will have to create best practices and policies that consider existing concerns and strategically prioritize future risks in ways that balance the need for proactive solutions while mitigating misinformation and hype. BCIs will likely augment and complicate many existing technologies that are currently on the market, and privacy professionals will have to stay abreast of recent developments to protect this quickly growing space.
*Image courtesy of Gerd Altmann from Pixabay
Call for Nominations: 12th Annual Privacy Papers for Policymakers
The Future of Privacy Forum invites privacy scholars and authors with an interest in privacy issues to submit finished papers to be considered for FPF’s 12th annual Privacy Papers for Policymakers Award. This award provides researchers with the opportunity to inject ideas into the current policy discussion, bringing relevant privacy research to the attention of the U.S. Congress, federal regulators, and international data protection agencies.
The award will be given to authors who have completed or published top privacy research and analytical work in the last year that is relevant to policymakers. The work should propose achievable short-term solutions or new means of analysis that could lead to real world policy solutions.
FPF is pleased to also offer a student paper award for students of undergraduate, graduate, and professional programs. Student submissions must follow the same guidelines as the general PPPM award.
We encourage you to share this opportunity with your peers and colleagues. Learn more about the Privacy Papers for Policymakers program and view previous year’s highlights and winning papers on our website.
FPF will invite winning authors to present their work at an annual event with top policymakers and privacy leaders in February 2022 (date TBD). FPF will also publish a printed digest of the summaries of the winning papers for distribution to policymakers in the United States and abroad.
Learn more and submit your finished paper by October 15th, 2021. Please note that the deadline for student submissions is November 5th, 2021.
