AI Verify: Singapore’s AI Governance Testing Initiative Explained

In recent months, global interest in AI governance and regulation has expanded dramatically. Many identify a need for new governance and regulatory structures in response to the impressive capabilities of generative AI systems, such as OpenAI’s ChatGPT and DALL-E, Google’s Bard, Stable Diffusion, and more. While much of this attention focuses on the upcoming EU AI Act, there are other significant initiatives around the world proposing different AI governance models or frameworks.

This blog post covers “AI Verify,” Singapore’s AI governance testing framework and toolkit, announced in May 2022. Our analysis has three key parts. First, we summarize Singapore’s overall approach to AI governance and the key initiatives that the Singapore Government released regarding AI governance, prior to the launch of AI Verify. Second, we explain the key components of AI Verify.. Finally, as we approach the anniversary of AI Verify’s roll-out, we explore what the future may hold for AI Verify and Singapore’s approach to AI governance and regulation. Briefly, the key takeaways are:

• Singapore adopts a light-touch approach to AI governance and regulation, with the Model AI Governance Framework establishing a guide for AI governance in the private sector.

• AI Verify, an AI governance testing framework and toolkit, was launched in May 2022. While still in a pilot phase, it represents Singapore’s efforts to further develop global discourse on AI governance and regulation, respond to growing demand for trustworthy AI systems, and foster interoperability in global AI regulatory frameworks.

• AI Verify is based on a testing framework of internationally accepted AI governance principles, and a testing toolkit that companies can use in testing their own AI systems. Rather than defining ethical standards, AI Verify provides verifiability by allowing AI system developers and owners to demonstrate their claims about the performance of their AI systems. 

• To succeed, AI Verify will likely require greater recognition and adoption. This, in turn, depends on factors like cost, convincing stakeholders of its value, and its relevance to and synergy with international regulatory frameworks. 

1. Singapore’s overall approach to AI governance

In Singapore’s high-level strategy for AI, the National AI Strategy (NAIS), the country announced it aims to be “at the forefront of development and deployment of scalable, impactful AI solutions,” hoping to cement itself as “a global hub for developing, test-bedding, deploying, and scaling AI solutions.” Among the five “ecosystem enablers” identified in the strategy to increase AI adoption is the development of a “progressive and trusted environment” for AI  – one that strikes a balance between innovation and minimization of societal risks. 

To create this “progressive and trusted environment,” Singapore has adopted so far a light-touch and voluntary approach to AI regulation. This approach recognizes two practical realities about Singapore’s AI ambitions. First, the Singapore Government sees AI as a key strategic enabler in developing its economy and improving the quality of life of its citizens. This explains why Singapore is not taking a heavy-handed approach in regulating AI lest it stifles innovation and investment. Second, given its size, Singapore is aware it is also likely to be a price-taker rather than a price-setter as AI governance discourse, frameworks and regulations develop globally. Thus, rather than introducing new AI principles afresh, the current approach is to “take the world where it is, rather than where it hopes the world to be.”

Before the release of AI Verify in 2022, Singapore’s approach to AI regulation – as overseen by the Personal Data Protection Commission of Singapore (PDPC) – had three pillars: 

1. The Model AI Governance Framework (Model Framework). 
2. The Advisory Council on the Ethical Use of AI and Data (Advisory Council).
3. The Research Programme on the Governance of AI and Data Use (Research Program). 

As we aim to highlight the substantive aspects of Singapore’s AI regulatory approach, the following paragraphs will focus on the Model Framework. 

The Model Framework

The Model Framework, first launched at the World Economic Forum Annual Meeting (WEF) in 2019, is a voluntary and non-binding framework that guides organizations in the responsible deployment of AI solutions at scale, noting that this framework does not concern the development phase of these technologies. As a guide, the Model Framework sets out practical recommendations for AI deployments for private sector entities, as the public sector’s use of AI is governed by internal guidelines and AI and data governance toolkits. The Model Framework is billed as a “living document,” as it is meant to evolve through future editions alongside technological and societal developments. The Model Framework is also technology-, industry-, scale- and business-model agnostic. 

Substantively, the Model Framework is guided by two fundamental principles to promote trust and understanding in AI. First, organizations using AI in decision-making should ensure that the decision-making process is explainable, transparent and fair. Second, AI systems should be human-centric: the protection of human well-being and safety should be primary considerations in designing, developing and using AI.

The Framework translates these guiding principles to implementable practices in four key areas of an organization’s decision-making and technology-development processes:

(a) Internal governance structures and measures;

(b) Determining the level of human involvement in AI-augmented decision-making;

(c) Operations management; and

(d) Stakeholder interaction and communication.

The table prepared below shows a summary of some suggested considerations, practices, and measures falling under each of these key areas.

Internal governance structures and measures	Human involvement in AI-augmented decision-making	Operations management	Stakeholder interaction and communication
Clear roles and responsibilities Use existing or set up new corporate governance and oversight processes Ensure staff are appropriately trained and equipped Internal controls Monitoring and reporting system to ensure awareness at appropriate level of management Manage personnel risk Periodic reviews	Appropriate level of human intervention Use probability-severity of harm matrix to determine level of human involvement Incorporate corporate and societal values in decision-making	Good data accountability  Data lineage, quality, accuracy, completeness, veracity, relevance, integrity, etc. Minimizing bias in data / model Heterogeneous datasets Separate training, testing and validation datasets Repeatability assessments, counterfactual testing, etc. Regular review and tuning	General disclosure Being transparent when AI is used in products and services Use simple language, with communication appropriate to the audience, purpose and context. Increased transparency Information on how AI decisions may affect individuals Feedback channels Avenues for feedback and review of decisions

Other initiatives accompanying the Model Framework

When Singapore released the second edition of the Model Framework at the WEF in 2020, it was released alongside two other documents: the Implementation and Self-Assessment Guide for Organisations (ISAGO) and the Compendium of Use Cases (Compendium – Volume 1 and Volume 2). The ISAGO is a checklist helping organizations assess the alignment of their AI governance processes with the Model Framework. The Compendium provides real-life examples of the adoption of the Model Framework’s recommendations across various sectors, use cases, and jurisdictions. 

Collectively, the Model Framework and its suite of accompanying documents anchored and outlined substantive thinking on AI regulation in Singapore. These initiatives led to Singapore winning a United Nations World Summit on the Information Society Prize in 2019, recognizing its efforts as a frontrunner in AI governance. 

2. AI Verify in a Nutshell

January 2020 marked a turning point for global discourse on AI regulation. On January 17, 2020, a leaked white paper from the European Commission brought international attention to the increasing possibility of government regulation of AI technology. In February 2020, the European Commission formally issued a White Paper on Artificial Intelligence, which, among other things, set out plans to create a regulatory framework for AI. In the following months, the European Commission began to make available drafts of a forthcoming AI Act. For the first time, a major government was making a serious attempt to introduce substantive rules to horizontally regulate the development and use of AI systems. Due to the expected extraterritorial nature of the AI Act, companies developing AI systems outside of Europe could potentially be covered by the new law. 

These developments influenced thinking about the future of Singapore’s AI regulatory and governance landscape. While the PDPC maintained its voluntary and light-touch approach to AI regulation, it acknowledged a future in which AI faces heightened oversight. The PDPC seemed to also be mindful of growing consumer awareness and demand for trustworthiness from AI systems and developers, a need for international standards on AI to benchmark and assess AI systems against regulatory requirements, and an increasing need for interoperability of AI regulatory frameworks. With these in mind, Singapore began developing the framework that eventually coalesced into AI Verify.

What is AI Verify?

Launched by the Infocomm Media Development Authority (IMDA) – a statutory board under the Singapore Ministry of Communications and Information, and the PDPC, AI Verify is an AI governance testing framework and toolkit. By using AI Verify, organizations are able to use a combination of technical tests and process-based checks to conduct a voluntary self-assessment of their AI systems. The system, in turn, helps companies attempt to objectively and verifiably demonstrate to stakeholders that their AI systems have been implemented in a responsible and trustworthy manner. 

Given that AI testing methodologies, standards, metrics and tools continue to develop, AI Verify is also currently at a “Minimum Viable Product” (MVP) stage. This has two implications. First, there are several technical limitations to the MVP version, and limitations to the types and size of AI models or datasets that it can test or analyze. Second, it is expected that AI Verify will evolve as AI testing capabilities mature. 

The four aims for developing an MVP version of AI Verify are:

(a) First, IMDA hopes that organizations are able to use AI Verify to determine performance benchmarks for their AI systems, and demonstrate these claimed benchmarks to stakeholders such as consumers and employees, thereby helping organizations enhance trust.

(b) Second, given that it was developed with various AI regulatory and governance frameworks, as well as common trustworthy AI principles in mind, AI Verify seeks to help organizations find commonalities across various global AI governance frameworks and regulations. IMDA is also continuing to engage regulators and standards organizations to map AI Verify’s testing framework onto established frameworks. These efforts are aimed at allowing businesses to operate and offer AI-enabled products and services in multiple markets, while allowing Singapore to act as a hub in AI governance and regulatory testing.

(c) Third, as organizations trial AI Verify and use its testing framework, IMDA will be able to collate industry practices, benchmarks and metrics. These can facilitate input into the development of international standards on AI governance, considering Singapore is participating in global AI governance platforms such as the Global Partnership on AI and ISO/IEC JTC1/SC 42, to contribute valuable perspectives towards the development of international standards on AI governance.

(d) Fourth, IMDA hopes AI Verify will allow Singapore to create a local AI testing community, consisting of AI developers and system owners (who are seeking to test AI systems), technology providers (who are developing AI governance implementation and testing solutions), advisory service providers (specializing in testing and certification support), and researchers (who are developing testing technologies, benchmarks and practices). 

It is also important to clarify several potential misconceptions about AI Verify. First, AI Verify is not an attempt to define ethical standards. It also does not attempt to classify AI systems with a clear bright line. Instead, AI Verify provides verifiability, as it allows AI system developers and owners to demonstrate their claims about the performance of their AI systems. Second, an organization’s use of AI Verify does not guarantee that tested AI systems are free from risks or biases, nor that they  are completely “safe” or “ethical.” Third, AI Verify is intended to preclude organizations from unintentionally divulging sensitive information from their AI systems (such as their underlying code or training data); one key safeguard – AI Verify will be used by AI system developers and owners themselves to conduct self-testing. This allows the organization’s data and models to remain within the organization’s operating environment. 

How does AI Verify work?

AI Verify consists of two parts. The first is a Testing Framework, which references eleven internationally accepted AI ethics and governance principles, grouped into five pillars. The second is a Toolkit that organizations use to execute technical tests and to record process checks from the Testing Framework.

AI Verify’s Testing Framework

The five pillars and eleven principles in AI Verify’s Testing Framework, as well as their expected assessment, are:

Pillar	Principles	Assessment method(s)
Transparency on Use of AI and AI systems: This pillar is about disclosing to individuals about AI use in a technological system, so that they can be aware and make informed choices on whether to use the AI-enabled system.	Transparency: Providing appropriate information to individuals impacted by AI systems.	Assessed through process checks of documentary evidence (e.g., company policy and communication collaterals) providing appropriate information to individuals who may be impacted by the AI system. The information includes (subject to the need to avoid compromising IP, safety, and system integrity): the use of AI in the system, its intended use, limitations, and risk assessments.
Understanding how an AI model reaches a decision: This pillar is about allowing individuals to understand the factors contributing to an AI model’s output, while also ensuring output consistency and accuracy in similar conditions.	Explainability: Understanding and interpreting the decisions and output of an AI system.	Assessed through a combination of technical tests and process checks. Technical tests are conducted to identify factors contributing to an AI model’s output. Process checks include verifying documentary evidence of considerations given to the choice of models, such as rationale, risk assessments, and trade-offs of the AI model.
	Repeatability / reproducibility: Ensuring consistency in AI output by being able to replicate an AI system, either internally or through a third party.	Assessed through process checks of documentary evidence, including evidence of AI model provenance, data provenance, and use of versioning tools.
Ensuring safety and resilience of the AI system: This pillar is aimed at helping individuals understand that the AI system will not cause harm, is reliable, and will perform according to its intended purpose even despite encountering unexpected input.	Safety: Ensuring safety by conducting impact / risk assessments, and ensuring that known risks have been identified / mitigated.	Assessed through process checks of documentary evidence of materiality assessment and risk assessment, including how known risks of the AI system have been identified and mitigated.
	Security: Ensuring the cyber-security of AI systems.	Presently NA
	Robustness: Ensuring that the AI system can still function despite unexpected input.	Assessed through a combination of technical tests and process checks. Technical tests attempt to assess if a model performs as expected even when provided with unexpected inputs. Process checks include verifying documentary evidence, review of factors that may affect the performance of AI model, including adversarial attacks.
Ensuring fairness: This pillar is about evaluating whether the data used to train the AI model is sufficiently representative, and testing to ensure that the AI system will not unintentionally discriminate.	Fairness: Avoiding unintended bias, ensuring that the AI system makes the same decision even if a certain attribute is changed, and ensuring that the data used to train the model is representative.	Assessing the mitigation of unintended discrimination through a combination of technical tests and process checks. Technical tests check that an AI model does not produce biased results based on protected or sensitive attributes specified by the system owner, by checking the model output against the ground truth. Process checks include verifying documentary evidence that there is a strategy for the selection of fairness metrics aligned with the desired outcomes of the AI system’s intended application; and the definition of sensitive attributes are consistent with legislation and corporate values.
	Data governance: Ensuring the source and quality of data by adopting good data governance practices when training AI models.	Presently NA
Ensuring proper (human) management and oversight of the AI system: This pillar is about assessing human accountability and control in the development and/or deployment of AI systems, and whether the AI system is aimed at beneficial purposes for general society.	Accountability: Ensuring proper management oversight during AI system development.	Assessed through process checks of documentary evidence, including evidence of clear internal governance mechanisms for proper management and oversight of the AI system’s development and deployment.
	Human agency and oversight: Ensuring that the AI system is designed in a way that will not diminish the ability of humans to make decisions.	Assessed through process checks of documentary evidence that the AI system is designed in a way that will not reduce human’s ability to make decisions or to take control of the system. This includes defining the role of humans in the oversight and control of the AI system such as human-in-the-loop, human-over-the-loop, or human-out-of-the-loop.
	Inclusive growth, societal and environmental well-being: Ensuring beneficial outcomes for people and the planet.	Presently NA

The actual Testing Framework has several key components:

(a) Definitions: The Testing Framework provides easy-to-understand definitions for each of the AI principles. For example, explainability is defined as the “ability to assess the factors that led to (an) AI system’s decision, its overall behavior, outcomes and implications.”

(b) Testable criteria: For each principle, a set of testable criteria is provided. These criteria are a mix of technical and/or non-technical (e.g. processes, procedures, or organizational structures) factors that contribute to the achievement of the desired outcomes of that governance principle.

Using the example of explainability, two testable criteria are provided. A developer can run explainability methods to help users understand the drivers of the AI model. A developer can also demonstrate a development preference for AI models that can explain their decisions or that are interpretable by default.  

(c) Testing process: For each testable criteria, AI Verify provides the processes or actionable steps to be carried out. The steps could be quantitative (such as statistical or technical tests) or qualitative (such as producing documented evidence during process checks). 

For explainability, a technical test could involve empirically analyzing and determining feature contributions to a model’s output. A process-based test would be to document the rationale, risk assessments, and trade-offs of an AI model. 

(d) Metrics: These are quantitative or qualitative parameters used to measure, or provide evidence for, each testable criterion.

Using the explainability example above, the metric for determining feature contributions could examine contributing features of a model output as obtained from a technical tool (such as SHAP and LIME). The process-based metric could be documented evidence of evaluations when choosing the final model, such as risk assessments and trade-off weighing exercises.

(e) Thresholds (where applicable): Where available, the Testing Framework will provide recognized values or benchmarks for selected metrics. Such values or benchmarks could be defined by regulators, industry associations, or other recognized standard-setting organizations. For the MVP model of AI Verify, thresholds are not provided given the rapid evolution of AI technologies, their use cases, as well as methods to test AI systems. Nevertheless, as the space of AI governance matures and the use of AI Verify increases, IMDA intends to collate and develop context-specific metrics and thresholds to be added to the Testing Framework.

AI Verify’s Toolkit

While AI Verify’s Toolkit is currently only available to organizations that have successfully registered for AI Verify’s MVP program, IMDA describes the Toolkit as a “one-stop” tool for organizations to conduct technical tests. Specifically, the Toolkit packages widely-used open-source testing libraries. Such tools include SHAP (Shapley Additive ExPlanations) for explainability, the Adversarial Robustness Toolkit for robustness, and AIF360 and Fairlearn for fairness.

Users of AI Verify can deploy the Toolkit within their internal environment. Users will be guided by a user interface to navigate the testing process. For example, the Toolkit contains a “guided fairness tree” for users to identify fairness metrics relevant for their use case. At the end, AI Verify produces a summary report that helps system developers and owners interpret test results. For process checks, the report provides a checklist stating the presence or otherwise of document evidence specified in the Testing Framework. The test results are then packaged into a Docker® container for easy deployment. 

3. Conclusion

When IMDA released AI Verify, the wave of interest in generative AI seen today had yet to materialize. With the wave currently upon us, interest in demonstrating governance, testability and trustworthiness of AI systems has grown significantly. Initiatives like AI Verify appear poised to respond to this interest.

Singapore has previously demonstrated its ability to contribute to global discourse and thought leadership on AI governance and regulation, namely through the Model Framework. The stakes for AI Verify are high, but so is the global need for such an initiative. To succeed, AI Verify will likely require greater recognition and adoption. This depends on several factors. First, the tool’s accessibility is critical: AI-driven organizations hoping to use AI Verify will need to be able to access it at little or no cost. Second, convincing organizations of its value is key. This will require IMDA to demonstrate that AI Verify is technically and procedurally sound, that it can be effectively used on more (and newer) kinds and sizes of AI models and data sets, that it does not impinge on commercial sensitivities around proprietary AI models and datasets. Third, and perhaps most importantly, it must remain relevant to international regulatory frameworks. IMDA will need to ensure that AI Verify can continue to help organizations address and interoperate within key emerging global AI regulatory frameworks, such as the EU AI Act, Canada’s AI and Data Act, the NIST AI Risk Management Framework in the US, and even Singapore’s own Model Framework.