AI Oversight lab
The objective of the AI Oversight Lab is to enable public entities to develop and use AI tools, while adhering to ethical principles and respecting human rights, and to facilitate oversight bodies in their auditing role.
AI and Machine learning based tools are more and more available and used both by public and private bodies, influencing decisions and processes in many key sectors of the society. It is becoming therefore extremely important and urgent to develop policies and practices to properly assess and evaluate AI systems under the lens of the Trustworthy AI principles, allowing the developers, the public entities, and the citizenship to answer questions about the fairness, the security, the compliance to privacy rules, and the social and ethical impact of a tool.
In the first year, the research conducted in the three lab rooms has been developed next to the concrete use case offered by the municipality of Nissewaard; TNO has conducted an independent evaluation of a commercial ML tool used by the municipality to detect possible abuses of social benefits. This evaluation showed that, despite the fact that the developers clearly put effort to satisfy the principles of Trustworthy AI, the ML algorithm used in the tool was still not robust enough to comply with the European guidelines. Based on this, the municipality of Nissewaard has decided to stop using the tool.
Guided by this specific use case, the research of Lab 2 has focused mainly on identifying guidelines to develop robust and reproducible AI, presented in a handbook. Both Lab 1 and Lab 3 have conducted an extensive review of existing literature, frameworks and tools to evaluate and audit AI, identifying the current state of the art, and future options and challenges. Finally, a first prototype for a visual and interactive tool for (internal) evaluation of AI systems is proposed. This prototype is based on the literature and the experience of different stakeholders. A list of yes/no questions guides the auditors to analyse the AI system and spot possible problems; where available, information about quantitative measures and tools is provided. At the end of the evaluation a report is generated including the answers, notes and comments of the auditor, a visual representation of the auditing flow; possible weaknesses of the system are also highlighted. At this stage of the development only questions about “Diversity, non-discrimination and fairness” are implemented.
- The report from the TNO consultation about a ML decision support tool used by the Nissewaard municipality.
- I. Georgieva , C. Lazo, T. Timan, A. F. van Veenstra. "A reflection and a gap analysis based on recent frameworks and practical experience", under review (available upon request - email@example.com).
- Marianne Schaaphok, Project manager, e-mail: firstname.lastname@example.org
- Merle Beaujon, Project manager, e-mail: email@example.com