AI4Justice

Problem Context

The use of AI in the justice domain is on the rise. The National Police in the Netherlands use a nationwide predictive policing tool. In the United States, they are using a system to predict recidivism, and together with the Netherlands Public Prosecution Service (Dutch: ‘Openbaar Ministerie’) we have developed and are now testing a risk assessment tool to determine the risk of fleeing of convicted criminals. However, it is much easier to develop such risk assessment tools than it is to develop them correctly. In this project, we focus on the question: What is needed to build a legitimate, fair and correctly working (also in the long run) risk assessment tool? Additionally, how can you keep users both trustful and critical?

Solution

To assess the long-term correctness of risk assessment tools, we have performed experiments to evaluate the risk of self-fulfilling prophecy and tunnel vision. Both are generic problems caused by one-sided feedback learning. For example, if you detain someone for life, you will never be able to assess their recidivism. Furthermore, we performed experiments together with the Netherlands Public Prosecution Service to assess if users are able to identify flaws in the reasoning of the risk assessment tool, when they are presented with the result of its risk assessment. For example, do users start to over-trust a system if it is constantly providing assessments that match their own?

Results

Through our experiments, we have identified several key factors of success and failure for risk assessment tools. Though generic, these key factors are often extra important in the justice domain. They present technical, socio-technical, and social solutions to problems such as the self-fulfilling prophecy, tunnel vision, and appropriate trust calibration for users.