Algorithms in the public sector: four case studies of ADMS in Spain

Case Studies

Download report

AI, algorithms and automation have increasingly become part of the daily vocabulary of the general public. As use of AI increases, there is a growing need to understand exactly how the digitalisation of public services and the use of automation can have a direct impact on peoples’ lives – a wrong tick on an online application can exclude a group from getting the aid they are eligible for, a miscalculation in a police assessment can be a deciding factor of life or death, an unnoticed alert may result in misuse of taxpayer money.

The more societies delegate decisions to algorithms, the more we need to understand how these decisions are made and what information is used to make them.

This report builds on previous work of Digital Future Society on automated decision-making systems (ADMS). It continues the effort to understand the social impact that derives from their use. Moreover, it seeks to bridge the knowledge from different experts from academia, the public sector and civil society organisations to illustrate the complexities behind each tool.

The report explores four case studies of ADMS used by the Spanish public sector that have drawn attention in the media.

BOSCO – Determines eligibility for a nationwide energy bill subsidy

RisCanvi – Calculates the risk of inmate recidivism in Catalonia

VioGén – Predicts the risk of gender violence for the national police force

SALER – Anticipates potential cases of corruption in Valencia

The more societies delegate decisions to algorithms, the more we need to understand how these decisions are made and what information is used to make them.

Tanya Álvarez, Researcher at Digital Future Society Think Tank

RisCanvi is a tool used in the Catalonian penitentiary system to predict the probability of recidivism and violent recidivism of the inmate population.

Algorithms in the public sector: four case studies of ADMS in Spain

Four case studies of ADMS used by the Spanish public sector that have drawn attention in the media

Bosco

In 2018, over half a million families’ applications to the energy bill subsidy were rejected. This called the attention of Civio, a non-profit organisation, whose primary mission is to hold the public administration accountable. According to Civio, the mass rejection lies, in part, to the government’s use of a software to determine eligibility of applicants. This software is called BOSCO. BOSCO was created by the public administration in 2017 to assess whether a user is entitled to the energy bill subsidy.

Civio’s director, Eva Belmonte, requested the source code used from the government to determine the basis of the mass rejections; however, the request was denied on copyright grounds. Civio, later appealed and is currently awaiting the final ruling in the case.

Cases like BOSCO are part of a larger trend of digitising welfare services. These tools are employed to improve the efficiency and cost-effectiveness. However, issues such as lack of transparency, diffuse liability and barriers to equity, put to question whether BOSCO respects the rights of its beneficiaries.

Read the report to find more about BOSCO.

RisCanvi

RisCanvi is a tool used in the Catalonian penitentiary system to predict the probability of recidivism and violent recidivism of the inmate population. As part of a generation of risk assessment instruments, tools like RisCanvi are increasingly used for sentencing, correction, and parole decisions worldwide.

Created in 2010, RisCanvi (named from the Catalan words for risk and change) helps prison management allocate resources and determine a treatment plan for the inmates. These assessments are recommended as advisory tools, and their predictive efficacy according to some experts has its limits.

Tools like RisCanvi have been under the spotlight in the US and UK, as they may perpetuate systemic bias, and consequently lead to discrimination. While advocates of RisCanvi stress that there is no way to eliminate bias, they highlight that professionals that use these tools need to be well-equipped to identify and counteract these biases.

Lawyer, Nuria Monfort, expert in criminal law, notes that societies must understand the political and ideological influence of these tools, and that they are far from being objective and scientific, that they prioritise efficiency over the human rights of inmates. She says, “We have a huge prison population… a limited number of professionals and they have to issue a number of resolutions. So we are going to have a machine. It’s got nothing to do with the social function, the collective responsibility, with the individual’s conduct, it’s about getting through the pile.”

Learn more about RisCanvi in the report.

VioGén

When an incidence of gender violence is reported to the police (which could be presented by a relative, witness or law enforcement agent – it need not be the victim) this initiates an administrative process in which the police officer opens an investigation and fills in an online form with the victim. This protocol is called, VioGén, a system used by Spanish police to help estimate the risk of recividism in gender violence.

The form includes 35 indicators under five categories, that range from history of violence in the relationship and characteristics of the aggressor, which the police can check off if they apply to the victim’s case. Depending on the risk assigned to each case, officers come up with a “custom security plan”.

VioGén has been generally perceived as a step in the right direction, as it has made gender violence more visible and provides a protocol for agents to follow. However, there are underlying issues, that question the effectiveness of the tool. Does the system adequately address the needs of victims? Are agents empowered to change the level of risk, to ensure more protection for the victims?

Find out more about VioGén in the report.

Saler

In the early 2000s, Valencia, the fourth largest autonomous community in Spain, had a bad reputation for political corruption. In its steps to regain public trust, the Valencian parliament passed a law for an Early Warning System to identify and prevent possible cases of bad practices. SALER is an ADMS that cross references public data to anticipate potential cases of corruption in the public administration. This tool is the first of its kind developed in Spain; however, it has not been fully implemented in the public administration.

Alfons Puncel Chornet, former undersecretary of the Ministry of Transparency from 2015 to 2019, was the father of the tool. He recounts the moment he was inspired to create it… A construction company was asking for reimbursement of a deposit they had paid to guarantee completion of a project. The comptroller overseeing the public works contract realised the company was asking for the deposit back before the project was finished. They had been using a paperwork management process and I saw the possibility to digitise the system so it could alert public servants of similar cases immediately.

SALER is at an initial stage incorporating a series of databases. One of the main challenges SALER has faced is the slow digital transformation of the public sector. Not only are there technical challenges stalling the full implementation of SALER, but the original advocates for the system are also no longer leading the implementation of the tool. The use of data-driven tools is increasingly seen by governments as an effective and efficient solution to manage public funds.

How would SALER work in practice? What are the main concerns regarding ADMS like SALER?

Read more on SALER, in the report.