Explained: What a Dutch ruling means for digital identification schemes worldwide
Last week, a Dutch district court ruled against an identification mechanism called SyRI (System Risk Indicator), because of data privacy and human rights concerns.
Digital ID systems are being rolled out at a fast pace in places like Kenya, Philippines, Nigeria, Mexico, and more.
In a first anywhere in the world, a court in the Netherlands recently stopped a digital identification scheme for reasons of exclusion. This has a context for similar artificial intelligence systems worldwide, especially at a time when identity, citizenship and privacy are pertinent questions in India.
What was the scheme?
Last week, a Dutch district court ruled against an identification mechanism called SyRI (System Risk Indicator), because of data privacy and human rights concerns. While the Hague district court found using new technology to control fraud was acceptable, it held SyRI was too invasive and violative of the privacy guarantees given by European Human Rights Law as well as the EU’s General Data Protection Regulation.
The Dutch Ministry of Social Affairs developed SyRI in 2014 to weed out those who are most likely to commit fraud and receive government benefits. Legislation passed by Dutch Parliament allowed government agencies to share 17 categories of data about welfare recipients such as taxes, land registries, employment records, and vehicle registrations with a private company.
The company, called “The Intelligence Agency”, used an algorithm to analyse data for four cities and calculate risk scores. The selective rollout was conducted in low-income and immigrant neighborhoods, which have a higher number of beneficiaries. Elevated risk scores were sent to relevant government arms, which stores these on government databases for a maximum of two years. The government, in that time period, could open an investigation on the targeted person.
What were the arguments in court?
After taking into account community concerns, civil society groups and NGOs launched a legal attack on this case of algorithmic governance. Legal criticism mounted, alleging that the algorithm would begin associating poverty and immigrant statuses with fraud risk.
The Dutch government defended the programme in court, saying it prevented abuse and acted as only a starting point for further investigation instead of a final determination. The government also refused to disclose all information about how the system makes its decisions, stating that it would allow gaming of the system.
The court found that opaque algorithmic decision-making puts citizens at a disadvantage to challenge the resulting risk scores. The Netherlands continuously ranks high on democracy indices.
The Ministry of Social Affairs in the country released a statement stating it will study the ruling, not declaring a complete removal of their system.
📢 Express Explained is now on Telegram. Click here to join our channel (@ieexplained) and stay updated with the latest
How relevant is this for India?
Similar to the Supreme Court’s Aadhaar judgment setting limits on the ID’s usage, the Hague Court attempted to balance social interest with personal privacy. However, the Aadhaar judgment was not regarding algorithmic decision-making; it was about data collection.
The ruling is also an example of how a data protection regulation can be used against government surveillance. The court ruled that SyRI was violative of principles of transparency and data minimisation laid out in their General Data Protection Regulation. Other European tech initiatives have been stalled by the regulation, including a facial recognition system on students in Sweden and France.
India’s pending data protection regulation, being analysed by a Joint Select Committee in Parliament, would give broad exemptions to government data processing in its current form. Some members of the committee have decided to take up government surveillance in the upcoming deliberative meetings.
India’s proposed regulation is similar to the US in the loopholes that could be potentially exploited. Hence, attempts to ban facial recognition in cities such as San Francisco have not had the same success as attempts in Europe. A system somewhat paralleling the Dutch SyRI system was a risk-scoring software being used by US court systems to establish bail times. The US Supreme Court declined to hear a related case in 2017.
Have other countries taken note of the Dutch court ruling?
Digital ID systems are being rolled out at a fast pace in places like Kenya, Philippines, Nigeria, Mexico, and more.
Experts worldwide have been watching the Netherlands case throughout, and agree that the ruling will ripple beyond south Rotterdam.
The UN Special Rapporteur on extreme poverty and human rights, Philip Alston, said: “This decision sets a strong legal precedent for other courts to follow. This is one of the first times a court anywhere has stopped the use of digital technologies and abundant digital information by welfare authorities on human rights grounds.”
Don’t miss from Explained: Indian scientists are in Antarctica. What are they studying?
The UK chairman of the House of Commons work and pensions select committee, Stephen Timms, said: “This ruling by the Dutch courts demonstrates that parliaments ought to look very closely at the ways in which governments use technology in the social security system, to protect the rights of their citizens.”
Share
