Opinion A call for algorithmic justice for SC/STs

Dominant caste groups perpetuate caste discrimination on social media, promoting violence and caste supremacy. According to a 2019 report based on a survey by the Delhi-based Lokniti and Centre for the Study of Developing Societies (CSDS), Scheduled Castes (SC) and Scheduled Tribes (ST) appear two times less frequently on social media than Hindu dominant caste users. Observer Research Foundation’s research report reveals that caste identities are reproduced, and caste norms violations have increased on social media in India.

Caste discrimination is continuing to expand online, including on social media platforms, communications, entertainment, online ads, education, job opportunities, and more. India has plenty of examples of emerging technologies reinforcing caste discrimination, such as discriminatory caste-based geolocation, digital surveillance leading to the criminalisation of Dalits, and Facebook’s caste discriminatory targeted ads. Indian AI scholars are concerned that data-driven algorithms may perpetuate caste bias in job and loan approval processes by reinforcing caste identities present in the name and address fields. These algorithmic biases lead to human rights violations such as increased surveillance of marginalised communities, data privacy, mental health issues, denial of socioeconomic and educational opportunities, and online social stigma. Emerging technologies are insensitive towards caste-based human rights issues online.

A recent Google research also calls for an anti-caste approach to algorithmic fairness by incorporating Dr B R Ambedkar’s insights on how caste hierarchies and patriarchies act as barriers to social justice commitments in India. Moreover, India’s digital divide significantly contributes to online social exclusion and lack of diversity in the tech workforce. Often, AI technologies’ fairness metrics and constraints are not primed to identify caste biases. This non-sensitivity and lack of policy regulations enable the perpetuation of caste discrimination online. However, there is no mention of caste in India’s recent Intermediary Guidelines and Digital Media Ethics Code Rules (2021). The proposed Digital India Act & Digital Personal Data Protection Bill 2023 briefly discusses upholding citizens’ rights and rights against discrimination. However, it is unclear how caste bias will be eradicated.

Marginalised communities and digital rights experts had hoped that social media would provide a pluralistic, democratic space for their voices and be accessible to people from all backgrounds. However, recent studies have shown that access to digital spaces is perceived as unfair and inequitable. Caste presence in emerging technologies risks perpetuating and even amplifying ongoing social injustices. In emerging technologies, caste can manifest bias in algorithms, exclusionary design practices, and in the lack of diversity at the workplace. India lacks an active tech policy and ethical framework to address caste-based discrimination in AI. We can reimagine Ambedkar’s vision of a casteless society within emerging technologies and apply the six ways to annihilate caste in AI and Metaverse technologies to tackle these systemic human rights issues.

Recognize Caste Bias: Technology developers must be trained to recognise potential caste biases and collaborate with social scientists to reflect and recontextualise datasets with socio-cultural values to mitigate algorithmic caste bias. Conducting regular anti-caste algorithmic audits of technologies with representatives from diverse social groups particularly SC and ST before deployment in real-world environments is also crucial to prevent these biases from being replicated in emerging technologies.

Develop Fairness Metrics & Constraints: In India, there are few efforts to prevent algorithmic gender bias. Currently, there are no fairness metrics or constraints in place to identify possible caste-based discrimination and biases in AI systems. Therefore, it is crucial to develop culturally contextual anti-caste fairness metrics and constraints to eliminate caste bias from emerging technologies and to effectively implement them.

Promote Diversity and Inclusivity: Developers from vulnerable communities such as SC and ST should have leadership roles in the design, development, and deployment process. This can help ensure emerging technologies are designed with a broad range of perspectives in mind and disentangle colonial ideologies of power to prevent the replication of existing bias, removing influences and privileges of dominant-caste members.

Meaningfully Support Marginalised Communities: Marginalised communities are socially and economically impacted by emerging technologies. To ensure meaningful participation of individuals and communities in the technology development process as co-designers, developers, deployers, and ethical auditors in AI systems, it is crucial to allocate sufficient and appropriate funding to promote inclusive AI.

Raise Public Awareness: Tech social workers should be engaged to educate the public, particularly SC and ST communities, about the harmful impact of all forms of caste-based discrimination online and their adverse impact on their social and economic opportunities. Tech social workers can organise outreach programmes such as community workshops, hackathons, and coding boot camps, as well as develop educational resources to engage the public in discussions about AI-driven caste discrimination. This would create a network of informed individuals who can educate others on the potential risks of AI.

Eliminate Caste in AI: India requires anti-caste ethical guidelines for emerging technologies and effective implementation and enforcement with tech companies and users. To prevent the expansion of the caste system in AI and the metaverse, Indian policymakers and tech companies should adopt Ambedkar’s vision of the annihilation of caste and apply it appropriately. The proposed Digital India Act-2023 should include a clause to annihilate caste in emerging technologies.

The annihilation of caste requires algorithmic justice in modern times. Modern digital India needs to recall Dr Ambedkar’s vision of the annihilation of caste and the appropriate application of his knowledge in emerging technologies to eradicate caste and build a safe, inclusive, just digital India.

The writer is associate director of SAFE Lab, University of Pennsylvania, and affiliate, The Berkman Klein Center for Internet & Society, Harvard University. sivam@upenn.edu

Suraj Yengde, author of Caste Matters, curates Dalitality and is currently at Oxford University