‘Can generate fake case citations’: Top court judge flags AI concerns

“Relying on AI for legal research comes with significant risks, as there have been instances where platforms like ChatGPT have generated fake case citations and fabricated legal facts,” he said.

Justice Gavai said, “While AI can process vast amounts of legal data and provide quick summaries, it lacks the ability to verify sources with human-level discernment. This has led to situations where lawyers and researchers…have unknowingly cited non-existent cases or misleading legal precedents…”

Justice Gavai also said “the essence of justice often involves ethical considerations, empathy, and contextual understanding — elements that remain beyond the reach of algorithms…”

He flagged concerns on short clips from court hearings being circulated and even monetised by social media users. “Short clips from court hearings are often circulated on social media, sometimes in a way that sensationalises the proceedings. These clips, when taken out of context, can lead to misinformation…”

“Many content creators, including YouTubers, re-upload short excerpts from proceedings as their own content, raising questions about intellectual property rights and ownership of judicial recordings. The unauthorised use and potential monetisation of such content blur the lines between public access and ethical broadcasting,” he said.