AI Fabricates Police Reports, Judge Warns of Eroding Trust

A federal judge has raised alarms over the use of artificial intelligence by U.S. Immigration and Customs Enforcement (ICE) agents to draft use-of-force reports, highlighting risks to accuracy and public trust. In a recent court opinion, Judge Sara Ellis noted that an agent used ChatGPT to generate a narrative after providing the AI tool with minimal input-a brief description and images-resulting in discrepancies between the official account and body camera footage. The judge criticized this practice as undermining the agents' credibility and potentially explaining the inaccuracy of the reports.

Experts warn that relying on AI for such high-stakes documentation is fraught with challenges. Ian Adams, a criminology professor and AI task force member, called the approach "the worst of all worlds", emphasizing that feeding an AI a single sentence and images invites the system to fabricate details rather than reflect objective facts. Andrew Guthrie Ferguson, a law professor, added that predictive AI tools could distort narratives by prioritizing what "should have happened" over factual accuracy, complicating legal defenses in court.

Privacy concerns further compound the issue. Katie Kinsey, a tech policy counsel at NYU's Policing Project, noted that uploading images to public AI platforms like ChatGPT could expose sensitive data to misuse, as uploaded content might become part of the public domain. She argued that law enforcement agencies are "building the plane as it's being flown" when it comes to AI governance, often adopting policies only after mistakes occur.

The Department of Homeland Security has not yet established clear guidelines for AI use by agents, and the body camera footage cited in the court order has not been publicly released. Meanwhile, some jurisdictions, like Utah and California, have begun requiring AI-generated documents to be labeled, offering a potential model for transparency.

Tech companies are also navigating AI's role in law enforcement. Axon, a provider of body cameras, has developed AI tools that limit themselves to audio-based narratives, avoiding the complexities of visual interpretation. Yet, the use of predictive analytics in policing remains contentious, with critics questioning whether AI-driven decisions align with professional standards or public expectations of accountability.

As AI adoption accelerates, the case underscores the urgent need for robust policies to ensure accuracy, privacy, and ethical use. Without clear guardrails, the integration of AI in law enforcement risks eroding both legal integrity and public confidence.