Are We Blindly Outsourcing Professional Integrity to AI?
The legal profession is currently facing a reckoning regarding AI-assisted due diligence. The New Mexico Supreme Court recently fined attorney Stephen Aarons $5,000 after he filed fabricated witnesses and police reports generated by an AI in a murder case. This incident serves as a stark reminder that generative models are not truth-engines, but rather probabilistic predictors that can hallucinate with high confidence. When lawyers outsource their fact-checking to software, they are not just risking professional sanctions; they are undermining the very integrity of the judicial system.
Simultaneously, the technical landscape is shifting toward self-verification, with companies like Cognition integrating tools like GPT-6 Astra into software engineering workflows to allow systems to test their own code. While this promises faster development, it creates a "black box" feedback loop where human oversight is increasingly pushed to the periphery. We are effectively automating the review process, trusting the machine to audit its own potential for error. As Anthropic acknowledges recent internal model instabilities and cybersecurity vulnerabilities, we must ask if we are trading long-term institutional reliability for short-term efficiency.
What we're arguing about
- At what point does the use of AI to "assist" with professional work product shift from a productivity tool to a breach of fiduciary or ethical duty?
- If software like Devin can now audit its own code, do human-led peer reviews become a bottleneck or a necessary failsafe that we are dangerously abandoning?
- Should regulatory bodies mandate "human-in-the-loop" requirements for high-stakes professions, or does that simply hamper the inevitable evolution of these tools?
Share your personal experiences with AI-generated errors in professional settings and how you have adjusted your verification workflows to maintain your integrity.