A recent High Court judgment delivered a stark warning for businesses using artificial intelligence, and it is not just a problem for lawyers.
A major UK law firm has been reprimanded for the use of generative AI by one of its junior lawyers in drafting correspondence to the Court containing fictitious references to the Insolvency Act.
The underlying matter of the correspondence in question was fairly straightforward, being an application to replace the respondent in his role of administrator or liquidator to various companies. The draft order submitted by the law firm, on behalf of its client, included reference to the respondent being released from liability without the need to apply to the Secretary of State. Insolvency and Companies Court Judge Mullen requested that the firm clarify where such a power in relation to liquidators existed and by return, the firm produced two letters which Judge Mullen concluded had been ‘almost entirely outsourced’ to AI and contained references to the Insolvency Act which did not exist.
Whilst it was subsequently admitted by the law firm that AI was used by a junior lawyer to draft the letters, the Court was particularly unimpressed by the adequacy of supervision and the supervisors’ failure to identify the hallucinations of the AI program. The seriousness of the error, in the mind of the Court, was exacerbated by the law firm failing to take the opportunity to clarify, in the second letter, that the provisions of the Insolvency Act relied upon in the first letter were indeed hallucinations of the AI program, having been given the opportunity by Judge Mullen to do so.
The case highlights the risks involved in using AI in any business. In particular, it is a useful reminder that where business engage with AI, employees should be extensively trained and monitored on their AI literacy; including with regard to using any generative AI. They should also be clear on the risks of AI usage relating to IP rights, data protection and confidentiality.
These days, an appropriate AI policy is not so much a nicety as an essential. It needs to be monitored and kept under periodic review, particularly as the general landscape of AI continues to rapidly evolve. Employees and senior people within the business need to be aware of the necessity of keeping the human element at the centre of quality control. There should always be appropriate safeguards and procedures in place to verify any AI output and, to ensure any facts and references included are double-checked.
It is clear, the Courts are very unlikely to take a sympathetic approach to AI usage where employees have not verified the AI outputs. Indeed, in this instance, there were significant cost consequences for the law firm, who covered their own client’s cost of the application, reputational damage to the firm (both in that they lost the client and to their existing and potential clients), as well as regulatory consequences for the firm and the solicitors.
Even sophisticated organisations with experienced professionals can get this wrong. The speed and convenience of AI makes it tempting to rely on outputs without scrutiny, but this case shows how quickly that can unravel.
Used well, AI is a valuable tool. Used carelessly, it becomes a serious business and reputational risk
The information on this site about legal matters is provided as a general guide only. Although we try to ensure that all of the information on this site is accurate and up to date, this cannot be guaranteed. The information on this site should not be relied upon or construed as constituting legal advice and Howes Percival LLP disclaims liability in relation to its use. You should seek appropriate legal advice before taking or refraining from taking any action.
