A bill passed on Thursday by the California Senate would require lawyers in the state to verify the accuracy of all materials produced using artificial intelligence, including case citations and other information in court filings.

The measure, which appears to be one of the first pending in a state legislature on the use of AI by lawyers, has gone to the State Assembly for consideration.

In addition to governing California lawyers' use of AI, the bill prohibits arbitrators presiding over out-of-court disputes from delegating decision-making to generative AI and from relying on information produced by AI outside case records without first telling the parties involved.

Courts across the country have grappled with the benefits and pitfalls of AI in legal practice as lawyers, judges and others have increasingly turned to chatbots and other specialized and publicly available AI tools. The technology can generate fictional details known as "hallucinations," which has led to sanctions or reprimands against lawyers and self-represented litigants in dozens of cases for not vetting AI-generated case citations and other material included in court filings.

Lawyers under the California bill would be required to take “reasonable steps” to verify the accuracy of material that was produced using artificial intelligence, correct any false or hallucinated output, and remove biased content in any AI-generated material they use.

Attorneys also would be barred from putting confidential, personally identifying, or other nonpublic information into public generative AI tools and must make sure that the use of AI does not "unlawfully discriminate" against people or communities.

Senator Tom Umberg, a Democrat who chairs the California Senate’s judiciary committee, introduced the bill, known as SB 574, opens new tab. "As A.I. becomes more common in the legal system, we need clear guardrails to protect clients’ confidentiality and ensure that real people, not algorithms, are making legal decisions," Umberg said in a statement to Reuters on Friday.

The measure is one of several in California aimed at managing the potential dangers of artificial intelligence, including a new law that requires big companies to disclose how they plan to mitigate potential catastrophic risks from their cutting-edge AI models.

Read more here: Reuters