Bill Text: CA SB896 | 2023-2024 | Regular Session | Amended

NOTE: There are more recent revisions of this legislation. Read Latest Draft

---

Bill Title: Generative Artificial Intelligence Accountability Act.

Spectrum: Moderate Partisan Bill (Democrat 4-1)

Status: (Passed) 2024-09-29 - Chaptered by Secretary of State. Chapter 928, Statutes of 2024. [SB896 Detail]

Download: California-2023-SB896-Amended.html

---

Amended  IN  Assembly  August 19, 2024

Amended  IN  Assembly  July 03, 2024

Amended  IN  Assembly  June 04, 2024

Amended  IN  Senate  May 16, 2024

Amended  IN  Senate  April 10, 2024

CALIFORNIA LEGISLATURE— 2023–2024 REGULAR SESSION

Senate Bill

No. 896

---

Introduced by Senator Dodd
(Coauthor: Senator Rubio)
(Coauthors: Assembly Members Low, Mathis, and Weber)

January 03, 2024

---

An act to add Chapter 5.9 (commencing with Section 11549.63) to Part 1 of Division 3 of Title 2 of the Government Code, relating to artificial intelligence.

LEGISLATIVE COUNSEL'S DIGEST

SB 896, as amended, Dodd. Generative Artificial Intelligence Accountability Act.

Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.

This bill, the Generative Artificial Intelligence Accountability Act, would, among other things, require the Government Operations Agency, the Department of Technology, under the guidance of the Government Operations Agency, the Office of Data and Innovation, and the California Privacy Protection Agency Department of Human Resources, to produce a State of California Benefits and Risk of Generative Artificial Intelligence Report that includes certain items, including an examination of the most significant, potentially beneficial uses for deployment of generative artificial intelligence tools by the state, and would require those entities to update the report, report to the Governor, as required by Executive Order No. N-12-23, as prescribed. The bill would require the Office of Emergency Services, the California Cybersecurity Integration Center, and the State Threat Assessment Center every 2 years to Services to, as appropriate, perform a joint risk analysis of potential threats posed by the use of generative artificial intelligence to California’s critical energy infrastructure, including those that could lead to mass casualty events and environmental emergencies. events and would require a high-level summary of the analysis be provided annually to the Legislature.

This bill would also require a state agency or department that utilizes generative artificial intelligence to directly communicate with a person regarding government services and benefits to ensure that those communications include both (1) a disclaimer that indicates to the person that the communication was generated by generative artificial intelligence and (2) clear instructions, information, as specified, describing how the person may contact a human employee of the state agency or department.

Digest Key

Vote: MAJORITY   Appropriation: NO   Fiscal Committee: YES   Local Program: NO  

---

Bill Text

The people of the State of California do enact as follows:

SECTION 1.

 This act shall be known as the Generative Artificial Intelligence Accountability Act.

SEC. 2.

 Chapter 5.9 (commencing with Section 11549.63) is added to Part 1 of Division 3 of Title 2 of the Government Code, to read:

CHAPTER  5.9. Generative Artificial Intelligence Tools

11549.63.

 The Legislature finds and declares all of the following:

(a) The Legislature recognizes the tremendous potential of artificial intelligence (AI) to improve the lives of its citizens and the functioning of government. However, the Legislature also recognizes that the use of AI must be guided by principles of fairness, transparency, privacy, and accountability to ensure that the rights and opportunities of all Californians are protected in the age of artificial intelligence.

(b) The Legislature further recognizes that generative artificial intelligence (GenAI) enables significant, beneficial uses through its unique capabilities, but GenAI raises novel risks compared to conventional AI across critical areas, including democratic and legal processes, biases and equity, public health and safety, and the economy, and requires measures to address insufficiently guarded governmental systems and unintended or emergent harmful effects from this technology. Additionally, because humans have explicit and implicit biases built into our society, GenAI has the capacity to amplify these biases as it learns from input data. Therefore, it is imperative to consider the implications on Californians of, among other categories, different regions, income, races, ethnicities, gender, ages, religions, abilities, and sexual orientation for all GenAI inputs, outputs, and products for both prioritizing implementations that may promote equity and guarding against bias and other negative impacts.

(c) No individual or group should be discriminated against on the basis of race, gender, age, religion, sexual orientation, or any other protected characteristic in the design, development, deployment, or use of AI systems. The unprecedented speed of innovation and deployment of GenAI technologies necessitates proactive guardrails to protect against potential risks or malicious uses, including, but not limited to, bioterrorism, cyberattacks, disinformation, deception, violation of privacy, and discrimination or bias.

(d) The Legislature affirms the importance of transparency in the use of GenAI systems. The public has the right to know when they are interacting with GenAI being used by the state and to have a clear and conspicuous an accessible identification of that interaction.

(e) The Legislature recognizes that the use of GenAI systems must be consistent with the protection of privacy and civil liberties and must be guided by a commitment to equity and social justice. It is the intent of the Legislature in enacting this legislation that all GenAI systems be designed and deployed in a manner that is consistent with state and federal laws and regulations regarding privacy and civil liberties and minimizes bias and promotes equitable outcomes for all Californians.

(f) This act, in addition to the 2022 White House Blueprint for an AI Bill of Rights, executive guidance from the governor, statutory or regulatory requirements, and evolving best practices should guide the decisionmaking of state agencies, departments, and subdivisions in the review, adoption, management, governance, and regulations of automated decisionmaking technologies. Further, state leadership on adopting these standards and best practices should encourage the private sector to adopt these practices and safeguards.

(g) Public and private colleges and universities should collaborate with the private sector and relevant state agencies Public-private cross-sectional partnerships should be established to train students to meet the GenAI workforce development needs of the state, including providing instruction on AI and related ethical, privacy, and security considerations while advancing research on best practices. Further, there is the need for the state to recruit, retain, and train GenAI professionals in certain state jobs, and agencies should collaborate to facilitate a pipeline and infrastructure to accomplish that goal, including adopting appropriate job classifications and incentive programs. goal.

(h) State agencies, departments, and boards should utilize their full range of authority to protect consumers, patients, passengers, and students from fraud, discrimination, and threats to privacy and to address other risks that may arise from the use of GenAI, including risks to financial stability. State agencies, departments, and boards should consider rulemaking and emphasize or clarify if existing regulations and guidance apply to GenAI or other automated decisionmaking systems. They should also clarify any responsibility of regulated entities to conduct due diligence with respect to third-party GenAI services they use and emphasize or clarify requirements and expectations related to the transparency of AI models and regulated entities’ ability to explain their use of AI models.

(i)The California Privacy Protection Agency should utilize its existing authority to promulgate automated decisionmaking technology regulations.

11549.64.

 As used in this chapter:

(a) “Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.

(b) “Generative artificial intelligence” or “GenAI” means an artificial intelligence system that can generate derived synthetic content, including text, images, video, and audio that emulates the structure and characteristics of the system’s training data.

(c) “Person” means a natural person.

(d) “Report” means the State of California Benefits and Risk of Generative Artificial Intelligence Report report to the Governor required by Section 11549.65. Executive Order No. N-12-23.

11549.65.

(a)(1)The Government Operations Agency, the Department of Technology, the Office of Data and Innovation, and the California Privacy Protection Agency shall produce a State of California Benefits and Risk of Generative Artificial Intelligence Report that includes all of the following:

(A)An examination of the most significant, potentially beneficial uses for deployment of GenAI tools by the state.

(B)An explanation of the potential risks of the uses described in subparagraph (A) to individuals, communities, and government workers with a focus on high-risk uses, including the use of GenAI to make a consequential decision affecting access to essential goods and services.

(C)An explanation of risks from bad actors and insufficiently guarded governmental systems, unintended or emergent effects, and potential risks toward democratic and legal processes, public health and safety, data privacy, and the economy.

(2)

11549.65.

 (a) The Government Operations Agency, the Department of Technology, under the guidance of the Government Operations Agency, the Office of Data and Innovation, and the California Privacy Protection Agency Department of Human Resources, shall update the report, as needed, to respond to significant developments and shall, as appropriate, consult with academia, industry experts, and organizations that represent state government employees. exclusive employee representatives.

(b) (1) (A)The Office of Emergency Services, the California Cybersecurity Integration Center, and the State Threat Assessment Center shall every two years Services shall, as appropriate, perform a joint risk analysis of potential threats posed by the use of GenAI to California’s critical energy infrastructure, including those that could lead to mass casualty events and environmental emergencies. events.

(B)The entities described in subparagraph (A) shall develop, in consultation with appropriate external experts from academia and industry, a strategy to assess similar potential threats to other critical infrastructure.

(2) The analysis required by paragraph (1) shall be provided to the Legislature and the Governor, and, if appropriate, shall include recommendations reflecting changes to artificial intelligence technology, its applications, and risk management, including further private actions, administrative actions, and collaboration with the Legislature to guard against potential threats and vulnerabilities.

(3) The A high-level summary of the analysis required by paragraph (1) shall be submitted annually to the Legislature in compliance with Section 9795. Legislature.

(c)(1)(A)The Government Operations Agency, the Department of General Services, the Department of Technology, the California Cybersecurity Integration Center, and the California Privacy Protection Agency shall develop, maintain, and periodically evaluate and revise general guidelines for public sector procurement, uses, and required trainings for the use of GenAI, including for high-risk scenarios, and including for consequential decisions affecting access to essential goods and services.

(B)The guidelines required by this paragraph shall incorporate guidance from the White House publication titled Blueprint for an AI Bill of Rights and the National Institute of Standards and Technology’s AI Risk Management Framework, or succeeding document, and shall address safety, algorithmic discrimination, data privacy, and notice of when materials are generated by GenAI.

(C)The Government Operations Agency shall engage and consult with organizations that represent state government employees and industry experts, including, but not limited to, trust and safety experts, academic researchers, and research institutions in developing the guidelines required by this paragraph.

(2)For purposes of the periodic evaluation and revision required by paragraph (1), the Government Operations Agency, the Department of General Services, the Department of Technology, the California Cybersecurity Integration Center, and the California Privacy Protection Agency shall periodically evaluate any need to revise the guidelines and establish a consultative process by which to do so with academia, industry experts, and organizations that represent state government employees.

(d)(1)(A)The Government Operations Agency, the Department of Technology, the Office of Data and Innovation, and the California Privacy Protection Agency shall develop, maintain, and periodically evaluate and revise guidelines for state agencies and departments to analyze the impact that adopting a GenAI tool may have on vulnerable communities, including criteria to evaluate equitable outcomes in deployment and implementation of high-risk uses.

(B)The guidelines required by this subdivision shall inform whether and how a state agency or department deploys a particular GenAI tool.

(C)The Government Operations Agency shall engage and consult with organizations that represent state government employees and industry experts, including, but not limited to, trust and safety experts, academic researchers, and research institutions in developing the guidelines required by this subdivision.

(2)For purposes of the periodic evaluation and revision required by paragraph (1), the Government Operations Agency, the Department of Technology, the Office of Data and Innovation, and the California Privacy Protection Agency shall consult with academia, industry experts, and organizations that represent state government employees.

(e)The Government Operations Agency, the Department of General Services, and the Department of Technology shall update, as needed, the state’s project approval, procurement, and contract terms to incorporate analysis and feedback obtained pursuant to subdivisions (c) and (d).

(f)(1)To assist the Government Operations Agency and the Department of Technology in their efforts to perform any periodic review and update under this section, all state agencies and departments shall, as requested by the Government Operations Agency or the Department of Technology, conduct and submit an inventory of all current high-risk uses of GenAI within the agency or department to the Department of Technology, which shall administer the inventory.

(2)A state agency or department shall appoint a senior level management personnel who will be responsible for maintaining, conducting, and reporting the results of the inventory described by paragraph (1) to the Department of Technology within 60 days of issuance of a request for an inventory pursuant to paragraph (1).

(g)

(c) Any state agency or department shall consider procurement and enterprise use opportunities in which GenAI can improve the efficiency, effectiveness, accessibility, and equity of government operations consistent with the Government Operations Agency, the Department of General Services, and the Department of Technology’s guidelines policies for public sector GenAI procurement.

(h)(1)The Government Operations Agency shall collaborate with the Department of Technology to establish and maintain the infrastructure to conduct pilot projects of GenAI projects, including Department of Technology-approved environments to test those pilot projects.

(2)An environment created pursuant to this subdivision shall be available to any state agency or department to help evaluate GenAI tools and services, to further safe, ethical, and responsible implementations, and to inform decisions to use GenAI consistent with state guidelines.

(3)An environment created pursuant to this subdivision shall measure all of the following:

(A)How GenAI can improve Californians’ experience with, and access to, government services.

(B)How GenAI can support state employees in the performance of their duties.

(C)Any domain-specific impacts to be measured by the state agency or department, including job displacement.

(D)How to mitigate risks of harms associated with the use of GenAI.

(E)Any pilot projects conducted pursuant to this subdivision shall be consistent with the requirements of Item 0511-001-0001 of Section 2.00 of the Budget Act of 2024.

(i)The Government Operations Agency, the Department of General Services, the Department of Technology, the Office of Data and Innovation, the California Cybersecurity Integration Center, and the California Privacy Protection Agency shall engage with the Legislature and relevant stakeholders, including historically vulnerable and marginalized communities and organizations that represent state government employees, in the development and revision of any guidelines, criteria, reports, or training pursuant to this section.

(j)A state agency or department shall support the state government workforce and prepare for the next generation of skills needed to thrive in the GenAI economy by complying with both of the following:

(1)The Government Operations Agency, the Department of Technology, and the California Privacy Protection Agency shall make available trainings for state government worker use of state-approved GenAI tools to achieve equitable outcomes and to identify and mitigate potential output inaccuracies, fabricated text, hallucinations, privacy risks, and biases of GenAI, while enforcing applicable state laws and policies. If appropriate, the Government Operations Agency, the Department of Technology, and the California Privacy Protection Agency shall collaborate with organizations that represent state government employees and industry experts on developing and providing training.

(2)The Government Operations Agency, in consultation with appropriate state agencies and organizations that represent state government employees, shall establish criteria to evaluate the impact of GenAI on the state government workforce and provide guidelines on how state agencies and departments can support state government employees to use these tools effectively and respond to these technological advancements.

(k)

(d) Legal counsel for any state agency or department shall consider any potential impact of GenAI on regulatory issues under the respective agency’s or department’s authority and recommend necessary updates, if appropriate, as a result of this evolving technology.

11549.66.

 A state agency or department that utilizes GenAI to directly communicate with a person regarding government services and benefits shall ensure that those communications include both of the following:

(a) A disclaimer that indicates to the person that the communication was generated by GenAI.

(1) For written communications involving physical and digital media, including letters, email, and other occasional messages, the disclaimer shall appear prominently at the start of each communication.

(2) For written communications involving continuous online interactions, including interactions with chatbots, the disclaimer shall be prominently displayed throughout the interaction.

(3) For audio communications, the disclaimer shall be provided verbally at the start and end of the interaction.

(4) For video communications, the disclaimer shall be prominently displayed throughout the interaction.

(b) Clear instructions Information, or a link to an internet website containing clear instructions information, describing how the person may contact a human employee of the state agency or department.