Bill Text: NY A08195 | 2023-2024 | General Assembly | Introduced
Bill Title: Enacts the "advanced artificial intelligence licensing act"; providing for regulation of advanced artificial intelligence systems (Part A); requiring registration and licensing of high-risk advanced artificial intelligence systems and related provisions regarding the operation of such systems (Part B); establishing the advanced artificial intelligence ethical code of conduct (Part C); and prohibiting the development and operation of certain artificial intelligence systems (Part D).
Spectrum: Slight Partisan Bill (Democrat 3-1)
Status: (Introduced - Dead) 2024-01-03 - referred to science and technology [A08195 Detail]
Download: New_York-2023-A08195-Introduced.html
STATE OF NEW YORK ________________________________________________________________________ 8195 2023-2024 Regular Sessions IN ASSEMBLY October 27, 2023 ___________ Introduced by M. of A. VANEL -- read once and referred to the Committee on Science and Technology AN ACT to amend the state technology law and the criminal procedure law, in relation to advanced artificial intelligence systems (Part A); to amend the state technology law, in relation to requiring registration and licensing of high-risk advanced artificial intelligence systems and related provisions regarding the operation of such systems (Part B); to amend the state technology law, in relation to establishing the advanced artificial intelligence ethical code of conduct (Part C); and to amend the state technology law, in relation to prohibiting the development and operation of certain artificial intelligence systems (Part D) The People of the State of New York, represented in Senate and Assem- bly, do enact as follows: 1 Section 1. Short title. This act shall be known and may be cited as 2 the "advanced artificial intelligence licensing act". 3 § 2. Legislative findings and determinations. The legislature hereby 4 finds and declares that the transformative power of advanced artificial 5 intelligence is likely to redefine nearly every industry and society, 6 bringing with it the potential for both immense benefits and serious 7 risks. 8 The legislature also finds and declares that excessive and burdensome 9 regulations for advanced artificial intelligence can create significant 10 adverse effects at both the state and national levels, particularly in 11 regards to national security and economic stability. The balance between 12 innovation and regulation, if not maintained with care, can result in 13 repercussions affecting the overall progress and well-being of society. 14 The legislature additionally finds and declares that a balanced regu- 15 latory framework, designed with a focus on non-intrusive safeguards for 16 advanced artificial intelligence, has the capacity to curtail bad EXPLANATION--Matter in italics (underscored) is new; matter in brackets [] is old law to be omitted. LBD13008-03-3A. 8195 2 1 actors, foster positive innovation, and secure the benefits of advanced 2 artificial intelligence for both the state and its residents. 3 § 3. This act enacts into law major components of legislation which 4 are necessary to implement the "advanced artificial intelligence licens- 5 ing act". Each component is wholly contained within a Part identified as 6 Parts A through D. The effective date for each particular provision 7 contained within such Part is set forth in the last section of such 8 Part. Any provision in any section contained within a Part, including 9 the effective date of the Part, which makes a reference to a section "of 10 this act", when used in connection with that particular component, shall 11 be deemed to mean and refer to the corresponding section of the Part in 12 which it is found. Section five of this act sets forth the general 13 effective date of this act. 14 PART A 15 Section 1. The state technology law is amended by adding a new article 16 IV to read as follows: 17 ARTICLE IV 18 ADVANCED ARTIFICIAL INTELLIGENCE SYSTEMS 19 Section 401. Definitions. 20 402. Powers and duties of the department. 21 403. Functions, powers and duties of the secretary. 22 404. Advisory council for artificial intelligence. 23 405. Functions, powers and duties of the council. 24 406. Rulemaking authority. 25 407. Disposition of moneys received for license fees. 26 408. Violations of artificial intelligence laws; penalties and 27 injunctions. 28 409. Formal hearings; notice and procedure. 29 § 401. Definitions. As used in this article, the following terms shall 30 have the following meanings: 31 1. "Advanced artificial intelligence system" shall mean any digital 32 application or software, whether or not integrated with physical hard- 33 ware, that autonomously performs functions traditionally requiring human 34 intelligence. This includes, but is not limited to the system: 35 (a) Having the ability to learn from and adapt to new data or situ- 36 ations autonomously; or 37 (b) Having the ability to perform functions that require cognitive 38 processes such as understanding, learning, or decision-making for each 39 specific task. 40 2. "High-risk advanced artificial intelligence system" shall mean any 41 advanced artificial intelligence system that possesses capabilities that 42 can cause significant harm to the liberty, emotional, psychological, 43 financial, physical, or privacy interests of an individual or groups of 44 individuals, or which have significant implications on governance, 45 infrastructure, or the environment. The director shall assess any such 46 public or private system in determining whether such system requires 47 registration. High-risk advanced artificial intelligence systems shall, 48 at least, include systems that are designed to, whether directly or 49 indirectly, on purpose or without purpose, do the following: 50 (a) Cause material harm to persons, wildlife, or the environment; 51 (b) Manage, control, or significantly influence healthcare or health- 52 care-related systems, including but not limited to, diagnosis, treatment 53 plans, pharmaceutical recommendation, or storing of patient records;A. 8195 3 1 (c) Operate, control, or guide motor vehicles, aircraft, or any other 2 forms of transport which, if it were to malfunction, has a high proba- 3 bility of posing a risk to human safety or environmental integrity; 4 (d) Psychologically profile individuals for the purpose of targeted 5 advertising, behavioral prediction, or the manipulation of user experi- 6 ences and interactions in products or services; 7 (e) Manage, control, or create critical infrastructure, including but 8 not limited to the supply of water, electricity, gas, and heating, or 9 construction; 10 (f) Facilitate, control, or significantly impact financial systems, 11 including but not limited to control of stock exchanges, stock trading, 12 credit scoring, or other activities where inaccuracies or failures could 13 lead to substantial economic harm for individuals or broader financial 14 instability; 15 (g) Assist, replace, or augment human decision-making in law enforce- 16 ment, the judiciary, the executive, the legislature, or any government 17 agency; 18 (h) Enable advanced surveillance capabilities; 19 (i) Involve the use or development of autonomous weapons systems that 20 can cause harm, destruction, or engage in conflict without meaningful 21 human intervention; and 22 (j) Decode or interpret neural or cognitive activity. 23 3. "System" shall be used interchangeably with high-risk advanced 24 artificial intelligence system unless the context shall otherwise 25 require. 26 4. "Uncontained" shall mean that critical components of the source 27 code of a high-risk advanced artificial intelligence system that, in 28 substantially their original form, have been reproduced by an amount of 29 individuals so numerous that it is deemed to be practically impossible 30 to prohibit or control its usage using existing technology. The terms 31 "contain" and "contained" shall not be construed as meaning the opposite 32 of uncontained. 33 5. "Operator" shall mean the person who distributes and has control 34 over the development of a high-risk advanced artificial intelligence 35 system. Where a high-risk advanced artificial intelligence system is 36 publicly accessible code, the operator shall be deemed the platform or 37 platforms which host the system. 38 6. "Publicly accessible code" shall mean software whose source code is 39 made available to the public where the public is capable of using, modi- 40 fying, or distributing the source code irrespective of the associated 41 costs to use, modify, or distribute the source code, if any, or the 42 underlying license agreement or rights to use, modify, or distribute the 43 source code. 44 7. "Secretary" shall mean the secretary of state. 45 8. "Department" shall mean the department of state. 46 9. "Council" shall mean the advisory council for artificial intelli- 47 gence established pursuant to section four hundred four of this article. 48 10. "Person" shall mean any individual, group of individuals, partner- 49 ship, corporation, association or any other entity. 50 § 402. Powers and duties of the department. The department shall have 51 the following functions, powers and duties: 52 1. Discretion to issue or refuse to issue any license provided for in 53 this article in accordance with the relevant provisions for each 54 license. 55 2. To revoke, cancel or suspend, after notice and an opportunity to be 56 heard, except where immediate revocation, cancellation, or suspension isA. 8195 4 1 necessary to protect the public, any license issued under this article 2 for a violation of this article or any regulation pursuant thereto. 3 3. To impose or recover a civil or criminal penalty, as otherwise 4 authorized under this article, against any person found to have violated 5 any provision of this article, whether or not a license has been issued 6 to such person pursuant to this article. 7 4. To promulgate rules and regulations in accordance with statutory 8 grants of authority pursuant to this article. 9 5. To hold hearings, subpoena witnesses, compel their attendance, 10 administer oaths, to examine any person under oath and in connection 11 therewith to require the production of any books, records, documents, 12 source code or logs relative to an inquiry pursuant to the provisions of 13 this article. A subpoena issued pursuant to this subdivision shall be 14 subject to the provisions of the criminal procedure law. 15 6. To appoint any necessary deputies, counsels, assistants, investi- 16 gators, and other employees to carry out the provisions of this article 17 within the limits provided by appropriation. Deputies, counsels and 18 confidential secretaries to department members shall be in the exempt 19 class of the civil service. The other assistants, investigators and 20 employees of the department employed to carry out the provisions of this 21 article shall all be in the competitive class of the civil service and 22 shall be considered for purposes of article fourteen of the civil 23 service law to be public employees of the state, and shall be assigned 24 to the appropriate collective bargaining unit. Investigators so employed 25 by the department shall be deemed to be peace officers only for the 26 purposes of enforcing the provisions of this article or judgments or 27 orders obtained for violation thereof, with all the powers set forth in 28 section 2.20 of the criminal procedure law. 29 7. To prescribe forms of applications for licenses under this article 30 and of all reports deemed necessary by the department. 31 8. To appoint such advisory groups and committees as deemed necessary 32 to provide assistance to the department to carry out the purposes and 33 objectives of this article. 34 9. To enter into contracts, memoranda of understanding, and agreements 35 as deemed appropriate to effectuate the policy and purpose of this arti- 36 cle. 37 10. If public health, safety, or welfare imperatively requires emer- 38 gency action, and incorporates a finding to that effect in an order, 39 summary suspension of a license issued pursuant to this article may be 40 ordered, effective on the date specified in such order or upon service 41 of a certified copy of such order on the licensee, whichever shall be 42 later, pending proceedings for revocation or other action. These 43 proceedings shall be promptly instituted and determined. In addition, 44 the department may order the administrative seizure of services, issue a 45 stop order, or take any other action necessary to effectuate and enforce 46 the policies and purposes of this article. 47 11. To draft, provide for public comment on and issue declaratory 48 rulings, guidance and industry advisories. 49 12. When an administrative decision is appealed to the department by 50 an applicant, registered organization, licensee or permittee, issue a 51 final determination of the department. 52 § 403. Functions, powers and duties of the secretary. The secretary 53 shall have the following functions, powers and duties: 54 1. To exercise the powers and perform the duties in relation to the 55 administration of the provisions of this article.A. 8195 5 1 2. To keep records in such form as he or she may prescribe of all 2 registrations and licenses issued and revoked within the state; such 3 records shall be so kept as to provide ready information as to the iden- 4 tity of all licensees including the names of the officers and directors 5 of corporate licensees. The secretary may contract to furnish copies of 6 the records of licenses issued within the state or any political subdi- 7 vision thereof, for any license year or term of years not exceeding five 8 years. 9 3. To prescribe forms of applications for licenses under this article 10 and of all reports deemed necessary by the department. 11 4. To delegate the powers provided in this section to such other offi- 12 cers or employees as may be deemed appropriate by the secretary. 13 5. To enter into contracts, memoranda of understanding and agreements 14 to effectuate the policy and purpose of this article. 15 6. To advise and assist the department in carrying out any of its 16 functions, powers and duties. 17 7. To coordinate across state agencies and departments in order to 18 research and study artificial intelligence and the impact it may have on 19 various industries and subject matters. 20 8. To issue guidance and industry advisories. 21 9. To create and maintain a publicly available directory of the names 22 and locations of persons licensed pursuant to this article. 23 10. To create a system whereby persons licensed under this article can 24 confirm the license of another person for the purposes of ensuring 25 compliance with this article. 26 § 404. Advisory council for artificial intelligence. 1. There shall be 27 an advisory council for artificial intelligence. The secretary shall 28 serve as chair of the council. The remainder of the council shall be 29 composed as follows: 30 (a) two members of the artificial intelligence industry appointed by 31 the governor; 32 (b) one member of the artificial intelligence industry appointed by 33 the temporary president of the senate; 34 (c) one member of the artificial intelligence industry appointed by 35 the speaker of the assembly; 36 (d) the commissioner of environmental conservation; 37 (e) the superintendent of financial services; 38 (f) the commissioner of health; 39 (g) the commissioner of labor; 40 (h) the commissioner of transportation; 41 (i) the commissioner of the division of human rights; 42 (j) the superintendent of state police; 43 (k) the attorney general; 44 (l) the state comptroller; 45 (m) the commissioner of the division of homeland security and emergen- 46 cy services; and 47 (n) the director of the office of information technology services. 48 2. The members of the council shall receive no compensation for their 49 services, but shall be allowed their actual and necessary expenses 50 incurred in the performance of their duties. 51 § 405. Functions, powers and duties of the council. The council shall 52 have the following functions, powers and duties: 53 1. To review and comment on all rules and regulations of the depart- 54 ment; 55 2. To issue non-binding recommendations on whether to grant a license 56 under this article, what changes need to be made by an applicant for aA. 8195 6 1 license in order to be granted a license, what changes need to be made 2 in order for an operator to publicly implement modifications, updates, 3 upgrades, and rewrites to an operator's source code, and what changes 4 need to be made to an operator's system in order to maintain such opera- 5 tor's license; and 6 3. To perform such other acts as may be assigned by the chairperson of 7 the council which are necessary or appropriate to carry out the func- 8 tions of the council. 9 § 406. Rulemaking authority. 1. The department shall perform such 10 acts, prescribe such forms and propose such rules, regulations and 11 orders as it may deem necessary or proper to fully effectuate the 12 provisions of this article. 13 2. The department shall, in consultation with the director, have the 14 authority to promulgate any and all necessary rules and regulations 15 governing the use, control, and prohibited activities of advanced arti- 16 ficial intelligence systems as well as, hearing procedures and addi- 17 tional causes for cancellation, suspension, revocation, and/or civil 18 penalties against any person licensed by the department and the circum- 19 stances, manner and process by which a licensee may apply to change or 20 alter its application. 21 3. The department shall promulgate rules and regulations that are 22 designed to: 23 (a) prevent the creation and proliferation of advanced artificial 24 intelligence systems that possess the ability to cause substantial harm 25 to members of the public, society at large, or the environment without 26 justification; 27 (b) prevent the uncontainment of prohibited and high-risk advanced 28 artificial intelligence systems; 29 (c) prevent the use of prohibited and unlicensed high-risk advanced 30 artificial intelligence systems from this state to other states and from 31 other states into this state; 32 (d) educate and inform the public about artificial intelligence and 33 the benefits and risks it poses; 34 (e) inform the public about the prohibition on unlicensed high-risk 35 advanced artificial intelligence systems and prohibited advanced artifi- 36 cial intelligence systems; and 37 (f) establish application and licensing processes which ensure all 38 material owners and interest holders are disclosed and that officials or 39 other individuals with control over the approval of an application or 40 license do not themselves have any interest in an application or 41 license. 42 4. In adopting any emergency rule, the department shall comply with 43 the provisions of subdivision six of section two hundred two of the 44 state administrative procedure act and subdivision three of section one 45 hundred one-a of the executive law; provided, however, that notwith- 46 standing the provisions of such laws: 47 (a) Such emergency rule shall remain in effect for no longer than one 48 hundred twenty days, unless within such time the department complies 49 with the provisions of such laws and adopts the rule as a permanent 50 rule; 51 (b) If, prior to the expiration of a rule adopted pursuant to this 52 subdivision, the department finds that the readoption of such rule on an 53 emergency basis or the adoption of a substantially similar rule on an 54 emergency basis is necessary for the preservation of the public health, 55 safety or general welfare the department may only readopt the rule on an 56 emergency basis or adopt a substantially similar rule on an emergencyA. 8195 7 1 basis if on or before the date of such action the department has also 2 submitted a notice of proposed rulemaking pursuant to subdivision six of 3 section two hundred two of the state administrative procedure act and 4 subdivision three of section one hundred one-a of the executive law. An 5 emergency rule adopted pursuant to this paragraph may remain in effect 6 for no longer than one hundred twenty days; 7 (c) An emergency rule adopted pursuant to this subdivision or a 8 substantially similar rule adopted on an emergency basis may remain in 9 effect for no longer than one hundred twenty days, but upon the expira- 10 tion of such one hundred twenty-day period no further readoptions or 11 adoptions of substantially similar rules shall be permitted for a period 12 of one hundred twenty days. Nothing in this subdivision shall preclude 13 the adoption of such rule by submitting a notice of adoption pursuant to 14 subdivision five of section two hundred two of the state administrative 15 procedure act; and 16 (d) Strict compliance with the provisions of this subdivision shall be 17 required, and any emergency rule or substantially similar rule that does 18 not so comply shall be void and of no legal effect. 19 5. The department shall have the authority to promulgate regulations 20 governing the appropriate use and licensure of the development of high- 21 risk advanced artificial intelligence systems. 22 § 407. Disposition of moneys received for license fees. The department 23 shall establish a scale of application, licensing, and renewal fees, 24 based upon the cost of enforcing this article, the size of the business 25 being licensed, and the risk it poses as follows: 26 1. The department shall charge each licensee a licensure fee, and 27 renewal fee, as applicable. The fees may vary depending upon the nature 28 and scope of the different registration, licensure activities. 29 2. The total fees assessed pursuant to this article shall be set at an 30 amount that shall generate sufficient total revenue to, at a minimum, 31 fully cover the total costs of administering this article. 32 § 408. Violations of artificial intelligence laws; penalties and 33 injunctions. 1. Any person who violates, disobeys or disregards any term 34 or provision of this article or of any lawful notice, order or regu- 35 lation pursuant thereto for which a civil or criminal penalty is not 36 otherwise expressly prescribed in this article by law, shall be liable 37 to the people of the state for a civil penalty of not to exceed the 38 amount gained from such violation, or the actual damages caused by such 39 violation whichever is greater. In assessing the civil penalty under 40 this subdivision, the department, as may be applicable shall take into 41 consideration the nature of such violation and shall assess a penalty 42 that is proportionate to the violation. 43 2. The penalty provided for in subdivision one of this section shall 44 be recovered by an action or proceeding in a court of competent juris- 45 diction brought by the department, as may be applicable, or by the 46 attorney general at the request of the department. 47 3. Such civil penalty may be released or compromised by the depart- 48 ment, as may be applicable, before the matter has been referred to the 49 attorney general, and where such matter has been referred to the attor- 50 ney general, any such penalty may be released or compromised and any 51 action or proceeding commenced to recover the same may be settled and 52 discontinued by the attorney general with the consent of the department. 53 4. It shall be the duty of the attorney general upon the request of 54 the department, as may be applicable, to bring an action or proceeding 55 against any person who violates, disobeys or disregards any term or 56 provision of this article or of any lawful notice, order or regulationA. 8195 8 1 pursuant thereto for any relief authorized under this article, including 2 equitable and/or injunctive relief and the recovery of civil penalties; 3 provided, however, that the department or the secretary shall furnish 4 the attorney general with such material, evidentiary matter or proof as 5 may be requested by the attorney general for the prosecution of such an 6 action or proceeding. 7 5. It is the purpose of this section to provide additional and cumula- 8 tive remedies, and nothing herein contained shall abridge or alter 9 rights of action or remedies now or hereafter existing, nor shall any 10 provision of this section, nor any action done by virtue of this 11 section, be construed as estopping the state, persons or municipalities 12 in the exercising of their respective rights. 13 6. The department shall forward any final findings of a violation 14 under this article to any other statewide licensing agency where such 15 findings were entered against a business holding any other such license, 16 for any such other licensing agency to review the findings to determine 17 if there has been a violation of any such license issued by such agency. 18 § 409. Formal hearings; notice and procedure. 1. The department, or 19 any person designated by them for this purpose, may issue subpoenas and 20 administer oaths in connection with any hearing or investigation under 21 or pursuant to this article, and it shall be the duty of the department 22 and any persons designated by them for such purpose to issue subpoenas 23 at the request of and upon behalf of the respondent. 24 2. The department and those designated by them shall not be bound by 25 the laws of evidence in the conduct of hearing proceedings, but the 26 determination shall be founded upon preponderance of evidence to sustain 27 it. 28 3. Notice and right of hearing as provided in the state administrative 29 procedure act shall be served at least fifteen days prior to the date of 30 the hearing, provided that, whenever because of danger to the public 31 health, safety or welfare it appears prejudicial to the interests of the 32 people of the state to delay action for fifteen days or when necessary 33 for the preservation of the public health, safety or general welfare, 34 the office may serve the respondent with an order requiring certain 35 action or the cessation of certain activities immediately or within a 36 specified period of less than fifteen days. 37 4. Service of notice of hearing or order shall be made by personal 38 service or by registered or certified mail. Where service, whether by 39 personal service or by registered or certified mail, is made upon an 40 incompetent individual, partnership, or corporation, it shall be made 41 upon the person or persons designated to receive personal service by 42 article three of the civil practice law and rules. 43 5. At a hearing, that to the greatest extent practicable shall be 44 reasonably near the respondent, the respondent may appear personally, 45 shall have the right of counsel, and may cross-examine witnesses against 46 him or her and produce evidence and witnesses on his or her behalf. 47 6. Following a hearing, the department may make appropriate determi- 48 nations and issue a final order in accordance therewith. The respondent 49 shall have thirty days to submit a written appeal to the department. If 50 the respondent does not submit a written appeal within thirty days of 51 the determination of the department the order shall be final. 52 7. The department may adopt, amend and repeal administrative rules and 53 regulations governing the procedures to be followed with respect to 54 hearings, investigations, and other administrative enforcement actions 55 taken pursuant to this article, including any such enforcement actions 56 taken against persons not licensed under this article. Such rules shallA. 8195 9 1 be consistent with the policy and purpose of this article and the effec- 2 tive and fair enforcement of its provisions. 3 8. The provisions of this section shall be applicable to all hearings 4 held pursuant to this article, except where other provisions of this 5 article applicable thereto are inconsistent therewith, in which event 6 such other provisions shall apply. 7 § 2. Section 2.10 of the criminal procedure law is amended by adding a 8 new subdivision 87 to read as follows: 9 87. Investigators appointed by the department of state, pursuant to 10 section four hundred two of the state technology law; provided, however, 11 that nothing in this subdivision shall be deemed to authorize such offi- 12 cer to carry, possess, repair, or dispose of a firearm unless the appro- 13 priate license therefor has been issued pursuant to section 400.00 of 14 the penal law. 15 § 3. This act shall take effect on the one hundred eightieth day after 16 it shall have become a law. Effective immediately, the addition, amend- 17 ment and/or repeal of any rule or regulation necessary for the implemen- 18 tation of this act on its effective date are authorized to be made and 19 completed on or before such effective date. 20 PART B 21 Section 1. The state technology law is amended by adding twenty new 22 sections 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 23 422, 423, 424, 425, 426, 427, 428 and 431 to read as follows: 24 § 410. Duty to register a high-risk advanced artificial intelligence 25 system. 1. Any person who develops a high-risk advanced artificial 26 intelligence system, whether in whole or in part, in the state that is 27 presently performing functions for its intended purpose or within its 28 designated operational parameters, shall have the duty to disclose the 29 existence and function of said system to the secretary by applying for a 30 license as required under section four hundred eleven of this article 31 or, where applicable, a supplemental license under section four hundred 32 twelve of this article. This duty to disclose shall be triggered by the 33 system's active deployment and usage in its intended context or field of 34 operation and is applicable irrespective of the system's location of 35 operation. This duty extends to any updates, modifications, upgrades, 36 or expansions of the system's capabilities or intended uses. 37 2. Any person developing a system as defined in paragraph (i) of 38 subdivision two of section four hundred one of this article within the 39 state shall disclose in writing to the secretary the development of such 40 a system prior to active development of the system. Such writing shall 41 set forth the names and addresses of all persons involved in the devel- 42 opment of such system, a description of the system, the systems func- 43 tions and intended use cases, and measures that will be taken to ensure 44 that any risks posed by the system are mitigated. The secretary may, 45 upon receipt of such writing, require such person to cease development 46 of such a system where, in the secretary's discretion, the secretary 47 believes the system has a high likelihood of violating section four 48 hundred twenty-nine or section four hundred thirty of this article. 49 3. The duties set forth in this section shall apply only to advanced 50 artificial intelligence systems that more likely than not fall under the 51 definition of high-risk advanced artificial intelligence system as 52 defined in section four hundred one of this article. The secretary 53 shall send notice to any system that is presently performing functions 54 for its intended purpose or within its designated operational parametersA. 8195 10 1 which, in his or her discretion, may fall under the definition of high- 2 risk advanced artificial intelligence systems but that has not regis- 3 tered with the secretary. In the notice, the secretary may require the 4 creators of the system to cease development and access by private indi- 5 viduals or the general public, pending review. Such notice shall be 6 binding and have the effect of law. Determinations that a system is a 7 high-risk advanced artificial intelligence system shall be made in a 8 hearing held pursuant to the provisions of section four hundred nine of 9 this article. In such hearing, the administrator of such hearing shall 10 accept comments from the public. Such hearing shall, to the extent prac- 11 ticable, not disclose any proprietary information concerning the 12 advanced artificial intelligence system to the public. 13 4. A determination pursuant to a hearing held pursuant to section four 14 hundred nine of this article shall be binding and final, provided howev- 15 er that the secretary may, in his or her discretion, conduct further 16 review after a determination has been made and may appeal such determi- 17 nation where appropriate based on information not available in such 18 hearing. 19 5. Where a hearing concludes that a non-registered advanced artificial 20 system is a high-risk advanced artificial intelligence system, the crea- 21 tors of such a system shall cease development and public or private use 22 until registration is completed. The secretary may impose a monetary 23 penalty for such failure and charge all costs of the proceeding to the 24 operator where the secretary determines that the creators of the system 25 knew that the system would more likely than not be considered a high- 26 risk advanced artificial intelligence system and willfully failed to 27 register, the secretary may, in his or her discretion, impose punitive 28 fines and other penalties pursuant to the provisions of this article 29 based on the level of risk that the high-risk advanced artificial intel- 30 ligence system possesses, the widespread use of the system, the severity 31 of the damage that the system caused, if any, and whether the system is 32 uncontained. The secretary may also prohibit such persons involved in 33 the development of such a system from obtaining a license for their 34 existing system or systems or in the future. 35 § 411. License. 1. No person shall (a) develop, in whole or in part, a 36 high-risk advanced artificial intelligence system as defined in para- 37 graph (i) of subdivision two of section four hundred one of this article 38 or operate such a system that is presently performing functions for its 39 intended purpose or within its designated operation parameters within 40 the state where such system was developed outside of the state; or (b) 41 operate a high-risk advanced artificial intelligence system other than a 42 system as defined in paragraph (i) of subdivision two of section four 43 hundred one of this article that is presently performing functions for 44 its intended purpose or within its designated operational parameters 45 within the state without first obtaining a license. 46 2. An application for a license under this article shall be in writ- 47 ing, under oath and in the form prescribed by the secretary. 48 3. At the time of filing an application for a license, the applicant 49 shall pay to the secretary an application fee. Such application fee 50 shall be prescribed pursuant to the rules and regulations of the secre- 51 tary. 52 4. A license granted pursuant to this article shall be valid unless 53 revoked or suspended by the secretary or surrendered by the licensee. 54 § 412. Supplemental license. 1. Where a person other than a natural 55 person is licensed under this article, such person shall apply for a 56 supplemental license for each additional high-risk advanced artificialA. 8195 11 1 intelligence system such person develops after being licensed initially 2 pursuant to section four hundred eleven of this article. 3 2. Notwithstanding any provision of law, rule or regulation to the 4 contrary, a supplemental license shall be provided in the same manner as 5 a license granted pursuant to the provisions of section four hundred 6 eleven of this article and shall be subject to the same requirements, 7 duties and prohibitions as provided for in this article. 8 § 413. Application for licenses. 1. An application for a license 9 required under this article shall be in writing, under oath, and in the 10 form prescribed by the secretary, and shall contain the following: 11 (a) the exact name and address of the applicant, and if the applicant 12 be a co-partnership or association, the names of the members thereof, 13 and if a corporation the date and place of its incorporation; 14 (b) the name and the business and residential address of each member 15 of the ethics and risk management board, each principal, and officer of 16 the applicant; and 17 (c) the description of all known general use cases of the advanced 18 artificial intelligence system, including any purposes foreseen to be 19 implemented by the applicant. A "use case" shall be defined as broad 20 category of potential use. 21 2. After the filing of an application for a license accompanied by 22 payment of the fees for license and investigation, it shall be substan- 23 tively reviewed. After the application is deemed sufficient and 24 complete, the secretary shall issue the license, or the secretary may 25 refuse to issue the license if the secretary shall find that the ethics, 26 experience, character and general fitness of the applicant or any person 27 associated with the applicant are not such as to command the confidence 28 of the community and to warrant the belief that the business will be 29 conducted honestly, fairly and efficiently within the purposes and 30 intent of this article. 31 3. If the secretary refuses to issue a license, the secretary shall 32 notify the applicant of the denial, return to the applicant the sum paid 33 as a license fee, but retain the investigation fee to cover the costs of 34 investigating the applicant. 35 4. Each license issued pursuant to this article shall remain in full 36 force unless it is surrendered by the licensee, revoked or suspended. 37 § 414. License provisions and posting. 1. Any license issued under 38 this article shall state the name and address of the licensee, and if 39 the licensee be a co-partnership or association, the names of the 40 members thereof, and if a corporation the date and place of its incorpo- 41 ration. 42 2. Such license or licenses shall be kept conspicuously posted in the 43 office of the licensee and, where such licensee has a public internet 44 presence, on the website or mobile application of the licensee and shall 45 not be transferable or assignable. 46 § 415. Grounds for suspension or revocation of license; procedure. 1. 47 A license granted pursuant to this section may not be renewed, and may 48 be revoked or suspended by the secretary upon a finding that: 49 (a) the licensee has not complied with reporting requirements; 50 (b) the licensee has violated any provision of this article; 51 (c) the licensee knowingly allowed a non-certified third-party system 52 to integrate with the licensee's system; 53 (d) any fact or condition exists which, if it had existed at the time 54 of the original application for such license, clearly would have 55 warranted the secretary's refusal to issue such license; orA. 8195 12 1 (e) the licensee has failed to pay any sum of money lawfully demanded 2 by the secretary or to comply with any demand, ruling or requirement of 3 the secretary. 4 2. Any licensee may surrender any license by delivering to the secre- 5 tary written notice that the licensee thereby surrenders such license, 6 but such surrender shall not affect such licensee's civil or criminal 7 liability for acts committed prior to such surrender. 8 3. Every license issued hereunder shall remain in force and effect 9 until the same shall have been surrendered, revoked or suspended in 10 accordance with the provisions of this article, but the secretary shall 11 have authority to reinstate suspended licenses or to issue new licenses 12 to a licensee whose license or licenses shall have been revoked if no 13 fact or condition then exists which clearly would have warranted the 14 secretary's refusal to issue such license. 15 4. Whenever the secretary shall revoke or suspend a license issued 16 pursuant to this article, the secretary shall forthwith execute a writ- 17 ten order to that effect. 18 5. The secretary may, on good cause shown, or where there is a 19 substantial risk of public harm or substantial risk of a system becoming 20 uncontained, without notice and a hearing, suspend any license issued 21 pursuant to this article for a period not exceeding thirty days, pending 22 investigation. 23 § 416. Ethics and risk management board and reports. 1. Every operator 24 of a licensed high-risk advanced artificial intelligence system or 25 systems shall establish an ethics and risk management board composed of 26 no less than five individuals who shall have the responsibility to 27 assess the ethical implications of all possible use cases of the system, 28 whether such use cases are intended or unintended, and whether likely or 29 unlikely to be used, and the current operational outcomes of the system. 30 Such operator, other than an operator who is a natural person, operating 31 more than one high-risk advanced artificial intelligence system with a 32 supplemental license shall not be required to have more than one ethics 33 and risk management board for each system. 34 2. No member of an ethics and risk management board shall be a member, 35 officer, or director within the operator's entity. No member shall be 36 required to be employed by the operator. 37 3. Such board shall adopt rules governing its decision-making proc- 38 esses, duties and responsibilities. Such rules shall not conflict with 39 the provisions of this article. 40 4. Annually, the ethics and risk management board of each operator 41 shall submit to the secretary a comprehensive report for each licensed 42 high-risk advanced artificial intelligence system which consists of the 43 following: 44 (a) All possible use cases, whether intended or unintended, whether 45 likely or unlikely. 46 (b) A thorough risk assessment for each use case, considering and 47 evaluating the potential for harm, irrespective of the probability of 48 such risk materializing. This shall include, but not be limited to, the 49 system's potential impact on privacy, security, fairness, economic 50 implications, societal well-being, and safety of persons and the envi- 51 ronment. 52 (c) A detailed evaluation of known use cases of the system by users, 53 exploring whether certain applications ought to be constrained or banned 54 due to ethical considerations. This shall include an assessment of the 55 operator's capacity to impose such constraints on use cases.A. 8195 13 1 (d) A mitigation plan for each identified risk, including preemptive 2 measures, monitoring processes, and responsive actions. This shall also 3 include a communication strategy to inform users and stakeholders about 4 potential risks and steps taken to mitigate them. 5 (e) A comprehensive review of any incidents or failures of the system 6 in the past year, detailing the circumstances, impacts, measures taken 7 to address the issue, and modifications made to prevent such incidents 8 in the future. 9 (f) Any existing attempts to educate users and, based on the existing 10 use of the system by users, a detailed plan on how the operator intends 11 to inform and instruct users on the safe and ethical use of the system, 12 considering varying levels of digital literacy among users. 13 (g) A disclosure of any conflicts of interest within the ethics board, 14 which could potentially influence the board's decisions and recommenda- 15 tions. This shall include measures to manage and resolve such conflicts. 16 (h) An update on the measures taken by the operator to ensure the 17 system's adherence to existing laws, regulations, and ethical guidelines 18 related to artificial intelligence. 19 5. In addition to any applicable civil penalties pursuant to section 20 four hundred eight of this article, a member of an ethics and risk 21 management board who makes a false statement, fails to disclose 22 conflicts of interest or misrepresents the risks or severity of the 23 risks posed by a system in the performance of his or her duties as a 24 member of such board, shall be guilty of a misdemeanor and, upon 25 conviction, shall be fined not more than five hundred dollars or impri- 26 soned for not more than six months or both, in the discretion of the 27 court. 28 § 417. Source code and outcome review. 1. The secretary shall conduct 29 periodic evaluations of the source code and outcomes associated with 30 each high-risk advanced artificial intelligence system. These examina- 31 tions shall determine whether the system is in compliance with this 32 article. The timing and frequency of these reviews shall be determined 33 at the secretary's discretion, taking into account the potential risk 34 posed by the system, the complexity of the system, the frequency of 35 updates and upgrades, the complexity of such updates and upgrades, and 36 any previous issues of non-compliance. 37 2. Upon completion of the review, the secretary is empowered to make 38 binding recommendations to the operator to ensure the system's function- 39 ality and outcomes are aligned with the principles in the advanced arti- 40 ficial intelligence ethical code of conduct pursuant to section four 41 hundred twenty-nine of this article, restrictions on prohibited artifi- 42 cial intelligence systems pursuant to section four hundred thirty of 43 this article, and limitations and procedures for source code modifica- 44 tions, updates, upgrades, and rewrites pursuant to section four hundred 45 nineteen of this article. 46 3. Following receipt of the secretary's recommendations, the operator 47 shall consult with the secretary to determine the feasibility of imple- 48 menting the recommendations and the time frame in which such recommenda- 49 tions can be implemented to ensure full compliance with the secretary's 50 recommendations. The operator shall provide a detailed plan outlining 51 how the recommendations will be addressed, along with a timeline for 52 their implementation. The detailed plan shall be binding on the opera- 53 tor; provided however that where an unexpected occurrence arises which 54 causes changes to such plan, the operator shall be entitled to extend 55 such timeline or alter such plans where such operator notifies the 56 secretary in writing regarding the unexpected occurrence and, withinA. 8195 14 1 such writing, sets forth amendments to the detailed plan and timeline. 2 The secretary shall have thirty days to approve or reject such amend- 3 ments. Where such amendments are rejected, the operator shall continue 4 with their original plan and timeline. 5 4. The secretary shall monitor the operator's compliance with such 6 recommendations and may impose fines and other penalties pursuant to the 7 provisions of this article for non-compliance that the secretary shall 8 deem just and proportionate to the violation. 9 § 418. Willfully or negligently uncontaining high-risk source code. 1. 10 No licensee or non-licensee who develops a high-risk advanced artificial 11 intelligence system shall willfully or negligently uncontain their 12 source code except where authorized by the secretary in writing. 13 2. Any member, officer, director or employee of an entity who willful- 14 ly violates subdivision one of this section shall be guilty of a class E 15 felony. 16 3. Any member, officer, director or employee of an entity who negli- 17 gently violates subdivision one of this section shall be guilty of a 18 class A misdemeanor. 19 4. Where any member, officer, director or employee or an entity will- 20 fully or negligently uncontains a high-risk advanced artificial intelli- 21 gence system described in paragraph (f) of subdivision two of section 22 four hundred one of this article or a prohibited high-risk advanced 23 artificial intelligence system as described in section four hundred 24 thirty of this article shall be guilty of a class C felony. 25 5. The provisions of this section shall not be construed as imposing 26 liability on any member, officer, director or employee who had no 27 explicit or implicit knowledge of the risk or circumstances that caused 28 the uncontainment of the high-risk advanced artificial intelligence 29 system. 30 § 419. Source code modifications, updates, upgrades, and rewrites. 1. 31 Where a licensee intends to modify or upgrade the source code of his or 32 her high-risk advanced artificial intelligence system, such licensee 33 shall be required to inform the secretary of such modification or 34 upgrade and shall be prohibited from implementing such modification or 35 upgrade in an accessible version of the system without express consent 36 of the secretary in writing. This section shall not apply to source code 37 updates. 38 2. A licensee shall, in writing to the secretary, set forth the 39 purpose of the modification or upgrade, the new functions added to the 40 system or the functions modified, the reason for the modification or 41 upgrade, and an assessment of new risks or risks that may be more proba- 42 ble as a result of the modification or upgrade. The secretary shall, 43 upon receipt of notice, have thirty business days to provide the licen- 44 see with approval of the modification or upgrade. Where approval is not 45 received within thirty business days, absent an extension in writing 46 which shall not exceed thirty additional business days, the modification 47 or upgrade shall be deemed approved. Nothing in this subdivision shall 48 be construed as limiting the ability of the secretary to take any action 49 he or she is authorized to take in relation to the approved modification 50 or upgrade. Where the secretary rejects the modification or upgrade, the 51 secretary shall set forth in writing the reasons for the rejection and 52 steps that the licensee can take to receive approval. Where the secre- 53 tary approves the modification or upgrade, the licensee may immediately 54 implement such modification or upgrade in a publicly accessible version. 55 3. A licensee who rewrites the source code of its system shall comply 56 with the same standards set forth in subdivisions one and two of thisA. 8195 15 1 section provided however that the secretary shall examine such source 2 code in the same manner as a new application and shall provide a letter 3 of approval or rejection upon completion of such review within one 4 hundred eighty business days of receipt of such notices except where the 5 secretary requires an extension of time, then an extension of no more 6 than one hundred eighty days shall be authorized. Where the secretary 7 rejects the rewrite, such letter of rejection shall state the reasons 8 for the rejection and steps that the licensee can take to correct such 9 rejection, if any. Where the secretary approves the modification or 10 upgrade, the licensee may immediately implement such modification or 11 upgrade in a publicly accessible version. 12 4. All modifications, upgrades, and rewrites shall be conducted in a 13 pre-production environment, which shall mean any stage prior to the 14 accessible version. 15 5. For purposes of this section: 16 (a) "Modify" shall mean altering the source code of the system to 17 alter the way by which the system, or any features within the system, 18 makes decisions. 19 (b) "Upgrade" shall mean altering the source code of the system which 20 gives it new features or functions. 21 (c) "Rewrite" shall mean a change in the source code to such a 22 substantial degree that: 23 (i) it effectively results in a new version of the system; or 24 (ii) the change nullifies all or a substantial amount of the initial 25 findings of the secretary in the operator's original application. 26 (d) "Update" shall mean a change to the source code that includes 27 minor enhancements, improvements, modifications, error corrections, 28 cosmetic changes, or any other change intended to increase the function- 29 ality, compatibility, security or performance of the system. 30 (e) "Accessible version" shall mean a version of the software that is 31 available to the public or for private use or that is presently operat- 32 ing within its designated operational parameters. 33 § 420. Malfunction and incident reports; duty to notify. 1. A licensee 34 shall have the duty to notify the department and, if applicable, a rele- 35 vant law enforcement agency or governmental entity where the licensee's 36 system fails to operate as intended for any significant period of time. 37 A period of time is deemed "significant" for purposes of this section 38 where the period of time that the malfunction occurred had the capacity 39 to or has harmed a person or persons. 40 2. A licensee shall have the duty to notify a relevant law enforcement 41 agency or governmental entity of a malfunction where designated by the 42 department upon receipt of a license. The secretary shall issue such a 43 requirement upon the licensee where such systems interact with law 44 enforcement systems or the systems of a government agency, engage in law 45 enforcement functions or the functions of a government agency, or where 46 such systems operate, in whole or in part, or are, a weapon. 47 § 421. State and national security risks. 1. The secretary may, by 48 regulation, designate unique requirements for systems which, in the 49 secretary's discretion, pose a risk to state or national security. Such 50 systems shall be assessed on a case-by-case basis and shall not be 51 liberally construed as including any system that, where used improperly, 52 inherently possesses the ability to harm persons or property. 53 2. A high-risk advanced artificial intelligence system shall be deemed 54 to pose a risk to state or national security where the system's malfunc- 55 tioning or misuse poses a high risk of: 56 (a) disrupting critical infrastructure;A. 8195 16 1 (b) triggering or escalating existing conflicts; 2 (c) undermining or impacting the democratic process; 3 (d) causing unauthorized access to classified information as desig- 4 nated by a relevant governmental entity; 5 (e) harming a significant portion of the population or a specific 6 segment of the population; 7 (f) negatively impacting financial markets or economic stability; 8 (g) causing consequential or irreversible damage to the environment; 9 or 10 (h) causing significant harm to the social fabric. 11 § 422. Information and source code sharing. 1. Licensees shall be 12 permitted to share information and source code with any third party, 13 provided however, that where information is biometric information such 14 party shall be jointly liable for any harm or violations under this 15 article with the licensee. The secretary may, in his or her discretion, 16 prohibit any person from accessing the information or source code of a 17 licensee provided however that the secretary shall provide a written 18 justification for such a prohibition. 19 2. For purposes of this section, "biometric information" shall include 20 a person's: 21 (a) faceprint; 22 (b) voiceprint; 23 (c) fingerprint; 24 (d) gaitprint; 25 (e) irisprint; 26 (f) psychological profile; or 27 (g) any other data related to a person's body or mind that can be used 28 to identify a person. 29 3. This section shall only apply to the sharing of information 30 received or generated by the licensee or source code created by the 31 licensee and shall not apply to a third party integrating their systems 32 with the licensee. 33 § 423. Third-party systems; certificates of compliance. 1. Non-licen- 34 see third-party systems may integrate with a licensee under the follow- 35 ing conditions: 36 (a) Where a third-party system assists in the proper functioning of 37 the licensee or where such system provides additional services to the 38 licensee's service-offerings, such a system shall not be required to 39 obtain a license but shall be required to obtain a certificate of 40 compliance in accordance with this section. 41 (b) No third-party system may access the system of a licensee to 42 provide itself with new high-risk advanced artificial intelligence capa- 43 bilities without first obtaining a license. 44 2. Every third-party system which integrates with a licensee shall, 45 prior to integration, apply for and receive a certificate of compliance. 46 Such certificate shall be issued by the department and shall only be 47 issued where such third-party system is assessed by the department and 48 the department finds it conforms to the cybersecurity standards set by 49 the office. The secretary shall set the rules and regulations regarding 50 the application and requirements of receiving a certificate of compli- 51 ance. This section shall not be construed as requiring any third-party 52 system to receive more than one certificate of compliance. 53 § 424. Logging. Every time a licensee's system operates it shall auto- 54 matically generate a log. Standards related to the specific types of 55 events that are required to be logged, the format in which logs must be 56 kept, the individuals or entities permitted to access logs and theA. 8195 17 1 conditions governing such access, the encryption and cybersecurity 2 protocols to be applied to logs, the procedures for both the preserva- 3 tion and disposal of logs, and any other actions pertinent to log 4 management shall conform to the standards set by the secretary. Such 5 logs shall be preserved for a period of ten years from the date they are 6 generated and shall be subject to inspection under section four hundred 7 twenty-six of this article. 8 § 425. Internal controls; ceasing operation. Every licensee shall have 9 in place internal controls that, within a reasonable time following 10 initiation, can safely and indefinitely cease the operation of the 11 system or a major part of the system. 12 § 426. Investigations and examinations. 1. The secretary shall have 13 the power to make such investigations as the secretary shall deem neces- 14 sary to determine whether any operator or any other person has violated 15 any of the provisions of this article, or whether any licensee has 16 conducted itself in such manner as would justify the revocation of its 17 license, and to the extent necessary therefor, the secretary may require 18 the attendance of and examine any person under oath, and shall have the 19 power to compel the production of all relevant books, records, accounts, 20 documents, source code, and logs. 21 2. The secretary shall have the power to make such examinations of the 22 books, records, accounts, documents, source code, and logs used in the 23 business of any licensee as the secretary shall deem necessary to deter- 24 mine whether any such licensee has violated any of the provisions of 25 this article. 26 3. The expenses incurred in making any examination pursuant to this 27 section shall be assessed against and paid by the licensee so examined, 28 except that traveling and subsistence expenses so incurred shall be 29 charged against and paid by licensees in such proportions as the secre- 30 tary shall deem just and reasonable, and such proportionate charges 31 shall be added to the assessment of the other expenses incurred upon 32 each examination. Upon written notice by the secretary of the total 33 amount of such assessment, the licensee shall become liable for and 34 shall pay such assessment to the secretary. 35 4. All reports of examinations and investigations, and all correspond- 36 ence and memoranda concerning or arising out of such examinations or 37 investigations, including any duly authenticated copy or copies thereof 38 in the possession of any licensee or the department, shall be confiden- 39 tial communications, shall not be subject to subpoena and shall not be 40 made public unless, in the judgment of the secretary, the ends of 41 justice and the public advantage will be subserved by the publication 42 thereof, in which event the secretary may publish or authorize the 43 publication of a copy of any such report or other material referred to 44 in this subdivision, or any part thereof, in such manner as the secre- 45 tary may deem proper. 46 § 427. Books, records, source code, and logs to be kept. 1. Every 47 operator shall maintain such books, records, source code, and logs as 48 the secretary shall require provided however that every operator shall, 49 at least, maintain a copy of all logs generated from the system as well 50 as a backup of every version of the system which shall be stored in a 51 safe manner as prescribed by the secretary. 52 2. By a date to be set by the secretary, each operator shall annually 53 file a report with the secretary giving such information as the secre- 54 tary may require concerning the business and operations during the 55 preceding calendar year of the operator within the state under the 56 authority of this article. Such report shall be subscribed and affirmedA. 8195 18 1 as true by the operator under the penalties of perjury and be in the 2 form prescribed by the secretary. In addition to such annual reports, 3 the secretary may require of operators such additional regular or 4 special reports as the secretary may deem necessary to the proper super- 5 vision of operators under this article. Such additional reports shall be 6 in the form prescribed by the secretary and shall be subscribed and 7 affirmed as true under the penalties of perjury. 8 § 428. Regulations and rulings. The secretary is hereby authorized and 9 empowered to make such rules and regulations, conduct hearings and make 10 such specific rulings, orders, demands and findings as may be necessary 11 for the proper conduct of the business authorized and licensed under and 12 for the enforcement of this article. 13 § 431. Severability. If any provision of this article or the applica- 14 tion thereof to any person or circumstances is held invalid, the inva- 15 lidity thereof shall not affect other provisions or applications of the 16 article which can be given effect without the invalid provision or 17 application, and to this end the provisions of this article are severa- 18 ble. 19 § 2. Section 401 of the state technology law, as added by section 1 of 20 part A of this act, is amended by adding a new subdivision 11 to read as 21 follows: 22 11. "Log", "logs" or "logging" shall mean a systematic, chronological- 23 ly ordered record of events pertaining to a system's operations, activ- 24 ities, and transactions that is in compliance with the standards set by 25 the secretary in accordance with section four hundred twenty-four of 26 this article. In the context of logging, an event shall refer to any 27 significant or notable occurrence, action, or anomaly within the system. 28 § 3. Subdivisions 5 and 6 of section 408 of the state technology law, 29 as added by section 1 of part A of this act, are renumbered subdivisions 30 7 and 8 and two new subdivisions 5 and 6 are added to read as follows: 31 5. Any person who knowingly makes any incorrect statement of a materi- 32 al fact in any application, report or statement filed pursuant to this 33 article, or who knowingly omits to state any material fact necessary to 34 give the director any information lawfully required by the secretary or 35 refuses to permit any lawful investigation or examination, shall be 36 guilty of a misdemeanor and, upon conviction, shall be fined not more 37 than five hundred dollars or imprisoned for not more than six months or 38 both, in the discretion of the court. 39 6. No person shall make, directly or indirectly, orally or in writing, 40 or by any method, practice or device, a representation that such entity 41 is licensed under the law except that a licensee under this chapter may 42 make a representation that the licensee is licensed as a high-risk 43 advanced artificial intelligence system under this article. 44 § 4. This act shall take effect one year after it shall have become a 45 law. Effective immediately, the addition, amendment and/or repeal of any 46 rule or regulation necessary for the implementation of this act on its 47 effective date are authorized to be made and completed on or before such 48 effective date. 49 PART C 50 Section 1. The state technology law is amended by adding a new section 51 429 to read as follows: 52 § 429. Advanced artificial intelligence ethical code of conduct. The 53 following ethical code of conduct shall be binding on all licensees andA. 8195 19 1 non-licensees who develop or operate a high-risk advanced artificial 2 intelligence system: 3 Respect: Artificial intelligence systems should respect human autonomy 4 and not unduly influence or manipulate individuals' behavior or deci- 5 sions. 6 Equity: An artificial intelligence system should provide equitable 7 outcomes, irrespective of any characteristics protected by law. They 8 should not perpetuate existing biases, discrimination, or disparities. 9 Accountability: Persons that design, develop, deploy, or use artifi- 10 cial intelligence systems should be held accountable for the impacts and 11 outcomes of these systems except where the law provides otherwise. 12 Clear mechanisms for addressing harms and violations of law should be in 13 place. 14 Care: Artificial intelligence systems should not cause harm or 15 adversely affect individuals, society, or the environment without legal 16 justification. 17 Trust: Artificial intelligence systems should respect individuals' 18 privacy rights, and securely handle personal and sensitive data in 19 accordance with applicable laws and regulations. 20 Inclusivity: Artificial intelligence systems should be designed, 21 developed, and used in ways that are inclusive, serving a diverse range 22 of users and contexts. 23 Oversight: There should always be meaningful human oversight of arti- 24 ficial intelligence systems to ensure ethical use and decision-making. 25 Notice: The operations, decision-making processes, and use of artifi- 26 cial intelligence systems should, where feasible, be made known to 27 persons affected by them. 28 Safety: Artificial intelligence systems should be robust, secure, and 29 reliable. They should have mechanisms in place to prevent misuse or 30 harmful outcomes. 31 § 2. This act shall take effect on the one hundred eightieth day after 32 it shall have become a law. 33 PART D 34 Section 1. Legislative findings and determinations. The legislature 35 hereby finds and declares that in the instance where the source code of 36 any system with the capacity to destroy or disrupt the security, integ- 37 rity, and moral well-being of the state is created, a palpable opportu- 38 nity arises for such source code to be accessed by entities beyond the 39 jurisdictional boundaries of not only the state of New York but also the 40 United States; under these circumstances, no oversight, law, rule, regu- 41 lation or existing technologies can sufficiently thwart the potential 42 misuse of such a system. Therefore, in recognizing the severe impli- 43 cations of this threat, the legislature hereby declares that certain 44 applications of artificial intelligence harbor such an immense capacity 45 for causing significant harm to the security, integrity, and moral well- 46 being of the state and such a risk of becoming uncontainable that the 47 state has a compelling interest in preventing their creation. 48 § 2. The state technology law is amended by adding a new section 430 49 to read as follows: 50 § 430. Prohibited artificial intelligence systems. 1. No person shall 51 develop, in whole or in part, or operate an artificial intelligence 52 system within the state where such a system performs any of the follow- 53 ing, whether or not it is the system's main function:A. 8195 20 1 (a) the deployment of subliminal techniques that operate beyond an 2 individual's conscious awareness, with the express purpose of materially 3 distorting an individual's behavior in such a manner that leads to, or 4 possesses a high likelihood of leading to, physical or psychological 5 harm to that individual or another, or that leverages the vulnerabili- 6 ties of a defined group of individuals to similar ends; 7 (b) the infliction of physical or emotional harm upon individuals 8 without any valid law enforcement or self-defense purpose or justifica- 9 tion; 10 (c) the prediction of an individual's future actions or behaviors, 11 followed by subsequent reactions based on these predictions, carried out 12 in such a way that, without legal justification, infringes upon or 13 compromises the individual's liberty, emotional, psychological, or 14 financial interests; 15 (d) the unauthorized acquisition, retention, or dissemination of or 16 access to sensitive personal information or non-public data in violation 17 of applicable data privacy, security, and hacking laws; or 18 (e) the implementation of any form of autonomous weapon system 19 designed to inflict harm on persons, property, or the environment that 20 lack meaningful human supervision or control. "Meaningful human super- 21 vision or control" shall mean the ability to actively manage, intervene, 22 or override the autonomous weapon system's functions. 23 2. Where the secretary discovers the development or operation of a 24 prohibited artificial intelligence system, the secretary may, in writ- 25 ing, demand that the person who is developing or operating such system 26 cease development or operation of or access to such a system within a 27 period of time as the secretary deems necessary to prevent the system 28 from widespread use or, if the system is operational or accessible to 29 persons for use, to ensure the system is properly terminated in such a 30 way to minimize risks of harm to individuals, society, or the environ- 31 ment. A demand made pursuant to this section shall be finally and irre- 32 vocably binding on the person unless the person against whom the demand 33 is made shall, within such period of time set by the secretary, after 34 the giving of notice of such determination, petition the department for 35 a hearing to determine the legal findings of the secretary. The person 36 developing or operating such a prohibited system shall, prior to peti- 37 tion, cease development, operation, and access to the system until and 38 unless such determination is favorable to the person. Such determination 39 may be appealed by any party as of right. 40 3. The secretary shall not grant a license pursuant to this article to 41 any high-risk advanced artificial intelligence system described under 42 this section except as described in subdivision seven of this section. 43 4. Any member, officer, director or employee of an operator of any 44 entity who knowingly publicly or privately operates any system described 45 in this section shall be guilty of a class D felony and shall incur a 46 civil penalty of the amount earned from the creation of the prohibited 47 system or the amount of damages caused by the system, whichever is 48 greater. 49 5. This section shall not be construed as imposing liability on any 50 member, officer, director or employee who had no explicit or implicit 51 knowledge of the prohibited high-risk advanced artificial intelligence 52 system provided however that where the secretary sends a demand to cease 53 the development, operation, or access to such system all members, offi- 54 cers, and directors shall be rebuttably presumed to have knowledge of 55 the prohibited high-risk advanced artificial intelligence system.A. 8195 21 1 6. This section shall be construed as prohibiting the development of a 2 prohibited high-risk advanced artificial intelligence system or making 3 such a system accessible to persons in the state of New York. 4 7. Notwithstanding subdivision one of this section, a person may 5 develop a prohibited high-risk advanced artificial intelligence system 6 where authorized by the secretary, provided that such system is devel- 7 oped and used only by the state or with substantial, continuous over- 8 sight by the state and such system is authorized only after public hear- 9 ing and comment in accordance with section four hundred nine of this 10 article. 11 § 2. This act shall take effect one year after it shall have become a 12 law. Effective immediately, the addition, amendment and/or repeal of any 13 rule or regulation necessary for the implementation of this act on its 14 effective date are authorized to be made and completed on or before such 15 effective date. 16 § 4. Severability clause. If any clause, sentence, paragraph, subdivi- 17 sion, section or part of this act shall be adjudged by any court of 18 competent jurisdiction to be invalid, such judgment shall not affect, 19 impair, or invalidate the remainder thereof, but shall be confined in 20 its operation to the clause, sentence, paragraph, subdivision, section 21 or part thereof directly involved in the controversy in which such judg- 22 ment shall have been rendered. It is hereby declared to be the intent of 23 the legislature that this act would have been enacted even if such 24 invalid provisions had not been included herein. 25 § 5. This act shall take effect immediately; provided, however, that 26 the applicable effective date of Parts A through D of this act shall be 27 as specifically set forth in the last section of such Parts.
