New York bill would require safety checks for powerful AI models

Large frontier developers would arrange independent safety evaluations before deploying covered models. This applies to frontier models deployed in whole or in part in New York, including models already deployed there when this section takes effect. Evaluators would need to meet office standards, and developers would pay reasonable evaluation costs. The evaluator would report findings but would not certify that a model is safe. The section would take effect one year after the act becomes law, with a different start if chapter 96 of the laws of 2026 takes effect later. Already deployed models would be evaluated within 180 days after this section takes effect.

Developers would monitor models they control, keep safety records, and secure privileged access and model weights. These provisions apply to large frontier developers. Monitoring would cover a model only while its developer retains reasonable technical or operational control; it would not cover an independently controlled copy after control is lost. Monitoring would need to limit collection and storage of unrelated user information where reasonably feasible. Developers would assess risks before broadly releasing or transferring model weights when that step could weaken their ability to maintain safeguards. Required safety records would be kept for at least five years.

Developers would report serious safety incidents, address certain risks, and protect workers who disclose safety concerns. Incident reporting applies to large frontier developers. Worker protections cover their employees and contractors, including workers for contractors and subcontractors. Reports would be due as soon as practicable and no later than seven days after the developer determines an incident occurred or learns enough to reasonably believe it occurred. The office could temporarily restrict a specific activity only after written findings tying it to an unreasonable risk; it could act without advance notice if notice would increase an imminent risk. Worker protections would cover disclosures based on a reasonable belief, and the text would not create a private right to sue. Temporary restrictions could last up to 30 days unless renewed. If a restriction remains in effect for more than 60 days in total, the office would refer the matter to the attorney general for review.

The attorney general could seek civil penalties for specified knowing or willful violations. The penalties would apply to large frontier developers, with revenue potentially including certain affiliates that materially supported the model involved. Penalties could reach one-half of one percent of annual gross revenue for specified knowing conduct, or one percent for willfully hiding a critical safety incident or repeatedly obstructing oversight with intent to conceal or frustrate enforcement. A good-faith legal, scientific, or technical disagreement alone would not count as concealment, obstruction, or interference.