SB 1047 and the open-weights question it never resolved

What the bill actually covered

Governor Newsom returned SB 1047 without his signature on September 29. The bill, authored by Senator Wiener, passed the Assembly on August 28 and the Senate on August 29 after ten rounds of amendment between March and August. It would have required developers of the largest models to adopt a written safety and security protocol before training, retain the ability to shut a model down, submit to annual third-party audits from 2026, and face civil action from the Attorney General for violations.

The threshold was compute and cost together. Before 2027 a covered model was one trained with more than 10^26 integer or floating-point operations at a cost above one hundred million dollars. A second threshold applied to fine-tuning. Anyone who adjusted the weights of a covered model using more than 3 times 10^25 operations at a cost above ten million dollars became a developer in their own right.

Those two numbers are where the open-weights argument lived. Everything else in the bill, the shutdown capability, the audits, the whistleblower provisions, the CalCompute cluster, followed from the answer to one question. When a model is released and someone else changes it, who is the developer?

Derivatives, and who controls them

The text handled this with a definition of covered model derivative. It included an unmodified copy of a covered model, a copy subjected to post-training modifications unrelated to fine-tuning, a copy fine-tuned below the ten million dollar threshold, and a copy combined with other software. Read that list as a developer of an open-weights model and it describes almost every downstream artefact anyone will ever build from your release.

The bill's definition of developer then drew the line. A person who fine-tunes a covered model using more than the threshold amount of compute and cost performs initial training of a new covered model and takes on the obligations. Below that line the modified model stays a derivative of the original. In practice that means a hobbyist LoRA on a released checkpoint is the original developer's derivative, and a ten million dollar continued pretraining run is someone else's model.

The shutdown provision is the part we think open-weights developers should have read most carefully, because it was narrower than the public debate suggested. Full shutdown was defined as ceasing the training of a covered model, ceasing a covered model controlled by a developer, and ceasing all covered model derivatives controlled by a developer. The word controlled does the work. A checkpoint on someone else's hard drive is not controlled by you, and the definition did not ask you to reach it.

The gap between the text and the fear

That leaves an odd shape. The shutdown duty only reached derivatives you control, but the definition of derivative swept in copies you do not control, and the assessment duty under section 22603 asked developers to assess whether the covered model was reasonably capable of causing or materially enabling a critical harm before deployment. Critical harm meant mass casualty weapons, attacks on critical infrastructure causing more than five hundred million dollars of damage, or autonomous conduct causing death or injury without adequate human oversight.

So the honest reading is that a developer of an open model could never be ordered to shut down what it had already released. It could still be asked whether releasing it was reasonable given what someone could do with a cheap fine-tune. The bill never said how a developer should weigh a modification it cannot see against harms it cannot predict. The amendments narrowed the fine-tuning threshold and the enforcement, but they did not add that guidance.

The bill also did not carve out non-commercial or open releases. It did reserve a seat on its nine-member Board of Frontier Models for the open-source community, alongside industry, weapons experts, safety and cybersecurity specialists and academics. A seat at a table that would set future thresholds is something. It is not an answer to the liability question.

What the veto said, and what it did not

The veto message is three pages and worth reading in full. Newsom writes that California is home to 32 of the world's 50 leading AI companies, and that the key debate is whether regulation should key off cost and compute or off a system's actual risks. His main objection is that the bill applies stringent standards to even the most basic functions, so long as a large system deploys it, without asking whether the system is used in a high-risk environment, involves critical decisions or touches sensitive data. He also warns that smaller, specialised models may emerge as equally or even more dangerous than the models targeted by SB 1047.

Notice what is absent. The message never uses the words open source, open weights or derivative. The governor's objection is that the threshold is too coarse. He does not say a downloaded checkpoint should or should not carry liability back to the lab that released it. The same-day announcement of an expert group led by Fei-Fei Li, Tino Cuellar and Jennifer Tour Chayes to develop guardrails likewise says nothing about open models. The seventeen AI bills he signed in the same month cover deepfakes, likeness rights, election content, health care review and training data disclosure, and none of them touch the question either.

What is still open

Two things survive the veto. The first is the fine-tuning threshold as a concept. Ten million dollars of compute is a specific, checkable line, and it will be the starting point for whoever drafts the next bill, in Sacramento or elsewhere. Whether that line is right depends on a question nobody has measured well, which is how much a cheap fine-tune can change what a model will do.

The second is the control test. If a future statute keeps shutdown duties limited to what a developer controls, open-weights releases are structurally exempt from the strongest remedy in the bill, and the whole burden shifts onto the pre-release assessment. That is the piece we would want to see written carefully next time. What is a reasonable assessment of a release whose most dangerous version will be built by someone else, and how would a developer show they made it? The bill did not say. The veto did not say. We expect the next draft to try, and we would like the people who publish open models to be in the room when it does.