Pause, or shut it all down: the two March letters

What the pause letter asked for

The Future of Life Institute published its open letter on March 22, 2023, two weeks after GPT-4 shipped. The text is shorter than the discussion around it. It calls on all AI labs to immediately pause for at least six months the training of AI systems more powerful than GPT-4. It asks that the pause be public and verifiable and include all key actors, and it says that if such a pause cannot be enacted quickly, governments should step in and institute a moratorium.

Two details in the wording got lost in most of the coverage. First, the scope is training runs above the GPT-4 threshold, so deploying existing models, fine-tuning them, and doing research on them was explicitly out of scope. Second, the letter describes what the six months are for. Labs and independent experts are meant to jointly develop shared safety protocols for advanced AI design, and the field is meant to refocus on making the current generation of systems more accurate, safe, interpretable, transparent, aligned and trustworthy. In parallel it asks for regulatory authorities, oversight and tracking of highly capable systems, provenance and watermarking, auditing and certification, liability rules and public funding for safety research.

The letter's page now shows 31,810 signatures. The names that carried it were Yoshua Bengio, Stuart Russell, Elon Musk, Steve Wozniak, Yuval Noah Harari, Max Tegmark, Emad Mostaque and Tristan Harris. It is a coalition of people who disagree with each other about almost everything else in AI, which is both why it got attention and why it could not agree on what a pause would mean in practice.

What the TIME piece asked for

On March 29, Eliezer Yudkowsky published an opinion piece in TIME explaining why he had not signed. His reason was that the letter understated the danger. He wrote that the most likely result of building a superhumanly smart AI under anything like current conditions is that literally everyone on Earth will die, and that a six-month pause does nothing to change that.

His proposal was categorical rather than temporary. Shut it all down. An indefinite worldwide moratorium on large training runs with no exceptions for governments or militaries. Shut down the large GPU clusters. Cap the compute allowed for training and ratchet the cap down over time. Track all GPUs sold. Enforce this through international agreement, and be willing to destroy a rogue datacenter by airstrike if a country builds one in violation. The piece closes with the line that we are not ready and are not on track to be significantly readier.

Placed next to the pause letter, the TIME piece reads as a deliberate refusal of the coalition strategy. The letter was written to be signable by a chip executive and a cognitive scientist at the same time. The TIME piece was written so that it could not be softened into a policy memo. Whatever one thinks of the argument, the two documents were not asking for two sizes of the same thing.

Who said no, and why

The refusals were as informative as the signatures. Sam Altman said the letter lacked technical nuance and disputed the claim that OpenAI was training GPT-5. Bill Gates declined to sign, arguing that asking one group to pause does not solve the challenges. Reid Hoffman called it virtue signalling that would have minimal impact. From a different direction entirely, Timnit Gebru and coauthors argued that the letter amplified a futuristic dystopian science fiction scenario and drew attention away from harms that models were causing right then.

Those objections came from three camps that had nothing else in common. The labs objected that they were the ones being asked to stop. The investors objected that a unilateral pause hands the lead to whoever ignores it. The critics from the algorithmic harms side objected that the letter's premise was wrong. A pause needs either voluntary compliance from the first camp or government enforcement over the objections of the second, and the letter had neither.

Why neither happened

The pause letter's own text contains the reason it failed. It asks for a pause that is public, verifiable and includes all key actors, and it offers no mechanism for any of those three properties. Verification of whether a lab is training a model above the GPT-4 threshold requires access to compute accounting that nobody outside the lab had in March 2023. Including all key actors requires labs in more than one country to agree, and no government had a body ready to convene them. The fallback, a government-imposed moratorium, would have required legislation that did not exist in draft form anywhere.

The TIME proposal fails for a simpler reason. It asks states to treat training runs as something closer to weapons programmes, and it does so at a moment when the public evidence of danger was a chatbot that was good at exams. The piece was written to shift the boundaries of the discussion rather than to be adopted. On that narrower goal it did something. For the rest of the year, arguments about timelines and moratoria took place in a space that the two documents had staked out at either end.

No pause occurred. Training of larger models continued at every major lab, and by every account since, development accelerated rather than slowed. What the letter did produce was a burst of governmental attention to regulation that would otherwise have taken longer to arrive.

What we take from the two documents

The lasting value of the pause letter, for us, is the list of things it said should be built during the six months. Auditing and certification, provenance and watermarking, liability rules, public funding for safety research. Every item on that list can be worked on without a pause and without anyone agreeing on whether the pause was a good idea, and most of them are what the field's governance effort has become. The pause itself was the least workable part of a document that was mostly a to-do list.

What we would like someone to try is to score the two letters against each other in a year. Take each concrete ask, from shared safety protocols to compute caps, and record which ones now have any institutional home. Our guess is that the pause letter's asks will mostly have one, in weak form, and the TIME piece's asks will have none. That result would tell us something about what open letters can do, which is set an agenda, and what they cannot, which is stop anyone from training anything.