Desks, badges and laptops
Anthropic's CEO asked the whole industry to slow down yesterday and his rivals agreed within hours. The one firm commitment is outside evaluators with desks, badges and laptops. Three earlier slowdowns explain why that small ask is the right shape.
Yesterday morning Dario Amodei, who runs Anthropic, published an essay called We Must Pace the Frontier. The line everyone quoted was “We must slow the pace at which we improve the capabilities of AI models.” Within hours Elon Musk had posted “Dario is right” and Sam Altman had agreed and said OpenAI would follow suit. Rivals who spend the rest of the year racing each other agreed on this in a matter of hours.
The part I found more interesting is what Anthropic actually committed to. Outside evaluators get “Desks in our offices, access badges, and company laptops,” roughly the same access as the internal risk teams, and the right to publish what they find. That’s the whole unilateral commitment. It sounds tiny next to a headline about slowing down an entire industry, so I went back and read about the other times the people building something asked everyone to slow down. I’ve come around to thinking tiny is the point.
1939. Leo Szilard worked out that uranium fission might make a bomb and asked physicists to stop publishing. He got Isidor Rabi to put the idea to Enrico Fermi, whose complete review was one word: “Nuts!” The Americans and the British mostly went along anyway. Frédéric Joliot in Paris refused, and on April 22 his team published in Nature that each fission throws off about 3.5 neutrons, enough to keep a chain reaction going. The official US history of the bomb records that “publication continued freely for about another year.” A big vague ask, one credible refusal, and the whole thing fell over.
1974. Paul Berg and ten colleagues wrote to Science asking biologists to hold off on two specific kinds of recombinant DNA experiment until they could meet and agree some rules. Seven months later about 150 of them did meet, on the beach at Asilomar in California. Sixteen journalists were let in on the condition that they filed nothing until it was over. After three days of arguing they voted the rules through by a show of hands. About four hands went up against, one of them James Watson’s. The NIH turned the result into a formal rulebook on June 23, 1976. Genentech had been founded eleven weeks earlier, and human insulin made in bacteria was in pharmacies by 1982. The pause held because it was short, it named the experiments it covered, and the people it applied to wrote it themselves.
2023. The Future of Life Institute published an open letter asking every lab to “immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.” Over thirty thousand people signed. Elon Musk was the famous name at the top. He had incorporated xAI in Nevada thirteen days earlier, and that May he told CNBC he’d signed “knowing it was futile” and just wanted to be on record. Nobody paused. The strange part is that governments moved anyway: voluntary commitments from the labs at the White House that July, the Bletchley summit in November, and the EU finishing its AI Act. The builders said no and the governments said yes.
2026. Now hold that up to a mirror. This time the builders are asking. A letter that went up in July now carries 1,386 signatures from people at OpenAI, Anthropic, Google DeepMind, Meta and a few others, asking Washington to help “deliberately pace the frontier.” In August OpenAI paused some frontier training for a couple of weeks, and Altman said “I think it is a good time to slow down.” Anthropic moved about 150 product engineers onto security and paused its riskier training environments for several weeks. And the government? Treasury Secretary Scott Bessent said last week, “We can’t pause. You can’t, because the Chinese won’t pause.” Nvidia’s Jensen Huang has called the extinction talk “complete nonsense.” David Sacks, the White House AI czar, said on Fox the warnings are “designed to scare the public” so NGOs can push “a government takeover.” The builders are saying yes and the government is saying no. It’s 2023 upside down.
You’d expect the public to have flipped as well. It hasn’t. In April 2023 YouGov asked Americans how concerned they were that AI “will cause the end of the human race on Earth,” and 46% said very or somewhat. It asked again this July: 47%. Three years of headlines moved the number by one point. People were already scared. What changed is who’s asking for the brakes.
Which brings me back to the desks. Dario’s essay says the 2023 pause idea “made little sense back then,” and that a real global pause is “unlikely to actually happen any time soon.” I think he’s read the same history. His commitment is Asilomar-shaped rather than pause-letter-shaped: small, specific, checkable, and written by the people it applies to. A stranger in your office with the same access as your own risk team, free to publish, is a much harder thing to fake than a press release about slowing down. Altman says OpenAI will do the same. So I’m not watching for a treaty. I’m counting badges.