Ilya's straight shot: what a lab with no product is betting on

What was announced

On June 19, Ilya Sutskever, Daniel Gross and Daniel Levy announced Safe Superintelligence Inc., with offices in Palo Alto and Tel Aviv. Sutskever left OpenAI on May 15 after the board fight over the company's direction, and this is where he landed. The founding statement is short, and every sentence in it is a claim about how research should be organised.

The company will have, in its own words, one goal and one product, a safe superintelligence. It says safety, security and progress will be insulated from short-term commercial pressures. It promises no distraction by management overhead or product cycles. And it says it will approach safety and capabilities in tandem, as technical problems to be solved through engineering and scientific breakthroughs. There is no product roadmap because the product is the thing at the end.

The structural claim

Strip the ambition out and what remains is a hypothesis about labs. The hypothesis is that product cycles corrupt research. A lab that has to ship every quarter optimises for what ships, hires for what ships, and lets safety work slide to whatever the launch calendar allows. Remove the product and the corruption goes with it. That is the bet, and the structure of the company is the experiment.

It is a reasonable reading of the last two years. OpenAI began as a nonprofit research lab and ended up with a consumer app, an enterprise business and a board crisis. Whatever one thinks of the details, the pull of a product on a research organisation is visible from outside. Sutskever was there for all of it and has chosen the opposite structure, and that choice is itself evidence about what he thinks went wrong.

Where the insulation leaks

Our doubt is about whether commercial pressure comes only from customers. A lab that has no revenue has investors instead, and investors have timelines. The statement says the business model is designed so that safety and progress are insulated from short-term pressure. It does not say how, and the how is the entire question. Money that expects a return in ten years applies less pressure than money that expects it in two, but it applies some, and a lab that has been running for eight years with nothing to show will feel it.

The second leak is talent. The statement says the offices are located where the founders have deep roots and can recruit top technical talent. But researchers who want to see their work used tend to go where it gets used, and a lab whose only output is a distant superintelligence has to hold people on conviction alone. That works for a founding team. It is harder for the two hundredth hire.

The third is compute. A frontier training run is one of the most expensive things a private company can do, and the money for it has to come from somewhere. Either SSI raises enough to buy it outright, which means large investors and their expectations, or it partners with someone who sells compute, which means a relationship with a firm that does have products. There is no version of frontier research in 2024 that is fully insulated from the commercial world, because the commercial world owns the chips.

What the structure gets right

Having said all that, we think the founders are correct about one thing that other labs get wrong. Treating safety and capabilities as a single technical problem, rather than as a research group and a compliance group, is the framing that produces results. The best safety work of the last two years has come from people who train models and people who interpret them working on the same team. A structure that refuses to split them is a structure that could produce that work at scale.

The other thing the statement gets right is scope. Most labs are trying to do everything, and their safety agendas are as broad as their product lines. A lab with one target can define what safe means for that target and measure against it. Whether it will publish the measurements is an open question, and the statement says nothing about openness at all, which we notice.

What would change our mind

A funding round will happen and will be large, and it tells us nothing. What tells us something is what the lab looks like in three years. If it has published technical work on the safety problems it says it is solving, if its researchers are visible at conferences, and if it has held to a single project through at least one period when the field's attention moved elsewhere, then the insulation worked and we were wrong to doubt it.

If instead it has started shipping something, or its research is invisible, or its founders have moved on, then the structure was a founding letter and nothing more. We would like to be wrong. A lab that actually does what this one says it will do would be worth having, and the only way to find out is to watch.