Real Alpha
Watch modeSubscribed

The Diary Of A CEO · 3 min · Jul 13, 2026

OpenAI's expected safety pause gave way to continued racing

At OpenAI, colleagues expected a pause once AI could automate research, but Daniel Kokotajlo says that expectation faded. He left after deciding safety was not driving the company and he lacked room to publish.

Daniel Kokotajlo

Daniel Kokotajlo is a former OpenAI researcher and founder of the AI Futures Project.

Lead author of AI 2027. He left OpenAI's governance team in 2024 and now runs the AI Futures Project, a forecasting nonprofit.

About this segment

At OpenAI, colleagues expected a pause once AI could automate research, but Daniel Kokotajlo says that expectation faded. He left after deciding safety was not driving the company and he lacked room to publish.

From “ChatGPT Whistleblower: They Offered Me $2m To Keep Quiet, No One Is Ready For What's Coming!”, Jul 13, 2026. Watch the full episode

“once we started getting to AIs that could maybe automate the AI research process, we would pause and figure out how to make it safe.”
Daniel Kokotajloat 13:39 · The Diary Of A CEO13:39

Takeaways

  1. Colleagues expected a safety pause when AI could automate research.
  2. By the time he left, he thought the company would keep racing.
  3. He also wanted more freedom to publish his research.

In this episode

Full episode
  1. 19:25AI labs plan to automate their own research first7:12
  2. 1:14:22Plan A would slow AI and make frontier research transparent7:41

  3. 13:18OpenAI's expected safety pause gave way to continued racing2:40

  4. 56:03AI could erase the usual path from one job to another3:04Up next
  5. 1:34:21Job loss could also strip people of political power4:04
  6. 1:46:11He would not permanently shut down AI, though he is torn4:05

Transcript

From the transcript The Diary Of A CEO publishes with the episode.

  1. 13:24

    And why are you no longer at OpenAI? Largely for the reason that I mentioned. So I became gradually disillusioned with how the company was going to behave. For example, when I first joined in 2022, at least the people I talked to, my colleagues at the company, there was this general sense of like, of course, we wouldn't actually just build superintelligence as soon as possible. Once we started getting really close, like once we started getting to AIs that could maybe automate the AI research process, we would pause and figure out how to make it safe.

  2. 13:55

    that's because we're the good guys and that's obviously the safe thing you should do rather than just going full speed ahead. But we're worried about other people who might not pause, you know, our competitors, Google, for example. And so that's why we need to be in the lead so that we have that room to do the safe stuff, right? That was sort of like a thing that seemed like... maybe like the median position or something among the colleagues I talked to when I was there when I started, including people like Sam, you know, including the leadership. And then by the time I left, I was like, oh man, they're really not going to do that, are they? Like they've sort of, you know, partly because this has become more politicized and they've become bigger and been under more scrutiny. People have started asking, like, why are you doing this in the first place if it's so risky? And so they've pivoted their narrative to being more like, actually, it's not that risky, you know? And so, yeah, I mean, it seems like they're just going to keep going roughly as fast as they can and hope that they can figure it out on the way. How did your time at OpenAI come to an end? I resigned in 2024. I had a nice goodbye party. What were the reasons you gave for quitting OpenAI?

  3. 14:56

    I thought that we were rationalizing too much and that we needed to think more about what would actually be good for the world. I wanted more freedom to publish. So AdOpeneye, as it became a bigger company, it became more of a normal tech company with incentives and, you know, a PR department and things like that. And so it started becoming more difficult to publish the sort of research that I was doing. For example, those scenarios that I mentioned, I couldn't publish those, right? They're just for internal use.

  4. 15:26

    I thought that that was a shame because right now most of the world is kind of asleep at the wheel and doesn't really realize what's going on with AI and doesn't really realize what's coming in the pipeline a couple years from now. And the companies aren't really incentivized to tell people that much about it. I mean, they say some vague stuff in a sort of hypey way, but, you know, well, they didn't want me to publish the scenario, for example, laying out like here's how things might actually look.