Replit automated the copy-paste loop between ChatGPT and code

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 2:06

   enlightening is watching some of our users code and so some some users were doing 100 percent of their code using AI. But the way they did it... No, you're talking about once you released Vibe coding. No, no, before that, before that, I'll tell you. How were they finished? Okay. So they would open ChatGPT in one window, and they would open Replit in another window. This is someone who's never coded before.

2. 2:34

   How could you tell that they were pasting code out of ChatGPT? I invited some users to the office. Oh, you would literally watch them? We would watch them. Okay, how do you do this? And basically, they would copy the code, put it into Replit. Replit would produce some error. They would copy the error, put it into ChatGPT, and then copy the fix. And it was such a manual process. And I looked at it.

3. 2:57

   All we have to do is make an API. Well, we have to automate the entire process. It wasn't obvious. It wasn't easy because back then, AI models would do a couple of tool calls. Oh, you know what's different? The difference is it's not just you have to automate the back and forth. You have to understand the errors, right? Exactly. Yeah, that's hard. Yeah, and you have to loop. And the terminal condition should be that the program is working as opposed to- Yeah, that's still true.

4. 3:26

   Right. As opposed to kind of like, we'll do one turn, right? Because that's how ChatGPT sort of worked. That's how most AI tools worked. And so can you loop indefinitely? Go to completion. Yes. Right. Which is, that's the new thing. That's what Claude does. Yes. Yeah. So that's what we demonstrated. That's what Replit first demonstrated is, so we demonstrated a couple of things. One is that you can run LLMs to completion.

5. 3:55

   Which I bet was news to the model companies. They did. They actually reached out to us and they were like, we're so, I'm surprised that ReplenAsia works. How did you make it work? And the crazy thing, Paul, is that the harness circa 24 is 10 times more complex than the harness today because the models were less capable. Yeah. So as models get more capable, like what the kind of the engineering you have to do to make them become agents is less and less so.

6. 4:23

   Does that mean it's easier for people to compete with you? Yes. But we have to do, well, that's the thing about the AI business today. is that you have to just accept that everything that you're doing is like getting commoditized constantly and you have to like- You're doing the next thing. You're doing the next thing, right. But the other big thing that we demonstrated is that AI can provision infrastructure.

7. 4:46

   So up until that point- And you actually have infrastructure to provision. Yes. That's the thing that's different between you and all these competitors, right? Exactly. So we had built like databases- You'd spent all these years suffering doing system software. Right, right. Right. So it was a little like tool call, spin up a database, you know, spin up a Redis, like host this, you know. And so what people found so surprising when we launched, I mean, you saw the product July 2024.

8. 5:13

   And when did you launch it? September 24. Okay. And so when people saw it, like Andrej Karpathy said, this was like an AGI moment, like a field AGI moment, because everyone was like, holy crap, like LLMs could do that. It freaked me out. Yeah.