AI-written code can outperform when agents run evals

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 18:08

   And so is there something that you can learn from how war has been fought or how war is being fought? And do you do that to actually inform how the technology should behave? I don't know if this question makes sense. I think to the extent that you can be better together, you can collaborate, like Arvind runs a program. We're now saying we're all finding vulnerabilities. Why don't we pull out vulnerabilities and find the defense once, as opposed to every company in the room trying to go test all the open source vulnerabilities they have, figure out how to patch them, go patch them individually.

2. 18:41

   So then you take some trusted parties, give us your vulnerabilities, we'll patch it, we'll distribute the patches so you can all be safe at the same time. It's kind of like how cybersecurity works today. If you find an attack technique in a different part of the world, we can take that attack technique, write machine learning code, ingest that into our product and make sure it gets available to every customer.

3. 19:00

   Is agentically written code error-free? No. No. Okay, explain that. So, look, but first on your war question. So what Nikesh talked about is you can have powerful alliances where the alliance effectively can become more powerful than the attacker. That's technique one. Technique two in warfare, what has been the most effective is asymmetry. And so often an attacker is cheaper than the defender, so the defender runs out of money and at some point then concedes.

4. 19:26

   Or can you make defense equally cheap, in which case you can basically have symmetric warfare, and then you have a much better chance of success. And the third, which has been the way all wars have been won, is when you get technological superiority on one side. So go back to World War II, go back to Korea, go back to Iraq, go back to anyone. If you get technological superiority on one side, you tend to win.

5. 19:49

   And so our goal is, can we get technological superiority? And that's what Nikos is referring to. Can we find vulnerabilities? Can we patch them? Can we put boundaries around them? That makes It's really expensive for the other side to now try to find a way through. So that's what I would answer there. Look, at the end, will agentic code be superior to human-written code? Absolutely. We can find that.

6. 20:12

   We use agents and coding agents now for probably half the code that is written inside our place, and it is superior to what was written before. But the reason it's superior is the agents can also do evals and basically make sure that everything they can find is actually fixed before it ever goes out. All that said, we're only finding all the flaws that we know how to find. And so you get into this weird loop in computer science.

7. 20:39

   It's not really properly verifiable. It's not fully verifiable because that is a task that is going to be almost computationally infeasible right now. Now, I'm not going to hold out hope that it doesn't become what you were talking about in number 30 and all that. This is a task in which maybe we'll find a way to actually get to be able to verify the correctness of these things, at least in where the inputs are limited.

8. 21:06

   The input can be anything, then I think it's really hard. Does that make sense? Yeah.