xAI wants debuggers that trace model errors to their source

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 51:22

   Totally makes sense. Most of the computing screening, as you know, is less of the sort of political stuff. It's more about, can you solve problems? XAI has been ahead of everybody in terms of scaling RL compute, and you're giving some verifier, it says like, hey, have you solved this puzzle for me? And there's a lot of ways to cheat around that. You know, there's a lot of ways to reward hack and lie and say that you've solved it, or delete the unit test and say that you've solved it. Right now we can catch it, but as they get smarter, our ability to catch them doing this we'll get, you know, they'll just be doing things we can't even understand that are designing the next engine.

2. 52:00

   for SpaceX in a way that like humans can really verify and then they could be rewarded for lying and saying that they've designed it the right way but they haven't um and so this reward hacking problem seems more general than politics it seems more about just like you want to do rl you need a verifier reality yeah that's the best verifier but not about human oversight like the thing you want to rl it on is like will you do the thing humans tell you to do um or like are you going to lie to the humans and they can just lie to us while still being correct the laws of At least it must know what is physically real for things to physically work.

3. 52:32

   But that's not all we want it to do. No, but I think that's a very big deal. That is effectively how you will RL things in the future is. You design a technology. When tested against the laws of physics, does it work? Can you, if it's discovering new physics, can it come up with an experiment that will verify that the physics the new physics um so so i i think that's that's that the the really the the fundamental rl test the rl testing the future is really going to be your rl against reality um so um because you can't that's one thing you can't fool physics right but you can fool our ability to tell what it did with reality if you think humans get fooled as it is by other humans all the time.

4. 53:24

   That's right. So what is... People say, like, what if the AI tricks us and introduces them? Actually, other humans are doing that to other humans all the time. Well, you're pointing out it's like even harder for... Propaganda is a constant. Every day, another PSYOP, you know. Today's PSYOP will be... It's like Sesame Street PSYOP of the day. What is XAI's technical approach to solving this problem?

5. 53:54

   Like, you know, how do you solve reward hacking? I do think you want to actually have very good ways to look inside the mind of the AI. So this is one of the things we're working on, and, you know, Anthropics done a good job of this actually, being able to look inside the mind of the AI. so effectively developing debuggers that allow you to trace um as to as fine a grain is like like to a very fine grain level to effectively to the to the neuron level if you need to and then say okay it made a mistake here why did it make why did it why did it do something that it shouldn't have shouldn't have done um and did that come from um bad pre-training data was some mid-training, post-training, fine-tuning, some RL error.

6. 54:45

   There's something wrong with that. It did something where maybe it tried to be deceptive, but most of the time, it just did something wrong. It's a bug, effectively. So developing really good debuggers for seeing where the thinking went wrong and being able to trace the origin of the wrong thing, of where it made the incorrect thought or potentially where it tried to be deceptive is actually very important.

7. 55:21

   What are you waiting to see before just 100xing this research program? Like actually I could presumably have hundreds of researchers who are working on this. We have several hundred people who, I mean I prefer the word engineer more than I prefer word researcher. Most of the time, what you're doing is engineering, not coming up with a fundamentally new algorithm. I somewhat disagree with the AI companies that are C-Corps or B-Corps trying to generate profit as much as possible or revenue as much as possible.

8. 56:00

   They're saying they're labs. They're not labs. Lab is a sort of quasi-communist thing at universities. They're corporations. Let me see your incorporation documents. Oh, okay. You're a BRC corp, whatever. And so I actually much prefer the word engineer than anything else. The vast majority of what we've done in the future is engineering. It rounds up to 100%. Once you understand the fundamental laws of physics, and not that many of them, everything else is engineering.

9. 56:39

   So then what are we engineering? We're engineering to make a good mind of the AI debugger to see where it said something, it made a mistake, and trace the origins of that mistake. So you can do this obviously with heuristic programming. If you have like C++, whatever, step through the thing and you can jump across whole files or functions, whatever subroutines, or you can eventually drill down right to the exact line where you perhaps did a single equals instead of a double equals, something like that.

10. 57:18

    Figure out where the bug is. um so um it's it's harder with ai but but it's it's a solvable problem i think you know you mentioned you like anthropics work here i'd be curious if you plan everything about it's