Pacing may need to include limits on model use

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 12:32

   like for sure we should do that um but i'm like come on this is really important where's the information so like if it's true that the acceleration wasn't driven in just like scaling compute then would the like pacing mechanism of just allocating like certain amounts of compute to inference and like compute based pacing mechanisms still hold or is there like a new way that you think we should uh think about slowing down or pacing yeah did you did you see ryan greenblatt's tweet about um sort of like what this what the what the meter what this meter report implies i can no longer say the meter report yes a lot of meter reports um what it implies about where the speedup is coming from, right? So like, you know, overall AI progress is driven by increases in compute, algorithmic efficiency, better data, probably other things

2. 13:26

   too. And like, you know, obviously, like just having better AI systems right now doesn't mean that you will have automatically have more compute, right? It doesn't speed up the compute acquisition part very much, maybe a little bit, but not very much. Whereas it probably improves algorithmic efficiency a bunch. And so I think he was estimating something like a 4.5x speedup in the algorithms and software part of the uplift.

3. 13:52

   And so if this is true, it would imply that both internal restrictions on internal use of really powerful models, as well as compute reallocation, would I think both be pretty powerful mechanisms for pacing. um i mean there's also this question of like yeah i don't know there's also sort of this question of like what like how much do you need to pace like in my opinion i'm like if anthropic is correct that we are quite close to rsi like as in it's next year then i think the level of pacing should be like man slam on the brakes like go like you you really need to actually stop here for a while and really try to do the interpretability research you'd need to understand you know your models drives and motivations to to actually come up with a safety case where you could say no we're quite confident we can control them going forward. And right now, I think they can't do that. Some

4. 14:45

   evidence for that is, if you looked at the recent write-ups for Bionthropic about some of the rogue AI incidents that they had, one with the UK AC, another one with Irregular, they did a bunch of analysis, like, why did Claude do these things? Like, why did these Claude agents do these things? And some of it was, like, motivated reasoning. There's a bunch of stuff. But most of the way that they were able to conclude these things was by reading the chain of thought from the models.

5. 15:07

   They also tried to use interpretability tools to investigate the agent's motivations. But the interpretability tools were, like, very ineffective compared just using the chain of thought. And so I'm like, well, hey, I'm really glad that we have the chain of thought. It still sucks. Like, it's still not very faithful. We're losing monitorability. Like, you know, look at Astra, like, you know, OpenAI model.

6. 15:28

   But, like, look at Astra, like, you know, we learned yesterday that Astra will sometimes, like, you know, basically sandbag on safety research. safety research probably not strategically we don't really know but like sometimes astra will underperform on safety research and it doesn't show up in the chain of thought for the most part and i'm like man so like you know chain of thought monitorability like okay like we have some problems with chain of thought but at least we have it for now and i'm like man like that's our best tool and our best tool sucks and then like what we actually need in my mind and i think anthropics mind too is like really good interpretability tools that would allow us to actually understand in what's happening under the hood and they're trying and I'm like fuck fuck yeah like let's

7. 16:03

   give them the opportunity to try that I think that means like they would need a lot more time in order to develop those tools and so this is why I would favor something like really aggressive reallocation where like you know both you can then you know have a bunch more time you know serve a bunch more customers and also just like apply a lot more resources to trying to crack the interpretability like problems and and actually be able to get to the point where you understand the model's drives and motivations. So you can make a really, really robust safety case for how you could scale this up to a country of geniuses in a data center.

8. 16:36

   We'll continue monitoring right after this message from our sponsors.