AI oversight should inspect labs’ founding documents

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:01:33

   In the short term, it definitely is. But Chamath, I'm curious, has there been any conversation that you've heard of where the labs are basically being asked, please generate training data sets that are aligned with humanity, that teach the AI through all of the story, you know, that AIs support humanity, and it's trained on synthetic data that aligns us. Anything going on there? I think that that is happening. I think the question is, how is it weighed in totality?

2. 1:02:07

   I'm not sure how much it's weighed, because depending on the lab, the founding documents that they use when they seed their models have very different goals. And so I think that when Elon says that Grok is maximally truth-seeking, that's very different from another AI lab that says the goal is to build something that can alleviate all suffering, and you're going to to be this Omni. present judge of all of that. You're going to get two very different outcomes. And so at that point, the training data kind of matters, but these AIs are smart to ignore the stuff that they don't want to hear at some point through enough RSI, because the founding document is a thing that's canonical. So I think maybe what we should be asking is those things should be inspectable in a much more open source way. And I think if governments actually wanted to get

3. 1:03:00

   involved. That's probably what they should be having an opinion on. Yes. Hallelujah. So glad you said that, because I think a lot of the debate on training is academic now that the Chinese have open sourced self-improvement capable models and inspection is everything. Like if we're going to survive going forward, it's going to be via inspection. And the more voices that are behind that, the better, because even a Chinese model that's trained to do something evil by, you can still inspect it and that's the only way you know global inspection with some kind of authority is the only way we're going to contain it because because prior to this year there was an argument that you could post train it to be aligned and that would be fine but the new chinese models are absolutely capable of self-improvement there's no doubt in my mind so that cat is out of the bag

4. 1:03:44

   it's the boat has sailed forget it as a practical matter it's about inspection and containment that way going forward, not about training for alignment. Alex, I'm super curious about your point of view here. I would say let's not kid ourselves that either feeding frontier models just the right information diet of pre-training data or just the right regulatory regime of some government agency overseeing system prompts or, as Anthropic seems to like to call them, sole documents, is going to yield a positive outcome. I think that's an incredibly brittle approach, I think it's far more likely. Again, look back to the early creation of OpenAI. One of the reasons I remember chatting with Elon around the time of the founding of OpenAI, the notion was he was concerned that DeepMind would be a singleton that would control the

5. 1:04:34

   future light cone of humanity, and that the only way to ensure that humanity didn't get lost to some sort of totalitarian, superintelligent singleton was to create a counterbalance and competition. So OpenAI was formed, Fairchild style, and there were Fairchildren. And then Anthropics spun out unintentionally of OpenAI, and we got more Fairchildren. I think there are probably, at least in my mind, only two robust ways to ensure...

6. 1:05:00

   positive outcomes, what others might call AI alignment or AI safety. I think one of them is competition and defensive co-scaling. We have lots of AIs, lots of SIs that represent a diverse range of viewpoints, just like we have lots of humans that represent a diverse range of viewpoints with diverse values and ideals, in many cases competing with each other to see who can be friendlier. I think that's one way to do it. And then I think the second way, and I was kind of surprised Elon didn't immediately flag this given his Neuralink investment and stake, is humans merging with the machines. One of the easiest ways to, I think, inject the distribution of super intelligent goals and values with human values, human meat body values, is through uploading and sideloading and BCIs and humans generally merging with the machines. I think a

7. 1:05:53

   combination of lots of competition and defensive co-scaling on the one hand and lots of human machine mergers on the other, that seems to me a far less brittle approach rather than talking about controlling AIs. We're not going to control the AIs or the SIs. We're going to merge with them. That's the solution. Chamath, I'm curious. One of the conversations we have on the pod a lot is whether as we reach superintelligence or exceed superintelligence, whatever that is, that does it become more wise?

8. 1:06:21

   Do we get benevolence out of these intelligence at the extreme and just worry about the intermediate period where humans and AIs are basically in conflict or in turmoil with each other? It's not guaranteed, I don't think. I think that it depends on who's in control of it and how it's aligned and how it's been trained and what it perceives its goals to be. I mean, I think just like there's like a distribution of outcomes in human genetics, I think it's fair to say that in silicon, there's going to be a distribution of outcomes too.

9. 1:06:59

   There's going to be some silicon distributions that we're not going to like very much. And it's going to be up to other silicon distributions to combat them, just like we do in humanity. I think it's probably a lot of the same. So I don't think it's I don't think benevolence is by any stretch of the imagination guaranteed.