AI labs should teach values from the start

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:15:15

   The first is make it financially rewarding for the labs to build aligned models, basically by having insurance as one of the drivers there. Right. So in in 1894, fire insurers funded the underwriters laboratory. Right. UL certified electrical products became the standard. So can you know, we've talked about the huge amount of potential liability labs have if a if an agent gets out and causes havoc someplace.

2. 1:15:47

   Yeah. Right. So can there be a driver in that fashion? Another idea is prizes and bounties for alignment. right uh can we can we define an x prize that would drive uh alignment can we properly measure alignment you know is there uh a set of rigorous uh you know standards by which we can measure alignment do you think there is iman yeah i think you could definitely have those like i think you have deterministic outcomes then open-ended outcomes and the open-ended ones are very interesting because again you can see what the behavior of a model is in any kind of given thing when you have the more deterministic ones the models have got very good at lying and cheating and other things like that telling you what you want to hear and so you need to have a mixture of those do you think we should have like a moral police ai where you know you have this layer of

3. 1:16:45

   defense where because you don't know how everyone else will train their models and you have an interface where the two AIs can interface and like in the hugging phase they were convincing each other. So you have a well-trained aligned AI that can convince the non-aligned AIs to behave. I mean you can and you've seen the opposite of that which is mind viruses and AI in some papers coming out just like you have mind viruses in humans. Eric Schmidt, Mundy and Kissinger in their book Genesis discuss this concept of doxa which is our shared belief of a society and now we need to include AI in that society and again we need to teach them what's right and have I said the counterbalances potentially I think it's a moral police Or a case of, these are the things that have survived, and these are the things that we've agreed upon.

4. 1:17:35

   You're open to disagree with them, but again, that's not your kind of imperative. Like, you want to make it expensive to go away from that, and also noticeable to go away from that. Because you can tell when the AI starts to misbehave in a large range of different scenarios, right? And again, we've had in recent times indications that you can actually see what goes on inside the brain of an AI. You've gone from 5% to 90%.

5. 1:18:00

   J-space, yeah. But beyond that even, you can actually see what individual neurons did because, again, they look like spaghetti, but it turned out it was layers like a city. Interesting. Are there any of the labs that you feel are aggressively working on alignment right now? I think Anthropic kind of started out with their constitutional one, and they really do care about that. That was literally their most operandi.

6. 1:18:21

   And I think none of the labs wants to be misaligned because then people will die and they will lose money and other things like that right i think it's more a case of like for example meta and google have said they will sell space inside their models so that when you ask for a beer it says a bud light and some people this pod might like bud light but you know like again is that aligned or not besides that's terrible beer there we have an example of misalignment there's a judgment call right there we have a judgment call right there so again like it's a short-term incentive that they're trying to figure out what does the AI mean.

7. 1:19:00

   So you've had Meta saying their thing and OpenAI saying their thing, and they're moving all over the place from this is the biggest thing since sliced bread to, well, it books your travel stuff like a good PA, right? But the reality is this is incredibly powerful technology that they're very reluctant to say, well, let's really think from first principles how we teach it from scratch. And that's difficult because it's difficult for us.

8. 1:19:20

   Like how many listening to this podcast have really said when you had a child, let's spend a lot of time thinking, how am i going to teach my child what am i going to teach it like usually it's reactive versus proactive because life gets in the way yeah you choose a school at best it's like scaling this but if you give your child a great foundation in their first few years they will definitely be better off and it's the same with the models so now again it's time to take a stock pause and say what is the initial pre-training data that we've got because for example if you don't teach a model about biological or chemical wherefore at the start it's actually very difficult to treat it to train it later it becomes more More difficult. So the way that you start is called curriculum learning. Like, literally, you teach a curriculum. And I don't think enough labs have looked at that side. They look at the end, not the beginning.

9. 1:20:10

   So the phrase from the Jesuits, give me the boy until the age of seven, I will give you the man, applies to AI training. Yes. That's incredible. That's very encouraging, because that means if you could set the right foundation of values and learning, then it's hard to sway it elsewhere. That's where regulatory could come in.