AI interfaces can borrow from how people think

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 36:06

   that you've learned after building AI products or working at OpenAI? Something that's just like, I did not expect that. I don't know, maybe I should have expected this, but one of the things that's been funny for me is the extent to which you can kind of reason, when you're trying to figure out how some products should work with AI, you can often or even why some ai thing happens to be true you can often reason about it the way you would reason about another human and it kind of works yeah so maybe a couple examples when we were first launching our um our reasoning model right we were the first to to build a model that could reason that could that could instead of giving you just a quick you know system one answer right away at every question you asked it was the third emperor of the holy roman empire like

2. 36:57

   you know here's an answer you could ask it hard questions and it would reason the same way that if i asked you to do a crossword puzzle you couldn't just like snap fill in everything you would be well okay on this one across i think it could be one of these two but that means there's an a here so that one has to be this oh wait you know like backtrack kind of step by step build up from where you are. Same way you answer any difficult logistical problem, any scientific problem. So this reasoning breakthrough was big, but it was also the first time that a model needed to sit and think. And that's a weird paradigm for a consumer product. You don't normally have something where you might need to hang out for 25 seconds after you ask a question. And so we were trying to figure out, you know, what's the UI for this? Because it's also not like with deep

3. 37:44

   research where the model is going to go and think for 25 minutes sometimes it's actually not that hard because you're not going to sit and watch it for 25 minutes you're going to go do something else you're going to go to another tab or go get lunch or whatever and then you'll come back and it's done when it's like 20 25 seconds or 10 seconds it's a long experience it's a long time to wait but it's not long enough to go do something else and so you actually need and you know so you you can think like if you asked me something that i needed to think for 20 seconds to answer what would i do i wouldn't just like go mute and not say anything and kind of um you know shut down for 20 seconds and then come back so we shouldn't do that we shouldn't just like have a slider sitting there that's annoying but i also wouldn't just start like babbling every single thought that

4. 38:33

   i had um so we probably shouldn't just like expose the whole chain of thought as the model's thinking but you know i might go like huh that's a good question all right i might approach it like that and then think and you know you're sort of like maybe giving little updates and that's actually what we ended up shipping you have similar things where you can like you can find situations where um you get better thinking sometimes out of a group of models uh that all try and attack the same problem and then you have a model that's looking at all their outputs and integrating it and then giving you a single answer at the end, I mean, sounds a little bit like brainstorming, right? I certainly have better ideas when I get in a room and brainstorm with other people because they think differently than me. And so anyways, there's just like all these situations where

5. 39:17

   you can actually kind of reason about it like a group of humans or an individual human and it sort of works, which I don't know, maybe I shouldn't have been surprised, but I was. That is so interesting because when I see these models operate, I never even thought about you guys designing that experience like it to me just feels like this is what the llm does it just sits there and tells me what it's thinking and i love this point you're making of like we like let's make it feel like a human operating and how does human operate well they just talk out loud they think here's the thing i should explore and i love that deep sequined like to the extreme of that right where they're just like here's everything i'm doing and thinking and i and people actually like that too i guess was that was that surprising to you like oh maybe that could work too people

6. 40:00

   seem to like everything yeah we learned from that actually um because we um when we first launched it we kind of gave you like the the subheadings of what the model was thinking about but not much more and then deep seek launched and they were it was a lot and we kind of went you know i don't know if everyone wants like that there's some novelty effect to seeing what the model's really thinking about we felt that too when we were looking at it internally it's interesting to see the model's chain of thought but it's not i you know i think at the scale of like 400 million people you don't want to see the model kind of like babble a bunch of things um and so what we ended up doing was summarizing it in interesting ways so instead of just getting the subheadings you're kind of getting like one or two sentences about how it's thinking about it and you can learn

7. 40:44

   from that so we kind of tried to find a middle ground that that we thought was an experience