AI assistants may shift from prompts to continuous sensory context

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 56:56

   and it dovetails with another story on the docket mira marati released thinking machines new real time that was impressive and this is super impressive and apple at the same time has patented putting cameras into your airpods so you're right and then i use this plot pin i've talked about it before these persistent hardware devices the watch the earpieces are going to know your entire reality they're going to monitor your entire desktop and this is going to lead to a use of tokens that would be a thousand times what user business users are currently using because the way mirrors we should have her on the pod or one of our events the way her model works friedberg is it's watching your desktop it's listening to all the voices and then it's watching your webcam all at the same time and every 200 milliseconds it's uploading it to two

2. 57:47

   different models one is deep thinking and looking backwards you know at a maybe a 30 second clip and the other ones in real time. That will 1000x the need for tokens if people are doing this for eight hours a day, because it's not the turn-based, here's my latest prompt, here's my question, I got a response. It's in real time, always querying an LLM, which is going to require a hardware upgrade to the average desktop. Mark Benioff, what are your thoughts on this brave new world. It's one small step for man, one giant leap towards AGI. I think that you said it really well. I mean, we've been so kind of think the LLM is the be all end all. We're going to go to AGI.

3. 58:32

   I don't really understand how large language models, which are only about language and words, and we know how it works. It's one word, one word, therefore the next word is this, is going to get us to where we want to go to, which is minority report or all the science fiction movies that we've seen. But what we saw, and I think, Jason, you'd really articulated that super well, by the way, multisensory models. And multisensory models, well, here's a good one. Oh, yeah, me, I'm a multisensory model at a biological computer.

4. 59:01

   I'm a multisensory model. I've got these eyes, ears, mouth, you know, work sometimes, some brain, heart, whatever, and some other things going on, too, that I don't even understand. And it's all running in this biological computer i'm not exactly a large language model though i do have one sometimes so i would say that multi-sensory models are the next big wave for ai and then but we're still not at agi at that point but those demos like the demo of that that was pretty everyone should see like that was pretty awesome and then i think we can kind of say hmm where are we now on this on the path but i think every model company is going to go hold on pivot you know again pivot like saying pivot everybody's pivoting and you're going to have to pick your poison and where where are you going next you know and you can't do everything you can't you said you were

5. 59:55

   spending 300 million on tokens with anthropic now imagine and what this would do to an average, you know, employee if they needed a thousand times the tokens they have now. So instead of spending 150K in tokens, you got to spend, you know, a hundred million, but you have some. I don't think that's true. I don't think that that, I think we're getting brainwashed on that. Okay. So I think that's a mistake to think that way.

6. 1:00:23

   See, I think we are wasting a lot of those tokens. So here's my, I think I convinced you this. So here we're using $300 million of Anthropic this year, and we're coding, we're coding, we're coding, right? The vast majority of those tokens don't need to go to Anthropic. There needs to be some intermediary layer that's saying, oh, that one has to go to Anthropic, but these ones can handle by a smaller model. You need a my show that can route it to the most affordable for the job. No, we're like, it's such an early moment. It's an early moment.

7. 1:00:53

   And I want to connect back to what you said before, which was brilliant, Jason, which was the importance of the edge. Because the edge and the cloud are going to come together, okay? And yes, you're going to have more distributed intelligence. It's going to work together. That's going to make a ton of sense what you said. What Chama said was also completely right. You were both right. Then you combine it to this idea. Let's put it together where we're going. Yes, we're going to have more distributed intelligence. We're going to have intelligence on the edge.

8. 1:01:22

   We're going to have multi-sensory models. And yes, you're going to do coding. And it's going to be more efficient though so to think that it's going to be always so expensive that is just the moment of time that we're at right now and i think there's going to be a hot new company that's going to come along and say this i'm going to sit between you and anthropic and open ai and i'm going to make sure that they oh you only need their tokens when you actually need them and that is not really where we are right now like yeah does that make sense what i'm saying like totally you're like it's like oh i need a shower and okay great all the water no no no i just need