Local AI could give users more control over information

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:42:50

   One of the big fears that people in America have about technology in particular is that without being aware that this was going to take place, everybody gave up their data. Everybody gave up their data and didn't recognize it was a commodity. That in turn made these corporations immensely wealthy and powerful. And then they have the ability to shape narratives. And that concerns people because using their ideological position as leverage to try to push that through technology that has immense control and influence over people and that.

2. 1:43:31

   We didn't see technology and corporations as having that much control over how society views itself and how we interact with each other. And there's a real, real concern that these companies got so big and have, like, there's a guy named Robert Epstein who's done a lot of work on narrated or curated search engine results and how much that can, have you seen any of his stuff? I think I've seen this, yeah.

3. 1:43:58

   How much that can affect elections, how much that can affect people's perceptions on any societal issue that's coming up. And it's concerning. It really is because they do curate search results. It's not simply you just run it out there and you get this is the data. No, you get – if you look for specific political figures, depending upon where they fall in the right or left spectrum and depending upon which way the company forms – the corporation forms – falls rather, you'll get different results and that sucks.

4. 1:44:31

   You know, that's very concerning that people don't recognize, they don't have the ability to see how that is dangerous for all of society to have that kind of power and wield it in that way where you're not being honest about accurate objective information. You're pushing particular ideologies. Yeah. So I think it's kind of like this is almost an effect of the asymmetry that exists between the amount of AI power that centralized systems and centralized companies have and the amount of AI power as you as a sovereign individual has.

5. 1:45:10

   so when you don't have the AIs to just go judge for yourself like what you should be reading and fed you're obviously like under the influence of what you know whatever big tech company is controlling the information for but when you have access to all those AIs you can actually just customize what you want to see by telling the AI like hey this is what I think you should actually question and tell me until now you never had that power for yourself you're finally getting it right and eventually we'll be able to have our own llms like our own models that we would be able to post in our own hardware we don't have to rely on like one centralized model given to us by like any specific model company and using that you can shape it to your your beliefs your custom you know know your your your custom data and and so when you're consuming a search result you can actually

6. 1:46:07

   task that ai that you control and you run so nobody can shut off access to it to tell you like hey like can you actually like give me a contrarian perspective on this or like can you tell me if these search results are actually biased so i think we need to give individuals more sovereignty with more access to their own ais that they own and run on a piece of hardware they own themselves and this is the whole like this is going to be leading to the whole rise of local ais so as ai models like today they're very power inefficient they're running on large data centers but in a year or two from now whatever capability that exists in the most power hungry data centers will be you it'll be possible to run it in some box that you own may not really yeah it's already happening it's already happening that like there are like interesting hardware projects like the

7. 1:46:57

   apple mac mini NVIDIA dgx where you can actually host a reasonable size model and and put it in a box and have it run and you don't have to pay for all the tokens that it produces you you just have to plug it into your power car and it works i know duncan my friend duncan trussell he does that yeah and and today the capability of that model that can run locally is not quite there so you would still prefer to use something that runs in the data center. But eventually this is going to be a spec.

8. 1:47:31

   There's going to be some percentage of tasks that you would start delegating to this local system. It'll be a hybrid model. And over time, it could end up being the case that you could buy something that feels like a refrigerator for your home, which is your own AI box, and host a model that you control. So nobody can arbitrarily shut off access to it one day. And then you can have that be your weapon against what the big tech wants you to be fed or believe in.

9. 1:47:59

   so this is the only way you can fight this because they have far more computing power far more data far more algorithms than you so the only way you can fight that is you have something you own yourself and with the rise of open source models open source lms you can just and and and and progress in local hardware and and both apple and video until they're all doing amazing work here you could potentially change the future and give people more power and this may not be as expensive as people think well that's a good solution because i've always wondered like is are these searches like using google is that going to be irrelevant one day because you already can just ask your phone like i most of the time if i want to have an answer for something i just ask perplexity i was like what is it and instead of like having to sift through

10. 1:48:50

    all these google searches yeah and try to figure out what it's showing me first and get to page three where it's what I really want to know. I can get the accurate information, then follow up questions. Oh, it's instantaneous. Yeah. And even the models that are running the Perplexity app today, they're all in the cloud. Eventually you'll be able to do that on a box that you own. You can still use the front end the UI of the app, but you can control the compute that runs on a piece of hardware. You may ask why? Why do I care?

11. 1:49:23

    Okay, like what if someday like the data center gets taken off like iran was bombing data centers right like what if someday like the government decides that model is no longer available you you want some control over like like what models you can run and like you can you may even want to shape it to like your context that you never want to be living on any data center and and and and uh i think that's where i believe the individual gets more sovereignty against big tech and um that's how like we fight uh surveillance or like centralization of power yeah and certainly pushing narratives