Self-improving AI still faces messy real-world systems

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 2:21:07

   When you think about the future of AI and you think of this, so what do you think about AGI in particular? You think about something that could potentially make better versions of itself. Self-replicating. Yeah. And then how far does it go? Yeah. So that is the ultimate form of... I think some people in Silicon Valley have started calling that as ASI. So when you see the word ASI being thrown around, people kind of think of ASI as an AGI that can recursively self-improve itself. So that's going to be no limits to how smart it can get.

2. 2:21:48

   And I used to think that ASI is bottlenecked by power because you need a ton of compute for this model to keep on training itself and running its own rollouts and collecting data and then going and updating itself. But you could imagine that once the algorithm is correct, the ASI could be tasked with just making itself more efficient too, where improvement doesn't just mean capability improvement.

3. 2:22:16

   Improvement could also mean power efficiency. and that way the recursive safe ASI that is improving itself also makes itself more compact and more efficient and it can run on less compute. So that would be the ultimate project in AI. Think of it as almost as the last project in AI is basically cracking recursive self-improvement. Once you crack that, you don't have anything else to work on. In practice, I think what's going to happen is because information is so muddled and fragmented and living in disjointed systems, just the way we have constructed our messy real world, it's going to be hard to point even a recursively self-improving AI at some metric and say, go improve this.

4. 2:22:57

   Or go reduce inflation by 5%. That'll be awesome if you can task an AI to do that. If that's the job of the government to just reduce inflation, have a deflationary effect on society and make goods and services a lot more abundant and efficient, it's going to have to deal with a lot of messy legacy systems if the task is to go improve the health care for good luck like who's going to deal with all the compliance of actually implementing these changes inside hospitals most hospitals are still using legacy software because that's the the software provider has lobbied the government in a way where only they're allowed to do that god what a stupid bottleneck exactly so a lot of the bottlenecks in in in actually having ai's just take over and massively improve the human society and our hospitals, our legal systems, our government

5. 2:23:49

   systems, where most of the payroll is going into, is just bottlenecked by a lot of compliance and regulation. And so that's why I feel the human beings are still necessary to effect the change because these laws and regulations were built for us. And it also seems like we have to demand that those systems be usurped. sure 100 and we need the help of AIs to rewrite all these laws it's going to be humanly impossible to go and change one specific line here and there right and then you're going to have a bunch of these software companies that are lobbying to try to stop that from happening yeah it's that's why like this messiness and this need for getting all people on the same page and actually steering the society in a positive way our jobs will probably be more steered towards that problem solving at a different level of abstraction, maybe more need for EQ, more need for actually

6. 2:24:45

   like understanding differences of opinion and still like a leadership quality, ability to understand people, and ability to convince people, these are the skills that will be even more important in the world where like actual work can be done by AIs. But effecting the change in our society, in our country, still needs human beings because the systems are messy.