China is competing through broad AI adoption and open models

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 3:45

   unpaid part-time advisor to the government. But we are in this high-tech competition with China. They obviously care about AI too. They're trying to race ahead. How do you, I understand that you recently made a How do you handicap this competition? Well, you and I just talked about this as part of your incredibly important work in the White House. I had thought that China and the United States were competing at the peer level in AI and that the good work that you have done and your predecessors did to restrict chips were slowing them down.

2. 4:21

   They're really doing something more different than I thought. They're not pursuing crazy AGI strategies, partly because of the hardware limitations that you've put in place, but partly because the depth of their capital markets don't exist. They can't raise, based on a wing and a prey, or $100 million or a memory equivalent to build the data centers. They just can't do it. And so the result is they're very focused on taking AI and applying it to everything.

3. 4:50

   And so the concern I have is that while we're pursuing AGI, which is incredibly interesting and we should talk about, and all of us will be affected by this, we better also be competing with the Chinese in day-to-day stuff. Consumer apps. This is something you understand very well, Chamath. Consumer apps, robots, and so forth and so on. I saw all the Shanghai robotics companies, and these guys are attempting to do in robots what they've successfully done with electric vehicles, right? And their work ethic's incredible. They're well-funded. It's not the crazy valuations that we have in America. They can't raise the capital, but they can win across that. The other thing the Chinese are doing, and I want to emphasize this as a major geopolitical issue, is that my own background is open source. In the audience,

4. 5:37

   you all know open source means open code, open weights means open training data. China is competing with open weights and open training data, and the U.S. is largely and majority focused on closed weights, closed data. That means that the majority of the world, think of it as the Belt and Road Initiative, are going to use Chinese models and not American models. Now, I happen to think the West and democracies are correct, and I'd much rather have the proliferation of large language models than that learning be done based on Western values.

5. 6:08

   Eric, we had a major open source initiative with Meta, you know, incredible balance sheet, tremendous technical firepower, but they seem to have mis-executed and now are taking a step back and reformulating something, to your point, that looks a little bit more closed source. It's not clear. Alex Wang's a good friend. He's come in. He's taking over. He's obviously incredibly capable. I would not say that they're going fully closed. And I think also they got screwed up because the DeepSeq people, R1, did such a good job, right? If you look at the reasoning model in DeepSeq and, in particular, their ability to do reinforcement learning forward and back, forward and back, and forward and back, this is a major achievement and it appears that they're doing it with less precision than, numeric precision than the

6. 6:59

   American models. As a bit of technical things, there's something called FP64, FP32, FP16. The American models are typically using 16-bit precision for their training. The Chinese are pushing eight and now even four. Is there something that the American, you know, bigger companies need to be doing in open source so that we can actually combat this? Well, a number of the large companies have said that they want to be leaders in open source as well.

7. 7:26

   Sam Altman indicated that the smallest version of the O3 model would be released, I believe, open weights, and they have done so. And he told me, anyway, that this model is much smaller than 10 to the 26. It's much easier to train, and it will fit or can fit on your phone. So one path is to say that we'll have these supercomputers doing AGI, which will always be incredibly expensive and so forth. But we also have to watch to make sure that the proliferation of these models for handheld devices is under America.

8. 8:00

   can control, whether it's OpenAI or Meta or Gemini or what have you.