Cerebras made architectural choices to adapt as AI changed

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 29:07

   Well, actually, I think that's really, you know, for entrepreneurs who listen to this, like, it was started as a training system. Like, part of the vision that you pitched and part of what you explained was, like, training is a harder problem because of back prop. top inference we thought was going to be on devices and computers and really distributed, which may still end up happening. And like fast forward to 2022, 2023, maybe 24, we pivoted the company, or I don't know if pivot's the right word, but like to inference, how and why and where did we get lucky and where did you have real foresight on that?

2. 29:45

   I think there's sort of, there are two ways to make decisions and to think about it. One, I think about like an old telephone circuit, right? They set up a dedicated view all the way to your brother in New York. And that's one way to do vision, right? That's one way, that you've got this idea of the way the future is going to look way out there. And that's a circuit, we think. The other way to do it is the way routers work, where the internet works, where you go hop to hop.

3. 30:26

   You go to Cleveland, and then there's a decision whether it's best to take the next hop. And that's sort of the way we worked, where we knew there was a pot of gold out there. But the path to it, we knew, was unknowable. And so each time you get over a new mountain, you look around and you re-decide. So it's this sort of hop-to-hop thinking. So can we build it? Yes. Can we make it work in routing? Yes. Is routing now where everybody's focusing?

4. 31:00

   That's where they're focusing. Can we be a fast router? Yes. But now we're seeing the lay of the land, we're seeing the unfolding of AI, the rapid growth of intelligence. Well, who's going to use it? We use AI through inference. So we're in a position, we're engaged in conversations, we had product being used where it allowed us to see something new. And that new thing was, holy cow, the trajectory of AI will make it smart enough that everybody will want to use it. If everybody wants to use it, inference is going to crush the compute infrastructure. We better be there and so each time we achieved something it sort of moved us up a mountain or a hill gave us a new view of the landscape we could make some new decisions and we were always sort of moving in the same direction but we didn't know how to how to get there and so decision hop you get to a point of

5. 31:59

   view Think carefully. Earn the next viewpoint. Earn the next viewpoint. Well, it's interesting because with software, you obviously can both think nimbly and change everything the next day. Right. It's hard where you can't do it like that. That's right. Ours, we have bigger discrete decisions, right? and that's why the choice in your chip of specialization versus flexibility is so important. We made a couple really good decisions in our first architecture where we decided not to sort of embed technology that would accelerate convolutional networks. Instead we said we don't know how long those will last if we work underneath that and accelerate the the algebra that underpins all ai we knew about that was a really good decision because when transformers came out we were the fastest at those two even though we've never seen them and never heard of them and they hadn't been

6. 33:03

   invented when we set the architecture and so it helps to make a few good decisions yeah it's