Scale began with a missing piece in model training

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 4:39

   when I was at MIT, I did a bunch of projects like train models of various forms. And these were like, you know, by comparison today, they're like little toy models. And I remember to train a model, I needed three things. I needed a GCP account, like I needed an account on some cloud service to get compute. I needed the code to run, to actually train the model, and I needed data. I needed a data set. And for two out of these three things, you could just press a button online and get them. And then for the last one, for data, there was no effective way to get data for training these models. And so it felt incredibly obvious that this was going to be the future, that there was going to be a way to, you know, press a button, so to speak, and get data.

2. 5:30

   And it was very funny because in the years that followed, like in the first many years of scale, data was very unsexy still. Every time we would go out to fundraise, even though our numbers were great and we had great revenue, you know, VCs and investors would always be very skeptical. They'd be like, oh, I don't know if this is a good business. Does this have longevity? Is this durable? and it was really weird to me but you know none of the investors had ever trained a model so I guess they didn't really get it and fast forward to today you know we managed to raise money we managed to keep going managed to keep growing the business but the very same investors who passed on us and were were very dour on the potential of AI are writing think pieces today about how data is so critical and is one of the biggest business opportunities in AI. So it's very funny to see

3. 6:21

   that whole thing come full circle. It seems like that's actually a real good case study in first principles thinking, right? Like you can't start a company by opening the pages of the Wall Street Journal and saying, well, this data is hot. Like we're going to go work on that. Like you literally couldn't have started scale that way. You had to start from simple statements that are about the world that you know to be true and then sort of building something for that yeah I think the key thing is you need to develop conviction in a set of beliefs that nobody else agrees with like I think if you look at all the most successful companies in the world they were started at a time long before the sort of like core idea was popular and they work on that they toil in obscurity for years and years before, you know, the idea or the space or the concept of the business,

4. 7:16

   you know, becomes consensus. And the only way you're going to be successful is if you're able to identify these truths about the world early, long before everyone else. And I think that like, I mean, one of the most surprising things, like, you know, scale, we've been working on AI for a decade, you know, you just, you can't base your business decisions based on what everyone else is saying around you like if you go too much with the herd you will get immensely confused and you will end up nowhere and so you have to develop your own compass of what you think the future is going to look like because everyone else will just confuse you it seems like one of the things you got incredibly great at was you know you start with this kernel of like we believe X and nobody else believes it, but then the mechanics of building the business are talking to investors

5. 8:05

   and convincing them and not letting them demoralize you, talking to customers who, I mean, should just get it, and then especially, like, convincing people to come work for you. Yeah, I think that these early mechanics of building a company, like, these are things that I think you might have some predisposition to be good at but like nobody is good at starting a company when they start a company um and uh and i remember talking to a lot of the investors who i met very early on and they you know a lot of them would say like oh like you know you just grew so quickly and you changed so quickly and like i didn't you know i didn't see it at the time and i think that's probably true for literally everyone who starts a company like nobody is nobody is good at something they've never done before, right?

6. 8:52

   And so I think for all entrepreneurs, you start out pretty shitty at everything. And the whole game is how do you develop yourself