Real Alpha
Try Pro free

Full episode · 1 h 14 m

Databricks CEO on Super AGI and Unlocking AI’s Untapped Potential

Ali Ghodsi argues that AI’s near-term value will come from reliable, specialized systems applied to companies’ data, not a single supermodel.

The Logan Bartlett Show · Logan Bartlett

The best minutes

  1. 8:18Compound AI systems can improve specific tasks more efficiently7 min
  2. 38:52Databricks pivoted from PLG to enterprise sales and paid innovationPro pick 6 min
  3. 54:45Test executive candidates by having them do the job5 min

Also worth watching

  1. 19:55Databricks customers use AI more to read than to write3 min
  2. 30:00Databricks turned down $20 million to stay focused on cloud3 min
  3. 34:25Databricks moved engineering pay to the market's 90th percentile3 min
  4. 44:16New products need startup-like autonomy inside a large company4 min
  5. 1:08:14A struggling executive may thrive in a different role3 min

Skip the AI adoption and model-economics discussion, the customer AI strategy, and the later culture and work-style conversation.

Summary · 5 min read

Ali Ghodsi is the cofounder and CEO of Databricks. Before starting the company, he and colleagues worked on Spark at Berkeley and entered Netflix’s recommendation competition.

Ghodsi argues that companies will get near-term value from reliable AI systems built for specific, repetitive tasks and connected to data they uniquely possess. He expects adoption to take time, even with current models, because companies need to organize data, align decision-makers and change workplace processes. He recounts Databricks’ shift toward enterprise sales, a more experienced leadership team and proprietary products, and shares how he approaches product development and hiring. Looking ahead, he expects AI applications to need a data-and-AI platform underneath them, a role he hopes Databricks can fill.

Minute by minute

  1. 0:41AI moved from niche work to broad adoption

    Ghodsi says he and colleagues began AI work at Berkeley around 2009–10; Spark earned the highest score alongside another team in Netflix’s recommendation competition. He recalls enterprise uses in ad placement, Expedia hotel rankings and Regeneron’s disease-marker research, while GPT-2 showed how quickly language systems were advancing. ChatGPT’s 2022 launch made the impact mainstream, though it did not change his underlying view.

  2. 5:30AI apps need data and task-specific evaluation

    Ghodsi says that calling a general LLM gives companies no edge; their data or customer relationships can. He expects both startups and incumbents to build successful AI businesses, but says he would bet more on startups because large companies move slowly. He places AI near the top of the hype cycle and expects disappointment before specialized applications emerge; compound AI systems need evaluations tailored to particular tasks.

  3. 10:38Specialized models can beat general models

    Ghodsi says customers often need a system that performs one repetitive task well and cheaply, rather than a giant general model. Databricks sees more aggregate revenue from tiny models than large ones, he says, because customers distill larger models to the smallest version that meets their needs. He expects scaling gains to slow as compute, energy and high-quality data become harder to expand, and expects humans to remain in the loop.

  4. 17:04Current AI has untapped workplace potential

    Ghodsi says current models could have major economic effects even if development stopped, because companies have not yet applied them across workplaces and processes. He expects value from making costly, repetitive tasks reliable, with production use cases likely to take two or three years to appear. Examples include tailored financial advice, insurance risk assessment, underwriting, SEC filing analysis and extracting structure from medical records. He estimates models are used to read data at least 10 to 1 more than to produce it.

  5. 22:28Enterprise AI starts with data and alignment

    Ghodsi advises companies to organize, secure and consolidate their data before rushing into AI, then focus on use cases where they have a data advantage. He says one person should coordinate AI decisions to limit internal disputes over vendors, models and projects. For Databricks, adjacent opportunities include governance, guardrails, privacy and compliance; he expects startups to create many of the applications.

  6. 28:12Databricks began as a non-consensus bet

    Ghodsi says his team saw AI as a way to move beyond backward-looking analytics, but customers were focused on the Hadoop wave and showed little interest. The researchers started Databricks because they thought no one else would commercialize the work. In 2016, a bank offered $5 million, then $10 million, then $20 million for an on-premises version; Ghodsi declined, saying Databricks needed to pursue a different path.

  7. 32:33Bigger ambitions changed Databricks’ talent strategy

    Ghodsi credits Ben Horowitz with pushing Databricks to pursue a $14 million Series A and think much bigger than the founders initially did. He also says Marc Andreessen challenged the company to compete for engineers from large technology companies. Databricks raised engineering compensation to the 90th percentile of the market, which helped it hire talent and increase product innovation.

  8. 36:39Databricks pivoted toward enterprise sales

    Ghodsi says he joined as CEO in 2016 after Databricks struggled to monetize Spark’s success; the company ended 2015 with $1.6 million in revenue. He describes three changes: focus on enterprise sales rather than relying on product-led growth, bring in more experienced executives, and keep some differentiating technology proprietary instead of open-sourcing everything. He says the changes were necessary for the company to survive.

  9. 44:16New products need startup-speed teams

    Ghodsi says new products need a small team that can iterate quickly, find users and test product-market fit without the main company’s full set of processes. Databricks built Genie outside its main codebase, then integrated it after seeing positive results. He says startups can move three to four times faster because large companies must handle security, legal review, multiple clouds and the risk of breaking existing customers.

  10. 48:01Leaders need trust and a long-term view

    Ghodsi says he would invest earlier in developing trusted leaders and make more bets for the future rather than take shortcuts for short-term revenue. He sees a CEO’s role as balancing delegation with stepping in to resolve overlapping decisions. He also says leaders must repeat important messages in fresh ways because employees may not absorb them the first time.

  11. 54:30Hiring should test the work itself

    Ghodsi says interviews are a noisy signal, so he asks candidates to work on real problems the job would involve. He prefers to conduct references early and says five to ten backdoor references can give a fuller picture of a candidate. He probes for specific examples and asks how candidates compare with others, while using references to understand strengths and weaknesses rather than simply disqualify someone.

  12. 59:11Executive hiring is a learning process

    Ghodsi says hiring Databricks’ sales leader took a year, and he treated the search as a chance to learn what excellent enterprise sales looked like by meeting and questioning candidates. He says the company’s culture principles came from examining both successful hires and people who failed. Those principles include raising the hiring bar, truth-seeking and making decisions from first principles.

  13. 1:04:36Hard work must remain sustainable

    Ghodsi says Databricks has always expected hard work, while acknowledging that people may decide the company is not the right fit or that the pace is not for them. He says leaders should face the truth when someone is no longer succeeding in a role, discuss the problem directly and consider another role before parting ways. He also says it is sometimes better to separate when someone is no longer interested in the work.

  14. 1:10:29AI applications will need a data platform

    Ghodsi expects AI applications to disrupt existing areas but says they will need a platform underneath them that brings together transactional data, analytics and AI. He describes Databricks as effectively an AI database and hopes it can power those applications. He points to log processing and threat detection, observability, and customer data platforms as areas where he sees this pattern, with disruption possibly arriving in the next two, three or four years.

Published Nov 22, 2024.

Transcript · 1 h 11 m read

The Logan Bartlett Show, hosted by Logan Bartlett, with Ali Ghodsi.

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

  1. Welcome to the Logan Bartlett Show. On this episode, what you're going to hear is a conversation I have with Ali Godsey. Ali is the co-founder and CEO of Databricks, a business valued at over $50 billion. Databricks sits in the center of the artificial intelligence revolution that we're seeing today. Ali and I talk about the state of AI as well as what he's seeing going on at the ground level, his view on the federation of models that he sees coming, as well as what he sees for the future of AI over the course of the next couple of years. We also go into the founding of Databricks, his journey to becoming ultimately the CEO, as well as the different operating principles that he runs the company with today.

  2. A really fun conversation that you'll hear with Ali now. Ali, thanks for doing this. Of course. So you've been doing artificial intelligence since 2013 with Databricks? With Databricks, yeah. Before that, we were at Berkeley. We were doing also AI. When did you actually start? I think 2009. We started participating early, 2009-10. Spark, when it was originally created, was to participate in the Netflix competition, which was, hey, they gave you all these movies and you had to do recommendations and they would give a million dollars, a half a million dollars to the winner. So we got the highest score on that together with another team.

  3. So you're 15 years in now. Yeah. It obviously was considered somewhat niche and maybe academic initially, and then obviously we're in a very different point today. I'm curious, as ChatGPT happened and that moment where it seemingly went from enterprise and business use to mainstream consumer adoption, did that change your perspective on artificial intelligence in any way? Not really. I think that it kind of changed I think from 2013, 14, 15, I just didn't understand why people don't really get into AI. You know, in 2014, I remember having this debate with many of the people at the company that I wanted to call it AI.

Read the whole transcript
  1. They said, no, that's like not authentic. We should call it machine learning. It's ML. And I said, look, once this stuff goes mainstream, you know, mainstream, it's not going to be called ML, right? They're like, no, but that's not accurate. AI is robotics. And I just never could understand why it's not much bigger of a deal. It was kind of a little bit esoteric, machine learning AI. And in, you know, 2022, November, it just made all the sense but then the magnitude of how much it sort of just shook everybody up that was kind of a shock it was surprising to you were there points that you looked back on actually in those 15 years that were actually maybe groundbreaking to you that you were but but not to the mainstream yeah i mean like when we started people weren't really using machine learning for other things than ad tech it was really ad placement you know click through you you know, click streams for logs to place ads and so on.

  2. So I would say 2014, 15, when we said really large enterprises started using this stuff for more interesting things like Expedia, how they listed the hotels, that how you ranked the listing of the hotels gave them a huge revenue uplift. It was kind of a game changer. Regeneron found sort of, so they found the marker in the genetic DNA for a disease and they were developing drugs. So you could start seeing like, Like, wow, that has a really big impact on a lot of these companies. That was a big deal. But then I would say 2018-19, we started seeing the natural language processing breakthroughs coming with GPT-2 and so on.

  3. It was already pretty amazing. So it was pretty clear that the Turing test would be broken soon in a couple of years or so. So, you know, it's been many of them. But yeah, who would have anticipated that the whole world would kind of wake up in this way and this would be like front news every day, you know. Yeah, hard to believe. Dave, I've heard you compare the ChatGP moment to the iPhone moment. Can you compare that analogy and what the implications are for apps and maybe how enterprises or users should think about artificial intelligence? Yeah, I mean, when the iPhone came out, everybody realized kind of what the smartphone...

  4. revolution would be and what happened immediately was that every company on the planet said we have to build lots of apps we have to build our own iphone apps um and so there was this flurry activity all the companies building apps but you know did the apps really make sense a lot of the apps that they built you know i i kind of joke about it and i say it's kind of like building flashlight application for your iphone uh but then a couple years later the iphone just has the flashlight built in and i think 2023 was one of those years that everybody wanted to say hey i own ai at our company i am the ai leader i am the gen ai so there were a lot of chatbots that were built and you know they'll be just like there's an app for that in 2007 or 8 there's going to be an ai company for that that just does that um so um you know for instance people building chatbot for hr documentation you can ask you know what's your policy on paternity leave or maternal leave and a company is building that but it doesn't make sense that a company builds that for themselves It makes more sense that there is a SaaS app for that and everybody just buys that.

  5. And a little bit we're in this moment where there's a lot of like, hey, we're going to build our own flashlight app or whatever chat app for everything. And now we're seeing slowly the companies being built and the apps being built that are going to specialize and we're all going to just leverage them. I think we're like in the first innings of that. So to take that analogy a step further, do you think that there ultimately will be disintermediation between the end customer and maybe the HRS system where a native AI business sort of takes in that natural language interface and no longer would Workday or ADP or whoever it is on the back end sort of serve as the interface to the user?

  6. Could be. I think that this dynamic is playing out that in this AI era, data is extremely important. Like, so why shouldn't everybody build their own chat app? Well, you know, you should really ask yourself, what's your competitive advantage as a company? What is it you can do uniquely that no one else in your industry can do? Everybody can call OpenAI as LLM. So you're just calling, you know, GPT-4 or 506. That's not going to help. So what does it have? Typically it comes back to the data or some customer relationship, but typically data, you must have some data advantage that nobody else has.

  7. So that's what it's going to come down to. The big companies, the incumbents, they have that data advantage, right? They have customers and they're collecting a lot of data. So the question is, can they leverage that data advantage and incorporate the AI and do the innovation that's needed? On the other hand, we've seen in these waves before that you'll have these smart founding teams that are just completely disrupted, thinking about things differently and they'll rise and they'll do things in a way that the big companies are just, they're asleep at the wheel and they're not going to do. I think we're going to see both.

  8. I think we're going to see some, we're going to see a lot of company startups that fail just like in 99 when the internet was about to come. But we're also going to see a few that become the next Google, Facebook, and so on. But I think we're also going to see some incumbents that actually figure out how to leverage the data that they have and do the investments needed to crack the code on this. We're going to see both of those. How exactly that, you know, fleshes out, I would probably bet more on the startups, to be honest, than the big ones. Interesting. They move too slow. The big ones move too slow.

  9. And, you know, if you have like a killer app, you can get the data flywheel probably collecting data on it and grow the usage. And then you'll have enough data advantage to be able to do that. And if the big guys are not paying attention, it'll move too fast. And before you know it, it's too late to react. Where do you think we are in the hype cycle of AI right now? I think that we're going to see this. I don't think like, hey, we're in trough of disillusionment. I think it's, you know, frankly, I think it's like somewhere at the top. And we're going to see that, you know, soon we're going to see more and more companies not doing so well and i think there's going to be a narrative that hey this ai thing was complete you know it was over promised there's nothing there the quality of the models aren't there they just don't work this stuff doesn't work it wasn't worth it all these companies and so on so i think we're kind of going in towards that very very soon uh but then i think right after that is you know when we're actually going to see those very companies start coming out with these applications that are completely game-changing for humanity.

  10. Yeah, the good news is crypto's back, so people can bounce back and forth between one and the other. I'm curious, as we record this, I guess, early November of 2024, there's been a bunch of reports about scaling laws and maybe the limitations that were reaching some of the outer ends and diminishing returns on that.

  11. Ali Ghodsi · 7 min

    “in aggregate, we get more revenue from tiny models.”

  12. agentic systems. So it sounds like a lot of lead bullets. There's no silver bullet that you sort of see on the horizon at this point. When you hear people talking about $50 billion models or throwing all this, it's very good for your business

  13. I'm sure as people think about this. But do you think that all of the stuff we were just talking about, it means that that's a little pie in the sky and overly ambitious for diminishing returns as people sort of talk about these models at that level of scale? Well, again, I don't have a crystal ball and maybe they're right. And if they are, it's amazing. You've been pretty good historically in predicting some of these things. I think that the reasoning that they have makes sense, which is like, hey, we're freaking creating AGI. It's worth any cost. It doesn't matter what it costs. And it doesn't matter what dilution.

  14. and your economic models done, I don't care about them because look at what we're creating. Makes sense to me if you actually achieve it. But, you know, I would probably lean more on the skeptical side. You know, spending that much money, like my, I guess I'm too, I guess I'm too much. of an operator running a company that needs to think about, you know, you know, growth rate, LTV, CAC, you know, you know, so on and so forth, operating margin that I think, you know, if you got to spend that much money to produce a model and then you have to do inference on that model to, you know, for it to sort of pay back the investments that you need to train that model.

  15. And it seems right now the case is that before you've even gotten the money back for how much it costs to train the model, it's already commoditized and there's a better version by someone somewhere. So then economically, it's just right now not feasible to do that. It seems that for those particular tasks, the cost to create these compound agentic systems, that's worthwhile. And the ROI is there. Why not do that? And it can drive a lot of value and continue doing that. And then maybe we'll be surprised when one of these labs come up with that super model. But I'm skeptical. I think that there's going to be a barrier.

  16. We've got to overcome some big barrier to get to that complete autonomous that can just get the accuracy right and it's reliable. If we froze model development right now, do you think that the existing capabilities that we have in AI is enough to catch up at some point with all the tooling and the applications and all that with the excitement and the promise that we're seeing? Or do you think these incremental gains that we're sort of talking about in the more niche sort of specialized models. We need to keep making those improvements to catch up with the potential of what we could actually accomplish.

  17. Absolutely. I think that humans are the longest pull in the 10th year. If no human uses any of this stuff, it's absolutely useless. You can't even come up with that super AGI thing. If nobody's using it, it doesn't matter. Humans take a very long time to change all of their behaviors, habits, processes in these companies and so on. It's just very, very hard. So I think with what just we have now, there's huge untapped potential, and it's not yet been applied in every workplace, in every function, in every process, in everything that we're doing. And I think we can even improve what we have today with those kind of techniques that we discussed.

  18. So I think like, even if we freeze it and we say, hey, that's it, actually, we didn't know how to create a single model that's even better. This is the best we'll see in the next 20 years. I think there's going to be massive implications on the economy and, you know, on what we can do and so on. So I would still be extremely bullish on this. That's really true to what I believe in. Is there something that you see from your seat and purview that you have that maybe wouldn't be intuitive to the average listener that's, you know, paying attention to all this stuff, but you're able to see so clearly the same way that you maybe did 10 years ago.

  19. You're like, hey, why isn't anyone, why does anyone care about this AI thing? Is there another insight, maybe not at that quantum, but something else that you sort of see that you feel like might be mainstream or well understood in a couple of years that is underappreciated today? Yeah, I mean, I think it's the fact that you're focusing the task. I kind of said it. You focus the tasks on the repetitive stuff that today is costing you a lot of money. And if you can just get the reliability up on those specific tasks, it's massive improvement for these organizations. And it's pretty game changing.

  20. We haven't actually reaped that yet. Those use cases are sort of right now being worked on. Folks are building them. They're not in production. It hasn't had impact on all the companies that are using this stuff, but it's just around the corner. It's like guaranteed it's going to appear. It will probably take two, three years before we see it, and then we'll see the impact of it. And actually, you know, I fear or, you know, suspect that there might be, before that we hit this trough of this illusion, and then actually these use cases land, and we'll see the companies that are doing that. So you can see it baking already.

  21. It's just not in production yet. Are there use cases that you've seen people use Databricks or other systems for that have gotten you particularly excited? Back to the Regeneron analogy that you were saying 10 years ago. Maybe something today that you guys see on the field that is exciting or makes you particularly looking forward to the future of what could happen.

  22. Ali Ghodsi · 3 min

    “it's probably at least a 10 to 1 ratio between how much people are using the models to read stuff than to produce.”

  23. where AI and automation can exist within the organization and how to continue to dive deeper into that journey, are there any takeaways or key points that you keep hitting that you feel like, hey, if you could tell all customers or this is how to set up a team, or this is how to think about it.

  24. Any frameworks or primitives or whatever that you think about? First of all, you've got to get your data strategy right. A lot of companies now want to sort of skip that step and jump straight to the AI. And the data teams have been at it, trying to get the data of their company organized, secured, get it in one place so they can actually get access to it, not in different silos. And that has been a journey, and it's not finished work. that work has not been on the CEO's radar of the Fortune 500 it's been sort of an esoteric topic, maybe IT was sort of worrying about it now the CEOs are waking up after ChatGPT and they're saying I need AI it's game changing, I see it now, I've used it myself give me AI and they're sort of skipping that step so it's extremely important that you go back to the basics the data is extremely important so you have to get that in a good place And if you don't have that in a good place, you're not going to actually going to be able to succeed with AI.

  25. The second thing is focus the use cases of AI that you want to do on where you have a data advantage. So for your company, what is your data advantage? And frankly speaking, oftentimes... find myself talking to customers, telling them, maybe don't do this project. And I say, why not? We're very excited about it. We're going to do this and that. We're ready to go. We'll pay you a lot of money for it. And it's like, yeah, but what's your advantage in this? You know, there'll be an app for that by someone else. So let's focus on places where you have a competitive advantage against your competition.

  26. And it typically has to come back to something you have around the data and you do have it. So let's focus on those use cases because we want our customers to be successful. Only then long-term will we succeed as well. So I would say that's the second thing I say. The third is there's a lot of people problems in large organizations. You know, there's now many people that are fighting over owning AI in the organization. Like I'm the person who owns AI at our company and so on. So figuring out a way so that you can get the strategy aligned in large organizations is, I know it's boring, but this is the hardest part of a large organization.

  27. with so many different people running in different directions, having so many ideas of which vendor to use, what kind of models to use, what kind of people to hire, which project to do. So if you can concentrate sort of the decision-making to one person in the organization and not have lots of departments have political fights over it, it streamlines and moves things faster. Typically, you actually have the head of that department being a change agent person. It's someone that doesn't do things that everyone will love. They'll come in and make the difficult decisions, streamline the data processing, the security around, the privacy around it, because everybody's worried about that.

  28. And then focus on those use cases that actually matter for that organization. This is as important as a technology. I care a lot about technology and geek out on that, but this is just as important. Projects will get not funded or they get stuck in security or they go nowhere because different departments are fighting each other. So we need to overcome these things as well. This is why I'm saying it's going to take a few years before we see the impact of these things. As you think about the landscape and the term that you used earlier to sort of talk about was a compound, what was the compound data and all that?

  29. Yeah, compound agent systems. How do you think about what areas to potentially pursue? There's a bunch of venture companies in spaces like inference or evals or whatever it is. As you think about different areas to potentially go into for Databricks as a business, how do you evaluate, no pun intended, I guess, those markets to potentially pursue. Yeah, it makes sense. Look, it's just for us, it makes a lot of sense as you already have all your data with Databricks. You already trust us that it's secure. You know that we can handle the security, the privacy. So the question is, what's really adjacent to that?

  30. And for me, the CEO, where is it we can actually bring extra value to the customer that they otherwise couldn't get elsewhere? Typically comes back to the data that they have with us already. So typically comes with governance around that data, with guardrails around that data. You know, you don't want to build, this actually happened with a customer. They had a chatbot, and if you asked it, what's the best, you know, car I could buy? And if this was a car manufacturer, it recommended a competitor's car, you know? So how do you put guardrails around that? You know, how do you deal with PII data?

  31. How do you deal with AI regulation that's coming? A lot of this we can standardize in Databricks. So it makes sense. We can like automatically produce documentation so you're compliant with the regulation that's coming. We can put guardrails so that it doesn't do those kind of things. And since we have the data, we can create this data loop so that you get this flywheel of when they use the model, we create the evals and automatically feed it back to making this model smarter and smarter and doing that. It seems we can do that because they already have the data with us. So it's just advantageous for us to do that.

  32. I think in this space, like for startups, they should focus on the applications. I think applications are going to be where a lot of value in the AI space is going to accrue in the future. We'd love to be their infrastructure. So they use us for their data processing, for their governance, and for their AI. The applications, I mean, there's a lot of creativity. There's a lot of things that you're seeing happening. I can't predict what's going to be the apps that... actually going to win in the future it's kind of like in 2000 trying to predict predict facebook or twitter no one could have done that no one ever would have said oh you know there's going to be a company that does you know 140 character tweets it's going to be the same thing now so we just want to be that infrastructure for them so that they can build their ai systems on top of us and we help them with that data flywheel we back up a little bit um to the original early days of spark so we talked about the netflix competition in that uh i had heard that maybe you tried to actually sell the technology in the early days and there weren't really any takers.

  33. Can you just go back to that early story? What did you see and what was your path to getting it off the ground into some commercial entity? Yeah, I mean, what was really happening is that we were seeing that the Silicon Valley tech companies were leveraging AI. This is 2009 or 10. We were seeing what Facebook was doing. We were seeing what Google was doing. And it was very clear that doing AI on the data is going to be the most important thing. And at the time, everybody was stuck on doing basic analytics, backwards looking. Call it BI, business intelligence, which is backwards looking. Like, how was my revenue last week?

  34. And they were excited that they could do that really, really fast. But we were asking, hey, what about, what's your revenue going to be next week? AI. And there was just not much appetite for that at the time. And we went around and we tried to push the technology out because we just wanted to do research. But it was very hard to actually get anyone to actually see this. You might remember the phrase, it was the Hadoop Wars, Hadoop days. It was impossible to get anyone's attention for that stuff. So it was kind of a little bit out of despair that we started the company and said, look, if we don't do it ourselves, no one is going to do it.

  35. I remember actually one of our first customers, we said, hey, this is open source, just take it because we don't want to make money on this. And I said, no, you got to charge us for it because we want an enterprise that's behind this. And that's kind of when we realized that, okay, there needs to be like an enterprise business. No one else is going to do that. We probably have to do it ourselves. Was it an issue of just pure focus for these vendors? Or was there something from a data structure standpoint

  36. Ali Ghodsi · 3 min

    “if you go on-prem, they're a much bigger company. So we have to have an asymmetric bet that we believe in”

  37. and then hopefully you're right as um as you reflect on the journey one of the stories that i had heard was um giving a lot of credit to to ben horowitz and actually turning this into a business can you tell that story typically i don't allow other vc plugs but i thought it was an interesting one that he actually was uh pushing you guys to think big and turn this into a commercial product yeah i mean look we came out of the research lab we really wanted to raise only two three four hundred thousand dollar seed and just code away for a year or so and he's the one that pushed hard no less to the series a and i want to be exclusive investor and um you know we were actually not very excited about it but then he gave us a price we couldn't refuse and he said no go big and we gave us you know 14 million at the time which is a lot of money uh back then i You know, it's not much these days.

  38. And he said, go, go, go, go, go. So he really pushed us, go aggressive, disrupt the market, think big. You know, he always was a couple of orders of magnitude, bigger sort of vision for Databricks than we ever had. Like whatever was our biggest grandiose vision of Databricks, he always was a couple of orders of magnitude ahead and was pushing aggressively saying, no, no, no, you're thinking too small, like way too small. And I think he really truly believed it as well. so that kind of set the tone from the early days for us. Because the early founders I don't know if you remember these numbers but you guys you remember the actual numbers so what were the bets for the early days?

  39. Well I mean like we were around the table when we started the company and said hey what's like a great outcome for the company and I think when we were around the table the numbers were something along the lines of like 150 million exit for us or 200 and someone said 250 and so on so that's how we were thinking and Ben was immediately like no no no billions for sure we're going to do and then And when we got to that, you know, it was like, this is easily going to be a $100 billion company. And, you know, now where we are now, he's like, definitely a trillion is what you need to think about.

  40. Ali Ghodsi · 3 min

    “We literally went back and moved the compensation for engineering to 90th percentile of the market.”

  41. what were you doing through that the first couple years in the journey and then what was the impetus for you to step in and become the CEO? I was running product and engineering which I loved and actually in that phase it made a lot of sense because what do you do the first two, three years of a company when you start a startup you want to get product market fit, build something that just works. So that was the focus for us. Just how do we get one customer that wants to use this and then how do we get a customer that wants to repeatedly use it and then How do we get a customer that repeatedly uses and pays us for it?

  42. And so that was the first two, three years of the focus. The technology that we had innovated with, with Spark, exploded in 2015. And, you know, at that point, we felt like we had succeeded, but we just didn't have any monetization. So we went through a difficult time in 2015. We had to figure out, okay, if we can't monetize, maybe there is nothing here. And the company kind of went through a little bit of sort of introspection. and then we said, okay, let's kind of do a little bit of reset of some of the things and rethink some of the stuff and start anew in 2016. So they came CEO in 2016 and we made a bunch of changes that actually started helping us with monetization.

  43. Initially, were you the interim CEO or were you the CEO when they were still talking to other potential outside candidates? Well, I thought I was, well, first of all, I heard through the grapevine that they're interviewing people. So I found out, okay, they're actually interviewing other external folks. Were you the CEO at the time? but they were like hey do you want to be the CEO I was like sure and then I found out okay they're actually interviewing other people they didn't tell me about that but then eventually I was like okay I think I am the CEO they've announced it to the company I am the CEO I'm running it but then I kind of you know my compensation didn't change and I learned kind of soon that hey I'm actually interim CEO I didn't know that you didn't have the title until so then it turned out no no no actually it was just a test period and then eventually I would say after one year I kind of I think they really kind of like okay no we're going to go with him. So the first, when you came in in 2016, there were three things that you kind of focused on. And it sounds like monetization

  44. Ali Ghodsi · 6 min

    “We were PLG. I know it's still hot. Everybody wants to build PLG companies. But we learned a hard way after two, three years that PLG doesn't really work.”

  45. Ali Ghodsi · 4 min

    “the startups move three to four times faster than a company like Databricks.”

  46. nine years-ish at the CEO job. Is that right? As you reflect on your leadership style, today you seem like the canonical leader of Databricks, and it's the job that you've always done. I assume leadership style has evolved in some ways over the course of this period of time. Anything you could wish you could go back and tell your earlier self taking the CEO job for the first time, or maybe your earliest founder self when you're sort of stepping into the seat?

  47. Yeah, I mean, I think that you would want to grow leaders earlier on, like build trust with few leaders that you groom and continue doing that. Because what you find nine, 10 years later, and I have a lot of awesome co-founders that are still active and they run the company where it's almost a partnership where we run it together. But it's not that many people, like, you know, even though we're almost 8,000 employees, how many people are there that can really help you build the business together it's hard to find those leaders that build your trust with them and grow the company with so I would just invest more in that much more actively and always be on the lookout for those kind of people in the company we were just busy executing and we didn't pay attention too much to that maybe so that's one thing I would do differently the second thing I would do differently is when these trade-off decisions come up and it's really hard to decide do we do, do we go on-prem or not or whatever it is?

  48. But make a bet that, you know, is much more for the future. Don't worry so much about, you know, the implications that it has on our finances or on the revenue. And don't, like, don't let short-termism creep in. Easy to say now, but in the early days when you're, it's far from clear that you're going to be successful as a company. And, you know, the VCs are asking you, how's it going? You have your board meetings. They're looking for your revenue targets. They are super obsessed. How's that? Is it hockey sticking? Is it going up? What's it looking like? How do you compare with others in the industry?

  49. Continue making the bets on the future and don't get carried away on those numbers. That's another advice I would give because we took many shortcuts that backfired. They were not worth doing. We kind of knew it, that this was a shortcut, but it was worth it because it's going to bring so much revenue if we just take this shortcut. I would have avoided many of those shortcuts now. Error on the side of the future. than the present. Now, of course, maybe it's path dependent, and I'm saying it just because we're where we are. If we had done the things that I'm saying we should do, we wouldn't have succeeded. But I still kind of, on balance, feel like we could have been more futuristic in our bets than we were. I get the feeling you focus on empowering individual leaders within your organization and having them deputized to make the best decisions for the organization rather than And you serving as a central hub for each department, is that a fair character?

  50. I think it's both. I think I also micromanage them. Okay. Maybe they would disagree. Yeah, they would probably not say, hey. How do you think about the tension between a central figure that's able to make definitive decisions that sort of sit across an organization versus more decentralization and empowerment at an individual level? Yeah, I mean, you get more clarity if it's one person and it's more consistent. And, you know, that's great. the disadvantage is that it slows things down and it bottlenecks things. And it's maybe less empowering. So you have to strike a balance. And you have to do it in places.

  51. Like, you know, you build trust over time. You figure out which are the parts which you can completely delegate and they're going to do an excellent job. And, you know, they're probably better than you at it. So don't pretend that you're better than them. And what are the areas where you really need to deep dive? And then there's some areas that there's things that only you can do as a CEO. Like, you know, it's literally tie-breaking. It might sound silly, but over time overlapping product areas appear and you get leaders that are kind of building similar-ish things and the Venn diagram intersects and if you leave it to them no matter how many times you tell them figure out how to unify this, do one we're not going to have two similar offerings they're not going to be able to do it on their own so you've got to step in and push you've got to use the CEO card judiciously I think that matters a lot and you can build a much simpler product so I think if you want a really simple unified product that makes sense one person makes much much more sense just have to watch out for bottlenecking things and slowing things down Why is a willingness to say the same thing over and over again an important part of the CEO's job in your mind?

  52. Because I think people are not paying attention to what you're saying. Like, you know, we all have busy lives. We're paying attention to our own life. You know, I didn't even notice this thing that you said. I wasn't even there. Or maybe you said it and I didn't really rock it. I was on my phone. And I had actually fired that day and I was having other issues that I was dealing with. So I didn't really, I didn't really internalize it. I didn't really get it. So it's many, many times you tell someone something for a third time and then only then you notice that they are like, you know, and then they repeat it back to you as if it's like they came up with it.

  53. That's great. I think it's just, you got to repeat it. But what I try to do is figure out a way to repeat it in different ways. Don't repeat it the same way all the time. It gets boring and old. Challenge yourself and come up with a new angle. Find a customer that says the same thing that you want to say. Or use case that's a little bit different. Use different slides, make new slides. And you'll be surprised because people didn't really get it the first time. They'll say, oh, this was completely awesome what you just said now. Even though you're like, well, that's literally what I said like a month ago, but I just refurbished it a little bit.

  54. So I think it's super important. Being truth-seeking, I think, is one of your values. Is that right? How do you implement that in practice or how does that manifest itself as a cultural practice for you guys? Make sure in the hiring process that you test folks on truth-seekingness. It's not that hard, actually, to ask someone, tell me what's your biggest screw-up. Tell me when you really made a mistake. Who did you really not hire well? Why did you make that mistake? How was it? Push them, push them, push them until they get uncomfortable. do they start to come up with lots of excuses well I really actually didn't want to hire this person but my boss made me hire them and that was a bad hire I knew it but yeah that was a mistake I made but technically my boss's mistake so like you know keep pushing them and you can in the interview process actually suss out if someone is truth seeking and honest or whether they're trying to a little bit you know make it look much nicer so that's number one number two don't promote the people in the company that are not truth seeking like at least if you made the mistake of hiring them in then at least make sure you don't promote them so in the promotions

  55. of the company make sure that you're really evaluating the cultural values like really do it it's easy to have the cultural values but are you evaluating the promotion I have a rock star rock star is doing great they move the needle on the business should they get the promotion or not you don't want to be the person saying well wait a minute can we like look at if they were truth seeking oh they're not let's not promote them it's like are you out of your mind this person had huge impact on the company yeah but they're not truth seeking but if you mean it if you want that to be the culture you have to actually do that in the promotion And then, of course, if it's an extreme case, we might have to part ways.

  56. You have to do that as well. You can't have people around that don't fit the culture. So that's what I would say. You spend 30-40% of your time on hiring? Is that right? One of the things you said is that when you're hiring executives, you'll just find reasons to keep in touch and find a bunch of different touch points. Can you maybe talk through that as a philosophy or how you go about doing it?

  57. Ali Ghodsi · 5 min

    “Basically, make them do the job as if they were already an employee in the company.”

  58. I don't think you had experience to it. How did you go about learning in that area? I think you copied maybe best practices or sort of internalized, stitched together to build your own model from a bunch of different organizations.

  59. Maybe can you talk through learning a new area or discipline and how you went about it? Yeah, I mean, first of all, I would just say, we hired Ron Grisco, who took us from essentially zero to now over 2 billion ARR, right? basically 3 billion euro and so he's been phenomenal and what did you see in him yeah I thought that he was extremely smart. So I thought he's actually different from many of the salespeople. He is the classical enterprise B2B sales guy, you know, so he's not like the engineer type that's, you know, he is the classic archetype of a sales leader, you know, with all the raw, raw and motivational speech and get everybody riled up. So it's like, he has all that.

  60. So that's great. That's the stuff we didn't understand or have. He had the big enterprise experience because there's all sellers that sell to mid-market SMB, but But he had actually sold to really big, gigantic enterprises. So that's also something we didn't understand anything about. He was thinking very big. Like when he came in, he was like, we're going to do a $10 million contract or $100 million. We were like, no one's going to ever pay $10 million for Databricks. It was our mindset. So that was also useful. But he also was, you know, he was cerebral and he was extremely smart and went by first principle.

  61. So there was a compatibility. He had an engineering degree. So he was sort of the unicorn of kind of someone who like spoke our language and couldn't understand us and was extremely smart, but also had this whole other thing that we couldn't understand. So that hire was important. We took us a year to hire. We went through a lot of candidates and I think we kind of nailed that one. It was luck. So that's important. The other thing I would say is in the interview process, when you're searching, like we took us a year to hire a salesperson. I've seen people make this mistake. Take the exec hiring process super seriously.

  62. it's a learning process. You're basically going to school to learn what a head of sales looks like. So in that, talk to all the top head of sales and grill them and pay attention and use them also in the interviews. Since I don't believe in interviews, as I said, use the interviews a little bit as a tutorial for yourself to learn how does excellent sales look like in your organization, for your company, for your company, for your company. And you get the, after, you know, talking to 20 people, you get a pretty good picture of, okay, this is kind of it. This is what they're doing. They all have that, All the stellar ones seem to have this in common.

  63. This is what it's about. And you can go deeper. And then in the backdoor process, you figure out who's who. You figure out who are the top people in this industry that you can't hire. Try to get some time with them as well, to just learn from them and build a relationship with them or get them to be advisors. So really put yourself in the, hey, you're going to university to learn. That's what that process is. You know, you're not an expert. And trust the people that are the best in the industry at that. Try to find the number one person and go with them. In hiring for Databricks, are there certain things that you feel you uniquely prioritize or the types of people that succeed within the organization that is maybe unique to Databricks?

  64. Yeah, well, so our culture principles have been reverse engineered to exactly answer your question. So you didn't start with the principles and went the other way. You started with the people that succeeded and came back. Succeeded and failed. Okay. We also looked at people that had perfect CVs that we hired and they bombed out. Why was that? And we went back and said, okay, we couldn't agree on the hiring bar. They were like, I've worked with these 10 people. They're stellar. And we're like, no, that doesn't look like our bar. That was a big issue. So raise the bar. Don't settle. We'd rather have more false negatives than have false positives.

  65. So we'd rather reject someone that's really great by mistake than hire someone by mistake that's not great. And that's painful. We learned that that way. truth seeking a lot of execs from the big companies had this attitude of you know everything's going great I got this my team's got that we're working on this here's a nice slide deck manage up nothing to look for here but that didn't work for us it's like that's where the truth seeking principle came in we really need to know like what's broken who did you who in your team is not working out well what's not what are we not doing well where are we not satisfying the customers so that was the second principle you know third principle I've done this before I know how it's done this is how a much bigger better company is doing it we should do it this way so that was not first principle we needed to know exactly how and why from first principles on every decision.

  66. So being first principle became another culture principle. So our culture principles are the recipe for succeeding at Databricks. Like execs ask me all the time at the end, you know, before just we shake hands and I hire them, they say, hey, so do you think I'm going to be successful here? What do you think is going to be needed? I mean, it's like, it's super simple. I swear to God, read the culture principles. If you really do those, you will be, I mean, of course, the results have to, the output has to be great, but you will work out here. If any of this stuff doesn't look good to you, let's argue about it now because that's going to be an issue.

  67. Those are really the principles of the company. So our culture principles aren't like the aspirational cool stuff that you'd like to have on the wall. It's literally what worked and didn't work in the past empirically. When looking for people within, or to come in Databricks, I think people that have a chip on their shoulder a little bit, is that something you can, do you try to tease out in the conversations? What are the types of things that you'll ask to try to figure that out? Yeah, I mean, the people that really, really, really went against all odds really push they have great one of the things that ron said in the sales uh you know interview that kind of i remember now from 2015 i think november was he said hey if if there's a deal lost you know i want to know like why because there's no way anyone else on the planet could have won that deal like i just know it's humanly impossible because i will have done everything humanly possible to win that deal so i just want to and it was a side comment he was doing with something and it was like clear to him that this this guy is going to go over and beyond and then of course the negotiation with him to hire him was a real pain so that was also

  68. reaffirming that okay he's probably going to be great at this job uh you don't want to head of sales that's you know terrible at negotiating their own comp for themselves um so you know you want to suss out you know is there something something that the people who have something to prove are they trying to prove to you if they're super content and happy with things you know then it's probably not going to work it's probably for people that are a little bit broken like all of us are like we're all we have something to prove and we're going to work our ass off we're going to do everything in our power and um you can suss that out in interviews it's not hard you mentioned work your ass off there i uh that's something that i think has went out of vogue working hard for a while, it's sort of come back in vogue. I'm curious, you had said something along the lines I had here, you'd never met someone successful who didn't work hard. Can you elaborate on that point? In research, in academia, like genius professors, you know, who had won amazing awards, you know, and that's like, you know, you can be really, really smart in research and maybe you don't need to work hard. But all of them that I met,

  69. they were, you know, working harder than us. And we're like shocked. They're there 24 seven. And then in industry, the same thing. So I just never, ever met someone who's just phoning it in and they're super successful. You find talented people, but they're not successful. It has to go with extremely hard work. So we had that from day one at Databricks and we never deviated, whether it was in vogue or not. We were a super hardworking culture from day one. We continued to be, and we said, look, we have lofty goals. To achieve them, we're going to work extremely hard. Totally understandable if you don't want to do it because maybe you have family, maybe other things you want to do in life.

  70. That's totally understandable, but then probably Databricks is not for you. And so it's been always true here at Databricks. In fact, we had, you know, our work-life balance scores weren't great in surveys. And, you know, I tried to be upfront. We don't want to actually have terrible work-life balance. We actually focus on teams that had terrible scores and try to, you know, change things. And we got actually improvements in those teams. But I also was upfront. I don't want to be best in class and have the highest work-life balance score in the industry. That's not, we're not going to win that way.

  71. So it's as simple as that. now i try to myself also match that so i don't want anyone to work harder than me like so i got to kind of match that myself um but and it has to be sustainable what i mean by that is i've been doing this for a decade more uh and i want it to be the same way for the employees so if they're like really burning themselves out and they can't do it for many more years that's not i don't want them to do that. I want them to be able to do it and it's sustainable. And then hopefully we can achieve great things together. And if they don't, that's okay too. No hard feelings, you know, or maybe in a period of their life, they want to do it, but they don't want to do it another.

  72. That's okay too. One of the hardest things our companies deal with in scaling is when the people

  73. Ali Ghodsi · 3 min

    “i have found that there are new roles for people where they can continue to crush it”

  74. I think you're pretty good on it.

  75. Are there elements of the world around Databricks or what you guys are operating that you think are on the cusp of being possible in the next five years? Just things that get you really excited to get out of bed every day and do this job and just the potential maybe at an industry level, not even at a company level that you think we're on the cusp of seeing? Yeah, I think what's going to happen is that we're going to see many of the things that we see today are going to be disrupted. And there's going to be a sort of AI app for those things in the future. We don't even know what they're going to look like, you know, but what do they all need at their core?

  76. Like what did apps need in the last 10, 20, 30, 40 years? They needed an Oracle database under the hood. You know, the joke is like you put all the complex, you stuff all the complexity into a database and then you build workflows and UIs on top or the two-tiered or three-tiered model. I think in the future, you need the AI database under the hood. And if you have an AI database, you have all your transactional data, you have all your analytics and AI in there with the model, and then you can do wonders on top. So I think we're going to see an ecosystem of these applications in the AI space being built, and they all need an AI database under the hood.

  77. who Databricks is. Databricks is basically an AI database. We don't call it that. We don't market it that way. But essentially, it's an AI database under the hood. And you will need that for every application. Every company on the planet will need that. And I hope that we can be the company that powers all of these applications in the future. There'll be others also that do that as well. But that's the goal. And I think we're seeing already the seeds of this happening. The seeds of it, I guess I'm curious just to elaborate on that. The seeds of this happening, are you seeing just particularly exciting use cases or just customers...

  78. Look, in the same space, they need to do massive log processing securely and detect zero-day attacks. That's kind of what Databricks bread and butter is, real-time processing of massive logs with AI built in. In the observability space, same thing. You need to observe your log processing, understand the metrics, detect anomalies, and so on. In the customer 360 space, CDPs and understanding what your customers are doing, same thing again there. It's like, you know, massive data that you have to collect from many, many different places and then get telemetry on it. It's huge data processing. You need to do it in real time.

  79. You need to do AI to get the insights and doing those. So each one of these spaces, I think, are going to be built on basically an AI database. I hope we can be the AI database that powers all of them. This disruption, I think, is going to happen in the next two, three, four years. It's exciting. Well, thank you for doing this. Likewise. guys thank you for joining this episode of the logan bartlett show with co-founder and ceo of databricks ollie godsey if you enjoyed this conversation we'd appreciate it if you shared to anyone else that you think might find it interesting as well as subscribe to us on whatever podcast platform you're listening to us on we look forward to seeing you back here next week with another great guest on the logan bartlett show have a good weekend everyone We'll see you next time.

More interviews