Free with a sign-in.
The best 10 min of 61
The Window Before AWS Wakes Up: Nebius CTO on Racing the Hyperscalers
Nebius CTO Danila Shtan says independent AI clouds can grow before hyperscalers adapt their infrastructure and chip supply.
Unsupervised Learning · Jacob Effron
- Danila Shtan is CTO of Nebius.
The best minutes
- 39:13Chip scarcity gave AI clouds room to competePro pick 5 min
- 25:05Open-model inference still has major room to improvePro pick 3 min
- 53:19Nebius lets engineers use AI but keeps accountability human3 min
Every moment plays free. PRO marks a write-up that's in Pro.
Also worth watching
- 15:36AI customers need shared storage built for extreme throughput4 min
- 10:12New Nvidia chips require constant iteration before launch5 min
- 21:47Prefill/decode disaggregation remains a case-by-case tool3 min
- 34:25Execution capacity is the bottleneck across AI infrastructure3 min
- 57:53Agent reliability may matter more than model size3 min
Skip the market overview and origin story, the Tavily acquisition and data-center comparison, and the quickfire and outro.
Summary · 4 min read
Danila Shtan is CTO of Nebius and says he has been with the company since before it was called Nebius. Nebius separated from Yandex in 2024; Shtan says the company drew on Yandex’s infrastructure team while rebuilding its cloud platform from scratch.
Shtan argues that Nebius competes through software that serves both large interconnected GPU clusters and small self-service customers, while independent AI clouds can meet demand that hyperscalers cannot prioritize. He says open-model inference remains poorly optimized and that 95% of Nebius customers run modified models, making model, kernel and workload-specific optimization important. He describes AI infrastructure bottlenecks as a test of execution across supply, facilities and project management, and says hyperscalers may invest more as chip supply improves. He also says Nebius is cautious about AI agents at work and is focused on improving the reliability of agent outcomes, not just model capability.
A Pro write-up, open to you here. Try Pro free for 7 days
A Pro write-up. The rest of the summary and the minute by minute are in Real Alpha Pro; the episode, its moments and the transcript stay free.
Read it with Pro, 7 days freeMinute by minute
- 1:21Software is the cloud differentiator
Shtan says Nebius was built to serve both customers using tens of thousands of GPUs in an interconnected cluster and self-service customers renting one GPU for half an hour. He expects AI demand to grow sharply and sees room for independent providers serving companies that need capacity beyond their own infrastructure.
- 6:12Nebius grew from Yandex
Shtan says Nebius separated from Yandex in 2024 and retained an experienced hardware team that had operated infrastructure at scale. The company rewrote its software platform from scratch; in 2024, he says, its software organization had 200 people, compared with roughly 40 to 50 on the hardware team.
- 10:12New chips take repeated testing
Shtan says NVIDIA’s designs can change between early schematics and production samples, so cloud providers must keep adapting. Nebius tests new hardware across virtualization, storage and networking, then reports issues and works with NVIDIA on fixes before exposing the hardware to customers.
- 14:55Customers need more than GPUs
Shtan says training and inference workloads are broadly familiar, but storage throughput has become a major requirement: some customers draw 700 gigabytes per second from shared storage. AI labs often prioritize compute and availability, while enterprise customers also need compliance, security reviews and certifications.
- 19:56Latency tuning sits in software
Shtan says Nebius aims for a universal hardware baseline rather than building different clusters for each latency target. For managed inference, the team tunes performance through orchestration, models and request routing to meet customer SLAs.
- 21:47Disaggregation is not routine
Shtan says prefill-and-decode disaggregation is widely discussed but remains difficult to operate and support with reliable SLAs. Nebius has promising prototypes and some manually configured production setups for demanding customers, but does not use it as a default solution.
- 24:53Open-model inference needs work
Shtan says open-model inference is not a solved problem: default server configurations can produce disappointing speed and cost. He says 95% of Nebius customers modify or fine-tune their models, and sees room for kernel work, compression and pruning tailored to narrower tasks.
- 28:28Nebius invests across optimization
Shtan says inference improvements can compound, so Nebius invests in areas including kernel optimization and model compression. He says an acquisition earlier in the summer added model-optimization talent, bringing the combined team to tens of people working on these problems.
- 30:07Tavily adds an agent tool
Shtan says Nebius was impressed by Tavily’s product and go-to-market approach, and believed its infrastructure expertise could help the web search company scale. He says Nebius is exploring ways to combine search with managed inference and other agent-building tools, including sandboxes.
- 32:10Facilities follow Nebius specifications
Shtan says Nebius controls the internal network and hardware whether a facility is owned, rented or funded through its asset-light model. He says GPUs, components, electricity, facilities, workers and transformers are all constrained, so execution—especially sourcing, scheduling and moving work between sites—is the central challenge.
- 36:59Workloads change cluster packing
Shtan says new workloads rarely require rebuilding infrastructure at the lowest level. They mainly change how Nebius packs and splits work, collocates workloads and exposes the right tools through its platform.
- 39:13Hyperscalers face competing priorities
Shtan says hyperscalers compete for limited chips while also reserving capacity for their own teams, which can make AI cloud a relatively small part of their business. He adds that adapting infrastructure built for other workloads requires major investment, and expects hyperscalers to pay more attention if chip shortages ease.
- 43:38Independent AI clouds can persist
Shtan expects some of the current 300 to 500 AI cloud providers to consolidate or fail, but says there can still be hundreds of providers serving distinct needs. He argues that larger platforms will broaden their offerings without eliminating independent specialists. On OpenAI’s partnership with Baseten, he says it could let users try open models as a lower-cost option while remaining in OpenAI’s ecosystem.
- 52:08Agent outcomes need more than models
Shtan says AI coding tools have improved sharply, but Nebius lets employees use tokens without per-person limits while monitoring usage and holding engineers responsible for agent-written work; fully autonomous agents cannot touch core services yet. He says he underestimated customer scale, including 30-megawatt facilities and shared storage at 700 gigabytes per second. He is now focused on improving agent outcomes through tools and harnesses, not just larger models.
Published Nov 2, 2026.
Transcript · 42 min read
Unsupervised Learning, hosted by Jacob Effron, with Danila Shtan.
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
If NVIDIA could have bought, could have produced like 10x the chips, this whole market would have been shaped differently. Because there is not enough capacity for their scale, this is a relatively small business for them. For the hyperscalers, it's not just a matter of like sales, it's also a matter of changing the infrastructure that was built at scale for other kinds of workloads. This is a massive investment on their part that needs to be justified by the size of a potential business. And it's not that there yet. the emergence of nebius and the other hyperscaler competitors these past years has been a fascinating part of the ai story it's something i've really been interested in and so it was a lot of fun to have nebius's cto neela on the podcast because we were able to dig into a bunch of interesting things about the business and how the space has unfolded we talked about nebius's really interesting origin story and the original relationship of the index and how the company came to be we talked about what it's like to be inside one of these companies how nebius works you know, with NVIDIA, what it looks like to get their next generation of chips live.
We talked about why the opportunity actually existed and why AWS, GCP, Azure didn't just capture this entire market. And we also talked about Xeniel's view on a bunch of new technologies like DISAG inference, among other things. All in all, just awesome to get to talk with someone who's at the epicenter of the AI infrastructure market about his thoughts on everything about where the space is headed. I think folks will really enjoy it. Without further ado, here he is. I guess I'd love to talk just maybe about the broader, like, AI cloud market, you know, today. And I think that there's a few maybe larger and prominent public, you know, I guess termed neoclouds.
And then, you know, it feels like there's a new one getting funded every day. I know you don't like that term, but maybe just talk a little bit about how you think about what differentiates. Like, some of our listeners are probably thinking, like, you know, all that really matters is just access to GPUs. And like, you know, wherever you can get them, you go. Maybe use that as a straw man and tell us why that's not the case. Well, I'm obviously biased because, like, first of all, I'm a software person and our product and business was launched on the idea that software is the mode, right? So your ability to serve the customers that consume, I don't know, tens of thousands of GPUs in one interconnected cluster with the same platform as you serve, like, a self-service customer that rents one GPU for half an hour.
Read the whole transcript
was our value prop from the very beginning. So we believe that this is extremely important, being able to build large scale but have the possibility almost like sell it in retail in a sense. This is what we think definitely differentiates us from the market, from everyone on the market. Like a small remark, NeoClouds hate that term with a passion because it started as a distinction as in these are kind of cloud companies that are doing this weird, strange new thing, but then everyone wants it in that market. So when we started, there were like, I don't know, five, seven companies that were in this area that were called NeoClouds.
Now, I think it's closer to like three to 500. Yeah. Depending on what you even are ready to call NeoCloud. the scale and we really don't like this stretched umbrella term anymore we are the ai cloud we don't care like let others other guys be neoclouds we are the ai cloud and this is our mental model we are not competing with the neocloud markets we kind of believe we are past that our next competition point is taking on aws and google cloud yes we are way smaller yes our like revenue is like what i think one percent of aws nowadays But we are running fast. We serve a very particular market. We have a bet that this is the market that is going to grow, that is going to explode.
So people are concerned that these investments are huge, people put a lot into the infrastructure, et cetera, et cetera, et cetera. My view, if I compare what we see in the market with what was there two years ago, we are barely scratching the surface. I think the overall demand in the next three years will increase tenfold, hundredfold, maybe. So we are at the verge of another explosive growth. Right now, we see these huge facilities, right? But a lot of those are privately owned for internal needs of certain companies, right? So Elon Musk is talking about his Colossus cluster, and it's immense scale.
Like, really, it's like real solid, large scale thing. But it's not for the broader market, right? It's for their internal needs, maybe for some partners that they decide. Maybe for Anthropik. Yeah, Anthropik can OpenAI amassing the capacity, but they do it for their own product. And this is like a wholly separate discussion. But these companies will essentially want to be everything. They will want to be the internet and contain everything and every product and every capacity angle within themselves. There is a question of Google, Meta, and Microsoft, who have a lot of capacity but are constantly juggling and struggling between giving it out to the market to rent or giving it out to internal teams.
because all of these companies are extremely machine learning, AI enabled. They need a ton of capacity for internal usage, for runtime and for research. So they have to constantly juggle this. So our bet is that there is a niche for independent, as in not affiliated with a large company with a huge internal consumption player that will serve the markets of the meet your players who want to be the large companies. I thought it'd just be interesting for our listeners to talk a little bit about the history of Nebius. I feel like it's a really interesting path that you had to end up here. And so maybe just talk a little bit about how the business came to be.
I'll start with the moment I've joined. So I am CTO of the company, but I've been with the company from the very beginning, like since before it was called Nebius. So the new company is established. I think during 2024 also, the deal was finalized. We finally announced our separation from Yandex, no connections to the Russian assets anymore, clean slate, et cetera, et cetera, et cetera. And that was it. People have come to these AI cloud businesses in different ways. And obviously, I feel like the Yandex team was legendarily really, really strong on the engineering side, you know, a lot of folks building iClouds came from, like, crypto or energy or, like, not necessarily the same, you know, maybe technical heritage. To what extent, I guess, was the Yandex experience really relevant? And to what extent was it like, you know, it turns out no one knows how to build a GPU cloud because it's all kind of new and, you know, there's everyone's learning on the fly. First, we had and still have the core team that supports the hardware side of business that came from yandex that had experience operating hardware like infrastructure at scale like at real scale hundreds of thousands millions of servers span over multiple facilities etc etc etc these
people like were the core of our build-outs like initial build-outs and expansion and they're still with the company so this definitely helped i think play. We would be way less successful if not for that core team. Yeah. On the software side, it's interesting because it's a two-fold. Like with hardware, it's very straightforward. We needed people with experience. We had people with experience. They started building out from the get-go. On the software side, it was a weird one. First of all, we had to rewrite everything from scratch. So our previous experience was kind of like garbled. Second, we had to grow so fast that by, for example, by the end of 2024, when the new platform launched, like a significant portion of our engineering team, software engineering team was not from Yandex anymore because we had to hire during this period extensively.
As a team, we were working on this cloud platform from the initial idea, right? So we kind of just went with it and doubled down on software. We are not doing a managed bare metal. We are not doing a patchwork of third-party services. We want to build a holistic cloud experience, the holistic cloud platform. And we kind of doubled down on that, even with the whole rewriting of the platform, like doing it from scratch, et cetera, et cetera, et cetera. And of course, having a ton of talented engineers on the team, obviously helped just for the scale of it you've mentioned people who came from crypto or came from other avenues of life, they are mostly concentrated on hardware, their idea is you need to buy GPUs, you need them to run and then you'll find the buyer in our case for example in 2024 when our hardware team that actually procured GPUs did the build doubts. We're working on our own servers and chassis, writing firmware and stuff, building the network.
That whole team was roughly like 40 to 50 people. The software organization was already 200. Right? So we kind of can certainly think of ourselves as a software-first company. We are building the cloud on the solid foundation of hardware. We're not a hardware-first company. Makes total sense.
“they are constantly iterating while these chips are being produced.”
further. And then in the end, yes, there is a product and the customers use it. And that's it.
One thing I think it would be so interesting to get your thoughts on is just the differences in kind of what end customers end up wanting. Like obviously it can be easy to think, okay, everyone just wants to come to you and get similar things. But I imagine you segment your customers or think about differences in what is important to different customers. How do you think about like the key differences in these customer desires today. Hi, I'm Anne Maris and I'm the producer of Unsupervised Learning. Thank you so much for listening to the show. And if you've enjoyed the conversation, please subscribe. Subscribing helps us grow, which helps us continue to lock down some amazing guests. Thank you so much for your listenership and see you next episode. How do you think about like the key differences in these customer desires today?
“five, six, seven hundred gigabytes per second of throughput.”
differentiating thing as opposed to what you would expect from like the size or the performance you know obviously i think a lot of people are thinking through this trade-off of like throughput latent or a trade-off of like latency and and kind of cost um and i imagine you have some customers that have workloads that are super latency sensitive and they'll you know do small batches or whatever is required uh to to get to that latency and then you have other customers that are much less latency sensitive maybe background agents or something is that all just you know uh determined on the software side i guess if you have those different customers of like basically how you know uh the batch size that you're running these workloads in or do you actually construct the clusters in any different ways based on those workloads?
No, no, no, no, no. This was the premise from the very beginning. We are building the universal cloud platform, and we need to make sure that our baseline is high enough to fit most of the workloads. Yeah. Then there is the SLA play, right? So customers come, like, for example, for the managed inference part of the business, customers come with the different demands. and then you are in this kind of a benchmark game, right? They want this particular footprint based on this particular traffic pattern fits these SLAs in terms of latency or latency is time to first token or the whole request latency.
And you start optimizing for that. But this is never done on the hardware level because hardware is good enough. This is rarely done on the lower infrastructure level because it's universal and, again, good enough, no compromise for performance, et cetera, et cetera. And you're going further away to the orchestration level, to the model level, way further up the stack, all the way to how exactly does the request from the customer, like the endpoint, gets to the inferencers and what are your performance hits along the way. Yeah, that makes sense.
“we are in the stage where everyone's talking about that, and no one's doing that properly.”
“95% of our customers do not run vanilla open source models. These are modified, fine-tuned”
There's a ton of work around inference optimization, and a lot of that gets done in open source, right, or contributions that share throughout the community.
There's obviously a bunch of work on smaller purpose-built models and compaction and a bunch of these things. Across all these problems, which do impact, you know, how enterprises consume inference, which ones like Nebius should have a solution or offer a product around and which ones you're like, hey, it's great that that exists and we'll benefit from that, but we don't need to be at the forefront. We want to be at the forefront of everything. In a sense, these are additive and even maybe multiplicative, right? Like, if you have a right subset of things, optimizations that you introduce for a particular use case, it means that you can reach the targets easier, better, faster, in a more predictable fashion.
So we invest in all avenues. Like, one of our acquisitions back in, like, the beginning of summer was targeting specifically that. We basically bought the company for the talent in model optimization, model compression, speculative decoding, et cetera, et cetera, combined it with existing talent. So we have a team that is 2x the size. And we have serious manpower, as in tens and tens of people who are working on these problems. We have specialized groups for kernel optimization, specialized groups for model compression problems. because this is, I feel, if you want to talk about running inference as a service at a serious level, this is the only way you can do it nowadays.
And I guess maybe along those lines, you guys have done a few acquisitions, and one of them that caught a few people's eyes was Tavoli in the web search space. Maybe talk through a little bit the rationale for that, how you think about that fitting into the broader platform. So first, we were very impressed by the product and their go-to-market motion. It was almost a year ago now, right? But we felt like we knew how to help them run even faster with our expertise in technology and low-level stuff. Because actually running a web search engine is not a very simple task, neither from the software development perspective nor from the...
A lot of you used to run one. And some of us dabbled a little bit in our previous lives, yes. Right? So this was the premise. Like, we have a great team that is running real fast in terms of, like, market penetration and, in general, go-to-market motion. And we know how to make it complementary with our expertise on the low-level stuff and running large-scale infrastructure and, for example, large-scale web index. It's not exactly embedded into the rest of the offerings that we have now today. We are exploring and we have something cooking for the managed inference combined with agentic tools because, for example, we also develop sandboxes nowadays, right?
So this is another building blocks, one of the building blocks that people use to build their agents. You have several other products that are in the earlier stages of development in that regard, too. Yeah, I mean, it seems that there's obviously a set of tools that folks are going to use to build agents. Build on open source, and it seems like you want to have a bunch of those tools to be able to offer to your end customers. And web search kind of being a key primitive for any long-running agent will consume a decent amount of tokens on just pulling information in. I guess I'd love to switch gears to just the broader industry build-out.
I think there's lots of questions folks are asking about that. I figured one place to start that actually might just be interesting for our listeners is, is to what extent are the data centers that you work with similar? And there's a reference design, and they all kind of like, I don't know, to you it doesn't really matter which one's which. And what are some of the differences in them that actually impact the business of running an AI cloud on top of them? Well, to be frank with you, I have no idea what the walls of the facility are made of. So they're probably different, but they doesn't really help.
But in all seriousness, what's within the walls is something that we fully control we do not rent data centers when someone like have built the like the network on their own or brought some gpu gpus or servers etc if we don't own the facility and we rent it for example we rent well the building electricity but that's it all the internal build out is controlled by us it's like it's done according to our standards so from the software side of things there is literally no difference which data center we're running is it our own facility is it something that is built within a rented facility or for example we've recently announced this so-called asset light model when a third party brings capital in order to set up the facility and GPUs, and we run our platform on top of that, it's going to be the same. These data centers are built according to our, like, fitted out according to our spec. Hardware is something that is familiar to us because, like, unknown hardware is quite hard to adopt and make sure that you provide proper SLAs.
And that's it. So we don't compromise there. We never rent something from people and present it as our own cloud. If you launch a virtual machine in Nebius cloud, you can be sure that it runs on our stack top to bottom.
“an experienced project manager is probably in highest demand I've ever seen throughout my career.”
Yeah. It's interesting because you obviously have, you know, the hardware itself changing. You have a bunch of workloads changing. Obviously, we have much more long-running agents. We have more post-training going on, you know, people running open-source models. How does that actually impact the way that, you know, the overall, like, I guess the overall build out of the clusters themselves?
To what extent is it like, hey, you know, workload's a workload and, you know, we can run it? No, the workload's a workload. Like, exactly like you said. It is slightly different, but in a sense, you need a high enough baseline to not be, like, constrained by that. And then the rest is just your, like, optimization problem. the way you pack things, the way you split things, right? It's like you don't need whole nodes for some of the workloads. So you need to collocate workloads. You need to have the right primitives. You need to expose those through the platform, et cetera, et cetera, et cetera.
But in a sense, like differentiated workloads almost never like impose any changes on how you build on the lowest level. Yes, of course, like nowadays with like this new architectures like what like like what is it called NVIDIA LPX like the outcome of the grog deal right this like adds to it but in a sense if you have experience running other generations this is just another like 0.5 generation right yes this will be a little bit novel and it's connected in in a weird like compared to the previous one's way and it kind of like there are some restrictions when you can run and not run on that but like there are efficiency gains and this is like just and like a niche local addition to the platform to what to what you offer to the broader market But in general, no, there is way less custom solutions that you would expect from that.
The workload is a workload.
“this is a massive investment on their part that needs to be justified by the size of a potential business. And it's not that there yet.”
you mentioned there's like these like, you know, 300 to 500, like you know neoclouds everyone trying to get into this market like as you kind of you know you mentioned you're kind of thinking the chip shortage comes to an end at some point like as you zoom five years ten years in the future like how many of these businesses are still around or like what is the end state of of this market listen i i i I think the whole like AI consumption is going to grow and it will be like universally used everywhere. Like AI is everywhere. Like right now, like web services are everywhere. AI services will be everywhere. So in a sense, like you have AWS, right? But you also have Hetzner.
You also have DigitalOcean. You also have Linode. You also have like tons and tons and tons of local like hosting providers that are like giving you virtual machines, and I'm experimental, et cetera, et cetera, et cetera. So this market is going to be vast. Will all of the current like 300 to 500 companies survive the next five years? I'm not sure. But I am pretty sure that there will be a place for the very long tail of providers down the line like some of this the current players will consolidate some of them will die off new new players will be launched etc etc etc it's a like it's not a bad market right now it's not a bad business to be in so it will constantly attract capital and attention and investments etc etc etc and i trust that like in five years time there's still going to be like hundreds of players in this market who found the investment and several hundred million dollars and they can offer something to someone in the market. Same as it is with the small scale clouds like DigitalOcean these days, right? DigitalOcean did not die because AWS is growing. You're obviously doing a lot of things in-house like sandboxes and you know, web search and, you know, helping people with their models over time.
And, you know, I think one question a lot of people are asking is, you know, what parts of the AI infrastructure stack end up just being subsumed into these future AI hyperscalers and like live, you know, in just one combined offering? And then what are like the standalone pieces of AI infrastructure that will persist as companies? In a sense, I think it will like, it will kind of mimic what's happened with like the legacy generation of clouds previous generation of clouds right when your growth kind of dictates that you need to start addressing new and like new sectors new markets etc because otherwise your growth is like bounded and you can't like can't show your projections can justify your valuation, et cetera, et cetera, et cetera.
So there will be a natural push for the larger companies to start, like, increasing their portfolio, like, trying to get into new products, into new services, et cetera, et cetera, et cetera. It doesn't mean that there will be a one-size-fits-all solution. Because in a sense, if you're, like, if you're concerned by that, you will, like, we just need to stop doing whatever we are doing and just wait for for like open ai to rule the world entropic to rule the world because these companies will eventually like you do realize that open ai has models runs inference they have their own search that they like they don't exactly expose it but if you're building the agent on their platform it has access to search they have all kinds of like sandboxing, local agents, cloud agents, et cetera, et cetera, et cetera.
They both OpenAI and Entropic are now investing in forward deployed organizations and are going to go and do the work that was traditionally like, I don't know, Deloitte's or like Accenture's piece of pie, right? So they will be everywhere. And I think this is a natural motion for the large company to start getting into a broader markets. but it doesn't mean this is they are going to be the only ones so even nowadays there is a place for independent object storage players all of a sudden right or independent compute players or independence small-scale inference players for that matter independent database players Like Google has BigQuery and whatever, all kinds of data stuff.
But then Snowflake kind of appears from out of nowhere. AWS has all kinds of managed databases, large-scale databases, like Aurora, Redshift, et cetera, et cetera, et cetera. But then MongoDB appears, and it's pretty successful. I can't say it's super large scale, and probably by nowadays standards will never be because nowadays standards for a large company is like trillion plus, right? But like two years ago, it was a very solid and very successful company or someone like ClickHouse appears with their cloud offering, right? Also a massive managed database, an independent company that finds its niche and works it real well.
So I think like larger company, larger clouds will grow. They will have products in every sector and will be pretty successful at selling those because of bundling, because of their sales power, et cetera, et cetera, et cetera. But there will always be a place for niche players. And because the market is huge, niche doesn't mean small. Niche doesn't mean small scale or low revenue or low valuation. This can be solid companies. Yeah. I mean, it's interesting you mentioned, like, you know, OpenAI Anthropic having this kind of broad suite of offerings, too. I mean, in many ways, they're obviously building their own AI clouds.
And I thought one thing that was interesting recently was, you know, obviously a limitation of those AI clouds versus others is the, you know, having just one model family, right, to be able to call from. But I thought it was really interesting that OpenAI announced that partnership with Base 10. I'm curious what your take was on that. I mean, is that them just trying to get into this general, you know, hyperscalar game across models? In a sense, we see a lot of push and concerns about people claiming that OpenAI models or entropic models are too expensive. Everyone is puzzled or surprised by the bills they get at the end of the month.
So this might be an answer to that. Well, if you're a global, go try OpenModels. At the same time, OpenModels are still behind the closed models, right? However good those models are, they're still lagging behind. And giving people an option to try, like, lower scale with the ability to switch back to the top frontier model might also be a good incentive. Anyway, I think OpenAI is doing a very interesting move with that partnership, and I like it really much because this kind of shows flexibility. right they're not certain so like they're exploring different options different things that they can or cannot do like looking for different options to like uh so but i i think in a sense this all plays into one single thing like open ai wants same as in tropic uh they want to be one single like touch point like they want to be one single thing for everything even that is like, okay, but sometimes I want to use open source models within my codex-controlled environment, by the way, right?
So, like, well, good for them. But if the company, the person, the user is still within your ecosystem, you still have a lot of options to steer them, to upsell them, to introduce them to new services. et cetera, et cetera, et cetera. So if that alleviates one of the concerns, so be it, why not? Well, I always like to end these interviews with some overly broad questions that I stuff into the very end. And so if you'll humor me, I guess the first question is just, what's one thing that you've changed your mind on in AI in the last year? In the last year? A lot has happened in the last year. No, the obvious answer would be software engineering.
like a lot of people forget but a year ago we lived in a totally different world yeah it's like dog years in the ai world way less capable yes way less capable in terms of the model capabilities and in terms of how good the modern harnesses and tools are it's like uh the ai coding because like a year ago it was total bullshit right now it's it's a solid part of like your everyday tooling. But for example, if you're not talking about the industry trends or like talking about more like my personal thing, I've started relying more on the like AI enabled like local agents and harnesses to afford a part of my cognitive load, to be honest.
I mean, obviously, you run a really large technical organization. And I think everyone's
“whatever you've done with the agent, it's still your result. It's your responsibility. It's your pull request. It's your code.”
What's the biggest mistake you feel like you've made in building Nevea so far? Obviously, a lot's gone right, but I'm curious, something that in retrospect, you wish you guys had done differently.
It might sound funny these days, but we underestimated the... scale like and it's not about the growth of the company we were pretty optimistic from the very beginning but uh i personally that was like my strong bias for the first like significant part i underestimated the scale that a single client operates at nowadays because like five years ago if you're talking about a 30 megawatt data center it was a serious facility it hosted like hundreds of thousands of servers like run glorious gloriously large foreclose by now 30 megawatts sometimes it is not enough for a single client which i'm still getting used to right the storage i've mentioned right like four years ago when you were talking about network attached storage you were talking about like hundreds of megabytes maybe gigabytes per second of throughput. Right now, some of our clients routinely fetch, I don't know, seven hundreds of gigabytes per second from a single shared file system.
I'm still, I underestimated it in the beginning, and I'm still getting used to this scale. I'm an infrastructure person. I've worked and I've seen these numbers throughout my career multiple times. And all of a sudden and you add a couple of zeros to the end and call it a new norm. Yeah. Still puzzling. That's crazy. And I guess lastly, there's obviously a lot of unsettled questions in the AI field today that lots of people are trying to figure out where the world goes. I'm curious what you're paying attention to and I guess with a lens toward if it goes one way or another, it would really change the strategy for Nebius as a whole.
What are you most paying attention to on these unsettled questions of where the AI world's going?
“I think a lot of future AI adoption lies in how reliable your agentic scenarios are and how reliably you reach those outcomes.”
I'm Jacob Effron, and this has been Unsupervised Learning, a podcast where I get to talk to the smartest people in AI and ask them tons of questions about what's happening with models and what it means for businesses in the world. As I hope is clear, I have a ton of fun doing this. It's a nights and weekends project, in addition to my day job as an investor at Redpoint.
But our ability to get these incredible guests on really comes from folks like you subscribing to the podcast, sharing it with friends. It's really what ultimately makes this whole thing work. And so please consider doing that. And thank you so much for your support and listening. We'll see you next episode.
