# The Window Before AWS Wakes Up: Nebius CTO on Racing the Hyperscalers

Markdown copy of https://realalpha.app/v/qJkcjURcAe8/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

Nebius CTO Danila Shtan says independent AI clouds can grow before hyperscalers adapt their infrastructure and chip supply.

- Show: Unsupervised Learning
- Hosts: [Jacob Effron](https://realalpha.app/people/jacob-effron/)
- Guest: [Danila Shtan](https://realalpha.app/people/danila-shtan/) is CTO of Nebius.
- Published: Nov 2, 2026
- Length: 1 h 1 m

## The best minutes

- [39:13](https://realalpha.app/v/qJkcjURcAe8/?t=2353) [Chip scarcity gave AI clouds room to compete](https://realalpha.app/s/chip-scarcity-gave-ai-clouds-room-to-compete/) (5 min)
- [25:05](https://realalpha.app/v/qJkcjURcAe8/?t=1505) [Open-model inference still has major room to improve](https://realalpha.app/s/open-model-inference-still-has-major-room-to-improve/) (3 min)
- [53:19](https://realalpha.app/v/qJkcjURcAe8/?t=3199) [Nebius lets engineers use AI but keeps accountability human](https://realalpha.app/s/nebius-lets-engineers-use-ai-but-keeps-accountability-human/) (3 min)

## Also worth watching

- [15:36](https://realalpha.app/v/qJkcjURcAe8/?t=936) [AI customers need shared storage built for extreme throughput](https://realalpha.app/s/ai-customers-need-shared-storage-built-for-extreme/) (4 min)
- [10:12](https://realalpha.app/v/qJkcjURcAe8/?t=612) [New Nvidia chips require constant iteration before launch](https://realalpha.app/s/new-nvidia-chips-require-constant-iteration-before-launch/) (5 min)
- [21:47](https://realalpha.app/v/qJkcjURcAe8/?t=1307) [Prefill/decode disaggregation remains a case-by-case tool](https://realalpha.app/s/prefill-decode-disaggregation-remains-a-case-by-case-tool/) (3 min)
- [34:25](https://realalpha.app/v/qJkcjURcAe8/?t=2065) [Execution capacity is the bottleneck across AI infrastructure](https://realalpha.app/s/execution-capacity-is-the-bottleneck-across-ai/) (3 min)
- [57:53](https://realalpha.app/v/qJkcjURcAe8/?t=3473) [Agent reliability may matter more than model size](https://realalpha.app/s/agent-reliability-may-matter-more-than-model-size/) (3 min)

Skip the market overview and origin story, the Tavily acquisition and data-center comparison, and the quickfire and outro.

## Summary

Danila Shtan is CTO of Nebius and says he has been with the company since before it was called Nebius. Nebius separated from Yandex in 2024; Shtan says the company drew on Yandex’s infrastructure team while rebuilding its cloud platform from scratch.

Shtan argues that Nebius competes through software that serves both large interconnected GPU clusters and small self-service customers, while independent AI clouds can meet demand that hyperscalers cannot prioritize. He says open-model inference remains poorly optimized and that 95% of Nebius customers run modified models, making model, kernel and workload-specific optimization important. He describes AI infrastructure bottlenecks as a test of execution across supply, facilities and project management, and says hyperscalers may invest more as chip supply improves. He also says Nebius is cautious about AI agents at work and is focused on improving the reliability of agent outcomes, not just model capability.

### Minute by minute

- [1:21](https://realalpha.app/v/qJkcjURcAe8/?t=81) **Software is the cloud differentiator.** Shtan says Nebius was built to serve both customers using tens of thousands of GPUs in an interconnected cluster and self-service customers renting one GPU for half an hour. He expects AI demand to grow sharply and sees room for independent providers serving companies that need capacity beyond their own infrastructure.
- [6:12](https://realalpha.app/v/qJkcjURcAe8/?t=372) **Nebius grew from Yandex.** Shtan says Nebius separated from Yandex in 2024 and retained an experienced hardware team that had operated infrastructure at scale. The company rewrote its software platform from scratch; in 2024, he says, its software organization had 200 people, compared with roughly 40 to 50 on the hardware team.
- [10:12](https://realalpha.app/v/qJkcjURcAe8/?t=612) **New chips take repeated testing.** Shtan says NVIDIA’s designs can change between early schematics and production samples, so cloud providers must keep adapting. Nebius tests new hardware across virtualization, storage and networking, then reports issues and works with NVIDIA on fixes before exposing the hardware to customers.
- [14:55](https://realalpha.app/v/qJkcjURcAe8/?t=895) **Customers need more than GPUs.** Shtan says training and inference workloads are broadly familiar, but storage throughput has become a major requirement: some customers draw 700 gigabytes per second from shared storage. AI labs often prioritize compute and availability, while enterprise customers also need compliance, security reviews and certifications.
- [19:56](https://realalpha.app/v/qJkcjURcAe8/?t=1196) **Latency tuning sits in software.** Shtan says Nebius aims for a universal hardware baseline rather than building different clusters for each latency target. For managed inference, the team tunes performance through orchestration, models and request routing to meet customer SLAs.
- [21:47](https://realalpha.app/v/qJkcjURcAe8/?t=1307) **Disaggregation is not routine.** Shtan says prefill-and-decode disaggregation is widely discussed but remains difficult to operate and support with reliable SLAs. Nebius has promising prototypes and some manually configured production setups for demanding customers, but does not use it as a default solution.
- [24:53](https://realalpha.app/v/qJkcjURcAe8/?t=1493) **Open-model inference needs work.** Shtan says open-model inference is not a solved problem: default server configurations can produce disappointing speed and cost. He says 95% of Nebius customers modify or fine-tune their models, and sees room for kernel work, compression and pruning tailored to narrower tasks.
- [28:28](https://realalpha.app/v/qJkcjURcAe8/?t=1708) **Nebius invests across optimization.** Shtan says inference improvements can compound, so Nebius invests in areas including kernel optimization and model compression. He says an acquisition earlier in the summer added model-optimization talent, bringing the combined team to tens of people working on these problems.
- [30:07](https://realalpha.app/v/qJkcjURcAe8/?t=1807) **Tavily adds an agent tool.** Shtan says Nebius was impressed by Tavily’s product and go-to-market approach, and believed its infrastructure expertise could help the web search company scale. He says Nebius is exploring ways to combine search with managed inference and other agent-building tools, including sandboxes.
- [32:10](https://realalpha.app/v/qJkcjURcAe8/?t=1930) **Facilities follow Nebius specifications.** Shtan says Nebius controls the internal network and hardware whether a facility is owned, rented or funded through its asset-light model. He says GPUs, components, electricity, facilities, workers and transformers are all constrained, so execution—especially sourcing, scheduling and moving work between sites—is the central challenge.
- [36:59](https://realalpha.app/v/qJkcjURcAe8/?t=2219) **Workloads change cluster packing.** Shtan says new workloads rarely require rebuilding infrastructure at the lowest level. They mainly change how Nebius packs and splits work, collocates workloads and exposes the right tools through its platform.
- [39:13](https://realalpha.app/v/qJkcjURcAe8/?t=2353) **Hyperscalers face competing priorities.** Shtan says hyperscalers compete for limited chips while also reserving capacity for their own teams, which can make AI cloud a relatively small part of their business. He adds that adapting infrastructure built for other workloads requires major investment, and expects hyperscalers to pay more attention if chip shortages ease.
- [43:38](https://realalpha.app/v/qJkcjURcAe8/?t=2618) **Independent AI clouds can persist.** Shtan expects some of the current 300 to 500 AI cloud providers to consolidate or fail, but says there can still be hundreds of providers serving distinct needs. He argues that larger platforms will broaden their offerings without eliminating independent specialists. On OpenAI’s partnership with Baseten, he says it could let users try open models as a lower-cost option while remaining in OpenAI’s ecosystem.
- [52:08](https://realalpha.app/v/qJkcjURcAe8/?t=3128) **Agent outcomes need more than models.** Shtan says AI coding tools have improved sharply, but Nebius lets employees use tokens without per-person limits while monitoring usage and holding engineers responsible for agent-written work; fully autonomous agents cannot touch core services yet. He says he underestimated customer scale, including 30-megawatt facilities and shared storage at 700 gigabytes per second. He is now focused on improving agent outcomes through tools and harnesses, not just larger models.

## Transcript

Unsupervised Learning, hosted by Jacob Effron, with Danila Shtan.

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[0:00](https://realalpha.app/v/qJkcjURcAe8/?t=0) If NVIDIA could have bought, could have produced like 10x the chips, this whole market would have been shaped differently. Because there is not enough capacity for their scale, this is a relatively small business for them. For the hyperscalers, it's not just a matter of like sales, it's also a matter of changing the infrastructure that was built at scale for other kinds of workloads. This is a massive investment on their part that needs to be justified by the size of a potential business. And it's not that there yet. the emergence of nebius and the other hyperscaler competitors these past years has been a fascinating part of the ai story it's something i've really been interested in and so it was a lot of fun to have nebius's cto neela on the podcast because we were able to dig into a bunch of interesting things about the business and how the space has unfolded we talked about nebius's really interesting origin story and the original relationship of the index and how the company came to be we talked about what it's like to be inside one of these companies how nebius works you know, with NVIDIA, what it looks like to get their next generation of chips live.

[1:01](https://realalpha.app/v/qJkcjURcAe8/?t=61) We talked about why the opportunity actually existed and why AWS, GCP, Azure didn't just capture this entire market. And we also talked about Xeniel's view on a bunch of new technologies like DISAG inference, among other things. All in all, just awesome to get to talk with someone who's at the epicenter of the AI infrastructure market about his thoughts on everything about where the space is headed. I think folks will really enjoy it. Without further ado, here he is. I guess I'd love to talk just maybe about the broader, like, AI cloud market, you know, today. And I think that there's a few maybe larger and prominent public, you know, I guess termed neoclouds.

[1:35](https://realalpha.app/v/qJkcjURcAe8/?t=95) And then, you know, it feels like there's a new one getting funded every day. I know you don't like that term, but maybe just talk a little bit about how you think about what differentiates. Like, some of our listeners are probably thinking, like, you know, all that really matters is just access to GPUs. And like, you know, wherever you can get them, you go. Maybe use that as a straw man and tell us why that's not the case. Well, I'm obviously biased because, like, first of all, I'm a software person and our product and business was launched on the idea that software is the mode, right? So your ability to serve the customers that consume, I don't know, tens of thousands of GPUs in one interconnected cluster with the same platform as you serve, like, a self-service customer that rents one GPU for half an hour.

[2:23](https://realalpha.app/v/qJkcjURcAe8/?t=143) was our value prop from the very beginning. So we believe that this is extremely important, being able to build large scale but have the possibility almost like sell it in retail in a sense. This is what we think definitely differentiates us from the market, from everyone on the market. Like a small remark, NeoClouds hate that term with a passion because it started as a distinction as in these are kind of cloud companies that are doing this weird, strange new thing, but then everyone wants it in that market. So when we started, there were like, I don't know, five, seven companies that were in this area that were called NeoClouds.

[3:11](https://realalpha.app/v/qJkcjURcAe8/?t=191) Now, I think it's closer to like three to 500. Yeah. Depending on what you even are ready to call NeoCloud. the scale and we really don't like this stretched umbrella term anymore we are the ai cloud we don't care like let others other guys be neoclouds we are the ai cloud and this is our mental model we are not competing with the neocloud markets we kind of believe we are past that our next competition point is taking on aws and google cloud yes we are way smaller yes our like revenue is like what i think one percent of aws nowadays But we are running fast. We serve a very particular market. We have a bet that this is the market that is going to grow, that is going to explode.

[4:09](https://realalpha.app/v/qJkcjURcAe8/?t=249) So people are concerned that these investments are huge, people put a lot into the infrastructure, et cetera, et cetera, et cetera. My view, if I compare what we see in the market with what was there two years ago, we are barely scratching the surface. I think the overall demand in the next three years will increase tenfold, hundredfold, maybe. So we are at the verge of another explosive growth. Right now, we see these huge facilities, right? But a lot of those are privately owned for internal needs of certain companies, right? So Elon Musk is talking about his Colossus cluster, and it's immense scale.

[4:56](https://realalpha.app/v/qJkcjURcAe8/?t=296) Like, really, it's like real solid, large scale thing. But it's not for the broader market, right? It's for their internal needs, maybe for some partners that they decide. Maybe for Anthropik. Yeah, Anthropik can OpenAI amassing the capacity, but they do it for their own product. And this is like a wholly separate discussion. But these companies will essentially want to be everything. They will want to be the internet and contain everything and every product and every capacity angle within themselves. There is a question of Google, Meta, and Microsoft, who have a lot of capacity but are constantly juggling and struggling between giving it out to the market to rent or giving it out to internal teams.

[5:46](https://realalpha.app/v/qJkcjURcAe8/?t=346) because all of these companies are extremely machine learning, AI enabled. They need a ton of capacity for internal usage, for runtime and for research. So they have to constantly juggle this. So our bet is that there is a niche for independent, as in not affiliated with a large company with a huge internal consumption player that will serve the markets of the meet your players who want to be the large companies. I thought it'd just be interesting for our listeners to talk a little bit about the history of Nebius. I feel like it's a really interesting path that you had to end up here. And so maybe just talk a little bit about how the business came to be.

[6:27](https://realalpha.app/v/qJkcjURcAe8/?t=387) I'll start with the moment I've joined. So I am CTO of the company, but I've been with the company from the very beginning, like since before it was called Nebius. So the new company is established. I think during 2024 also, the deal was finalized. We finally announced our separation from Yandex, no connections to the Russian assets anymore, clean slate, et cetera, et cetera, et cetera. And that was it. People have come to these AI cloud businesses in different ways. And obviously, I feel like the Yandex team was legendarily really, really strong on the engineering side, you know, a lot of folks building iClouds came from, like, crypto or energy or, like, not necessarily the same, you know, maybe technical heritage. To what extent, I guess, was the Yandex experience really relevant? And to what extent was it like, you know, it turns out no one knows how to build a GPU cloud because it's all kind of new and, you know, there's everyone's learning on the fly. First, we had and still have the core team that supports the hardware side of business that came from yandex that had experience operating hardware like infrastructure at scale like at real scale hundreds of thousands millions of servers span over multiple facilities etc etc etc these

[7:50](https://realalpha.app/v/qJkcjURcAe8/?t=470) people like were the core of our build-outs like initial build-outs and expansion and they're still with the company so this definitely helped i think play. We would be way less successful if not for that core team. Yeah. On the software side, it's interesting because it's a two-fold. Like with hardware, it's very straightforward. We needed people with experience. We had people with experience. They started building out from the get-go. On the software side, it was a weird one. First of all, we had to rewrite everything from scratch. So our previous experience was kind of like garbled. Second, we had to grow so fast that by, for example, by the end of 2024, when the new platform launched, like a significant portion of our engineering team, software engineering team was not from Yandex anymore because we had to hire during this period extensively.

[8:45](https://realalpha.app/v/qJkcjURcAe8/?t=525) As a team, we were working on this cloud platform from the initial idea, right? So we kind of just went with it and doubled down on software. We are not doing a managed bare metal. We are not doing a patchwork of third-party services. We want to build a holistic cloud experience, the holistic cloud platform. And we kind of doubled down on that, even with the whole rewriting of the platform, like doing it from scratch, et cetera, et cetera, et cetera. And of course, having a ton of talented engineers on the team, obviously helped just for the scale of it you've mentioned people who came from crypto or came from other avenues of life, they are mostly concentrated on hardware, their idea is you need to buy GPUs, you need them to run and then you'll find the buyer in our case for example in 2024 when our hardware team that actually procured GPUs did the build doubts. We're working on our own servers and chassis, writing firmware and stuff, building the network.

[9:56](https://realalpha.app/v/qJkcjURcAe8/?t=596) That whole team was roughly like 40 to 50 people. The software organization was already 200. Right? So we kind of can certainly think of ourselves as a software-first company. We are building the cloud on the solid foundation of hardware. We're not a hardware-first company. Makes total sense. I guess I'd love to take our listeners inside Nebius, and I feel like there's so many interesting parts of the business, but maybe one place to start would just be, I imagine as you're getting new NVIDIA chips that come in, there must be a bunch of stuff you have to do within the business to get this all up and running.

[10:33](https://realalpha.app/v/qJkcjURcAe8/?t=633) So what does that look like from, I don't know, you're talking to the NVIDIA folks and they're giving you early insight into the next generation of chips to actually getting these live for a set of customers? that is a mess no i mean like in a good way you know like this whole this whole market this whole field is running at such a pace that like everyone is like uh trying to catch up right and NVIDIA despite being like in the lead of this like essentially leading and showing the direction for this market is not the exemption so first NVIDIA folks like way before show you the prototypes of what they want to build in the next like year right they kind of show you the schematics like how it will be like optimized internally where is your like PCI connections how would that look like what is the like future platform would look like then some time later they come to you and said oh like we've optimized things it's now gonna be different you're like all right then they come to you and like oh we have like the first lab samples do we want to have a look yes of course we do want to have a look oh by the way it doesn't look like the stuff you've showed us two months ago oh we've changed stuff because like we found like right because they are so compressed

[11:52](https://realalpha.app/v/qJkcjURcAe8/?t=712) **Danila Shtan:** in time same as they everyone else they are constantly iterating while these chips are being produced.

[11:59](https://realalpha.app/v/qJkcjURcAe8/?t=719) Right? If it's not your, like, you know, there is like an ocean in the consumer market, like Apple iPhone that you see today was actually planned three years ago, and final design more or less was settled like a year ago, and they take the final year to polish it to build this product. Like, AI market is running at such a pace, like, that whatever, like, the announce is, it's indicative. And then teams are constantly iterating. Well, because the architectures and the workflows are changing, right? Right, so I'm sure they're iterating as that. Yes, like they're iterating, like going further and further and further.

[12:35](https://realalpha.app/v/qJkcjURcAe8/?t=755) And NVIDIA is a behemoth now, right? Whatever they do influences the whole market, supply chains, there's everything. So they have to like do that. So at some point you get a sample and it's more or less like what you'll get in production. Then you start building, like you are testing everything. you like incorporate support for the new hardware for the new restrictions or capabilities into the software platform if you figure out a way to isolate workloads from from the chat we are we are cloud we are not providing bare metal hosts right like so we need to split we make sure that our virtualization works that our storage works that our networking works sometimes we are on the bleeding edge of this so we like for some of the generations we were probably the first team like like maybe one of the first teams that actually built like a cluster of a certain size.

[13:32](https://realalpha.app/v/qJkcjURcAe8/?t=812) So we've hit some issues with like with the software that no one has seen before. So we had to report that back to NVIDIA, like propose a patch, get there, like fix from it, make sure that it runs, et cetera, et cetera, et cetera. So like we kind of feel like we are at the forefront of it. But all in all, it's a pretty normalized process. it may sound messy right there are like multiple iterations you go back and forth you're in constant uh contact with like NVIDIA hardware folks with NVIDIA software folks uh you are trying to solve the issues on your end at the same time incorporating into a more complex platform thinking how can you expose that to customers because for example working with uh like your usual black hole chips like you know the like b300s and then transferring that experience to gb300s which i call different architecture in terms of isolation you are talking racks rather than servers you need to find a way how to do maintenance on the rack instead of the server which you just switch off and you move the workloads to another one despite all of that this kind of works all the time and every day you like push it a little bit further. And then in the end, yes, there is a product and the customers use it. And that's it.

[14:55](https://realalpha.app/v/qJkcjURcAe8/?t=895) One thing I think it would be so interesting to get your thoughts on is just the differences in kind of what end customers end up wanting. Like obviously it can be easy to think, okay, everyone just wants to come to you and get similar things. But I imagine you segment your customers or think about differences in what is important to different customers. How do you think about like the key differences in these customer desires today. Hi, I'm Anne Maris and I'm the producer of Unsupervised Learning. Thank you so much for listening to the show. And if you've enjoyed the conversation, please subscribe. Subscribing helps us grow, which helps us continue to lock down some amazing guests. Thank you so much for your listenership and see you next episode. How do you think about like the key differences in these customer desires today?

[15:40](https://realalpha.app/v/qJkcjURcAe8/?t=940) oh it's not that different to be honest because like every customer is uh like its own unique flower right but at the same time workloads in principle are not that different it's either training and you kind of know how training looks like I think. Or inference. And you kind of know how inference looks like. Circling back a couple of years ago, for example, we kind of underestimated the storage component. We thought GPUs are the most important thing, and they are. But alongside the whole training angle, you see a massive demand for storage. And it's not for the amount of like HDDs or VMUs that you can provide to the customer. It's about throughput. It's about latency.

[16:26](https://realalpha.app/v/qJkcjURcAe8/?t=986) In my prior life, pre-AI workloads and infrastructure build-outs, I was talking about gigabytes per second if we were talking about network-attached storage, right? It was like, it's willing.

[16:39](https://realalpha.app/v/qJkcjURcAe8/?t=999) **Danila Shtan:** Right now, we have customers routinely kind of like hitting a common share, like one volume, right, connected to a number of GPU servers with like, I don't know, five, six, seven hundred gigabytes per second of throughput.

[16:55](https://realalpha.app/v/qJkcjURcAe8/?t=1015) this is extreme like two years ago I didn't even think about that and now this is more or less like something we expect at the same time if you are targeting these numbers for the highest demand like the most demanding customers your baseline is so high you kind of can't serve anyone right so if you hit this like use case where a customer and needs 700 gigabytes per second from three petabytes of shared file storage connected to a thousand of servers or like 8,000 GPUs, you kind of know how to serve a massive cohort. So I'd say in terms of like performance or features per se, the customers are not that different.

[17:44](https://realalpha.app/v/qJkcjURcAe8/?t=1064) Then you meet another subset of like what the customers really want because there are definitely different customers in terms of not their scale, not their size, not their workloads, but rather how mature they are or what are their obligations for their clients, right? Because we can talk about like a Neolabs. Like Neolabs routinely have seed rounds in the billions nowadays, right? So most of that budget goes into commute. So you can have like a contract for a billion dollars with the newly founded Neolab, they don't care that much about anything but GPU, storage, performance, availability. This is it.

[18:34](https://realalpha.app/v/qJkcjURcAe8/?t=1114) Oftentimes, they don't even care about the location of those GPUs. They don't care if that's US, Europe, France, or Iceland. Why bother? Internet connection is good, storage is there, GPUs are working, and we are talking about a billion dollar contract, right? At the same time, we now see and start onboarding enterprise customers. These are companies whose contract with us will be smaller, sometimes significantly smaller, but they are more established. They have customers and they have applications for their customers. They have procedures. They need to make sure they're compliant to a number of regulations in multiple countries.

[19:24](https://realalpha.app/v/qJkcjURcAe8/?t=1164) So now we are getting into that whole angle of being the real cloud, you know, like certifications, compliance, security posture checks. Let us look at your source code. How do you solve problem X? How do you protect your customers from attack vector Y? etc etc etc and i'd say if you are talking about differentiated customers this is the most differentiating thing as opposed to what you would expect from like the size or the performance you know obviously i think a lot of people are thinking through this trade-off of like throughput latent or a trade-off of like latency and and kind of cost um and i imagine you have some customers that have workloads that are super latency sensitive and they'll you know do small batches or whatever is required uh to to get to that latency and then you have other customers that are much less latency sensitive maybe background agents or something is that all just you know uh determined on the software side i guess if you have those different customers of like basically how you know uh the batch size that you're running these workloads in or do you actually construct the clusters in any different ways based on those workloads?

[20:31](https://realalpha.app/v/qJkcjURcAe8/?t=1231) No, no, no, no, no. This was the premise from the very beginning. We are building the universal cloud platform, and we need to make sure that our baseline is high enough to fit most of the workloads. Yeah. Then there is the SLA play, right? So customers come, like, for example, for the managed inference part of the business, customers come with the different demands. and then you are in this kind of a benchmark game, right? They want this particular footprint based on this particular traffic pattern fits these SLAs in terms of latency or latency is time to first token or the whole request latency.

[21:14](https://realalpha.app/v/qJkcjURcAe8/?t=1274) And you start optimizing for that. But this is never done on the hardware level because hardware is good enough. This is rarely done on the lower infrastructure level because it's universal and, again, good enough, no compromise for performance, et cetera, et cetera. And you're going further away to the orchestration level, to the model level, way further up the stack, all the way to how exactly does the request from the customer, like the endpoint, gets to the inferencers and what are your performance hits along the way. Yeah, that makes sense. So this is like 100% software. I guess a lot of people have talked to me.

[21:54](https://realalpha.app/v/qJkcjURcAe8/?t=1314) You know, it feels like this DISAG, you know, splitting pre-fill and decode is very in the air these days, and a lot of people talking about that. Like, is that something you guys think about now, or at least how do you think about that kind of in the future?

[22:05](https://realalpha.app/v/qJkcjURcAe8/?t=1325) **Danila Shtan:** I think we are in the stage where everyone's talking about that, and no one's doing that properly.

[22:11](https://realalpha.app/v/qJkcjURcAe8/?t=1331) Because, like, it is a complex problem to solve. And particularly in inference, you have a lot of other problems, like way lower hanging fruits that you can fix address and hit those SLAs. So we are quite cautious about that. We don't have it, for example, as a go-to solution for every customer, for every workload, because like, first, it doesn't make sense. Second, it's really hard to operate and work with, like provide proper SLAs around that is like proving quite difficult as of today. We have some promising results in our like prototype services, but not as of today. At the same time, we have some like disaggregated setups for the most demanding customers in production right now.

[23:01](https://realalpha.app/v/qJkcjURcAe8/?t=1381) It's just more, you know, wide glove manual approach. Yeah. But especially for this newer architectures of like, you know, those gray black walls, whole node that has really fast interconnect, a lot of challenges that you've had before with disaggregated steps are kind of solved if you're able to fit in the footprint of a rack. because, like, this throughput is so much faster, like, interconnect is so much faster, you kind of stop caring and that this is a way of just balancing the perfil and decode nodes to match the particular pattern that you see from the customer. What's your mental model for, like, the workloads that do fit in a rack?

[23:43](https://realalpha.app/v/qJkcjURcAe8/?t=1423) I don't know. It's a very case-by-case. Well, these are definitely significant workloads. Like, we are talking, like, thousands of requests per second, right? We are talking like billions of tokens that flow through. But at the same time, because we are the cloud, can for example like share the rack between two workloads and not be that bothered about that right yeah so for example on the cloud side we for this particular product we only allow to consume it like at track level you cannot order two grace blackwell's uh notes it doesn't make sense yeah right like you'll have to allocate the whole rack and then like structure workloads within it but for example when you're a client of such a solution as we do with managed inference for example you can collocate workloads and you can find a way to do that so in a sense you don't need to fit anything in the rack it's just is there a workload that will benefit from this disbalance like intentional disbalance of pre-filling decode nodes yeah and can you fit more than one into a single rack and how do you solve the packing problem? Yeah, that makes sense. You kind of mentioned there's like lower hanging fruit on the inference side than doing that maybe. What are

[25:05](https://realalpha.app/v/qJkcjURcAe8/?t=1505) some of the lower hanging fruits today that you still think we're not as an industry optimized on? Well, first of all, I think as an industry we didn't optimize shit. So like, pardon my French, but in a sense, I strongly believe that running inference, particularly in the variety of open source models that are available today, is not a solved problem. If you take something, just run with it as it's published in the initial support in some of the open source inference servers run with the novel models, your results will probably be underwhelming. It's not that fast. It's not that cheap, especially with current prices on the infrastructure it's easier to go and use like fable 5.1 astro 6 etc etc etc but because of the open architecture because like of everything that we know as of today you can optimize the way you process those requests you can optimize the way the inference server is configured you'll put a lot of effort in kernel optimizations, sometimes kernel over-optimizations based, like, on the particular pattern that you see in the traffic, right?

[26:25](https://realalpha.app/v/qJkcjURcAe8/?t=1585) There is a lot of underexplored potential in model weights optimization. And I'm not just talking about, like, stuff like speculative decoding, for example. I'm talking about, like, model pruning, selective pruning, compression. because I believe that, and we see that within our customer base,

[26:47](https://realalpha.app/v/qJkcjURcAe8/?t=1607) **Danila Shtan:** 95% of our customers do not run vanilla open source models. These are modified, fine-tuned, et cetera, et cetera, et cetera.

[26:55](https://realalpha.app/v/qJkcjURcAe8/?t=1615) It means that these are the models that start to address a particular use case that the customer has. Particular use case usually means limited domain, scoped domain. Scoped domain usually means that you need to retain frontier intelligence and reasoning for a particular subset of tasks, and you don't care about the rest. And now we are getting somewhere in terms of maybe you don't need such a large model anymore. Maybe you need just like a model that is selectively compressed in a way where it works for a particular task and it doesn't work as good for the rest of it. Now, of course, that doesn't work with like open ended, you know, deep research or broad coding tasks anymore.

[27:48](https://realalpha.app/v/qJkcjURcAe8/?t=1668) But I strongly believe that for real AI adoption, when we are talking about serious enterprise business, when we are talking about like practice. that actually solves problems, you are getting more and more into this specialized agents, limited domains, where you even stop caring about the costs. You care more about predictability of the outcome, and you care about time, as in like real seconds and minutes till you get that outcome. Yeah. And I think there is a lot of potential there, and we will still see a couple of nice breakthroughs there. There's a ton of work around inference optimization, and a lot of that gets done in open source, right, or contributions that share throughout the community.

[28:39](https://realalpha.app/v/qJkcjURcAe8/?t=1719) There's obviously a bunch of work on smaller purpose-built models and compaction and a bunch of these things. Across all these problems, which do impact, you know, how enterprises consume inference, which ones like Nebius should have a solution or offer a product around and which ones you're like, hey, it's great that that exists and we'll benefit from that, but we don't need to be at the forefront. We want to be at the forefront of everything. In a sense, these are additive and even maybe multiplicative, right? Like, if you have a right subset of things, optimizations that you introduce for a particular use case, it means that you can reach the targets easier, better, faster, in a more predictable fashion.

[29:24](https://realalpha.app/v/qJkcjURcAe8/?t=1764) So we invest in all avenues. Like, one of our acquisitions back in, like, the beginning of summer was targeting specifically that. We basically bought the company for the talent in model optimization, model compression, speculative decoding, et cetera, et cetera, combined it with existing talent. So we have a team that is 2x the size. And we have serious manpower, as in tens and tens of people who are working on these problems. We have specialized groups for kernel optimization, specialized groups for model compression problems. because this is, I feel, if you want to talk about running inference as a service at a serious level, this is the only way you can do it nowadays.

[30:11](https://realalpha.app/v/qJkcjURcAe8/?t=1811) And I guess maybe along those lines, you guys have done a few acquisitions, and one of them that caught a few people's eyes was Tavoli in the web search space. Maybe talk through a little bit the rationale for that, how you think about that fitting into the broader platform. So first, we were very impressed by the product and their go-to-market motion. It was almost a year ago now, right? But we felt like we knew how to help them run even faster with our expertise in technology and low-level stuff. Because actually running a web search engine is not a very simple task, neither from the software development perspective nor from the...

[30:56](https://realalpha.app/v/qJkcjURcAe8/?t=1856) A lot of you used to run one. And some of us dabbled a little bit in our previous lives, yes. Right? So this was the premise. Like, we have a great team that is running real fast in terms of, like, market penetration and, in general, go-to-market motion. And we know how to make it complementary with our expertise on the low-level stuff and running large-scale infrastructure and, for example, large-scale web index. It's not exactly embedded into the rest of the offerings that we have now today. We are exploring and we have something cooking for the managed inference combined with agentic tools because, for example, we also develop sandboxes nowadays, right?

[31:42](https://realalpha.app/v/qJkcjURcAe8/?t=1902) So this is another building blocks, one of the building blocks that people use to build their agents. You have several other products that are in the earlier stages of development in that regard, too. Yeah, I mean, it seems that there's obviously a set of tools that folks are going to use to build agents. Build on open source, and it seems like you want to have a bunch of those tools to be able to offer to your end customers. And web search kind of being a key primitive for any long-running agent will consume a decent amount of tokens on just pulling information in. I guess I'd love to switch gears to just the broader industry build-out.

[32:18](https://realalpha.app/v/qJkcjURcAe8/?t=1938) I think there's lots of questions folks are asking about that. I figured one place to start that actually might just be interesting for our listeners is, is to what extent are the data centers that you work with similar? And there's a reference design, and they all kind of like, I don't know, to you it doesn't really matter which one's which. And what are some of the differences in them that actually impact the business of running an AI cloud on top of them? Well, to be frank with you, I have no idea what the walls of the facility are made of. So they're probably different, but they doesn't really help.

[32:51](https://realalpha.app/v/qJkcjURcAe8/?t=1971) But in all seriousness, what's within the walls is something that we fully control we do not rent data centers when someone like have built the like the network on their own or brought some gpu gpus or servers etc if we don't own the facility and we rent it for example we rent well the building electricity but that's it all the internal build out is controlled by us it's like it's done according to our standards so from the software side of things there is literally no difference which data center we're running is it our own facility is it something that is built within a rented facility or for example we've recently announced this so-called asset light model when a third party brings capital in order to set up the facility and GPUs, and we run our platform on top of that, it's going to be the same. These data centers are built according to our, like, fitted out according to our spec. Hardware is something that is familiar to us because, like, unknown hardware is quite hard to adopt and make sure that you provide proper SLAs.

[34:14](https://realalpha.app/v/qJkcjURcAe8/?t=2054) And that's it. So we don't compromise there. We never rent something from people and present it as our own cloud. If you launch a virtual machine in Nebius cloud, you can be sure that it runs on our stack top to bottom. Yeah, makes sense. I mean, I guess there's a lot of dialogue around the bottlenecks to the AI buildup today. And I think people talk about a bunch of different ones, you know, whether it's power or the labor to actually make, you know, data centers, cooling, networking, you know, HBM, GPUs, I guess, as overall. Like, how do you think about what the key bottlenecks are today? Or if you were to kind of rank across all of them, the few that feel most poignant right now?

[34:54](https://realalpha.app/v/qJkcjURcAe8/?t=2094) I'm sorry to disappoint there is one single bottleneck and that's your ability to execute because yes all of those are issues like there are GPU shortage in the market, components shortage nowadays it's CPU shortage in the market, like NVMe shortage in the market there are not enough facilities not enough electricity not enough qualified workforce to help you build it out not enough like small components like network transceivers for example what else is uh not in abundance oh you know like uh those uh large scale uh i'm not sure i like i i know the proper english terminology but like you know those machines that actually convert electricity in place like transformators transformers yeah transformers yeah like do you know the lead times on those these days?

[35:54](https://realalpha.app/v/qJkcjURcAe8/?t=2154) Ooh, years. So, like, everything is a problem. And I think the... A single bottleneck and at the same time, the single mode that you can impose on that is your ability to execute around all these problems and concerns, your ability to source, your ability to cut corners where appropriate, your ability to execute in general, your ability and flexibility to move stuff between facilities. Okay, we're going to move this one from here to there, and then we're going to move it back. Yes, we'll postpone this one, but this means we'll be in time with this one. This is important, et cetera, et cetera, et cetera.

[36:33](https://realalpha.app/v/qJkcjURcAe8/?t=2193) **Danila Shtan:** So I think that in this particular field, an experienced project manager is probably in highest demand I've ever seen throughout my career.

[36:45](https://realalpha.app/v/qJkcjURcAe8/?t=2205) Yeah, I guess the bottleneck is really good project managers. Yeah, exactly. Like, your overall ability to, like, have this large context of, like, problems everywhere in mind and navigate that becomes a mode. Yeah. It's interesting because you obviously have, you know, the hardware itself changing. You have a bunch of workloads changing. Obviously, we have much more long-running agents. We have more post-training going on, you know, people running open-source models. How does that actually impact the way that, you know, the overall, like, I guess the overall build out of the clusters themselves?

[37:23](https://realalpha.app/v/qJkcjURcAe8/?t=2243) To what extent is it like, hey, you know, workload's a workload and, you know, we can run it? No, the workload's a workload. Like, exactly like you said. It is slightly different, but in a sense, you need a high enough baseline to not be, like, constrained by that. And then the rest is just your, like, optimization problem. the way you pack things, the way you split things, right? It's like you don't need whole nodes for some of the workloads. So you need to collocate workloads. You need to have the right primitives. You need to expose those through the platform, et cetera, et cetera, et cetera.

[38:01](https://realalpha.app/v/qJkcjURcAe8/?t=2281) But in a sense, like differentiated workloads almost never like impose any changes on how you build on the lowest level. Yes, of course, like nowadays with like this new architectures like what like like what is it called NVIDIA LPX like the outcome of the grog deal right this like adds to it but in a sense if you have experience running other generations this is just another like 0.5 generation right yes this will be a little bit novel and it's connected in in a weird like compared to the previous one's way and it kind of like there are some restrictions when you can run and not run on that but like there are efficiency gains and this is like just and like a niche local addition to the platform to what to what you offer to the broader market But in general, no, there is way less custom solutions that you would expect from that.

[39:15](https://realalpha.app/v/qJkcjURcAe8/?t=2355) The workload is a workload. Why didn't the hyperscalers move into this market faster or win this market more convincingly early on, in this first chapter at least? First of all, sometimes there is an ocean that they didn't even try. Like, there is a solid, like, solid amount of business that hyperscalers get from out of this market. Like, huge deals, they are very competitive. Sometimes people say, oh, have you seen AWS prices? Well, these are public prices. I assure you, like, for proper deals, they are doing proper prices and they're very competitive and, like, they're very aware of the market.

[39:56](https://realalpha.app/v/qJkcjURcAe8/?t=2396) I think, in a sense, like, there are two angles to that. First, like what I've mentioned before, a lot of these companies need to juggle like their internal consumption and what they can put for like for the broad market. And there is like we are in cheap shortage world, right? So like if NVIDIA could have bought, like could have produced like 10x the chips, probably this whole market would have be shaped differently because they were like Microsoft's and Google's of the world would both buy capacity for themselves and for the broader market to operate. This was part, like simply not enough capacity for their scale, right? Because there is not enough capacity for their scale, this is a relatively small business for them.

[40:47](https://realalpha.app/v/qJkcjURcAe8/?t=2447) AWS revenue is what, like 100 billion or something? I mean, like a business for $2, $3, $4 billion, it's not insignificant by any margin. But compared to what their conservative, large-scale, established businesses, it's not that easy either. But your point is basically they get a chip allocation that is not as large as they'd want. They have to use a large amount of that for training themselves. There's only so big that the cloud business can be, given all that. And yeah, it's interesting to figure out how to prioritize. Do you go after the few folks with large contracts or do you try and get this mid-market enterprise up-and-coming segment?

[41:34](https://realalpha.app/v/qJkcjURcAe8/?t=2494) But I guess you're kind of doomed either way if you only have a certain number of chips. Yes, exactly. And then at the same time, this becomes a relatively small part of your revenue portfolio and you're like, do I even need to allocate resources to that? And that is especially a pressing question because, like, for the hyperscalers, it's not just a matter of, like, sales. It's also a matter of changing their infrastructure that was built at scale for other kinds of workloads. The networking, the, like, throughput performance, the storage performance, by the way, right, that I've mentioned earlier.

[42:17](https://realalpha.app/v/qJkcjURcAe8/?t=2537) **Danila Shtan:** So this requires, if you want to have it universally implemented through the whole stack, this is a massive investment on their part that needs to be justified by the size of a potential business. And it's not that there yet.

[42:33](https://realalpha.app/v/qJkcjURcAe8/?t=2553) I think if we start getting out of this cheap shortage in, I don't know, two, three, five years, maybe, we will definitely see this shift. and they will pay more attention because this will become like increasing a large chunk of business and they will start investing. They will like reorganize and redesign their like internals to be more accommodating to these demands, et cetera, et cetera, et cetera. And then with their whole sales power, they'll go and try to win this market. And the question is whether the novel players like us, for example, will be fast enough to grow and represent like significant competition when they get there.

[43:21](https://realalpha.app/v/qJkcjURcAe8/?t=2601) I'm like, I'm 100% sure that they will pay more attention in the future. Yeah. Our bet is that we'll be fast enough to be in the league with them and fight as equals rather than a small player trying to run in their trails. Makes sense. I mean, what do you think happens to like, you mentioned there's like these like, you know, 300 to 500, like you know neoclouds everyone trying to get into this market like as you kind of you know you mentioned you're kind of thinking the chip shortage comes to an end at some point like as you zoom five years ten years in the future like how many of these businesses are still around or like what is the end state of of this market listen i i i I think the whole like AI consumption is going to grow and it will be like universally used everywhere. Like AI is everywhere. Like right now, like web services are everywhere. AI services will be everywhere. So in a sense, like you have AWS, right? But you also have Hetzner.

[44:26](https://realalpha.app/v/qJkcjURcAe8/?t=2666) You also have DigitalOcean. You also have Linode. You also have like tons and tons and tons of local like hosting providers that are like giving you virtual machines, and I'm experimental, et cetera, et cetera, et cetera. So this market is going to be vast. Will all of the current like 300 to 500 companies survive the next five years? I'm not sure. But I am pretty sure that there will be a place for the very long tail of providers down the line like some of this the current players will consolidate some of them will die off new new players will be launched etc etc etc it's a like it's not a bad market right now it's not a bad business to be in so it will constantly attract capital and attention and investments etc etc etc and i trust that like in five years time there's still going to be like hundreds of players in this market who found the investment and several hundred million dollars and they can offer something to someone in the market. Same as it is with the small scale clouds like DigitalOcean these days, right? DigitalOcean did not die because AWS is growing. You're obviously doing a lot of things in-house like sandboxes and you know, web search and, you know, helping people with their models over time.

[45:50](https://realalpha.app/v/qJkcjURcAe8/?t=2750) And, you know, I think one question a lot of people are asking is, you know, what parts of the AI infrastructure stack end up just being subsumed into these future AI hyperscalers and like live, you know, in just one combined offering? And then what are like the standalone pieces of AI infrastructure that will persist as companies? In a sense, I think it will like, it will kind of mimic what's happened with like the legacy generation of clouds previous generation of clouds right when your growth kind of dictates that you need to start addressing new and like new sectors new markets etc because otherwise your growth is like bounded and you can't like can't show your projections can justify your valuation, et cetera, et cetera, et cetera.

[46:38](https://realalpha.app/v/qJkcjURcAe8/?t=2798) So there will be a natural push for the larger companies to start, like, increasing their portfolio, like, trying to get into new products, into new services, et cetera, et cetera, et cetera. It doesn't mean that there will be a one-size-fits-all solution. Because in a sense, if you're, like, if you're concerned by that, you will, like, we just need to stop doing whatever we are doing and just wait for for like open ai to rule the world entropic to rule the world because these companies will eventually like you do realize that open ai has models runs inference they have their own search that they like they don't exactly expose it but if you're building the agent on their platform it has access to search they have all kinds of like sandboxing, local agents, cloud agents, et cetera, et cetera, et cetera.

[47:36](https://realalpha.app/v/qJkcjURcAe8/?t=2856) They both OpenAI and Entropic are now investing in forward deployed organizations and are going to go and do the work that was traditionally like, I don't know, Deloitte's or like Accenture's piece of pie, right? So they will be everywhere. And I think this is a natural motion for the large company to start getting into a broader markets. but it doesn't mean this is they are going to be the only ones so even nowadays there is a place for independent object storage players all of a sudden right or independent compute players or independence small-scale inference players for that matter independent database players Like Google has BigQuery and whatever, all kinds of data stuff.

[48:31](https://realalpha.app/v/qJkcjURcAe8/?t=2911) But then Snowflake kind of appears from out of nowhere. AWS has all kinds of managed databases, large-scale databases, like Aurora, Redshift, et cetera, et cetera, et cetera. But then MongoDB appears, and it's pretty successful. I can't say it's super large scale, and probably by nowadays standards will never be because nowadays standards for a large company is like trillion plus, right? But like two years ago, it was a very solid and very successful company or someone like ClickHouse appears with their cloud offering, right? Also a massive managed database, an independent company that finds its niche and works it real well.

[49:13](https://realalpha.app/v/qJkcjURcAe8/?t=2953) So I think like larger company, larger clouds will grow. They will have products in every sector and will be pretty successful at selling those because of bundling, because of their sales power, et cetera, et cetera, et cetera. But there will always be a place for niche players. And because the market is huge, niche doesn't mean small. Niche doesn't mean small scale or low revenue or low valuation. This can be solid companies. Yeah. I mean, it's interesting you mentioned, like, you know, OpenAI Anthropic having this kind of broad suite of offerings, too. I mean, in many ways, they're obviously building their own AI clouds.

[49:50](https://realalpha.app/v/qJkcjURcAe8/?t=2990) And I thought one thing that was interesting recently was, you know, obviously a limitation of those AI clouds versus others is the, you know, having just one model family, right, to be able to call from. But I thought it was really interesting that OpenAI announced that partnership with Base 10. I'm curious what your take was on that. I mean, is that them just trying to get into this general, you know, hyperscalar game across models? In a sense, we see a lot of push and concerns about people claiming that OpenAI models or entropic models are too expensive. Everyone is puzzled or surprised by the bills they get at the end of the month.

[50:24](https://realalpha.app/v/qJkcjURcAe8/?t=3024) So this might be an answer to that. Well, if you're a global, go try OpenModels. At the same time, OpenModels are still behind the closed models, right? However good those models are, they're still lagging behind. And giving people an option to try, like, lower scale with the ability to switch back to the top frontier model might also be a good incentive. Anyway, I think OpenAI is doing a very interesting move with that partnership, and I like it really much because this kind of shows flexibility. right they're not certain so like they're exploring different options different things that they can or cannot do like looking for different options to like uh so but i i think in a sense this all plays into one single thing like open ai wants same as in tropic uh they want to be one single like touch point like they want to be one single thing for everything even that is like, okay, but sometimes I want to use open source models within my codex-controlled environment, by the way, right?

[51:45](https://realalpha.app/v/qJkcjURcAe8/?t=3105) So, like, well, good for them. But if the company, the person, the user is still within your ecosystem, you still have a lot of options to steer them, to upsell them, to introduce them to new services. et cetera, et cetera, et cetera. So if that alleviates one of the concerns, so be it, why not? Well, I always like to end these interviews with some overly broad questions that I stuff into the very end. And so if you'll humor me, I guess the first question is just, what's one thing that you've changed your mind on in AI in the last year? In the last year? A lot has happened in the last year. No, the obvious answer would be software engineering.

[52:28](https://realalpha.app/v/qJkcjURcAe8/?t=3148) like a lot of people forget but a year ago we lived in a totally different world yeah it's like dog years in the ai world way less capable yes way less capable in terms of the model capabilities and in terms of how good the modern harnesses and tools are it's like uh the ai coding because like a year ago it was total bullshit right now it's it's a solid part of like your everyday tooling. But for example, if you're not talking about the industry trends or like talking about more like my personal thing, I've started relying more on the like AI enabled like local agents and harnesses to afford a part of my cognitive load, to be honest.

[53:18](https://realalpha.app/v/qJkcjURcAe8/?t=3198) I mean, obviously, you run a really large technical organization. And I think everyone's trying to figure out, you know, I mean, obviously, the models only recently got really good on the coding side. And so everyone's trying to figure out what it means to build teams given the new model capabilities and run an organization. I'm curious where you've settled and how you thought about that. Cautiously, conservatively. I'm very concerned that modern models and the attractive user experience that they provide, like model agents, modern models, etc etc etc is something that will lead us to two issues you like you see slop like being revealed in like in chats in merge requests all over and i like i try to paste that and i try to like be very observant and vigilant about that so for example like we don't restrict talking usage like people can use whatever they want our budget is quite significant we don't impose like per person limitations or per department limitations but the at the same point we try to like look at the usage uh figure out what it is used for look at the most ineffective ways people

[54:47](https://realalpha.app/v/qJkcjURcAe8/?t=3287) **Danila Shtan:** use it. We constructed a set of policies that doubles down on the idea that whatever you've done with the agent, it's still your result. It's your responsibility. It's your pull request. It's your code. It's you who is on call. It's you who's going to wake up in the middle of the night

[55:11](https://realalpha.app/v/qJkcjURcAe8/?t=3311) if something goes awry, et cetera, et cetera, et cetera. And if something was created by a completely autonomous agent it is marked like that uh completely autonomous agents will not touch like the core services for a while because like this is like this is like out of bounds like sometimes people are surprised that like i work in a company that's kind of participates in this whole ai revolution and i'm very conservative but at the same time like i've seen how this sources we've made, and I'll be cautious. What's the biggest mistake you feel like you've made in building Nevea so far? Obviously, a lot's gone right, but I'm curious, something that in retrospect, you wish you guys had done differently.

[55:56](https://realalpha.app/v/qJkcjURcAe8/?t=3356) It might sound funny these days, but we underestimated the... scale like and it's not about the growth of the company we were pretty optimistic from the very beginning but uh i personally that was like my strong bias for the first like significant part i underestimated the scale that a single client operates at nowadays because like five years ago if you're talking about a 30 megawatt data center it was a serious facility it hosted like hundreds of thousands of servers like run glorious gloriously large foreclose by now 30 megawatts sometimes it is not enough for a single client which i'm still getting used to right the storage i've mentioned right like four years ago when you were talking about network attached storage you were talking about like hundreds of megabytes maybe gigabytes per second of throughput. Right now, some of our clients routinely fetch, I don't know, seven hundreds of gigabytes per second from a single shared file system.

[57:09](https://realalpha.app/v/qJkcjURcAe8/?t=3429) I'm still, I underestimated it in the beginning, and I'm still getting used to this scale. I'm an infrastructure person. I've worked and I've seen these numbers throughout my career multiple times. And all of a sudden and you add a couple of zeros to the end and call it a new norm. Yeah. Still puzzling. That's crazy. And I guess lastly, there's obviously a lot of unsettled questions in the AI field today that lots of people are trying to figure out where the world goes. I'm curious what you're paying attention to and I guess with a lens toward if it goes one way or another, it would really change the strategy for Nebius as a whole.

[57:52](https://realalpha.app/v/qJkcjURcAe8/?t=3472) What are you most paying attention to on these unsettled questions of where the AI world's going? Well, I stopped caring about the new models being larger or smarter. I just accepted that it's a given. I'm kind of trying to look out for new novel architectures and approaches, but I don't see any as of today. I hope somewhere deep down in a stealth lab, something new is cooking but for now i frankly don't see anything that like is a thing that i would be even like would place even a small bet on but i'm more and more interested in the whole outcome optimization part on the agent side right because like we are used to talking about models and size of models and families of models, et cetera, et cetera, et cetera.

[58:51](https://realalpha.app/v/qJkcjURcAe8/?t=3531) But now that we've shifted to this multi-turn, long-running agents that we are talking not about requests, but rather outcomes, companies are talking about bidding per outcome, right? I'm very interested in how that part will shape up. And we have like a small research group internally with the neighbors that's working on this problem, actually. is like, how do you increase the chances of getting to that outcome? Not by increasing the model or changing the model. That is, of course, part of the discussion. But I believe there is a lot of optimization in the part where you introduce more sophisticated or maybe more simplified tools.

[59:31](https://realalpha.app/v/qJkcjURcAe8/?t=3571) You introduce the harnesses that start to evolve according to a particular task. And this kind of gets us back to significant CPU consumption because a lot of modern AI workloads actually go back to becoming code. It's codified, it's very formal. You see quality gates, you see sophisticated tools, et cetera, et cetera, et cetera. And sometimes your agent works for mince and mince and mince. without even issuing a backend request to an LLM because the workloads start to sit on this part. And this is the part that really interests me

[1:00:09](https://realalpha.app/v/qJkcjURcAe8/?t=3609) **Danila Shtan:** because I think a lot of future AI adoption lies in how reliable your agentic scenarios are and how reliably you reach those outcomes.

[1:00:22](https://realalpha.app/v/qJkcjURcAe8/?t=3622) And I think the whole optimization loop outside the model is an important part of that. 100%. Well, this has been an awesome conversation. Really appreciate you coming on the show. And yeah, thanks so much for coming on. Thank you for having me. I'm Jacob Effron, and this has been Unsupervised Learning, a podcast where I get to talk to the smartest people in AI and ask them tons of questions about what's happening with models and what it means for businesses in the world. As I hope is clear, I have a ton of fun doing this. It's a nights and weekends project, in addition to my day job as an investor at Redpoint.

[1:00:53](https://realalpha.app/v/qJkcjURcAe8/?t=3653) But our ability to get these incredible guests on really comes from folks like you subscribing to the podcast, sharing it with friends. It's really what ultimately makes this whole thing work. And so please consider doing that. And thank you so much for your support and listening. We'll see you next episode.