Real Alpha
Watch mode

Sourcery · 3 min · Oct 12, 2026

Long-context agents make local hardware cost-competitive with data centers

Aravind Srinivas

Founder & CEO, Perplexity

Held research roles at OpenAI and Google DeepMind before co-founding the AI search engine Perplexity in 2022.

  • Molly O'SheaHost · Founder, Sourcery

    Ex-Upfront Ventures investor; has interviewed Alex Karp and Palmer Luckey.

About this segment

Srinivas explains that chat is cheaper in the data center, but as context grows past about 100k tokens the cloud advantage shrinks to roughly 3x or 1x. That is why Perplexity bets on local hardware plus data centers.

From “Perplexity CEO: SMBs Could Drive the Next Economic Boom”, Oct 12, 2026. Watch the full episode

“as context expands, and as individual user has a lot of personal context, the context window increase dominates the batch size.”
Aravind Srinivasat 30:08 · Sourcery30:08

Takeaways

  1. Chat inference is about 30x more expensive on local hardware
  2. At 100k+ context, the data center advantage falls to about 3x or 1x
  3. Data center build-outs are bottlenecked by permits, power and interconnection queues
Watch 3:28 from 29:26

In this episode

Full episode

  1. 29:26Long-context agents make local hardware cost-competitive with data centers3:28

  2. 26:26AI products compound through recursive usage, not network effects1:57Up next
  3. 42:23The real bottleneck is deciding what to build, not writing code2:14

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

  1. 29:30

    And also talk through maybe the partnerships associated with them. Sure. So as a company, we believe that we should be orchestrating across everything in AI. And AI is a five-layer cake, as Jensen described it. And the chip layer is very important, too. and most of AI inference runs on the cloud today, the data center. There's a reason for that. Batch inference gets you high throughput. And a lot of repeatable workloads with small context windows benefits from batch inference.

  2. 30:07

    That's just the nature of how caching works. But as context expands, and as individual user has a lot of personal context, the context window increase dominates the batch size. And your context is unique to you. When everybody wants personal agents or every business wants its own personal agents that's always on and working 24 seven and running a lot of automations, the context window matters even more than the model weights in the memory.

  3. 30:39

    And so, for example, chat is best suited for data center inference. It would be 30x more expensive to do that on local hardware. But as things get more and more into the long context trajectories, especially like 100k context and so on, the data center only buys you like 3x or even like close to 1x over running it on local hardware as far as like cost efficiency is concerned. And so that's when local hardware starts to matter.

  4. 31:11

    And couple this fact with the fact that the biggest bottleneck to diffusing AI right now to more people who want more agents is the data center build-outs, the long interconnection queues, the permits, people protesting against the build-outs, the gigawatt requirements they have, the need for them to have their own substations and gas turbines to do that. It's just very slow and like very problematic.

  5. 31:42

    Also consider that there are like millions, hundreds of millions of devices that we already have that can potentially run models up to like 100 billion parameters even in future and can be plugged into our own power, which is already running on the grid. If you just multiply 50 watts per Apple Silicon and 100 million Apple Silicon, there's already gigawatts of power. Of course, it's envelope math.

  6. 32:10

    It's not that always our Macs are plugged in for AI and so on. I get that. But there's a lot of hygienic workloads that can run uniquely to you that we're not even tapping into yet. Think of the data center like a bus. Shared ride is obviously cheaper. But that didn't stop people from wanting to have their own car. and having your own car is equivalent of having your own local hardware. And you're going to have both.

  7. 32:36

    Just like how sometimes you might want, like, the frontier to do repetitive tasks that are very, very reliable. But a lot of personal contacts and long-running agents where the context and the harness matters even more than the model,