Real Alpha
Watch modeSubscribed

Minus One · 3 min · Dec 13, 2024

AI agents need to leave chat and enter existing workflows

Alexandr Wang says model performance drops across multiple turns, but the near-term agent bottleneck is product design. He sees a major startup opportunity in finding the right agent experience.

Alexandr Wang

Alexandr Wang is founder and CEO of Scale AI.

Dropped out of MIT to co-found Scale AI in 2016; Meta invested $14.3B in it in 2025 and hired him to lead Superintelligence Labs.

  • Aditya Agarwal, Host · General partner, South Park Commons

    One of Facebook's first engineers, he helped build its first search engine; later CTO of Dropbox.

About this segment

Alexandr Wang says model performance drops across multiple turns, but the near-term agent bottleneck is product design. He sees a major startup opportunity in finding the right agent experience.

From “Scale AI & the Startup Hunger Games with Alexandr Wang”, Dec 13, 2024. Watch the full episode

Answers “Are AI agents ready for real work?” See all 50 answers

“breaking the models out of the chat paradigm into something that's a little bit more baked into the fundamental workflows.”
Alexandr Wangat 29:48 · Minus One29:48

Takeaways

  1. Model reliability falls as tasks require more turns.
  2. Wang sees product design as the key agent bottleneck.
  3. Embedding models in workflows could unlock consumer use.

In this episode

Full episode
  1. 34:31AI founders need to separate conviction from industry hype4:18
  2. 42:08AI startups can win by focusing where giants cannot3:13
  3. 30:15Model progress now depends on specialized human data3:55
  4. 20:02Countries will compete to adopt US or Chinese AI stacks4:13

  5. 26:53AI agents need to leave chat and enter existing workflows3:25

  6. 12:59Scale rewards economic impact over technical difficulty4:50Up next
  7. 49:45Enterprise startups should budget time for bureaucracy3:31
  8. 10:42Enterprise sellers must shape perception as well as reality2:20

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

  1. 26:55

    seen a lot of, um, kind of like excitement about agentic AI agents kind of means everything and also means nothing right now. Um, what, what do you make of it? Like, what do you, how do you think in 2025 that kind of plays out both from a consumer perspective, but then maybe also, I think it's a little bit clearer what it means maybe for companies but like for consumers i would still argue that i i mean i haven't used an agent to kind of do anything interesting for me yet uh well i haven't really used one in the company sense either but i'm curious what do you make of that yeah i think there's i think where we are with agents so where we are with models overall is that they're um they're quite good in within one turn you know with one prompt response they perform pretty well and then the performance just

  2. 27:48

    goes down a cliff as you increase the number of turns um and this is kind of like i mean nobody will really go out and say it but this is really this is the reality of where we are so where they work really well it's kind of like the Google use case where you ask a query and kind of look at the answer, where they work really poorly is if you actually need to like work with the model and do something more complicated, you know, they just kind of fall off a cliff.

  3. 28:11

    And so I think there's kind of two things. I think the first is the industry is gonna, like the model companies are gonna have to work very hard to improve the reliability with greater numbers of turns and ultimately get these models to have some greater level of internal coherence and and ultimately sort of you know be more like entities right now models don't even know what they don't know and so they kind of and you know in general you're sort of like you know in interacting the model on multiple terms you're sort of like it's like this machine that you know is statistically more likely to be correct than incorrect but it's sort of it's not like a it's not like an entity with any theory of mind that you're interacting with um so i think that'll that'll have to change but i i actually think that like for a lot of for for this like watershed agents moment um the

  4. 29:04

    biggest blocker is just um the product design yeah and i think the like you know there is the models are good enough already for there to be some agent product or agent kind of um product experience that actually will will be pretty mind-blowing and pretty great for a lot of people um i think for the people in this room it's probably not obvious because you know maybe we play with models all the time and so we know that they're actually really good at a lot of things but most people don't even know that the models are that good and so cursor is like a good example of this like i think most engineers didn't actually know that the models were super good and you put into cursor and all of a sudden it's like a part of their workflow it's much easier to use So I think that kind of moment will happen for consumers.

  5. 29:48

    And I think it really is just breaking the models out of the chat paradigm into something that's a little bit more baked into the fundamental workflows. I think this is one of the big, this is like the biggest startup opportunity in 2025, if I had a guess, is like actually iterating to find the right agent modality. I agree. Great. Maybe let's move on to talking about, obviously, you know, everybody is talking about