Real Alpha
Watch modeSubscribed

The Cognitive Revolution · 3 min · Aug 10, 2026

Lindy cut its agent error rate eightfold in a week

Flo Crivello says Lindy’s self-improvement loop cut its error rate eightfold in its first week. It’s a concrete example of improving an agent in production without waiting for a stronger base model.

FC

Flo Crivello

Flo Crivello is founder and CEO of Lindy.

  • Nathan Labenz, Host · Host, The Cognitive Revolution

    Founded AI video company Waymark and red-teamed GPT-4 before its release.

About this segment

Flo Crivello says Lindy’s self-improvement loop cut its error rate eightfold in its first week. It’s a concrete example of improving an agent in production without waiting for a stronger base model.

From “Lindy Teammate: Flo Crivello on Multiplayer Agents, Memory & Why He'd Ban the Chinese Models He Uses”, Aug 10, 2026. Watch the full episode

“It went down by 8x, the error rate.”
FCFlo Crivelloat 24:45 · The Cognitive Revolution24:45

Takeaways

  1. Lindy’s self-improvement loop cut error rate by 8x in week one.
  2. The team tracks error rates as the loop improves.
  3. Reliability work can improve performance after launch.

In this episode

Full episode
  1. 27:43Context buckets keep huge histories searchable5:54
  2. 53:11Lindy uses its AI teammate to optimize its own CI7:10
  3. 1:41:42Flo wants restrictions on Chinese frontier models7:48

  4. 22:04Lindy cut its agent error rate eightfold in a week2:54

  5. 14:58Shared memory can be controlled with editable text instructions4:46Up next
  6. 40:45Memory retrieval can improve from logged questions3:00
  7. 10:59The memory agent learns which Slack channels to ignore3:49
  8. 1:21:00A date validator can catch scheduling errors before execution6:14

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

  1. 22:11

    What has been hard about getting multiplayer to work in an intuitive way that might be unobvious to somebody who hasn't been through the slog himself? Yeah, I actually think it's been the scaffold, the context management, like this piece, like the context buildup. It's in a way inspired by Carpathy's auto-wiki idea. And so we've had to do a lot of work around context management because once you talk to an AI employee, it's actually quite unlike just talking to like a chat GPT or a cloud because you actually expect it to keep a very rich representation of its past context and of its past memories.

  2. 22:51

    You really expect a level of consistency and coherence out of an AI employee that you don't expect out of your cloud. Like cloud sort of like, it's cute. Sometimes it plugs like previous memories about you in your chat, But you don't really do heavy work on an ongoing basis with it in the same way that you do with an employee. So managing all of that context, so both the memory agent that builds up the context, that's millions and millions and millions of tokens per user.

  3. 23:17

    And then the core agent that actually uses that context and how does it retrieve the right information at runtime. This has been a major, major, major challenge. Frankly, I've sometimes been telling the team, and I can't imagine we're the only company thinking that. I sometimes feel like we should be publishing. because I think we're doing stuff that's like seriously state-of-the-art and it's quite frequently that we do stuff and like three to six months later, we see a paper come out and blow up about that thing.

  4. 23:43

    There was one time when literally the paper was named what we had called the thing internally because it was obvious. And so I think like context and memory management has been a really, really big part of the challenge. Reliability is always another part of the challenge, right? Like you want... to make your model work. And so we've worked quite a bit on reliability. Like we've called it like a validator.

  5. 24:07

    It's basically a sort of, it's an LLM as a judge that triggers like multiple times during the tasks, but it's modular. So you have multiple LLMs as a judge fan out and it's sort of counsel. And then they talk to each other and they decide what to do. So that's been one. The self-improvement loop, I think like since GPT-4 models have become so capable that now you can actually have self-improvement loops.

  6. 24:29

    And so, you know, we're not the first ones to talk about it with no exception. You know, Lindy is now self-improving. So we can literally see a curve of error rate go down and to the right. You know, that's the direction you want to see error rate go down. It went down by like literally within the first week of us putting the self-improvement loop online, which was like two months ago or something.

  7. 24:52

    It went down by 8x, the error rate.