Guest
Ben Hylak
Co-founder & CTO, Raindrop
Built Raindrop’s tools to detect, investigate, and fix failures in production AI agents.
Follow Ben Hylak
We’ll email you each new moment, one email a day at most. Plus the best of every big interview, Wednesday and Saturday.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
It’s free.
Real Alpha has cut 4 moments of Ben Hylak (Co-founder & CTO, Raindrop) talking, from an interview on MTS published Nov 8, 2026.
What Ben Hylak said
- 9:07
Simulations replay production traces to test agent changes
“most teams right now are not able to iterate on their agent in a meaningful way without shipping to production.”
at 10:21 - 1:29
More capable agents make failures harder to detect and costlier
“as models were getting better this problem was getting worse”
at 1:47 - 4:52
Agent memory failures need separate storage and retrieval metrics
“if you actually want to improve your agent you need to understand if it's from storage or retrieval”
at 6:36 - 12:53
Frontier models can be worse at adapting their tone
“i do think that frontier models are actually worse at these things than earlier models”
at 14:03
- 9:07
Is this you? Add the badge to your site.
