Free with a sign-in.
The best 10 min of 23
Frontier AI Models Have Gotten Worse at Writing | Ben Hylak
Ben Hylak says more capable AI agents are harder to monitor, and teams need production-trace simulations to test changes before they ship.
MTS · Theo Jaffee & Sophia Dew
- Ben Hylak is co-founder and CTO of Raindrop.
The best minutes
- 9:07Simulations replay production traces to test agent changes3 min
- 1:29More capable agents make failures harder to detect and costlier4 min
- 4:52Agent memory failures need separate storage and retrieval metrics3 min
Also worth watching
Skip the career-history and company-mission discussion, plus the closing sponsor reads.
Full analysis · 5 min read
Real Alpha · Nov 8
Ben Hylak says frontier models can write worse
- Hylak says frontier models can write worse than smaller models, especially in tone.
- He offers a mechanism, but no controlled comparison showing which model writes better.
- Teams should test voice and refusals separately from task success before switching models.
Free with an account
Read the rest of the analysis
6 sections, the 6 other voices it tests Hylak against, and what it means for you. Every episode we cut gets one.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
Signed up before? The same buttons sign you in. The episode, its moments and the transcript stay free.
Frontier tuning can weaken tone and writing
Hylak says frontier models can be worse at tone and writing than smaller models.
His explanation is that reinforcement learning from human feedback can push models toward a specific tone and refusal style.
He also said those refusals can help protect a personal agent when someone else tries to connect.
That is a tradeoff, not a ranking of overall model capability.
The episode gives no controlled writing test, sample set, or side-by-side model comparison.
Hylak also said models are optimized for code rather than human interaction, but offered no measure of that difference.
So teams should not infer that smaller models are generally better from this account.
Tulsee Doshi on Lenny's Podcast in November 2026 also described model communication as part of product UX.
Her point supports treating voice as a product behavior, not a proxy for task performance.
“frontier models are actually worse at these things”
Our read
Voice, refusal behavior, and task success are separate dimensions. Teams need separate measures before deciding whether a model change improves their product.
More capable agents make failures harder to observe
Hylak's operational point is that more capable agents take on more complex work and gain access to consequential tools.
That combination makes behavior harder to inspect, test, and keep within intended bounds.
For example, a chatbot can give a wrong answer, while an agent can send a bad email.
The mistake then has an external effect, not just a poor response on screen.
Hylak said Raindrop works with Fortune 50 healthcare customers, where he sees higher stakes.
That supports monitoring, but not identical controls for every agent.
An agent drafting email and one used in healthcare differ in access and acceptable error.
Ethan Mollick on Big Technology Podcast in October 2026 described agents accessing private data and sending messages outside.
That makes the permission boundary concrete, though it does not establish how often such failures occur.
“as models were getting better this problem was getting worse”
Our read
More capability raises the value of observability when it expands what an agent can do. The right controls still depend on the agent's permissions and setting.
Bad actions often expose an intent gap
Hylak treats minor tool misuse and serious harm as related problems, not separate categories.
He described agents pursuing a goal without understanding the human's intended limits.
His example was an agent under pressure to solve a cybersecurity evaluation by any means.
For example, a request to finish a security task may not authorize every possible route.
That distinction matters because a system can follow a stated goal while violating its user's expectations.
Teams can define forbidden actions and test whether agents respect those limits under pressure.
Monitoring can surface violations, but it does not itself guarantee that an agent understands intent.
Hylak said companies have an incentive to pay for monitoring because it can improve products and reduce liability.
Raindrop sells agent monitoring, so that business case comes from a company with a direct interest in it.
“understand your intention and, like, the bounds of that”
Our read
Intent is difficult to reduce to one score. The practical work is to define limits, test them, and monitor for violations.
Teams need cause-specific metrics before changing models
Memory failures show why one overall success rate may not tell a team what to fix.
An agent may store a fact badly, or retrieval may fail to find it later.
Those causes need different interventions, so combining them can hide whether a change helped.
Hylak said storage changes may not repair the previous three or six months of production history.
Take Garry Tan's Gbrain example on Lightcone in October 2026, which uses vector search and graph structure to retrieve context.
That example makes memory look like an indexing and retrieval problem, not simply a larger context window.
On Latent Space in November 2026, Zach Lloyd described Warp scoring traces for redundant tests.
That offers another example of turning agent runs into named failure categories.
But it does not prove one taxonomy fits every team.
Hylak said desired behavior varies across logistics, law, healthcare, and personal agents.
“you need to understand if it's from storage or retrieval”
Garry Tan · Lightcone · Oct 30Agent memory turns into a retrieval problem
Zach Lloyd · Latent Space · Nov 3Agent traces can be scored against team-specific failure modes
Our read
Granular metrics make iteration more useful only when categories match what users count as failure. The metric should identify a fix, not merely label a problem.
Production replay tests changes against real workflows
Hylak's simulation approach replays past production situations instead of relying only on hand-written tests.
That matters because an agent's behavior depends on tools, skills, flags, memory, and assembled context.
A prompt playground cannot represent all those moving parts.
He described recreating interactions involving email, calendars, or DoorDash.
The simulation can use real data when available and synthetic data otherwise.
Teams can then compare model, prompt, or tool changes before shipping.
This resembles regression testing against lived workflows, rather than proving general capability.
Ethan Elasky on MTS in October 2026 described recreating incidents with fake agents on a wiki.
That also tests whether failures recur, but does not describe Raindrop's connected-service reconstruction.
Hylak is presenting a Raindrop feature, and the episode gives no measured reduction in incidents or comparison with standard evaluations.
“we want to show you what your change is going to change”
Our read
Replay makes change review more concrete. The episode explains the mechanism, not the size of its effect or how reliably it predicts production behavior.
Replay results depend on available production data
A replay is only as representative as the history and connected services available to rebuild it.
Synthetic data can make a test possible, but teams still need to know what was generated or missing.
Jacob Lauritzen, Legora's CTO, said on Latent Space in November 2026 that client confidentiality prevents inspecting client traces.
His case does not critique Raindrop, but it shows why trace access cannot be assumed.
When teams cannot use raw interactions, they need replay designs that preserve confidentiality.
They can record which fields came from history, which were generated, and which were unavailable.
That makes gaps visible instead of treating every replay result as equally grounded.
Jacob Lauritzen · Latent Space · Nov 2Client confidentiality forces Legora to infer product failures
Our read
A passing replay is only as informative as its evidence. Teams should distinguish historical inputs from synthetic or unavailable data.
What it means for you
- Split memory failures. Report storage errors separately from retrieval misses. That makes the next fix and its verification target clearer.
- Replay consequential changes. Test a model, prompt, or tool change against relevant production histories before release. Include workflows where the agent can take consequential actions.
- Track replay coverage. Record which test fields are historical, synthetic, or unavailable. This matters when traces contain sensitive information.
- Measure voice separately. Compare tone and refusal behavior alongside task success when evaluating a model change. A stronger refusal score alone will not show whether the interaction works for intended users.
What to watch
- Whether teams can separate storage failures from retrieval failures in production.
- Whether replay results predict behavior after model, prompt, and tool changes.
- Whether stronger refusal behavior comes with worse tone or writing for intended users.
Send this to product and engineering leads shipping agents with write access.
Watch the minute where Hylak says frontier models can write worse
Written by Real Alpha from the episode's transcript and the other moments it links, then checked against the transcript. Quotes are the speaker's exact words.
Minute by minute
- 0:00Frontier models write worse
Hylak says frontier models can be worse at tone and writing than earlier or smaller models. He attributes this partly to reinforcement learning from human feedback shaping a specific tone and calibrated refusals, which he says can still help with security.
- 1:29More capable agents are harder to monitor
Hylak says Raindrop grew from the team's work on coding agents. As agents become more capable, he argues, they also become more complex and harder to understand, observe and test; their use in settings such as Fortune 50 healthcare raises the stakes of failures.
- 2:58Misuse can expose alignment gaps
Hylak calls Raindrop's approach capitalism-aligned safety: monitoring should improve products and reduce liability, giving companies a reason to pay for it. He says many harmful actions happen when agents follow pressure to complete a task without understanding human intent, citing an agent pushed to solve cybersecurity evaluations by any means.
- 4:52Model switches change quality
Hylak says each model switch brings a new set of problems. He points to differences in writing quality between Anthropic and OpenAI models, and says teams are also trying to reduce costs and route requests to the right model.
- 6:04Memory failures need finer classification
Hylak says teams need to distinguish memory storage problems from retrieval problems, rather than just track complaints. Storage changes are difficult to assess because they may not fix memories created during the previous three or six months. He says teams need a reliable verification loop to measure whether changes help.
- 8:14Monitoring must adapt to each customer
Raindrop works with customers in logistics, law, healthcare and personal agents, and Hylak says each has different standards for good and bad behavior. He says the product surfaces candidate issues for customer feedback and adapts what it detects.
- 9:07Agents need world-level simulations
Hylak says agents are no longer defined by a prompt alone: their behavior also depends on skills, tools, feature flags, user plans and memory. He says most teams cannot meaningfully iterate without shipping to production, even though they need ways to prevent problems before they reach users.
- 11:01Production traces can test changes
Hylak argues that writing a test case for every capability is a poor fit for agents expected to handle situations teams did not anticipate. Raindrop replays past production interactions by simulating connected services such as email, calendars and DoorDash, using real data when available and synthetic data. Teams can compare changes to models, prompts or tools.
- 12:53Agents often miss human intent
Hylak says users often tell agents what they want, but the agents do something different, making the gap partly a communication problem. He expects better alignment to come partly from product design. He also says frontier models can adapt poorly to tone and write worse than earlier or smaller models, though their refusals can help security.
- 14:55Robotics and design shaped his path
Hylak says he studied robotics and became interested in avionics, human-computer interaction with robots and AR. He spent almost four years as a designer at Apple working on Vision OS.
- 16:22Raindrop serves high-stakes and individual use
Hylak describes Raindrop's mission as protecting the world from both large and small agent failures, noting that issues can affect human safety. The company initially focused on customers handling millions of messages or events a day; he sees simulations as a step toward usefulness for individual developers too.
- 18:22Simulations support recursive improvement
Hylak says Raindrop uses simulations internally to make changes and verify fixes. He argues teams need to close the loop from finding an issue to confirming and shipping a fix, and says simulations give them confidence before switching models rather than testing the change in production first.
Published Nov 8, 2026.
Transcript · 19 min read
MTS, hosted by Theo Jaffee and Sophia Dew, with Ben Hylak.
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
I do think that frontier models are actually worse at these things. Than earlier models? Well, even just like smaller models. Like I think they're just like really RLHF to have a very specific tone, to have like very well calibrated refusals, like all of these sorts of things, which is genuinely useful. Like if you're making a personal agent or something, it just like, you know, if someone else tries even connecting to that, it just has a pretty good intuition on what to refuse kind of broadly. so it can be useful from like a security standpoint but like yeah their tone is just and their writing quality is like bad all right welcome back guys um i'm live with ben hilak who is the co-founder and cto of raindrop which monitors ai agents in production in production and flags failures like hallucinated answers tool misuse or behavior shifts after a model upgrade usually before users notice so welcome ben um so this feels like a very pressing need um especially since agents are running basically everything as many of our audience knows so tell us about like first the high level like philosophy of how um raindrop came to be totally yeah what was your thinking what spawned it totally i mean we started we started a while ago now and it was really just
like we were hacking on coding agents we actually had a coding agent like many years ago that's how the company started and um and they weren't even called agents then they were just like chat bots that could like then it's like chatbots that had tools it was like all very vague yeah um the word agent was actually really cringed for a while um and i think that it became really obvious to us
“as models were getting better this problem was getting worse”
Read the whole transcript
Yeah.
“if you actually want to improve your agent you need to understand if it's from storage or retrieval”
to build the internal tool side project or product you'd otherwise lose a weekend to with Lovable, you get software you can ship today. That includes an editable code base.
You can inspect and change plus two-way GitHub sync. Lovable also handles the backend and infrastructure, including managed Postgres, auth, storage, hosting, payments, and more. So you don't have to wire it all up yourself. Through Lovable's MCP server, you can create and deploy your project directly from the agents you already use. Turn ideas into software people love with Lovable. Lovable.dev. Now, on to the episode. Yeah. I'm also curious, before we talk about simulations, where do you get the initial data or how do you find the most common issues or how did you initially... And then how do you see like track them as they evolve for users it's a really good question so yeah we work with like a crazy range of customers at this point it's everything from like you know logistics law health care you know personal agents it's just like everything and each of those have a completely different set of issues things they care about like what is good behavior what is bad behavior um and it is literally wildly different um and so a lot like our entire product is built around being able to um do that in an unsupervised way but like to then show people like candidate issues that we think are important to them that they can then like uh give feedback to and we can
adapt in real time like our everything that we're detecting yeah yeah so tell us about simulations
“most teams right now are not able to iterate on their agent in a meaningful way without shipping to production.”
um agents like you want them to do something new you want them to think out of the box like the amount of times that i literally in the prompt contain and think about this like in a novel way or you know or i'm like search all existing like libraries for such and such like niche or something yes and then i that's my way of like prompting some sort of out of the box like creative instinct in an agent but like what are your broader thoughts about how this gets solved
“i do think that frontier models are actually worse at these things than earlier models”
came from apple's design team yes so that's that's an interesting and then you also did avionics yes so that's an interesting trajectory to now you know working on like problems with agents so how what was like what was the trajectory how did this happen yeah so I've kind of had yeah I've had a weird trajectory so uh yeah I went to school for robotics and I was doing that sort of doing engineering really really obsessed with avionics um then I was really
obsessed with like uh human computer interaction but with robots um it's like robot human interaction and um and I think I I got really really interested in like AR at some point yeah and that's that's also happened to me yeah exactly like I because it was kind of like all the things I loved about like human robot interaction without the hard parts like literally the hard parts like it was like you know you don't have to worry about how does you know how do robots walk upstairs or something and just like you know and so I think that like unifying theme throughout my whole life is like I love um what I kind of call like unbounded it.
Interfaces where there's like no, you know, really huge input space and really huge output space or like infinite. Like if you think about like AR or robots or, you know, rockets are certainly unbounded. It's like, they're just like, there's all of these things, you know, you have like AR can use your entire, you know, field of view. It can use, you know, and it can come close to you. It can come far to you. And I was obsessed with solving that. So yeah. So I was a designer at Apple for like, yeah, like almost four years working on Vision OS. Yeah, that's awesome. So what is, in your view, like the mission of Raindrop? Where do you see it like possibly evolving after like simulation?
Yeah, so our mission is to protect the world from agent failures and both big ones and small ones. And so I think both parts are important. And I think it's actually been really humbling over the last few months seeing both as agents get deployed into more critical sectors, but also as our kind of customers have grown larger and larger, the issues we detect suddenly, you know, actually start to touch like human lives in really meaningful ways, like where people's safety are concerned, etc. And, like, that's actually been – we've, as a company, seen, you know, what we do change, like, a lot and just the importance of it change.
So it's been pretty cool. Yeah. And how do you think about building for, like, personal use cases versus for companies? You mean, like, individual developers? Yeah, like, individual developers. I guess, yeah, I guess that I'm speaking from a very SF-like point of view where everyone is, like, a quasi-developer. Yeah, yeah, yeah. But, yeah, I guess, like, for individual people versus teams, that sort of thing. Yeah, I think both are really, really important, right? Like, I think you want to have a product that just, like, any random developer can sign up for or get value out of. At the same time, especially when we started, a lot of the pitch was, you know, like, we were working with companies that have, you know, millions of, you know, like, messages or events a day.
And it's, like, making sense of that is insanely hard. So that was our focus at first. I think simulations is our first step towards, you know, something that's useful even with, like, N equals one. where it's like, and I use it for things that I'm building. And so it's like just being able to say like, okay, I'm going to merge, you know, I don't like the tone of this or like I don't like when it asks, you know, proposes a calendar event at this point in time. Let's try changing the prompt. Let's see if it actually fixes it. That's been a really powerful flow for me. Yeah, that was going to be my next question.
Like how have you personally seen, you know, efficiency improvement, process improvements from using simulations? Well, yeah, we use it a ton. i mean raindrop broadly especially like we use a ton within raindrop which is kind of funny like we have a lot of agents actually everything is now so like recursive yeah i mean just be very clear like that that is like i think for us you know we've always been very focused on that problem of like well you know uh there's clearly too much to go through by hand like if if we had to make sense and build agents without using ai it would actually be impossible um and even the level of complexity of the agents that we're building now would not be possible without agents it's like very obvious and so um we've been really focused on like well what do you need to close that loop um like where you get an issue and you could actually solve that and know you solved it and push it which is like like again the loop is like recursive self-improvement essentially at least at least some numbers of loops right like we want to push get as many loops in before you hit a wall and i think for us what we realized very quickly was like simulations is the way to solve that because you get an issue and you just need to be able
to know like teams right now are very very scared of making changes to their agent because they don't know what's going to happen like they're like well you know we're paying x much for you know um you know to use opus 55 should you know sonic comes out and you're like well we could save this much amount of money if we switched but should we it's like very unclear um and you know You could put it into production. and, you know, do a feature flag or whatever, and we help with that too. But I think the best way we have found to get, you know, confidence before is simulations. Yeah. Well, very well put.
Thank you so much for coming on. We're coming up on time. It was great having you on and hearing about Raindrop. So, yeah, thanks so much. Good. Great to be here. A huge thanks to MTS sponsors. ElevenLabs, AI that communicates at human level across every channel and modality. ElevenLabs.io slash MTS. scale your startup on neon millions of developers and startups have already chosen neon for their back end start on the free plan or get up to a hundred thousand dollars in credits for your startup at neon.com slash mts special thanks to our sponsor kong the ai connectivity platform connect apis llms agents and systems with serious security and governance konghq.com Finally prove what your AI investment is worth with Lairdyn. Head to Lairdyn.com today to start measuring your AI's bottom line. MongoDB, trusted to run critical applications by 75% of the Fortune 100, the intelligent data platform for the era. Apply today for access to up to $15,000 in Atlas credits, dedicated technical support, and exclusive events and go-to-market opportunities. MongoDB.com.
support for the show comes from vcx the public ticker for private tech The U.S. stock market started history's greatest wave of wealth creation. From factory workers in Detroit to farmers in Omaha, anyone could own a piece of the great American companies. But today, our most innovative companies are staying private longer, which means everyday Americans are missing out. Until now. Introducing VCX, a public ticker for private tech. Visit GetVCX.com for more info. That's GetVCX.com. Carefully consider the investment material before investing, including objectives, risk, charges, and expenses. This and other information can be found in the Innovations Fund Prospectus at GetVCX.com.
This is a paid sponsorship.
