MTS · 3 min · Oct 28, 2026

Mercor uses training runs to validate its data

Edward Hu · Head of AI Modeling, Mercor

Edward Hu explains why Mercor trains models on its own data and aims to attach a training recipe to each delivery.

In this episode

9:22 Benchmarks can punish capable models for unfair reasonsPro pick 6 min
14:52 Realistic AI evaluations must reflect how work happensPro pick 3 min
5:29Mercor uses training runs to validate its data3 min
17:29 Mercor published the steps behind its 400-billion-parameter run2 min

“And our goal is actually to, with every data delivery, we actually attach a training recipe and some training results based on an open training stack.”

Edward Huat 6:40 · MTS

Takeaways

Training models tests whether data helps with a benchmark or domain.
Mercor aims to include a training recipe and results with each delivery.
The research team links model training, benchmarks and economic research.

About this segment

Edward Hu explains why Mercor trains models on its own data and aims to attach a training recipe to each delivery. That connects data validation with benchmark and economic research.

Edward Hu is Head of AI Modeling at Mercor and former OpenAI researcher.

Leads model training and research at Mercor; invented LoRA and μTransfer at Microsoft.

Theo Jaffee, Host · Co-founder & host, MTS
One of the first members of a16z's new media team; earlier hosted the Emergent Ventures-backed Theo Jaffee Podcast.
Sophia Dew, Host · Host, MTS

From “AI Outperforms Junior Accountants on Real Work | Edward Hu”, Oct 28, 2026. Watch the full episode

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

5:33
Yeah, that's something I wanted to talk about with you as well. I know your team is expanding. Lots of new folks are joining. You yourself used to be a researcher at OpenAI and you've joined McCore. You're now leading all of their AI modeling. Tell me a little bit about your role during McCore. You say you came in doing one thing and now your role has expanded a ton and the type of research you're doing has expanded a ton.
5:57
Yeah, we were just talking right before going live. Initially, when I joined the team, my role was let's start training some models. And training model from our core is important in many ways. First of all, as we serve model training companies, as we serve post-training researchers, it's so important to know what it takes to train these models. And that directly affects the quality of our data. And increasingly, the best way to know if a piece of data is valuable or not, if a data delivery for a particular benchmark actually helps with that benchmark, with the domain, is to actually train a model on it.
6:40
And increasingly, we're doing that for a lot of our customers. And our goal is actually to, with every data delivery, we actually attach a training recipe and some training results based on an open training stack. So that's initially what I set out to do at Mercore. and very quickly we realized the research opportunity at Mercor is enormous. There are so many things to tackle, especially things that we're uniquely positioned to tackle, like what we talked about, building these frontier benchmarks.
7:09
And I'm working very closely with the benchmark authors I just mentioned on building that next iteration of the frontier, also working with folks, pushing the frontier of model training. And I'm very proud to say most of our work for model training is going to be open source because we want to support the open model ecosystem. And like what Aiden talked about here on the show two weeks ago, we're also working on econ research.
7:36
And to me, it's important for all these research directions to be integrally connected. It's really important that we are not just a team with some people training models and some other people building benchmarks. The culture we're cultivating is that everyone, on the research team, first of all, they are part of one team and they all work closely together. One example is the APEX accounting benchmark is a frontier benchmark we built on the research team in collaboration with RAMP.
8:09
And that became the vehicle for our economic research team to do this never-been-done research that became quite impactful. And we also have our upcoming benchmarks actually measuring another way that we believe AI should be economically valuable, but it's undercover by the benchmarks today.