Real Alpha
Try Pro free

Unsupervised Learning · 2 min · Jun 4, 2025

Mercor's AI interviews increased onsite conversion

Brendan Foody · Co-founder & CEO, Mercor

Brendan Foody says Mercor uses AI interviews for nearly every role except executives.

In this episode

Full analysis
  1. 13:51AI labs are shifting from crowdsourcing to expert dataPro pick 3 min
  2. 18:55Mercor followed demand signals into AI data work4 min
  3. 35:00Foundation models may split between APIs and custom applications3 min
  4. 23:46Mercor's AI interviews increased onsite conversion2 min
  5. 26:44Knowledge workers may shift toward creating evals4 min
  6. 7:35AI can assess candidates while people focus on selling the role3 min
  7. 11:17Missing workplace context can be the main evaluation bottleneck2 min
  8. 5:13Mercor tunes candidate evaluations with customer outcomes3 min
“And then we switched over to fully an AI process before the onsite and the conversion went up on the onsites.”
Brendan Foodyat 24:50 · Unsupervised Learning
24:50

Takeaways

  1. Mercor uses AI interviews for almost all non-executive roles.
  2. A standardized assessment replaced a human case study before the onsite.
  3. Onsite conversion increased after the change.

About this segment

Brendan Foody says Mercor uses AI interviews for nearly every role except executives. Standardizing the pre-onsite assessment raised conversion into onsites, he says.

Brendan Foody

Brendan Foody is co-founder and CEO of Mercor.

Co-founded Mercor, a labor marketplace supplying AI companies with expert data, paying tens of millions weekly.

  • JS

    Jordan Segall, Host · Partner, Redpoint

From “Mercor CEO: Evals Will Replace Knowledge Work, AI x Hiring Today & the Future of Data Labeling”, Jun 4, 2025. Watch the full episode

Answers “Is AI paying off for companies?” See all 54 answers

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

  1. 23:53

    And all of those translating to them finding ways to trip up the model and improve capabilities. Do you guys use your own product today? And how does it get used in your own hiring process? Absolutely. We use it for every role except our executive roles. I mean, we still have the listing for our executive roles, but most of our executives, I would take the first interview rather than sending them.

  2. 24:11

    straight to the AI interview for the selling reason, not the vetting reason. Yeah, I mean, it's extremely effective. In fact, we've found that in many cases, it's like the most predictive signal. I think one thing people underestimate hiring processes is that humans have this very strong bias towards thinking that they're right in this vibes-based assessment. And like hiring is like the original vibe, everything.

  3. 24:34

    VCs definitely do not suffer from that. Yeah, and it's like, it's like, let's ground everything in the performance data of who's actually doing well on the job. I remember actually like we, so we have this role we're hiring for strategic project leads. And we used to have a human case study before like the strategic project lead onsite to, and the onsite is like working with us for a day to see how they would do on various parts of the job and figure out who to hire. And then we switched over to fully an AI process before the onsite and the conversion went up on the onsites.

  4. 25:02

    And so it's like through using the AI interviewer, just being a lot more objective about the comparisons, having it standardized throughout everyone who's applying to the role rather than just like mixed across like three different interviewers. It was allowing us to have a lot better conversion. What about on the eval side? Are you guys using kind of a bunch of people that you source for your own evals?

  5. 25:24

    Do you do a lot of that internally? Yeah, we use a lot of people or we work with a lot of people from our marketplace to create our own evals. And so it's a similar process to what we go through with our customers. And, of course, we still need the researchers involved with those people and understanding what are the reasons that the model is making mistakes. How can we create our taxonomy, have our post-training data reflect that taxonomy, and he'll climb on the eval.

  6. 25:49

    But it's all the same processes and people.