# AI in NYC: How Tenex Builds an AI-Native Engineering Team — Alex Lieberman & CJ Hess Markdown copy of https://realalpha.app/v/Vki6cklFqxo/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy. Alex Lieberman and CJ Hess argue that AI-native engineering rewards outcomes and requires teams to standardize processes and preserve context for agents. - Show: Latent Space - Hosts: [Shawn Wang (swyx)](https://realalpha.app/people/swyx/) - Guest: [Alex Lieberman](https://realalpha.app/people/alex-lieberman/) · Co-founder & Managing Partner, Tenex - Guest: [CJ Hess](https://realalpha.app/people/cj-hess/) · Special projects lead, Tenex - Published: Oct 21, 2026 - Length: 44 min ## The best minutes - [7:16](https://realalpha.app/v/Vki6cklFqxo/?t=436) [Tenex keeps agent plans and decisions in version control](https://realalpha.app/s/tenex-keeps-agent-plans-and-decisions-in-version-control/) (3 min) - [19:23](https://realalpha.app/v/Vki6cklFqxo/?t=1163) [Only 5% of large enterprises run non-engineering agents](https://realalpha.app/s/only-5-of-large-enterprises-run-non-engineering-agents/) (5 min) - [30:26](https://realalpha.app/v/Vki6cklFqxo/?t=1826) [Rebuilding SaaS depends on value and maintenance burden](https://realalpha.app/s/rebuilding-saas-depends-on-value-and-maintenance-burden/) (4 min) ## Also worth watching - [3:00](https://realalpha.app/v/Vki6cklFqxo/?t=180) [Output-based pay still has to account for non-coding work](https://realalpha.app/s/output-based-pay-still-has-to-account-for-non-coding-work/) (4 min) - [14:37](https://realalpha.app/v/Vki6cklFqxo/?t=877) [Tenex's token spend is roughly one to two times employee spend](https://realalpha.app/s/tenexs-token-spend-is-roughly-one-to-two-times-employee/) (5 min) - [24:26](https://realalpha.app/v/Vki6cklFqxo/?t=1466) [Vertical AI agents face a hard path to software margins](https://realalpha.app/s/vertical-ai-agents-face-a-hard-path-to-software-margins/) (6 min) - [34:48](https://realalpha.app/v/Vki6cklFqxo/?t=2088) [AI job changes demand action from employers and workers](https://realalpha.app/s/ai-job-changes-demand-action-from-employers-and-workers/) (3 min) Skip the model and tool preference chat and the New York ecosystem and hiring discussion. ## Summary Lieberman describes Tenex as an applied AI partner that helps large companies move from AI absent to AI native. He says its 75-person team is made up mainly of engineers and AI strategists and serves Fortune 2000 and mid-market customers. Hess was Tenex’s first hire after trying startups and running an engineering consultancy; his special projects include agent workflows and an internal platform. Lieberman and Hess say Tenex’s engineering model rewards output rather than hours, though the company is still refining how it pays for architecture and client work. Hess describes a MetaHarness workflow that stores plans, decisions and project context in Git; Lieberman says engineering output has roughly doubled as models take on longer tasks. Lieberman says enterprise AI adoption drops sharply between general-purpose LLMs and production agents, especially outside engineering, and argues that agents need standardized processes and data controls. They debate vertical AI and SaaS replacement, the effects of automation on jobs, and New York’s potential as an AI hub. ### Minute by minute - [1:25](https://realalpha.app/v/Vki6cklFqxo/?t=85) **Engineers earn for output.** Hess says he joined as Tenex’s first hire after seeing AI raise his engineering output while hourly consulting paid him less. He says Tenex ties engineering pay largely to story points, with roughly a 4x gap between its highest- and lowest-paid engineers. Lieberman says the model still needs nuance for architecture and client work. - [7:16](https://realalpha.app/v/Vki6cklFqxo/?t=436) **Agents need durable project context.** Hess describes MetaHarness as a structured set of documentation and tools for planning work, tracking decisions and producing client readouts. Plans, code and decisions are checked into Git, he says, so agents and engineers can access not just what the team built but why. - [10:14](https://realalpha.app/v/Vki6cklFqxo/?t=614) **Long-running tasks raise output.** Lieberman says engineering output has roughly doubled over the year as models took on longer-running tasks. He says Tenex revisits story-point criteria and project estimates at least quarterly to keep compensation tied to value as models improve. - [14:37](https://realalpha.app/v/Vki6cklFqxo/?t=877) **Token costs are falling.** Tenex estimates its raw token spend at between 1:1 and maybe 2:1 relative to employee spend. Lieberman says falling per-task prices have made excessive token use less threatening than it was six to eight months earlier, while cheaper frontier models and competition could keep pushing costs down. - [19:23](https://realalpha.app/v/Vki6cklFqxo/?t=1163) **Enterprise agent adoption lags.** Lieberman estimates that, among roughly 100 enterprises Tenex is speaking with, 90% have rolled out general-purpose LLMs, 60% offer training, about 50% have coding agents, and 5% have non-engineering agents in production. He says employee access to company data also creates permissioning problems for CIOs and CISOs. - [23:13](https://realalpha.app/v/Vki6cklFqxo/?t=1393) **Agents need a defined process.** Lieberman says companies cannot automate a process that employees perform inconsistently. Tenex first helps standardize the work and define evaluations, then hands repeatable tasks to background agents for people to review. - [24:26](https://realalpha.app/v/Vki6cklFqxo/?t=1466) **Vertical agents need durable advantages.** Lieberman is skeptical of vertical AI agents without a durable data, compliance or workflow advantage, arguing that frontier models could absorb their tasks and that one-off customer demands make software margins harder to maintain. Wang counters that specialist products can win through data, integrations and tailored interfaces, citing Bloomberg and Superhuman. - [30:26](https://realalpha.app/v/Vki6cklFqxo/?t=1826) **SaaS replacement depends on the stakes.** Wang says his Kill My SaaS project replaced a mid-tier product quoted at $40,000 a year with a $1,000 build planned for production at the next conference. Lieberman says replacement depends on risk, savings and whether the capability is core; Tenex has built its own CRM and OS. Wang adds that better fit and faster iteration matter alongside cost. - [34:48](https://realalpha.app/v/Vki6cklFqxo/?t=2088) **AI shifts work and displaces jobs.** Lieberman says AI will augment jobs but also displace some, and expects companies to reassign employees to other problems rather than simply run out of work. Wang says employers and workers both need to reskill, while warning that people should assess how replaceable their roles may become. - [38:06](https://realalpha.app/v/Vki6cklFqxo/?t=2286) **New York can pair AI with commerce.** Lieberman says New York’s AI community is a smaller bubble than San Francisco’s, but its concentration of commerce gives it strong potential for real-world adoption. He says Tenex wants to widen the local AI community; Wang says bringing San Francisco builders to meet New York customers provides a useful reality check. - [41:36](https://realalpha.app/v/Vki6cklFqxo/?t=2496) **Tenex is hiring in New York.** Lieberman says Tenex is hiring deployed engineers and AI strategists and has 75 people in New York. Hess says his special projects focus on MetaHarness and an internal platform for deploying background agents while solving enterprise IT problems. ## Transcript Latent Space, hosted by Shawn Wang (swyx), with Alex Lieberman and CJ Hess. Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there. [0:00](https://realalpha.app/v/Vki6cklFqxo/?t=0) and I just felt my output getting better and faster and ultimately getting paid less. Like I was charging hourly to my clients. I phrase it to the team as like, you can't automate a process that does not exist. Okay, we're here at AI Engineer checking with 10x. Welcome back, Alex. Thanks. Thanks for having us. You brought your director of engineering, CJ. Welcome to the pod. Special projects lead at 10X. Okay, special projects lead. All right, you're the SWAT team. Yes, yes. So you've been on the pod before, and people are vaguely familiar with the 10X story, but can you sort of reintroduce from the top what is the 2026, how you introduce yourself? [0:51](https://realalpha.app/v/Vki6cklFqxo/?t=51) Yeah, very simply, 10X is an applied AI partner for some of the biggest companies in the world to help them go from AI absent to AI native. our belief is that AI is incredible but it is raw intelligence and the hardest problem to solve is the final mile to actually diffuse it through the economy and so our view is to act as the player to help enterprises to figure all this can I curse on? to figure all this shit out yeah yeah we have the explicit tag on Apple Podcasts although I did recently discover that YouTube doesn't like it oh really? they're so like yeah you can't see the F word or they'll sort of like turn off your video what are you going to do? [1:29](https://realalpha.app/v/Vki6cklFqxo/?t=89) I don't care. Okay. We're for adults. CJ, what drew you to 10X? So I was actually the first hire. I don't even know if Alex and Armand had the name 10X quite yet, but I have like a graveyard of startups I tried to start before 10X, ultimately culminating in like my own kind of engineering consulting company. And I spent some time there doing traditional engineering, eventually moved over to doing some of this like AI consulting and just like building more and more with it. I'm a software engineer by trade. And I just felt my output getting better and faster and ultimately getting paid less. Like I was charging hourly to my clients. [2:13](https://realalpha.app/v/Vki6cklFqxo/?t=133) And I really liked kind of the ethos of 10x that Alex had first posted when I saw it on Twitter about kind of looking to reward engineers for their output as opposed to rewarding them for their time. I think you know fast forwarding two years you can see like how engineers can now parallelize a ton of work and just have a ton more output at the time with you know Sonnet 3.5 Gemini 2.5 Pro it might not have been that obvious but I basically took the bet that I think with my skill set and leaning more into AI I could have much higher output and ultimately have like a higher earning capacity to do that. And since then, I've just like fell in love with that model. [2:55](https://realalpha.app/v/Vki6cklFqxo/?t=175) I think that's like what the future of engineering really looks like. And ultimately, when AI is solving all these tech problems, companies are going to pay based on outcome and they're not going to want to pay based on the time that's required to build something. Yeah. So for more detail on that, people can refer back to our previous pod where you literally pay people by story points. Yep. And you just have to agree what the story points are. But that also means that some 10x engineers can literally make millions of dollars a year and some others may not make as much just because they're not as productive. Yeah, exactly. Like the thesis is that AI is really just a leveraged bet on a human's intellect and work ethic. And so you're just going to have more variance than ever before in earning capacity. And our view is our model isn't for everyone, but for the person who basically wants to go along themselves, who has this incredible foundation in software engineering, but also is constantly pushing themselves to leverage the frontier to stretch their time as much as possible while having quality output, there's no better way to earn. [3:53](https://realalpha.app/v/Vki6cklFqxo/?t=233) What's the state of, like last time I talked to you, I think you had like maybe two or three people making a million a year or over a million a year. Is there an update on that? Yeah. So just to paint a picture, we started 10X 18 months ago when we started the business. Really, we started as like an AI powered dev shop. Really like the original idea was engineering is kind of like the canary in the coal mine for AI. And part of the impetus was my co-founder, Armand, who's technical by trade, was like starting to use coding agents a lot. And then he had some people he knew that were still typing into their ID, like not adopting the technology at all. And so we're like, why don't we just supercharged a company's engineering org who isn't adopting this? So we started there. Then as we started moving up market from startups and like mid-market companies to now, you know, 50% of our customers are Fortune 2000 and then 50% are mid-market doing 100 to 500 million in revenue. They weren't just asking for basically staff org and AI engineering. They were asking for like AI transformation as a service. [4:59](https://realalpha.app/v/Vki6cklFqxo/?t=299) Today, the business is at 75 people. 90% of those employees are either engineers or AI strategists, which are basically think of as like technical consultants. And I think the thesis has proved out true, which is there's a very wide dispersion in earning capacity between the highest paid engineer at 10X and the lowest paid engineer at 10X. It's probably something around 4X difference. Oh, not 10X? Not 10X difference, yeah. That's the dream. Yeah, exactly. But everything is proved right about the thesis. I do think like there's been interesting learnings around our model of paying based on story points, where there are limits to the model that we knew would be limits and how we think about that moving forward. [5:42](https://realalpha.app/v/Vki6cklFqxo/?t=342) But overall, I think the concerns that people had around this model of like paying people based on output, like the classic concerns are, well, story points are subjective. People are gonna either sandbag story points or they're just going to ship a bunch of bad code. Like barely check the box. Exactly. Right, but not actually do the job. Yeah, exactly. [6:02](https://realalpha.app/v/Vki6cklFqxo/?t=362) **Alex Lieberman:** And I would say those two concerns have not been concerns at all because of our AI strategists. [6:09](https://realalpha.app/v/Vki6cklFqxo/?t=369) We incentivize, I think, perfectly at odds with engineers where AI strategists are incentivized for customer retention and growth. Actually, the biggest challenge has been, how do you compensate engineers for all the time that's not spent coding, whether it's architecting, meeting with clients? And so that's where we've had to find nuance in the comp model, but it's still largely driven based on output. Anything to add? Yeah, on that, I spent a lot of time thinking about this because if I think about my tenure at 10X, I was coding a lot, coding a lot, coding a little less. And now we're kind of getting to the point where the models are more or less given the right context and can get you to 90% of the solution. [6:52](https://realalpha.app/v/Vki6cklFqxo/?t=412) and I do think there is some opportunity here to think about like really the value is sitting down with a customer, hearing everything that they can brain dump and then just providing that to the model to actually get closer to the solution. So I imagine, you know, in the future, we're going to start to look more at like true what this output is, how it works, how it looks as opposed to, you know, kind of that traditional story point model but still anchoring to those as the unit. I think one thing you'd also, I don't know if we've talked about this before, but like the this idea of like the meta harness, which is like the the engineering workflows at 10x that we use that has allowed us to do, I think, like really meaningful work on long running tasks. [7:35](https://realalpha.app/v/Vki6cklFqxo/?t=455) Do you want to just talk through like what our process is? Because I feel like people find that interesting. Yeah. So we basically started trying to kind of standardize a set of skills. I think most engineering orgs have done something similar by this point. And for us, that grew into what is actually like a very structured set of documentation within a repo and then a CLI that's around it that kind of the models are using to actually drive the work that's being done, whether it's creating a plan, a visualization, you know, creating some readout for a client. Effectively, like building all the pieces of the process around actually writing the code to just happen more automatically and standard across our team. [8:14](https://realalpha.app/v/Vki6cklFqxo/?t=494) So we have a couple of things like one is this like big canonical plan document that has some similar structure. But I'd say as the models have gotten better, we've actually reduced the enforcements there in the structure. Kind of similarly across all of these. But once you have your plan, you actually build it out. And we have a very opinionated way on how agents will go and log their decisions and log what they're doing and just keep track of what they're up to. Everything checked into Git. Yeah. All of this is checked in. all of this is really kind of marked down for me now more HTML I like to see these picture books that the agents fit out but all of that gets checked in and then it serves as this surface where not only do you have the code as the source I think there's a debate on do you even need docs the code can actually be the memory of a code base and I do think you do you need to know why you made a decision you need to know the context around the conversation that was had but ultimately that plus the code base if there's a question, if there's something we need to answer, if we need to look back on why [9:16](https://realalpha.app/v/Vki6cklFqxo/?t=556) **CJ Hess:** we did something, we just always have that in some durable record. And it's never like in someone's head. It's always in the repo. So that just gives like our agents and our tools, these superpowers [9:27](https://realalpha.app/v/Vki6cklFqxo/?t=567) to have all the context that I think a lot of engineering teams still just have in conversations or have in their heads. And our goal is to kind of continue to push that forward so that all the context is in one place and more and more of that work is just driven by the agent as opposed to us having to provide everything up front. Yeah, amazing. And one way we even measure like if any of these kind of like processes are working is are we seeing an increase in trend in average story points per month of every engineer in the company? Right. And what do you think it is, let's say, compared to started this year? I don't know. What do you think? Do you think Plus 50? [10:05](https://realalpha.app/v/Vki6cklFqxo/?t=605) I think so. It really depends because we have some people that I, there's some picture that I saw on Twitter of a guy over like a wizard orb and he's got these like things in the sky. And that's how I think of some of our engineers when they're whipping agents around. I'd really say overall it's been like a 2x increase this year in particular. I think basically if you think of December as the like Opus 4-5 moment where the models really started to take on these long running tasks to today we're back to Opus, but Opus 5.5. I mean, that's just a magnificent model that can really do a ton of work. [10:39](https://realalpha.app/v/Vki6cklFqxo/?t=639) Over that period, I've seen the actual output of engineers, I would say roughly double in terms of what they were able to do to what they're able to cover in the same time span now. Is a single story point worth the same now as it was a year ago? We're always trying to calibrate. It's a tricky equation. Is there like inflation or deflation, right? I don't know. I mean, in theory, there should be deflation, right? Yeah. Yeah. Yeah. I don't know. And that's roughly how we think about it. I would say at least once a quarter, we kind of sit down and evaluate, like, what is our criteria for these story points? [11:13](https://realalpha.app/v/Vki6cklFqxo/?t=673) How are we actually estimating these projects? There's a fine balance to be made between, like, still rewarding this output, but also as the models get better, we do need to be able to kind of calibrate, like, the value that we're creating. so overall like I would say roughly that 2x has played a similar role in like how we do these estimations but I mean if I just look at some of the features and things people are building and pushing out now versus this time last year I mean it's it's insane what the difference has been yeah well so there's a couple ways I can take this but I'm just going to check in on the sort of model preference stuff since that's always a topic conversation you I think you just let people choose whatever the tools they like but then you maybe you have some statistics on what is preferred in the office uh it seems like opus is leading yeah what's the like care to estimate like the market share whatever the vibe right now is opus 5.5 i think no fable i don't particularly use fable and i think a lot of people right now are just like opus is good enough opus is so good it's fast it's cheap it kind of hits all my check boxes um i would say i previously have had like [12:19](https://realalpha.app/v/Vki6cklFqxo/?t=739) two to three models depending on whether they're cheap and fast whether they're really smart like whether they can solve specific problems and now opus is kind of covering my my full range um i think a lot of the team still reaches for tools like codex when we need to do something maybe more in the computer use world um and really workflow wise i think using both is still the best option whichever your driver is which i think for most of our team is opus 5.5 using a codex to review and having them actually work back and forth on your machine, I think still produces the best results. But right now we're pretty aligned on those two. [12:57](https://realalpha.app/v/Vki6cklFqxo/?t=777) And are you still AMP maxing or no? Oh, I am AMP maxing. So you're using Opus through AMP? So, yeah, they recently... Or that's the default model for AMP. Yeah, they recently launched the ability to hook in your Cloud Code subscription. So that's been a pretty big game changer for me. because that'll really cut your cost by like 80% totally I mean I love AMP because it takes everything off my computer they have this like cloud setup the orbs we actually co-hosted a happy hour with them last week so it was cool to meet the team and orb out but that's never going to be a thing oh yeah this may be the first time in history that any human has said we're orbin out they do say it's orbin time Yeah. [13:44](https://realalpha.app/v/Vki6cklFqxo/?t=824) That's it. But yeah, that's my interface of choice. And models are really between that if I'm leaning on codecs. It's Sol6-1. Did you give CloudTag a shot? Isn't it weird that Anthropic never captured the remote environment thing in the way that Orbs did? Yeah, I think there's a piece of that, I think, on the AMP side of them being very opinionated and not trying to build this general solution. tools like cloud code in the cloud and cloud tag i think provide value for teams that are really trying to like collaborate maybe on non-technical tasks but part of the reason i love amp is that it has all these cross model functionalities and i can do that opus writes it you know i can even hop to another orb and say look at that orb and review it and all of those things still come together to basically replicate what i'm doing on my machine and i just couldn't quite get there with a cloud code in the cloud, a cloud tag. [14:39](https://realalpha.app/v/Vki6cklFqxo/?t=879) Yeah, fair enough. Another thing I always track, I'm starting to track this year, is percent of let's say token spend to employee spend. I don't know if you have that number. So roughly, most companies are at maybe let's call it $10 to $200 in spend per employee. The more AI native ones would convert that to a percentage, so roughly 10%. The average AI startup UpEmployee is about 1x in terms of employee spend and token spend. And then it goes up to maybe 5 to 10x in terms of token spend. Where are you guys in that range? I actually don't have a number off the top of my head. [15:19](https://realalpha.app/v/Vki6cklFqxo/?t=919) **Alex Lieberman:** I'd say we're somewhere between, if we're talking like raw tokens, somewhere between 1 to 1 and maybe like 2 to 1. [15:29](https://realalpha.app/v/Vki6cklFqxo/?t=929) Really? That's pretty high. And what we think about a lot is like when to reach for the biggest model. I think every company is starting to think about that. But I will say recently our costs have been driven just way down from Opus 5.5. We kind of had an era of like everyone was using Astra and Fable and they just gave the best results. But ultimately those models are expensive. And like a lot of like, you know, since the last time we talked, a lot of people had this sort of token maxing error, token leaderboards, everything. And then they were like, oh, no, no, no, no. Now we have too much cost. [16:01](https://realalpha.app/v/Vki6cklFqxo/?t=961) And like, well, it's the most predictable thing in the world. Did you ever fear about it? or like you were just getting good ROI regardless? Yeah, I mean, there's a few ways that I think about it. It's like we were for sure worried about it just as like our headcount was growing. And I think also like even though our team, I would say like I don't feel like anyone on our team ever really ascribed to like the token maxing zeitgeist because... I had a few moments. Yeah, me too. But it's like, well, if the boss is going to pay for it, I'm just going to go nuts, right? Because then you do the weird stuff so that you can scale back. [16:44](https://realalpha.app/v/Vki6cklFqxo/?t=1004) But you have to do the weird stuff first to discover the lessons that you can take with you. Yeah, totally. I mean, this may sound crazy, and I'd be curious of your thoughts, but I was talking to someone earlier about the debate that continues around, basically, I promise this is related, I was talking to someone earlier around the data companies, right like um micro one record handshake and just like what is like what does the long term for these businesses look like right because like not 90 plus percentage of their their revenue is from the labs and a number of these founders would argue that they think it's going to flip-flop where in the next few years it's going to be enterprise driven and then like the thing that comes to mind like the next thing i assume you have to believe if enterprises are going to be the core buyer of that data is that enterprises are going to be doing a lot of fine-tuning on their own models. And then the question becomes, where is that going to make sense? It's either you think there's going to be better quality, which again, like the bar is just going to keep moving on the frontier. So that's like a hard bet to make. Then it has to be price. The price one [17:46](https://realalpha.app/v/Vki6cklFqxo/?t=1066) feels hard to me because it's like, we're just seeing token costs continue to go down on these models. And so I guess part of my, the reason we're not concerned internally is because it It almost feels like we've kind of hit this like Pareto point of like costs are coming down at a per task basis. Then it is way harder today for someone to token max and like really F the company in terms of cost than it was six to eight months ago. Like almost think six to eight months ago was the point at which like you'd be really concerned. Yeah. Do you agree with that? Yeah. I think there's a couple ways like the future might play out there. [18:28](https://realalpha.app/v/Vki6cklFqxo/?t=1108) Like, if there's a world where there's a Astra 6.5, a Fable 5.5, and they're super expensive, but they get the best results, I could see us trending back that direction. But generally, I agree that, you know, you look at something like Muse Wonder that they launched yesterday, today, and it's on the frontier. It's dirt cheap. The competition is heating up, both in open source and closed source. And I think that just will continue to drive costs down. And when you really look at it, a lot of tasks that are economically valuable can be done with Opus 5.5. So, you know, it's that cliff of like, would a 10 times smarter model really justify the cost of like moving some cells in a spreadsheet? [19:14](https://realalpha.app/v/Vki6cklFqxo/?t=1154) Probably not. Right. So coming back to your customers and what you're seeing from the market, what is the most common ask that you guys get? And then what is something that people come to you with and you're like, no, we don't actually want that job yet? Yeah. So one way I think about this is I think about enterprises adopting AI as almost like these different milestones in a race. And I think, especially at AIE, when I tell people about what percentage of companies have reached these milestones, they're kind of like mind blown by it. But it's like, OK, first milestone in the enterprise AI race is just rolling out a general purpose LLM to your people. [19:53](https://realalpha.app/v/Vki6cklFqxo/?t=1193) And I'd say of the, you know, let's call it 100 enterprises we're talking to today, 90% have rolled out either Claude or Codex to their employees. Then the next milestone is like actually teaching people how to use it, like training and enablement, which like never gets the love that it should get, but is arguably more important or as important as the technology itself. I would say that's at like 60%. Oh, that's pretty good. Yeah, yeah. Like there's some group there of companies that are like, our people will just figure it out. But then there's like a good chunk that is like, okay, we need to teach our people what skills are. [20:31](https://realalpha.app/v/Vki6cklFqxo/?t=1231) We need to teach our people the features and functionality of these surfaces. Then the next after training and enablement within a general purpose LLM is coding agents. And the crazy thing is like, I was asking a few people here, like what percentage of enterprises do you think have rolled out coding agents to the engine product org in their company? And most people are like, it has to be 70 or 80%. The number is closer to 50%. Like every other enterprise that we speak to either hasn't rolled out a coding agent or what they are using is closer to not having a coding agent than having a coding agent. [21:08](https://realalpha.app/v/Vki6cklFqxo/?t=1268) Then the next, and this is where like the biggest, I would say, drop off is, where like the biggest diffusion gap is from like raw intelligence to enterprise usage, [21:17](https://realalpha.app/v/Vki6cklFqxo/?t=1277) **Alex Lieberman:** is agents in production for non-engineering work drops to like 5%. [21:22](https://realalpha.app/v/Vki6cklFqxo/?t=1282) Wow. It's like, I mean, you're in these conversations all day long, but like most enterprises that we talk to that have, let's just say more than a thousand employees do not have an agent in production that is doing work that is not building software. Sure. And so I think there's a few things. One is there is because like there's just this whole wave is driven by a combination of some small group of people understanding what is possible from this technology in a way larger group of people driven by FOMO. And by the way, FOMO isn't bad. It creates some forward momentum. But it also means that people will latch onto the things that are most in the zeitgeist. [22:01](https://realalpha.app/v/Vki6cklFqxo/?t=1321) I guess now for the last year and a half, people have been saying it's the year of the agent or however long it is. Every year is the year of the agent. Like enterprises are like, can you help build us an agent for some back office task within the organization? Like that is one of the most common things. And it's why now a lot of the work we're doing at 10X is deploying agents into production outside of the entor for companies for repeatable work that they're doing. The second is permissioning and like RBAC. Basically, as every employee in a company has Claude or they have Codex and they need access to certain data, it is turned into an absolute nightmare for CIOs and CISOs. [22:41](https://realalpha.app/v/Vki6cklFqxo/?t=1361) and they don't have basically a good gateway for easily doing almost like a control panel for data permissioning. Literally just what can you read, what can you not read. Exactly. And that is a huge issue. So that's like something we spend a lot of time on. So you make them for people or do you like buy something off the shelf? So basically we started by building bespoke for companies and now we actually productize that where we have our own gateway solution. Exactly. Exactly. Is there anything else that comes to mind that's like we're hearing a lot from customers? I think that covered what we're hearing when it comes to like particularly the agent deployments. [23:18](https://realalpha.app/v/Vki6cklFqxo/?t=1398) I think one of the pains that just always gets discovered, I phrase it to the team, is like you can't automate a process that does not exist. So when you have 40 people that are all roughly doing the same function, but they're all doing it a little bit of a different way, there's actually it's it's way harder to build an agent that's kind of proactively doing that in the background you can totally sit down with them give them the tools walk them through how they could augment their own workflow but when you really start to get the leverage is when you have these agents that are surfacing work to you you're reviewing it you're throwing it back and you become more of this like you know everyone's been talking about becoming an agent manager and i think we're on the cusp of that but when you zoom out oftentimes our recommendation to companies is like, hey, first, we need to build a process around this organization within your company. And then here are the steps for us to actually hand off that work to an agent in the background. But if we don't have some form of standardization, it's going to be impossible for the agent to know whether it's doing it right for us to build evals around this system and actually [24:23](https://realalpha.app/v/Vki6cklFqxo/?t=1463) deliver the value that an agent that is doing this function would have. And even one thing I'd be curious your thoughts on, we were literally talking about this right before, is and like CJ will be okay with me sharing this is like the we were talking about like vertical specific AI solutions and like what does the future look like for these solutions and we were generally expression expressing an overall bearishness with vertical specific AI solutions in different functions or industries and part of it is just from our vantage point of [24:55](https://realalpha.app/v/Vki6cklFqxo/?t=1495) **Alex Lieberman:** spending all day in like the guts of enterprises, there is so much nuance and so much inertia. And so like so many systems you have to wrangle and people you have to wrangle. And also when a buyer [25:08](https://realalpha.app/v/Vki6cklFqxo/?t=1508) out of a vertical solution is expecting truly a one of one experience, like we're in this, we are in an age where people are expecting one of one software solutions. And so if you're expecting that. It's no wonder that every software company is basically building up a team of FDEs because there's just this convergence happening of services and software when every company wants all this nuance that was not traditionally expected from software. And so I think it's really interesting to think about, yeah, like four vertical specific solutions. Either one, how will they be able to do this at software margins if everyone expects a one-of-one solution? [25:52](https://realalpha.app/v/Vki6cklFqxo/?t=1552) And also, why should we not expect frontier models to just gobble them up? Yeah. So to give some concrete examples, you're talking like Harvey for lawyers. Yep. Open evidence for doctors. These are the classic examples of people that are winning their domains. And you're bearish on them? I would say generally bearish on them, correct. Wow. Well, so far you've been wrong. and you know i think that the the answer very simply is like access to data that chat gp is not going to have which is like very common that's a that is a moat obviously anthropic and open ai are also acquiring those same data sources but they're just not as dedicated uh you know as as uh you know these guys are going to be um they do have innovations uh if you look if you've like tried it harvey um they have like the tabular review that lagora also have them we've had the cto on the podcast um where they They have affordances and UIs and all that that is just customized because they only serve them. [26:50](https://realalpha.app/v/Vki6cklFqxo/?t=1610) The pitch is focus, basically. Focus, affordances, maybe. What do you mean by affordances? In the way that you care about the last mile for your customers, they care about their last mile in terms of the software. And as long as people train in that software, it never goes away. As a banker or in my finance career, I could always access the same exact information from the NASDAQ, from the New York Stock Exchange. but why do I pay $50,000 a year for Bloomberg? Well, it's just like that's what everyone, all my peers do, and it's all consolidated for me, and it's all specialized for me so I don't make mistakes or whatever. [27:24](https://realalpha.app/v/Vki6cklFqxo/?t=1644) It's tied into my trading system or my chat or whatever, my news system. No, I think people play for Bloomberg as the social network. I think people love the DMs in Bloomberg. There's that. There's that. But as a trader, I know I actually use that for trading. Yeah, yeah, yeah. So, I mean, I think there's a bunch of integrations and tools, And I think ultimately it's hard to be bearish on any of them, mostly because the demand for software, the demand for tooling to do your job is just huge. And some of it, yes, is going to be fulfilled by horizontal, generalized tools like the Claws and ChatGPTs. [27:55](https://realalpha.app/v/Vki6cklFqxo/?t=1675) And some of it, you're just going to want something better. And I always think about maybe the success of Superhuman as an email client. Everyone has Gmail, but here's just premium Gmail with more stuff for people who live in email all day long. And that's a market. It maybe wasn't the largest startup in the world, but I think reasonable people can disagree how large each thing is. Yeah, totally. I think what's not to be disagreeing with is the total market's going to grow and all rising tide is going to lift all boats. And then you're just talking percentage pressure market at that point. Yeah, I agree with that. Totally agree. [28:30](https://realalpha.app/v/Vki6cklFqxo/?t=1710) I think there's even, there's like a differentiation between some new app built from the ground up with AI involved. That's what comes to mind when I think about Superhuman. I love Superhuman. I don't even spend a ton of time in email, but anything that has... No, you pay $30 a month just to not worry about it. Anything that has cool hotkeys, I'm into. Yeah. But when I think about, I guess, products where they are really pushing some agent as the product that is doing work in some specific vertical, unless there is some very strategic compliance data, whatever it might be, moat, I think those are going to struggle going forward. [29:09](https://realalpha.app/v/Vki6cklFqxo/?t=1749) As we start to see, like, quite literally the ability to either spin up your own really quickly, continue to compound as the models get better, or just, you know, those tasks start to fall into the world of the models just generally. But anything that has this, like, app and tool built around it, kind of to my previous point, changes the process and has an agent on top of it. And I think, like, that kind of builds on the same philosophy, and I think that can still be super helpful, and those will be successful. Yeah. Oh, OK. So there is a related view about this that maybe isn't quite as spicy as yours, which is basically, it's not like nobody wants to use. [29:49](https://realalpha.app/v/Vki6cklFqxo/?t=1789) People probably don't want to use the agent that you built. They want to use the agent that they already have. And they want to plug in your data into their agent. So everyone is sitting on some part of your data. And they're like, use my agent, please. Please pay me that $20 a month. But you can't just stack up all these $20 a month subscriptions. Ultimately, probably a small set of companies are going to own the interface, and then everyone just plugs into that overall agent, which I agree with. Then it's interesting that Salesforce, being one of the system of record companies, the biggest probably, or the most well-known, is embracing that path as, like, you will not use our agents. [30:26](https://realalpha.app/v/Vki6cklFqxo/?t=1826) We will just provide APIs and MCPs for you to plug into your stuff. Yeah, which is kind of cool. OK, so I actually wanted to also walk through one example. So I just released my results for my Kill My SaaS competition, where we replace a mid-tier SaaS. I call it mid-tier basically in the sense that it's probably not that trivial, but it's not Salesforce. It's somewhere in the middle. And I think there's a lot of mid-tier SaaS out there that people are always talking about, like the death of software, right? We were quoted $40,000 a year. We spent $1,000 to replace it. It's going to go into production for the next conference. [30:59](https://realalpha.app/v/Vki6cklFqxo/?t=1859) So we are that customer that says, like, I want the one-on-one tool, and I'm expecting all these things. and we're building a domain specific or vertical specific thing for our business. And so far it's going okay. Tell me where it's going wrong. Also, I'm wondering how many story points that is. Yeah, well, CJ can size up the story points on him. I mean, my general view is I think the ripping out of, my general view is the more you move up marketing business, the harder the proposition of ripping out and building from scratch gets. I agree with you. But part of the reason when, as people have said, a very small group have said, Salesforce is really at risk. [31:41](https://realalpha.app/v/Vki6cklFqxo/?t=1901) SAP is really at risk. Don't agree with that at all. It's just like those businesses in my mind just accrue value in this moment in time. And part of the reason they accrue value is there's so much money at stake where if a company gets it wrong ripping them out, you're talking about potentially hundreds of millions or billions of dollars. You're going to be risk averse. Yeah, exactly. I think for mid-tier, it actually comes down to what is the size of the opportunity and is that ever going to be the worthiest opportunity to rebuild inside of a company? I'm trying to even think. In an enterprise, let's just say there's some mid-tier SaaS that is a few million bucks a year. [32:23](https://realalpha.app/v/Vki6cklFqxo/?t=1943) I think the first question is, from a percentage of revenue or cost perspective, is that going to ever move the needle enough to basically allocate engineer or resource to? I think it's going to be case by case. I think for some enterprises, the answer may be yes. For other enterprises, the answer may be no. I think more of it is going to be the thing that gets companies outside of capability. Like, you know, we talk about how I forgot who created the repo. That is basically the reverse engineer repo. Yeah, exactly. Like everyone's talking about that right now. And even at 10X, like we built our own like CRM from scratch. [33:05](https://realalpha.app/v/Vki6cklFqxo/?t=1985) What did we build? Our OS we built from scratch. And like one thing I think we realized and every company realizes this is one of the beauties of software is not just the utility that you get from it. [33:17](https://realalpha.app/v/Vki6cklFqxo/?t=1997) **Alex Lieberman:** It is all of the headaches you don't have to think about in maintaining it and improving it when it's not in your core. [33:24](https://realalpha.app/v/Vki6cklFqxo/?t=2004) So I think it basically comes down to, is the size of the prize big enough to justify the cognitive load that is going to take from some amount of resource in your company to nurture something that is not core to your business? Evidently, like I think for your business, the difference of basically $39,000 a year, you're implicitly saying that it is worth it. It is not just about costs. That's defense. I also think about the offense of like, no, actually our experience could be a lot better for literally every party concerned. Totally, totally. And so, yeah, I think part of the thought there is actually just, there's a significant, basically it's not just that. [34:08](https://realalpha.app/v/Vki6cklFqxo/?t=2048) So there's two pieces to it. One is like, not just we're being overcharged for a product, but it's like the product has been mediocre for a long period of time. what would the 10x better version of this product look like? Because we are the customers and we can iterate with like minutes and not months. Totally. I think, I actually think what's interesting is I think startups and like even like high growth, mid-sized companies can be nimble in that way. I think for enterprises where like they're so entrenched in a certain capability, rarely is that going to be the thing that, rarely are they going to say like, oh we can create a 10x better version of this sass we use and we're going to spin it out potentially as its own thing or whatever it is like i just don't think enterprises function that way for sure yeah yeah it's been discussed i actually i was i would say that you know you talk about automation uh the the more spicy thing is are they automating humans right and like once everything is automated and and smooth sailing uh do we you know replace those jobs uh i don't know how much you can speak to that. Yeah, I mean, I think there's two pieces here. I really deeply believe in the like, AI will produce good for humanity. It's going to augment people's jobs, [35:18](https://realalpha.app/v/Vki6cklFqxo/?t=2118) it will absolutely change and displace. But I think it's more of a movement as opposed to just like, cutting off work. Yeah. I think A16Z posted something about how like call centers are truly now on the decline and AI starting to handle more and more of that work. And ultimately, if that is a skill you've built up your whole career that is devastating to have something come in and take that function over. That being said, part of what we do when we're talking to customers about, hey, an agent is now going to do this work is actually around reallocation. I don't think any business would claim like once we have this solved, there's nothing for this group of people to work on and try to solve. [35:59](https://realalpha.app/v/Vki6cklFqxo/?t=2159) So our approach really is about what can we replace and then what are the next biggest problems to look at and i'd make an argument in most large companies that that cycle kind of continues forever there's always the next problem if i think about how we've grown infinite demands there's always been another problem and actually taking those people that may have this like super deep subject matter expert and bringing them above water to start to maybe work more across teams or like think about parts of the business less so on like the level of a sell in a spreadsheet and more about how these teams are working together. Ultimately, in the long run, I think that keeps most people there and allows those businesses to become more nimble and actually tackle like the big meaty problems and not the like drudge of most people's day to day when they you, you know, when you're thinking about someone that's in like a finance division, they're mostly in spreadsheets, but how can we elevate them to think strategically about the business and where money is allocated and things like that. Yeah. And I also feel like when we have these conversations around like job displacement or job reallocation, we talk about it as this [37:10](https://realalpha.app/v/Vki6cklFqxo/?t=2230) thing like happening to people. And one of the reasons I don't like that is I feel like it doesn't give enough credit to people to like have agency in their own lives. Meaning I think you can adjust. Yeah. People really should be thinking like when I think about like enablement or people re-skilling, I think it is the job of an employer because they want to drive growth in their business, but it is also the job of an employee. And I think when people say that AI is going to replace all of the work that people don't enjoy doing, I think that's a little bit [37:41](https://realalpha.app/v/Vki6cklFqxo/?t=2261) **Shawn Wang:** of like cope for the reality of just people are going to be put out of jobs because a lot of people don't have the luxury of liking or not liking a job. They do a job to make money. [37:50](https://realalpha.app/v/Vki6cklFqxo/?t=2270) And also I think people, to the extent that it's possible, should have agency in saying to themselves, okay, what is the stability or replaceability of my job over the next few years? And if I am to try to skill up in a way that I'm better protected, what does that look like? People have agency to say that to themselves and actually take action. Yeah. Yeah. I think that's a good message for people to reflect on. The last piece I wanted to touch on before I let you guys go is just New York. You know, greatest city in the world, according to some people, some people. What's the mood generally among the people they talk to, whether it's at work personally, what's the feeling, right? Like I live in San Francisco all day long. Everyone's just in agents all day long. Like, you know, it's the discussion is instincts or grok or muse. [38:36](https://realalpha.app/v/Vki6cklFqxo/?t=2316) It's not like, are you, do you like AI or not? Right. And I feel like I want to step out of that bubble for a bit or, but it's still New York. It's still progressive. It's still forward looking. Yeah. I mean, what I'd say is I feel like basically I would just say there's a bubble in New York that is a smaller bubble than exists in SF. If SF is like a big bubble, I think we at 10X feel like we are in a, what I would say is like a microcosm of San Francisco because of the people that self-select to work at the company. And then also the companies that we're partnered with and like the people that we engage with in New York. But I would for sure say also it is true that it is a smaller universe than like in SF, when you go to SF, it feels all-encompassing where it's like everyone everywhere you look there is someone who similar to like aie how i was saying earlier when i went on stage like if people go on twitter or x because they feel like they're on edge city they're living in the future like aie is like that on steroids like i feel like sf is similar to ai i mean it's like i probably like the the overlap of aie uh E generally and people who have lived at some point in their life in SF is probably a very [39:49](https://realalpha.app/v/Vki6cklFqxo/?t=2389) large overlap. We're roughly 1% of the city now. Actually? For World's Fair, yeah. That's crazy. Yeah. But like to that point, I just think like it is in the culture and the DNA of SF where a very big population of people live in the future. I think you can find that group in New York and I think we at 10X have found that people And part of our goal is to drive like AI as a hub or New York as a hub for AI in the U.S. because there is so much commerce here that like it is going to be a place where diffusion of the technology actually has such a meaningful impact on the GDP of this country. I think the bubble is smaller now and we take it almost as like a charge of ours to increase our bubble. [40:31](https://realalpha.app/v/Vki6cklFqxo/?t=2431) Maybe I don't know that it's possible to get to SF size, but I do think we can have a meaningful impact on getting more people to live in the future here. There's a physical, like you see it on the billboards. It's slowly creeping in. Oh, totally. People used to, remember the friend campaign where people would like spray all sorts of graffiti on that? I think that was like a little bit of an intentional rage bait. But now you have the unironic AI billboards and people are like, I guess that's part of life now. 100%. Yeah. So I think it's like, I think there's an opportunity to bring kind of like the edge and like the, just like the living in the future nature that SF has created because of just like the economic climate that exists there because of venture, because of tech companies, then because of the people that are attracted there. But I also think of us as like, we're in a, it's weird, we're in a smaller pond for AI, but I would argue we're in as big of a pond, if not a bigger pond, for commercial value. [41:30](https://realalpha.app/v/Vki6cklFqxo/?t=2490) Yeah, totally. And so I actually... The way I put it is the reason I do AI in New York is I bring all the people from San Francisco to come here and meet their customers and also get a reality check. Totally. And I actually think there's a really interesting arbitrage there. Yeah. Yeah, totally. Okay, awesome. Call for hiring. I know you guys are always hiring. Yep. What are you hiring for? What's changed? Yep. We're hiring for deployed engineers at 10X just go to 10x.co. We're also always hiring AI strategists, which are basically technical consultants who are leading client engagements. And the team is 75 strong in New York City. [42:06](https://realalpha.app/v/Vki6cklFqxo/?t=2526) We're continuing to make New York City our home. And then if you are an enterprise who is looking to actually get AI native and you've tried it, it's not working, hit us up. I promise we'll do better work than, not going to name them, but a lot of other traditional consultancies. Ooh, spicy. CJ? Yeah, I mean, I echo all of that. I'm particularly looking for engineers. What's a special project? What is that? A special project involves anything outside of kind of our client work. So I think about my brain right now. It is building that meta harness and system internally that basically it's like find the bleeding edge of what the models can do and give them tools to augment how well they can do it. [42:51](https://realalpha.app/v/Vki6cklFqxo/?t=2571) And then also internally working on the platform that Alex mentioned, big goal there, diffuse these background agents and actually get them out to enterprises in a way that's truly, truly helpful to them. And solve a lot of those like messy IT problems that we hit on earlier. Dude, when you have some solutions, we take guest posts on the Latent Space blog. I'd love to welcome some sharing of knowledge. But otherwise, thank you so much for your time. Thanks, man. And good to see you again in New York. Yeah, likewise. Appreciate it. Once a year. Once a year. Hell yeah. Thank you.