# Why FLUX 3 Trains on Images, Video and Sound — Andreas Blattmann & Dustin Podell, Black Forest Labs

Markdown copy of https://realalpha.app/v/8cGfJOU3AlQ/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

Andreas Blattmann and Dustin Podell argue that training across images, video and audio can build visual intelligence for generation and robotics.

- Show: Latent Space
- Hosts: [Shawn Wang (swyx)](https://realalpha.app/people/swyx/) & [Alessio Fanelli](https://realalpha.app/people/alessio-fanelli/)
- Guest: [Andreas Blattmann](https://realalpha.app/people/andreas-blattmann/) · Black Forest Labs
- Guest: [Dustin Podell](https://realalpha.app/people/dustin-podell/) · Black Forest Labs
- Published: Nov 16, 2026
- Length: 1 h 38 m

## The best minutes

- [57:12](https://realalpha.app/v/8cGfJOU3AlQ/?t=3432) [A small dataset unlocked general image editing in hours](https://realalpha.app/s/a-small-dataset-unlocked-general-image-editing-in-hours/) (4 min)
- [34:53](https://realalpha.app/v/8cGfJOU3AlQ/?t=2093) [A video model learned enough 3D to fine-tune in 1,000 steps](https://realalpha.app/s/a-video-model-learned-enough-3d-to-fine-tune-in-1-000-steps/) (3 min)
- [40:12](https://realalpha.app/v/8cGfJOU3AlQ/?t=2412) [Training should focus compute on informative denoising steps](https://realalpha.app/s/training-should-focus-compute-on-informative-denoising-steps/) (5 min)

## Also worth watching

- [1:24:25](https://realalpha.app/v/8cGfJOU3AlQ/?t=5065) [BFL aims to crack one-shot robot learning by 2027](https://realalpha.app/s/bfl-aims-to-crack-one-shot-robot-learning-by-2027/) (8 min)
- [1:13:06](https://realalpha.app/v/8cGfJOU3AlQ/?t=4386) [Autoregression is more data-efficient; diffusion can sample faster](https://realalpha.app/s/autoregression-is-more-data-efficient-diffusion-can-sample/) (5 min)
- [27:39](https://realalpha.app/v/8cGfJOU3AlQ/?t=1659) [The SDXL refiner came from a different, unreleased model](https://realalpha.app/s/the-sdxl-refiner-came-from-a-different-unreleased-model/) (6 min)
- [47:01](https://realalpha.app/v/8cGfJOU3AlQ/?t=2821) [BFL was founded to capture value its models had created](https://realalpha.app/s/bfl-was-founded-to-capture-value-its-models-had-created/) (4 min)
- [1:05:17](https://realalpha.app/v/8cGfJOU3AlQ/?t=3917) [More than ten image references are hard to review](https://realalpha.app/s/more-than-ten-image-references-are-hard-to-review/) (5 min)

Skip the 17-minute opening introductions and career histories and the Deepgram sponsor break.

## Full analysis: Andreas Blattmann says about 1,000 steps produced a 3D model

- Blattmann estimated about 1,000 fine-tuning steps yielded a 3D model.
- That suggests transferable spatial structure, but does not measure reliable physical understanding.
- Teams should test what transfers before funding shared image, video, and action systems.

Its 6 sections, what it means for you and what to watch are on [the page](https://realalpha.app/v/8cGfJOU3AlQ/#analysis), free with an account:

- [36:53](https://realalpha.app/v/8cGfJOU3AlQ/?t=2213) About 1,000 steps produced a 3D model
- [24:14](https://realalpha.app/v/8cGfJOU3AlQ/?t=1454) Observation builds priors but does not establish robot control
- [18:34](https://realalpha.app/v/8cGfJOU3AlQ/?t=1114) One multimodal model remains a commercial thesis
- [44:27](https://realalpha.app/v/8cGfJOU3AlQ/?t=2667) Compute allocation and data quality shape transfer
- [59:30](https://realalpha.app/v/8cGfJOU3AlQ/?t=3570) Small editing datasets can unlock existing model structure
- [1:26:13](https://realalpha.app/v/8cGfJOU3AlQ/?t=5173) Robotics still needs demonstrations before one-shot learning

## Summary

Andreas Blattmann co-invented latent diffusion and co-created Stable Diffusion before co-founding Black Forest Labs. Dustin Podell moved from software development into research at Stability AI; he was lead author of SDXL and led FLUX Kontext.

Blattmann and Podell say Black Forest Labs is extending image generation toward visual intelligence, using a shared model trained across images, video and audio, with action prediction for robotics. Blattmann argues that learning from 2D video can build implicit 3D understanding, and points to an experiment that produced a 3D model after 1,000 fine-tuning steps on 3D data. Podell says FLUX Kontext’s editing ability grew from a small, high-quality dataset and distillation. They describe autoregressive models as more data-efficient to train and diffusion models as faster to sample after distillation, while arguing that neither approach has won out.

### Minute by minute

- [4:22](https://realalpha.app/v/8cGfJOU3AlQ/?t=262) **Diffusion took root in Germany.** Blattmann says he and the early team began studying diffusion in Germany, drawing on their physics backgrounds; work at Heidelberg led to latent diffusion and later Stable Diffusion. He says they founded BFL in Freiburg, near the Black Forest, rather than relocating to a tech hub.
- [9:22](https://realalpha.app/v/8cGfJOU3AlQ/?t=562) **Stable Diffusion drew Podell into research.** Podell says he had spent about 10 years as a software developer before Stable Diffusion’s release prompted him to leave his job. He joined Stability AI as an engineer, then learned by reading papers, experimenting with models and working with researchers in the open-source community.
- [17:16](https://realalpha.app/v/8cGfJOU3AlQ/?t=1036) **Video can teach implicit 3D.** Blattmann says BFL wants models to learn intuitive physical knowledge from natural signals, not just language. He argues that 2D video can teach implicit 3D understanding, such as how the unseen back of an object might look, and says interaction could further develop that understanding.
- [27:31](https://realalpha.app/v/8cGfJOU3AlQ/?t=1651) **SDXL split structure and detail.** Podell says SDXL brought multi-aspect-ratio training and more improvements into the base model. He describes its refiner as a separate diffusion model that added high-frequency detail, while Blattmann says the work helped show how structure and texture emerge at different points in denoising.
- [33:57](https://realalpha.app/v/8cGfJOU3AlQ/?t=2037) **Temporal data matters for video.** Blattmann says latent diffusion naturally extends to video, but stresses that data quality and motion matter as much as the algorithm. He cites an experiment where a model pretrained on 2D video produced a 3D model after 1,000 fine-tuning steps on 3D data. The guests say temporal modeling helps prevent flicker.
- [39:56](https://realalpha.app/v/8cGfJOU3AlQ/?t=2396) **Flow matching targets useful compute.** Blattmann says SD3 marked BFL’s shift to rectified flow and transformer architectures. He describes flow matching as a simpler way to model a path between distributions, which can support faster sampling. The guests also discuss allocating training compute to the parts of the denoising trajectory that matter most.
- [46:50](https://realalpha.app/v/8cGfJOU3AlQ/?t=2810) **BFL began with a small team.** Blattmann says 13 or 14 people started training FLUX.1 after a founding offsite in a French chalet, and the model launched three months later. Podell says the team also wanted to capture more value from its models than it felt it could at Stability AI. Blattmann said BFL had raised $450 million and that revenue had passed the last published figure of $100 million.
- [51:38](https://realalpha.app/v/8cGfJOU3AlQ/?t=3098) **Video was BFL’s longer-term thesis.** Podell recalls a photo-personalization app that drew a surge of interest, and says the team saw image generation as a broad technology rather than a single use case. Blattmann says BFL had long viewed video modeling as a way to understand how the world works; image generation gave the company a successful starting point.
- [54:54](https://realalpha.app/v/8cGfJOU3AlQ/?t=3294) **Kontext makes edits iterative.** Podell says FLUX Kontext began as a pitch to use a pretrained model’s spatial knowledge for general image editing. A small, high-quality dataset describing relationships in photos showed early promise after roughly four to six hours of training; the team then expanded the project and distilled the model. He says the goal was to move beyond repeated generation and selection.
- [1:03:13](https://realalpha.app/v/8cGfJOU3AlQ/?t=3793) **Data and evaluations depend on the task.** Blattmann says useful training data depends on the problem and stage, and teams must test the trade-off between quantity and quality; he notes that robots need egocentric data. Podell says FLUX.2 was trained to handle 10 reference images. He also describes Context Bench as an arena-style comparison and says teams should test models on real use cases.
- [1:13:06](https://realalpha.app/v/8cGfJOU3AlQ/?t=4386) **Autoregression and diffusion have trade-offs.** Blattmann says autoregressive models use training data more efficiently, while distilled diffusion models can generate in fewer steps. He sees strengths and limitations on both sides and says the debate is unsettled. The guests add that conversational editing depends on training and data, not only on an autoregressive architecture.
- [1:20:39](https://realalpha.app/v/8cGfJOU3AlQ/?t=4839) **A shared model predicts robot actions.** Blattmann says BFL pretrains on images, video and audio, then adds robot action states so the model can predict action chunks from video and instructions. He says the approach works with real robots and in simulation; one industrial task required four hours of fine-tuning. The guests say a 7B version is intended for on-device, real-time use.
- [1:30:07](https://realalpha.app/v/8cGfJOU3AlQ/?t=5407) **BFL expects robotics to expand in 2027.** Blattmann predicts commercial manufacturing applications and progress toward one-shot learning, pointing to industry partnerships in Germany. Podell separately speculates that niche home robots could become useful later in 2027, while noting that this is only a prediction.
- [1:32:12](https://realalpha.app/v/8cGfJOU3AlQ/?t=5532) **Photo restoration found an audience.** Blattmann says BFL wants FLUX used in professional workflows to improve productivity. Podell says a surprising use case was restoring and colorizing degraded family photos while preserving their similarity to the originals. The guests say it is difficult to predict which uses will catch on.

## Transcript

Latent Space, hosted by Shawn Wang (swyx) and Alessio Fanelli, with Andreas Blattmann and Dustin Podell.

From the transcript Latent Space publishes with the episode.

[0:00](https://realalpha.app/v/8cGfJOU3AlQ/?t=0) Happy to bring to you our episode with Black Forest Labs. Our guests today are Andreas Blattmann, he also goes by Andi, who is co-inventor of latent diffusion, the technique that eventually got scaled up by Stable Diffusion and Stability AI, and they eventually split with a bunch of, uh, the core Stability AI image gen folks to form Black Forest Labs. And Dustin Podell, who I also met with Anj, uh, at one of the stories that we tell, and, uh, who worked his way up from just being a regular engineer, a freelance, like, web developer person, into a model and research lead at Stability AI, and now, uh, Black Forest Labs, creator of SDXL, Control LoRA, post-training lead on SD3, Flux Ultra, Flux.1, and FLUX Kontext.

[0:41](https://realalpha.app/v/8cGfJOU3AlQ/?t=41) And we go into all of these details. Um, as you may know, sometimes we're very honored to have an extended history with some of our guests, and this is one of them. You might have heard in our Anj Mittal episode where we talked about introducing each other at one of the NeurIPS's [smacks lips] and meeting Black Forest Labs for the first time. And, uh, this was right after Stable Diffusion. But what you may not have heard, and we only tell this story in this episode, is how latent space itself was started because of Stable Diffusion, directly, uh, crediting the launch for me basically quitting my job to go all in on AI based on what we saw.

[1:19](https://realalpha.app/v/8cGfJOU3AlQ/?t=79) And today's guest, Dustin, also [chuckles] had that experience, and so did Vibhu. And so we have a very, very strong tie with the team that basically led up to Black Forest Labs, and now we're also interested in what's next for them. Like, they have created some of the best open and closed weights models for image generation and video generation. I think especially FLUX Kontext, if you're not aware of image editing models, I do think that this is in the ex-in the limit one of the things to fully replace Canva and, um, and Photoshop and a bunch of other workflows and editors that you might use in a normal creative workflow.

[1:57](https://realalpha.app/v/8cGfJOU3AlQ/?t=117) It is not going to replace the top end of the market. But for regular, normal, everyday edits, why would you not want anything else other than to query a model and ask it to do, uh, image editing for you? Uh, we do this obviously with this kind of video editing. It's all Clanker done, and I do review it, and they're also moving into robotics. And so this ties in with a miniseries of model labs that have been pursuing this direction, building world models from image gen, like with Runway, uh, as well as, um, video models and now robotics models. And so I think a lot of things rhyme in a lot of our recent episodes, like even the artificial analysis episode that we're gonna release soon, they are also going into robotics, and, like, it seems pretty s- obvious that when you see a whole bunch of smart people who are at the frontier all leaning one direction, that you're probably six to 12 months out from some kind of really big breakthrough.

[2:51](https://realalpha.app/v/8cGfJOU3AlQ/?t=171) This has been a consistent theme over my last four years of podcasting. It's not gonna change now. So look out for it. Um, that's the [chuckles] most obvious call-out I can make. But there's also a lot of interesting lore for people interested in the general diffusion space. Uh, we're gonna d- do a little bit more over time. I, I'm thinking about an AI- AIE LA, where we talk about the generative media field and the creative arts and how, uh, that complicated relationship is evolving, um, and I think maturing, to be honest. And finally, for those longtime listeners, uh, we are starting to open up to advertisers, people that we really admire and respect and wanna showcase what they're working on.

[3:30](https://realalpha.app/v/8cGfJOU3AlQ/?t=210) Um, the first one was with Supabase, which I think a lot of you enjoyed, and the second one today is brought to you by Deepgram, and you'll hear more from us at the midpoint of the episode. [upbeat music] You should be showing a robot an action one time, and then it should be able to do this, and this would be truly a superhuman intelligence like- At Stability that we couldn't essentially leverage the value of what the models could be as well as the mistakes of not being able to actually capture the value of the models themselves. We raised all in all four hundred and fifty million, uh- Oh my God... dollars of funding so far. Yeah. Currently, the last, uh, that was published in, uh, December was one hundred million in revenue.

[4:04](https://realalpha.app/v/8cGfJOU3AlQ/?t=244) BFL was not founded to be an image generation lab. Video modeling is actually the central means to understanding how our world basically works. It was maybe, like, six hours of training, four hours of training. It, it was, it was a very, very short training run, and all of a sudden, like, I saw what felt like a clear generality coming out of it, 'cause it was [upbeat music] Okay. Finally, our BFL pod. Welcome, guys, Andreas and Dustin. [chuckles] Great to be here. Thank you so much. Thank you. Yeah, definitely thank you for having us. I know we've chatted about doing this for a long time. Yes.

[4:35](https://realalpha.app/v/8cGfJOU3AlQ/?t=275) So we've had Anj on a couple times, and that's where I met you, at Vienna, I think one of the ICMLs or- Was this, like, one of the, like, was, was the, like- The-... rooftop, uh, event, or, or what was it? There was a year where there were two Viennas back to back. Yeah, yeah, yeah, yeah. That was first ICLR, and then it was ICML. Yeah, yeah. I don't know how they do this thing. [laughs] Yeah. Anyway, anyway, uh, and, and at the time you guys had-- were, like, just about to, uh, launch BFL. Like, we had a dinner and, like, you were like, oh, and thinking about what's next after Stability and, uh, all that.

[5:06](https://realalpha.app/v/8cGfJOU3AlQ/?t=306) So, uh, anyway, congrats. How do you like, guys, like to introduce yourself? Maybe we'll sort of get a crisp, uh, self-intro up front. So I'm Andi. I'm a co-founder of Black Forest Labs. Been developing diffusion models for, oh, wow, I think seven years now. Uh, met Robin and Patric way, way back in the day. I think it's, yeah, seven years now since we met at Hei- in Heidelberg. Uh, I'm from Germany o- uh, originally. And there we started looking into, yeah, just diffusion modeling for images. Uh, we were all having a kind of a physics background, um, in some sense, and this, this paradigm of just, like, mapping this general physical phenomena of diffusion to machine learning and generative modeling had a, a kind of a, a philosophical kind of beauty to us.

[5:50](https://realalpha.app/v/8cGfJOU3AlQ/?t=350) That's why we got into this. And then, yeah, developed a lot of the, I guess, fundamental algorithms that are powering diffusion models today, or image generation, video generation, and also action prediction today. I guess we'll talk about this later We'll talk about it. But, um- Yeah, uh, this of, of a kind related to... It's not immediately obvious to people, image generation to action prediction. Yeah. But we can talk about that. Uh, and then one, one thing I did wanna comment is a lot of people don't know the relationship. Black Forest is a forest in Germany? It's a mountain range, actually. Ah.

[6:19](https://realalpha.app/v/8cGfJOU3AlQ/?t=379) But, um, yeah, it's a, it's a region in Germany, basically. And you named it after that. And finally, Robin and me, we f- we first met during our PhD, and then we figured out that we were actually born, I don't know, 10 miles away from each other. Um, and that was, that was pretty fun, and then eventually that led us also, uh, to come back to Germany. Beautiful. [laughs] All of Europe looks like this. Yeah. You can see the lab right in the middle there. [laughs] Yeah. Yeah. Yeah. Um, so- So you named it because you're from here. Exactly. We named it because actually, um, we founded this research lab, and back in the day, we could basically pick where we wanted to do it.

[6:56](https://realalpha.app/v/8cGfJOU3AlQ/?t=416) No one said, "That's okay. You have to come to, say, SF or, or whatever place." So we decided, okay, we wanna be actually starting this at the place where we were born in Freiburg, which is, like, at the foot of the Black Forest, basically, or a city in the Black Forest, and that's how it came, and that's how, uh, then the company also started there. Yeah. ComVis, also Heidelberg? ComVis was in Heidelberg. Exactly. That's where we met. We then later moved on to Munich. That was all- Which is the team that did latent diffusion... through our PhD. Exactly. And, and that is when we developed this latent diffusion algorithm. Yeah. I really like your s- uh, name by the way, your podcast name- Yeah... Latent Space. Uh, I, I, I, I would actually be curious how that came together. Uh- I can talk, I can talk about it- Yeah... but, yeah, go ahead. Uh, yeah.

[7:35](https://realalpha.app/v/8cGfJOU3AlQ/?t=455) Um, and exactly, we developed then this latent diffusion algorithm at the university, and then based on this research project, we scale out the algorithm which led to Stable Diffusion, um, then effectively, and yeah. Since then, it's been a, a constant, uh, journey on a rocket ship, you could say, which is super, super fun to, to have. Yeah. And we've met amazing people like Dustin as well, um- Yeah... who's, who's one of the, the, the kind of oldest and, and, like, mo- yeah, mo- most, I think, relevant researchers that we have. But yeah, we'll, we'll learn more about that [laughs] in a- We'll, we'll gas him up la- later.

[8:09](https://realalpha.app/v/8cGfJOU3AlQ/?t=489) [laughs] Uh, I think latent diffusion and just g- generally, like VAEs as well, and, and then for us Latent Space, uh, is this general idea, and, and we- even our logo is, uh, actually a VAE, and it's this idea that you can translate things into spaces and then explore things that are adjacent to what exists today. Um, and then you can sort of retranslate back. Yeah. And I thought that that was, like, such a cool insight for creativity and generation that I never really understood until I studied Stable Diffusion. Like, the launch of Stable Diffusion changed my life. I started Latent Space because of Stable Diffusion. I quit my job because of Stable Diffusion. Oh, wow.

[8:42](https://realalpha.app/v/8cGfJOU3AlQ/?t=522) 'Cause I was like, "Okay, everything I know about AI is wrong because I can run this on my laptop." Yeah. And, like, if I was wrong about this, what else am I wrong about? Uh, so there, there's that. Um, but and then there, there's another relationship with, um, Terry Pratchett, which I, I don't know if people have read. Writer, right? Writer. He has this concept of L-space, uh, which is library space, um, where he has a, a orangutan that's, like, the librarian in Unseen University, which is wizard university, uh, and it basically is a library with all possible books. So if you have a library with all possible books, you can just theoretically go in there and find a book that is right for you, uh, and that is Latent Space. Yeah. Uh, so I originally called it L-Space.

[9:19](https://realalpha.app/v/8cGfJOU3AlQ/?t=559) Uh, one of my readers donated Latent Space, and then it became Latent Space. Amazing. [laughs] Dustin, like, uh, yeah, what was your journey into diffusion and AI in general? Yeah, yeah. So I, I came from a little bit of a different background than, than Andy and Robin and team. So I'm, I'm not from the German side, but I've still had a fantastic time working with these guys. And, uh, honestly, it's funny you mentioned that you quit your job when Stable Diffusion came out because, uh, the original Stable Diffusion I was not working with these guys yet, and it was actually a cause for me to quit my own job at the time.

[9:47](https://realalpha.app/v/8cGfJOU3AlQ/?t=587) So I, um- You caused mass unemployment. [laughs] Yeah. I'm actually curious, like, the numbers of people that just com- like, changed their life direction when Stable Diffusion dropped. I, I would actually be very curious to know kind of the branching effects from that. Nice. But I don't know, we can, we can theorize on it. [laughs] That's actually super interesting because here in SF, in the Bay, it had this huge impact, whereas in Germany- Mm-hmm... where we stayed back in the day, that day, people didn't really notice it because, like- Oh... the people were so, so far away from tech. Mm-hmm.

[10:17](https://realalpha.app/v/8cGfJOU3AlQ/?t=617) Like, there are a couple of people obviously that are, that have been involved in the open source, uh, community also in Germany, but the majority of people didn't really notice, uh, that drop in summer 2022- Mm-hmm... which was really interesting. And then you see what happens here, like he-- Yeah. That, that was... To, to us, that was really kind of super interesting. Yeah. But, but anyway, so I, I guess to give a little background on me, so I, I, uh, at BFL, I, uh, I'm one of the founding researchers that joined in with the, uh, we had a, a bit of a larger team that I'm sure we'll get into the history of kind of how that all came to be.

[10:48](https://realalpha.app/v/8cGfJOU3AlQ/?t=648) But my background is not actually traditionally in, you know, the ML space whatsoever. I was traditionally a software re-research- or, or a software developer for, you know, around 10 years or so. Early in 20- you know, around the 2020s, I became very, very interested in just, like, open source AI. I saw, like, the kind of the early work in GANs and other, you know, in generative media, and I was tracking with it. I was experimenting with it a little on the side. It got me quite excited. I mean, and I remember following originally the, the latent diffusion work 'cause actually I was, I was following that and quite excited about it.

[11:20](https://realalpha.app/v/8cGfJOU3AlQ/?t=680) And then, uh, the one, you know, one of the things that kind of tipped me off to go, "Okay, maybe this is something a little bit, you know, a little bit more serious," is actually, uh, shout out to, uh, Midjourney V1. Wow. Um, back when that came out and I, I remember thinking, like, "Okay, this, this is actually going in, in a direction. I, I can see this being quite impactful." And then the one big moment, the big drop that really changed everything for me, as we talked about, was the Stable Diffusion release. And at the time I was working at a software studio down in Los Angeles. Stable Diffusion came out, and basically that day I turned in my resignation- [laughs]... and said, "Okay-" But what was, what was the plan? Like, what did... [laughs] The, the plan?

[11:57](https://realalpha.app/v/8cGfJOU3AlQ/?t=717) So, so the way, the way that I saw it, the way that I, like, envisioned it at the time- I saw this thing that felt like, it, it literally felt like a magical thing that came out, and I viewed, okay, the people that are working on this must be like arcane wizards, and I need to do whatever I can to understand like what magic or what runes or what things that they did to imbue in this to make this happen and let this go out to the world and have this, have this impact. It wasn't like I was just totally disconnected. At the time, I was in the Stable Diffusion Discord. I actually had friends in there. I was, I was kind of interacting with the community.

[12:27](https://realalpha.app/v/8cGfJOU3AlQ/?t=747) So when I, when I- So what, what, was it viewed as basically open source Midjourney, or like, like do people not make the connection? There, there was... I mean, I, I was in both. I remember there were a lot of kind of, uh, discussions around this, like, oh, Stable Diffusion was, it's having such a... You know, there, there was a lot of discussion, and I remember the Midjourney Discord and the Stable Diffusion Discord around that time around the, the impacts it was having. I mean, it, it's been a long time so I can't remember the exact- Yeah... nature of it. I don't wanna speak on the, you know- Midjourney's doing fine. Like they're doing- They're doing good.

[12:52](https://realalpha.app/v/8cGfJOU3AlQ/?t=772) I mean, they're, they're, they're doing great. [laughs] Um, but for me it was more just like I was very interested in the fundamental technology, and then the fact that I could now run this at home. And then really the thing I saw was then just this community of developers actually like taking it home, developing on it. I was hacking away at it. I was meeting a bunch of other really, really smart, great people, some of which have, are actually at BFL now that have, have come along for the journey. That moment of just community interaction and, and learning and developing and building together was s- really an impactful moment for me.

[13:23](https://realalpha.app/v/8cGfJOU3AlQ/?t=803) Lucky, luckily enough for me, I was able to find a kind of a job opportunity to get myself into Stability at the time, where, uh, you know, th- this, this was starting. And it, it, I didn't actually start working with these guys right away. It took me a while to kind of, to kind of work my way up to that, and, and we can go into that history later if we want. But- 'Cause Stability had a bunch of different m- uh, teams, right? It was a- Stable LM, Stable Music. There was- Yeah, Harmoni, yeah, the audio team Yeah. It, it was, it was like, I don't know, the way I would kind of describe it is like, uh, if you grabbed a bunch of... Uh, Discord was kind of the space that a lot of this happened at the time. [laughs] There were...

[13:55](https://realalpha.app/v/8cGfJOU3AlQ/?t=835) I don't know, I don't know why the whole AI- It's like if a Discord had a- It was a-... a physical manifestation [laughs] It was, yeah. It was like- Effectively it was a melting pot of like very different people that came together who were just like interested in this- Yeah... emerging new technology that came obviously, and that's also how we, uh, like this is one of the m- most exciting things about our team. Yeah. Our founding team is, it comes from very different backgrounds. Like we have this hardcore academia research branch, and then we have people who were from the open source community, like, like Dustin, um, who actually got via a very different kind of trajectory into this space.

[14:28](https://realalpha.app/v/8cGfJOU3AlQ/?t=868) But I think this is what makes our team so strong. Yeah, absolutely. We have these diverse backgrounds, and there's, yeah, there's rarely a team I think that, that, that has this history, also this, the kind of joint history together. Yeah, yeah, yeah. And this is just amazing that, that like- Yeah. Yeah... yeah. And we can go into more of your other co-founders first. Yeah. Uh, I, I did wanna... So I was also in that space, right? Like literally, you know- Yeah, yeah... I was also very influenced by Stable Diffusion. I did the wrong thing. [laughs] I basically went down the path of ComfyUI where I build UIs for Stable Diffusion. You seem to have done the right thing where- [laughs]... you went to research.

[14:59](https://realalpha.app/v/8cGfJOU3AlQ/?t=899) Well, it, it's funny you mentioned Comfy, 'cause ki- like I used to work directly with... I mean, Com- Comfy joined somebody that I- We've had him on the pod, yeah. Yeah, yeah, yeah. And he's- And I, I think he, he moved here. Yeah, yeah. Yeah, yeah. He's, he's, he's like my neighbor. How does a... That you, he's a researcher. How do you become a researcher? [laughs] I'm someone that's like, when I get interested in something, I am obsessive to the point of I cannot put it down until I pull it apart and figure out every aspect of it. And so to me, I'd already been experimenting with pulling the models apart, talking to a lot of different people in that open source space, so not just like developers, but other researchers.

[15:30](https://realalpha.app/v/8cGfJOU3AlQ/?t=930) I mean, remember talk about another Discord, the, you know, the Eleuther Discord back then. At that time, I met a lot of people who I w- I was just learning from, and I was reading a lot of papers and just trying to uptake and understand for myself. Joining Stability was also another major point for me. Originally, I didn't join as a researcher. I joined as an engineer. Mm. But all of my free time I spent with essentially any researchers I was able to make friends with, you know. Shout out to, uh, Dr. Scott Hawley there who worked on the audio team, who mentored me quite a lot there. And at some point, I was able to, to sneak my way onto the, uh, onto the cluster there and got myself access to, you know, some idle compute.

[16:06](https://realalpha.app/v/8cGfJOU3AlQ/?t=966) [laughs] And I, I, I really just like, I treated it like I treated how I learn most things, which is just open source hacking, trying things, reading papers, trying to implement different pieces of work, and just seeing how, I don't know, how models trained, how they learned and, and, and I mean, that was really it. Yeah. Like there was, there was no magic to it other than just I was excited [laughs] and I just wanted to learn about it. I wanted to tear it apart until I understood how it worked. I mean, this is open source. It is better. Yeah, yeah. I think that that's generally how you learn best, just trying out things- Yeah...

[16:32](https://realalpha.app/v/8cGfJOU3AlQ/?t=992) and then like- And he was also Eleuther? Yeah, yeah. Yeah. Affiliated, but not on the, uh, image gen side. But back then everything was like very hackable, right? Yeah. The, the delta between open models was like pretty drastic between what was closed and good, right? So people on language model side were doing post-training. That's where like a lot of the news guys started. Mm-hmm. Uh, in this, you know, you can look at early Stable Diffusion- This look, this looks so fast and then like it, it-... uh, and, and like where we get to today. Yeah, yeah, yeah. So it's pretty crazy. But you both had pretty, you know, impactful roles. Like you did SDXL. Mm-hmm. You did latent diffusion. We'll get through all the history, but for people that haven't super followed the space, where are we today?

[17:11](https://realalpha.app/v/8cGfJOU3AlQ/?t=1031) Like what's the, what's the state of things? I know Flux.3 video is out. Uh, Flux.3 action, we're now like 7B doing robotics. What's the latest in this space with you guys? We started as an image generation lab- [laughs]... when we started Black Forest Labs, right? As a kind of legacy of Stable Diffusion and the likes. Um, and that has been I think our, our thing for the first year, but then we realized, okay, you actually, image generation is nice, but this technology can unlock so much more. Mm-hmm. And that's when we slowly went into what we call visual intelligence, which is basically trying to get our intuitive intelligence that we people have when we s- say when we catch a ball and just understand physics without actually knowing how to mathematically describe it or something, or when we grab a bottle and just know exactly how to do this because we've done it 100,000, maybe not 100,000 times, I don't know, but a lot of times, uh, in our, in our childhood, right?

[18:07](https://realalpha.app/v/8cGfJOU3AlQ/?t=1087) We wanna be transforming that Knowledge and that form of intelligence, which is very different from language model intelligence, into models. And learning based on pixels, which means not only images, but also videos, and maybe also audio, say not pixels, but natural signals is a, maybe a better term- [laughs]... that really unlocks this. And that is what we realized, and we realized, hey, we have the algorithm to actually do this, which is diffusion or flow matching. And that is when we just said, "Okay, we need to extend from image generation to basically all the natural, uh, representations," which led to FLUX 3 basically, which is one model that learns multimodal from images, videos, audios, also actions.

[18:48](https://realalpha.app/v/8cGfJOU3AlQ/?t=1128) We can talk about how this comes together later. To really get this, to, to keep it with Daniel Kahneman, I don't know if you read- Thinking, fast and slow... Thinking Fast and Slow. It's a kind- system one intelligence. It's not system two intelligence. Mm. And you can maybe say- Oh, my God... maybe, maybe- Everyone uses system one models now. [laughs] May- may- maybe, maybe language modeling- The original, yeah... is system two, but we are focused for now on system one, which is extremely important for everything that is intuitive and that requires real-world interaction. That's why at, on a high level, it's also so good for robotics. So that's been the journey, and now we're really focusing on that.

[19:20](https://realalpha.app/v/8cGfJOU3AlQ/?t=1160) And in the, in, in the end, obviously, um, it's about combining both of these systems into this broad, very, very broad way of intelligence that humans also have. I wanna go into sort of, uh, history and all, all these things. Uh, but just to also recap for the, for people at the front, you're already making a s- sort of the money as a generative media company, right? Exactly. Um, I don't know what the, the numbers are and, and maybe you can sort of recap there. I, I think you have, you have some- You raised, like, you raised $300 mil on- You raised a ridiculous amount of money... multi- yeah.

[19:48](https://realalpha.app/v/8cGfJOU3AlQ/?t=1188) Yeah. So- We, we raised, I think- Any re-... uh, all, all in all, $450 million- Oh, my God... of funding so far. Um, yeah. So there's way more than- A lot of it from Squa- Twitch and stability... you know, way more than stability. Yeah. Uh, any revenue milestones that you've, that you've announced? Kahneman Slow is currently- Haven't. Yeah... the last, uh, that was published in, uh, December, was $100 million in revenue. Yeah. We've crossed that since. I can, I can tell you that. [laughs] Clearly, clearly, clearly you've crossed that if you raised 450. Um- Yeah... yeah, exactly. So- So, you know, yeah. A- anyway, I'm just saying, like, you did not have to pivot. Great. And to be clear, we're not pivoting- Yeah, exactly... we're just extending. Extending. Yeah, yeah.

[20:17](https://realalpha.app/v/8cGfJOU3AlQ/?t=1217) Like, this, this model is really bu- it's trained on everything. It's, it's always been the plan. And obviously to, to create artworks, you also need this visual kind of intelligence. To create a, a video that we see here- Yeah... you need to know that, how physics play out. I, I need to know that the model needs to understand this is actually interesting. Uh, but, but the model needs to understand that the arm of this, say, this, this, this, uh, capybara or whatever it is, is not just falling away, that it has to follow some law that is described by physics. It's just not describing the law itself, it's just describing the output in a similar way that we humans don't understand physics, but we understand that if I let a ball fall, it just falls downward. Yeah.

[20:57](https://realalpha.app/v/8cGfJOU3AlQ/?t=1257) Well, I think there's, there's different, uh, concepts of a world model, and obviously that's something that, uh, we can get into, where can you learn... This is sort of the bitter lesson ver- version where you just learn on a bunch of input sources- Mm-hmm... or you have to actually physically structure the 3D model and, and kind of learn from that. No, no 3D model at all. Yeah. [laughs] Just learning on, ba- based on natural data. We, we also, we have a bit of tri- triangulation be- like, from our eyes wise, right? We have two eyes, and that gives us a kind of slight understanding of 3D perception. But effectively to a very large extent, even if we would, like, close one eye, we could still kind of learn a lot about the world.

[21:31](https://realalpha.app/v/8cGfJOU3AlQ/?t=1291) That is what these models are basically learning. Like, you just learn from natural data, 2D video data. That obviously has a lot of 3D understanding, because if you wanna generate a pan around an object, you need to be hallucinating how it looks from behind. And if I look at you, I have a pretty good understanding of how your back looks, basically. Yeah. Because I've just seen it before, right? So this is gives you an implicit 3D understanding, but it's really trained on kind of 2D videos. Yeah. I, I'm trying to steelman the counterargument- Uh-... the, the opposite argument, right? Like, this is like the Gary Marcus, like, "Oh, LLMs don't learn anything," or... I, I forget who are the famous critics of this kind of stuff. But basically, it's like, it's like I have never lived...

[22:10](https://realalpha.app/v/8cGfJOU3AlQ/?t=1330) I, I as a model, I only see on inputs. I've never-- I don't have personal experience. You can model my back because you are also a human. But b- and you can model other humans. But, uh, maybe I, maybe this analogy isn't, isn't exactly correct because I was, uh... Obviously, I'm sympathetic to your, your side of the story. But, like, I, I think that is how people criticize this approach to world modeling, which is that, yes, you can approximate closer and closer, closer, but you'll never quite get something core to due to the existence of the thing because you never modeled the thing. You modeled, uh, uh, the most likely, you know, uh, sort of, uh, constraints around the thing. Yeah, I mean, I, I think to some degree you could argue that we operate under the same principle, right?

[22:48](https://realalpha.app/v/8cGfJOU3AlQ/?t=1368) Like, you can say, like, that we are, you know... Okay, we have this embodiment and we're constantly, like, taking in data and, like, obviously we, we regularly update. But in some principle, like, we're still just taking in these, like, filtered signals from the world. I mean, there's all sorts of wacky things with how our, our eyes process and we have to, you know, part of our brain kind of has to hallucinate some of the info 'cause we have the blind spot in the center and, and, and other aspects of this. But fundamentally, like, I don't think there's that much of a difference between, you know, how we-- like we're, we're out here taking in this natural data and building this model of the world that then we're, we're, we're able to act on versus, like, these models also learning from...

[23:23](https://realalpha.app/v/8cGfJOU3AlQ/?t=1403) Obviously, it's, it-- the signals are different that they're, you know, okay, they're turned into, to, to pixels or, you know, into, into waveforms and processed through this other mean. But, like, the, the information is still there for the model to take in and, and build this, like, representation of the world that then you can essentially operate with. Yeah. And yeah. I was actually arguing this with Luca- Lukas Bayer. Uh, he's also a, a, um... He's, he's with, uh, Meta now, I think, formerly Google, um, and OpenAI. Created the bits. He's in, in Zurich, which is pretty close to Freiburg, and he's, he's coming over from time to time. We were, we were debating this, um, on, on human learning. Like, he was, you, you, he was-- we were saying, okay, we actually don't need interaction, which would be kind of the, I think s- well, what I would say is important for learning, like, higher forms of, of intelligence, it's like first it's observation.

[24:08](https://realalpha.app/v/8cGfJOU3AlQ/?t=1448) That's also how we humans learn when we are babies, right? We just lie around first and we, we observe things, and from that alone we learn so much. This is what, what you can actually Exactly map to what, how we are currently training these models. They just observe a lot of 2D videos, audio, and then they learn from it. And I think next step will also be getting, closing the loop to actually interaction with the world, because that's also in how- Mm-hmm... I, I think that that's very important in how, how, how we learn basically around the world. We can then tap and we see, oh, there's a correlation between I knock on wood and there's a sound that has occurred, right?

[24:40](https://realalpha.app/v/8cGfJOU3AlQ/?t=1480) I, I can just like manipulate the world then and to, to understand it better. That's how I think how humans are mainly learning- Mm-hmm... at least how [laughs] I personally learned. Um, and he was actually saying he thinks the, the majority is observation, which I found very interesting because, uh, that would, that would... If, if that would be true, then we just need to do what we're currently doing more, and that could basically unlock, uh, an insane amount of intelligence basically. Yeah. I wonder how much of this is like emergence with good priors, right? We've seen adding new modalities to, say, language models.

[25:10](https://realalpha.app/v/8cGfJOU3AlQ/?t=1510) You don't get like the full benefit of, you know, doubling your performance since you've added vision to a language model. But if you have good priors, um, you know, is there light benefit? Is there big benefit? It's, it's interesting to see. I think there's a lot of technical stuff that'll be interesting to walk through, like Gaussian splatting was a big thing. World Labs did a lot of 3D data. Where are the trade-offs between video generation output versus what else? But I think something fun for listeners of the podcast would be the history. You worked on, you know, very early Stable Diffusion, SDXL, latent diffusion even before that. So- Video diffusion as well... video diffusion. Basically, there's a lot of buzzwords here.

[25:47](https://realalpha.app/v/8cGfJOU3AlQ/?t=1547) Um, you know, LCM, LoRA, flow matching- [laughs]... old prompting. Like- Can you teach us everything? [laughs] Yeah. I mean, what, what, what's interesting? Like how big are these scaling wise? But- I, I seriously... No, so, uh, I've actually built like a tech tree. Like I wanted to- Yeah... build this game where it's like a Civilization, but the, the tech tree is AI. Uh, like key AI papers. Oh, okay. [laughs] So you spend research points and get it. And so like these are the tech trees, right? Like, yeah, first it's GANs then, uh, you know, dot, dot, dot, like latent diffusion happened. But [laughs] you know, like there's a lot of core points beyond that. I'd play that game. Yeah. [laughs] I'd play that game.

[26:22](https://realalpha.app/v/8cGfJOU3AlQ/?t=1582) [laughs] So, so, so, okay, anyway, just to pitch the game, it, it, it's like you role play as a Neo lab and you have to s- [laughs]... you have to spend points on like- Spend that money [laughs]... uh, GPUs or researchers, and then [laughs] you have to collect data and like there's, there's all these like little things that you can sort of research. Fine tune. Yeah. I... Come on, whip it up. Let's see. [laughs] We're in the mod coding world now. I wanna play it tomorrow. Yeah. Uh, it turn- I think, uh, game balance is really hard because I hadn't-- You guys played... I, I, I have, I built it. Oh, fantastic.

[26:46](https://realalpha.app/v/8cGfJOU3AlQ/?t=1606) [laughs] Uh, but, uh, there's like game balance is really hard. It's actually like a AGI hard problem I feel like because, you know, once they find a way to win, they just always win like that. Mm. And it's not interesting as a game. Mm-hmm. You need multiple ways to win, and like you need high-risk ways to, you probably lose but maybe you win. Yeah, yeah. Yeah. Yeah. Mod crafts, classic problem in GANs, by the way. [laughs] So I think, I think one thing that if you were interested in, I'm interested in teaching our audience, like, like what are the key points that like if you are, uh, if you've heard a lot of terms, like what are the key things that have really impacted the fields?

[27:17](https://realalpha.app/v/8cGfJOU3AlQ/?t=1637) And, you know, a lot of you have, you've lived through a lot of this. I'd also say this is a great way for people that wanna get into this space, good interview questions if you're hiring as well, like you should know most of these things. I think there's some points where there's like spicy, you know, different paths that's popped up, so autoregressive image gen versus diffusion, uh- Two backbones or hourglass- Yeah. Yeah... diffusion... so all, all these things, what are the key things people should know? You know, I'll leave it, I'll leave it pretty blank. I mean, first would be SDXL maybe- Yeah, man... if we're talking about SDXL a bit.

[27:45](https://realalpha.app/v/8cGfJOU3AlQ/?t=1665) I was gonna say, what were the, what were the things we did in SDXL? [laughs] The refiner. The... Oh, no please, not the refiner. Wait, why, why that reaction? So SDXL was a big piece of work in trying to make the models feel good to use for the first time, if I was to kind of like lay anything out. Not to say like Stable Diffusion 1 and 2 weren't, you know... I mean, they were very exciting, they were very fun to use, but, uh, to anyone that was like in the space at the time between the period of SD, you know, SD1 to SDXL, uh, we saw a ton of work out in the, in the open source community working on how do we fine-tune these models, how do we make these models, you know, more aesthetic, how do we make these models into like, you know, uh, multi, you know, uh...

[28:24](https://realalpha.app/v/8cGfJOU3AlQ/?t=1704) If anyone remembers like the original Stable Diffusion were only able to do square images. And so SDXL was the first work of trying to actually release like a found, like the, the base foundation model kind of with a lot of these general improvements implicit in the model. So, uh, SDXL involved, you know, the first time we did large scale, uh, multi-aspect training. There was a, an... You know, at the time, uh, this was before, uh, packing. There was like an efficient way we could do essentially bucket style training. So training, you know, on all sorts of different resolutions of, of images in a, you know, a, a large scale.

[28:57](https://realalpha.app/v/8cGfJOU3AlQ/?t=1737) This was the first time we did multiple actual training stages, so taking it from pre-training, actually developing like post-training sets. So the, there was a lot of data work that was done in SDXL. I'm trying to think like all the other aspects. And then, uh, I'll, I'll give a kinda little, a little bit of a backstory on the refiner and maybe, maybe some things that, uh, didn't quite make it in. So here, here was the whole process of the final release model is essentially SDXL fundamentally works similar to the original Stable Diffusion. It was a latent diffusion model, took in text. It had...

[29:25](https://realalpha.app/v/8cGfJOU3AlQ/?t=1765) You know, you had a noisy latent, took the context of the text, actually generated the, you know, generated the imagery in latent space and then a VAE decoded it. And then from there, the refiner was actually another full diffusion model that the, uh, the whole point of it was to take essentially what was not a rough image, but SDXL wasn't the best at what we would call high frequency detail, which is this like the fine detail, like skin texture, texture of grass, texture of- Yeah, upscaling. Up- upscaling. Yeah. It wasn't an upscaler because it didn't actually change the resolution, but it, it, y- yeah, upscaled in terms of- It's a refiner... in terms of the, in terms of... It refined it.

[30:00](https://realalpha.app/v/8cGfJOU3AlQ/?t=1800) The reason I thought it was janky is because I worked, I... Not just me, the, the whole team that worked on this was, was fantastic. I don't want to take all the credit at all because there were, there were a lot of fantastic people that, that worked on this. We did a lot of work to try to push the detail quality of the model while trying to keep this like generality And we just didn't quite get there. And so then there was this like side track of work of like, "Okay, what are other methods to do this?" And this is where the refiner came into being with actually...

[30:26](https://realalpha.app/v/8cGfJOU3AlQ/?t=1826) **Dustin Podell:** The refiner was actually a model that was, it was what was supposed to-- So th- this is a little bit of the secret lore, is there was a version of SDXL that was never released. Uh, it was a different architecture.

[30:38](https://realalpha.app/v/8cGfJOU3AlQ/?t=1838) It didn't quite work as well. But it was trained pretty far along, even to like the post-training stages. Uh, the refiner was that model. It is a full diffusion model. I actually completely forgot about that. [laughs] Yeah, yeah. It- it-- Like the refiner was a full diffusion model. And the one thing about that model is it had a little bit of better performance on these high-frequency details. It just had very poor structure. And so this whole idea came of like, what if we essentially like- Different model, different job... different model, different job. We, we took it, we did a lot of work to really push the, you know, the work on how well it could actually refine this high-frequency detail.

[31:09](https://realalpha.app/v/8cGfJOU3AlQ/?t=1869) And then we made this, this, essentially this pipeline to- Yeah, which, which is the XL part... which, which then became, okay, this together is like a more performative model. You know, or a more performative system altogether. And then that was what we ended up releasing here. Which is actually interesting. It was one of the early pioneering works of like actually-- That looked into, okay, what kind of- Yeah... properties learned in which part of the trajectory of this diffusion kind of, uh, trajectory, right? Because you have these structural things that are learned closer to the noise, if you- Mm-hmm... if you come closer to the noise.

[31:39](https://realalpha.app/v/8cGfJOU3AlQ/?t=1899) And then you have the fine high resolution, uh, high-frequency stuff that comes further down and, and is closer to a kind of, um, the clean image, basically. And like today there's a bunch of different works that actually analyze this. But you do actually, you, you kind of, you regenerate from low frequency to high frequency if you go from like noise to clean image, right? Along this trajectory. But that was not really known. Like, it was informally maybe a bit known by then. Yeah. And this was, it was one of the first works to basically exploit this- Yep... and say, "Okay, I, I actually s- I dissect this kind of trajectory, and I, I have two kind of experts, one for the structure and the next for the texture, basically." Yeah.

[32:14](https://realalpha.app/v/8cGfJOU3AlQ/?t=1934) And yeah, right now we have modulation, which is like condition on the time step to actually account for that as, as- Mm-hmm... well, and stuff like that in the architecture. But that was really the first, uh, one of the first works to, to really, um, make this distinction. And it, as you see, it worked, right? [laughs] Yeah. It, it did work quite well. I guess the reason I, I still hold frustration, because at the time I wanted this one model to be so good at everything. Mm. And I felt- It was a cop-out. Yeah... may- maybe, yeah, maybe it was like a little bit of a cop-out. Now, nowadays I take a, a bit of a different view, 'cause I mean, we see systems of models do incredible work together.

[32:45](https://realalpha.app/v/8cGfJOU3AlQ/?t=1965) But at the time, I remember feeling, and I still just have this like sort of old frustration- Chippery, yeah... you know, just, just, just from back then. Um, but- Well, it's also very fair- Yeah... because you should always go with the simplest solution that works. Yeah. And if, if you then have a kind of a, you, you need to understand, okay- It's just not, like- At, at which point do I need a- A machine learning-... evolve the refiner?... approach. Yeah. Yeah. It's just like- Exactly. Yeah... when it comes to scaling, whatever the simplest solution is that works- But-... you should go with kind of a bit of Occam's razor, uh, as well in, in statistics, basically. Yeah.

[33:12](https://realalpha.app/v/8cGfJOU3AlQ/?t=1992) Well, so you know Runway was also kind of doing their Gen-1 and Gen-2. And, uh, Anastasios on, on this pod also said, admitted that Gen-2 was a pipeline mix. [laughs] Oh, okay. [laughs] Yeah, yeah. So- Like when they- No, like, like if it works, it works. Yeah... they, they just didn't admit it. Whereas yours was public, so you had to admit it. [laughs] Yeah. [laughs] I think if you go back to the time too, like people I don't-- People coming into the space today haven't seen 200, 500 steps of diffusion, right? Like, as you say, like you have a background, and then minutes later you have little fine details coming in. Uh, s- it was also very tinkerable, right? This came out as what, like a 3B, works on eight gigs of RAM.

[33:48](https://realalpha.app/v/8cGfJOU3AlQ/?t=2028) So as much as you could bitter lesson do the general approach, if you're not a lab and you're just doing this at home, you, you have like one or two GPUs- Yeah... you're gonna do pipeline stuff, right? Yeah. It's like that's how you can see the differences right there. Yeah, yeah. Uh, so video? I've been working on video for quite a, actually already before Stable Diffusion, I was working on, uh, video generation, uh, for NVIDIA. Um, had a paper there, Align Your Latents, where we basically mapped the-- It was actually in parallel to Stable Diffusion. We j- basically mapped this latent diffusion algorithm to video, which was the, the na- natural next thing to, to, to do. Mm-hmm.

[34:23](https://realalpha.app/v/8cGfJOU3AlQ/?t=2063) It was very exciting to kind of then this, at, at once see things moving. And you can imagine the kind of compression, um, reasoning behind why Stable Diffusion was so efficient compared to other models in the, in the time. Also holds for video, where you actually have this temporal dimension that you can also dis- uh, compress, right? So that made a ton of sense. Uh, worked on this for, first for NVIDIA, and then when we, when we basically joined Stability, we, um, basically after, I think after SDXL, um- Mm-hmm... we worked on this, this, um, Stable Video Diffusion, which basically first applied this, this latent diffusion, um, approach to video asset.

[34:59](https://realalpha.app/v/8cGfJOU3AlQ/?t=2099) But importantly, this, this work was a lot about investigating the effects of data filtering, which, as we felt, or as I personally felt back in the day, were like in research, and they still are very much underappreciated. Like data [laughs] is just like, when, when, when we, when we think about what these models are doing, effectively we're estimating a probability distribution, and the data set is what gives rise to this probability distribution, right? And if you have the perfect algorithm, the best algorithm in the world that, that just learns anything you want, but your data is very bad and you have like on average blurry videos or images and like, like not a lot of motion in the videos, your model will learn that no matter how good your algorithm is.

[35:39](https://realalpha.app/v/8cGfJOU3AlQ/?t=2139) So what comes out of the model will actually be very bad and like, like- Mm-hmm. Yeah... not nice. So the data is one of the most important levers that you have as one who, who wants to train these models. And in the entire research field, there was never much talked about data because a lot of people from the research community, they just worked on benchmarks and, uh, like, like tweaked the algorithms. But if you talk about frontier models, obviously we know that by now that data is extremely important, right? And, and that is what we wanted to be really like sharing a bit, like sharing more context on.

[36:10](https://realalpha.app/v/8cGfJOU3AlQ/?t=2170) So we investigated how much data... Like, and then you have this, this trade-off, I wanna be training on a, a lot of data. Quantity versus quality, basically. Exactly. Where is the- Mm... where's the sweet spot? That was, uh, what we investigated there. And then also a very interesting chart- Is there, is there a chart we should pull up? Uh-... co- coming, coming. Ah. We, we can actually t-take it. The, the, the most- Yeah... exciting experiment that we did was not the data one. But actually if you go down a bit, um- We then-- And it comes back to what we discussed earlier. We then showed that just pre-training on plain 2D videos allows you to fine-tune a explicit 3D model extremely fast from it. So if you go to the, to the last paragraph here of the, of the paper. Here we go.

[36:49](https://realalpha.app/v/8cGfJOU3AlQ/?t=2209) The- these are these 3D, uh, 3D, um, views here of, of the model.

[36:53](https://realalpha.app/v/8cGfJOU3AlQ/?t=2213) **Andreas Blattmann:** So what we actually did is we took this video model that we trained, and then we had, I think, 1K fine-tuning steps on 3D data, and we got a 3D model out of it. Mm.

[37:01](https://realalpha.app/v/8cGfJOU3AlQ/?t=2221) And, and th-that, that comes back to the question, okay, what, what do you actually need to, to kind of understand the world in 3D, right? Only 1K [laughs] [laughs]... steps of fine-tuning that shows that this video model, although only trained on, on- It was learning something... plain video- Mm-hmm... actually had this insane amount of 3D understanding, and you just took a bit of a, a, a, a focus to get a 3D model out of it. Mm. And that, that is really like, that was one of the early experiments where we saw, okay, this is actually kind of-- You can get world understanding through- Got it... training on plain video.

[37:31](https://realalpha.app/v/8cGfJOU3AlQ/?t=2251) And like from a research perspective, this was like, I think, the most interesting experiment, um, in this paper. Also pretty short paper. There's like a section here where you even explain the pipeline, right? There's like three, four stages, pre-training, some post-training. Uh, yeah, stage one, image pre-training, video post-tr- uh, video pre-training- Exactly... video fine-tuning, and then- Exactly... 1,000 steps you can also make it 3D, right? Mm-hmm. Yep. Crazy. Uh, at the time, I, I don't remember what I did, but there was basically this temporal consistency problem where if you just stretch out diffusion and try to stitch frames, it wouldn't work.

[38:05](https://realalpha.app/v/8cGfJOU3AlQ/?t=2285) So a very old hackathon project where I, I forget the method we even used, but we fixed this temporal consistency to like scale it out to 15, 20 seconds. It took forever, but it was a very hacky solution 'cause you could do this in an afternoon- Yep... right? So fun. But you, yeah, critically you need this kind of learning- Yes... over spatia- uh, temporal kind of- Yes... over the temporal dimension. Mm-hmm. And if you just stitch together kind of a very fast- Yeah, yeah... diffusion model, you don't-- You have this flickering. Even- Yes... even if you've-- You fix the seed and you do a ton of shit on effects- Mm-hmm... but you realize you need to, like in, in currently every-- Or, uh, today everyone is using transformers.

[38:37](https://realalpha.app/v/8cGfJOU3AlQ/?t=2317) So in the transformer suite you need the attention over the temporal- Mm-hmm... dimension as well to kind of learn from that. Yeah. Right? And then understand what is actually-- what-- how does consistent motion look like, basically. And as far as I understand, we asked this of the, on the Grok Imagine part as well, uh, there's no compression in the video side that is useful, right? You just have frame, frame, frame, frame, frame. Exactly. [laughs] Which is the simplest for- Yep... version. But it's- I mean, well, well if, if you then look at the-... the most intense... MP4 codecs you, you have very smart compressions. Mm-hmm.

[39:04](https://realalpha.app/v/8cGfJOU3AlQ/?t=2344) But that is lossy obviously, yeah, and, and you need to kind of, you need to connect the frames in a way and like, like extract what, what, what, how, how- Yeah... how kind of consistent motion looks like basically. Yeah. I don't think any video model exposes this, but kind of how far you look back in the, uh, temporal, uh, attention, because sometimes you do need longer, uh, or it needs to be more adaptive maybe. Uh, for example, if, uh, something gets occluded and then it comes out again- Yep... you want them, you want them to be the same. Mm-hmm. But what if-- How long is it occluded? I mean, that, principally that, that depends on the, on the, on the problem you want to be solving, right?

[39:36](https://realalpha.app/v/8cGfJOU3AlQ/?t=2376) If, if you, if you want to be creating a kind of a, a long memory model that, that, that can, can create you virtual worlds, then you obviously need to find a way to look back very, very, very long. A super memory hungry. And that, that is, that is just- Yeah... uh, I think also one of the most exciting research problems, uh, to work on- Yeah, absolutely... these days. Long context is just, uh, extremely important, right? I don't know if there's more there before we cut over to Black, uh, Black Forest Labs. Uh, anything else in the stable world? Uh, S, I mean, there's SD3. SD3. Yeah, SD3 was kind of- And that was the last paper that you did before you- Yeah, yeah, SD3 was a, a, I felt like a pretty big paper. [laughs] That, that was a- Yeah. [laughs]...

[40:08](https://realalpha.app/v/8cGfJOU3AlQ/?t=2408) it was a, I think one, a culmination of our work of, and stability, I would say, like- Yeah, absolutely... we- This is your graduation thesis. [laughs] If you want, yes, I, I mean... [laughs] No, this, this paper was really about understanding more in which part of this trajectory from data to noise does the model actually learn what properties and where do we have to spend most of our computation that we put in- Yeah... training to maximize the performance of a model. Yeah. So b-before-- A-and obviously, that was also the transition from classic diffusion to flow matching with a, a slightly different, uh, uh, paradigm basically.

[40:41](https://realalpha.app/v/8cGfJOU3AlQ/?t=2441) We call it rectified flow. Exactly. Um- Th- this was also the first time we used like diffusion transformers, right? With the, the, the- Yeah. And then, and then, and then we took it a bit further, yeah. So lot, lot, lots of changes like going from a convolutional architecture to a transformer architecture- Yeah... changing the probabilistic model from, from a diffusion, um, to basically flow matching- Mm-hmm... which is way simpler mathematically, which is super, super nice. I think you should actually define some of those terms for people that haven't heard. [laughs] So latent division, you know, flow matching, what.

[41:12](https://realalpha.app/v/8cGfJOU3AlQ/?t=2472) How did you come up with the concept of these ter- things here, right? Well, he didn't come up with flow matching. Well, I- We, we, exactly. We, we didn't come up with flow matching. We just saw this algorithm as, as something that is in a way more general because it can-- Like for diffusion, you always need one of your two distributions- Yes... between which you're basically translating needs to be a standard normal. That does not hold for flow matching anymore. And instead of approaching this kind of problem to, to, to, um, to transform one probability distribution into the other with a-- Seeing it as, as a solution of a stochastic differential equation, you're actually seeing it as a d- a solution of a ordinary differential equation, which simplifies everything completely, like mathematically completely.

[41:55](https://realalpha.app/v/8cGfJOU3AlQ/?t=2515) It makes also fast sampling much more, um, feasible you could say. Because what we always do, obviously, is when we have trained the model, we want to get-- be making as many steps, as little st-steps as possible to basically sample as fast as possible, right? Some people would say it's, it's actually a special case of, of diffusion, flow matching. Other, others might say, okay, it's actually a generalization of flow matching, uh, of, of diffusion, the flow matching algorithm, because you, you, you go from-- you can translate between two distributions no matter how they are shaped- Mm... and you don't need this, this kind of, um, standard normal basically as, to, to be one of, one of them.

[42:30](https://realalpha.app/v/8cGfJOU3AlQ/?t=2550) But yeah, in, in general, it can be seen as, again a, a, a model that learns basically to a vector field between two d- that, that, that translates between two distributions And once the model has learned that vector field, you can just integrate it numerically, which is then as easy as solving a ODE by means of an Euler forward, uh, step basically. So it's, it's, it's very similar in, in many respects. Um, but it was just like, I would say mathematically easier, more general from, from the, the, the kinds of probability distributions that it can slate be- can, can translate between, and that's why we basically, uh, went with it.

[43:05](https://realalpha.app/v/8cGfJOU3AlQ/?t=2585) And maybe as a s- a third part I said, it's also easier to derive a fast model from those, uh, flow matching ones basically. It, it was, it was much simpler and more scalable. Like it was, it was, it was much, much easier to work with and then had this like nice performance benefit as well. So like, I don't know, there were... the, the, the diffusion space got quite complicated. [laughs] Uh, are we satisfied that there's no loss? It's like, or is it, is it lossy in acceptable ways? [sighs] Lossy, you mean like do you, do you mean- No, loss, lossy compared to a full diffusion. No. All right. Yeah. Okay. Yeah, yeah. So then it's good. It's a strict improvement. Exactly. It's, it's- It's great... it, it is an improvement definitely.

[43:39](https://realalpha.app/v/8cGfJOU3AlQ/?t=2619) It-- Yeah, I mean, it's, it's effectively just drawing a line between like, you know, noise to data really. Yeah. It's, it's, it's like, it, it, it kind of took this like there was this complex space on like how do you map between these two, uh, the, you know, the, like these two distributions as Andy was saying. Yeah. And it simplified it down to [laughs] like the simplest of just drawing a line. It, it's just like, y- y- because you're trying to do a shortcut, like what if it's a bumpy distribution? Yeah, yeah. So that's the, that's the thing. And like the most important part of the paper or one of the most important like on the, on the, on the probabilistic modeling side, um, was actually looking at the trajectory from noise to image and which parts really matter and which don't. Because- Mm-hmm... if you just like we, we, we train this in kind of steps, right?

[44:18](https://realalpha.app/v/8cGfJOU3AlQ/?t=2658) We, we, we al- we always on, on this part from data to noise, we only always train on one kind of step, and then we approximate this direction in flow matching, um, with the model. And if you just sample those steps in between data and noise uniformly, you might actually spending a lot of time optimizing for things that actually don't matter. Mm-hmm. So one, one central part of the paper was

[44:38](https://realalpha.app/v/8cGfJOU3AlQ/?t=2678) **Andreas Blattmann:** where should we spend our compute? Which parts of this trajectory from data to noise carry the information that's most important?

[44:45](https://realalpha.app/v/8cGfJOU3AlQ/?t=2685) Because there-- this is where you should spend the bulk of your compute at training, right? And oversampling these parts during training and then undersampling other parts that are less important. It, it's effectively like a further, like a, a, a much further better exploration of part of what we talked about with SDXL, which was like the refiner. We spent time like training the refiner on the end of this trajectory to learn these high frequency details, and this was like more of an exploration of the entire space of where like you should spend time to make the most like general- Yeah... good model to learn the data. Yeah. Amazing. Uh, one thing I did pick up with SD3 was text, uh, in images, which you guys- Yes. Mm-hmm... you guys showed off a lot.

[45:23](https://realalpha.app/v/8cGfJOU3AlQ/?t=2723) There was a time when images could just straight up not do text, and then they did more and more text and then now it's like, you know, "Here's, here's, uh, here's our, our nice typography." And I feel like it was-- Can we attribute that to the diffusion transformer or? I, I don't know if I could say it was the diffusion transformer. Yeah. I mean, I, I think it really was just better modeling- Better data? ... b- it was better modeling and better data. Like it was fundamentally-- Like as, as much as I wanna- The boring answer... as much as I wanna like, as, as much as like, I don't know, you always want to attribute some fancy thing, it really was just that it, it can model the world better. It's a, it's a, it's a data thing. Yeah.

[45:50](https://realalpha.app/v/8cGfJOU3AlQ/?t=2750) It's also like what, what you're-- You always need like for everything that, that, that you train on, you need a caption, right? Yes. If your caption just don- Yeah... doesn't describe the text that is, uh, visualized somewhere, your model will not learn it, right? Yeah. So, um, that's also something, uh, that, that is often... Sometimes we spend a lot of time- Yeah... talking about the nitty-gritty details of some algorithm. In the end, it's again, uh, the better lesson is- Right... is- It's more easy data... simple scalable is- Yeah... is almost always the most effective, I would say al- al- always the most effective. Yeah. Yeah.

[46:16](https://realalpha.app/v/8cGfJOU3AlQ/?t=2776) But you can obviously scale in a smart way and still if, if, if you, if you spend tons of compute, you should be spending it as efficiently as possible- Absolutely... and only where it matters. And that is also at the heart of this, this paper basically, right? Yeah. I think prompting also got a lot easier at this time. Mm-hmm. Um, we used to have the 4K- Prompt extenders. Yeah. The 4K upscale, ultra- Our solutions massive with all the lists... 8K. Yes, yes, yes. Greg. Who was the Greg guy? The, the poor artist that just- Greg Rutkowski. Yeah. And now you- Oh. Greg Rutkowski, yes. [laughs] Yeah.

[46:44](https://realalpha.app/v/8cGfJOU3AlQ/?t=2804) I wonder how he's doing. [laughs] I hope he's doing well. I hope he's doing well. Yeah. Yeah. And also, you know, for context, this is when DALL-E 3, Midjourney, Ideogram V1, nothing was open. Yeah. So, uh, anything that you would see of this quality was either old stable diffusion or- Yeah... pretty much a closed, closed- Mm-hmm... portal. Yeah. Then you had to big-- make the big break to Black Forest. Uh, what was that whole period like, right? Like basically you had to leave and start a lab from scratch. I think it was probably one of the most exciting times, uh, we all had because when you're in stealth, you're just like, "You're a small team." We were 13, 13 people I think- Yeah... when, when, when we started.

[47:22](https://realalpha.app/v/8cGfJOU3AlQ/?t=2842) 14, yeah. We first went to France, had a, had a, had a, a fun little week in, in, in the- I remember that. It was fun. Yeah... some homes in France in a, in a, in a chalet. Spent the entire week just hacking, hanging out. Dustin figured out some really cool, uh, um, [laughs] uh, stuff to make our models way more consistent. I, I, I, I remember that. That was so fun. That was a magical week. That really was our kind of founding offsite. That is when we started, um, training Flux.1, which three months later, um, resulted in this nice company launch basically with a model that, that I think blew pretty much everyone away.

[47:55](https://realalpha.app/v/8cGfJOU3AlQ/?t=2875) Uh, at least blew us, us also away. [laughs] Oh, yeah. Like commercially, what, what, what this directly unlocked, like only, only having, having kind of incorporated the company for three months and then releasing this model and it was like... Yeah, just like was received very well, right? That was, that was amazing. Yeah. Uh, so that was- But like-... a lot of fun... did, did you know that when you started this... I mean, you had to get all the, the, uh, you know, the clusters together, like all the data. You had to start from scratch, scratch. We had to start from scratch- Mm-hmm... yes, but we knew what to do basically. Yeah.

[48:24](https://realalpha.app/v/8cGfJOU3AlQ/?t=2904) Obviously. We, we had a lot of experience, um, and- But like, do you... Did you have a h- You just did SD3. Did you have a hypothesis like that you were exploring of like- I mean-... what you would do differently? We, we, we've obviously gathered a lot of, lot of, uh- Yeah... knowledge like in these entire times even, even before, uh, Stability as well, right? And that all culminated, and then we also had a team that, that just like was in one place basically- That was very focused, but which, which had never been the case- Yeah... before because like, the-that's just like if you're, if you're working from, from very different parts of one company. Also, as Dustin said, Dustin was actually not in our team before. Yeah.

[49:00](https://realalpha.app/v/8cGfJOU3AlQ/?t=2940) And then like we basically had, had, had a super nice team together, spent a lot of time together discussing how we would actually design the, the, the kind of learning process of this model, which was at this offsite, and then we just kicked off, and that resulted basically in this episode. Yeah. Was it like discussions of what is Flux 1 or was it broader where do things go in a few years? How much- Well-... of this vision was like with- Well, I, I guess I can jump in and say like, I mean, one of the reasons the company was started, uh, I, I will say there were some frustrations of time at Stability.

[49:31](https://realalpha.app/v/8cGfJOU3AlQ/?t=2971) You know, we saw from, from our side, our, our side, we saw the impact that our models had made out in the world. We saw not just the impact from like the open source world, but also we saw companies that were started and run and, and, and quite successfully used our models out in the world. Like we knew there was a lot of draw and a lot of potential in what our models could do, and unfortunately, I, I think we also felt that [laughs] uh- Yeah...

[49:55](https://realalpha.app/v/8cGfJOU3AlQ/?t=2995) **Dustin Podell:** at Stability that we couldn't essentially leverage the value of what the models could be as well.

[50:00](https://realalpha.app/v/8cGfJOU3AlQ/?t=3000) So not only did we have the belief in like our team's ability to actually do the research and perform and build these models and, and get them out in the world, but this would be our shot to kind of correct those, um, inefficiencies or mistakes of not being able to actually capture the value of the models themselves. And, and so that was also a big part of the belief of why we thought we could, you know, go forward 'cause we-- I mean, we have this record of doing, doing this and making this impact, and we figured that could continue on and we can actually make this, uh, you know, successful company out of it, which now, I mean, I think it's, I think it's gone pretty- [laughs] I think it was a good, I think it was a good thesis.

[50:33](https://realalpha.app/v/8cGfJOU3AlQ/?t=3033) Yeah, it's gone pretty good. [laughs] I think it was a good thesis. So- As, as I've said before. Yeah. [laughs] Uh- We've been talking about how we can generate images and video and improve the perceptual understanding of models, but what about voice? And that's the subject of today's ad. Uh, we've been working with Deepgram for a while for on the AI side, and they recently acquired Pipecat, which has also been a big sponsor of AI. So Deepgram is becoming one of the critical pieces of voice infrastructure for all agents. So this episode is sponsored by Deepgram. If you're building a voice agent, you need it to keep up with how people actually talk.

[51:06](https://realalpha.app/v/8cGfJOU3AlQ/?t=3066) That means understanding words and context and keeping latency low enough for a conversation to flow. That's what Deepgram's speech models and real-time infrastructure are built for. One in 5,000 organizations uses speech-to-text, text-to-speech, and voice agent APIs in production. The focus is accuracy, low latency, and cost efficiency so you can build and run voice experiences at scale. You can use the cloud APIs or deploy it yourself, including on premises. And there's an offer on Flux text-to-speech, spend a dollar, get a dollar in credits, up to $500. To sign up for free or find out more, go to deepgram.com/contextiseverything. Deepgram, context is everything. Thanks to Deepgram for bringing us this episode, and now we'll go back to the interview.

[51:43](https://realalpha.app/v/8cGfJOU3AlQ/?t=3103) Well, I, I-- and like what use cases were you seeing, right? Because I think a lot of people went straight to like Hollywood, like we-- you know, that's our goal. Or like, uh, Imad used to say, "We will make Game of Thrones Season 8 again." [laughs] I mean, I, I'm trying to remember this. I kinda-- I don't know if I'll be able to remember the name of this company, but th-this, this might have even been before SDXL, um, but it was a company that for like a week or two, it went viral where anyone could take like a picture of themselves and then it- Yeah... like created a- Uh, the older version of SD, right? The older version, and it used- Yeah...

[52:11](https://realalpha.app/v/8cGfJOU3AlQ/?t=3131) like the old Dream Booth style fine-tuning, which was extremely inefficient and then, uh... But it w- it like blew up. Like everyone- Yes... was using this. Everyone was excited about it. The models could make so many things, and we saw that there was a lot of potential in how everyone wanted to use it, and you know, we, we saw where the models were going to... You know, at the time they couldn't do typography. We knew obviously, okay, typography is gonna be important and we, we eventually got there with SD3 and, and then so on. So there wasn't the just one use case. It was we saw that this was like a general technology that had a lot of potential and that was going to continue to grow and have potential in the future, which now is expanding into, you know, the, the physical world a lot more.

[52:47](https://realalpha.app/v/8cGfJOU3AlQ/?t=3167) And- But- But yeah... thi-this was actually already the potential we saw that like- Yeah. Yeah, absolutely... BFL was not founded to be an image generational lab. It was more of a kind of a matter of the, the, the Flux movement being so good that we then capitalized on it as well- Yeah... because it gave us a kickstart. But when we pitched the company to investors for the C series, it was clear we wanna be working on video because we see that as a- Yeah... I think we shared this on a podcast actually back in 2023 already when we were still at Stability. We were saying, "Okay, this like video modeling is actually the central means to understanding how our world basically works."

[53:21](https://realalpha.app/v/8cGfJOU3AlQ/?t=3201) Because again, what I said earlier, right, essentially we humans are also first observing the world, then starting to interact with it, and by that we, we learn so much. So that was always the thesis, and then obviously we had to kind of take this detour, which was very successful, I would say, to, through image generation. And eventually-- and, and then- Mm-hmm... you always like face the, the, the kind of the issue of, okay, where do I spend my time on? That- Yeah... that's unfortunately a, a thing we, we, we, we cannot get across, that the day only has 24 hours, unfortunately. But eventually I think Flux 3 is really like where we saw this going for a long time and what we- Mm-hmm...

[53:56](https://realalpha.app/v/8cGfJOU3AlQ/?t=3236) wanted to be doing for a long time, and that's why we're so excited a- about like having the model out there, being able to create videos, being able to create images, being able to create audio, being able to act as a robot brain to actually kind of work in real manufacturing workflows. And there's, there's a lot of stuff coming this year still that we, we have to announce on, on, on, on the physical AI side. Mm-hmm. But really like, as Dustin said, our goal was always to create the most general technology, and we see this happening currently, which is why we're so, yeah, so excited basically. Yeah.

[54:29](https://realalpha.app/v/8cGfJOU3AlQ/?t=3269) I mean, I think you guys also took the right paths to stay focused, right? Like if I'm not mistaken, Flux 1 launched with xAI. It was their image gen model. [laughs] Good partnership. Yes, that's right. Yep. Uh, before they had their own, but like, you know, that's a good way to solve do we build distribution ourselves. Yeah. Even to the latest Flux 3 Max is post trained with FAL. They do a lot of entrance distillation. Mm-hmm. They're good at that, right? So it's interesting. But I- It seems like you're expanding now towards, uh, you have... What's the latest one with robotics, um- Flux reaction? Flux reaction. Yeah.

[54:59](https://realalpha.app/v/8cGfJOU3AlQ/?t=3299) Yeah, but before we get there- Okay. Yeah... there's one more I want to cover- Yeah, yeah. That's okay... which is image editing in context. Sure. Yeah, yeah. Uh, we've okay, uh- Also, really nice, uh, history right here on the website. Yeah. [laughs] I, I, I was actually- Like that is- It's, it's weird we're now at the stage of company that, like, things just happen, and I d- I'm like, "Oh, we have this." Um, but I was, I was kind of sad Flux 1 wasn't on there to view, but it's, like, the original one. Right. Yeah. [laughs] Yeah. Yeah, but it's, it's fine. It's, uh- We don't- Pro's, pro, pro looked good.

[55:24](https://realalpha.app/v/8cGfJOU3AlQ/?t=3324) I, I think, like, even just the visuals here, like maybe someone will clip it of- Yeah... original Stable Diffusion into XL to all the papers- You see the progression, right? You see the progression. Yeah, yeah. But one thing that I do wanna cover at some point, I no longer personally see the progression, right? You show me state-of-the-art image video models. A lot of Chinese labs are putting them out. It's hard to eval. I know people care. I've talked to Summit at BFL about, okay, no, actually, people in the industry, aggressively A/B test industry, there's, there's a lot that people care about on evals, but there's also a level of, it all looks very realistic to me.

[55:57](https://realalpha.app/v/8cGfJOU3AlQ/?t=3357) [laughs] Um- The rest is prompting skill issue. I think there's, uh, yeah. [laughs] I think there's a lot of little nitpicks. Like there were hands, there are still issues in text sometimes. So- There's still hands issues? No, there were hands issues. Yeah. There, there's issues in perception and layout, like I'm redesigning my living room. It's horrible. Controllability is a huge thing. Controllability. I think controllability is the, the thing we need to solve, like large scale. There's so many ways you can control these models, and maybe that, that is a good seg into, into, uh- Context... Flux context as well- Yeah...

[56:25](https://realalpha.app/v/8cGfJOU3AlQ/?t=3385) because that, that was a pain point that we saw for a lot of time. People were s- training so many LoRAs for- Yes... just to get this character consistency into models, right? A shout out- Yeah... to replicate back then with, you know- Yeah, exactly... all the LoRAs in front. [laughs] Oh, yeah. And, and, and then we had this, uh, amazing pitch of Dustin. It was, uh, also in Northside- [laughs]... also in France, I think. It was, it was another cabin, right [laughs], like a lot of things happen in cabins. Also, I think [laughs] we should go to the cabin. We really like, um- You should do more. Yeah, yeah. Yeah.

[56:50](https://realalpha.app/v/8cGfJOU3AlQ/?t=3410) And, and then we had this pitch of Dustin, who actually kind of saw this potential and, okay, I need to, I need to get this, like, context. I, I, I need to be able to prompt not only with text, but also with images for image editing and that. Um, yeah. Mm. Yeah, yeah. No, I mean, I, so I did this, like, I, I originally- That, that made pre-hype as well. Crazy. I've heard the story that Flux context was, like, you and one or two other people. Yeah. So it was- [laughs]... for, for a long time, it was basically my side project, and then I stole one of our, one of our data guys a little bit into it and said- [laughs]... "I'm, I'm-- Come, we're, we're, we're building this thing."

[57:18](https://realalpha.app/v/8cGfJOU3AlQ/?t=3438) So for me, like, so, so yeah, a- a- as Andy was saying, it originally started as a pitch of mine at some, uh, some cabin in France that, uh, I did essentially like a... I, I, I like to do these presentations every, you know, every offsite or every time we get together, where I try to look at, like, the state of the field and where I think the field is going, and kind of where... Like, what, what is something we're not doing or something where these models have a potential to do something novel and useful that, you know, maybe we're not looking at right now or maybe isn't solved. And so I actually did a presentation on, I know this is like the most overloaded word in the world at this point, but I was like, "Oh, it's the omni, the omni model presentation at that point," because, um, there was a lot of research work out there at the time on, you know, other editing models.

[58:02](https://realalpha.app/v/8cGfJOU3AlQ/?t=3482) If we remember back to, I think there was the old prompt to prompt, and then there's, uh... Why, why can I not remember the name of this, um... The pitch was essentially these models are clearly learning, like, these deep relationships about the world, and this is gonna keep coming up. It's about these, these relationships. And to me, you know, like, the relationship of like, okay, this Coke can is sitting on top of a table. This bottle is sitting here. And you prompt the model, and you go, "Okay, I want a photo of a dog wearing a hat." And it needs to know, okay, the hat is on top of the dog. It, it, it needs to have this general understanding.

[58:30](https://realalpha.app/v/8cGfJOU3AlQ/?t=3510) And my kind of theory in mind at the time was that since the model already has all of this, like, spatial knowledge, if we essentially build the right data and train the model, like train a, a, an already pre-trained model the right way, we should be able to evoke essentially, like, general image editing with very, very little training [laughs], capturing these already, like, pre-built relationships of when things are, are, are there, when they're not there, uh, and how these, like, relationships, these, these space relationships inside of an image work. And then essentially what it became is like a three to form. I don't remember the timeline of, uh, me here over in our SF office with the one data guy I stole, building out essentially just early test training data sets.

[59:12](https://realalpha.app/v/8cGfJOU3AlQ/?t=3552) And, and I remember the moment I saw that this was, like, clearly working quite well when we made this effectively tiny data set of just very, like, high quality labeled. It was essentially describing, like, simple relationships in an image, and that was really it. Uh, like, it wasn't a way that you would prompt a, a modern, like, edit instruct model.

[59:30](https://realalpha.app/v/8cGfJOU3AlQ/?t=3570) **Dustin Podell:** It was just literally describing these relationships. And I trained the model for, like, I mean, it was maybe like six hours of training, four hours of training. It was, it was a very, very short training run.

[59:39](https://realalpha.app/v/8cGfJOU3AlQ/?t=3579) And all of a sudden, like, I saw what felt like a clear generality coming out of it, 'cause it was-- I think it was, like, a photography data set. It was just, like, photos of stuff. But even just from that, I was seeing, like, I can put a logo on a shirt. I can put, like, I can do... And it wasn't good. It was just, like, the model was clearly learning to attend between these two concepts and use these pre-trained relationships. And then I took that to our next offsite, showed results, and then it became this essentially more company-wide project for about, you know, um, a, a couple months, where we really put effort in, brought people in.

[1:00:11](https://realalpha.app/v/8cGfJOU3AlQ/?t=3611) You know, really improved essentially... I, I, it was mostly a data set project. Like I'll say, I'll say it was mostly a data set project. And then in the end of it, the distillation. Um, and then there, there was a lot of, there was a lot of very nice distillation work by Axel that was done to, to, to speed it up. Then we essentially were able to put out, put this model out into the world, and I, I was quite proud of. Yeah. And- That was the first kind of model to really do this character consistent editing, right? In, in, like, high speed. Yeah. That made it, again, accessible to so many people. For most people, it's very common to generate something, chat with it, change it.

[1:00:42](https://realalpha.app/v/8cGfJOU3AlQ/?t=3642) Before this, image generation took quite a while, and you need to recreate the image. You couldn't- Yeah... change the parent to be blue. Just, just getting the same character, it won't- It wouldn't be the same. You- Or you need to mask, uh, something that you don't like- Yeah... and, and then fill it in the blank. And then- Yeah, I just, I wanted to get away from the slot machine, essentially. Everyone had to do the slot machine. [laughs] Yes. Like prompt and prompt and prompt and prompt, and it was very clear that, like, iterative editing was, like- Oh, no... much more effective way to do this.

[1:01:04](https://realalpha.app/v/8cGfJOU3AlQ/?t=3664) So my, my crappy Comfy UI was that. It just- [laughs]... slopped out like 10 different versions and you could pick. There's, there's, yeah, yeah. And I guess I can- My, my favorite part, I still slop out. Like, I'm- [laughs]... changing, living in my generate four, generate six, redoing something. Just make 10 iterations and [laughs] And then, yeah, and eventually that one's good enough. My favorite is that you pray to RN Jesus. [laughs] Because it's a random number. Uh, basically, you know what, to me- This concept of what is a foundation model is something like this where it's like, okay, you know all these little workflows that you guys do to like do that and do that and do that and they- they all use different tools?

[1:01:35](https://realalpha.app/v/8cGfJOU3AlQ/?t=3695) We're gonna replace that with one model. Yeah. [laughs] And that- that- that's- To this, that was-... actually what we are building towards, right? Yeah. And that- that was where FluxV came from and- This was a very clear aha moment for me. Yeah, yeah. Mm-hmm. I don't know the... I- I never tried Prompt2Prompt, but I think, uh, to me the closest it has come to a FLUX Kontext is Qwen Image, which, like, the Qwen folks are doing really good, like- Mm-hmm... they had a Photoshop layer model. Yeah. It was incredible. Yeah, no, no, I mean, they're doing- [laughs]... doing pretty cool stuff over there. Yeah. It's interesting how much of this just comes back to there's a lot of good data.

[1:02:02](https://realalpha.app/v/8cGfJOU3AlQ/?t=3722) [laughs] Yeah, I mean, it was, it was targeted good data for... Was it a good concept? I- I don't know what- how to even describe this here. Yeah. But yeah. I- I think one thing that I'm surprised by is, so, you know, comment on stability one more time. Stable Japan, right, Japan had this, like, whole law of, like- Yes... we will protect you if you trade on copyright- I'll open anything up... in- as a country. Yeah. And everyone was like, "Ooh, Stable Japan," like, it's interesting. Nothing happened. [laughs] Yeah. I always wanted to visit that office. I was sad I never did, actually. [laughs] I think, uh, on your Omni point as well, it was an interesting time. There were the two others. Gemini is obviously the latest Omni.

[1:02:37](https://realalpha.app/v/8cGfJOU3AlQ/?t=3757) Uh, Image Bind, which I didn't realize is 2023 from Meta, they threw in one embeddings to- Heat maps... heat maps- Heat maps. Depth... and depth. That's a big one. [laughs] Which, you know, I'm like, oh, very early, but, like, it's cool. I- Yeah... I think this work has progressed. There's definitely newer, newer- Yeah, so like, but you can, you can-... work on it, but like IMU... you know, this is just another color, right? It's RGBA what- whatever that is. Yeah, yeah, yeah. Yeah. Right? So like how many colors are there? Oh, the fun thing, this was also- [laughs]... all, all in, all out, so you can go text to heat map if you want. You can go IMU to depth. It's a big, big market for that. [laughs] But- [laughs]... interestingly, no, no video out, which is I think the- Yeah...

[1:03:11](https://realalpha.app/v/8cGfJOU3AlQ/?t=3791) the kind of the- Yeah, yeah... the central thing that- that- Yeah... we need to l- to, to, to understand basically the world better, right? This, this is, I think what, what's... Uh, as said, what has always been at the core of what we were, we're doing. Yeah. Okay. Yeah. Well, before we leave the, the sort of data discu- or IP discussion- Yes... is there a particular kind of data that is, uh, everyone in an industry knows is, like, valuable? Uh, if people... If- if I have a data vendor listening, how do they sell to you? What, what... How do they approach you? It, it depends heavily on what you, what type of problem you want to be solving. That's just super important. If I, if I train a, uh, a robot, I obviously need a lot of eccentric data. I think that's not a secret, right?

[1:03:48](https://realalpha.app/v/8cGfJOU3AlQ/?t=3828) Yeah. Because that, that is just how this thing will be, will be basically. E- egocentric or? Yeah. I mean, there's also a benefit for you guys, right? On the language model side, there's a lot of people doing RL environments. Is there a parallel that people should come to you with that kind of stuff? Yeah. I'm, I'm trying to get you vendors, yeah. [laughs] It's like get you- Oh, we have so many data vendors actually that are, that are approach- Too many? Yeah, yeah. Okay. It's, it's a lot. Oh, okay. What, what, what is good versus bad? It, it again depends on which stage of the training you are- Yeah... um, and what your effective goal is.

[1:04:14](https://realalpha.app/v/8cGfJOU3AlQ/?t=3854) I think there, there's a, there's a trade-off between we had this already, right, scale of the data versus, um, versus the quality. Yeah. And that is what you have to, for every training run, like large scale pre-training and you do again, you have to kind of find, find that sweet spot out because depending on the architecture you, you, you might be using, uh, that might look very different again, and that is one of the central pre-training experiments like, or pre-scaling experiments that you need to do. You need to do a ladder basically, um, to understand at scale how much data-- or to, to predict at scale how much data do I actually need and h- how should the quality look like.

[1:04:48](https://realalpha.app/v/8cGfJOU3AlQ/?t=3888) So where is the sweet spot between like these two dimensions of data quantity versus data quality? Mm-hmm. Any other major jumps that we want to call out? So Flux.1 Context was a big one. Flux.2, Flux Tools, Action, we talked a bit about. Any other- Yeah... things that- I mean, I think the, the three sizes, that's very important- Mm-hmm. Yeah... in terms of like, you know, the, the dev models and the Schnell. Schnell? Schnell? Yeah. What is Schnell? Schnell means fast in German. Ah. Ah. Yeah, I think you probably covered that in a blog post. But yeah, I, I think that, that like is you have open source, you have the closed source, and you, you have the range of performance. I think that makes sense. Yeah, absolutely. Yeah.

[1:05:23](https://realalpha.app/v/8cGfJOU3AlQ/?t=3923) I mean, yeah, I mean, I guess if like anything to track on is like okay, f- well, the one, like, thing that made me quite sad that we didn't quite get in with FLUX Kontext is that it was only single image edit because, uh, I mean, I s- it was very clear that there was like a huge need for, for multi-image. Like, people, you know, want to be able to put in many, many different images and, and relate those- Uh, references... references is a better word here. Yeah. Yeah. Many, many different references. Which I think Groq is, like, king right now. Or I, I, I don't know this state of it, but yeah. Yeah. I know, I know Midjourney does quite a bit on style books where you can build up references. Yeah.

[1:05:52](https://realalpha.app/v/8cGfJOU3AlQ/?t=3952) I mean, they, they've always been very good at like the sty- style reference, but th- this is more just like general. Like, like Flu- FLUX Kontext would only take one image- Yeah... and so it was just you can only iterate with that. So you have a bad photo of one person, that's all you have. Yeah. But- But you could have three different photos of the same person. But it was actually general enough. I remember it was, it was actually quite funny. People were, like, doing workflows where they were stitching a bunch of, like, images into one image to, like, emulate it. But then with Flux.2, we finally were able to, like, spend the time to, to, to kind of build out the data and actually do this larger scale training for handling, I think it was 10 reference images at the time.

[1:06:20](https://realalpha.app/v/8cGfJOU3AlQ/?t=3980) So essentially building up much more context to work within. Yeah. Yeah. Which is again [laughs] also a lot of a, a, to, to a large extent the data, a data, um, problem that you, you just need the data- Yeah... to have these many references, right? Yeah. Is there a practical limit? So, you know, let's say the industry standard is like 10 characters or 18 characters. That's it, right? Yeah, I mean- I don't need 100... obviously you can show crazy numbers- [laughs]... on, on, on how many, on how many characters you, or like, like images you can get into one image or also video, but there, there are like, that is agent work then to, to, to get me 100 references.

[1:06:52](https://realalpha.app/v/8cGfJOU3AlQ/?t=4012) [laughs] Which is the easy thing, right? It's agent work. But then I need to check it, like I, I, then I have to review 100 images if that is actually what I... It's actually also cr- [laughs] Yeah, yeah... a hard creativity problem. Like how should I, where should I put these set of- Composite these things. Yeah... 100 images, right? So I think without like, like over-indexing on the number, everything that goes up like, like over, over 10, 10 references is also hard to follow for a human, I think. Like if you, if you say- Yeah... okay, we want to be creating a, a video, right? I obviously can, can put in 50 people there But if you look at the mo- Like, the, the, the most important movies, how many central characters do we have? Mm.

[1:07:23](https://realalpha.app/v/8cGfJOU3AlQ/?t=4043) Like, five, six maximum. Unless you're- Yeah... Avengers. S- exactly. [laughs] The, the, the, that is then also it's at some point approaching the, the kind of limit of what we humans can actually- [laughs]... kind of process in, in, in a two-minutes clip or something. This also gets way, like, you coming back to the, I know you mentioned the evals earlier. Building evals for larger, like, contextual models- Yeah... becomes a, a much bigger process. Yeah. Like, even for, like, we, we ended up releasing a, uh, an open eval set for, um, I think it was Context Bench, which was built off the internal bench that we use to evaluate context.

[1:07:54](https://realalpha.app/v/8cGfJOU3AlQ/?t=4074) And then, as you scale up how many references- Oh, I haven't seen this. Okay. Yeah, yeah, please, please look up Conne- I'll pull it up. This was a lot of work, but it'll, kind of a... It was actually quite fun to put this together. Make the cat very fat. Yeah. I mean, it was, we lit- I remember sitting in the back room of our SF office with, like, a few of the other guys. Like, me and Tim were l- uh, were laughing. Tim Dawhorn is another one of the great, great guys I work with. I feel like, so, like, do you just, like, take some edibles and just go, like [laughs] Dude, I, I mean, we just, we're sitting there late- What's the process?

[1:08:19](https://realalpha.app/v/8cGfJOU3AlQ/?t=4099) We're sitting there late at night literally just typing in this, like, we're just pulling images and typing in the stupidest things. [laughs] I mean, it works, right? Yeah, that, that like... It, but it's, like, real. Like, I don't know. So, because, because- And in the end, you always, you need to kind of make one step to make it a kind of- And we, we-... enjoy the comedy... what is, what is ground truth? You, you, you need to make sure that you have a ground truth somewhere. Yeah. No, this is just a benchmark, basically. It's just inputs. Right, but where's the- How do you- Where's the-... eval the output? Yeah.

[1:08:40](https://realalpha.app/v/8cGfJOU3AlQ/?t=4120) Where's the eval? I mean, the, the, that is the interesting thing, right? You just ask people, this is a public bench to give everyone, like- I see. Oh, so this is- This is half of an eval... some arena for improvement. Yeah. Yeah, no. This, this is the eval and then you, you have two models that you want to be comparing, right? Then you judge. Yeah. Yeah, yeah, exactly. Arena style, yeah. Okay. Yeah, yeah. It's- So it's not a validation set in that sense. Yeah. It's just a eval bench. This is just Dustin sitting in a room [laughs] This is, we remove this. It, it, I, genuinely, like, 80% of this is that.

[1:09:03](https://realalpha.app/v/8cGfJOU3AlQ/?t=4143) Like, it, it's, um [laughs] also heavily- It's heavily influenced by our, by our, uh, I think, by our Italy offsite. Uh- Yeah. No, but- Yeah. Why don't you do, why don't you do, like, a Mad Libs, like, like, like, a combinatorial explosion type thing that is more structured than just random inspiration? So, so, so, I mean, now we're do- we definitely do- Oh, yeah... more stuff like this, I will say. The, so one of the thing is there were, I mean, there were certain things we had to, like, filter out for the public set on here, 'cause it's just, there were some very dumb things in the internal thing, you know, in the, in the internal set.

[1:09:31](https://realalpha.app/v/8cGfJOU3AlQ/?t=4171) **Dustin Podell:** But, um, I don't know. I, I spend a lot of time, I like to see how people actually use our models. Okay? And I, I'm someone that actually really likes to use our models.

[1:09:38](https://realalpha.app/v/8cGfJOU3AlQ/?t=4178) I, I think it's very, very important that you test and play, like, really just play. I'll say play with your models. Like- Like memes... like me. Like, yeah. Like, use it for what people actually want to use it for. Like, I came, like, like I said, I came from Discord, open source communities on this, and you see people doing the stupidest things, but it's like, this is, people are having fun with it. And it's, so a lot of the set, you know, besides trying to cover a bunch of actual use cases, uh, like production use cases, which, which, which are very important, I also want to make sure it's just like the models are fun to use, you know?

[1:10:06](https://realalpha.app/v/8cGfJOU3AlQ/?t=4206) Yeah. And so, yeah. Good, good vibe eval vibe bench. Exactly. Yeah. Well, I think it's like I, I, I underappreciated three years ago, which was like, yes, this is memes. Yes, this is like a mid-journey, whatever subscription, but like, it actually becomes very real work. Yeah, exactly. Like, because- Absolutely, like- Because they've done it for fun, then they're like, "Oh, I can do this at work." And if effectively also, like, when you, when you automatically eval these, these models, like, you, there's a bunch of obviously human evals that everyone is doing in the field, but every evaluation will have their blind spots.

[1:10:36](https://realalpha.app/v/8cGfJOU3AlQ/?t=4236) Yeah. And you, you should be combining as many independently as possible to get- Yeah... a picture that is as good as possible of, like, how the model is performing. But that doesn't, like, re- remove the burden from you to kind of... Or it's not a burden. It's actually super fun to then sample the model yourself and, like, try- Yeah... understand where are the strengths, where are the weaknesses. Like, this is, this is a core part of developing a model is actually constantly sampling when developing it and checking, a, is it, is it where I would expect it to be, right? At some level, it's like what you guys care about too, right? Like- Mm-hmm... these are your use cases. It's a good way for if you want something to get better, like, make a little benchmark and see how things perform.

[1:11:10](https://realalpha.app/v/8cGfJOU3AlQ/?t=4270) No, a- absolutely. I, I think it's very important that the people building the models actually, like- Use them... like, want to use the model. Yeah. Like, I, I think this is very important. There's a big distinction in this and, like, the microphone brand trying to, you know, use it for product photos. Mm-hmm. A very niche use case that they can min-max on compared to remove- It's very general. Yeah. Yeah. I, I, I think one thing I do doubt or question is the, it was a very well-known Midjourney's loop, which is the, it's Discord, everyone sees, like, other people prompting. You get the four options. You choose the four.

[1:11:42](https://realalpha.app/v/8cGfJOU3AlQ/?t=4302) That is feedback for your next model. How come you didn't do any of that? We didn't do it publicly at scale. Yeah. So, so every one of- But isn't it valuable to do? It can be valuable, but I think only up to a certain extent that if you then, like, cross certain limits, the signal that you get is pretty, pretty- Yeah... Like early days, we saw this insane potential, right, of this, this, this, this tuning, uh, say of, of Midjourney's models, like V- V5 or something crazy. But if your base model just reaches a certain, certain kind of level of strength, it's actually, you, you don't get these super good examples anymore where you can clearly see the model improving from this, and this is also a lot of noise.

[1:12:18](https://realalpha.app/v/8cGfJOU3AlQ/?t=4338) Yeah. So there are, there are a lot of other ways w- how you can actually do this. I, I had this debate as well with someone where back then it was very useful because prompt adherence wasn't great, right? Uh, it's pretty judgmental on which one follows- Yeah... the prompt better when neither [chuckles] of them do it great, right? Yeah. So you have a lot of preference tuning towards- Yeah... there's very high signal and okay, the people prefer this. Now back to my point a lot earlier, like, if I take the top five models and tell them to remove a bed, it's all gonna look the same to me. The preference, there's not that much signal, right? They're all filling it in pretty good- Yeah... because there's, like, they understand this task very well. So that side of the eval for me is kind of gone.

[1:12:55](https://realalpha.app/v/8cGfJOU3AlQ/?t=4375) It all, it all looks fine. Yeah. But yeah. I was gonna jump and say that was actually one of my favorite parts of, of moving into context, is finally it wasn't just like- [laughs]... I don't know. It was finally just like, okay, did, did it actually just do the task? Yeah. And, and, and I really enjoyed getting to these, like, very kind of clear, concrete tasks. One more part of the text tree, and then we want to go into robotics in action. Mm-hmm. Um, autoregressive- Yes... image gen. It seems like a big trend with both Gemini and ChatGPT doing it. Yeah. Uh, like- Is that as big an impact in, in your mind as it is in ours? You know what I mean? Like, I, I-- it seemed like a very big change because then you could conversationally edit.

[1:13:33](https://realalpha.app/v/8cGfJOU3AlQ/?t=4413) It is the, like, the next obvious step from conte-uh, i- FluxContext, where instead of editing with a image and a, and, and text prompt, you're conversationally working with a, with a model. That I would say is not a, a feature of autoregressive models- Yeah. Mm-hmm... uh, only. It's just based on the data you train on it. And like you, you, you can get language capabilities into a diffusion model as well, right? Yes. Which we, which we know is, is, uh, working. And, um, I said in, in long term, this is where we believe this will, this will also go into and where we're working towards to also get more modalities in, including this language model, right? Mm-hmm.

[1:14:08](https://realalpha.app/v/8cGfJOU3AlQ/?t=4448) So it's not, it's not, it's not a question of the, of the, of the algorithm there. Yeah. Um, it's just a question of like, what do you focus on, basically.

[1:14:14](https://realalpha.app/v/8cGfJOU3AlQ/?t=4454) **Andreas Blattmann:** I think both approaches are extremely exciting and have a lot of strengths and a lot of like shortcomings. I, I would say autoregressive models are way more data efficient.

[1:14:23](https://realalpha.app/v/8cGfJOU3AlQ/?t=4463) You can train them way better in data because every, like, you have the-- for, for diffusion models, you have this trajectory from data to noise, right? And every training example you're training on is effectively because you have this continuous path from data to noise in between. It's giving you r-rise to infinitely many training examples because you can pick every possible spot to denoise from, from this trajectory between data and noise for one example. Obviously, the model would overfit if you would only train on, on, on this one, uh, example. Theoretically, it's just like one training example is giving or one data point is giving rise to infinitely many training examples.

[1:15:02](https://realalpha.app/v/8cGfJOU3AlQ/?t=4502) Whereas autoregressive models can just train on all the tokens in parallel, basically, and get the maximum gradient from each training example. That's the training side. So there I would say autoregressive models are more data efficient. However, at the sampling, at sampling time, autoregressive models have to-- don't get across that. They have to autoregressively create one token after the other- Yes... because they're iterative in this na- like, like data direction, right? And diffusion models, you can just go down and distill fifty steps down to four, and by that you can speed that up so much, which is on the autoregressive side, just not possible because you have to autoregressively generate token after token after token.

[1:15:41](https://realalpha.app/v/8cGfJOU3AlQ/?t=4541) So these are just two, two, um- Well, you, you make it sound like it's just worse then. So, so- Yeah... autoregressive generation is in that respect, it has a limitation, but on the other side, diffusion has in, in the terms of data efficiency or training, has also a limitation. So we, we actually-- I think it's-- I, I like both approaches. We've also been working on autoregressive models like- Mm-hmm... long, long, long back, and both are super exciting, I would say. Yeah. So on, uh, data efficiency, autoregressive wins out though? Yeah. Okay. So scale. Um, I guess one of the other big things that you see, having talked to people that worked on both like ChatGPT and Imagen is how you can add in the loop RL, like agentically.

[1:16:18](https://realalpha.app/v/8cGfJOU3AlQ/?t=4578) So models can do a-agentically, you know, generate something, reason on it, edit the image and continue that workflow. Yeah, they can do that too. Yeah. And this is, but this is separate than pipelining in diffusion generation, right? It's all the one same model doing it, where as you train that in on a data level, you know, you can scale that. Um, I guess, is there a counter to this in diffusion, or? There's no inherent limitation to-- that you couldn't have a model that is diffusion based- Yeah... do the exact same process that can do all of these aspects. Yeah. That haven't been enabled yet. Yeah. This is like really blowing my... Yeah.

[1:16:49](https://realalpha.app/v/8cGfJOU3AlQ/?t=4609) Like, I, I had the wrong, basically nano banana happened and I was like, "Holy shit, like autoregressive is a thing," but you're saying it's not. I think, am I wrong that isn't nano banana diffusion? I thought nano banana was diffusion. It's- I think it is diffusion based. I think the auto, like it is- Oh, it was, it was 16 bit? And I, I don't have like insider knowledge here, so I might be wrong, but I'm pretty sure it's, it has the language component to do like autoregressive, but it isn't like, I think it's what, what we'd call like a transfusion model. Mm-hmm. So like it's, uh, you know, it, it can essentially do both. But there's no, yeah, there's no inherent limitation that says you, you, you have to do it one way or another. And it's not like we're...

[1:17:21](https://realalpha.app/v/8cGfJOU3AlQ/?t=4641) They're, they're explorations. [laughs] Yeah. So yeah, yeah. Like, uh- What I'm getting is that- Yeah... it's not, it's not settled consensus. And I thought it was that auto, there was a wave of people shifting from diffusion to autoregression for Imagen 2. I think there's a lot of, there, there's a lot of exciting research on that end right now going on. Yeah. Like I, I will say there's, there's some very impressive work out there. Obviously, we've seen models work extremely well on that front. The same is going on for, for flow matching is also improving. Like it, uh, I would say the debate is not settled yet. Yeah. Like there's still significant re-research on both fronts- That's exciting... going on right now. Yeah.

[1:17:49](https://realalpha.app/v/8cGfJOU3AlQ/?t=4669) And I think, I think it's important to, to watch both and be ex- you know, try both, exci- be excited about the, you know- And finally, the same is happening for robotics as well. We have, we have- Absolutely... LASERs versus, uh- Yes... VAPS basically, which is super, super interesting as well. But yeah. The, the last other thing I guess is there's also the diffusion for text, right? So there's also some of that. So it's, there's benefits. There's of course speed, right? You can- We see that already, like- Yeah... with text, we see it clearly, right? How like- Well, we got Diffusion Gemma that people- Yeah...

[1:18:15](https://realalpha.app/v/8cGfJOU3AlQ/?t=4695) can play around with too. Will, will they ever reach parity? Is it just a matter of investment in money and data or like, you, you know? I would say a lot of it is data. If, if it doesn't work, why doesn't it work? My simple answer is that we, we reason left to right. We don't reason, you know, and diffuse into, into vision. At the same time, you like, I don't know, we-we, with autoregressive models, I think a lot of people like to focus on the, the, the, you know, step-by-step of the, of the tokens, but at the same time, it also can parallelize. Like we, we can't just take in a whole book at the same time as well, which those models can. Yes. So there, yeah, yeah. Um- And you can fake it in. Yeah. I totally get it. Yeah.

[1:18:48](https://realalpha.app/v/8cGfJOU3AlQ/?t=4728) So that's, I mean, that is the valid question. Like how come- Yeah... diffusion for text never took off and, you know, now it's like slowly, slightly better, but like how, how- I think it's just like a matter of which approach was, was earlier and has been like becoming for language model the standard. I mean, I think as your point, you said if it's more data efficient and the world has been doing a lot of scaling, there's a side there. That doesn't mean that there aren't pockets where there's use cases that you can get really good things at a certain size. Yeah. The example is like, yes, Diffusion Gemma is pretty good and it...

[1:19:21](https://realalpha.app/v/8cGfJOU3AlQ/?t=4761) meets a bar and can run really fast, right? If you want that, that's great. Yeah, I mean, on the autoregressive language front, it- it's-- I think it really comes down to it was simple, it was scalable- Yeah... it was there early. It was, like, a very... I mean, it's clearly been a, a very successful trajectory. A lot of time and, and money and research has gone into to make it not work. There's right now a lot of very exciting research on the diffusion front, you know, and I, and I'd like to separate the notion there's, uh, 'cause there's-- Diffusion language is really two camps, and I think a lot of people tend to not realize this because there are discrete diffusion language models which I almost don't-- I don't like to call them diffusion models really, and this is where, like, the Diffusion Gemma and, like, the Mercury code- Mm...

[1:20:03](https://realalpha.app/v/8cGfJOU3AlQ/?t=4803) like Mercury models live, which is, is more of this, like, masking style model where, where it, it jumps from a mask to a token, but it's still in the discrete space versus now there's a, a big wave of research right now going on continuous diffusion language models which those are actually operating as, like, you know, I don't wanna say flow matching models, but they're, they're, they're operating in this, like, essentially denoising space. So, so more like in- This was used for audio, if I'm not mistaken. The- Uh, audio generation... continuous diffusion. Continuous diffusion. I mean, well, it's definitely- Diffusion has also been used for- It's for everything, yeah... visual modeling, audio modeling.

[1:20:35](https://realalpha.app/v/8cGfJOU3AlQ/?t=4835) My last fun question there is, um, do we have a good way to do, like, GRPO style RL on pure diffusion text models? Yeah, we, we're doing this actually. Okay. So for Flux.3, we did this and, and it has had tr- tremendous, uh, gains that it follows, yeah. So Flux Action, this is coming to very, very recent launches and, and work that you're doing. Mm. How do you approach this? What is your-- what is the feedback and impact so far? Yeah, it's actually like how, how we approach it. Basically, you, you pre-train on, again, videos, images, audio, everything together, and then you have this very general model that understands the world.

[1:21:10](https://realalpha.app/v/8cGfJOU3AlQ/?t=4870) And then actually you can, can take a bit of action prediction data. Basically, you add kind of predictions in the state space of this robot, which is basically a kind of a parameterization of here this robot arm, um, which every joint will be described by a, a translation and a rotation, six degrees of freedom, basically. And then you predict that together with a video. And since the model can already reason about the video and what it needs to create or to, to fulfill a certain task in video space, it's actually very simple to then add this new dimensionality or this, this, this new dimension, this state space, how we call it, and then basically predict that as well.

[1:21:48](https://realalpha.app/v/8cGfJOU3AlQ/?t=4908) Given a chunk of earlier video, condition it on that, enter an instruction. Basically, the model creates the next chunk of actions to basically fulfill- Mm... um, this action. And this, this works extremely good. It's-- We, we can do it in the real world, like for real world robots. We can do it in a simulation. Like computer use is also kind of a mapping of that to the digital world, and we can, like we- Ooh... can make the model play games right and stuff like that. Mm-hmm. So this is all enabled by this, this single Flux.3 model. And then you see also that it's extremely capable because we have this very general pre-training, which is not only limited to, say, classic robot data, but actually all kinds of data that gives the model just this inherent world understanding that you also see then in performance.

[1:22:30](https://realalpha.app/v/8cGfJOU3AlQ/?t=4950) Yeah. And being a diffusion model is also pretty speedy, as we see here, um, with, uh, with, uh, with these distilled, uh, versions basically, right? Would you call this a world model or world action? Do you have a term you like to use? [laughs] It's a world-- This, this one is a world action model. I think the, the word- World action... the term world model I don't like to use. It's so overloaded. I'm in the same, I'm in the same boat. It feels like a hominy to me at this point. It's just- Let's, let's, let's call this a visually intelligent model because it can do all these, these basic things, right?

[1:22:55](https://realalpha.app/v/8cGfJOU3AlQ/?t=4975) I guess the, the distinction that I see in these models is you're not just predicting video or world, you're also prediction action states, right? Mm-hmm. Yeah. So one is what is happening in the world, what is the state, but also what is the next action to predict. So it's like- Exactly... sort of two outputs, right? Mm-hmm. So. Yeah. Is there a target frame rate, uh, that you target that is industry standard? So the- Because you don't have to predict out that far is, is, is my, is my idea. You mean in terms of the real-time factor? Like how, how- Yeah... it's like ten FPS, I think, um- Yeah, FPS... what the model currently runs. Very s-small model.

[1:23:27](https://realalpha.app/v/8cGfJOU3AlQ/?t=5007) And, and you're projecting out like one FPS or five F- Uh- Sorry, one frame, five frames, whatever, right? I would, I would have to look it up. I think it's- Yeah... it's a small chunk of, of frames. I would say like in between nine and 21 frames that it then predicts again. Yeah. And then the model is conditioned on the previous, uh, generations again. And what the camera-- When it runs on the robot, what the camera actually perceives, right? And then you basically predict this, uh, further. So it's, it's, it's a bit of a chunk. It's not, uh, running autoregressively, so to say, in, in, in time, basically, but it's a chunk prediction.

[1:23:57](https://realalpha.app/v/8cGfJOU3AlQ/?t=5037) What I'm driving at is like this video prediction side that is very much for predicting the results of what you do. Yeah. But so probably you need some kind of long range planner existing outside of that model that will coordinate the actions that- I mean, this, this is why we're discussing the language side earlier. Yeah. Yeah. Yeah. So as it runs right now, what we see is that there's no pl- like, like high level planner involved or something. Yeah. This is just a model running on a, on a, um- On a loop. Yeah. Some of the interesting things here- This is what we did with, uh, Dyna Robotics. Yeah. Um, Dyna has one, uh, Runway has one. I guess talk about the fine-tuning side.

[1:24:31](https://realalpha.app/v/8cGfJOU3AlQ/?t=5071) So if I wanna take your general model, fine-tune it for a new robot with a new task. Not something super crazy, but you know, something that's out of distribution it hasn't seen. Say it's my own home robotics. Yep. I can program in how to use a humanoid arm or whatever. Mm-hmm. Uh, what is that process like? How long does it take? How important is that in fine-tunability? Also, again, like since this model is so general and, and the pre-training was so nice, um, it, it takes like compar... Basically- Exactly, yeah... you, you need a bunch of hours. I think we showed one example, um, where we compared it with, um, I think it was Paleo5, which was also, uh, open source, right? Yeah. It took eight times less data to basically train on it.

[1:25:08](https://realalpha.app/v/8cGfJOU3AlQ/?t=5108) That was for industry application. It was like four hours of data that we, that we needed basically to get to a success rate that was like good enough for this basic application. And is that a thing you kinda wanna help climb on right now? Or is that like- Yeah. So we go back to the context thought. Yeah. We shouldn't be doing fine-tuning What we, what we are targeting and what we'll- Absolutely... working on right now extremely carefully and with a lot of focus in, in context learning for robotics. Absolutely.

[1:25:34](https://realalpha.app/v/8cGfJOU3AlQ/?t=5134) **Andreas Blattmann:** This is-- You should be showing a robot an action one time, and then it should be able to do this, and this would be truly a superhuman intelligence.

[1:25:42](https://realalpha.app/v/8cGfJOU3AlQ/?t=5142) Like, if you, if you look at how long it takes us as children to learn walking, to learn using our thumb correctly to grab this can, for instance, right? This is like weeks and even months of, of learning that it takes us to do this. Our vision is that robotic-- like an autonomous robot should be able to observe any action once and then be able to do it. Mm-hmm. That is what would be a clearly superhuman kind of visual intelligence, and that is what we wanna be reaching. What is the standard now? Like, how many demonstrations do we need? It's around like- Hours... four, four hours of fine-tuning- Four hours, yeah, yeah... you, you need at least. But this will- Yeah, and even, even four hours is pretty good. I, I would say 2027 will be the year where we will crack this task.

[1:26:21](https://realalpha.app/v/8cGfJOU3AlQ/?t=5181) Single shot learning. Yeah. Mid-journeying is an interesting one in here too, right? So is this like I wanna go from manufacturing to say maybe medical stuff like filling pill capsules, like specifically like different niches, for example? Is that where- I, I mean, the model, the model is trained quite generally. Yeah. So even, even the mid-train, like before fine-tuning, it might be able to handle many of the... Like even this version might be able to handle many of the tasks you throw at it. Yeah. The fine-tuning is really just nailing. You know, in the robotics world, you wanna, you wanna act...

[1:26:48](https://realalpha.app/v/8cGfJOU3AlQ/?t=5208) You wanna- You, you want a certain level of accuracy- Yes... you expect out of it. Exactly. This is just a standard- This is where, yeah... plain large scale mid-training. Yeah. And then we move on to this fine-tuning stage- Right... where we add the high-quality robotics data in, and then we get this, this, this further bump. But you, you see that already to a, to a, to a somewhat a good extent, um, on mid-training, the model can actually do these tasks. I guess the other interesting thing is it's a 7B, right? Yep. Is that mostly just constrained from these things are deployed- Exactly... on device? You wanna be having- Yeah... you want wanna be having a model that is as small as possible because- Yeah...

[1:27:20](https://realalpha.app/v/8cGfJOU3AlQ/?t=5240) you need to be able to act first on device if you wanna be adding this to a manufacturer, basically, and then also in real time, which- And this is on the backbone of Flux.3? Yes. [chuckles] And do we-- So is like the image and audio, all that stuff- All-- Like, this is, this is derived from this one model, basically. This-- We, we have this multimodal model. Yeah. That, that is really important to understand. This has been trained on a huge corpus of images, videos, audio, and from that, we can derive everything that we basically released this summer, which was Flux.3 video, Flux.3 video edit, Flux.3 action, Flux.3 image, Flux.3 image client.

[1:27:58](https://realalpha.app/v/8cGfJOU3AlQ/?t=5278) Yeah. We also derived an upscaler from it. We have a super nice 4K, uh, video upscaler, which is, I think, super fast also. Can only recommend. It's really good. And this is all powered by this one model, and this, this shows how training these very general models actually compounds then over time because you can derive- Mm-hmm... all these applications from them, right? Is there a difference between a video upscaler and an image upscaler? Isn't it- A video upscaler, we, we talked about the temporal consistency. Yeah. Obviously, if you would have a- The same upscaling... image scaler- Yeah... that could have like, like no temporal understanding, then you would have this flickering again.

[1:28:27](https://realalpha.app/v/8cGfJOU3AlQ/?t=5307) [laughs] This-- Especially in the high frequencies, you would then real-- Like, like you don't see this too much when you com... Like look, look at two images side by side, you don't see that much. But if you switch between the images, that's, that's one of the basic, uh, tasks that everyone who's looking at a lot of images does, just opening two images in two tabs- You do notice, yeah... and then, then switching left, right- [laughs]... and you see every single, single difference, and that would just amount to kind of flickering in, in- Yeah... temporal space if you wouldn't have this temporal component in the video upscaler. Got it. And that's what makes the difference from a image upscaler. Any notes on real robot evals and how to scale this?

[1:29:00](https://realalpha.app/v/8cGfJOU3AlQ/?t=5340) Like I, I see, you know, sponge in bowl three times. [laughing] Like... And, and at some level, like if you look at our Dynapod, there are literally people just one-off testing robots 'cause you have to test in real scale. But like, come on, two out of three runs on sponge in bowl. Just restart, do it again. You can be three out of three, right? [laughing] It's, it's a hard problem, and I think one, one, one, uh, thing that's constantly being worked on is just closing the sim-to-real gap, and this- Yeah... will get better and better and better over time. But to be honest, evals, no matter where you do it, it's a painful thing. Yeah. It's also painful when you, when you look at the content creation evals.

[1:29:35](https://realalpha.app/v/8cGfJOU3AlQ/?t=5375) Like ask three different groups of voters what a good image looks like, and you would get-- you, you, you will get three different answers back. You get wars over it. Yeah. We, we- [laughs] We, we have wars over it internally. We have wars over this. [laughs] Yeah. But that's e- evals are just for generative modeling. Evals are just hard. Yeah. Yeah. Is this already-- Like how developed is the commercial side? We will have- More of these coming... soon [laughs] I cannot tell, tell too much, but there will be soon first- We can in my goal here... DFL, DFL-powered robots in productions live. And it's open, right? 2027 will be the year of robotics. Oh. Oh, absolutely. Okay. Say more on predictions. Where, where do we go from this in 2027?

[1:30:13](https://realalpha.app/v/8cGfJOU3AlQ/?t=5413) How, how fast do things pick up? Single shot learning. Yeah. In context learning is a, is, is a big, is a big thing we'll see first. Is this a- We'll see first commercial applications of these things in real manufactories. That's, that's why it's actually good that we have a, this German base because there's a ton of, uh, industry data- Mm... and, um, obviously lots of potential for partnerships. People are really hungry for robotics, I would also say. Sure. So there's a, there's a good bunch of, um, of synergies that we wanna be using to actually make 2027 the year of robotics. I, I, I'm very excited about it. The research, the research on this is also moving very quickly, so I, I suspect- Yeah... that robotics will improve very quickly. Yeah.

[1:30:53](https://realalpha.app/v/8cGfJOU3AlQ/?t=5453) What about the side of like robotics is very physical manufacturing constraints. Like you can scale up tech, you can scale up models, but that's-- It's pretty hard to affect, right? So- Mm... predictions 2027, where, where, where, where are we realistically on robotics? Can I buy something off the shelf that can single shot learn, like? I suspect like later 2027, like people will be able to get some, some quite interesting robotics in their house. I mean, I-- Whether it's, you know-- Is it gonna be able to do like fine motor tasks and all this? I, I don't know. But I suspect like... And again, this is just entirely predictive. I've, I've no baseline for this.

[1:31:27](https://realalpha.app/v/8cGfJOU3AlQ/?t=5487) But I, I, I suspect next year There might be some, some interesting like, you know, at least niche for like the, the tech side of people like us. Not like, you know, uh, the average person would be picking this up, but some interesting potentially useful robotics in your house. And for manufacturing- Mm... I think 2027 will be a, a year. Because manufacturing- Yeah, absolutely. Yeah... is, like, more restricted, right, than, than kind of... It, it, it isn't that long tail less like, say, autonomous driving and stuff like that, right? Or, or like consumer robotics where, where we can just like do a- a- You would ideally like to do this thing to do anything you want.

[1:31:58](https://realalpha.app/v/8cGfJOU3AlQ/?t=5518) But for manufacturing, I think there's a very good chance that we will see like a lot of applications actually being unlocked by robotics in 2027. Very much so. And that's what we are also working towards, basically. Very much so. Awesome. I mean, exciting. It seems like very fun place. You guys have robot labs- [laughs]... image video model. Building the future. You're also- You- You're just missing the flying car. You mentioned Germany a lot- But it's coming... but you also have a pretty big SF office. I actually didn't know there was a big European side. There's a substantial SF presence. Yeah, it's called Black Forest Labs.

[1:32:26](https://realalpha.app/v/8cGfJOU3AlQ/?t=5546) Yeah. [laughs] And, and so like- And this is all like, like inviting everyone to come over to our place, uh- Yeah... 410 Townsend Street. [laughs] Uh, super, super nice. Yeah. Come, come by. And, and we, we, we, we're not a European company, not, like- Yeah... we're a company with roots both in Europe and the US- Absolutely... and we are acting globally, basically. That's fun. We got you on because you're visiting in town, and we're like, "Okay, we should- Yes... have you on." Yeah. How can people help you? Yeah. You hiring? Yeah. Looking for... I think just, just, uh, supporting us in marketing- Mm... talk about our models. Um, helping us also here, I think in the US, getting, getting our name out, out. Here in the community, we are well known. Yeah.

[1:33:02](https://realalpha.app/v/8cGfJOU3AlQ/?t=5582) I think for, for like in the broader society, not that much. There's the big players, right? That's also what we need to change in 2027- Yeah... really kind of getting our name out, because we have extremely strong technology. I think that, that, that's what we saw. We have a long history of having built like the core pillars of like what everyone is using, basically. It's now also time to get our name out, um, to the kind of broader audience. Yeah. But the, the tech community, I think we are well known, but- Yeah... apart from that- Tell your mom about Flux. So it's, it's very interesting that the overall gen- Ob- obviously, I'm a marketing guy, and I help companies in marketing all the time.

[1:33:36](https://realalpha.app/v/8cGfJOU3AlQ/?t=5616) You guys are very different, 'cause it, generative media in general is, has a weird, mixed reputation in the broader population, right? Hmm. Like, normally I would suggest, "Oh, have a Ti- TikTok or Instagram marketing campaign," but pretty much you'll be killed in, i- i- just because they hate AI so much. Oh. Right? And so i- it's very interesting to find the niches where this is, isn't regarded as AI. It's just normal, a, a thing you can do with tech. Mm-hmm. But it's not AI. Yeah. 'Cause I think the moment you say it's AI, the c- population will just- Yeah... have pitchforks. Yeah, yeah. I don't know if you have any response, or thoughts, or explanations about that.

[1:34:10](https://realalpha.app/v/8cGfJOU3AlQ/?t=5650) I think it's important to, to just always also emphasize that we, we, we take this responsibly. We're like, like n- not... Our models are not creating s- like avalanches of slop or something, right? They, they, they actually live in real use cases, uh, lots of like professional workflows, and they just help making the lives of, like, professional people who work with this- Mm... way easier, and just increasing productivity so much. And, uh- It's Photoshop. Yeah, e- exactly. It's, you know, Photoshop. Yeah. It's, it's so crazy, like how, how much more productive you are then. And that, that's really the, the kind of the, the goal of this on the content side. It's not about putting people out of work, not at all. It's just like making everyone's life so much easier. Oh.

[1:34:47](https://realalpha.app/v/8cGfJOU3AlQ/?t=5687) Yeah, I wouldn't say that. I, I'm just like i- if I worked on your team, like this is the, the my GTM problem right now, which is like, how do, how do we get people to basically, you know, go viral in the same way that maybe WhisperFlow goes a little bit viral 'cause they're like, "Oh, yeah, I understand how to use this. This is not AI. It's just WhisperFlow." Yeah. Or it's like, it's Gondola, same thing. I think they're, the, they're, the generative media hasn't had that crossover except for NanoBanana, and then- Mm... whenever ChatGPT had the- Image gen... Studio Ghibli moment. Yeah. Mm, yeah. Those kinds of like, that is the crossover where it's like, "Oh, like this is a fun trend that I wanna hop on." Yeah.

[1:35:23](https://realalpha.app/v/8cGfJOU3AlQ/?t=5723) There were moments that hit well, like your point of the see yourself as you're older, like that had a huge moment. Yeah. And people would look at themselves as their 80s, send it to their family. Like there have been moments, but nothing is super... I, I think one of the difficult pieces is it's actually very hard to predict one of these like viral moments. Like, I, like NanoBanana was their, like their internal name, and then they just kept it because it, it, like people just liked the name and it stuck. Mm-hmm. For context, actually, the one big use case that we saw a ton of usage on that like, you know, we didn't necessarily expect was image restoration as well. Mm. Yeah.

[1:35:54](https://realalpha.app/v/8cGfJOU3AlQ/?t=5754) So there was a ton of like, I saw amazing, amazing examples of people, you know, taking old- Bringing back... degraded family photos and then restoring- Colorizing them... colorizing them, fixing them up, but, but it, and it was able to keep like a, a very, very high level of, uh, of similarity to the, to what the original would've been. Yeah. And then, I don't know, that, that was actually one of my favorite moments that we've had, uh, you know, of people showing like their family these kind of restored photos and the reactions from it. And I, I really, really enjoyed that like positive moment that, that it brought. But like I said, like we, so often we can't predict like what the thing is going to be. You know?

[1:36:30](https://realalpha.app/v/8cGfJOU3AlQ/?t=5790) It's, it's, it's even with, uh, even with our old, you know, even back in the stable diffusion days, it was always surprising what the trend was, you know, online. Mm. Like, it, it just kind of, they just kind of- Yeah, you just never- They, they emerge. They emerge out of- Yeah. You cannot plan them. I think o- once you have them, you need to, to kind of exploit them, obviously. Yeah, yeah. And you should be always asking, "What is the next big thing- Absolutely... the next capability that it has not been able, or like, like that models have not been able to do? What would that be?" And then trying to f- to bake this into your model. Absolutely, yeah. Making bets. So- some of them will work, some others will not work.

[1:37:02](https://realalpha.app/v/8cGfJOU3AlQ/?t=5822) But this is just extremely important to, to kind of always have this mindset of, "Hey, what, what is the next thing we should be working on that will just bring a lot of joy or like productivity increases to people," right? Yeah. Well, that is the domain of research. You guys are doing amazing job, so thank you so much for sharing your story. Uh, I hope more people come to know you as a team because they've, they've seen a lot, a lot of the model buzz. But, uh, I, I feel like this is also the first time I've seen, I've heard the full story of PFO. Yeah. Yeah, no, thank you. Thank you so much for having us. Thank you so much. Yeah. It's amazing presentation. I mean, this, this was a... It, it's fun to finally have gotten to do this. Yeah. Yeah.

[1:37:37](https://realalpha.app/v/8cGfJOU3AlQ/?t=5857) I mean, it's a year in the making. But also, you know, I bought a bunch of, uh, MicroDucks, and, uh, you know, I'm looking forward to, to them running around. [laughs] Oh. Do, do, do you have... Like I, I wanna get my hands on one. Okay. I feel like I missed the window of these being available. Eh. You have connections. I'm, I'm sure I can, I'm sure I can- [laughs] Whoever out there can get me a MicroDuck, get me a MicroDuck. [laughs] I, I, I- I'll get you a MicroDuck. Okay, perfect. [laughs] I bought a bunch 'cause I know- Fantastic... I know I'm just, this is gonna be a hot gift for Christmas. Yeah, yeah, yeah, yeah. So. It's, you know, like, uh- But, but yeah, you know, robotics, 2027, that's, you know, one year away.

[1:38:05](https://realalpha.app/v/8cGfJOU3AlQ/?t=5885) Like, you know, we'll, we'll see, we'll see where, uh, this all learns. But I'm very optimistic about this approach, and I think you guys have explained it very well. Fantastic. Yeah. Amazing. Awesome. Thank you. Cool. Thanks so much. All right. Thank you. [upbeat instrumental music]
