Free with a sign-in.
The best 12 min of 98
Why FLUX 3 Trains on Images, Video and Sound — Andreas Blattmann & Dustin Podell, Black Forest Labs
Andreas Blattmann and Dustin Podell argue that training across images, video and audio can build visual intelligence for generation and robotics.
Latent Space · Shawn Wang (swyx) & Alessio Fanelli
- Andreas Blattmann · Black Forest Labs
- Dustin Podell · Black Forest Labs
The best minutes
- 57:12A small dataset unlocked general image editing in hoursPro pick 4 min
- 34:53A video model learned enough 3D to fine-tune in 1,000 steps3 min
- 40:12Training should focus compute on informative denoising stepsPro pick 5 min
Every moment plays free. PRO marks a write-up that's in Pro.
Also worth watching
- 1:24:25BFL aims to crack one-shot robot learning by 20278 min
- 1:13:06Autoregression is more data-efficient; diffusion can sample faster5 min
- 27:39The SDXL refiner came from a different, unreleased model6 min
- 47:01BFL was founded to capture value its models had created4 min
- 1:05:17More than ten image references are hard to review5 min
Skip the 17-minute opening introductions and career histories and the Deepgram sponsor break.
Full analysis · 5 min read
Real Alpha · Nov 16
Andreas Blattmann says about 1,000 steps produced a 3D model
- Blattmann estimated about 1,000 fine-tuning steps yielded a 3D model.
- That suggests transferable spatial structure, but does not measure reliable physical understanding.
- Teams should test what transfers before funding shared image, video, and action systems.
Free with an account
Read the rest of the analysis
6 sections, the 5 other voices it tests Blattmann against, and what it means for you. Every episode we cut gets one.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
Signed up before? The same buttons sign you in. The episode, its moments and the transcript stay free.
About 1,000 steps produced a 3D model
Blattmann described taking a model trained on plain 2D video and fine-tuning it on 3D data. He said he thought about 1,000 steps produced an explicit 3D model.
That is stronger than a visual analogy. It is evidence that video pretraining can transfer into a different output representation.
But the episode gives no 3D dataset size, baseline, or scored quality comparison. Those omissions limit what the step count can establish.
For example, Blattmann said generating a view around an object requires inferring its unseen side. That makes spatial priors in video learning plausible, without proving complete geometric understanding.
The result supports a testable training hypothesis, not a general score for spatial intelligence. Teams still need to establish whether video pretraining makes the 3D fine-tune more effective.
Our read
Our reading is narrow. The experiment supports transferable structure, not a general measure of spatial intelligence.
Observation builds priors but does not establish robot control
Blattmann said models first observe large amounts of video, much as babies observe before acting. He presented interaction as a next step, not a solved capability.
That makes the 3D result relevant to robotics, but not sufficient for it. Predicting a plausible view differs from controlling an arm and checking the result.
Still, Blattmann said closing the loop with physical interaction may help models learn cause and effect. That marks a boundary around the passive-video thesis.
Sergey Levine, on Minus One in October 2026, described task planning, robot-specific annotations, and visual reasoning as routes to generalization. Those are additional ingredients, not evidence against video pretraining.
Teams should test which capabilities arrive from pretraining, then measure what interaction data adds. Otherwise, a rich visual prior can be mistaken for task competence.
Our read
Observation may create a strong starting representation. The episode leaves open how much interaction is needed for dependable action.
One multimodal model remains a commercial thesis
Blattmann said FLUX 3 is one model trained on images, video, audio, and actions. He frames those signals as a route to visual intelligence, not merely a wider image generator.
That is a clear product thesis, but the episode gives no independent measurement of gains from shared training.
At 1:27:43, he listed video, video editing, action, image, and image editing products. He also named a 4K video upscaler, but not audio as a derived product.
The distinction matters. Audio appears as a training modality, not in that spoken product list.
Podell said BFL formed partly because the team felt Stability could not capture value its models created. That is relevant context for a company presenting reuse as both a research direction and a business opportunity.
Sagar Vaze, on MTS in October 2026, also described multimodal training from the start, with images included early. That parallel supports the strategy as a live research direction, not BFL's claimed cross-task gains.
Our read
A shared backbone could make capabilities reusable. The episode explains BFL's bet, but does not independently measure its returns.
Compute allocation and data quality shape transfer
Blattmann said uniform sampling along flow matching's noise-to-data trajectory can waste compute on less informative regions. His team's work asked where training effort should go.
That could matter more than raw scale when a lab has a fixed compute budget. But the episode gives no ablation magnitudes or cost curve, so it offers no purchasing formula.
SDXL's refiner also complicates any single-model story. Podell said it came from a different, unreleased architecture with better fine detail but weaker structure.
That example shows how separate models can complement each other. It does not prove that a shared model cannot work.
Nishant Ranka, on Latent Space in November 2026, described checking data with spot checks, model runs, ablations, and embedding clusters. His practical point was to test whether data teaches the intended capability.
The guests described data quality and quantity as a tradeoff that must be tested at scale. Teams should run their own scaling ladder rather than assume a universal recipe.
Our read
The episode favors better allocation over simply adding compute. The right allocation remains task-specific and needs measurement.
Small editing datasets can unlock existing model structure
Podell said an early editing experiment used a tiny, high-quality set of labels describing relationships inside images. He recalled maybe six hours, maybe four, for the initial training run.
For instance, he saw the model place a logo on a shirt. That was an early sign that its pretrained spatial relationships could support an edit.
But he said the result was not good yet, and the episode gives no measured success rate for that first run. The anecdote suggests a capability, not its breadth.
Podell later called the larger effort mostly a dataset project, followed by distillation. That makes the account useful for teams testing targeted data, but not a recipe they can assume will transfer.
BFL described human comparisons for ContextBench and said more than ten references become hard for people to follow. More context is not automatically more useful if reviewers cannot track it.
Stefano Corazza, on The TWIML AI Podcast in November 2026, said design editing differs from image editing. Design benchmarks can favor first-pass images, while real design work changes existing compositions.
Our read
Targeted data may expose capabilities already present in a pretrained model. Product teams still need task-specific evaluations.
Robotics still needs demonstrations before one-shot learning
Blattmann said the current workflow needs at least about four hours of fine-tuning. He also called even four hours pretty good.
They cited a comparison using eight times less adaptation data than an open-source model. Blattmann hedged the comparator's identity.
That is an efficiency result, not one-shot learning. The action model predicts robot state alongside video, then runs in chunks conditioned on prior observations.
At 1:24:15, the host summarized the setup as a loop with no high-level planner, and Blattmann agreed. The demonstrated system still depends on task-specific data and repeated observations.
Blattmann predicted 2027 would be the year they cracked one-shot learning. That remains a forecast, while the current evidence includes task-specific demonstrations and a data-efficiency comparison.
Brett Adcock, on Moonshots in November 2026, said choreographed robot demos do not prove practical autonomy. For operators, narrow pilots with repeated task-success measurements are the stronger test.
Our read
The robotics thesis is promising, but the gap to one-shot learning remains explicit. Treat 2027 as BFL's target, not a delivery date.
What it means for you
- Audit 3D transfer. Compare video-pretrained and baseline models after identical 1,000-step 3D fine-tunes. Score geometry consistency, not just whether each model produces an output.
- Price the training ladder. Test data quality and quantity at several training scales. Track capability gains against compute instead of treating BFL's allocation approach as a universal recipe.
- Separate editing tasks. Evaluate image edits separately from design tasks involving layout and readability. Include reference-heavy work and human comparisons, not only first-pass image quality.
- Log robot adaptation. For each pilot, record demonstrations, fine-tuning hours, and success across repeated trials. Compare results across tasks before treating one data reduction as general.
What to watch
- Whether BFL demonstrates one-shot task learning in 2027, its stated target.
- Whether the reported eightfold data reduction holds across different robot tasks.
- Whether shared image, video, audio, and action training produces measured gains across applications.
Send this to product and robotics leads.
Watch the minute where Blattmann gives his estimate of roughly 1,000 steps.
Written by Real Alpha from the episode's transcript and the other moments it links, then checked against the transcript. Quotes are the speaker's exact words.
Minute by minute
A Pro write-up, open to you here. Try Pro free for 7 days
A Pro write-up. The minute by minute is in Real Alpha Pro; the episode, its moments and the transcript stay free.
Read it with Pro, 7 days free- 4:22Diffusion took root in Germany
Blattmann says he and the early team began studying diffusion in Germany, drawing on their physics backgrounds; work at Heidelberg led to latent diffusion and later Stable Diffusion. He says they founded BFL in Freiburg, near the Black Forest, rather than relocating to a tech hub.
- 9:22Stable Diffusion drew Podell into research
Podell says he had spent about 10 years as a software developer before Stable Diffusion’s release prompted him to leave his job. He joined Stability AI as an engineer, then learned by reading papers, experimenting with models and working with researchers in the open-source community.
- 17:16Video can teach implicit 3D
Blattmann says BFL wants models to learn intuitive physical knowledge from natural signals, not just language. He argues that 2D video can teach implicit 3D understanding, such as how the unseen back of an object might look, and says interaction could further develop that understanding.
- 27:31SDXL split structure and detail
Podell says SDXL brought multi-aspect-ratio training and more improvements into the base model. He describes its refiner as a separate diffusion model that added high-frequency detail, while Blattmann says the work helped show how structure and texture emerge at different points in denoising.
- 33:57Temporal data matters for video
Blattmann says latent diffusion naturally extends to video, but stresses that data quality and motion matter as much as the algorithm. He cites an experiment where a model pretrained on 2D video produced a 3D model after 1,000 fine-tuning steps on 3D data. The guests say temporal modeling helps prevent flicker.
- 39:56Flow matching targets useful compute
Blattmann says SD3 marked BFL’s shift to rectified flow and transformer architectures. He describes flow matching as a simpler way to model a path between distributions, which can support faster sampling. The guests also discuss allocating training compute to the parts of the denoising trajectory that matter most.
- 46:50BFL began with a small team
Blattmann says 13 or 14 people started training FLUX.1 after a founding offsite in a French chalet, and the model launched three months later. Podell says the team also wanted to capture more value from its models than it felt it could at Stability AI. Blattmann said BFL had raised $450 million and that revenue had passed the last published figure of $100 million.
- 51:38Video was BFL’s longer-term thesis
Podell recalls a photo-personalization app that drew a surge of interest, and says the team saw image generation as a broad technology rather than a single use case. Blattmann says BFL had long viewed video modeling as a way to understand how the world works; image generation gave the company a successful starting point.
- 54:54Kontext makes edits iterative
Podell says FLUX Kontext began as a pitch to use a pretrained model’s spatial knowledge for general image editing. A small, high-quality dataset describing relationships in photos showed early promise after roughly four to six hours of training; the team then expanded the project and distilled the model. He says the goal was to move beyond repeated generation and selection.
- 1:03:13Data and evaluations depend on the task
Blattmann says useful training data depends on the problem and stage, and teams must test the trade-off between quantity and quality; he notes that robots need egocentric data. Podell says FLUX.2 was trained to handle 10 reference images. He also describes Context Bench as an arena-style comparison and says teams should test models on real use cases.
- 1:13:06Autoregression and diffusion have trade-offs
Blattmann says autoregressive models use training data more efficiently, while distilled diffusion models can generate in fewer steps. He sees strengths and limitations on both sides and says the debate is unsettled. The guests add that conversational editing depends on training and data, not only on an autoregressive architecture.
- 1:20:39A shared model predicts robot actions
Blattmann says BFL pretrains on images, video and audio, then adds robot action states so the model can predict action chunks from video and instructions. He says the approach works with real robots and in simulation; one industrial task required four hours of fine-tuning. The guests say a 7B version is intended for on-device, real-time use.
- 1:30:07BFL expects robotics to expand in 2027
Blattmann predicts commercial manufacturing applications and progress toward one-shot learning, pointing to industry partnerships in Germany. Podell separately speculates that niche home robots could become useful later in 2027, while noting that this is only a prediction.
- 1:32:12Photo restoration found an audience
Blattmann says BFL wants FLUX used in professional workflows to improve productivity. Podell says a surprising use case was restoring and colorizing degraded family photos while preserving their similarity to the originals. The guests say it is difficult to predict which uses will catch on.
Published Nov 16, 2026.
Transcript · 1 h 39 m read
Latent Space, hosted by Shawn Wang (swyx) and Alessio Fanelli, with Andreas Blattmann and Dustin Podell.
From the transcript Latent Space publishes with the episode.
Happy to bring to you our episode with Black Forest Labs. Our guests today are Andreas Blattmann, he also goes by Andi, who is co-inventor of latent diffusion, the technique that eventually got scaled up by Stable Diffusion and Stability AI, and they eventually split with a bunch of, uh, the core Stability AI image gen folks to form Black Forest Labs. And Dustin Podell, who I also met with Anj, uh, at one of the stories that we tell, and, uh, who worked his way up from just being a regular engineer, a freelance, like, web developer person, into a model and research lead at Stability AI, and now, uh, Black Forest Labs, creator of SDXL, Control LoRA, post-training lead on SD3, Flux Ultra, Flux.1, and FLUX Kontext.
And we go into all of these details. Um, as you may know, sometimes we're very honored to have an extended history with some of our guests, and this is one of them. You might have heard in our Anj Mittal episode where we talked about introducing each other at one of the NeurIPS's [smacks lips] and meeting Black Forest Labs for the first time. And, uh, this was right after Stable Diffusion. But what you may not have heard, and we only tell this story in this episode, is how latent space itself was started because of Stable Diffusion, directly, uh, crediting the launch for me basically quitting my job to go all in on AI based on what we saw.
And today's guest, Dustin, also [chuckles] had that experience, and so did Vibhu. And so we have a very, very strong tie with the team that basically led up to Black Forest Labs, and now we're also interested in what's next for them. Like, they have created some of the best open and closed weights models for image generation and video generation. I think especially FLUX Kontext, if you're not aware of image editing models, I do think that this is in the ex-in the limit one of the things to fully replace Canva and, um, and Photoshop and a bunch of other workflows and editors that you might use in a normal creative workflow.
Read the whole transcript
It is not going to replace the top end of the market. But for regular, normal, everyday edits, why would you not want anything else other than to query a model and ask it to do, uh, image editing for you? Uh, we do this obviously with this kind of video editing. It's all Clanker done, and I do review it, and they're also moving into robotics. And so this ties in with a miniseries of model labs that have been pursuing this direction, building world models from image gen, like with Runway, uh, as well as, um, video models and now robotics models. And so I think a lot of things rhyme in a lot of our recent episodes, like even the artificial analysis episode that we're gonna release soon, they are also going into robotics, and, like, it seems pretty s- obvious that when you see a whole bunch of smart people who are at the frontier all leaning one direction, that you're probably six to 12 months out from some kind of really big breakthrough.
This has been a consistent theme over my last four years of podcasting. It's not gonna change now. So look out for it. Um, that's the [chuckles] most obvious call-out I can make. But there's also a lot of interesting lore for people interested in the general diffusion space. Uh, we're gonna d- do a little bit more over time. I, I'm thinking about an AI- AIE LA, where we talk about the generative media field and the creative arts and how, uh, that complicated relationship is evolving, um, and I think maturing, to be honest. And finally, for those longtime listeners, uh, we are starting to open up to advertisers, people that we really admire and respect and wanna showcase what they're working on.
Um, the first one was with Supabase, which I think a lot of you enjoyed, and the second one today is brought to you by Deepgram, and you'll hear more from us at the midpoint of the episode. [upbeat music] You should be showing a robot an action one time, and then it should be able to do this, and this would be truly a superhuman intelligence like- At Stability that we couldn't essentially leverage the value of what the models could be as well as the mistakes of not being able to actually capture the value of the models themselves. We raised all in all four hundred and fifty million, uh- Oh my God... dollars of funding so far. Yeah. Currently, the last, uh, that was published in, uh, December was one hundred million in revenue.
BFL was not founded to be an image generation lab. Video modeling is actually the central means to understanding how our world basically works. It was maybe, like, six hours of training, four hours of training. It, it was, it was a very, very short training run, and all of a sudden, like, I saw what felt like a clear generality coming out of it, 'cause it was [upbeat music] Okay. Finally, our BFL pod. Welcome, guys, Andreas and Dustin. [chuckles] Great to be here. Thank you so much. Thank you. Yeah, definitely thank you for having us. I know we've chatted about doing this for a long time. Yes.
So we've had Anj on a couple times, and that's where I met you, at Vienna, I think one of the ICMLs or- Was this, like, one of the, like, was, was the, like- The-... rooftop, uh, event, or, or what was it? There was a year where there were two Viennas back to back. Yeah, yeah, yeah, yeah. That was first ICLR, and then it was ICML. Yeah, yeah. I don't know how they do this thing. [laughs] Yeah. Anyway, anyway, uh, and, and at the time you guys had-- were, like, just about to, uh, launch BFL. Like, we had a dinner and, like, you were like, oh, and thinking about what's next after Stability and, uh, all that.
So, uh, anyway, congrats. How do you like, guys, like to introduce yourself? Maybe we'll sort of get a crisp, uh, self-intro up front. So I'm Andi. I'm a co-founder of Black Forest Labs. Been developing diffusion models for, oh, wow, I think seven years now. Uh, met Robin and Patric way, way back in the day. I think it's, yeah, seven years now since we met at Hei- in Heidelberg. Uh, I'm from Germany o- uh, originally. And there we started looking into, yeah, just diffusion modeling for images. Uh, we were all having a kind of a physics background, um, in some sense, and this, this paradigm of just, like, mapping this general physical phenomena of diffusion to machine learning and generative modeling had a, a kind of a, a philosophical kind of beauty to us.
That's why we got into this. And then, yeah, developed a lot of the, I guess, fundamental algorithms that are powering diffusion models today, or image generation, video generation, and also action prediction today. I guess we'll talk about this later We'll talk about it. But, um- Yeah, uh, this of, of a kind related to... It's not immediately obvious to people, image generation to action prediction. Yeah. But we can talk about that. Uh, and then one, one thing I did wanna comment is a lot of people don't know the relationship. Black Forest is a forest in Germany? It's a mountain range, actually. Ah.
But, um, yeah, it's a, it's a region in Germany, basically. And you named it after that. And finally, Robin and me, we f- we first met during our PhD, and then we figured out that we were actually born, I don't know, 10 miles away from each other. Um, and that was, that was pretty fun, and then eventually that led us also, uh, to come back to Germany. Beautiful. [laughs] All of Europe looks like this. Yeah. You can see the lab right in the middle there. [laughs] Yeah. Yeah. Yeah. Um, so- So you named it because you're from here. Exactly. We named it because actually, um, we founded this research lab, and back in the day, we could basically pick where we wanted to do it.
No one said, "That's okay. You have to come to, say, SF or, or whatever place." So we decided, okay, we wanna be actually starting this at the place where we were born in Freiburg, which is, like, at the foot of the Black Forest, basically, or a city in the Black Forest, and that's how it came, and that's how, uh, then the company also started there. Yeah. ComVis, also Heidelberg? ComVis was in Heidelberg. Exactly. That's where we met. We then later moved on to Munich. That was all- Which is the team that did latent diffusion... through our PhD. Exactly. And, and that is when we developed this latent diffusion algorithm. Yeah. I really like your s- uh, name by the way, your podcast name- Yeah... Latent Space. Uh, I, I, I, I would actually be curious how that came together. Uh- I can talk, I can talk about it- Yeah... but, yeah, go ahead. Uh, yeah.
Um, and exactly, we developed then this latent diffusion algorithm at the university, and then based on this research project, we scale out the algorithm which led to Stable Diffusion, um, then effectively, and yeah. Since then, it's been a, a constant, uh, journey on a rocket ship, you could say, which is super, super fun to, to have. Yeah. And we've met amazing people like Dustin as well, um- Yeah... who's, who's one of the, the, the kind of oldest and, and, like, mo- yeah, mo- most, I think, relevant researchers that we have. But yeah, we'll, we'll learn more about that [laughs] in a- We'll, we'll gas him up la- later.
[laughs] Uh, I think latent diffusion and just g- generally, like VAEs as well, and, and then for us Latent Space, uh, is this general idea, and, and we- even our logo is, uh, actually a VAE, and it's this idea that you can translate things into spaces and then explore things that are adjacent to what exists today. Um, and then you can sort of retranslate back. Yeah. And I thought that that was, like, such a cool insight for creativity and generation that I never really understood until I studied Stable Diffusion. Like, the launch of Stable Diffusion changed my life. I started Latent Space because of Stable Diffusion. I quit my job because of Stable Diffusion. Oh, wow.
'Cause I was like, "Okay, everything I know about AI is wrong because I can run this on my laptop." Yeah. And, like, if I was wrong about this, what else am I wrong about? Uh, so there, there's that. Um, but and then there, there's another relationship with, um, Terry Pratchett, which I, I don't know if people have read. Writer, right? Writer. He has this concept of L-space, uh, which is library space, um, where he has a, a orangutan that's, like, the librarian in Unseen University, which is wizard university, uh, and it basically is a library with all possible books. So if you have a library with all possible books, you can just theoretically go in there and find a book that is right for you, uh, and that is Latent Space. Yeah. Uh, so I originally called it L-Space.
Uh, one of my readers donated Latent Space, and then it became Latent Space. Amazing. [laughs] Dustin, like, uh, yeah, what was your journey into diffusion and AI in general? Yeah, yeah. So I, I came from a little bit of a different background than, than Andy and Robin and team. So I'm, I'm not from the German side, but I've still had a fantastic time working with these guys. And, uh, honestly, it's funny you mentioned that you quit your job when Stable Diffusion came out because, uh, the original Stable Diffusion I was not working with these guys yet, and it was actually a cause for me to quit my own job at the time.
So I, um- You caused mass unemployment. [laughs] Yeah. I'm actually curious, like, the numbers of people that just com- like, changed their life direction when Stable Diffusion dropped. I, I would actually be very curious to know kind of the branching effects from that. Nice. But I don't know, we can, we can theorize on it. [laughs] That's actually super interesting because here in SF, in the Bay, it had this huge impact, whereas in Germany- Mm-hmm... where we stayed back in the day, that day, people didn't really notice it because, like- Oh... the people were so, so far away from tech. Mm-hmm.
Like, there are a couple of people obviously that are, that have been involved in the open source, uh, community also in Germany, but the majority of people didn't really notice, uh, that drop in summer 2022- Mm-hmm... which was really interesting. And then you see what happens here, like he-- Yeah. That, that was... To, to us, that was really kind of super interesting. Yeah. But, but anyway, so I, I guess to give a little background on me, so I, I, uh, at BFL, I, uh, I'm one of the founding researchers that joined in with the, uh, we had a, a bit of a larger team that I'm sure we'll get into the history of kind of how that all came to be.
But my background is not actually traditionally in, you know, the ML space whatsoever. I was traditionally a software re-research- or, or a software developer for, you know, around 10 years or so. Early in 20- you know, around the 2020s, I became very, very interested in just, like, open source AI. I saw, like, the kind of the early work in GANs and other, you know, in generative media, and I was tracking with it. I was experimenting with it a little on the side. It got me quite excited. I mean, and I remember following originally the, the latent diffusion work 'cause actually I was, I was following that and quite excited about it.
And then, uh, the one, you know, one of the things that kind of tipped me off to go, "Okay, maybe this is something a little bit, you know, a little bit more serious," is actually, uh, shout out to, uh, Midjourney V1. Wow. Um, back when that came out and I, I remember thinking, like, "Okay, this, this is actually going in, in a direction. I, I can see this being quite impactful." And then the one big moment, the big drop that really changed everything for me, as we talked about, was the Stable Diffusion release. And at the time I was working at a software studio down in Los Angeles. Stable Diffusion came out, and basically that day I turned in my resignation- [laughs]... and said, "Okay-" But what was, what was the plan? Like, what did... [laughs] The, the plan?
So, so the way, the way that I saw it, the way that I, like, envisioned it at the time- I saw this thing that felt like, it, it literally felt like a magical thing that came out, and I viewed, okay, the people that are working on this must be like arcane wizards, and I need to do whatever I can to understand like what magic or what runes or what things that they did to imbue in this to make this happen and let this go out to the world and have this, have this impact. It wasn't like I was just totally disconnected. At the time, I was in the Stable Diffusion Discord. I actually had friends in there. I was, I was kind of interacting with the community.
So when I, when I- So what, what, was it viewed as basically open source Midjourney, or like, like do people not make the connection? There, there was... I mean, I, I was in both. I remember there were a lot of kind of, uh, discussions around this, like, oh, Stable Diffusion was, it's having such a... You know, there, there was a lot of discussion, and I remember the Midjourney Discord and the Stable Diffusion Discord around that time around the, the impacts it was having. I mean, it, it's been a long time so I can't remember the exact- Yeah... nature of it. I don't wanna speak on the, you know- Midjourney's doing fine. Like they're doing- They're doing good.
I mean, they're, they're, they're doing great. [laughs] Um, but for me it was more just like I was very interested in the fundamental technology, and then the fact that I could now run this at home. And then really the thing I saw was then just this community of developers actually like taking it home, developing on it. I was hacking away at it. I was meeting a bunch of other really, really smart, great people, some of which have, are actually at BFL now that have, have come along for the journey. That moment of just community interaction and, and learning and developing and building together was s- really an impactful moment for me.
Lucky, luckily enough for me, I was able to find a kind of a job opportunity to get myself into Stability at the time, where, uh, you know, th- this, this was starting. And it, it, I didn't actually start working with these guys right away. It took me a while to kind of, to kind of work my way up to that, and, and we can go into that history later if we want. But- 'Cause Stability had a bunch of different m- uh, teams, right? It was a- Stable LM, Stable Music. There was- Yeah, Harmoni, yeah, the audio team Yeah. It, it was, it was like, I don't know, the way I would kind of describe it is like, uh, if you grabbed a bunch of... Uh, Discord was kind of the space that a lot of this happened at the time. [laughs] There were...
I don't know, I don't know why the whole AI- It's like if a Discord had a- It was a-... a physical manifestation [laughs] It was, yeah. It was like- Effectively it was a melting pot of like very different people that came together who were just like interested in this- Yeah... emerging new technology that came obviously, and that's also how we, uh, like this is one of the m- most exciting things about our team. Yeah. Our founding team is, it comes from very different backgrounds. Like we have this hardcore academia research branch, and then we have people who were from the open source community, like, like Dustin, um, who actually got via a very different kind of trajectory into this space.
But I think this is what makes our team so strong. Yeah, absolutely. We have these diverse backgrounds, and there's, yeah, there's rarely a team I think that, that, that has this history, also this, the kind of joint history together. Yeah, yeah, yeah. And this is just amazing that, that like- Yeah. Yeah... yeah. And we can go into more of your other co-founders first. Yeah. Uh, I, I did wanna... So I was also in that space, right? Like literally, you know- Yeah, yeah... I was also very influenced by Stable Diffusion. I did the wrong thing. [laughs] I basically went down the path of ComfyUI where I build UIs for Stable Diffusion. You seem to have done the right thing where- [laughs]... you went to research.
Well, it, it's funny you mentioned Comfy, 'cause ki- like I used to work directly with... I mean, Com- Comfy joined somebody that I- We've had him on the pod, yeah. Yeah, yeah, yeah. And he's- And I, I think he, he moved here. Yeah, yeah. Yeah, yeah. He's, he's, he's like my neighbor. How does a... That you, he's a researcher. How do you become a researcher? [laughs] I'm someone that's like, when I get interested in something, I am obsessive to the point of I cannot put it down until I pull it apart and figure out every aspect of it. And so to me, I'd already been experimenting with pulling the models apart, talking to a lot of different people in that open source space, so not just like developers, but other researchers.
I mean, remember talk about another Discord, the, you know, the Eleuther Discord back then. At that time, I met a lot of people who I w- I was just learning from, and I was reading a lot of papers and just trying to uptake and understand for myself. Joining Stability was also another major point for me. Originally, I didn't join as a researcher. I joined as an engineer. Mm. But all of my free time I spent with essentially any researchers I was able to make friends with, you know. Shout out to, uh, Dr. Scott Hawley there who worked on the audio team, who mentored me quite a lot there. And at some point, I was able to, to sneak my way onto the, uh, onto the cluster there and got myself access to, you know, some idle compute.
[laughs] And I, I, I really just like, I treated it like I treated how I learn most things, which is just open source hacking, trying things, reading papers, trying to implement different pieces of work, and just seeing how, I don't know, how models trained, how they learned and, and, and I mean, that was really it. Yeah. Like there was, there was no magic to it other than just I was excited [laughs] and I just wanted to learn about it. I wanted to tear it apart until I understood how it worked. I mean, this is open source. It is better. Yeah, yeah. I think that that's generally how you learn best, just trying out things- Yeah...
and then like- And he was also Eleuther? Yeah, yeah. Yeah. Affiliated, but not on the, uh, image gen side. But back then everything was like very hackable, right? Yeah. The, the delta between open models was like pretty drastic between what was closed and good, right? So people on language model side were doing post-training. That's where like a lot of the news guys started. Mm-hmm. Uh, in this, you know, you can look at early Stable Diffusion- This look, this looks so fast and then like it, it-... uh, and, and like where we get to today. Yeah, yeah, yeah. So it's pretty crazy. But you both had pretty, you know, impactful roles. Like you did SDXL. Mm-hmm. You did latent diffusion. We'll get through all the history, but for people that haven't super followed the space, where are we today?
Like what's the, what's the state of things? I know Flux.3 video is out. Uh, Flux.3 action, we're now like 7B doing robotics. What's the latest in this space with you guys? We started as an image generation lab- [laughs]... when we started Black Forest Labs, right? As a kind of legacy of Stable Diffusion and the likes. Um, and that has been I think our, our thing for the first year, but then we realized, okay, you actually, image generation is nice, but this technology can unlock so much more. Mm-hmm. And that's when we slowly went into what we call visual intelligence, which is basically trying to get our intuitive intelligence that we people have when we s- say when we catch a ball and just understand physics without actually knowing how to mathematically describe it or something, or when we grab a bottle and just know exactly how to do this because we've done it 100,000, maybe not 100,000 times, I don't know, but a lot of times, uh, in our, in our childhood, right?
We wanna be transforming that Knowledge and that form of intelligence, which is very different from language model intelligence, into models. And learning based on pixels, which means not only images, but also videos, and maybe also audio, say not pixels, but natural signals is a, maybe a better term- [laughs]... that really unlocks this. And that is what we realized, and we realized, hey, we have the algorithm to actually do this, which is diffusion or flow matching. And that is when we just said, "Okay, we need to extend from image generation to basically all the natural, uh, representations," which led to FLUX 3 basically, which is one model that learns multimodal from images, videos, audios, also actions.
We can talk about how this comes together later. To really get this, to, to keep it with Daniel Kahneman, I don't know if you read- Thinking, fast and slow... Thinking Fast and Slow. It's a kind- system one intelligence. It's not system two intelligence. Mm. And you can maybe say- Oh, my God... maybe, maybe- Everyone uses system one models now. [laughs] May- may- maybe, maybe language modeling- The original, yeah... is system two, but we are focused for now on system one, which is extremely important for everything that is intuitive and that requires real-world interaction. That's why at, on a high level, it's also so good for robotics. So that's been the journey, and now we're really focusing on that.
And in the, in, in the end, obviously, um, it's about combining both of these systems into this broad, very, very broad way of intelligence that humans also have. I wanna go into sort of, uh, history and all, all these things. Uh, but just to also recap for the, for people at the front, you're already making a s- sort of the money as a generative media company, right? Exactly. Um, I don't know what the, the numbers are and, and maybe you can sort of recap there. I, I think you have, you have some- You raised, like, you raised $300 mil on- You raised a ridiculous amount of money... multi- yeah.
Yeah. So- We, we raised, I think- Any re-... uh, all, all in all, $450 million- Oh, my God... of funding so far. Um, yeah. So there's way more than- A lot of it from Squa- Twitch and stability... you know, way more than stability. Yeah. Uh, any revenue milestones that you've, that you've announced? Kahneman Slow is currently- Haven't. Yeah... the last, uh, that was published in, uh, December, was $100 million in revenue. Yeah. We've crossed that since. I can, I can tell you that. [laughs] Clearly, clearly, clearly you've crossed that if you raised 450. Um- Yeah... yeah, exactly. So- So, you know, yeah. A- anyway, I'm just saying, like, you did not have to pivot. Great. And to be clear, we're not pivoting- Yeah, exactly... we're just extending. Extending. Yeah, yeah.
Like, this, this model is really bu- it's trained on everything. It's, it's always been the plan. And obviously to, to create artworks, you also need this visual kind of intelligence. To create a, a video that we see here- Yeah... you need to know that, how physics play out. I, I need to know that the model needs to understand this is actually interesting. Uh, but, but the model needs to understand that the arm of this, say, this, this, this, uh, capybara or whatever it is, is not just falling away, that it has to follow some law that is described by physics. It's just not describing the law itself, it's just describing the output in a similar way that we humans don't understand physics, but we understand that if I let a ball fall, it just falls downward. Yeah.
Well, I think there's, there's different, uh, concepts of a world model, and obviously that's something that, uh, we can get into, where can you learn... This is sort of the bitter lesson ver- version where you just learn on a bunch of input sources- Mm-hmm... or you have to actually physically structure the 3D model and, and kind of learn from that. No, no 3D model at all. Yeah. [laughs] Just learning on, ba- based on natural data. We, we also, we have a bit of tri- triangulation be- like, from our eyes wise, right? We have two eyes, and that gives us a kind of slight understanding of 3D perception. But effectively to a very large extent, even if we would, like, close one eye, we could still kind of learn a lot about the world.
That is what these models are basically learning. Like, you just learn from natural data, 2D video data. That obviously has a lot of 3D understanding, because if you wanna generate a pan around an object, you need to be hallucinating how it looks from behind. And if I look at you, I have a pretty good understanding of how your back looks, basically. Yeah. Because I've just seen it before, right? So this is gives you an implicit 3D understanding, but it's really trained on kind of 2D videos. Yeah. I, I'm trying to steelman the counterargument- Uh-... the, the opposite argument, right? Like, this is like the Gary Marcus, like, "Oh, LLMs don't learn anything," or... I, I forget who are the famous critics of this kind of stuff. But basically, it's like, it's like I have never lived...
I, I as a model, I only see on inputs. I've never-- I don't have personal experience. You can model my back because you are also a human. But b- and you can model other humans. But, uh, maybe I, maybe this analogy isn't, isn't exactly correct because I was, uh... Obviously, I'm sympathetic to your, your side of the story. But, like, I, I think that is how people criticize this approach to world modeling, which is that, yes, you can approximate closer and closer, closer, but you'll never quite get something core to due to the existence of the thing because you never modeled the thing. You modeled, uh, uh, the most likely, you know, uh, sort of, uh, constraints around the thing. Yeah, I mean, I, I think to some degree you could argue that we operate under the same principle, right?
Like, you can say, like, that we are, you know... Okay, we have this embodiment and we're constantly, like, taking in data and, like, obviously we, we regularly update. But in some principle, like, we're still just taking in these, like, filtered signals from the world. I mean, there's all sorts of wacky things with how our, our eyes process and we have to, you know, part of our brain kind of has to hallucinate some of the info 'cause we have the blind spot in the center and, and, and other aspects of this. But fundamentally, like, I don't think there's that much of a difference between, you know, how we-- like we're, we're out here taking in this natural data and building this model of the world that then we're, we're, we're able to act on versus, like, these models also learning from...
Obviously, it's, it-- the signals are different that they're, you know, okay, they're turned into, to, to pixels or, you know, into, into waveforms and processed through this other mean. But, like, the, the information is still there for the model to take in and, and build this, like, representation of the world that then you can essentially operate with. Yeah. And yeah. I was actually arguing this with Luca- Lukas Bayer. Uh, he's also a, a, um... He's, he's with, uh, Meta now, I think, formerly Google, um, and OpenAI. Created the bits. He's in, in Zurich, which is pretty close to Freiburg, and he's, he's coming over from time to time. We were, we were debating this, um, on, on human learning. Like, he was, you, you, he was-- we were saying, okay, we actually don't need interaction, which would be kind of the, I think s- well, what I would say is important for learning, like, higher forms of, of intelligence, it's like first it's observation.
That's also how we humans learn when we are babies, right? We just lie around first and we, we observe things, and from that alone we learn so much. This is what, what you can actually Exactly map to what, how we are currently training these models. They just observe a lot of 2D videos, audio, and then they learn from it. And I think next step will also be getting, closing the loop to actually interaction with the world, because that's also in how- Mm-hmm... I, I think that that's very important in how, how, how we learn basically around the world. We can then tap and we see, oh, there's a correlation between I knock on wood and there's a sound that has occurred, right?
I, I can just like manipulate the world then and to, to understand it better. That's how I think how humans are mainly learning- Mm-hmm... at least how [laughs] I personally learned. Um, and he was actually saying he thinks the, the majority is observation, which I found very interesting because, uh, that would, that would... If, if that would be true, then we just need to do what we're currently doing more, and that could basically unlock, uh, an insane amount of intelligence basically. Yeah. I wonder how much of this is like emergence with good priors, right? We've seen adding new modalities to, say, language models.
You don't get like the full benefit of, you know, doubling your performance since you've added vision to a language model. But if you have good priors, um, you know, is there light benefit? Is there big benefit? It's, it's interesting to see. I think there's a lot of technical stuff that'll be interesting to walk through, like Gaussian splatting was a big thing. World Labs did a lot of 3D data. Where are the trade-offs between video generation output versus what else? But I think something fun for listeners of the podcast would be the history. You worked on, you know, very early Stable Diffusion, SDXL, latent diffusion even before that. So- Video diffusion as well... video diffusion. Basically, there's a lot of buzzwords here.
Um, you know, LCM, LoRA, flow matching- [laughs]... old prompting. Like- Can you teach us everything? [laughs] Yeah. I mean, what, what, what's interesting? Like how big are these scaling wise? But- I, I seriously... No, so, uh, I've actually built like a tech tree. Like I wanted to- Yeah... build this game where it's like a Civilization, but the, the tech tree is AI. Uh, like key AI papers. Oh, okay. [laughs] So you spend research points and get it. And so like these are the tech trees, right? Like, yeah, first it's GANs then, uh, you know, dot, dot, dot, like latent diffusion happened. But [laughs] you know, like there's a lot of core points beyond that. I'd play that game. Yeah. [laughs] I'd play that game.
[laughs] So, so, so, okay, anyway, just to pitch the game, it, it, it's like you role play as a Neo lab and you have to s- [laughs]... you have to spend points on like- Spend that money [laughs]... uh, GPUs or researchers, and then [laughs] you have to collect data and like there's, there's all these like little things that you can sort of research. Fine tune. Yeah. I... Come on, whip it up. Let's see. [laughs] We're in the mod coding world now. I wanna play it tomorrow. Yeah. Uh, it turn- I think, uh, game balance is really hard because I hadn't-- You guys played... I, I, I have, I built it. Oh, fantastic.
[laughs] Uh, but, uh, there's like game balance is really hard. It's actually like a AGI hard problem I feel like because, you know, once they find a way to win, they just always win like that. Mm. And it's not interesting as a game. Mm-hmm. You need multiple ways to win, and like you need high-risk ways to, you probably lose but maybe you win. Yeah, yeah. Yeah. Yeah. Mod crafts, classic problem in GANs, by the way. [laughs] So I think, I think one thing that if you were interested in, I'm interested in teaching our audience, like, like what are the key points that like if you are, uh, if you've heard a lot of terms, like what are the key things that have really impacted the fields?
And, you know, a lot of you have, you've lived through a lot of this. I'd also say this is a great way for people that wanna get into this space, good interview questions if you're hiring as well, like you should know most of these things. I think there's some points where there's like spicy, you know, different paths that's popped up, so autoregressive image gen versus diffusion, uh- Two backbones or hourglass- Yeah. Yeah... diffusion... so all, all these things, what are the key things people should know? You know, I'll leave it, I'll leave it pretty blank. I mean, first would be SDXL maybe- Yeah, man... if we're talking about SDXL a bit.
“there was a version of SDXL that was never released.”
Well, so you know Runway was also kind of doing their Gen-1 and Gen-2. And, uh, Anastasios on, on this pod also said, admitted that Gen-2 was a pipeline mix. [laughs] Oh, okay. [laughs] Yeah, yeah. So- Like when they- No, like, like if it works, it works. Yeah... they, they just didn't admit it. Whereas yours was public, so you had to admit it. [laughs] Yeah. [laughs] I think if you go back to the time too, like people I don't-- People coming into the space today haven't seen 200, 500 steps of diffusion, right? Like, as you say, like you have a background, and then minutes later you have little fine details coming in. Uh, s- it was also very tinkerable, right? This came out as what, like a 3B, works on eight gigs of RAM.
So as much as you could bitter lesson do the general approach, if you're not a lab and you're just doing this at home, you, you have like one or two GPUs- Yeah... you're gonna do pipeline stuff, right? Yeah. It's like that's how you can see the differences right there. Yeah, yeah. Uh, so video? I've been working on video for quite a, actually already before Stable Diffusion, I was working on, uh, video generation, uh, for NVIDIA. Um, had a paper there, Align Your Latents, where we basically mapped the-- It was actually in parallel to Stable Diffusion. We j- basically mapped this latent diffusion algorithm to video, which was the, the na- natural next thing to, to, to do. Mm-hmm.
It was very exciting to kind of then this, at, at once see things moving. And you can imagine the kind of compression, um, reasoning behind why Stable Diffusion was so efficient compared to other models in the, in the time. Also holds for video, where you actually have this temporal dimension that you can also dis- uh, compress, right? So that made a ton of sense. Uh, worked on this for, first for NVIDIA, and then when we, when we basically joined Stability, we, um, basically after, I think after SDXL, um- Mm-hmm... we worked on this, this, um, Stable Video Diffusion, which basically first applied this, this latent diffusion, um, approach to video asset.
“I think, 1K fine-tuning steps on 3D data, and we got a 3D model out of it.”
There's like three, four stages, pre-training, some post-training. Uh, yeah, stage one, image pre-training, video post-tr- uh, video pre-training- Exactly... video fine-tuning, and then- Exactly... 1,000 steps you can also make it 3D, right? Mm-hmm. Yep. Crazy. Uh, at the time, I, I don't remember what I did, but there was basically this temporal consistency problem where if you just stretch out diffusion and try to stitch frames, it wouldn't work.
So a very old hackathon project where I, I forget the method we even used, but we fixed this temporal consistency to like scale it out to 15, 20 seconds. It took forever, but it was a very hacky solution 'cause you could do this in an afternoon- Yep... right? So fun. But you, yeah, critically you need this kind of learning- Yes... over spatia- uh, temporal kind of- Yes... over the temporal dimension. Mm-hmm. And if you just stitch together kind of a very fast- Yeah, yeah... diffusion model, you don't-- You have this flickering. Even- Yes... even if you've-- You fix the seed and you do a ton of shit on effects- Mm-hmm... but you realize you need to, like in, in currently every-- Or, uh, today everyone is using transformers.
So in the transformer suite you need the attention over the temporal- Mm-hmm... dimension as well to kind of learn from that. Yeah. Right? And then understand what is actually-- what-- how does consistent motion look like, basically. And as far as I understand, we asked this of the, on the Grok Imagine part as well, uh, there's no compression in the video side that is useful, right? You just have frame, frame, frame, frame, frame. Exactly. [laughs] Which is the simplest for- Yep... version. But it's- I mean, well, well if, if you then look at the-... the most intense... MP4 codecs you, you have very smart compressions. Mm-hmm.
But that is lossy obviously, yeah, and, and you need to kind of, you need to connect the frames in a way and like, like extract what, what, what, how, how- Yeah... how kind of consistent motion looks like basically. Yeah. I don't think any video model exposes this, but kind of how far you look back in the, uh, temporal, uh, attention, because sometimes you do need longer, uh, or it needs to be more adaptive maybe. Uh, for example, if, uh, something gets occluded and then it comes out again- Yep... you want them, you want them to be the same. Mm-hmm. But what if-- How long is it occluded? I mean, that, principally that, that depends on the, on the, on the problem you want to be solving, right?
If, if you, if you want to be creating a kind of a, a long memory model that, that, that can, can create you virtual worlds, then you obviously need to find a way to look back very, very, very long. A super memory hungry. And that, that is, that is just- Yeah... uh, I think also one of the most exciting research problems, uh, to work on- Yeah, absolutely... these days. Long context is just, uh, extremely important, right? I don't know if there's more there before we cut over to Black, uh, Black Forest Labs. Uh, anything else in the stable world? Uh, S, I mean, there's SD3. SD3. Yeah, SD3 was kind of- And that was the last paper that you did before you- Yeah, yeah, SD3 was a, a, I felt like a pretty big paper. [laughs] That, that was a- Yeah. [laughs]...
it was a, I think one, a culmination of our work of, and stability, I would say, like- Yeah, absolutely... we- This is your graduation thesis. [laughs] If you want, yes, I, I mean...
“where should we spend our compute? Which parts of this trajectory from data to noise carry the information that's most important?”
There was a time when images could just straight up not do text, and then they did more and more text and then now it's like, you know, "Here's, here's, uh, here's our, our nice typography." And I feel like it was-- Can we attribute that to the diffusion transformer or? I, I don't know if I could say it was the diffusion transformer. Yeah. I mean, I, I think it really was just better modeling- Better data? ... b- it was better modeling and better data. Like it was fundamentally-- Like as, as much as I wanna- The boring answer... as much as I wanna like, as, as much as like, I don't know, you always want to attribute some fancy thing, it really was just that it, it can model the world better. It's a, it's a, it's a data thing. Yeah.
It's also like what, what you're-- You always need like for everything that, that, that you train on, you need a caption, right? Yes. If your caption just don- Yeah... doesn't describe the text that is, uh, visualized somewhere, your model will not learn it, right? Yeah. So, um, that's also something, uh, that, that is often... Sometimes we spend a lot of time- Yeah... talking about the nitty-gritty details of some algorithm. In the end, it's again, uh, the better lesson is- Right... is- It's more easy data... simple scalable is- Yeah... is almost always the most effective, I would say al- al- always the most effective. Yeah. Yeah.
But you can obviously scale in a smart way and still if, if, if you, if you spend tons of compute, you should be spending it as efficiently as possible- Absolutely... and only where it matters. And that is also at the heart of this, this paper basically, right? Yeah. I think prompting also got a lot easier at this time. Mm-hmm. Um, we used to have the 4K- Prompt extenders. Yeah. The 4K upscale, ultra- Our solutions massive with all the lists... 8K. Yes, yes, yes. Greg. Who was the Greg guy? The, the poor artist that just- Greg Rutkowski. Yeah. And now you- Oh. Greg Rutkowski, yes. [laughs] Yeah.
I wonder how he's doing. [laughs] I hope he's doing well. I hope he's doing well. Yeah. Yeah. And also, you know, for context, this is when DALL-E 3, Midjourney, Ideogram V1, nothing was open. Yeah. So, uh, anything that you would see of this quality was either old stable diffusion or- Yeah... pretty much a closed, closed- Mm-hmm... portal. Yeah. Then you had to big-- make the big break to Black Forest.
“we couldn't essentially leverage the value of what the models could be”
[laughs] Uh- We've been talking about how we can generate images and video and improve the perceptual understanding of models, but what about voice? And that's the subject of today's ad. Uh, we've been working with Deepgram for a while for on the AI side, and they recently acquired Pipecat, which has also been a big sponsor of AI. So Deepgram is becoming one of the critical pieces of voice infrastructure for all agents. So this episode is sponsored by Deepgram. If you're building a voice agent, you need it to keep up with how people actually talk.
That means understanding words and context and keeping latency low enough for a conversation to flow. That's what Deepgram's speech models and real-time infrastructure are built for. One in 5,000 organizations uses speech-to-text, text-to-speech, and voice agent APIs in production. The focus is accuracy, low latency, and cost efficiency so you can build and run voice experiences at scale. You can use the cloud APIs or deploy it yourself, including on premises. And there's an offer on Flux text-to-speech, spend a dollar, get a dollar in credits, up to $500. To sign up for free or find out more, go to deepgram.com/contextiseverything. Deepgram, context is everything. Thanks to Deepgram for bringing us this episode, and now we'll go back to the interview.
Well, I, I-- and like what use cases were you seeing, right? Because I think a lot of people went straight to like Hollywood, like we-- you know, that's our goal. Or like, uh, Imad used to say, "We will make Game of Thrones Season 8 again." [laughs] I mean, I, I'm trying to remember this. I kinda-- I don't know if I'll be able to remember the name of this company, but th-this, this might have even been before SDXL, um, but it was a company that for like a week or two, it went viral where anyone could take like a picture of themselves and then it- Yeah... like created a- Uh, the older version of SD, right? The older version, and it used- Yeah...
like the old Dream Booth style fine-tuning, which was extremely inefficient and then, uh... But it w- it like blew up. Like everyone- Yes... was using this. Everyone was excited about it. The models could make so many things, and we saw that there was a lot of potential in how everyone wanted to use it, and you know, we, we saw where the models were going to... You know, at the time they couldn't do typography. We knew obviously, okay, typography is gonna be important and we, we eventually got there with SD3 and, and then so on. So there wasn't the just one use case. It was we saw that this was like a general technology that had a lot of potential and that was going to continue to grow and have potential in the future, which now is expanding into, you know, the, the physical world a lot more.
And- But- But yeah... thi-this was actually already the potential we saw that like- Yeah. Yeah, absolutely... BFL was not founded to be an image generational lab. It was more of a kind of a matter of the, the, the Flux movement being so good that we then capitalized on it as well- Yeah... because it gave us a kickstart. But when we pitched the company to investors for the C series, it was clear we wanna be working on video because we see that as a- Yeah... I think we shared this on a podcast actually back in 2023 already when we were still at Stability. We were saying, "Okay, this like video modeling is actually the central means to understanding how our world basically works."
Because again, what I said earlier, right, essentially we humans are also first observing the world, then starting to interact with it, and by that we, we learn so much. So that was always the thesis, and then obviously we had to kind of take this detour, which was very successful, I would say, to, through image generation. And eventually-- and, and then- Mm-hmm... you always like face the, the, the kind of the issue of, okay, where do I spend my time on? That- Yeah... that's unfortunately a, a thing we, we, we, we cannot get across, that the day only has 24 hours, unfortunately. But eventually I think Flux 3 is really like where we saw this going for a long time and what we- Mm-hmm...
wanted to be doing for a long time, and that's why we're so excited a- about like having the model out there, being able to create videos, being able to create images, being able to create audio, being able to act as a robot brain to actually kind of work in real manufacturing workflows. And there's, there's a lot of stuff coming this year still that we, we have to announce on, on, on, on the physical AI side. Mm-hmm. But really like, as Dustin said, our goal was always to create the most general technology, and we see this happening currently, which is why we're so, yeah, so excited basically. Yeah.
I mean, I think you guys also took the right paths to stay focused, right? Like if I'm not mistaken, Flux 1 launched with xAI. It was their image gen model. [laughs] Good partnership. Yes, that's right. Yep. Uh, before they had their own, but like, you know, that's a good way to solve do we build distribution ourselves. Yeah. Even to the latest Flux 3 Max is post trained with FAL. They do a lot of entrance distillation. Mm-hmm. They're good at that, right? So it's interesting. But I- It seems like you're expanding now towards, uh, you have... What's the latest one with robotics, um- Flux reaction? Flux reaction. Yeah.
Yeah, but before we get there- Okay. Yeah... there's one more I want to cover- Yeah, yeah. That's okay... which is image editing in context. Sure. Yeah, yeah. Uh, we've okay, uh- Also, really nice, uh, history right here on the website. Yeah. [laughs] I, I, I was actually- Like that is- It's, it's weird we're now at the stage of company that, like, things just happen, and I d- I'm like, "Oh, we have this." Um, but I was, I was kind of sad Flux 1 wasn't on there to view, but it's, like, the original one. Right. Yeah. [laughs] Yeah. Yeah, but it's, it's fine. It's, uh- We don't- Pro's, pro, pro looked good.
I, I think, like, even just the visuals here, like maybe someone will clip it of- Yeah... original Stable Diffusion into XL to all the papers- You see the progression, right? You see the progression. Yeah, yeah. But one thing that I do wanna cover at some point, I no longer personally see the progression, right? You show me state-of-the-art image video models. A lot of Chinese labs are putting them out. It's hard to eval. I know people care. I've talked to Summit at BFL about, okay, no, actually, people in the industry, aggressively A/B test industry, there's, there's a lot that people care about on evals, but there's also a level of, it all looks very realistic to me.
[laughs] Um- The rest is prompting skill issue. I think there's, uh, yeah. [laughs] I think there's a lot of little nitpicks. Like there were hands, there are still issues in text sometimes. So- There's still hands issues? No, there were hands issues. Yeah. There, there's issues in perception and layout, like I'm redesigning my living room. It's horrible. Controllability is a huge thing. Controllability. I think controllability is the, the thing we need to solve, like large scale. There's so many ways you can control these models, and maybe that, that is a good seg into, into, uh- Context... Flux context as well- Yeah...
because that, that was a pain point that we saw for a lot of time. People were s- training so many LoRAs for- Yes... just to get this character consistency into models, right? A shout out- Yeah... to replicate back then with, you know- Yeah, exactly... all the LoRAs in front. [laughs] Oh, yeah. And, and, and then we had this, uh, amazing pitch of Dustin. It was, uh, also in Northside- [laughs]... also in France, I think. It was, it was another cabin, right [laughs], like a lot of things happen in cabins. Also, I think [laughs] we should go to the cabin. We really like, um- You should do more. Yeah, yeah. Yeah.
And, and then we had this pitch of Dustin, who actually kind of saw this potential and, okay, I need to, I need to get this, like, context. I, I, I need to be able to prompt not only with text, but also with images for image editing and that. Um, yeah. Mm. Yeah, yeah. No, I mean, I, so I did this, like, I, I originally- That, that made pre-hype as well. Crazy. I've heard the story that Flux context was, like, you and one or two other people. Yeah. So it was- [laughs]... for, for a long time, it was basically my side project, and then I stole one of our, one of our data guys a little bit into it and said- [laughs]... "I'm, I'm-- Come, we're, we're, we're building this thing."
“it was maybe like six hours of training, four hours of training. It was, it was a very, very short training run.”
So my, my crappy Comfy UI was that. It just- [laughs]... slopped out like 10 different versions and you could pick. There's, there's, yeah, yeah. And I guess I can- My, my favorite part, I still slop out. Like, I'm- [laughs]... changing, living in my generate four, generate six, redoing something. Just make 10 iterations and [laughs] And then, yeah, and eventually that one's good enough. My favorite is that you pray to RN Jesus. [laughs] Because it's a random number. Uh, basically, you know what, to me- This concept of what is a foundation model is something like this where it's like, okay, you know all these little workflows that you guys do to like do that and do that and do that and they- they all use different tools?
We're gonna replace that with one model. Yeah. [laughs] And that- that- that's- To this, that was-... actually what we are building towards, right? Yeah. And that- that was where FluxV came from and- This was a very clear aha moment for me. Yeah, yeah. Mm-hmm. I don't know the... I- I never tried Prompt2Prompt, but I think, uh, to me the closest it has come to a FLUX Kontext is Qwen Image, which, like, the Qwen folks are doing really good, like- Mm-hmm... they had a Photoshop layer model. Yeah. It was incredible. Yeah, no, no, I mean, they're doing- [laughs]... doing pretty cool stuff over there. Yeah. It's interesting how much of this just comes back to there's a lot of good data.
[laughs] Yeah, I mean, it was, it was targeted good data for... Was it a good concept? I- I don't know what- how to even describe this here. Yeah. But yeah. I- I think one thing that I'm surprised by is, so, you know, comment on stability one more time. Stable Japan, right, Japan had this, like, whole law of, like- Yes... we will protect you if you trade on copyright- I'll open anything up... in- as a country. Yeah. And everyone was like, "Ooh, Stable Japan," like, it's interesting. Nothing happened. [laughs] Yeah. I always wanted to visit that office. I was sad I never did, actually. [laughs] I think, uh, on your Omni point as well, it was an interesting time. There were the two others. Gemini is obviously the latest Omni.
Uh, Image Bind, which I didn't realize is 2023 from Meta, they threw in one embeddings to- Heat maps... heat maps- Heat maps. Depth... and depth. That's a big one. [laughs] Which, you know, I'm like, oh, very early, but, like, it's cool. I- Yeah... I think this work has progressed. There's definitely newer, newer- Yeah, so like, but you can, you can-... work on it, but like IMU... you know, this is just another color, right? It's RGBA what- whatever that is. Yeah, yeah, yeah. Yeah. Right? So like how many colors are there? Oh, the fun thing, this was also- [laughs]... all, all in, all out, so you can go text to heat map if you want. You can go IMU to depth. It's a big, big market for that. [laughs] But- [laughs]... interestingly, no, no video out, which is I think the- Yeah...
the kind of the- Yeah, yeah... the central thing that- that- Yeah... we need to l- to, to, to understand basically the world better, right? This, this is, I think what, what's... Uh, as said, what has always been at the core of what we were, we're doing. Yeah. Okay. Yeah. Well, before we leave the, the sort of data discu- or IP discussion- Yes... is there a particular kind of data that is, uh, everyone in an industry knows is, like, valuable? Uh, if people... If- if I have a data vendor listening, how do they sell to you? What, what... How do they approach you? It, it depends heavily on what you, what type of problem you want to be solving. That's just super important. If I, if I train a, uh, a robot, I obviously need a lot of eccentric data. I think that's not a secret, right?
Yeah. Because that, that is just how this thing will be, will be basically. E- egocentric or? Yeah. I mean, there's also a benefit for you guys, right? On the language model side, there's a lot of people doing RL environments. Is there a parallel that people should come to you with that kind of stuff? Yeah. I'm, I'm trying to get you vendors, yeah. [laughs] It's like get you- Oh, we have so many data vendors actually that are, that are approach- Too many? Yeah, yeah. Okay. It's, it's a lot. Oh, okay. What, what, what is good versus bad? It, it again depends on which stage of the training you are- Yeah... um, and what your effective goal is.
I think there, there's a, there's a trade-off between we had this already, right, scale of the data versus, um, versus the quality. Yeah. And that is what you have to, for every training run, like large scale pre-training and you do again, you have to kind of find, find that sweet spot out because depending on the architecture you, you, you might be using, uh, that might look very different again, and that is one of the central pre-training experiments like, or pre-scaling experiments that you need to do. You need to do a ladder basically, um, to understand at scale how much data-- or to, to predict at scale how much data do I actually need and h- how should the quality look like.
So where is the sweet spot between like these two dimensions of data quantity versus data quality? Mm-hmm. Any other major jumps that we want to call out? So Flux.1 Context was a big one. Flux.2, Flux Tools, Action, we talked a bit about. Any other- Yeah... things that- I mean, I think the, the three sizes, that's very important- Mm-hmm. Yeah... in terms of like, you know, the, the dev models and the Schnell. Schnell? Schnell? Yeah. What is Schnell? Schnell means fast in German. Ah. Ah. Yeah, I think you probably covered that in a blog post. But yeah, I, I think that, that like is you have open source, you have the closed source, and you, you have the range of performance. I think that makes sense. Yeah, absolutely. Yeah.
“I like to see how people actually use our models.”
Yes, this is like a mid-journey, whatever subscription, but like, it actually becomes very real work. Yeah, exactly. Like, because- Absolutely, like- Because they've done it for fun, then they're like, "Oh, I can do this at work." And if effectively also, like, when you, when you automatically eval these, these models, like, you, there's a bunch of obviously human evals that everyone is doing in the field, but every evaluation will have their blind spots.
Yeah. And you, you should be combining as many independently as possible to get- Yeah... a picture that is as good as possible of, like, how the model is performing. But that doesn't, like, re- remove the burden from you to kind of... Or it's not a burden. It's actually super fun to then sample the model yourself and, like, try- Yeah... understand where are the strengths, where are the weaknesses. Like, this is, this is a core part of developing a model is actually constantly sampling when developing it and checking, a, is it, is it where I would expect it to be, right? At some level, it's like what you guys care about too, right? Like- Mm-hmm... these are your use cases. It's a good way for if you want something to get better, like, make a little benchmark and see how things perform.
No, a- absolutely. I, I think it's very important that the people building the models actually, like- Use them... like, want to use the model. Yeah. Like, I, I think this is very important. There's a big distinction in this and, like, the microphone brand trying to, you know, use it for product photos. Mm-hmm. A very niche use case that they can min-max on compared to remove- It's very general. Yeah. Yeah. I, I, I think one thing I do doubt or question is the, it was a very well-known Midjourney's loop, which is the, it's Discord, everyone sees, like, other people prompting. You get the four options. You choose the four.
That is feedback for your next model. How come you didn't do any of that? We didn't do it publicly at scale. Yeah. So, so every one of- But isn't it valuable to do? It can be valuable, but I think only up to a certain extent that if you then, like, cross certain limits, the signal that you get is pretty, pretty- Yeah... Like early days, we saw this insane potential, right, of this, this, this, this tuning, uh, say of, of Midjourney's models, like V- V5 or something crazy. But if your base model just reaches a certain, certain kind of level of strength, it's actually, you, you don't get these super good examples anymore where you can clearly see the model improving from this, and this is also a lot of noise.
Yeah. So there are, there are a lot of other ways w- how you can actually do this. I, I had this debate as well with someone where back then it was very useful because prompt adherence wasn't great, right? Uh, it's pretty judgmental on which one follows- Yeah... the prompt better when neither [chuckles] of them do it great, right? Yeah. So you have a lot of preference tuning towards- Yeah... there's very high signal and okay, the people prefer this. Now back to my point a lot earlier, like, if I take the top five models and tell them to remove a bed, it's all gonna look the same to me. The preference, there's not that much signal, right? They're all filling it in pretty good- Yeah... because there's, like, they understand this task very well. So that side of the eval for me is kind of gone.
It all, it all looks fine. Yeah. But yeah. I was gonna jump and say that was actually one of my favorite parts of, of moving into context, is finally it wasn't just like- [laughs]... I don't know. It was finally just like, okay, did, did it actually just do the task? Yeah. And, and, and I really enjoyed getting to these, like, very kind of clear, concrete tasks.
“I would say autoregressive models are way more data efficient.”
LASERs versus, uh- Yes... VAPS basically, which is super, super interesting as well. But yeah. The, the last other thing I guess is there's also the diffusion for text, right? So there's also some of that. So it's, there's benefits. There's of course speed, right? You can- We see that already, like- Yeah... with text, we see it clearly, right? How like- Well, we got Diffusion Gemma that people- Yeah...
can play around with too. Will, will they ever reach parity? Is it just a matter of investment in money and data or like, you, you know? I would say a lot of it is data. If, if it doesn't work, why doesn't it work? My simple answer is that we, we reason left to right. We don't reason, you know, and diffuse into, into vision. At the same time, you like, I don't know, we-we, with autoregressive models, I think a lot of people like to focus on the, the, the, you know, step-by-step of the, of the tokens, but at the same time, it also can parallelize. Like we, we can't just take in a whole book at the same time as well, which those models can. Yes. So there, yeah, yeah. Um- And you can fake it in. Yeah. I totally get it. Yeah.
So that's, I mean, that is the valid question. Like how come- Yeah... diffusion for text never took off and, you know, now it's like slowly, slightly better, but like how, how- I think it's just like a matter of which approach was, was earlier and has been like becoming for language model the standard. I mean, I think as your point, you said if it's more data efficient and the world has been doing a lot of scaling, there's a side there. That doesn't mean that there aren't pockets where there's use cases that you can get really good things at a certain size. Yeah. The example is like, yes, Diffusion Gemma is pretty good and it...
meets a bar and can run really fast, right? If you want that, that's great. Yeah, I mean, on the autoregressive language front, it- it's-- I think it really comes down to it was simple, it was scalable- Yeah... it was there early. It was, like, a very... I mean, it's clearly been a, a very successful trajectory. A lot of time and, and money and research has gone into to make it not work. There's right now a lot of very exciting research on the diffusion front, you know, and I, and I'd like to separate the notion there's, uh, 'cause there's-- Diffusion language is really two camps, and I think a lot of people tend to not realize this because there are discrete diffusion language models which I almost don't-- I don't like to call them diffusion models really, and this is where, like, the Diffusion Gemma and, like, the Mercury code- Mm...
like Mercury models live, which is, is more of this, like, masking style model where, where it, it jumps from a mask to a token, but it's still in the discrete space versus now there's a, a big wave of research right now going on continuous diffusion language models which those are actually operating as, like, you know, I don't wanna say flow matching models, but they're, they're, they're operating in this, like, essentially denoising space. So, so more like in- This was used for audio, if I'm not mistaken. The- Uh, audio generation... continuous diffusion. Continuous diffusion. I mean, well, it's definitely- Diffusion has also been used for- It's for everything, yeah... visual modeling, audio modeling.
My last fun question there is, um, do we have a good way to do, like, GRPO style RL on pure diffusion text models? Yeah, we, we're doing this actually. Okay. So for Flux.3, we did this and, and it has had tr- tremendous, uh, gains that it follows, yeah. So Flux Action, this is coming to very, very recent launches and, and work that you're doing. Mm. How do you approach this? What is your-- what is the feedback and impact so far? Yeah, it's actually like how, how we approach it. Basically, you, you pre-train on, again, videos, images, audio, everything together, and then you have this very general model that understands the world.
And then actually you can, can take a bit of action prediction data. Basically, you add kind of predictions in the state space of this robot, which is basically a kind of a parameterization of here this robot arm, um, which every joint will be described by a, a translation and a rotation, six degrees of freedom, basically. And then you predict that together with a video. And since the model can already reason about the video and what it needs to create or to, to fulfill a certain task in video space, it's actually very simple to then add this new dimensionality or this, this, this new dimension, this state space, how we call it, and then basically predict that as well.
Given a chunk of earlier video, condition it on that, enter an instruction. Basically, the model creates the next chunk of actions to basically fulfill- Mm... um, this action. And this, this works extremely good. It's-- We, we can do it in the real world, like for real world robots. We can do it in a simulation. Like computer use is also kind of a mapping of that to the digital world, and we can, like we- Ooh... can make the model play games right and stuff like that. Mm-hmm. So this is all enabled by this, this single Flux.3 model. And then you see also that it's extremely capable because we have this very general pre-training, which is not only limited to, say, classic robot data, but actually all kinds of data that gives the model just this inherent world understanding that you also see then in performance.
Yeah. And being a diffusion model is also pretty speedy, as we see here, um, with, uh, with, uh, with these distilled, uh, versions basically, right? Would you call this a world model or world action? Do you have a term you like to use? [laughs] It's a world-- This, this one is a world action model. I think the, the word- World action... the term world model I don't like to use. It's so overloaded. I'm in the same, I'm in the same boat. It feels like a hominy to me at this point. It's just- Let's, let's, let's call this a visually intelligent model because it can do all these, these basic things, right?
I guess the, the distinction that I see in these models is you're not just predicting video or world, you're also prediction action states, right? Mm-hmm. Yeah. So one is what is happening in the world, what is the state, but also what is the next action to predict. So it's like- Exactly... sort of two outputs, right? Mm-hmm. So. Yeah. Is there a target frame rate, uh, that you target that is industry standard? So the- Because you don't have to predict out that far is, is, is my, is my idea. You mean in terms of the real-time factor? Like how, how- Yeah... it's like ten FPS, I think, um- Yeah, FPS... what the model currently runs. Very s-small model.
And, and you're projecting out like one FPS or five F- Uh- Sorry, one frame, five frames, whatever, right? I would, I would have to look it up. I think it's- Yeah... it's a small chunk of, of frames. I would say like in between nine and 21 frames that it then predicts again. Yeah. And then the model is conditioned on the previous, uh, generations again. And what the camera-- When it runs on the robot, what the camera actually perceives, right? And then you basically predict this, uh, further. So it's, it's, it's a bit of a chunk. It's not, uh, running autoregressively, so to say, in, in, in time, basically, but it's a chunk prediction.
What I'm driving at is like this video prediction side that is very much for predicting the results of what you do. Yeah. But so probably you need some kind of long range planner existing outside of that model that will coordinate the actions that- I mean, this, this is why we're discussing the language side earlier. Yeah. Yeah. Yeah. So as it runs right now, what we see is that there's no pl- like, like high level planner involved or something. Yeah. This is just a model running on a, on a, um- On a loop. Yeah. Some of the interesting things here- This is what we did with, uh, Dyna Robotics. Yeah. Um, Dyna has one, uh, Runway has one. I guess talk about the fine-tuning side.
“You should be showing a robot an action one time, and then it should be able to do this”
You mentioned Germany a lot- But it's coming... but you also have a pretty big SF office. I actually didn't know there was a big European side. There's a substantial SF presence. Yeah, it's called Black Forest Labs.
Yeah. [laughs] And, and so like- And this is all like, like inviting everyone to come over to our place, uh- Yeah... 410 Townsend Street. [laughs] Uh, super, super nice. Yeah. Come, come by. And, and we, we, we, we're not a European company, not, like- Yeah... we're a company with roots both in Europe and the US- Absolutely... and we are acting globally, basically. That's fun. We got you on because you're visiting in town, and we're like, "Okay, we should- Yes... have you on." Yeah. How can people help you? Yeah. You hiring? Yeah. Looking for... I think just, just, uh, supporting us in marketing- Mm... talk about our models. Um, helping us also here, I think in the US, getting, getting our name out, out. Here in the community, we are well known. Yeah.
I think for, for like in the broader society, not that much. There's the big players, right? That's also what we need to change in 2027- Yeah... really kind of getting our name out, because we have extremely strong technology. I think that, that, that's what we saw. We have a long history of having built like the core pillars of like what everyone is using, basically. It's now also time to get our name out, um, to the kind of broader audience. Yeah. But the, the tech community, I think we are well known, but- Yeah... apart from that- Tell your mom about Flux. So it's, it's very interesting that the overall gen- Ob- obviously, I'm a marketing guy, and I help companies in marketing all the time.
You guys are very different, 'cause it, generative media in general is, has a weird, mixed reputation in the broader population, right? Hmm. Like, normally I would suggest, "Oh, have a Ti- TikTok or Instagram marketing campaign," but pretty much you'll be killed in, i- i- just because they hate AI so much. Oh. Right? And so i- it's very interesting to find the niches where this is, isn't regarded as AI. It's just normal, a, a thing you can do with tech. Mm-hmm. But it's not AI. Yeah. 'Cause I think the moment you say it's AI, the c- population will just- Yeah... have pitchforks. Yeah, yeah. I don't know if you have any response, or thoughts, or explanations about that.
I think it's important to, to just always also emphasize that we, we, we take this responsibly. We're like, like n- not... Our models are not creating s- like avalanches of slop or something, right? They, they, they actually live in real use cases, uh, lots of like professional workflows, and they just help making the lives of, like, professional people who work with this- Mm... way easier, and just increasing productivity so much. And, uh- It's Photoshop. Yeah, e- exactly. It's, you know, Photoshop. Yeah. It's, it's so crazy, like how, how much more productive you are then. And that, that's really the, the kind of the, the goal of this on the content side. It's not about putting people out of work, not at all. It's just like making everyone's life so much easier. Oh.
Yeah, I wouldn't say that. I, I'm just like i- if I worked on your team, like this is the, the my GTM problem right now, which is like, how do, how do we get people to basically, you know, go viral in the same way that maybe WhisperFlow goes a little bit viral 'cause they're like, "Oh, yeah, I understand how to use this. This is not AI. It's just WhisperFlow." Yeah. Or it's like, it's Gondola, same thing. I think they're, the, they're, the generative media hasn't had that crossover except for NanoBanana, and then- Mm... whenever ChatGPT had the- Image gen... Studio Ghibli moment. Yeah. Mm, yeah. Those kinds of like, that is the crossover where it's like, "Oh, like this is a fun trend that I wanna hop on." Yeah.
There were moments that hit well, like your point of the see yourself as you're older, like that had a huge moment. Yeah. And people would look at themselves as their 80s, send it to their family. Like there have been moments, but nothing is super... I, I think one of the difficult pieces is it's actually very hard to predict one of these like viral moments. Like, I, like NanoBanana was their, like their internal name, and then they just kept it because it, it, like people just liked the name and it stuck. Mm-hmm. For context, actually, the one big use case that we saw a ton of usage on that like, you know, we didn't necessarily expect was image restoration as well. Mm. Yeah.
So there was a ton of like, I saw amazing, amazing examples of people, you know, taking old- Bringing back... degraded family photos and then restoring- Colorizing them... colorizing them, fixing them up, but, but it, and it was able to keep like a, a very, very high level of, uh, of similarity to the, to what the original would've been. Yeah. And then, I don't know, that, that was actually one of my favorite moments that we've had, uh, you know, of people showing like their family these kind of restored photos and the reactions from it. And I, I really, really enjoyed that like positive moment that, that it brought. But like I said, like we, so often we can't predict like what the thing is going to be. You know?
It's, it's, it's even with, uh, even with our old, you know, even back in the stable diffusion days, it was always surprising what the trend was, you know, online. Mm. Like, it, it just kind of, they just kind of- Yeah, you just never- They, they emerge. They emerge out of- Yeah. You cannot plan them. I think o- once you have them, you need to, to kind of exploit them, obviously. Yeah, yeah. And you should be always asking, "What is the next big thing- Absolutely... the next capability that it has not been able, or like, like that models have not been able to do? What would that be?" And then trying to f- to bake this into your model. Absolutely, yeah. Making bets. So- some of them will work, some others will not work.
But this is just extremely important to, to kind of always have this mindset of, "Hey, what, what is the next thing we should be working on that will just bring a lot of joy or like productivity increases to people," right? Yeah. Well, that is the domain of research. You guys are doing amazing job, so thank you so much for sharing your story. Uh, I hope more people come to know you as a team because they've, they've seen a lot, a lot of the model buzz. But, uh, I, I feel like this is also the first time I've seen, I've heard the full story of PFO. Yeah. Yeah, no, thank you. Thank you so much for having us. Thank you so much. Yeah. It's amazing presentation. I mean, this, this was a... It, it's fun to finally have gotten to do this. Yeah. Yeah.
I mean, it's a year in the making. But also, you know, I bought a bunch of, uh, MicroDucks, and, uh, you know, I'm looking forward to, to them running around. [laughs] Oh. Do, do, do you have... Like I, I wanna get my hands on one. Okay. I feel like I missed the window of these being available. Eh. You have connections. I'm, I'm sure I can, I'm sure I can- [laughs] Whoever out there can get me a MicroDuck, get me a MicroDuck. [laughs] I, I, I- I'll get you a MicroDuck. Okay, perfect. [laughs] I bought a bunch 'cause I know- Fantastic... I know I'm just, this is gonna be a hot gift for Christmas. Yeah, yeah, yeah, yeah. So. It's, you know, like, uh- But, but yeah, you know, robotics, 2027, that's, you know, one year away.
Like, you know, we'll, we'll see, we'll see where, uh, this all learns. But I'm very optimistic about this approach, and I think you guys have explained it very well. Fantastic. Yeah. Amazing. Awesome. Thank you. Cool. Thanks so much. All right. Thank you. [upbeat instrumental music]


