# Design editing needs benchmarks beyond single-shot generation

Markdown copy of https://realalpha.app/s/design-editing-needs-benchmarks-beyond-single-shot/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Stefano Corazza](https://realalpha.app/people/stefano-corazza/) · Stefano Corazza is head of AI research at Canva.
- Host: [Sam Charrington](https://realalpha.app/people/sam-charrington/) · Host, The TWIML AI Podcast
- Show: The TWIML AI Podcast
- Episode: [AI Design Still Feels Like a Slot Machine](https://realalpha.app/v/heE80uxOmus/), Nov 10, 2026
- Moment: 22:51 to 26:32 of the episode (4 min), playing on https://realalpha.app/s/design-editing-needs-benchmarks-beyond-single-shot/; the episode from there: https://realalpha.app/v/heE80uxOmus/?t=1371

## Quote

> “And you have benchmarks for image editing, which is great, but **design editing is different** than just image editing.”
> — [Stefano Corazza](https://realalpha.app/people/stefano-corazza/) at [23:33](https://realalpha.app/v/heE80uxOmus/?t=1413) · The TWIML AI Podcast

## Takeaways

1. Current benchmarks favor first-generation images over iterative design edits.
2. Canva has not found an industry standard for editing existing designs.
3. Structured rubrics help VLMs score readability and composition.

## About this segment

Design benchmarks still emphasize first-pass images, while real design work includes editing existing compositions. Stefano Corazza uses structured VLM rubrics alongside human evaluation to score readability and layout.

From “AI Design Still Feels Like a Slot Machine”, Nov 10, 2026. [Watch the full episode](https://realalpha.app/v/heE80uxOmus/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[22:55](https://realalpha.app/v/heE80uxOmus/?t=1375) about benchmarks for image generation. You know, there are well-established benchmarks, but I always wonder, you know, how well they capture kind of the human perspective and human judgment and taste, as you just referenced. And I'm wondering, you know, someone who works on this constantly, like how do you think about, you know, the applicability of the popular benchmarks, how you supplement them, that kind of thing?

[23:19](https://realalpha.app/v/heE80uxOmus/?t=1399) That's a great question. I would say the benchmarks overall are lagging a little bit behind to where the power of AI in design is going to be because they're mostly focused on single shot, the first generation. And you have benchmarks for image editing, which is great, but design editing is different than just image editing. And so as of today, There is no established benchmark for how good a model is editing a design, for example.

[23:53](https://realalpha.app/v/heE80uxOmus/?t=1433) There's some public benchmark for generating, let's say, a presentation, like there's a present bench. In one, you know, has like 200 examples of like presentation-related slides, related tasks. And you can test it, but again, it's new generation. And so there's not a lot of benchmarks that really can assess the effectiveness, editing and design, the speed and the cost also to go with that. So I hope there's going to be more.

[24:23](https://realalpha.app/v/heE80uxOmus/?t=1463) Of course, we have internally, we have a lot of those, But yeah, we haven't found an industry-established benchmark for the ability to actually edit a design because the industry has focused on, oh, AI is so magic, it's going to single-shot everything, which we know is not the case. And so I think now the understanding is finally landing and I hope we're going to see more benchmarks that really start from like a given set of pre-existing designs and defines rubrics for how those designs can be modified and then does the comparison, the benchmarking on that.

[25:00](https://realalpha.app/v/heE80uxOmus/?t=1500) That kind of implies that you can use an LLM or a VLM as a judge to look at an image and assess it visually, you know, from any number of perspectives. Are we there yet? Or, you know, how good are VLMs at kind of articulating the qualities that you might want to optimize an image for? I think they're pretty amazing. And we have also to keep in mind that if you structure the rubrics really well, they are really good at defining, I don't know, is this design readable?

[25:39](https://realalpha.app/v/heE80uxOmus/?t=1539) Like, what's the score of readability? What's the score of layout and composition? So we have very well articulated rubrics about design quality. And then we can use VLMs to judge that. And we get quite far along. Of course, we always do also human evaluation and some of the arenas are strongly, you know, leveraging that because it's the ultimate test. But I think, yeah, we came a long way. It forced us also to really define what a good design means, right?

[26:11](https://realalpha.app/v/heE80uxOmus/?t=1571) And so if humans are not able to articulate that, then VLMs are not going to be able to score it, right? And so I think we had to really go back and really understand what makes the design really good to be able to leverage VLMs to do it. But I think now we are getting to a place where it's quite reliable.