# Prompt iteration matters most when optimizing the final few percent

Markdown copy of https://realalpha.app/s/prompt-iteration-matters-most-when-optimizing-the-final-few/ on Real Alpha, the best minutes from long interviews with the people building AI. Cite and link the page, not this copy.

- Speaker: [Amanda Askell](https://realalpha.app/builders/amanda-askell/) · Head of personality alignment, Anthropic · Leads Anthropic's personality alignment team, shaping Claude's character; co-authored the GPT-3 paper at OpenAI.
- Host: [Lex Fridman](https://realalpha.app/people/lex-fridman/) · Host, Lex Fridman Podcast · Robotics and AI researcher at MIT; started the show in 2018.
- Show: Lex Fridman Podcast
- Episode: [Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity | Lex Fridman Podcast #452](https://realalpha.app/v/ugvHCXCOmm4/), Nov 11, 2024
- Moment: 3:05:54 to 3:11:35 of the episode (6 min), playing on https://realalpha.app/s/prompt-iteration-matters-most-when-optimizing-the-final-few/; the episode from there: https://realalpha.app/v/ugvHCXCOmm4/?t=11154

## Quote

> “I think that prompting actually only really becomes relevant when you're really trying to eke out the **top like two percent** of model performance”
> — [Amanda Askell](https://realalpha.app/builders/amanda-askell/) at [3:10:08](https://realalpha.app/v/ugvHCXCOmm4/?t=11408) · Lex Fridman Podcast

## Takeaways

1. Define ambiguous qualities such as rude or polite.
2. Probe failure cases and add examples to the prompt.
3. Iterate deeply when a prompt will be used at scale.

## About this segment

Amanda Askell treats prompting as iterative specification: define the task, test edge cases and add examples. She says extensive iteration mainly pays off when you need the last few points of performance.

From “Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity | Lex Fridman Podcast #452”, Nov 11, 2024. [Watch the full episode](https://realalpha.app/v/ugvHCXCOmm4/)

## Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

[3:05:59](https://realalpha.app/v/ugvHCXCOmm4/?t=11159) I really do think that philosophy has been weirdly helpful for me here, more than in many other respects. So in philosophy, what you're trying to do is convey these very hard concepts. like one of the things you are taught is like and i think it is because it is i think it is an anti-bullshit device in philosophy philosophy is an area where you could have people bullshitting and you don't want that um and so it's like this like desire for like extreme clarity so it's like anyone could just pick up your paper read it and know exactly what you're talking about is why it can almost be kind of dry like all of the terms are defined every objection's kind of gone through methodically and it makes sense to me because I'm like when you're in such an a priori domain like you just clarity is sort of a this way that you can you know prevent people from just kind of

[3:06:55](https://realalpha.app/v/ugvHCXCOmm4/?t=11215) making stuff up and I think that's sort of what you have to do with language models like very often I actually find myself doing sort of mini versions of philosophy you know so I'm like suppose that you give me a task I have a task for the model and I want it to like pick out a certain kind of question or identify whether an answer has a certain property like I'll actually sit and be like let's just give this a name this property so like you know suppose I'm trying to tell it like oh I want you to identify whether this response was rude or polite I'm like that's a whole philosophical question in and of itself so I have to do as much like philosophy as I can in the moment to be like here's what I mean by rudeness and here's what I mean by politeness and then there's a like there's another element that's a bit more um i guess i don't know if this

[3:07:41](https://realalpha.app/v/ugvHCXCOmm4/?t=11261) is scientific or empirical i think it's empirical so like i take that description and then what i want to do is is again probe the model like many times like this is very prompting is very iterative like i think a lot of people where they're if a prompt is important they'll iterate on it hundreds or thousands of times um and so you give it the instructions and then i'm like what are the edge cases so if i looked at this so i try and like almost like you know uh see myself from the position of the model and be like what is the exact case that i would misunderstand or where i would just be like i don't know what to do in this case and then i give that case to the model and i see how it responds and if i think i got it wrong i add more instructions or i even add that in as an example so these vary like taking the examples that are right at the edge of what you

[3:08:25](https://realalpha.app/v/ugvHCXCOmm4/?t=11305) want and don't want and putting those into your prompt as like an additional kind of way of describing the thing um and so yeah in many ways it just feels like this mix of like it's really just trying to do clear exposition um and I think I do that because that's how I get clear on things myself so in many ways like clear prompting for me is often just me understanding what I want um it's like half the task so I guess that's quite challenging there's like a laziness that overtakes me if I'm talking to Claude where I hope Claude just figures it out.

[3:08:55](https://realalpha.app/v/ugvHCXCOmm4/?t=11335) So, for example, I asked Claude for today to ask some interesting questions. Okay. And the questions that came up, and I think I listed a few sort of interesting, counterintuitive, and or funny, or something like this. Yeah. All right. And it gave me some pretty good, like, it was okay. But I think what I'm hearing you say is like, all right, well, I have to be more rigorous here. I should probably give examples of what I mean by interesting and what I mean by funny or counterintuitive and iteratively build that prompt to better, to get it like what feels like is the right, because it's really, it's a creative act.

[3:09:38](https://realalpha.app/v/ugvHCXCOmm4/?t=11378) I'm not asking for factual information. I'm asking to together write with Claude. So I almost have to program using natural language. Yeah, I think that prompting does feel a lot like the kind of the programming using natural language and experimentation or something. It's an odd blend of the two. I do think that for most tasks, so if I just want Claude to do a thing, I think that I am probably more used to knowing how to ask it to avoid like common pitfalls or issues that it has. I think these are decreasing a lot over time.

[3:10:09](https://realalpha.app/v/ugvHCXCOmm4/?t=11409) um but it's also very fine to just ask it for the thing that you want um I think that prompting actually only really becomes relevant when you're really trying to eke out the top like two percent of model performance so for like a lot of tasks I might just you know if it gives me an initial list back and there's something I don't like about it like it's kind of generic like for that kind of task I'd probably just take a bunch of questions that I've had in the past that I've thought worked really well and I would just give it to the model and then be like now here's this person I'm talking with give me questions of at least that quality um or I might just ask it for some questions and then if I was like oh these are kind of trite or like you know I would just give it that feedback and then hopefully it produces a better list um I think that kind of iterative prompting

[3:10:52](https://realalpha.app/v/ugvHCXCOmm4/?t=11452) at that point your prompt is like a tool that you're going to get so much value out of that you're willing to put in the work like if I was a company making prompts for models I'm just like if you're willing to spend a lot of like time and resources on the engineering behind like what you're building then the prompt is not something that you should be spending like an hour on it's like that's a big part of your system make sure it's working really well and so it's only things like that like if i'm using a prompt to like classify things or to create data that's when you're like it's actually worth just spending like a lot of time like really thinking it through what other advice would you give to people that are talking to claude sort of generally more general because right now we're talking about maybe the edge cases like eking out the two percent