AI-generated code still needs expertise to verify

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:11:17

   go buy it yeah the um you know with with this this ai and you know all these lms one of the big things that we've seen is that they can write all the code actually like a lot of my peers genuinely could say a hundred percent of the code in the last so many months was written by an LLM. And so a lot of people who really love computing or love writing code, it's actually kind of sad for them to see, you know, these things become obsolete, like typing things out and stuff. And I was curious in your career in computing, there must have been points.

2. 1:12:00

   where something became obsolete, maybe like the punch cards or I don't know, something like that. And I guess, how did you feel at that time? Yeah, well, I don't miss punch cards in the slightest. No way. I was thinking, you know, my thesis was done on, my PhD thesis was done on punch cards, and that was three boxes of cards. And I realized that that was 30 pounds of stuff I had to carry around for my thesis.

3. 1:12:25

   So there are certain things like that I don't miss. There are other things that you miss the kind of, call it the intellectual challenge or the puzzle solving or the whatever, where you write a piece of code. So right at the moment, I'm teaching a class that involves C programming. So it's back to early days. And, you know, I haven't written much C for a longish time. And it's kind of fun to get back into it and sort of rethink and relearn, you know, that's what pointers are all about, kind of saying.

4. 1:12:56

   so I suspect most people don't miss that sort of thing because the memory management aspects are a pain on the other hand the challenge and the fun of figuring it out and getting it to work and so on is there now the same fun and challenge and getting it to work is going to be true if you're writing in Python is it there if you prompt your favorite plot or whatever to generate it for you? I don't think so. I don't think that at all. And even once the program is generated for you, you still have to see whether it works and whether it's the most effective way to do what you want.

5. 1:13:33

   And is it really doing what you want? Or should you have wanted something different? And all of those sorts of things. And that's in an environment where you know what you're doing. I've been using mostly Claude, but others as well, to generate SQL, because I'm not an SQL expert. And I have a colleague over at the Institute for Advanced Studies who has a database. She's not a programmer at all, but she wants to query her database effectively. So I need something to generate SQL from natural language queries. So I want to say, give me all of the whatevers that have this property and get the large language model to generate the SQL and then run it through the database. How do I know whether it gave me the right query? Because I'm not an SQL expert. And if we already knew the answer, we wouldn't need it. And so I'm wrestling with that. And my

6. 1:14:29

   observation is 90% of the time it generates a perfectly fine query and 10% of the time it doesn't, gives you a totally wrong answer, and I don't know which is which. It sounds like what you're saying is you need some domain expertise, otherwise you can't be the judge of if it's... You need domain expertise, probably it's the same domain expertise you always needed. You need ways of validating what you do because you can't be sufficiently expert that you can test a big program all by yourself.

7. 1:14:58

   So you need something, AI again, to help you with generating the test, but then you need something to evaluate whether the tests were properly generated and then there's this infinite regress of you know check the checker who checked the checker kind of thing um now i'm oversimplifying and i'm not a high-powered user and i know plenty of people who as you described are you know essentially everything they used to do they have outsourced to something like a large language model. And I don't know what they're giving up by doing that. In one sense, they're getting a lot out of it. They're probably 10 times as productive as they used to be in terms of code to get some functionality. And at the same time, they're probably losing 10% of their brain cells or failing to exercise them properly. And I just don't know. It's early and things, whatever we

8. 1:15:53

   say today in a week or a month will not probably be correct anymore the place where this shows right around here, at least, is kids worrying about what's going to happen when I get out of here. I wanted to be a software engineer. Does that job still exist? And the answer is maybe, but it's different than it would have been two or three years ago. And by the time they graduate in a year, it'd probably be different yet again. And the question is, how do we, on the faculty side of things, let's say, help them manage this? What do we try to teach them? How do we get them to have the right habits of thought or whatever, so that they are able to go thrive in this brave new world? And the short answer is, I have no idea either. And I don't think anywhere does. In the world you describe, which is today's world, the models are not perfect. There's a lot