Grade the process, not just the code

Transcript

From the transcript The Pragmatic Engineer publishes with the episode.

1. 1:30:59

   But also I think a really simple solution is stop grading just the outcome. Focus on the process, right? Like even if you don't come up with wildly harder assignments for assessment, stop grading just the code that they turned in. Grade instead, what are the prompts that you used to get here? Also, what are the tests that you provided? Like how did you make sure that this thing is actually doing what you need it to do?

2. 1:31:29

   And like when you start focusing on like the process that got you there instead of just the outcome, like I think not only CS, but most disciplines, right? Like we've been using go write a new essay on capture in the rye. Like we don't need another essay on capture, right? Go write a new linked list. We don't need another linked list. We've been using the artifact that's produced from that as a proxy for you put in the time and the struggle.

3. 1:31:59

   And like faced the friction that we assume that you now have learned something. It might be more appropriate to just focus on do we have evidence that you've struggled and learned? Turn in the whole package, right?

4. 1:32:13

   Curriculums are designed by committees, which is for a good reason. But if you had Titus' software engineering school, where you could yourself put together the curriculum and study course for an undergrad course of, let's say, three to four years, how would it look like right now? And we're talking about getting engineers, ideally having engineers who can hit the ground running at software and engineering companies, obviously using AI tools, building software, that kind of stuff.

5. 1:32:34

   Yeah, so... I think the first course, like the first semester starts unaided by AI. Like we need to get you through at least the level that you can read code and have a vague idea of what's going on. Like you can understand control flow and those sorts of things. I think there's some really interesting things on well scaffolded and harnessed AI for educational purposes. Of we're intentionally not just going to give you the answer.

6. 1:33:02

   And in conjunction with if you're assessing the whole package and process of creation, then you don't need to worry about them going to chat GPT and just getting the whole answer. Some of that's covered. I mean, I'll be honest. I don't know. Nobody knows. There is no playbook. But one of the ideas. I want to get opinionated. So one of the ideas that I've had is let's make CS0 like non-majors and majors alike.

7. 1:33:29

   Intro to programs, like specifically, how do you describe the inputs and outputs and structure and design of the interfaces, right? So we could talk about what's the difference between a checkbox and a radio button. When do you use a button versus some other UI widget for non-UI things like what command line flags, you know, positional parameters? How do you like interface these components specifically so that if that's the only class that you ever took, At least the AI vibe-coded slop that you produce for your own use cases is going to have a slightly more coherent, sensible design language and be more fit for purpose.

8. 1:34:10

   I live in New York. The condo building I'm in, there's an app for the laundry room. And drives me insane because, like, they use, like, menu UI for button, right? And, like, just the design language is completely wrong. Start with that. And then graduate kind of along the lines of what you were saying of, like, we want to move up into, like, projects and teamwork and things like that. Try to motivate them into themselves wanting to solve a bigger problem and then needing some Background on, oh, this is how the web works.

9. 1:34:49

   This is how a remote API RPC call works. You can't give people a solution to a problem they don't realize they have, right? And until they're trying to solve these problems, those solutions that we present in theory are all just trivia. They're just isolated, academic, not connected to anything. Like motivate that by like getting them into a situation where they need that as the answer first. And then you can provide that background.

10. 1:35:16

    I also desperately want more projects for two reasons. I don't want the assignments to be, there is a correct answer because that's not how the industry works. Mostly not how research works. And I want them to grade, part of your grade is critique of others. Like look at their outcomes or their code or both, right? And be able to say like, that was great. Did you think about doing a different thing here?

11. 1:35:49

    And then also grade them on can you give feedback without being a jerk? Because psychological safety and teamwork, right? And all of this sounds like a lot of work, except you can use AI to like do some of that analysis. Like let's do the sentiment analysis. Was that rude? Things like that. And I think we could have a much more practical, hands-on, and much more effective educational structure. And I worry that that's going to be software engineering departments and not CS departments.