Wanted: Advice from CS teachers

@futurebird

Software developer trainer here.

Silly question, but do they read the *entire* error description? And once they solved it do they understand the relationship between the cause, the message, and the fix?

It might be worthwhile to force them to slow down by asking them to write down the entire message on paper.

Try making a game out of it: Who can generate the strangest error message with the fewest amount of “bad” code.

@futurebird

When they reach more advanced levels, you will have to teach them to step even farther back, look outside the frame of the monitor, and account for a different layer of the stack.

What if your code works, so well in fact, that it threatens the empires of incumbent elites? What if they break down your door, unperson you, arrest you illegally dozens of times, jail you, brutalize you, deny you legal and medical care, and leave you in pain?

1/n

@EricLawton @david_chisnall

"Now I'm curious about whether LLMs' code compiles and executes error-free on their first attempt."

At first it did not, but they have added a routine to run it through a compiler until it at least runs without syntax errors and probably produces output that seems like what you asked for for a limited example of input.

This is a bolted on extra check, not some improvement in the base LLM.

But some people are acting like it does represent advances in the LLM.

@DevWouter

They are middle school kids totally new to programming so reading the whole thing is a challenge and they will encounter too many words they are still learning.

And really their errors are nearly always typos. Or not putting their function call in main body of the program. Or using a variable they have not defined.

So, for them reading deep isn't that important IMO.

The responses can be very verbose and technical.

@futurebird

How would you regain digital identity, access to your domains, and your money, and get back to doing what you love?

That's what I would like to learn. How to repair a broken body and mind, heal yourself, and go after the systemic injustices that made possible the violence done to a hacker, who could do things the rich and powerful could not? How to fix the system so you won't be bothered on your next coding run, in case your code really works!

It's more fun to compute!

2/2

@futurebird @david_chisnall In my (very limited) experience, all that resulted in is pseudocode with // before each line

@futurebird @EricLawton @david_chisnall
"I have improved my LLM"
*looks inside*
*improvement is 100% classical handcrafted algorithmic code*

@ben @david_chisnall

They've added some new feature that will pull up a little virtual machine and it will let you run the code in there. It also seems to test that the code will at least compile first.

I worry that people seem to think that the LLM just... "evolved" these features when clearly a human person had to add them.

@futurebird Software engineering is the stuff around software coding. The AI-vangelists have decided to re-discover the field from scratch, while expressing increasingly hubristic contempt for the lived experience of the entire field.

@EricLawton @david_chisnall

@futurebird a couple things I do. (It sounds like you're already doing some of these with slight variation):

1. Tell them I make errors when programming. My former boss who worked for 30 years as a programmer still makes errors.
Errors are not a bad thing. They are part of programming. Part of the process.
Errors are hints. They are clues to figure out how to make the program work.

@futurebird I tell my students that debugging is 50+% of the work, *even* for me with 10+ years of experience. Then I tell them that what makes me a good programmer is not my ability to write error-free code the first time, but my ability to quickly debug code based on oodles of experience.

You could try intentionally making a few common mistakes and showing off the error messages that result if you're projecting an IDE instead of just writing on the whiteboard.

I've also built a Python library called "optimism" that can do "did you use X construct" or even "X inside Y" which helps write certain kinds of practice problems for beginners. It can generate an error message for "you're not using an if statement inside a loop but the problem requires that" for example. Reading other responses here I'm reminded that I should probably put more effort into the error messages...

@futurebird
2. When a student tells me their code doesn't work, I almost never tell them how to fix it. Here are my steps:
a. Did you read the error? Okay then read the error.
b. What does it say? What line is the error on? Do you know what the words mean? Okay let's break it down. *Go word by word through the error to check their understanding and explain the words they don't know*
c. What part of the line do you think is the problem? How can we change it? What else could we put there instead

@futurebird @david_chisnall As a retired coder. The job description hasn’t changed. Just the ratios on tasks performed. Much more reading code vs lots less writing actual lines of code. I like the sound of this debugging approach, as it forces reading and understanding. I have long said the CS courses needed more reading/reviewing code. Like a large part of traditional art courses teach art “criticism”. You have to be able to tell “good” art from “ bad” art to be able to judge if you work is achieving the goals you want from it. The same applies to code.

@futurebird maybe you already do something like this, but I wonder if some sort of preamble to set expectations around errors will help? Something like "computer programming can look like math but the the instant feedback makes it feel very different. Eventually that will feel like a good thing, but know that you may get error messages when you dont expect them and that a large part of learning to code is learning how to read and understand what those error messages are saying."

@raganwald

Having spent thousands of hours working with various involved parties, trying to understand what they did and how IT could help, I came up with Lawton's Law,
> Everybody's job is a lot more complex than you think it is. Including IT folks' jobs.

@futurebird @david_chisnall