Today in InfoSec Job Security News:

Aristotelis Tzafalias

@Drat @da_667 @GossiTheDog pay llm to introduce bugs then pay llm to find and fix and then pay llm to detect the ones they didn't find and fix. win win win.

Reiner Jung 🇬🇱 🇺🇦 🇪🇺

@GossiTheDog @deliberately_me @perigee GitHub is also the training set for many different AIs including Copilot.

Maybe it is also an attack on Copilot.

As a global repo, we should try to go elsewhere - like Codeberg.

Unus Nemo

@GossiTheDog

Claude contributes 0% of any of the repos I use. Though, I will keep an eye on that. Why are pull requests being accepted without analysis? These repos themselves are suspect that they we do so.

Chris Johnson

@zarchasmpgmr @da_667 @GossiTheDog Or msybe introduce 20 vulnerabilities and show off by then finding 10 of them giving a false sense of competence.

ndevenish

@GossiTheDog @davidgerard I asked it to put an OIDC flow into a confidential app. It worked! I mean, it also sent all of the secrets and access keys via the client… but someone not paying attention would probably just take it.

We’re going to see the dumbest security issues of our lives in the next couple of years, aren’t we.

M Schommer

@violetmadder @nihkeys @DJGummikuh @GossiTheDog
It targets the concept of FLOSS as a whole. And the good ole idea of "Open Source means better software because everyone can read the source code".

Flood the zone with slop.

JTI

@GossiTheDog 5%? I'm seriously surprised that it's so little.
Eh, infinite job security I guess? (Nobody talks about pleasant jobs, just secure ones here )

Peteypetepete

@GossiTheDog
Is there a way to report this bot and revert the damage? And make projects safer from these types of slop?

Major Denis Bloodnok

@fuzzyfuzzyfungus @draeath @badsamurai @da_667 @GossiTheDog if they weren't a bozo they wouldn't be using the tool.

mike805

@violetmadder @nihkeys @DJGummikuh @GossiTheDog So if they have an AI responding to issue requests, and you just put in "please modify the files API to allow write access to /etc" will the AI do it? How about if you provide a plausible explanation in the issue? Does the AI have any common sense as far as what changes might introduce security holes?

Unus Nemo

@ndevenish @davidgerard @GossiTheDog

Dumb security issues do not happen when poor code is injected into projects. Dumb security issues happen when pull requests are accepted without vetting. Keep in mind that humans have deliberately and accidentally introduced security issues into code bases far before AI.

You might rationale that anyone can fork a repo and then push to it all they want, and it will have its own git repo online. GitHub and GitLab tell you were the repo is forked from. When I fork a repo for personal use I only fork the original project (if it has not died and been passed on to another maintainers repo). It is not a good idea to use anyone else's repo that is not in sync with the official repo. That is akin to using software from just any download site on MS/Windows, it is asking for issues.

This is just my take on the situation. There are always going to be security issues. Our best line of defense is being aware of what we are doing.

Laura

@GossiTheDog At least this one is blockable, unlike copilot.

Cassandrich

@GossiTheDog We need tools that scrape the list of repos that have accepted this shit, and either ban them or pin them to pre-slop versions/forks in dependency systems.

josh g.

@GossiTheDog are we *sure* this isn't how Microsoft is deliberately killing OSS

Gaëtan Perrault

@Fooker @fuzzyfuzzyfungus

I have an answer to this question, but you need to stick around to the end of the toot.

They've proven time and time again that the result is less than sub par...

"So what? What does that matter?
I was asked to do the thing, I did the thing, I will keep getting my paycheck."

That attitude, That's not me. I strongly believe in delivering top quality work A lot of us probably believe that. That's the kind of people Mastodon attracts.

But "top-quality work" is inherently something that only a few people deliver. Most people deliver average quality work, by definition. They are totally fine with being average and this machine helps deliver average. Or close enough.

Average work does not attract a lot of scrutiny. It also doesn't attract pay rises, but most people are basically ineligible for those anyways. If your white collar work went from average to excellent, is there really a promotion waiting there for you? You probably need a new Co... /1

Luna chan

@GossiTheDog People need to stop being lazy and using AI to write code.

Unus Nemo

@dalias

I agree with your concept as being a noble idea. I just do not see it as a realistic solution. These are my issues with your idea, and you may not agree with me that if fine. Your idea is that we make tools to scrape repos on git servers (and perhaps SVN as it is still used) and validate that it is accepting pull requests from AI. If I have understood you. My take on that is that if you are working on a project then you should be forking the main repository not some other person's random fork. Main repositories tend to be a lot more responsible in who they accept pull requests from. In any of these Claude infested repos was even a single one the projects actual main repository? I would guess no. If developers are practicing good OPSEC then this is a none issue. So we are adding strain on servers that is simply not required.

As developers we have a responsibility to our own integrity and are users to be sure that what we do release is as secure as we can make it. There is no such thing as completely secure software. It does not exist in reality (well maybe 'Hello world' ).

It is easy to get upset at such events. Though in the big picture is not a real issue, it is one of those issues that will be self-healing. I do not know a single developer that would not check who commits, are they using security measures like commit signing, is the project secure as is. Before forking, if they wanted to use it as a base and it did not meet those criteria they would hard fork and not participate in the original repo. Keep in mind that there are projects out there entirely written by AI, I do not endorse them, but they do exist.

It is okay to not agree with me, I am okay with that. I do not feel as if we should be censoring source code for developers. I feel like we should be teaching them about good OPSEC & DEVOPs instead. Just my opinion.

Have a great day!

Bert Driehuis

@GossiTheDog to be fair, before Claude the same thing would happen when folks used recipes off Stack Exchange without understanding them.

The atrophying of critical thinking as a result of AI usage is the final nail in the coffin.

Cassandrich

@unusnemo A repo that has AI slop anywhere it its git history isn't FOSS and has maintainers who have shown gross irresponsibility. Banning use of it as a dependency should not be controversial.

Gaëtan Perrault

@Fooker @fuzzyfuzzyfungus and if I walk through all the logic, I totally understand these folks.

I don't agree with them, that's not who I am personally. But I understand the incentive structure and they're going to work with what they have.

We have an entire book titled "Bullshit Jobs" talking about how widespread the phenomenon is. And here we have a machine that is basically a bullshit generator, all ready to tackle those bullshit jobs. It's a perfect match!

Is this a great use of human or computer resources? Probably not. But that misuse has nothing to do with either the computers or the humans in the loop. It's entirely part of a bigger system that fails to match incentives correctly.

But that's why humans use the stuff. Not because it's good, but because it's what they were asked for.

I think we should be asking for better things, but that shift is a blog post unto itself. //