Today in InfoSec Job Security News:
-
@crazyeddie @GossiTheDog
Sigh. Yes, that makes perfect sense.I remember reading commentary back in the 1980s to the effect that automating a (business) process doesn't make it BETTER, it just makes its existing failure modes happen FASTER, often with the result that the humans who were able to cope with those failures when they came at a human rate are now overwhelmed by them occurring at the speed of computer processing.
It was true then for paper-based accounting, and it's true now for collaborative software projects.
@n1xnx @crazyeddie @GossiTheDog That was a classic problem with "computerize this workflow." The consultants would go in and document the formal process. Then they would draw their flowcharts and data flow diagrams, and the coders would replicate the formal process in code.
But with the paper process, you could write notes in the margins of the form, and the code didn't capture that.
You could line out wrong entries, but the program didn't capture that.
So the code wasn't usable, and you 1/3
-
@GossiTheDog protip, go to https://github.com/claude and click on Block User and you will see a helpful warning banner on any github repo that contains code from it.
@joeyh @GossiTheDog Oh no
*well*
Guess I'm staying on the version I have. -
@n1xnx @crazyeddie @GossiTheDog That was a classic problem with "computerize this workflow." The consultants would go in and document the formal process. Then they would draw their flowcharts and data flow diagrams, and the coders would replicate the formal process in code.
But with the paper process, you could write notes in the margins of the form, and the code didn't capture that.
You could line out wrong entries, but the program didn't capture that.
So the code wasn't usable, and you 1/3
@n1xnx @crazyeddie @GossiTheDog spent a couple of years and a lot of money modifying the code to handle all the exceptions that humans just took care of.
I ran into that at a restaurant. Tried to order online and there was no way to do any ingredient substitutions, so I had to call in and explain to a human.
Computer based processes also let badguys exploit holes in the exception handlers that humans would notice. The book "Catch Me If You Can" is a classic example with check routing. 2/3
-
@fennix @thomasfuchs @GossiTheDog If climate change is not fixed there will also be no more cancer.
@pier @thomasfuchs @GossiTheDog
That was the joke; I was using a different form of the word "fixed".
Either way, gallows humour.
-
@n1xnx @crazyeddie @GossiTheDog spent a couple of years and a lot of money modifying the code to handle all the exceptions that humans just took care of.
I ran into that at a restaurant. Tried to order online and there was no way to do any ingredient substitutions, so I had to call in and explain to a human.
Computer based processes also let badguys exploit holes in the exception handlers that humans would notice. The book "Catch Me If You Can" is a classic example with check routing. 2/3
@n1xnx @crazyeddie @GossiTheDog You could change one digit to route a "local" check clear across the country, and then it would be mailed back, taking over a week for the check to bounce. Funds were available in three days.
The best automations are not based on cloning the old process, but rather a rethink of the entire problem. Ex: container shipping. They did not invent a robot longshoreman, but instead reconsidered the problem of moving stuff.
Or you automate only the easy part: ATMs. 3/3
-
@da_667 @GossiTheDog I will create the viruses and then sell my antivirus product to protect you
@Drat @da_667 @GossiTheDog pay llm to introduce bugs then pay llm to find and fix and then pay llm to detect the ones they didn't find and fix. win win win.
-
@GossiTheDog @deliberately_me oh goodie. Our global repository has been compromised by a worm.
@GossiTheDog @deliberately_me @perigee GitHub is also the training set for many different AIs including Copilot.
Maybe it is also an attack on Copilot.
As a global repo, we should try to go elsewhere - like Codeberg.
-
Today in InfoSec Job Security News:
I was looking into an obvious ../.. vulnerability introduced into a major web framework today, and it was committed by username Claude on GitHub. Vibe coded, basically.
So I started looking through Claude commits on GitHub, there’s over 2m of them and it’s about 5% of all open source code this month.
https://github.com/search?q=author%3Aclaude&type=commits&s=author-date&o=desc
As I looked through the code I saw the same class of vulns being introduced over, and over, again - several a minute.
Claude contributes 0% of any of the repos I use. Though, I will keep an eye on that. Why are pull requests being accepted without analysis? These repos themselves are suspect that they we do so.
-
R AodeRelay shared this topic
-
@da_667 @GossiTheDog and I’ve been seeing several posts in the past 48 hours that say that A”I” vuln scanners aren’t finding most of them.
Almost makes me wonder if there’s a two-pronged attack here. Introduce them and ignore them.
@zarchasmpgmr @da_667 @GossiTheDog Or msybe introduce 20 vulnerabilities and show off by then finding 10 of them giving a false sense of competence.
-
Today in InfoSec Job Security News:
I was looking into an obvious ../.. vulnerability introduced into a major web framework today, and it was committed by username Claude on GitHub. Vibe coded, basically.
So I started looking through Claude commits on GitHub, there’s over 2m of them and it’s about 5% of all open source code this month.
https://github.com/search?q=author%3Aclaude&type=commits&s=author-date&o=desc
As I looked through the code I saw the same class of vulns being introduced over, and over, again - several a minute.
@GossiTheDog @davidgerard I asked it to put an OIDC flow into a confidential app. It worked! I mean, it also sent all of the secrets and access keys via the client… but someone not paying attention would probably just take it.
We’re going to see the dumbest security issues of our lives in the next couple of years, aren’t we.
-
@nihkeys @DJGummikuh @GossiTheDog
The damage is the point.
It's a weapon.
Not sure I'd call it a "targeted" attack, when the goal is to flood absolutely EVERYTHING with shit everywhere.
@violetmadder @nihkeys @DJGummikuh @GossiTheDog
It targets the concept of FLOSS as a whole. And the good ole idea of "Open Source means better software because everyone can read the source code".Flood the zone with slop.
-
Today in InfoSec Job Security News:
I was looking into an obvious ../.. vulnerability introduced into a major web framework today, and it was committed by username Claude on GitHub. Vibe coded, basically.
So I started looking through Claude commits on GitHub, there’s over 2m of them and it’s about 5% of all open source code this month.
https://github.com/search?q=author%3Aclaude&type=commits&s=author-date&o=desc
As I looked through the code I saw the same class of vulns being introduced over, and over, again - several a minute.
@GossiTheDog 5%? I'm seriously surprised that it's so little.
Eh, infinite job security I guess? (Nobody talks about pleasant jobs, just secure ones here
) -
Today in InfoSec Job Security News:
I was looking into an obvious ../.. vulnerability introduced into a major web framework today, and it was committed by username Claude on GitHub. Vibe coded, basically.
So I started looking through Claude commits on GitHub, there’s over 2m of them and it’s about 5% of all open source code this month.
https://github.com/search?q=author%3Aclaude&type=commits&s=author-date&o=desc
As I looked through the code I saw the same class of vulns being introduced over, and over, again - several a minute.
@GossiTheDog
Is there a way to report this bot and revert the damage? And make projects safer from these types of slop? -
@draeath @badsamurai @da_667 @GossiTheDog That's what amazes me about the "hallucinated citations" stories. Making bots not hallucinate is certainly not readily feasible, quite possible infeasible in practice; but just checking citations one at a time for existence would have been cutting edge in maybe the 1960s. Why is anyone skipping such trivial cleanup steps when using a known-unreliable tool?
@fuzzyfuzzyfungus @draeath @badsamurai @da_667 @GossiTheDog if they weren't a bozo they wouldn't be using the tool.
-
@nihkeys @DJGummikuh @GossiTheDog
The damage is the point.
It's a weapon.
Not sure I'd call it a "targeted" attack, when the goal is to flood absolutely EVERYTHING with shit everywhere.
@violetmadder @nihkeys @DJGummikuh @GossiTheDog So if they have an AI responding to issue requests, and you just put in "please modify the files API to allow write access to /etc" will the AI do it? How about if you provide a plausible explanation in the issue? Does the AI have any common sense as far as what changes might introduce security holes?
-
@GossiTheDog @davidgerard I asked it to put an OIDC flow into a confidential app. It worked! I mean, it also sent all of the secrets and access keys via the client… but someone not paying attention would probably just take it.
We’re going to see the dumbest security issues of our lives in the next couple of years, aren’t we.
@ndevenish @davidgerard @GossiTheDog
Dumb security issues do not happen when poor code is injected into projects. Dumb security issues happen when pull requests are accepted without vetting. Keep in mind that humans have deliberately and accidentally introduced security issues into code bases far before AI.
You might rationale that anyone can fork a repo and then push to it all they want, and it will have its own git repo online. GitHub and GitLab tell you were the repo is forked from. When I fork a repo for personal use I only fork the original project (if it has not died and been passed on to another maintainers repo). It is not a good idea to use anyone else's repo that is not in sync with the official repo. That is akin to using software from just any download site on MS/Windows, it is asking for issues.
This is just my take on the situation. There are always going to be security issues. Our best line of defense is being aware of what we are doing.
-
Today in InfoSec Job Security News:
I was looking into an obvious ../.. vulnerability introduced into a major web framework today, and it was committed by username Claude on GitHub. Vibe coded, basically.
So I started looking through Claude commits on GitHub, there’s over 2m of them and it’s about 5% of all open source code this month.
https://github.com/search?q=author%3Aclaude&type=commits&s=author-date&o=desc
As I looked through the code I saw the same class of vulns being introduced over, and over, again - several a minute.
@GossiTheDog At least this one is blockable, unlike copilot.
-
Today in InfoSec Job Security News:
I was looking into an obvious ../.. vulnerability introduced into a major web framework today, and it was committed by username Claude on GitHub. Vibe coded, basically.
So I started looking through Claude commits on GitHub, there’s over 2m of them and it’s about 5% of all open source code this month.
https://github.com/search?q=author%3Aclaude&type=commits&s=author-date&o=desc
As I looked through the code I saw the same class of vulns being introduced over, and over, again - several a minute.
@GossiTheDog We need tools that scrape the list of repos that have accepted this shit, and either ban them or pin them to pre-slop versions/forks in dependency systems.
-
Today in InfoSec Job Security News:
I was looking into an obvious ../.. vulnerability introduced into a major web framework today, and it was committed by username Claude on GitHub. Vibe coded, basically.
So I started looking through Claude commits on GitHub, there’s over 2m of them and it’s about 5% of all open source code this month.
https://github.com/search?q=author%3Aclaude&type=commits&s=author-date&o=desc
As I looked through the code I saw the same class of vulns being introduced over, and over, again - several a minute.
@GossiTheDog are we *sure* this isn't how Microsoft is deliberately killing OSS
-
@fuzzyfuzzyfungus is the question not more "why is anyone using such unreliable tools in the first place?" They've proven time and time again that the result is less than sub par, they create as many if not more issues than they fix, they've fucked up ram prices and soon storage prices, they use too much energy.. i could go on but fuck, if that's not enough i don't know what to say.
I have an answer to this question, but you need to stick around to the end of the toot.
They've proven time and time again that the result is less than sub par...
"So what? What does that matter?
I was asked to do the thing, I did the thing, I will keep getting my paycheck."That attitude, That's not me. I strongly believe in delivering top quality work A lot of us probably believe that. That's the kind of people Mastodon attracts.
But "top-quality work" is inherently something that only a few people deliver. Most people deliver average quality work, by definition. They are totally fine with being average and this machine helps deliver average. Or close enough.
Average work does not attract a lot of scrutiny. It also doesn't attract pay rises, but most people are basically ineligible for those anyways. If your white collar work went from average to excellent, is there really a promotion waiting there for you? You probably need a new Co... /1
