Nobody knows how the whole system works

@lproven

great article, back in the 80s you had to know how your language of choice interacted with the cpu nd memory.

amazed at no comments

@lproven It's not even that no single person knows how the whole system works - that is inevitable in any complex system, I bet no single person could have designed Rome's water distribution system either.

The great risk of LLM deskilling is that now people don't even know how their little part of the system works.

@lproven

Yes, [AI] represents a significant shift in how we build software, it moves us further away from how the underlying stuff actually works, but the benefits exceed the risks.

I don't know about that last one. If it weren't for the numerous massive externalities the current crop of AI currently has, it might have been easier to argue for it, but in the current situation, I don't think it's that clear-cut.

Of course it's easier to clump the people who are concerned about these never-before-seen externalities with the people who are averse to change in their craft in order to dismiss both wholesale.

@lproven Also the difference between AI and the entirety of the technologies mentioned in this article is that machine learning (of which LLMs are an outcrop) is deliberately designed to obscure its inner working. It is a feature that makes it incredibly effective to reach its goal, but as a result even machine learning experts cannot explain how a system produced a given outcome, by design.

So "nobody can't understand everything" as a hand-waving for AI is terribly reductionist. We voluntarily made computer systems inaccurate in order to bring about God in the machine and we ended up with a shitty climate change acceleratory Eliza.