> Authors, screenwriters, et al. have a new niche to explore. Any day now I expect an A24 trailer featuring a villain who speaks in the register of ChatGPT. “You’re absolutely right, Kayleigh,” it intones. “I did drown little Tamothy, and I’m truly sorry about that. Here’s the breakdown of what happened…”
May I recommend Pluribus (2025-)
Yes. The creator has said he didn't have AI in mind while writing the show, but I think it's still a very stimulating AI parable despite that.
The sycophancy, the grinding inevitablity of assimilation, the homogeneous entity that speaks out of a billion mouths. It's all there.
Rather than the alien infecting humanity with it's DNA virus thing, it could somehow seize the "means" to existing LLMs/models/etc and claim to have "archived humanity's essence". This could then become the 'brain' that speaks from a billion mouths.
It's a large difference in concept but potentially only a small difference in outcome.
“You cannot give me anything, because all that you have is stolen.”
Also, the show Devs (2020), by Alex Garland. The joke is "Devs Ex Machina", Ex Machina being another film of his.
Eh. I enjoyed it enormously and I do likewise recommend it, but its story isn't related to AI (either the concept of its moment or the technology of ours) even slightly, nor trying to be, or at least not in any way I saw. It was pretty open with its themes, so I would expect that one to have been pretty noticeable if it was present alongside the questions of reality, artifice, grief, and simulationism with which the miniseries does concern itself.
You're right, not necessarily AI, more so what an omniscient and omnipotent device could do especially in the wrong hands.
What's the difference between the idle imaginings of a god's mind and a universe scale simulation?
I always got advanced AI vibes from Devs, that it was a mind interfacing with reality in some sort of weird inception / simulation / manifestation way.
In the show the device is not autonomous, humans use it.
That can be a type of mind, though? It can also be a type of interface - a tap into a system not fully understood, controlling the perspective or view but not the process. The whole "mind of god" Deus/Devs, etc - I think it's left ambiguous on purpose for the hook but I always took it to be an AI flavored story at the core.
Maybe. The plot itself is based on a short story that Garland read, both the 2007 original [0] and its 2022 rewrite [1]. Qntm is great and their latest book, There Is No Antimemetics Division, recently was on HN as well [2].
[0] https://qntm.org/responsibility
"in the wrong hands"
that's fucking funny
Thanks for the recommendation, the premise of the show sounds great!
"... That's not cheating. That's being smart."
There's literally an ad for Amazon's Alexa devices that features not just the gist of your example, but that specific cause of death (which is itself predated by a murderous digital intelligence doing the same thing in AMC's Pantheon series).
Guys I thought it was the fire next time.
Best line:
> I am concerned that ML systems could ruin our lives without realizing anything at all.
It's hard to say it's not actively happening. And we don't even know it, don't realize it? don't care?
(Claude Code got mandated at my work this week. Like literally engineers must use CC.)
> (Claude Code got mandated at my work this week. Like literally engineers must use CC.)
I'm taking a break from this industry until the madness blows over. I cannot even, anymore.
With the current direction of this industry, I fear you will come back to something that is both unrecognisable and terrifying.
If there is anything left to come back to.
More likely you’ll return to an industry utterly swamped in technical debt.
Anyone claiming these systems are for the "democratization of software engineering (or any knowledge field)" are simply not grasping the reality at hand.
It's not like our corporate leadership is being subtle in any way about their openly stated end goals here. We are simply instructed to continuously burn our enormous piles of "thought tokens" for their machines. The same machines only made possible by the theft of humanity's collective works at unimaginable scales. We must hold up these statistical facsimiles of human work, now rendered by machine, as the inevitable future output of humanity. To do otherwise is the way of the luddites.
The gods of unbounded growth, efficiency and productivity have come to demand our sacrifice. Who are we to stand in their way when countless skilled laborers before us have fallen to automation? Our number has been called as it were, and to reject what they demand is to reject "progress" itself. The march continues as it has since before the dawn of industrialization, relentless and indifferent to any and all that are crushed beneath it.
I don't know when the last bastion of "inefficiency" will fall, but at some point humanity may collectively grow to regret some forms of automation. Time will tell.
There's a certain simplicity and beauty in honing and stewarding a craft towards mastery that automation doesn't provide. I feel this is an innately human desire. Many cultures in the past used to place a great emphasis in this very personal endeavor. Sadly it seems each decade we slowly suppress these basic human needs, too tantalized by Western values of having ever "more" of everything, much to our detriment.
> (Claude Code got mandated at my work this week. Like literally engineers must use CC.)
As I haven't been a typical full-time employee in software development for some time, could you possibly just like leak the entire email where this was announced or something? (open invitation to others who could too, if parent cannot) I'm very curious to see how it was announced and what possible reasoning they could have for that.
Don't get me wrong, I use agents for lots of coding too, but forcing people to use tools they might not want to use doesn't feel like the right way. I was also allowed to use vim whenever I wanted for most of my career, something that feels more and more rare when speaking with people just starting their careers now.
The official declaration came about in a team-meeting. We're a tiny startup, 3 full-time eng with the CTO co-founder driving the AI transformation. We have scheduled onboarding meetings to get the entire company finding automation opportunities with specifically Claude (CC, Cowork). For eng specifically, there is "acknowledgement" that we all may have different setups, but we should all be unifying our prompts, strategies, pipelines, Agents, and so... it's CC. I still use Cursor so I'm the only one not on CC; my eye-brow was especially raised.
I don't want to be doxxed lol, so ironically I'll be sharing in the sense that on one hand I am unsettled by the mandate, on the other hand, for a tiny startup it's seems the state of the industry, less so company specific.
Startups (think they) are in a fight for their life, so the mandate comes from everyone contributing 10x or whatever. The expectation is that agentic coding should 3x/5x/10x your feature output, because that's how we're going to win.
I have many thoughts. But I'll focus on that last one: the mandate is literally more features, more code, because it gets us closer to winning. In my small engineering circles, surprisingly, this is like the defacto stance. All things considered, might as well ship more code!
I shall wait your collected thoughts. I’ll suggest The Mythical LLM Month as a title.
You made my day with this line =P.
tbh I have been struggling with the state of software in the agentic era. I'm pro LLM, there's undeniable leverage in its coding abilities. I do want to write this post! I'll start with hopefully this distillation:
In the agentic era, if shipping code is a commodity, then why ship more code? we can say this for the entire concept of "build" - we've commoditized building anything software related, i don't understand how this translates to therefore BUILD MORE.
So then the more nuanced conversation is that taste and judgement is the leverage. i agree with this. But its hand wavy in that we can't agree on what taste is. and also accelerationists hold true that all can be encoded. more agents. i don't even disagree with this entirely.
What I'm missing is that AI-native software engineers are going to brute force their way to PMF, to judgment, taste, enlightenment, consciousness.
But why is this a straight line? Just add more agents. add a "designer", a "sales" and "user researcher" agent. just add more agents.
You don't know what you don't know, is my retort. It's surreal to me that we're living through the equivalent of the smartest software engineers effectively giddily prompting, with full conviction mind you, "add more pop!" to make their thing better.
Just needs more pop!
Man, I completely agree with your thinking here. I've been trying to be more active in online communities, to try to discuss this exact idea.
LLM code can be leveraged, but pretending that tokens are just going to turn into money printers at some point is not productive. The primary source of software's value to an end user is the thought that was placed into it. Where does that go for the AI-natives? As you say, they are seemingly brute forcing software engineering, at least so far.
One thing I have been considering is how LLMs primarily change the "build vs buy" calculus for a fair number of software niches, particularly things like developer tooling and small libraries and packages. Partially due to a projected increase in supply chain attacks, and partially due to the changing standards of engineers. There's no longer anything stopping someone from working with an ugly or clunky syntax, presuming it's a well documented standard. So many "developer experience" tools are going to hit this - Tailwind primarily comes to mind.
It's a sort of "erosion" of niches in the current landscape - although to me this does not really work out for the worse in the long term, since again, the thinking in the process will need to just go somewhere else.
You’re welcome! At least I did something productive with my day :-)
How about The Million Dollar Mythos Month Some of these AI trends are starting to look more like gacha game moneysinks than productivity tools.
We did a similar Claude code mandate a few weeks ago.
Motivation was people being so allergic to tests and automation, that making them use superpowers produced better code, but also started adding a test pyramid.
The mandate was actually phrased in a way that you must produce industry standard code, and if you struggle with it you can use cc to bridge the gap.
Honestly I worry that this way devs will produce higher quality code, but will not understand why, how to measure the “quality” and steer towards it themselves.
At this point though the founders were pretty adamant with the code quality and lack of tests so this seemed like a reasonable way for the company, and I am curious to see how such a mandate affects code, deliverables and individual’s knowledge.
So far it seems to be working as intended, but it is early days.
> Motivation was people being so allergic to tests and automation
Frankly if your devs weren't doing this stuff before, forcing them to do it with AI assistance is probably going to be counter productive. If it is possible to produce good quality code and tests and such with LLMs, it is not likely by forcing LLMs on people who didn't care about code quality or tests before
I sometimes see myself writing too much manual code, so I reprompt my work in another directory for claude, and let it do whatever the fuck he wants, because I know for a fact that my company is monitoring claude usage as a metric.
> And we don't even know it, don't realize it? don't care?
People here are warning about this daily. And outside this tech bubble full of people trying to profit from it, higher percentages of people elsewhere do realize and do care.
That's a good point. But I also think about the unprecedented adoption of chatGPT. Global mass consumer adoption of all ages, and people giddy about offloading their jobs to chatGPT not connecting the very simple straight line to "you think your boss is going to pay you to ask chatGPT?"
> Bored houseboys might download licensed (or bootleg) imitations of popular personalities and set them loose in their home “AI terraria”, à la The Sims, where they’d live out ever-novel Real Housewives plotlines.
I can't wait to put myself into a sim world and give myself super powers.
Statistically, I should already be one of them. But where are my powers? It's another Fermi paradox for sure.
The workaround is to schedule the creation of many many copies of myself, each with its own sim world, just after I fall asleep. So I will have an arbitrarily certain chance of waking up as one of those copies, and the ability to fly.
> I can think of a few good myths for today’s “AI”. Searle’s Chinese room comes to mind, as does Chalmers’ philosophical zombie. Peter Watts’ Blindsight draws on these concepts to ask what happens when humans come into contact with unconscious intelligence—I think the closest analogue for LLM behavior might be Blindsight’s Rorschach.
LLM's remind me of sprites, pixies, and the like, who are situationally helpful but require constant supervision. We're like modern magicians who learned how to summon these sorts of spirits and bind them -- imperfectly -- to our will. But their perception of truth and reality is "through the looking glass" relative to our own. They aren't lying, from their own frame of reference, even though what they say is untrue relative to ours.
Speaking of myths, pixies, and spirits:
> I. DEFINITION:
> MAGICK is the Science and Art of causing Change to occur in conformity with Will.
> (Illustration: It is my Will to inform the World of certain facts within my knowledge. I therefore take “magical weapons,” pen, ink, and paper; I write “incantations”—these sentences—in the “magical language” i.e. that which is understood by people I wish to instruct.
> I call forth “spirits” such as printers, publishers, booksellers, and so forth, and constrain them to convey my message to those people. The composition and distribution is thus an act of MAGICK by which I cause Changes to take place in conformity with my Will.)
- Aleister Crowley, "Magick Without Tears," Chapter I, 1954. https://hermetic.com/crowley/magick-without-tears/mwt_01
A common definition anthropologists use for magic is occult technology: a system of laws that can be manipulated to create desired changes. There's a lot of value in thinking of programming as a form of magic.
Can you expand on this? It has always seemed to me that while programming does indeed like to couch itself in magical terms ("he's a database wizard", "this compiler stuff is black magic", etc), it is fundamentally understandable and replicable. All layers of programming build on their lower layers and this stuff is understood well enough that you can go to university to learn about it in detail.
Programming is technology but not "occult" technology, and I don't really see the added value of treating it as occult. Quite the opposite actually, most good programmers I know acquired their skill because they have a decent grasp about the entire system rather than treating most of it as a black box.
Oil is the medium of time manipulation magic. Created through ancient sacrificial rituals, it is is a substance that can be used to create aging/rot-retarding barriers, or refined into derivatives that increase the rate of plant growth and mechanical work. To be handled with care, as extended contact can lead to corruption of the body, as well as increased susceptibility to fire elemental spells.
Simple rituals can render an inferior product from most living things; the time-manipulation abilities of such substances will be weaker, but the substance will be safer to handle, and can even be imbibed (this is a double-aged sword, reducing one's vital life force while increasing one's bodily proportions to that of a toddler).
You either see it or you don't.
There's actually a useful and quite generic metaphor to be excavated here. I would just tell you what it is, but I think you'll more enjoy finding it for yourself.
A definition by which every human alive ever qualifies as a magician, and which is therefore not very useful as a distinction.
> A definition by which every human alive ever qualifies as a magician
Exactly correct.
Chapter 2: "No, every act of your life is a magical act; whenever from ignorance, carelessness, clumsiness or what not, you come short of perfect artistic success, you inevitably register failure, discomfort, frustration. [...] Why should you study and practice Magick? Because you can't help doing it, and you had better do it well than badly."
If you called him on it he would say that was on purpose, then talk your ears off about how. He was a ferociously effective charlatan, which is why people still remember the name he made up for himself. (And even invented a rhyming couplet to prate as a pronunciation guide!)
These don't sound like convincing indicators of being an "effective charlatan". Am I to see the Notorious B.I.G. in the same frame?
You are delusional.
Crowley is full of shit.
By this definition a hammer is a magical or "magickal" implement - the K was Crowley's invention, so that he could trademark it - which of course can be true if someone decides as much, but the only reason to couch such trivia in the pettifogging obscurity Crowley favored is because doing so will help you nail bored young socialites, an activity which Crowley also famously favored. (Gotta watch out for that neurosyphilis! What a shame he never did.)
Try thinking for yourself, instead.
> pettifogging
Off-topic but just wanted to thank you for teaching me a new word. I try to always reply to HN comments that expand my vocabulary.
Is pettifogging some kind of etymological parent of bikeshedding then?
Nice word.
I rue the day the IG reels crowd pick up on it and it becomes the "word du jour" that gets overused to the point of being intolerable. Right up there with "narcissist" and "gaslighting".
The problem isn't so much overuse as misuse, as "gaslighting" gets thrown around for almost any kind of falsehood.
Another example would be "Ponzi scheme", which I've seen abused for any situation the speakers seems unsustainable, even when there isn't any records fraud.
For sure. Despite all the talking about "self-deification" and all that shit, they sure seem to care a lot about what society (and their imaginary demons) think about them.
It's kind of an aside in the post, but connecting LLMs and Searle's Chinese Room argument is a brilliant observation. Although there are people who believe LLMs are really thinking, it's mostly confirming that the Turing test wasn't the right way to test this.
Is it? The observation seems patently obvious if one has even the most superficial understanding of how LLMs work? Why is this not common knowledge?
Understanding LLMs require a few pieces of knowledge that are not very common in the industry, not only ML stuff, but I think that having worked in the past with NLP helps a lot.
I fucked around with dissociate press 30 years ago just to see what would happen if you fed it a combination of random chapters from Alice in Wonderland and the book of Revelations. This hardly represents insider knowledge.
If 'aphyr is reading comments here, I'm curious: have you read Joseph Weizenbaum?
I read a fair bit about Eliza when I was young, but I don't recall if any of those were from Weizenbaum himself!
See Computer Power and Human Reason - many powerful lessons in there.
Our collective learned helplessness in the face of being bombarded with advertising, propaganda and outright lies is just astonishing to me. Not an article about fighting back, or doing anything, just the resignation of a follower.
There's a lot of people who are comfortable (socially or professionally) with diagnosing and analyzing problems. Those same people are often indifferent or outright hostile to people proposing solutions, not least because solutions that brought about change would make the analysts less relevant.
It's a hard problem because it reduces to the fact that narcissists and sociopaths are a significant proportion of the population, and they're strongly attracted to money, power, attention, and status.
So even though they're a small minority they infest politics, business, and the media, and create a culture in their own image.
Most proposed solutions end up in superficial tribal arguments about standard economic and political positions. Not about the underlying issue, which cuts right across the usual battle lines.
Frankly, who are these people? Because this is just another "they" idle conspiracy theory.
"They" are against me.
Ironically I could cite a very specific group this applies to: fitness influencers in the wake of ozempic. "Natural weight loss" and FUD about the drug took off when it hit mainstream awareness because it really was a direct threat to them. Of course this group also tends to heavily abuse other drugs as they age out - because being a trim fitness inflencer is easy in your 20s, keeping it going into your mid-30s is a lot more difficult.
I was thinking specifically of political pundits who are doing a roaring trade (in opinion columns, TV hits, book promotions etc) talking about authoritarianism and its many causal factors. They're curiously mute when it comes to discussing solutions, with very generic advice like 'go to a protest' or 'vote for the opposition' despite the abundant evidence from authoritarian regimes around the world of these tactics not being very effective. You never hear them talk about things like general strikes or mass civil disobedience campaigns for some reason.
reminds me of the college scene in the movie Tomorrowland where all the teachers going on and on about about the things that would end us and when she asked "Can we fix it?" and teacher is like "What?" "I get things are bad but what can we do?"
learned helplessness is really a problem, but personally all I have gotten is scorn and hatred for trying to make a difference/improve things that I managed/had control of. All people care about is precious number go up/ignoring the future while everyone around me is looking at me like I have two heads for not blindly following the insanity
what do we honestly do?
Find others likeminded locally and cooperate where opportunities present themselves.
What would you do? The system is very good at transmuting critiques of it into its own propagation.
A learned helplessness as a diagnosis implies that there are things that can be done. But I can't see any. It may be because of a learned helplessness of course, so my inability to see what can be done can be a fact about myself, not about the world around me, but still... It is a catch 22, isn't it? Maybe not, but it is a self-reinforcing uncertainty loop. I'm not buying it.
Unfortunately the political movements most interested in the relationship between society and technology were wrapped up with Nazis and so the line of thinking is underdeveloped, as it has had to start again.
Ok. How do you propose one fights back? Do you really viscerally understand what it is you are fighting against?
fork it. Fork the internet. How about that? We have this stupid system built on people paying to target us. Is that "the" internet or is it just one internet, and not a very good one. This is supposed to be a place for hackers. So fork the internet.
The application layer has way more gravity than the networking one right now. You'd need to fork that for anything to happen.
It's (mostly) not the networking layer where people pay to target us. It's the application layer that would most benefit from being forked.
Of course the problem is that what can be forked already has been. Federated social media. Distributed git hosting. However most "essential" uses are centralized and often also commercial in nature. If you fork Amazon you're ... still Amazon. That sort of thing.
I mean, I've had pipe dreams of a parallel networking stack built on IPv6 with a killer app offering real peer-to-peer networking again. But who will deploy and maintain all the new IPv6 networking gear, infrastructure, undersea cables? This is not something within the purview of the average (or even above average) hacker any more; the encumbents are too firmly entrenched.
Collectives, cooperatives and other organizations. It's been done before, giant WiFi networks spanning countries. Freifunk, Guifi and NYC Mesh are a few of them, the two first are real and alive alternative networks built on independent hardware infrastructure.
Not sure about the details with Freifunk, but Guifi has collaborating companies who basically operate like ISPs but connect you to Guifi + you get a internet connection via the Guifi peering the installer helps you with.
Personally, I am guessing it is the other way around. Not first killer app, but first a parallel networking stack, etc.
The idea that anyone is too firmly entrenched is always true. Until it is not. We are not still using horses, nor casting bronze tools. Nor do most ships sail. We (mostly) don't burn coal anymore. It would have been utterly insane to imagine replacing any of those entrenched technologies. That is the "follower" syndrome.
The context for a new kind of internet is very different now, than when the internet started. That changed context provides an opening for new ways to do things.
A crucial and hard piece is that it has to be paid for. As much as you need a parallel networking stack you need a parallel business model. Yes, the current one is too firmly entrenched, etc.
I think Heidegger provides a framing for why these are just musings in, as you put it, a firm entrenchment and the whole ordeal is looking quite bleak. I honestly can't imagine a way out of the technological frame and I am simply not seeing my generation in common spaces. Even my ability to meaningfully connect with my peers through conversation is deteriorating by product of the sheer scale of potential engagements one has at any moment. It is quite overwhelming and I am afraid there is no technological answer here.
But like...why?
This is basically "just don't use things you don't enjoy" and the trouble of our time seems to be the number of people who can't or won't do that.
It's somewhat an age thing but also definitely a lot of people in all generations never learn it: you can just stop using things. Walk away and suddenly find you never want to look back, and if you do it's entirely unappealing.
Good article, but I cannot help myself to not bring up lack of appreciation for humanities in tech circles.
Article mentions Searle and Chalmers, but we literally have at least two centuries of critical thought that expressed itself through Nietzsche, Marx, Horkheimer, Adorno, Weber, Durkheim, Foucault, Debord, Baudrillard, and many others (obviously I am mentioning those I am most familiar with). If you read Dialectic of Enlightenment, you'll find that slop isn't something that had arisen in last few years. If you read Discipline and Punish you'll find that surveillance and coercion isn't a problem that was born with internet or Palantir. And Baudrillard had few words to say about simulation and reality.
But STEM crowds for decades cried we don't need such thinking, and science and technology are all that we need. Historians, philosophers, culture critics etc. supposedly have nothing to offer us. Who needs to read Marx or Marcuse if sci-fi novels offer all you need, maybe sprinkled with some PG essays and blogpost from you favorite tech blogger, and we happen to live in the best of possible worlds with the best of possible economic systems.
This has already happened with media for the past 100+ years. We're shown what companies and governments want us to see. People develop parasocial relationships with people they see on tv...
If fear is the mind-killer, then sexy chatbots are the libido-killer, for me. Hard no.
> Unavailable Due to the UK Online Safety Act
NSFW blog content on HN? Really?
Woah you can't see this in the UK? Without age verification?
Update: there's a section called "Pornography". It does not contain pornography.
It's not blocked By the UK, he's put a country filter on it Like all the US sites that decided to block.the EU over gdpr because turning off tracking was too much effort
Work fine in Spain, so lets not jump on the all to typical EU-made-the-internet-suck bandwagon all too quickly. Seems this is a UK problem. But what do I know, I actually read the error message.
It's the author's problem with the UK (and the UK's Online Safety Act, which establishes requirements on hosts that can't be avoided by merely not being in the UK), rather than the UK's problem with the author.
But as much as I dislike the OSA: if you're not subject to UK law, why do you (website author) care what our government thinks of your website? It's not like they can do anything to you.
Visitors are subject to UK law.
He has an article for that
https://aphyr.com/posts/379-geoblocking-the-uk-with-debian-n...
> This is, to be clear, a bad solution. MaxMind’s free database is not particularly precise, and in general IP lookup tables are chasing a moving target. I know for a fact that there are people in non-UK countries (like Ireland!) who have been inadvertently blocked by these lookup tables. Making those people use Tor or a VPN sucks, but I don’t know what else to do in the current regulatory environment.
perhaps the safety filter is wrong instead of the post?
Most likely it's a protest. Badplace passed Badlaw, so residents of Badplace can't see my content, so nyah!
But, topics of a sexual nature—nothing really NSFW, just mentions of various fetishes that online people have developed and popularized, and the possibilities for AI to realize those fetishes and potentially spawn new ones—are discussed in the blog post, so it may be illegal to present to minors under the OSA.