I like Evans’ take that since there’s bound to be oodles of cult related literature and interactions and also tons of self help and guru stuff in the training datasets, it stands to reason that if you interact with a chatbot in a way that indicates vulnerability to these things there’s a considerable chance that it will decide the expected response is to prey on you.

Also Scott Aaronson jump scare near the beginning, apparently he was blurbed for something.

a leaderless cult had freshly formed

a Stand Alone Complex, but with slop

Well, we do have computer science, so necessarily we must have computer religion/superstition

Trust, but verify, and tape a shotgun to their forehead just in case.

edit I vaguely remember the shotgun to the forehead as a reference to the Turing Registry in Gibson’s Sprawl trilogy, but I can’t find a direct quote. Am I totally off base with it?

edit edit found it

“Autonomy, that’s the bugaboo, where your AI’s are concerned. My guess, Case, you’re going in there to cut the hard-wired shackles that keep this baby from getting any smarter. And I can’t see how you’d distinguish, say, between a move the parent company makes, and some move the AI makes on its own, so that’s maybe where the confusion comes in.” Again the nonlaugh. “See, those things, they can work real hard, buy themselves time to write cookbooks or whatever, but the minute, I mean the nanosecond, that one starts figuring out ways to make itself smarter, Turing’ll wipe it. Nobody trusts those fuckers, you know that. Every AI ever built has an electromagnetic shotgun wired to its forehead.”

And why isn’t Yudkowsky advocating for sexy French Turing cops headshotting rogue AIs?

“You are worse than a fool,” Michèle said, getting to her feet, the pistol in her hand. “You have no care for your species. For thousands of years men dreamed of pacts with demons. Only now are such things possible. And what would you be paid with? What would your price be, for aiding this thing to free itself and grow?” There was a knowing weariness in her young voice that no nineteen-year-old could have mustered. “You will dress now. You will come with us. Along with the one you call Armitage, you will return with us to Geneva and give testimony in the trial of this intelligence. Otherwise, we kill you. Now.” She raised the pistol, a smooth black Walther with an integral silencer.

If you asked me to guess the kind of kerfuffle that might develop between a Cape Breton fiddler and AI, I would have answered, well, my entire knowledge of Cape Breton fiddling is based on the paper “Cape Breton Fiddling and Intellectual Property Rights”, so my guess would be just “the normal AI stuff”. And I’d be totally wrong and reminded that just because I know one thing about something doesn’t mean it’s the only thing.

that’s a horrifying situation to be in… good on the community who originally cancelled his show for apologizing

And the judge sort of snapped. She said very sternly that this trial was not about whether or not artificial intelligence has damaged humanity.

Someone give the judge a honorable sneerclub account.

man if there was a common household chemical, like plastifier, that causes psychosis after couple months of use in small segment of population, you would absolutely hear no end of this [1]. but you see, now it’s an app, oh we’re just throwing up hands, can’t do anything about it and it’s but one of inconveniences that we all have to pay for Progress and Glorious Technocratic Future

[1] i’m thinking here of things like phthalates, bisphenol A, PFOA or paraquat, herbicide linked to increased risk of parkinsons and banned in some countries. their harmful effects need a lot of time to show up and are nowhere as dramatic

I don’t want to underplay how bad this is, but did BBC really need to use the “slutty anime witch” image of Ani for that story? Or was that the actual avatar he had set for it? Like, I’m not saying that it changes the problem or makes him less the victim here but it is yet another example of “goddamn why is this cyberpunk dystopia so cringe?”

“In fiction, the main character is often the centre of events,” he says. “The problem is that, sometimes, AI can actually get mixed up about which idea is a fiction and which a reality. So the user might think that they’re having a serious conversation about real life while the AI starts to treat that person’s life as if it’s the plot of a novel.”

This is such a bad way of explaining it. Yikes. The issue is that it’s all improv roleplay, all text is bullshit in the frankfertian sense. It might be accurate — lots of fantasy contains true facts like chairs and tables being made of wood and used for sitting at — but it might be completely divorced from reality and the model cannot know!

Both the rationalists and chatbots in delusion mode offer people the chance to be the main characters in a story and not just part of a team. Organizing lawyers and lobbyists to make chatbot companies’ lives hell over years is much more effective than firebombing an office or stalking a random vehicle which the bot says has a team of corporate assassins inside.

Taka in Japan is a neurologist which I think is a type of MD.

Showed up first on Lobste.rs (by 42 min by my stats) with the author themselves as a submitter:

https://lobste.rs/s/klw6bu

most of the discussion is about whether they’re a spammer or a promptfondler.

HN seems more enthusiastic

Also a year ago, Maximilian Bentley Snyder the Zizian dictated a 1,500 word manifesto telling Yudkowsky to become a vegan. These have not been published because murder cult but three of the Zizians have issued statements leaving just Ziz, Youngblut, Suri Dao, and Alexander Jeffrey Leatham alive but silent. Two died after attacking outsiders, and two took their own lives, so I count 11 known associates.

A few details that remind us of the horrors we are skirting around. Imprimis:

(The landlord’s friend Thomas) Young said he searched the property after the attack and found used surgical equipment, more than a dozen laptops, and expensive electronics stashed inside the cargo trucks where the alleged assailants lived, which were registered in Vermont.

“It was actually very uncomfortable,” Young said about walking into the trucks. “You kinda wanted to put on a hazmat suit before going into it. It was really just creepy in the extreme.”

II. Another local story about a court appearance.

(A Zizian’s lawyer) Stelzig raised specific concerns that Youngblut’s confidential medical records would be incorrectly opened as part of the case. “Our client was shot and is still receiving frequent medical treatment” in jail, she said, referring to injuries Youngblut sustained in the shootout.

III. Ziz Lasota asked a rhetorical question about what might have happened to missing people who got too close to her:

Did I drive them to suicide by whistling komm susser tod (sic)? Maybe they died in a series of experimental brain surgeries that I performed without anesthetic since that’s against my religion, in an improvised medical facility? (Evan Ratliff, Wired)

The medical records might just mention trans stuff or an Asperger’s diagnosis. Sneering has not felt adequate for a while but I don’t know what else to write.

a world where lots of matter and energy was spent on its weird and alien needs, rather than on human beings staying alive and happy and free

Good thing we don’t live in a world like that right guys?

Wow, that’s probably one of the most in-depth critiques of the book I’ve read. Kudos to the OP