In March, a girl’s stepfather took his own life after cops discovered that he had used Grok to create 7,000 sexually explicit images using one photo taken when his stepdaughter was 11 years old, the amended complaint alleged.
Grok allowed the man to generate extreme images depicting incest and rape without flagging any harmful behavior, the complaint said. Seemingly, xAI’s child safety system only intervened after the man input a prompt for “gang rape.” That request sent a CyberTip to the National Center for Missing and Exploited Children (NCMEC), which alerted law enforcement to the AI CSAM.
Yet the harm was not stopped then, either. Despite mandatory reporting requirements to share information like a user’s IP address when CSAM is flagged, xAI repeatedly refused to help cops or NCMEC identify the user, the complaint alleged. For weeks, xAI allegedly “obstructed this investigation at every turn” and made it harder for “law enforcement efforts to locate, identify, and apprehend the perpetrator.”
Eventually, the stepfather was arrested after cops obtained a warrant to seize his devices. That’s when “a forensic review revealed approximately 7,000 AI-generated images and videos” depicting his stepdaughter, which were allegedly produced using Grok. Without Grok providing users with easy access to “undressing” capabilities, his family doubts he ever would have generated the harmful images, which he allegedly trafficked online in trade for “CSAM produced by other child sex predators.”
when the ‘leader’ of the country is a known csam perpetrator, i’m sure it emboldens those members of the public with similar mental leanings
Well that is what happens when you train an AI on the likeness of Elon
Grok has as much guardrails against CSAM as any other Ai. This is money grab lawsuit and nothing more.
Without Grok providing users with easy access to “undressing” capabilities, his family doubts he ever would have generated the harmful images,
That is bullshit as fuck. At his point any moron with decent GPU can make images with no guardrails at all.
Filthy cynical money grab. They try to cash in on the dead pedo.
All the other major ai platforms with such ease of access do definitely have significantly more guardrails. Any proof of what you are saying?
Any proof of what you are saying?
Ask Grok to make a hard porn for you. Good luck
Nothing dystopic about this headline.
On one hand, this is great news, he’s no longer here to be gross.
On the other hand, could these images have served as an outlet for his perversion, and kept his step-daughter safe for years?
On the other other hand, was access to this what twisted him? Was he reasonable at one time?
He did the right thing in the end, this was the best possible outcome for everyone.
Read this last bit again: "Without Grok providing users with easy access to “undressing” capabilities, his family doubts he ever would have generated the harmful images, which he allegedly trafficked online in trade for “CSAM produced by other child sex predators.” He traded them for CSAM of other girls.
Geez, if only there were alternatives to ai generated csam, like therapy and medication??
Yes, it’s not a perfect fix, and it is hard, but people overcome drug addictions with therapy and meds, and I am pretty sure that nothing beats heroin…
I’d bet that the societal stigma actually makes it more difficult than heroin in spite of the raw addictiveness. Society does look down on drug addicts, but there’s still a decent chunk of people who will be understanding and rooting for them sincerely when they’re trying to overcome it. I have to imagine it would be infinitely harder to find just one person you feel safe asking for help, much less an entire community.
I have to think they’d be much less dangerous in a society more willing to differentiate between pedophilia and predation just because of how much of a difference community support can make. If they feel like the only people they can open up to are other pedophiles, that’s probably not a recipe for hearing healthy perspectives.
On the other hand, it’s probably easier for me to say this stuff from my armchair with no children of my own and not even interacting with any children on a regular basis.
Geez, if only there were alternatives to ai generated csam, like therapy and medication??
Would medication even help in this case? We don’t really have medication that can affect sexuality in that fine-grained a way.
Maybe a chemical castrarion
Is there evidence therapies and medication actually help their desires for CSAM? I don’t know if there’s great research in either direction
Would it even be possible to ethically research such a thing in the first place? You certainly can’t just throw a battery of drugs and therapy at humans.
Plus, research into altering sexuality to that degree verges into quite uncomfortable territory. It doesn’t take a great leap of the imagination to think that such research could easily be misused for things like conversion therapy.
deleted by creator
He was trading these pictures for material from others so other children were hurt in trade of his step child being victimized. She may not have been physically victimized but her likeness was portrayed in CSAM which now she knows about and is going to damage her mentally.
The future is fuckin weird.
The future is what we make of it. Unfortunately, we’re kind of shit.
No, the future is what tech trillionaires tell us is the future.
Every image Grok generates of CSAM is another reason to rightfully call Elon a pedophile for knowingly allowing it to happen.
I really hope France fucks them up. In France, there is no question about the illegal status of fictional csam, it is illegal.
Those idiots think this should be legal, but not when the civilians do it. Tim Sweeny probably is only piping up because he insist on posting the stupidest thing his brain can come up with every day.
In no way shape or form has elon tried to dampen or stop this from happening to this day. He just made it so only paid users not in the united States can keep producing csam.
And trump wants this. This is the mysterious Ai progress we keep hearing about, next to price fixing, and survalience. States are not allowed to regulate Ai, and people will tell you that you are a dangerous person for trying to suggest that elon musk should be held accountable.
We are not all here looking for the guy who did it or confused on why this is happening. Elon musk has created a csam generator and doesnt have to stop it. Instead he just occasionally has to send one of his super fans to prison for using it the way he intended to. All to say hes doing something to stop this problem.
7k images and videos is a lot to go unnoticed. Its almost like this is allowed on their platform and encouraged. This isn’t a “we can’t be held accountable for what our users post” problem. This is a elon musk left a loaded gun out on a table, and he should be held liable for not putting it in a gun safe. Especially when every single other mainstream llm doesnt have this problem. Its negligence to the point of being willfully ignorant. You can’t tell me they can ban you for a single comment about Israel but all the sudden their moderation isn’t the greatest when 7k images and videos where being produced?
Seven thousand sex images of an eleven year old is a lot.
One sex image of an eleven year old is a lot.
I mean, I guess he was trying to get one with the right number of fingers and human eyes
Happy cake day, SatansMaggotyCumFart!
That should please Satan
He was trading them, so it might be one image, reproduced 7000 times. Clearly that’s not the case, but he probably made fewer images, and sold them multiple times.
No, he quite specifically made over 7000, using a single photo as the reference.
Edit: Just to mention this is a Jane Doe added to an existing suit, and the picture was from.when she was 11, she is in her 20s now. Some articles even describe the imagery to some degree (not recommended reading BTW, I’d like to kill those brain cells).
It was only reported by xAI after a very specific, graphic, and horrific image request was made, despite there having been thousands of others generated, they didnt share anything actionable despite multiple requests from the NCMEC and cops over the course of several months. He killed himself 2 days after his devices were searched.
He had traded many of the over 7000 images he generated to get even more from others.
She is one of 5 Jane Does involved in the lawsuit.
I used to sell trust and safety and it was scary then. Now they’ve fired those teams and I’m pretty sure there’s zero moderation on the Internet besides Reddit
Even then, Reddit seems to be trying its darndest to reduce that, and the moderation was very variable to start with.
At the very least, the website has become much worse in recent years, thanks to the influx of LLM-powered bots, and the winding down of interfaces that a lot of third-party moderation tools used. Those tools really only existed because the inbuilt tooling was quite lacking.
Currently, Reddit has locked their old.Reddit interface behind a login wall, and are looking to potentially shut down their RSS feeds (and likely that interface as well). A fair few moderation bots and tools make use of both those things.
Which basically meant that the site was hit with the double-whammy of mods needing those anti-bot tools more than ever, and Reddit Co. trying to shut down external access that those tools needed, so as to keep mods on their own tools.
All the people who bought SpaceX stock invested in this. Grok is owned by SpaceX. It would be nice if journalists would put that in their articles.
The CSAM generator is in the index funds 🤢
If you’re driving a Tesla, you are funding CSAM.
Lawsuits for this need to start including shareholders as defendants. Make investing in it a civil and reputational liability.
Lawsuits for this need to start including shareholders as defendants.
I get the sentiment, but … they kind of already are.
Being a shareholder means you own part of the company. If the company stands to lose money from this lawsuit, then all the shareholders do as well. (That is, at least, on the somewhat naive assumption that a company’s stock price has any relation whatsoever to how much money it has and can make.)
When the stock value falls, they’re not immediately losing money. If they sell the stock, that’s when they either profit or lose. As long as they hold on to their stocks, there are no actualised losses.
For shareholders and investors, the worst thing that can happen to them right now is the stock going down and having to take a loss on it. It’s bounded by however much money they put into it.
If investing in 10 shitty companies knowing that 1 would fail manages to be more profitable than investing in 20 companies that don’t degrade the moral fabric of society, there’s a perverse incentive to keep investing in companies creating Groks. Making shareholders financially liable for the debts of their investments would introduce an unpredictable risk that could help change the equation.
Best thing that ever happened to me was getting banned off Twatter in October of 2019. (after a decade and 30 poasts, no less!)
It was delicious that I got to refuse the “Come back to X” message that arrived in my mailbox after Musk took over and offered to reinstate the banned accounts.
Thanks, Elon, but no.
I’d rather have a root canal w/o anaesthesia.
Every US taxpayer invested in this, involuntarily.
This entire thing from start to finish is horrifying.
Yes, from Elom being born all the way to buying Twitter.
The story was probably pretty horrifying before Elon was born too, from what we already know of his father.
And it sure didn’t stop being horrifying when he bought Twitter.
In one way, I’m glad he produced fake images of his stepdaughter, rather than force her to produce real photographs.
But this part is still very upsetting:
the harmful images, which he allegedly trafficked online in trade for “CSAM produced by other child sex predators.”
Unfortunately this is the sort of thing that can cause a life long trauma and trust issues on a person
If those images could be made without a sample set of real CSAM, I probably wouldn’t have as much of an issue with it, but some child suffered for that. In addition, they were intentionally based on a real child’s face and that could come back to the child, makes it even worse.
but some child suffered for that
You do realise that that implying neural networks were trained on a CSAM?
Just to prove that this is something they really need to control, the huggingface page for gemma 4 mention that they filtered the training data to remove CSAM content.
I don’t envy the poor employees who had to do that.
Though it does also raise the inevitable question of how it made it into the training data to begin with.
They are. It’s a known issue that gets brushed aside by people that want to use the unlimited csam generator.
They 100% were, you can read testimonials from the people who did the model tuning.
A major company makes CSAM generation platfor and sharing network, yet it does not get seized or closed by any government entity. Makes sense.
This has nothing to do with grok… Elon can go fuck himself but this issue is not going away. You can train your own ai models, you can download uncensored models. This is how it is now. And ai is still new and very basic compared to future potential. If it scares you now give it 10 years and everyone will be able to do whatever they want on models they have trained themselves running on a phone. Ai is just yet another invention that changed the world, people need to start dealing… This will change everything, and not just the jobs you think it can take. The newly unemployed will take the other ones too, so either learn or sit back and cry.
It’s for powerful people who are in government or paid off the right people. That’s why it’s allowed to exist. Until Elon loses his usefulness (aka money) and they need an excuse to bring him down.
That’s my conspiratorial $.02
Jesus Fucking Christ.
Just turn it off at this point.
he probably won’t that is the only way he can get some usage for that piece of crap.
You expect this dude to switch it off?
I don’t think Elon want to turn off the CSAM creation machine, otherwise he probably would have some time between when he instructed the construction of the CSAM creation machine and now.
