Hey folks!
I made a short post last night explaining why image uploads had been disabled. This was in the middle of the night for me, so I did not have time to go into a lot of detail, but I’m writing a more detailed post now to clear up where we are now and where we plan to go.
What’s the problem?
As shared by the lemmy.world team, over the past few days, some people have been spamming one of their communities with CSAM images. Lemmy has been attacked in various ways before, but this is clearly on a whole new level of depravity, as it’s first and foremost an attack on actual victims of child abuse, in addition to being an attack on the users and admins on Lemmy.
What’s the solution?
I am putting together a plan, both for the short term and for the longer term, to combat and prevent such content from ever reaching lemm.ee servers.
For the immediate future, I am taking the following steps:
1) Image uploads are completely disabled for all users
This is a drastic measure, and I am aware that it’s the opposite of what many of our users have been hoping, but at the moment, we simply don’t have the necessary tools to safely handle uploaded images.
2) All images which have federated in from other instances will be deleted from our servers, without any exception
At this point, we have millions of such images, and I am planning to just indiscriminately purge all of them. Posts from other instances will not be broken after the deletion, the deleted images will simply be loaded directly from other instances.
3) I will apply a small patch to the Lemmy backend running on lemm.ee to prevent images from other instances from being downloaded to our servers
Lemmy has always loaded some images directly from other servers, while saving other images locally to serve directly. I am eliminating the second option for the time being, forcing all images uploaded on external instances to always be loaded from those servers. This will somewhat increase the amount of servers which users will fetch images from when opening lemm.ee, which certainly has downsides, but I believe this is preferable to opening up our servers to potentially illegal content.
For the longer term, I have some further ideas:
4) Invite-based registrations
I believe that one of the best ways to effectively combat spam and malicious users is to implement an invite system on Lemmy. I have wanted to work on such a system ever since I first set up this instance, but real life and other things have been getting in the way, so I haven’t had a chance. However, with the current situation, I believe this feature is more important then ever, and I’m very hopeful I will be able to make time to work on it very soon.
My idea would be to grant our users a few invites, which would replenish every month if used. An invite will be required to sign up on lemm.ee after that point. The system will keep track of the invite hierarchy, and in extreme cases (such as spambot sign-ups), inviters may be held responsible for rule breaking users they have invited.
While this will certainly create a barrier of entry to signing up on lemm.ee, we are already one of the biggest instances, and I think at this point, such a barrier will do more good than harm.
5) Account requirements for specific activities
This is something that many admins and mods have been discussing for a while now, and I believe it would be an important feature for lemm.ee as well. Essentially, I would like to limit certain activities to users which meet specific requirements (maybe account age, amount of comments, etc). These activities might include things like image uploads, community creation, perhaps even private messages.
This could in theory limit creation of new accounts just to break rules (or laws).
6) Automated ML based NSFW scanning for all uploaded images
I think it makes sense to apply automatic scanning on all images before we save them on our servers, and if it’s flagged as NSFW, then we don’t accept the upload. While machine learning is not 100% accurate and will produce false positives, I believe this is a trade-off that we simply need to accept at this point. Not only will this help against any potential CSAM, it will also help us better enforce our “no pornography” rule.
This would potentially also allow us to resume caching images from other instances, which will improve both performance and privacy on lemm.ee.
With all of the above in place, I believe we will be able to re-enable image uploads with a much higher degree of safety. Of course, most of these ideas come with some significant downsides, but please keep in mind that users posting CSAM present an existential threat to Lemmy (in addition to just being absolutely morally disgusting and actively harmful to the victims of the abuse). If the choice is between having a Lemmy instance with some restrictions, or not having a Lemmy instance at all, then I think the restrictions are the better option.
I also would appreciate your patience in this matter, as all of the long term plans require additional development, and while this is currently a high priority issue for all Lemmy admins, we are all still volunteers and do not have the freedom to dedicate huge amounts of hours to working on new features.
As always, your feedback and thoughts are appreciated, so please feel free to leave a comment if you disagree with any of the plans or if you have any suggestions on how to improve them.
Personally I say just leave hosting of images to dedicated sites for that purpose. Your efforts are better left to dealing with how to render them. That being said, I use to be in charge of managing abuse on a site that has an average of 20 million posts a month (seriously).
The way I essentially defeated these kinds of attacks was with an image scanning service. It scans for anything NSFW and blocks it. Sometimes things would make it through but once an admin flagged it we could use that to block the users IP and account. It’s not cheap but the volume is also not huge yet for lemm.ee so it might not be too bad.
This is my opinion also. Reddit turned to shit around the time they started self-hosting. Imgur only exists because people needed a place to host reddit images.
Is there a fediverse instance of Imgur?
I’ve seen people link to uploads on Pixelfed, though this is probably not the intended use case.
No, but there’s nothing stopping you from using direct links from imgur, in traditional fashion.
It’s a little bit convoluted, though. You have to post the image, then hover over and select “Get share links”, and then pick the option for BB code (forums). This has the [img] tags at the start and finish, but importantly it has the direct link to the image file. If you use this on lemmy then it will load in the instance, rather than directing to imgur itself.
That solution sounds overly complicated.
That produces
https://i.imgur.com/4K8r2Bo.pngImgur is deleting images over a certain age posted anonymously. And they might continue to decrease the number of images they keep to try to be closer to profitability. So that will be bad for longevity of content.
reddit is the new imgur. I post stuff to my reddit profile grab the image link and post here. let spezz pay the bill for hosting our images.
This is brilliant. someone should make a way for us to provide login info to reddit that will just “login” and “post” an image to some random private sub, then return the url. A browser plugin would probably do this easily.
I would love it!
Personally I say just leave hosting of images to dedicated sites for that purpose.
They aren’t profitable, so they’ll eventually go down. If no one is looking at their site, why keep it going just to serve other sights?
The same can be said about Lemmy.
But if the instance goes down, no one will care that the images in the posts are also gone.
Can we get image upload back?
I would appreciate having it for avatars and banners. I was going to change mine and couldn’t figure out why even the smallest time size was “too large. Would it be possible to have an avatar be a link for a hosted picture somewhere else or is that too much of a security concern?
For step 6 - are you aware of the tooling the admin at dbzero has built to automate the scanning of images in Lemmy instances? It looks pretty promising.
Yep, I’ve already tested it and it’s one of the options I am considering implementing for lemm.ee as well.
it’s been 4 months-- any update on this?
You forgot getting the authorities involved when somebody does upload csam
The Lemmy.world team is getting some authorities involved already for this particular case. I am definitely in favor of notifying law enforcement or revelant organizations, and if anybody tries to use lemm.ee to spread such things, I will definitely be involving my local authorities as well.
getting the authorities involved
How do you imagine that playing out? This isn’t some paedophile ring trading openly, this is people using CSAM as an attack vector. Getting over-enthusiastic police involved is exactly their goal, and will likely do very little to help the victims in the CSAM itself.
Yes, authorities should be notified and the material provided to the relevant agencies for examination. However that isn’t truly the focus of what’s happening here. There is no immediate threat to children with this attack.
I think the images should never be cached from other instances in the first place, that is a huge oversight in pictrs since not only does it have the potential to cache unwanted content but also causes the images hosted to rapidly accumulate which isn’t ideal as it increases storage requirements which is unfair to people who want to self-host a personal instance. Hosting a personal instance should not have monstrous storage requirements or serious liability risk due to caching all images automatically, it should only cache what is uploaded to the Instance like profiles and banners, and posts that include images from the Instance.
I have reservations about allowing fully-invite based registrations on lemmy instances. While I do think it might be good to have invites as a way for users to skip filling out an application I don’t really like the idea of requiring them like Tildes does, makes it feel like an elitist exclusive club of sorts having to beg for an invite from users. I don’t think it should be an alternative to application-based registration, but rather a supplement to it, if someone can get an invite from users that’s great but if not they should still be able to write an application to join, this could be extensive and also lower priority since you could get invites but should still be an option available.
Account requirements really depends on what they are and what they restrict (also who on the instance is allowed to impose restrictions). For example on instances with downvotes enabled I think score/upvote requirements are a bad idea since it essentially means that people who disagree are locked out, like on Reddit with karma restrictions, I do not support this, it creates an echo-chamber where unpopular opinions. It’ll also lead to upvote farming if there are negatives due to having a lower score.
Comment or post requirements would just lead to post or comment farming similar to vote farming, though it’s not as bad as score-requirements since people making posts and comments naturally (whether they are liked or not) can’t be taken away by other people based on opinions (only if they break the rules and get posts removed, which isn’t even remotely similar since they broke the rules).
Limiting image uploading is a fair requirement in my opinion since uploads can be particularly harmful if the uploads are malicious, and also uploads aren’t really needed since people can externally host almost all their images without the need for uploads.
When it comes to DMs and restrictions around them I feel like that should be up to individual users to decide to allow private communication from certain users or not, or even to allow DMs at all, this shouldn’t be something globally applied to people, maybe it could be a default in User settings and have a requirement set by the Admins but people should be able to turn it off if they don’t care or want to accept messages from new users, I know I certainly will, I hate being nannied when it comes to who’s allowed to send me messages, IMO Annoying or uncomfortable DMs are a fact of life and I prefer to deal with issues when they happen rather than block anyone who’s a new user that might want to talk to me, it’s one of the things I hated that Reddit does without giving me the option to opt out and receive messages from everyone.
I think having a Machine-Learning based system to identify Malicious images is actually a pretty good idea going forward, I know how some people feel about AI and Machine-Learning but I think it’s probably our best defense considering that none of us want to see it, it might have False positives but I’d rather than than to allow CSAM to live here. Ultimately the choice is have ML scanning or Disable pictrs here, I think ML is the better option because people are going to want to have Avatars and without pictrs that isn’t possible (unless Lemmy adds support to the UI for externally hosted Avatars and Banners).
You need to update the site to reflect these changes - I was trying to upload a banner and got the incorrect error “image too large”
Going from completely open to invite-based is what I’d call overcompensation (you’re overestimating how willing people will be to invite)
I’m going to be a part of an invite only community?! Of course, given the circumstances, this is pretty fucked. But I feel kinda fancy right now.
Thanks for all you do on lemm.ee
Forums have existed on the internet forever and and have already dealt with this thousands of times previously. You don’t need to overthink it or reinvent the wheel. It didn’t stop forums existing very comfortably in the past and isn’t an issue that should be that different to deal with today.
Simply limit image uploads to a certain account age threshold and karma threshold and you will eliminate 99% of the ability to abuse this.
Forums have existed on the internet forever and and have already dealt with this thousands of times previously
The main difference is that forums aren’t federated. On Lemmy you not only need to keep in check internal users, but also external instances, and as everyone can host one, federation ads extra complexity
I like almost everything on this plan, except for the last 2 items. The account requirements for “extra activities” best be chosen carefully as to not encourage the good old “karma farming” that we got away from in leaving Reddit.
And the ML thing for recognizing NSFW is also something to be carefully considered. Too strict and it gets annoying with false positives, it can restrict posting actual content, and too lax won’t make a difference for the people actually looking to circumvent it. I think a “vetting” system like the previous item could be better in the long run, in only letting “trusted” people upload content.I will apply a small patch to the Lemmy backend running on lemm.ee to prevent images from other instances from being downloaded to our servers
If possible, could you tell others how to apply this patch to their own server?
See this comment: https://lemm.ee/comment/2901943
How would one go about implementing this change on their server?
so how do I upload an avatar?
It is now impossible to add an avatar or banner to profiles because the only way to do so through the UI is uploading to the instance. There’s no way to add an external URL. Just wanted to point that out in case it wasn’t intentional. Very understandable if that’s something we have to sacrifice for the time being.
Edit: I noticed that images will upload to the account’s home instance instead of the community’s home instance. This means that one workaround for the time being to change your lemm.ee community’s icon and banner is to create an account on another instance and then add that account as a moderator to your lemm.ee community. You can then use that external account to change the icon and banner of your lemm.ee community because images will be uploaded to whatever instance your account is on instead of lemm.ee.
Nor sure about links but avatars and banners were intentional because those would be images, and all image uploads are suspended for now.
Unfortunately it’s a side effect of disabling uploads, it is possible to have externally hosted avatars (field is just a URL linking to the image) but isn’t officially supported by lemmy-ui yet, hopefully it can be in the future though, I made an issue for it on the GitHub to hopefully get it supported (linked in my other comment).
IMO Lemmy shouldn’t have media uploading of any kind. Aside from the CSAM risk, it’s unsustainable and I think one of the reasons Reddit went to shit is by getting into the whole image/video/gif hosting.
Dozens of media hosts exist out there, and the mobile/web clients should focus instead on showing remote content better.
Got to be honest, having an invite based system and locking certian features behind age of accounts, karma, etc seems like the opposite of the freedom everyone promised me the Fediverse represented when we moved over.
I personally don’t really care about images and would prefer image uploads just stay deactivated and we operate as a text only forum but with open membership.
Leaving image uploads completely disabled would also be an option to fight this particular type of attack, but there are also other issues with open registrations. For example, while our sign-up captcha seems to be preventing automated registrations, we are still having to ban advertiser accounts almost daily. I think an invite system would really help to reduce sign-ups by any kind of users intending to abuse the system.
I’m all for an invite-based system, although we will need some way of combating ‘invite trees’, where one bad actor invites several others, who subsequently invite an exponentially increasing number. A reasonable delay on the invite allowance would go a long way, I think.
I have to say that an invite based signup system makes my toenails curl backwards. IMO this will let instances die out slowly. I didn’t know anyone using lemmy and just stumbled upon it. ppl like me wont ever be able to join an instance if it is invite only.
Don’t misunderstand me: I do understand how critical it is for the operators of instances to protect themselves. Lemmy is a rather young project and still needs better admin tools. However, there are some good discussions happening on GitHub. Untill the operators and admins have the tooling to protect themselves, I see disabling img upload as preferable. It also took reddit some time to allow uploading images, instead of linking them.
I very much agree, invite-only systems are a bad idea for this reason.
That’s a fair point. It’s not a situation with an easy answer.
I 100% agree! An invite-based system means that a new user has to find some way of contacting someone in order to request an invite. I think that only allowing X posts per day for e.g. the first week or 2 for new accounts would be a way to combat companies and spammers. Not allowing images or limiting image posts for new accounts, and using automated CSAM detection methods, which I understand are in the works, seems to be a good way to combat that problem.
