Hi all,
If you’re just now signing in for the first time in 12+ hours, you may just now be finding out that Lemmy World and other instances where hijacked. The hijackers had the full abilities of hijacked user, mod, and admin accounts. At this time, I am only aware of instance defacing and URL redirections to have been done by the hijackers.
If you were not forced to sign back in this morning, contact your instance admin to verify mitigations were completed on your instance.
How?
This occurred due to an XSS attack in the recently added custom emojis. Instance admins should follow the issue tracker on the LemmyNet GitHub, as well as the Matrix Chat. Post-Incident Activity is still on-going.
Currently, it is likely that just your session cookie was stolen, with instance admins being targeted specifically by checking for navAdmin, an HTML element only instance admins had. I do not believe this to affect users across instances, but I have yet to confirm this.
What happens next?
As I am not the developers or affected instance admins, I cannot make any guarantees. However, here is what you’ll likely see:
- Post Incident investigation continues. This will include inspecting code, posts, websites, and more used by the hijackers. An official incident writeup may occur. You should expect the following from that report:
- Exactly what happened, when.
- The incident response that occurred from instance admins
- Information that might have helped resolve the issue sooner
- Any issues that prevented successful resolution
- What should have been done differently by admins
- What should be improved by developers
- What can be used to identify the next attack
- What tools are needed to identify that information
-
A CVE is created. This is an official alert of the issue, and notifies security experts (and enthusiasts), even those not using lemmy, about the issue.
-
A code security audit is done. This will likely just be casual reviews by technical lemmy users. However, I will be reaching out to the Mozilla Foundation and Cure53 as they recently did an audit of Mastodon. If there is interest in an external audit of lemmy and the costs are affordable, I’ll look into crowdfunding this cost.
You must log in or register to comment.
Is the sudden unannounced disappearance of Vlemmy related to this ? I had my main account there and it just disappeared
The most probable theory is that the server owner found illegal federated content on their server and decided to nuke the place entirely.
Dumb take. Lemmy.world devs simply said it’s not likely Threads will federate with Lemmy anytime soon anyways, and they’ll make a decision when there is actually a decision to be made.
removed by mod
I’ve made many comments saying why we should defederate with Threads but I still agree with the admins that there’s no point in doing a knee-jerk reaction on a threat that
A) won’t happen until at least a few months later
B) likely won’t federate with Lemmy anyways
C) isn’t actually a threat to Lemmy, but could pose a threat to microblogging websites like Mastodon
So yeah, waiting and seeing isn’t stupid.
“Wait and see” is fine.
Federating in the meantime is not.
Stop trying to dance around the heart of the issue.
Correct me if I’m wrong, but I don’t think Threads even has the ability to federate yet?
Federating in the meantime is not.
Dude, Threads isn’t federating with anything right now. That’s the point, we’re not federated so there’s no reason to make a decision on something that won’t happen yet.
I’m not sure you understand the takeaway from the admin post last night.
They haven’t made a decision, as it’s no issue as there is no federation to threads and won’t be for the forseeable future.
Making a fuss about it now is as usefull as making a fuss over the sun dying in several million years.
You’re missing the forest for the trees. You need to stop and listen to your fellow federation users.
https://lemmy.world/post/1274909
Conclusion:
From the points discussed above, the possible lack of moderation alone justifies considering defederation from Threads. However, it remains to be seen how Meta will handle moderation on such a large scale. Additionally, the inability of individuals to block an instance means we have to do what is best for the community.
Where does it say in the admin post that they are default federating with Threads?
While this is definitely a discussion to be had (I’ve created a few accounts on different instances). Posting that here is just adding onto the drama.
Not sure how you view it that way.
It’s just good advice for people who want to stay the hell away from a company like Meta that was willing to hand over private data to assist authorities in arresting a woman for exercising her reproductive rights.
Really not thrilled with the Lemmy.world users trying to downplay just how ridiculous the idea of federating with Meta is.
Is it possible to block .world entirely yet?
User-level instance blocking is not yet implemented. You could use an instance that is defederated from lemmy.world, such as beehaw.org.
Look, Lemmy.world takes the rational way where it says there is no use to panic. Let’s be rational and analyze the situation rationally and go from there.
I support Lemmy.world. Let’s see how the situation develops and let’s make a rational decision with cool heads.
“Ma’am… this is an Arby’s.”
Threads is not a Lemmy issue. Mastodon is where the concern lies.
Not the time or place for this.
I would strongly recommend making decisions for yourself.
I ran screaming from FB. Like, hair on fire running. That place is a roiling pot of piss. I would 100% not interact with ANY instance that has federated with them. They are a cancer that will infect this next big thing
So much drama
This post is weird. You’re typing like you’re in charge of things, but you’re apparently not.
It’s one thing to show some initiative, but you’re literally demanding a full report like the Lemmy devs work for you. You sound like someone who does this kind of thing for a living and felt the need to flex. Because otherwise, what the hell are you even doing?
Setting neurotically-specific demands for the developers makes sense if you represent a big instance or something, but you’re literally just a dude. You could have framed this entire post in a different way and gotten away with it. Right now, it’s creepy to anybody who actually reads the entire thing.
These aren’t demands, but I can definitely see how they can come off that way. These are industry standard post cybersecurity incident review questions by defined by NIST (NIST SP 800-61 Rev 2 Section 3.4.1) slightly rephrased.
If there is interest in an external audit of lemmy and the costs are affordable, I’ll look into crowdfunding this cost.
It could get VERY, VERY expensive… depends on code complexity.
Yeah, specifically why I mentioned “affordable”.
Agreed. It might be hard to swing right now, but imo this is going to be a crucial step moving forward.
Was lemmy.ml affected ?
XSS, Seriously?
Yeah it’ll be hard to regain my trust after this one. I mean I’ll still use Lemmy but for now I’ll assume mine or any other account could be hacked at any time and act accordingly. This is a really amateur mistake even by FOSS standards.
To add context to this. What I’ve been told is that a community running on a lemmy fork with 5 digit users had used this code for a while and backported(?) the code upstream when they federated back. I guessing there was an assumption of safety as they had been using the custom emojis code for quite a while without it being exploited.
How do I contact my instance admin?
I went ahead and logged out and back in on my own, but I was not prompted to do so by Lemmy and I don’t know enough about this stuff to say whether or not doing it on my own would’ve helped. I’m still pretty overwhelmed trying to figure this site out, so I apologize if this is a dumb question, just want to make sure I’m doing what I can to protect my own privacy.
I don’t want to fall for the millennial version of a Nigerian prince scam, lol.
XSS, classic
Thanks for your hard work, fuck the trolls that always have to poke holes in shit
Honestly, I see it as a win.
The people that did this didn’t really act out in a coordinated attack. They were just kind of playing around, redirecting to lemonparty, changing page elements.
It could have been a lot worse. The site could have been redirecting to malicious websites, downloading trojans, doing a lot of bad things. Instead, we got direct attention to the security vulnerabilities in question, and they’re being worked on and patched out relatively quickly. Helps that a lot of those on these communities are focused in programming and cybersecurity.
That’s fair, I was a little harsh. Not all hackers are looking to cause damage!
Agree completely. In the grand scheme of things the damage that appears to have happened here is small potatoes, but it brought attention to the vulnerability so it was patched quickly. Going forward now, the authors and contributors to the project might be a bit more focused on hardening the software against these types of vulnerabilities. Pen testing is invaluable on wide user-base internet accessible platforms like this because it makes better, more secure software. Unfortunately this breech wasn’t under the “ethical pen testing” umbrella but it sure as hell brought the vulnerability to the mindshare of everyone with a stake in it, so I view it as a net win.
Yes, exactly. This could’ve been way worse.
thanks for the update
also, I cannot properly login into my lemmy.world account anymore. username/password work, but when I try to upvote it tells me i gotta be logged in. Tried apps and web.
I had the same problem on the liftoff app. Had to delete the account then sign back in and everything worked perfectly again. Personally I also changed my password because it made me feel better.
For liftoff, you can just add the same .world account again and it’ll work, it worked for me.
Clear cache of the app
Thank you so much! I had the same problem but didn’t think of clearing the cache…
Cheers. Happy to help
For me it was cache and storage. Just cache didn’t work. (Using Liftoff)
I had to clear the app cache and local storage before it would work. Cache alone didn’t work.
Try clearing cookies (or app cache, on app), that will most likely resolve the issue.
sometimes just logging in again works. if you have it set to automatically log in, that happens sometimes.
The way the hack was utilized is honest very creative and interesting; either way, if all big communities could crowdsource money for security audits, I believe that it could help prevent something like this in the future.
Edit for clarification: I am NOT talking about how the exploit itself was created or achieved, I am talking about the act of having the power they did and just doing shock crap.
HTML injection / XSS vulnerabilities tend to be a sign of amateur hour to be honest. Made me a bit worried that I’m hitching my reddit escape wagon to the wrong technology. But sounds like this was due to instance-specific customizations rather than the core Lemmy tech, so hopefully we’re still on solid ground.
The way the hack was utilized is honest very creative and interesting;
That’s often the case with exploits.
Your not wrong, however, most of the time when these events happen it’s meant for something more insidious than lemon party.
Sure. Not sure how that’s relevant though?
In general, finding an exploit requires looking for little tiny details that could exist in, really, any area of a given system; looking for a bug, and then exploiting that bug by understanding how input data can be used to create a deterministic chain of events.
This almost always requires thinking outside of the box.
There are people who are also paid to find these before malicious actors do.
It’s always going to be creative in some way, at least in the beginning.
It’s like when people first discover Quake’s fast inverse square root. Sure, the first time around it seems genius. In reality, code like that is actually everywhere, and there is a somewhat trivial aspect to optimizing those kinds of problems.
deleted by creator
So if you dont have to resign-in?
Timception you should contact the admin for your lemmy server. If you didn’t need to log back in, your server likely was not fixed. This means someone could hack it and steal the session cookies to impersonate server members.
I was able to scroll through content, but had to relogin to upvote. However, today I was forced to re-login since I entered the Memmy app. So I guess, all good? Thx for the tips.
No problems from me
I haven’t been able to change my password on Lemmy.world. When I click save, nothing happens and the password doesn’t update.
That’s probably something someone wants to look into.
had a login/not logged in loop for a while yesterday. reset lemmy.world site data seems to have fixed
Currently, it is likely that just your session cookie was stolen, with instance admins being targeted specifically by checking for navAdmin, an HTML element only instance admins had. I do not believe this to affect users across instances, but I have yet to confirm this.
Probably because the hackers used some http request to get the data and didn’t want to wade through thousands of rows of JWT strings.
