this post was submitted on 19 Nov 2024

245 points (96.6% liked)

Lemmy.World Announcements

29084 readers

164 users here now

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news 🐘

Outages 🔥

https://status.lemmy.world

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to info@lemmy.world e-mail.

Report contact

DM https://lemmy.world/u/lwreport
Email report@lemmy.world (PGP Supported)

Donations 💗

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Join the team

founded 2 years ago

MODERATORS

ruud@lemmy.world

lwadmin@lemmy.world

lwCET@lemmy.world

jelloeater85@lemmy.world

Serinus@lemmy.world

lw_mod_notification@lemmy.world

MrKaplan@lemmy.world

245

Federation issues with lemmy.ml (lemmy.world)

submitted 5 days ago* (last edited 1 day ago) by lwadmin@lemmy.world to c/lemmyworld@lemmy.world

73 comments fedilink hide all child comments

We're aware of ongoing federation issues for activities being sent to us by lemmy.ml.

We're currently working on the issue, but we don't have an ETA right now.

Cloudflare is reporting 520 - Origin Error when lemmy.ml is trying to send us activities, but the requests don't seem to properly arrive on our proxy server. This is working fine for federation with all other instances so far, but we have seen a few more requests not related to activity sending that seem to occasionally report the same error.

~~Right now we're about 1.25 days behind lemmy.ml.~~

You can still manually resolve posts in lemmy.ml communities or comments by lemmy.ml users in our communities to make them show up here without waiting for federation, but this obviously is not something that will replace regular federation.

We'll update this post when there is any new information available.

Update 2024-11-19 17:19 UTC:

~~Federation is resumed and we're down to less than 5 hours lag, the remainder should be caught up soon.~~

The root cause is still not identified unfortunately.

Update 2024-11-23 00:24 UTC:

We've explored several different approaches to identify and/or mitigate the issue, which included replacing our primary load balancer with a new VM, updating HAproxy from the latest version packaged in Ubuntu 24.04 LTS to the latest upstream version, finding and removing a configuration option that may have prevented logging of certain errors, but we still haven't really made any progress other than ruling out various potential issues.

We're currently waiting for lemmy.ml admins to be available to reset federation failures at a time when we can start capturing some traffic to get more insights on the traffic that is hitting our load balancer, as the problem seems to be either between Cloudflare and our load balancer, or within the load balancer itself. Due to real life time constraints, we weren't able to find a suitable time this evening, we expect to be able to continue with this tomorrow during the day.

As of this update we're about 2.37 days behind lemmy.ml.

We are still not aware of similar issues on other instances.

top 50 comments

sorted by: hot top controversial new old

[–] trespasser69@lemmy.world 12 points 21 hours ago* (last edited 21 hours ago) (1 children)

Oh, come on. Again?

The Usual suspects, Network troubleshooting editon

Source: https://lemmy.world/post/22166289

[–] Olgratin_Magmatoe@lemmy.world 4 points 21 hours ago

FUCK active directory

[–] yokonzo@lemmy.world 4 points 18 hours ago (2 children)

There's actually been some issues with Reddthat as well, not seeing users ect

[–] ticoombs@reddthat.com 2 points 11 hours ago

I too would love to know what your experiencing (so I can fix it!)

[–] MrKaplan@lemmy.world 2 points 18 hours ago

do you have some more details about that?

[–] Lost_My_Mind@lemmy.world 40 points 1 day ago (3 children)

I mean.....if you wanted to defederare from lemmy.ml I'd be fine with that.

[–] taipan@lemmy.world 8 points 1 day ago* (last edited 1 day ago)

It's certainly an efficient way to resolve the problem.

[–] NateNate60@lemmy.world 4 points 1 day ago (2 children)

There's too much content on lemmy.ml to defederate. We'd lose like a quarter of all the content.

[–] Draconic_NEO@lemmy.world 7 points 21 hours ago

We would adapt, just like when Beehaw was a significant amount of the userbase on Lemmy and they cut us and sh.itjust.works off, we adapted and they got smaller. Lemmy.world is a bigger server than lemmy.ml. Only reason their communities are still so big is network effect. Which would be curbed by them being cut off. As I've said already, network effect is curbed by force, taking away a choice, not providing 7 more choices while leaving the original.

[–] Lost_My_Mind@lemmy.world 17 points 1 day ago (1 children)

We'd lose the .ml version of that content.

But I thought the whole point of federation was that if mods, or communities become problematic, you could always create your own. Then all the non-problematic people will move to that.

Is that not the very foundation of the concept of the fediverse?

[–] NateNate60@lemmy.world 9 points 1 day ago (1 children)

It's not wrong, but I don't regard Lemmy.ml as being problematic enough to defederate from. Their moderation practices are questionable and their user base is annoying but it's otherwise generally tolerable. People can block the instance if they don't want to see content from it.

[–] lurklurk@lemmy.world 3 points 20 hours ago

It would be nice to have a simple way to block the user base too

[–] IndustryStandard@lemmy.world -4 points 1 day ago

I wouldn't.

[–] MrKaplan@lemmy.world 76 points 5 days ago (6 children)

this comment section is not a place to rant about other instances

[–] EndlessApollo@lemmy.world 1 points 2 hours ago

I'll just post sentence instead of a rant then, defederate from .ml please :3

[–] cm0002@lemmy.world 7 points 18 hours ago

Ookie dookie

https://lemmy.world/post/22359447

[–] Psythik@lemmy.world 2 points 19 hours ago (2 children)

Please defederate from them.

[–] Glytch@lemmy.world -5 points 8 hours ago

Block the instance on your own client, not all of us are as closed minded as you are.

[+] Hamartia@lemmy.world -12 points 1 day ago (1 children)

The hypocracy of people exaggerating the scale of the problems and piling-on about other instances in their manic drive to turn this place into an echo chamber is depressing.

[–] auzy@lemmy.world 11 points 19 hours ago (1 children)

It's been far better the past few days without them.

Honestly some of the discussions I have with lemmy.ml people don't make sense in any capacity

[–] Hamartia@lemmy.world -3 points 16 hours ago (1 children)

I haven't really had that experience with anyone from .ml yet. There are a handful of actual tankies there but for the most part, from want I have seen, it's mostly various forms of far left opinion which no one should be afraid of.

Some people here do spend an unseemly amount of time complaining about them. Which is more toxic than anything they're usually complaining about.

[–] auzy@lemmy.world 6 points 15 hours ago (1 children)

I had interactions like this person: https://lemmy.world/comment/13434977

A lot of them seem to be like the crazy Facebook crowd (not all).

But that being said, unfortunately, as Lemmy grows, I'm seeing more of that in general where we slowly move away from a "scientific" oriented approach, to half-truths and biased information.

So you're probably right.

But yeah, people do complain a disproportionate amount about them (I feel like a lot of people screaming "tankie", are similar to the trump crowd people)..

[–] Hamartia@lemmy.world 1 points 4 hours ago

That whole argument was painful to read. By the end he was so frustrated he became very obnoxious. Not cool.

I didn't get the sense of someone deliberately sowing misinformation however. Being misinformed isn't a dealbreaker nor is a poor rhetorical style.

I think it is important to continue forward with users with a wide range of sources. None of us have a monopoly on the truth and all of our 'trusted' sources palter and decontextualise their facts to some degree.

I know it's probably an unrealistic expectation but I'd rather mods, in general, had higher standards for curating civility in discussions.

load more comments (2 replies)

[–] greencactus@lemmy.world 49 points 5 days ago (9 children)

Ah, I just today in the morning blocked Lemmy.ml. Seems to have been interesting timing :)

[–] Chozo@fedia.io 49 points 5 days ago (2 children)

So it's your fault!

load more comments (2 replies)

load more comments (8 replies)

[–] finitebanjo@lemmy.world 32 points 4 days ago

I can think of a solution.

load more comments