Downtime, bugs, and failures on any kind of service (email, web, XMPP, etc)

39 readers

3 users here now

Report problems with a service here that is unrelated to user-controlled client side software. Thus servers and web clients are fair game. This is a catch-all for situations not accommodated by the following related communities:

!bugs@sopuli.xyz - for bugs in third-party software apps
!isitdown@infosec.pub - for downtime status on Lemmy instances specifically (although they accept reports on any fedi node)
!assholedesign_web@infosec.pub - for deliberate enshitification of web services specifically

founded 6 months ago

MODERATORS

freedomPusher@sopuli.xyz

lemmyverse.net missing many communities - is it no longer crawling? (slrpnk.net)

submitted 7 hours ago by activistPnk@slrpnk.net to c/bugs_in_services@sopuli.xyz

3 comments fedilink hide all child comments

When querying for “legal” and “law” on lemmyverse.net, it’s starkly clear that many communities are missing from the database. I get far more results when querying on specific instances. So what’s the problem? Is it no longer crawling?

you are viewing a single comment's thread
view the rest of the comments

[–] FrostyTrichs@walledgarden.xyz 2 points 6 hours ago (1 children)

The crawler has been wonky for a week or more. If you check the recent posts at !trendingcommunities@feddit.nl you'll see the list has been very unpredictable lately as some of the bigger instances aren't being reported by the bot. Simultaneously there seems to be a federating bug that no one has solved yet. It was/is affecting .world communities federating to lemmy.ml and it's been causing communities on walledgarden and some other instances to fall in and out of sync with each other.

[–] activistPnk@slrpnk.net 2 points 5 hours ago (1 children)

The law communities I was expecting to find are old and on small instances. When I searched on just the domain portion of a small instance, Lemmyverse finds only like 10% of the communities on that node.

You would expect LV to not forget a community once it is in the DB. But it must also have a removal mechanism for communities that are deleted. It’s as if the removal detection is getting false positives for removals.

[–] FrostyTrichs@walledgarden.xyz 1 points 5 hours ago

I understand the confusion.

Maybe I should've been clearer that lemmyverse performance has been unreliable, not just the crawler. Entire instances, communities, and mixes of the two have disappeared and reappeared between scans. It also hasn't been reliably picking up changes to instances it already knows about.

TLDR lemmyverse has something wrong with it.