I've noticed a lot of broken images around the place and can't quite work out what's happening. It doesn't seem to be a caching issue, at least not on my end, as trying to visit the image URL directly returns a 404.

As noted in a post in the Melbourne daily thread a few weeks ago, there are also a lot of broken community icons about the place. And as I think was raised a while ago, thumbnails seem to be broken most of the time as well.

As an example, here's what the trending communities tab looks like:

The communities page is a little better, but it's about 50/50 as to whether the icon does or doesn't load:

Here's a list of my own posts with missing images I've noticed recently

That last image seems to be super broken at the time of writing. Instead of just returning a generic 404, it returns:
{"error":"unknown","message":"Request error: error sending request for url (http://pictrs:8080/image/original/b4df332b-82a9-4ec8-b115-ac1b1b1824b0.jpeg): connection error: Connection reset by peer (os error 104)"}

Additionally, this is how thumbnails on links have been previewing:

Right-clicking and opening in new tab, it seems to be pulling a random pixel from somewhere in the image and then enlarging it and proxying it (500% zoom):

Example URL

But then other URL thumbnails don't work at all, for instance this post on trending:

Which links to Wikipedia, but fails for whatever reason, then tries to load an image called "broken-image-fallback.png", and, ironically, can't even load that (leads to a 404 page).

And some thumbnails on working images don't work, for example this post in c/GardeningAustralia doesn't load - even if I click on the thumbnail (which should normally load the full size/original image), but does work if I click the image URL itself.

One more thing: this new image proxy thing is getting super annoying. I realise it's a Lemmy-side implementation issue, but a couple of sites (Imgur, for example), don't like all the image requests from a single server/IP and end up rate limiting us. This effectively means that Imgur is unusable and no images load.

I actually thought there might have been another mass image purge like last years' incident which required the purging of 48 hours of images, but there hasn't been any announcements about it, and the other broken images dotted around the joint makes me think Pictrs is on its deathbed.

Any ideas what's going on?

Sidenote: image uploads whether in posts or as embedded images within posts/comments seem to fail about half the time for me on both mobile and desktop. I thought this was an issue with Boost again, but it's happening to me on PC as well. On mobile, it usually either returns no error at all, or a 500. On PC, no UI errors are displayed, it just doesn't embed the image, but network tools tell me that something somewhere is returning a 500, likely the image uploader thingy.

I thought the recent patches were supposed to fix the slow federation, but we're still not getting comments or vote tallies through for over a week.

My web dunce question of the week is,

Why would the images url be linked to aussie.zone when the community is blahaj.zone and the poster is lemm.ee?

Is it as simple as the lemm.ee poster might have seen it on aussie.zone and cross posted, or copied over from there?

Edit,

So, just checked this post on the sync app, (using Jerboa for the post), and sync has recognised the image as a URL? I'm confused, i loaded it in Jerboa like i'd normally load a screenshot as an image. Anyway, sorry if the picture doesn't display.

I'm just trying to understand the lemmy system a little better, not trying to highlight a perceived problem.

I know there's been a few posts about this before, but it's been a month since the last one and it's still ongoing. It doesn't seem that any of the LW admins responded to Zag's post on their help community, and the last response from lodion/Nath I'm aware of was from 4 months ago when there were outright federation failures as opposed to just lengthy delays.

@Nothing4You@programming.dev posted a comment on the post from last month about the delays stating that it's an issue on our end as our server isn't keeping up. I'm not sure whether this is the case or not, and I'm not sure how to interpret the Grafana dashboard they linked to, but as it's a new reply on an old post, I wanted to note it.

Current federation delays seem to be around 7 days. It doesn't seem to be affecting posts themselves on Lemmy.world communities, but does affect all replies to them (even from users on other instances), and all upvotes on the posts. [Edit: on further investigation, this isn't the case. The current delays are at least 13 days, and this does actually affect posts too]

I don't want to sound too pushy, since the LW admins and Lodion/Nath are all volunteers, but I was hoping we might be able to get an update on what the cause is, and if it's an issue in Lemmy itself, if anybody's opened an issue on GitHub and the developers are aware.

(NB: I don't interact that much with LW, so all of my testing has been on the Boost for Lemmy community.)

cross-posted from: https://lemmy.ml/post/2222531

It's here!

Download Mlem for Lemmy on the App Store

Please note: it may take a couple minutes for it to be available in all regions. We just pressed the button.

Mlem for Lemmy is out of beta and live on the App Store!

Thank you to everybody who tested our app and submitted feedback, whether through TestFlight or a GitHub issue or a post on the community. Your insight has been an integral part of making this app what it is, and we look forward to working with you to make it even better in the months to come.

The past month and a half of hard work has been an amazing journey, but this is just the beginning. We hope you give it a download and join us as we strive to bring you the best mobile Lemmy experience we can.

Cheers,

The Mlem Group