this post was submitted on 03 Sep 2024
851 points (99.4% liked)

Programmer Humor

32561 readers
354 users here now

Post funny things about programming here! (Or just rant about your favourite programming language.)

Rules:

founded 5 years ago
MODERATORS
 
you are viewing a single comment's thread
view the rest of the comments
[โ€“] vortexsurfer@lemmy.world 2 points 2 months ago

Had a similar thing at work not long ago.

A newly deployed version of a component in our system was only partially working, and the failures seemed to be random. It's a distributed system, so the error could be in many places. After reading the logs for a while I realized that only some messages were coming through (via a message queue) to this component, which made no sense. The old version (on a different server) had been stopped, I had verified it myself days earlier.

Turns out that the server with the old version had been rebooted in the meantime, therefore the old component had started running again, and was listening to the same message queue! So it was fairly random which one actually received each message in the queue ๐Ÿ˜‚

Problem solved by stopping the old container again and removing it completely so it wouldn't start again at the next boot.