101
dfox
149d

Hey everyone,

During some backend improvements to the devRant infrastructure, some of our async queue processors (SQS) stopped working which caused many notifs to not go out/stop working. Unfortunately our alerting didn’t pick up on this since there were still queues being processed (just not specific ones) and some aspects of notifs working. Big apologies for this issue!

It is now resolved, and while very delayed, no notifications were lost and all were processed after the queue processors started up again. Sorry for the bulk notifs, but we wanted to make sure all that were supposed to go out went out.

Additional alerting will be put in place to prevent this from happening again.

Thanks for your patience!

Comments
  • 14
    I was wondering where the 100 notifs suddenly came from.
  • 6
    Thanks for the fix !
  • 9
    Yay. These notifs are like crack, man.
  • 14
    It's been a while since I've been bombed with 300 odd notifications in 1 go.

    Felt like a trip down nostalgia lane.
  • 10
    @C0D4 for real I saw two in a row from @-red and was like shit here we go again
  • 5
    @PrivateGER you are a rising star baby *wink wink*

    Thanks Dave for the fix.
  • 3
    To all the prophets of doom...

    It's alive!!!!!
  • 3
    He comes through! Thanks much man!
  • 2
    Ah thank you! I was worried.
  • 6
    150+ notifications
    Felt like old times :)

    As an aside, I’m surprised the mention notification uses the processing timestamp instead of the comment post time.
  • 4
    Even missed this notification in the wave. Must say though, only having a 50-ish of notifications feels weak 🙈
  • 1
    No doubt why I left dR for such time period. Now, everything in place. Thanks!
  • 1
    No worries I revisited a whole bunch of rants and things people said 😁
  • 3
    @ScriptCoded I only had one and it was because I replied to myself. Sad face
  • 0
    This is unacceptable. I demand a refund!
  • 0
    EVENTUAL CONSISTENCY FTW!!
Add Comment