103
dfox
3y

Hey everyone,

During some backend improvements to the devRant infrastructure, some of our async queue processors (SQS) stopped working which caused many notifs to not go out/stop working. Unfortunately our alerting didn’t pick up on this since there were still queues being processed (just not specific ones) and some aspects of notifs working. Big apologies for this issue!

It is now resolved, and while very delayed, no notifications were lost and all were processed after the queue processors started up again. Sorry for the bulk notifs, but we wanted to make sure all that were supposed to go out went out.

Additional alerting will be put in place to prevent this from happening again.

Thanks for your patience!

Comments
Add Comment