17
tytho
8y

I accidentally started a reindex on a collection that had 14 million records in the middle of the day. Caused an outage in a major portion of our applications for about 3 hours. Worst thing was that once I pressed enter, I realized that it was for the production database, and not the staging database like I intended. I immediately went to go tell the dev ops lead, and he basically said, "whelp, let's just sit back and watch the world burn. Not much we can do about it"

Comments
  • 3
    I've found such experiences to be quite interesting, you know like watching a medical drama on tv and seeing them do open chest surgery/something "graphic"(? idk if thats the right word)

    You grimace, you feel a bit squeamish. It ain't a pretty sight, but you can't look away either. Dat morbid curiosity man
Add Comment