27
fyzrn
7y

The first time I caused a massive error on production.

The good news was the site didn't go completely down. The bad news, however, was that it went down for 60% of our users, and because it's only partial, it got detected only after about two hours.

Everyone halted what they were doing to help investigate the issue. When it turned out that my latest commit caused the error, I was told to fix it... with the CTO and senior software architects watching.

It all happened because I deleted one too many line, an if statement, making the accompanying else statement a complete nonsense. It was a corner case code unforeseen by the QA guy.

The attached meme perfectly describes my feeling for the rest of the month following that accident.

Comments
  • 2
    I had partial site failure it effected 0.02% of customers and I was scared to death 😂
  • 3
    Shit can happen, but it's not just your fault.

    Atleast your supervisor should stand up and also take a slice of responsibility. Also the QA guy is questionable.

    You're working as a team and if someone makes a fault, the team did something wrong and not just you ;)

    And the best way out is to learn from it and make it better next time
Add Comment