9

Three days ago my focus was shifted from a development role to a support role. I was shifted to replace another support guy who had used fraud to get the position. I have no experience with this role but there was decent KT and I'm catching on fine. During onboarding and KT I'm serving as the first contact for new tickets and whatnot...

Today I got a ticket with an error on our production instance that no one had ever seen before. It prevented the guy from using our service entirely. I tried to reproduce it and... I couldn't use the service either. No one could. Everything was down. I could see the sweat building on my manager's forehead.

Thankfully another member on my team has done a bit of support before, so we collaborated with each other and other teams throughout the day to figure out what's wrong and how to fix it. I'm listening to them chat remotely as we speak - so far I've been working on it 9 hours straight.

This service is used by everyone - it's a business critical service with due dates on actions and escalations to managers... Imagine if the support ticketing service for your company crashed. That means a lot of people are asking what's wrong, requiring extensions, etc. I've been answering to managers and seniors in the business throughout the day.

The best part? We figured out why the server went down, and the reason is fantastic: someone updated the server's code without telling anyone, and all they had done was remove critical parsing code. Just took it right out, pushed, redeployed. We don't know who did it or who even has access to do that. I guess I have some detective work cut out for me after we've fixed everything that was broken by that.

I miss coding already.

Comments
  • 2
    Sounds like the support guy who was a "fraud" knew more than he let on.
Add Comment