39
xawos
290d

~RANT
I started two months ago a new role in a new company.
Together with 3 colleagues I manage a HUGE OpenStack platform (I’m talking about more than 300 customers, many petabytes of data, dozens of compute nodes 😍).

On Friday night a well-known multinational had the GREAT idea to update the software of one of the core components of the platform (Neutron).
Obviously it all went wrong and customers started to call and open tickets, so even if it was already past midnight we’ve been all called to arms.
As a workaround we had to rebuild ALL the networks, detach all the ports and reattach them to the VMs, with all the errors that came with this kind of operations, such as failing to get an internal/external IP, fail to boot etcetera.

What amuses me is the fact that the above-mentioned multinational just asked for logs with no solution (except for the above workaround) until Monday morning.

It has been a good weekend, i talked with professionals from all over the world, in various languages, and worked multi-tasking with two different computers, with a third that was used just to share my screen and VPN for the management network.

During this weekend I virtually met all the possible colleagues that I will meet in this new role, and it was pretty funny to see how they handle stress. (Badly if you ask)

I’ve been woken up at 7:30 after falling asleep at 6:45, I’ve slept 17 hours in 5 days, haven’t taken a shower in 5 days (I know, it happens, I worked from home) but it was worth it.
Not only and merely for the money (overtime work), but mainly for the job itself.
I love to handle this HUGE infrastructures for now, even if I think they have many limitations and many components could have been written better.

The best thing about this weekend was to meet a group of people from all over the country and the world that worked non-stop from Friday night until Wednesday night to solve the situation.

All of them understood me when I asked them questions, and didn’t treat me like the [insert random]science nerd that I am.

I felt like in the movies, in front of three monitors, working from the comfort of my chair, in my underwear, eating as fast as I could because I was constantly in call with someone.

Apart from that, WHAT THE FUCK WENT THROUGH YOUR HEAD TO DEPLOY AN UPDATE ON FRIDAY NIGHT, YOU FUCKING PIECE OF adorable man???

WHY COULDN’T YOU WAIT UNTIL MONDAY??? NOBODY USES THOSE SERVICES AT NIGHT APART FROM THE SCHEDULED JOBS!!!!!

I HOPE YOU’LL HAVE A LONG AND PROSPER LIFE, FULL OF JOY AND HAPPINESS

Comments
  • 4
  • 1
    Is it up and running now ?
  • 2
    @Julien00859
    Yes, we applied the workaround an all the tenants via semi-automated scripts 💪🏻😎
  • 3
    So nice to see somebody that loves his work and that can get the best out of hard times. Well done Sir/Madam
  • 3
    @xawos can I borrow your teams??? Mine make me cry when this happens to us.
  • 2
    We had a production update this Friday night, and it was my first such offshore support gig.

    I was kind of all excited that I might get a call (I was nervous as hell, but excited all the same). But I guess everything went well and I didn't get any call. :/

    Dunno whether I should be happy or sad.
Your Job Suck?
Get a Better Job
Add Comment