30

Had 2 days of vacation. Theoretically (plus weekend, plus 2 days) 6 days.

Worked today… At Saturday.

Some administrators forgot to properly check bandwidth limitations....

*rolls eyes*

We had a major version upgrade of some server software at Monday.

Guess why I got called...

Of course it MUST be the software upgrade.

It couldn't be the new hardware that was setup 2 weeks ago and on which a lot of "important" VMs were migrated.

*eyes roll inside till only white is visible*

The even more annoying thing is that it wasn't that hard to figure out.

Looking at monitoring, we had spikes on 20 Gbit/s (roughly 2.x Gigabyte/sec - Ethernet) connection of some server at roughly 1.9 plus Gigabyte/sec.

IO latency spikes that made the graph look like a heartbeat EKG with severe tachycardia...

*additionally to white eyes starts cursing in reverse latin*

Incompetent admin answer: Booboo that can only be your fault - the developers must investigate.

Me (just a tad more polite): Meep Meep mother fucker, get your shit together. If the software would eat that much, the network would be a niece chunk of charcoal. Plus the time (sending instead of links to monitoring pictures… guess the lazy fucktard who's brain is a vacuum didn't even bother to check it)...

NOTICE SOMETHING?!

Incompetent admin: It starts at the same time. Always.

After wasting roughly another hour of time discussing with him, I just hanged up the video call.

Called someone I knew from the admin department and turns out that - drumrolls please - the incompetent admin was someone who got recruited 3 months ago…

*turning into antichrist*

I then had a not so polite discussion about how the only competent people could take days off (all except incompetent admin were on vacation) and the seemingly incompetent fresh recruit - who by the way NEVER mentioned this - was the only one left of the admin department. Which would be bad alone, but no - he even got the 24/7 emergency support role for the whole weekend.

Sometimes this company and HR especially notoriously drive me insane...

Guess next week there will be some HR barbecue.

But yeah. After a lot of raging around we nailed it down to the traffic of backups and could fix it.

Roughly 4 hours of analysis, communication, raging and hatred.

Just one hour implementing shit.

*goozfraba*

Comments
  • 1
    At the first part I was thinking dude you guys need monitoring. But than you have it... Hope you get at least compensated properly.

    I would say that this is more a management fuckup. Way too much responsibility for the new hire. Also if the vetting process is solely done by HR that is a huge management fuckup up.
  • 8
    @hjk101 yeah.

    The reason I'm grumpy with HR: They *must* give their blessing to vacations... They *must* be informed who does 24/7 support (legal reason / taxes / extra compensation).

    We're not so that large that it would be an issue to notice that someone does 24/7 support with no experience (worst still in probation time.... Oh lord.)

    Admin management fucked up without any question, but that's the reason why HR exists: If a management fucks up it's their responsibility to prevent disaster.

    They know exactly who belongs to which department and that no (small) department is allowed to have less than 3 employees working. 3 - 1 24/7 support, 1 regular, 1 regular as fallback.

    It's really not that hard. These rules were made for a reason. 🤬
  • 1
    Mmm barbeque 😏 🍖
  • 0
    !!tse erocrets sunisa mulucicsaf diuQ
  • 1
    @IntrusionCM ah than it is a perfectly good reason (as if we need more) to be pissed at HR. Never seen a HR backstop but it makes sense. Could even be automated rules.

    At my current employer management doesn't even have decent control. If your team is ok with it you can take off. Basically that is always. They just auto sign off whatever request comes in.
  • 1
    @Jedidja !!!sumabib ogrE
  • 2
    @IntrusionCM already there, mate.
    Cheers!
  • 0
    @Jedidja Cheers! Looks fascinating, self brew?
  • 1
    @IntrusionCM no, from a local brewery that sells refills

    (the small amount of self brewing that I was part of didn't bring anything worth the effort)
  • 0
    Why still VM and not Containers or SaaS?
  • 0
    @max19931 Containers still need a server to run on, which makes sense to be a VM...

    SaaS is in my opinion a broken concept...

    Or better a "concept to make profit by exploitation of weak links in understaffed companies with limited resources".

    I could elaborate on that, but that would be a rant on its own.

    Our company isn't that large, nor is it extremely small. Somewhere in the middle tier - we need our own infrastructure, as a lot of workflows depend on it.
Add Comment