6

The conversations that come across my DevOps desk on a monthly basis.... These have come into my care via Slack, Email, Jira Tickets, PagerDuty alerts, text messages, GitHub PR Reviews, and phone calls. I spend most of my day just trying to log the work I'm being asked to do.

From Random People:
* Employee <A> and Contractor <B> are starting today. Please provision all 19 of their required accounts.
* Oh, they actually started yesterday, please hurry on this request.

From Engineers:
* The database is failing. Why?
* The read-only replica isn't accepting writes. Can you fix this?
* We have this new project we're starting and we need you to set up continuous integration, deployment, write our unit tests, define an integration test strategy, tell us how to mock every call to everything. We'll need several thousand dollars in AWS resources that we've barely defined. Can you define what AWS resources we need?
* We didn't like your definition of AWS resources, so we came up with our own. We're also going to need you to rearchitect the networking to support our single typescript API.
* The VPN is down and nobody can do any work because you locked us all out of connecting directly over SSH from home. Please unblock my home IP.
* Oh, looks like my VPN password expired. How do I reset my VPN password?
* My GitHub account doesn't have access to this repo. Please make my PR for me.
* Can you tell me how to run this app's test suite?
* CI system failed a build. Why?
* App doesn't send logs to the logging platform. Please tell me why.
* How do I add logging statements to my app?
* Why would I need a logging library, can't you just understand why my app doesn't need to waste my time with logs?

From Various 3rd party vendors:
* <X> application changed their license terms. How much do you really want to pay us now?

From Management:
* <X> left the company, and he was working on these tasks that seem closely related to your work. Here are the 3 GitHub Repos you now own.
* Why is our AWS bill so high? I need you to lower our bill by tomorrow. Preferably by 10k-20k monthly. Thanks.
* Please send this month's plan for DevOps work.
* Please don't do anything on your plan.
* Here's your actual new plan for the month.
* Please also do these 10 interruptions-which-became-epic-projects

From AWS:
* Dear AWS Admin, 17 instances need to be rebooted. Please do so by tomorrow.
* Dear AWS Admin, 3 user accounts saw suspicious activity. Please confirm these were actually you.
* Dear AWS Admin, you need to relaunch every one of your instances into a new VPC within the next year.
* Dear AWS Admin, Your app was suspiciously accessing XYZ, which is a violation of our terms of service. You have 24 hours to address this before we delete your AWS account.

Finally, From Management:
* Please provide management with updates, nobody knows what you do.

From me:
Please pay me more. Please give me a team to assist so I'm not a team of one. Also, my wife is asking me to look for a new job, and she's not wrong. Just saying.

Comments
  • 0
    @JustThat saying no is a big part of my job.
  • 1
  • 2
    If you keep overloading me with tasks, we'll enter triage mode.

    Triage mode? What's that?

    Simply put, I decide what service gets priority and what service has no priority.

    But we decide the priorities.

    ... You try to decide the priorities, but ultimately based on your priorities, I need to shut down some services as they're becoming a risk.

    You're kidding?

    Not at all. I cannot solve all problems at once, but running an service unmonitored or even worse insecure is no option. So shutdown or reducing the workload are the options. Hence triage - I need to decide which service can still run and which service needs to be shut off as there is no time left for maintenance.

    Most often the discussion becomes then rage fueled, but all in all... That's all we as administrators can do.

    Either management gets their shit fixed up, or the services will be taken offline.

    Management deciding priorities is one of the things that can never work. Simply because their perspective is too limited.

    I usually keep a very accurate list of how many servers, services, licenses, costs, tickets, ticket priorities, estimated outstanding work hours etc are there.

    It's always good to have ammunition. The more, the better.
Add Comment