SkillsAWS, Azure, Python, Go, Shell, PowerShell
Joined devRant on 12/13/2018
Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
Proudest bug squash? Probably the time I fixed a few bugs by accident when I was just trying to clean up an ex-coworker's messy code.
So I used to work with a guy who was not a very good programmer. It's hard to explain exactly why other than to say that he never really grew out of the college mindset. He never really learned the importance of critical thinking and problem-solving. He did everything "by the book" to a point where if he ran into an issue that had no textbook solution, he would spin his wheels for weeks while constantly lying to us about his progress until one of us would finally notice and take the problem off his plate. His code was technically functional, but still very bad.
Quick Background: Our team is responsible for deploying and maintaining cloud resources in AWS and Azure. We do this with Terraform, a domain-specific language that lets us define all our infrastructure as code and automate everything.
After he left, I took on the work to modify some of the Terraform code he'd written. In the process, I discovered what I like to call "The Übervariable", a map of at least 80 items, many of them completely unrelated to each other, which were all referenced exactly once in his code and never modified. Basically it was a dynamic collection variable holding 80+ constants. Some of these constants were only used in mathematical expressions with multiple other constants from the same data structure, resulting in a new value that would also be a constant. Some of the constants were identical values that could never possibly differ, but were still stored as separate values in the map.
After I made the modification I was supposed to make, I decided I was so bothered by his shitty code that I would spend some extra time fixing and optimizing it. The end result: one week of work, 800 lines of code deleted, 30 lines added, and a massive increase in efficiency. I deleted the Übervariable and hardcoded most of the values it contained since there was no possible reason for any of them to change in the future. In the process, I accidentally fixed three bugs that had been printing ominous-sounding warnings to the console whenever the code was run.
I have a lot of stories about this guy. I should post some more of them eventually.3
Most awkward recruiter interaction was probably when I had to tell a staffing agency that I didn't want to work for them because I had already worked for them and left.
I got into IT by working for a staffing agency and I was contracted out to a large company's IT call center. Doing the usual sort of level-one tech support stuff. After my contract term ran out (and upon reminding my boss that she wouldn't be my boss for much longer if she didn't hire me away from the staffing agency), I was hired on full-time.
Six months later I left the call center and moved on to a cloud server development job in the same company. Not long after that, I got a message on LinkedIn from the staffing agency, offering to hire me on as a contractor working for one of their largest partners in the area.
I responded asking for more details, just for fun. The company I'd be working for, etc. Then I had to inform them that I had in fact previously worked for their firm, and now worked at the company that they were offering to contract me out to, and earning a fair bit more money than they were offering.
They didn't even look at my employment information on LinkedIn before sending the InMail. Just glanced over my skills, saw the magic buzzword "devops," and sent me a message.2
I used to have imposter syndrome when I first started at my current job. But then I discovered that one of my coworkers was an actual imposter. He didn't lie on his resume or anything but he was basically incapable of thinking for himself. If there was no step-by-step process to follow, he'd spin his wheels for weeks before doing it in the worst possible way, refusing all offers of assistance from the rest of our team.
After he quit and the true extent of his incompetence came to light, I no longer felt like an imposter.1
So today I logged into devrant and noticed that all of the ++ buttons were gone. Apparently the Fanboy's Annoyances filter list (which I enabled recently in AdGuard) includes the "plusone" CSS class, which is the class name Devrant uses for the ++ button. Whoops.
I think I might remove that filter list entirely. There's a lot of other very non-specific stuff it blocks that might break a lot of websites.3
Haven't started this project yet, but I've been considering ordering another Raspberry Pi and turning it into a voice assistant using Mycroft. I've been wanting to play around with IoT stuff for a while, and since I'm practically confined to my apartment for the foreseeable, there's no better time to get started.
Why is it that an issue is only critical-priority until the person who's raising the biggest fuss has to do something about it?
I was notified that a website hosted in AWS went down overnight and never came back up. I was then bombarded with email after email after email while I logged into our AWS account and poked around. I'm responsible for cloud infrastructure stuff, like VMs or virtual networking or security or whatever, not the actual applications running on said infrastructure. Once I confirmed their EC2 instance was reachable and I could login with SSH, I told them they'd have to fix their application.
They told me that they had no backend developer on their development team. I'm still getting a deluge of emails from multiple people on this team and their managers and managers' managers and so on.
"Perfectly understandable," I told them, though it was anything but. "You should probably look into obtaining one."
The emails stopped immediately. I assumed they were handling it and closed my ticket and moved on. But apparently I was wrong.
Six weeks later, the site is still down, they still have no backend dev, and I'm convinced that they were lying to me when they stressed the importance of this web app because now that it's no longer my problem, not a single person seems to care that it's still broken.3
I'm not a fan. Harder to collaborate and discuss things with coworkers. I'm constantly fighting the urge to turn towards my gaming pc and start playing something. I only have one monitor instead of the 2 I have in the office. I suffer from depression and the social isolation will definitely exacerbate that. The commute to my office is less than 15 minutes so I'm barely saving any time there. Pretty much the only benefit for me is not needing headphones to listen to music.2
Most unusual place I've coded would probably at a bar while utterly wasted. I fixed a production outage and even got on the phone with tier 1 support when they reported the issue.4
I don't like interview coding challenges. At the same time, given the skill level of some developers I've worked with who work for a contracting firm and presumably didn't get a coding test in their intervies...I understand the necessity. Some people are so bad at coding that even the simplest of coding tests can show how bad they are.
I think my favorite is being given a simple task to write code for. And that's it. No "use this specific language feature to do this specific thing". Just a task and that's it.
I got a really simple coding test once. I had to reverse a string. I could choose any language. Presumably they wanted to see loops or something, but I just used Python and did this: string[::-1]
I got the job.3
My goal for 2020: actually finish a personal project.
Last year, I built 25% of a website, 10% of a companion app for a video game, and half of a command-line utility. I would really like to finish an entire project this year, but considering that I just started a new project that promises to be huge and unwieldy, that will probably have to wait for whatever project I try next after burning out on this one.1
The one I use the most often is the AWS API (usually via a wrapper like boto3), but I hate it so it's not my favorite.
Ive been playing with the Bungie.Net API for a while, with the end goal of building some kind of dashboard for information about the various weekly events and such in Destiny 2. I guess that's currently my favorite.
One of my former coworkers was either completely incompetent or outright sabotaging us on purpose. After he left for a different job, I picked up the project he was working on and oh my God it's a complete shitshow. I deleted hundreds of lines of code so far, and replaced them with maybe 30-40 lines altogether. I'm probably going to delete another 400 lines this week before I get to a point where I can say it's fixed.
He defined over 150 constants, each of which was only referenced in a single location. Sometimes performing operations on those constants (with other constants) to get a result that might as well have been hard-coded anyway since every value contributing to that result was hard-coded. He used troublesome and messy workarounds for language defects that were actually fixed months before this project began. He copied code that I wrote for one such workaround, including the comment which states the workaround won't be necessary after May 2019. He did this in August, three months later.
Two weeks of work just to get the code to a point where it doesn't make my eyes bleed. Probably another week to make it stop showing ten warnings every time it builds successfully, preventing Jenkins from throwing a fit with every build. And then I can actually implement the feature I was supposed to implement last month.6
Fuck the Fourth of July.
I was already feeling pretty unpatriotic after all the terrible things the USA has been doing lately, and then to top it all off, they closed down almost every street within a one-mile radius of my apartment just because I happen to live near the place where they're shooting off fireworks. I spent the evening with my girlfriend, enjoying a nice dinner and sort of forgetting it was even really a holiday, then got in my car at 9:00 expecting to be home by 9:30. Nope. I spent over an hour driving in a huge circle around my apartment, searching for even one single road that the police hadn't closed to traffic. Finally managed to get into the area on a back road and get home, but it took so long that I went through all five stages of grief before I got home at 11:00.
They also closed the nearest bridge across the river so I couldn't go shopping for groceries today, either. My pantry is almost completely empty. At least I'm working from home tomorrow so I'll have time to buy food then.8
Got an email earlier this week. It went something like this:
"It looks like your team still hasn't delivered the logging and monitoring solution that we asked for. Can you get it done in time for our production deployment next Friday?"
Um, wait, excuse me, WHAT?
1. You never actually asked for the thing you claim we didn't deliver. In fact, when we brought up the fact that you should probably have some monitoring set up for your servers, you said it would be handled entirely by your own team.
2. I HAVE BEEN WORKING ON THIS PROJECT FOR SIX MONTHS WHY DIDN'T YOU TELL ME YOUR DEADLINE UNTIL NOW
3. I won't even have time to start working on this until the Monday after your prod deployment date. Sorrynotsorry.
I really shouldn't be surprised though. This project has been a clusterfuck from the very beginning so this is just par for the course.2
The bug I never fixed isn't a bug in code I wrote, but rather an OS problem I've given up on fixing.
I dual-boot Windows and Linux on my desktop PC. Every time Windows updates, it switches from grub to the Windows bootloader, making it impossible to boot into Linux. I've fixed it three times (each time requiring a different fix, from disabling fast startup to reinstalling Grub from a live USB), then gave up. My desktop PC is now a Windows machine. I'm upgrading some parts soon (including replacing my boot drive with an NVMe SSD) so I decided when I do that, I'm just going to reinstall Linux on the new drive and see how long I can last without installing Windows at all.5
The last eight years were fun, but I ran out of space while trying to compile a project, and, well, your number came up. I'm sorry...
I need a bigger SSD. I launched Visual Studio (which I rarely use so it only had the default extensions installed) to clone and build the new Windows Terminal to see what it's like. Had to download over 10GB of extensions and features first, and then compiling the project ate up every last byte of remaining space.8
My team handles infrastructure deployment and automation in the cloud for our company, so we don't exactly develop applications ourselves, but we're responsible for building deployment pipelines, provisioning cloud resources, automating their deployments, etc.
I've ranted about this before, but it fits the weekly rant so I'll do it again.
Someone deployed an autoscaling application into our production AWS account, but they set the maximum instance count to 300. The account limit was less than that. So, of course, their application gets stuck and starts scaling out infinitely. Two hundred new servers spun up in an hour before hitting the limit and then throwing errors all over the place. They send me a ticket and I login to AWS to investigate. Not only have they broken their own application, but they've also made it impossible to deploy anything else into prod. Every other autoscaling group is now unable to scale out at all. We had to submit an emergency limit increase request to AWS, spent thousands of dollars on those stupidly-large instances, and yelled at the dev team responsible. Two weeks later, THEY INCREASED THE MAX COUNT TO 500 AND IT HAPPENED AGAIN!
And the whole thing happened because a database filled up the hard drive, so it would spin up a new server, whose hard drive would be full already and thus spin up a new server, and so on into infinity.
Thats probably the only WTF moment that resulted in me actually saying "WTF?!" out loud to the person responsible, but I've had others. One dev team had their code logging to a location they couldn't access, so we got daily requests for two weeks to download and email log files to them. Another dev team refused to believe their server was crashing due to their bad code even after we showed them the logs that demonstrated their application had a massive memory leak. Another team arbitrarily decided that they were going to deploy their code at 4 AM on a Saturday and they wanted a member of my team to be available in case something went wrong. We aren't 24/7 support. We aren't even weekend support. Or any support, technically. Another team told us we had one day to do three weeks' worth of work to deploy their application because they had set a hard deadline and then didn't tell us about it until the day before. We gave them a flat "No" for that request.
I could probably keep going, but you get the gist of it.7
Spent three days banging my head against my desk trying to get an AWS Lambda function to work, only to finally discover that my code was perfectly functional and it was a security group problem. It was supposed to send a POST request to a load balancer's URL but couldn't resolve the hostname because the security group blocked a necessary outbound port for DNS requests.
That's what I get for not troubleshooting at the infrastructure level when experiencing connection issues. I did not spend two years doing tech support just to forget basic troubleshooting steps now that I'm in the DevOps field...1
Meetings, responding to emails, handling urgent tickets, etc. If I could just get four uninterrupted hours of coding in a day, I'd be happy. But I'm basically in meetings all morning and usually have at least 1 more in the middle of the afternoon.2
My very first rant here was about the mess of ticket submission and ticket tracking applications we use, and about how we were moving to a single unified system some day.
Well, that day is today. And, predictably, it went horribly wrong.
So the way it's supposed to work is people login to the portal, search for what they want to request, then fill in details and submit. It creates a request ticket assigned to the appropriate team. (The old way involved a bunch of nonsense that you can see in my first rant).
The thing is, I found out about this today, when I got a company-wide email saying the new system was live as of this morning. None of us knew it would happen today. Not that I could've foreseen any issues just by getting the announcement early, but still, usually people find out about these things beforehand.
So, ecstatic to finally be rid of the old ticket tracking system, I log into the new system and look for our request form, which is, of course, not there. I check the old system and see that they combined every single "general request" into a single request where you pick which team the request goes to.
So I finally find the right request, pick the right department from the drop-down, and see that the request looks much better than it did on the old system. Out of curiosity, I look at the list of people who are part of that department.
I am not on the list.
My ENTIRE TEAM is not on the list.
Because they migrated the team data to the new system a year ago, when the issue tracking/reporting portion of it went live. My current team was hired approximately six months after that and apparently updating the team data in the new system isn't part of our Onboarding process yet.
So... Bright side is I guess I will have a lot of free time soon since nobody can submit new project work to my team?
tl;dr: they took a great software product and implemented it so poorly that our team can't use it.3
The second time I dropped out of college. I wasn't sure why at the time, but I just couldn't manage to keep myself motivated and interested. Later on I was diagnosed with ADD and all my school problems made sense, but at the time, I thought that since I'd tried and failed twice to get a computer science degree, I wasn't qualified to be a developer.
And now I have a degree and a dev job and I know what's actually wrong with my brain and how to deal with it.13
Wow, Corsair pulled no punches today. I think I like their April Fools joke the best so far.
Mostly indie games these days, since I have hundreds of them in my Steam library and haven't played most of them. Last week I tried Factorio and now I'm obsessed with it.11
Finish a personal project.
Any personal project. I'm not picky. The last time I completed a project that wasn't for my job was like five (well, now six) years ago.
On the nth day of Christmas, my true love gave to me:
1 misconfigured autoscaling group
200 unnecessary servers
29 urgent emails
3 support teams that would have fixed the problem in an hour of anybody had bothered going to them first
Ugh. Idiots. Somehow the whole issue was caused by a single full hard disk, which caused database transactions to fail, which caused the group to scale out and spin up new instances, which didn't actually fix the problem so it kept scaling until it hit the limit and then continuously failed to create new instances for several hours straight, generating loads of notification emails and generally causing problems for everyone involved.
Thinking of developing a mod for a game I play, but only thinking about it so far. No coding yet. Probably no coding until I'm back at work on the 26th.1
Our ticket tracking system and our IT service request system are from two different companies that are direct competitors. The source code is full of temporary hacks to just make them play nice until a better solution is worked out. Fast forward a few years and we're abandoning both systems in favor of a single, unified system that handles everything. We currently have maybe 20% of the new, unified system done, which is now hacked together with both of the legacy systems until we finally transition fully to the new system. The current plan is for next year, but the plan six months ago was for this year, and almost no progress has been made since then, so we're probably going to have two ticket trackers and two request systems for a while.
Actually, three ticket trackers and three request systems. The third ticket tracker is used to track work done on tickets that exist in the legacy tracker because the legacy tracker can't do that on its own, while the third request system is the oldest and most cumbersome legacy system of them all.1