Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
What an absolute fucking disaster of a day. Strap in, folks; it's time for a bumpy ride!
I got a whole hour of work done today. The first hour of my morning because I went to work a bit early. Then people started complaining about Jenkins jobs failing on that one Jenkins server our team has been wanting to decom for two years but management won't let us force people to move to new servers. It's a single server with over four thousand projects, some of which run massive data processing jobs that last DAYS. The server was originally set up by people who have since quit, of course, and left it behind for my team to adopt with zero documentation.
Anyway, the 500GB disk is 100% full. The memory (all 64GB of it) is fully consumed by stuck jobs. We can't track down large old files to delete because du chokes on the workspace folder with thousands of subfolders with no Ram to spare. We decide to basically take a hacksaw to it, deleting the workspace for every job not currently in progress. This of course fucked up some really poorly-designed pipelines that relied on workspaces persisting between jobs, so we had to deal with complaints about that as well.
So we get the Jenkins server up and running again just in time for AWS to have a major incident affecting EC2 instance provisioning in our primary region. People keep bugging me to fix it, I keep telling them that it's Amazon's problem to solve, they wait a few minutes and ask me to fix it again. Emails flying back and forth until that was done.
Lunch time already. But the fun isn't over yet!
I get back to my desk to find out that new hires or people who got new Mac laptops recently can't even install our toolchain, because management has started handing out M1 Macs without telling us and all our tools are compiled solely for x86_64. That took some troubleshooting to even figure out what the problem was because the only error people got from homebrew was that the formula was empty when it clearly wasn't.
After figuring out that problem (but not fully solving it yet), one team starts complaining to us about a Github problem because we manage the github org. Except it's not a github problem and I already knew this because they are a Problem Team that uses some technical authoring software with Git integration but they only have even the barest understanding of what Git actually does. Turns out it's a Git problem. An update for Git was pushed out recently that patches a big bad vulnerability and the way it was patched causes problems because they're using Git wrong (multiple users accessing the same local repo on a samba share). It's a huge vulnerability so my entire conversation with them went sort of like:
"Please don't."
"We have to."
"Fine, here's a workaround, this will allow arbitrary code execution by anyone with physical or virtual access to this computer that you have sitting in an unlocked office somewhere."
"How do I run a Git command I don't use Git."
So that dealt with, I start taking a look at our toolchain, trying to figure out if I can easily just cross-compile it to arm64 for the M1 macbooks or if it will be a more involved fix. And I find all kinds of horrendous shit left behind by the people who wrote the tools that, naturally, they left for us to adopt when they quit over a year ago. I'm talking entire functions in a tool used by hundreds of people that were put in as a joke, poorly documented functions I am still trying to puzzle out, and exactly zero comments in the code and abbreviated function names like "gars", "snh", and "jgajawwawstai".
While I'm looking into that, the person from our team who is responsible for incident communication finally gets the AWS EC2 provisioning issue reported to IT Operations, who sent out an alert to affected users that should have gone out hours earlier.
Meanwhile, according to the health dashboard in AWS, the issue had already been resolved three hours before the communication went out and the ticket remains open at this moment, as far as I know.5 -
EoS1: This is the continuation of my previous rant, "The Ballad of The Six Witchers and The Undocumented Java Tool". Catch the first part here: https://devrant.com/rants/5009817/...
The Undocumented Java Tool, created by Those Who Came Before to fight the great battles of the past, is a swift beast. It reaches systems unknown and impacts many processes, unbeknownst even to said processes' masters. All from within it's lair, a foggy Windows Server swamp of moldy data streams and boggy flows.
One of The Six Witchers, the Wild One, scouted ahead to map the input and output data streams of the Unmapped Data Swamp. Accompanied only by his animal familiars, NetCat and WireShark.
Two others, bold and adventurous, raised their decompiling blades against the Undocumented Java Tool beast itself, to uncover it's data processing secrets.
Another of the witchers, of dark complexion and smooth speak, followed the data upstream to find where the fuck the limited excel sheets that feeds The Beast comes from, since it's handlers only know that "every other day a new one appears on this shared active directory location". WTF do people often have NPC-levels of unawareness about their own fucking jobs?!?!
The other witchers left to tend to the Burn-Rate Bonfire, for The Sprint is dark and full of terrors, and some bigwigs always manage to shoehorn their whims/unrelated stories into a otherwise lean sprint.
At the dawn of the new year, the witchers reconvened. "The Beast breathes a currency conversion API" - said The Wild One - "And it's claws and fangs strike mostly at two independent JIRA clusters, sometimes upserting issues. It uses a company-deprecated API to send emails. We're in deep shit."
"I've found The Source of Fucking Excel Sheets" - said the smooth witcher - "It is The Temple of Cash-Flow, where the priests weave the Tapestry of Transactions. Our Fucking Excel Sheets are but a snapshot of the latest updates on the balance of some billing accounts. I spoke with one of the priestesses, and she told me that The Oracle (DB) would be able to provide us with The Data directly, if we were to learn the way of the ODBC and the Query"
"We stroke at the beast" - said the bold and adventurous witchers, now deserving of the bragging rights to be called The Butchers of Jarfile - "It is actually fewer than twenty classes and modules. Most are API-drivers. And less than 40% of the code is ever even fucking used! We found fucking JIRA API tokens and URIs hard-coded. And it is all synchronous and monolithic - no wonder it takes almost 20 hours to run a single fucking excel sheet".
Together, the witchers figured out that each new billing account were morphed by The Beast into a new JIRA issue, if none was open yet for it. Transactions were used to update the outstanding balance on the issues regarding the billing accounts. The currency conversion API was used too often, and it's purpose was only to give a rough estimate of the total balance in each Jira issue in USD, since each issue could have transactions in several currencies. The Beast would consume the Excel sheet, do some cryptic transformations on it, and for each resulting line access the currency API and upsert a JIRA issue. The secrets of those transformations were still hidden from the witchers. When and why would The Beast send emails, was still a mistery.
As the Witchers Council approached an end and all were armed with knowledge and information, they decided on the next steps.
The Wild Witcher, known in every tavern in the land and by the sea, would create a connector to The Red Port of Redis, where every currency conversion is already updated by other processes and can be quickly retrieved inside the VPC. The Greenhorn Witcher is to follow him and build an offline process to update balances in JIRA issues.
The Butchers of Jarfile were to build The Juggler, an automation that should be able to receive a parquet file with an insertion plan and asynchronously update the JIRA API with scores of concurrent requests.
The Smooth Witcher, proud of his new lead, was to build The Oracle Watch, an order that would guard the Oracle (DB) at the Temple of Cash-Flow and report every qualifying transaction to parquet files in AWS S3. The Data would then be pushed to cross The Event Bridge into The Cluster of Sparks and Storms.
This Witcher Who Writes is to ride the Elephant of Hadoop into The Cluster of Sparks an Storms, to weave the signs of Map and Reduce and with speed and precision transform The Data into The Insertion Plan.
However, how exactly is The Data to be transformed is not yet known.
Will the Witchers be able to build The Data's New Path? Will they figure out the mysterious transformation? Will they discover the Undocumented Java Tool's secrets on notifying customers and aggregating data?
This story is still afoot. Only the future will tell, and I will keep you posted.6 -
So we've got a gif that doesn't show up in our React Native application. Of course, the designers assume it's me: "are you sure the gif is in the codebase? how are you using it in your component"? yeah ok boomer. I'm like, look at this other gif, works fine. "oh" So I tell them, double check the export options on how you are building the gif, maybe there is something there. so now they are asking ME for those export options. I'M A DEV, NOT A DESIGNER, DO YOUR JOB AND FIGURE IT OUT. I DON'T EVEN KNOW WHAT PROGRAM YOU ARE USING
oh as an aside, I was putting up a website for a client and they are like "my logo is quite similar to many others, is this something to worry about legally?" OH, SO NOW I'M A LAWYER TOO??!!?!? WHAT IS WRONG WITH YOU PEOPLE GOOGLE IS LITERALLY IN FRONT OF YOU
why do people continually think just because we can code we are fucking designers / lawyers / astronauts
/ god?
man this pisses me off - i think of that draw red lines with blue ink expert video, in the end, just smile and nod: "i can do... absolutely anything... trust me, I'm an expert"4 -
I had a pretty good year! I've gone from being a totally unknown passionate web dev to a respected full stack dev. This will be a bit lengthy rant...
Best:
- Got my first full time employment dev role at a company after being self-taught for 8+ years at the start of the year. Finally got someone to take the risk of hiring someone who's "untested" and only done small and odd jobs professionally. This kickstarted my career, super grateful for that!
- Started my own programming consulting company.
- Gained enough confidence to apply to other jobs, snatched a few consulting jobs, nailed the interviews even though I never practiced any leet code.
- Currently work as a 99% remote dev (only meet up in person during the initialization of some projects.) I never thought working remotely could actually work this well. I am able to stay productive and actually focus on the work instead of living up to the 9-5 standard. If I want to go for a walk to think I can do that, I can be as social and asocial as I want. I like to sleep in and work during the night with a cup of tea in the dark and it's not an issue! I really like the freedom and I feel like I've never been more productive.
- Ended up with very happy customers and now got a steady amount of jobs rolling in and contracts are being extended.
- I learned a lot, specialized in graph databases, no more db modelling hell. Loving it!
- Got a job where I can use my favorite tools and actually create something from scratch which includes a lot of different fields. I am really happy I can use all my skills and learn new things along the way, like data analysis, databricks, hadoop, data ingesting, centralised auth like promerium and centralised logging.
- I also learned how important softskills are, I've learned to understand my clients needs and how to both communicate both as a developer and an entrepeneur.
Worst:
- First job had a manager which just gave me the specifications solo project and didn't check in or meet me for 8 weeks with vague specifications. Turns out the manager was super biased on how to write code and wanted to micromanage every aspect while still being totally absent. They got mad that I had used AJAX for requests as that was a "waste of time".
- I learned the harsh reality of working as a contractor in the US from a foreign country. Worked on an "indefinite" contract, suddenly got a 2 day notification to sum up my work (not related to my performance) after being there for 7+ months.
- I really don't like the current industry standard when it comes to developing websites (I mostly work in node.js), I like working with static websites (with static website generators like what the Svelte.js driver) and use a REST API for dynamic content. When working on the backend there's a library for everything and I've wasted so many hours this year to fix bugs and create workarounds related to dependencies. You need to dive into a rabbit hole for every tool and do something which may work or break something later. I've had so many issues with CICD and deployment to the cloud. There's a library for everything but there's so many that it's impossible to learn about the edge cases of everything. Doesn't help that everything is abstracted away, which works 90% of the time but I use 15 times the time to debug things when a bug appears. I work against a black box which may or may not have an up to date documentation and it's so complex that it will require you to yell incantations from the F#$K
era and sacrifice a goat for it to work properly.
- Learned that a lot of companies call their complex services "microservices". Ah yes, the microservice with 20 endpoints which all do completely unrelated tasks?
Top Tags
Weekly Rant
View