AboutBuilding software to make the electric utility industry .05% less shitty
SkillsC#, Java, Kotlin, TypeScript, Python, etc etc etc
Joined devRant on 1/9/2020
Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
Fun day, lots of relief and catharsis!
Client I was wanting to fire has apparently decided that the long term support contract I knew was bullshit from go will instead be handled by IBM India and it's my job to train them in the "application." Having worked with this team (the majority of whom have been out of university for less than a year), I can say categorically that the best of them can barely manage to copy and paste jQuery examples from SO, so best of fucking luck.
I said, "great!," since I'd been planning on quitting anyways. I even handed them an SOW stating I would train them for 2 days on the application's design and structure, and included a rider they dutifully signed that stated, "design and structure will cover what is needed to maintain the application long term in terms of its basic routing, layout and any 'pages' that we have written for this application. The client acknowledges that 3rd party (non-[us]) documentation is available for the technologies used, but not written by [us], effective support of those platforms will devolve to their respective vendors on expiry of the current support contract."
Contract in hand, and client being too dumb to realize that their severing of the maintenance agreement voids their support contract, I can safely share what's not contractually covered:
- Stream based programming
- Angular 9
- Any of the APIs
- Dotnet core
- pretty much anything not in a commit
I'm a little giddy just thinking about the massive world of hurt they've created for themselves. Couldn't have happened to nicer assholes.7
In the before time (late 90s) I worked for a company that worked for a company that worked for a company that provided software engineering services for NRC regulatory compliance. Fallout radius simulation, security access and checks, operational reporting, that sort of thing. Given that, I spent a lot of time around/at/in nuclear reactors.
One day, we're working on this system that uses RFID (before it was cool) and various physical sensors to do a few things, one of which is to determine if people exist at the intersection of hazardous particles, gasses, etc.
This also happens to be a system which, at that moment, is reporting hazardous conditions and people at the top of the outer containment shell. We know this is probably a red herring or faulty sensor because no one is present in the system vs the access logs and cameras, but we have to check anyways. A few building engineers climb the ladders up there and find that nothing is really visibly wrong and we have an all clear. They did not however know how to check the sensor.
Enter me, the only person from our firm on site that day. So in the next few minutes I am also in a monkey suit (bc protocol), climbing a 150 foot ladder that leads to another 150 foot ladder, all 110lbs of me + a 30lb diag "laptop" slung over my shoulder by a strap. At the top, I walk about a quarter of the way out, open the casing on the sensor module and find that someone had hooked up the line feed, but not the activity connection wire so it was sending a false signal. I open the diag laptop, plug it into the unit, write a simple firmware extension to intermediate the condition, flash, reload. I verify the error has cleared and an appropriate message was sent to the diagnostic system over the radio, run through an error test cycle, radio again, close it up. Once I returned to the ground, sweating my ass off, I also send a not at all passive aggressive email letting the boss know that the next shift will need to push the update to the other 600 air-gapped, unidirectional sensors around the facility.11
Whelp, so much for Dell making good on the 32gb sku for the xps 13 9300. 8 days late and 16gb short with minuscule storage at a $100 premium.
"Developer edition" indeed.11
I am about to fire this client.
I can't take any more of this abject fucking stupidity.
I can't take any more sentence fragment responses to detailed questions and thorough responses.
I can't take any more expectations that I deliver consistent metadata and hundreds of pages of documentation, yet no one else has to do the same
I can't take any more rules only applying to/hamstringing me and my team
I can't take any more fucking gross incompetence and grossly undereducated shitfucks that get to send ridiculous bills and have 0 accountability while playing developer
I can't take any more obviously nepotistic and racist hiring that walks back every step of progress we've made in the last 50 years
I can't take not being able to call a spade a spade and being the villain when there's obvious graft occuring at every level
I can't take these old fucks padding their retirements while rendering everyone else contractors and cutting off opportunity for future generations
I can't take how absurdly, blisteringly stupid the business people are, or the fact that one average project managers with a recent PMI cert somehow bills what I do
I'm 100% going to drop dime on these fucks to every regulatory body they are beholden to, their investors, their corporate owners and USCIS, since I've already doxxed the shit out of all of my coworkers that don't remotely qualify for the positions they occupy.3
So IBM finally jettisoned the cancer that was Virginia Rometty a few weeks back. They had an opportunity to move fresh blood and solid managerial background into the top slot with Jim Whitehurst (Redhat) and try and recover their flagging market share and do some sane business strategy. They passed on that opportunity and instead appointed the old guard bootlicker who overpaid for Redhat to the tune of 20x what it was worth, and signalled their intent to continue staying the course of the Titanic and it's slow inevitable trek towards the bottom of the ocean. The board wants a yes man, and they got one.
This is basically what I assumed would happen, but I have some other predications as well:
- Whitehurst will leave to a better company
- the redhatters that haven't already left will be replaced with commodity labor
- Redhat will be the least stable Linux offering 2 years after the last hatter leaves
- they will sell off most of their existing software assets to HCL/ similar consulting partners like they did with domino and websphere to stem the bleed
- the displaced in that move will either quit or be replaced
- their cloud initiative will collapse under the weight of its own stagnation and glacial pace of development
- they will attempt to salve these wounds by moving focus to global services, reducing profit loss by cutting salary costs, further diluting their eroding ability to innovate
- they'll buy at least one other trendy software company at ridiculous valuation, and sell it off within 2 years at a massive loss
- the CEO slot will start to resemble the late Roman empire with a new CEO every other week
- Redhat assets will be sold to Google inside of 5 years
Last prediction: I will be overjoyed being able to witness the death of IBM in my lifetime. Fuck them 🍻8
I might break my "no beta-testing unless paid" rule for this. I am beyond ready for quic-ness.👍🏻
Story time! Promised this, so making good on the promise. Eh-hem.
Misunderstandings [A slice of life short play that actually happened]
Dramatis Personae (anonymized, bc of course):
Moi ........ me, myself and possibly some lint
Robert ..... co-architect
Daisy ...... line dev
Lisa ....... also line dev
Prologue: the beginninning
[A project is starting up, new devs are coming on, including the two individuals who drive this story.
Daisy, of Indian origin, an exceptional dev and lovely person. Mother, wife, very conservative by upbringing in her early 40s.
Lisa, also exceptional dev, lovely person. Mother, also wife, self-made immigrant with liberal views derived from personal pride and self-bootstrapping]
Enter the office, We introduce everyone, off to a nice start, everyone is happy and excited to be working on [large bank project].
Lisa and Daisy form a friendship of commonality, they have similar backgrounds by all appearances and similar concerns due to children the same age and shared employment. They seem to become fast friends and things proceed normally for some months. Smooth sailing, all is well.
The fuse is lit.
Scene: Lunchtime gossip
[Robert, middle 40s architect adjacent Moi, also architect, age is my own damn business [old, so very old].]
Robert: "So, it seems like Daisy and Lisa are getting along great."
Moi: *snerfs a little, almost chokes on enchilada* Yes, yes they are, It's nice to see...
Robert: *eyebrow, having learned to read my expressions* "Aaaaaaand..."
Moi: "I adore both of them, but they are primarily friends because they don't actually understand most of what the other says"
[Lisa has a thick Taiwanese accent, Daisy has a standard northern indian accent. Never the two shall meet]
Robert: "Are you sure, they seem to have a lot of conversations?"
Moi: "Positive, you weren't at lunch with the three of us. They're polar opposite in terms of values, it'll be fine so long as that never comes up"
Robert: "I'm not even digging into that"
Scene: This is bat country
[More months pass, everything is fine, project is humming along nicely, save a few blips of personality conflicts. Moi takes a vacation. A gas station, somewhere in the middle of Wyoming, a snowstorm, a sports car full of luggage]
Moi: *looks down, sees it's Robert, eyebrow raises, answer* What's on fire?
Robert: "We had to let Lisa go"
Moi: "Ah, they finally understood each other."
Robert: "Yes..." *deep sigh*
[Fade to flashback]
Scene: The office, Lisa's desk
[Daisy and Lisa are discussing non-descript conversation. Daisy broaches the subject of Lisa's past divorce and being a single mother]
Daisy: "It must have been hard, how did you manage?"
Lisa: "I had my daughter, she was my motivation. We made it here, I met my current partner"
Daisy: "That's good! It is so hard, coming to something new. I could never imagine leaving my husband."
Lisa: "He left us, we weren't important, I don't want to marry every again"
Daisy: "Surely you do though? Marriage is great for a woman, my parents found a great husband for me."
Lisa: "Haha, lucky you. Most indian marriage is like prostitution."
[At this moment, Daisy's demeanor takes a nose dive. Whatever was actually said, what she heard was, "Indian marriage is prostitution"]
Daisy: *tears begin pouring down her face, she flings herself back in her chair, head shaking violently she screams* "I AM AN HONORABLE WOMAN!"
[Daisy runs out of the room, straight to HR. Lisa sits there, stunned, not really understanding what just happened or the consequences]
Scene: Back in bat country
[Robert finishes the story, the emotions are a mixture of hilarity at the absurdity of the situation and frustration in the work void it has created]
Moi: "Satan, well. Fuck me. Fuck us. Fuck. Is Daisy alright, is she at least staying? We can't lose two devs at the same time."
Robert: "She got a few days off, she seems fine now, but she's... yeah, I never laughed so hard"
Moi: *double facepalm* "Yeah, the word choice was a bit outrageous. It's not like we didn't know it was coming. I'm going to get back on the road."
Robert: "Alright, enjoy yourself, I'll try and prevent any other forest fires."20
Main language, do people have such a thing? Some quixotic shit about various languages I use:
F#: never quite committing to 100% functional paradigms while still being one of the most enjoyable langs I use
Java: null handling via optional and maybes are ugly, but better than it was
C++: surprisingly functional in 2020
Haskell: no one else seems to ❤️ it, despite it being fucking beautiful
Kotlin: Ternary warfare, Czech sports opinions on how languages should work
Clojure: parenthetical scoping makes me far happier than it should
Typescript: makes ever more sense as it changes7
We'd just finished a refactor of the gRPC strategy. Upgraded all the containers and services to .Net core 3, pushed a number of perf changes to the base layer and a custom adaptive thread scheduler with a heuristic analyzer to adjust between various strategies.
Went from 1.7M requests/s on 4 cores and 8gb ram to almost 8M requests/s on the same, ended up having to split everything out distributed 2 core instances because we were bottlenecking against 10gb/e bandwidth in AWS.2
It's maddening how few people working with the internet don't know anything about the protocols that make it work. Web work, especially, I spend far too much time explaining how status codes, methods, content-types etc work, how they're used and basic fundamental shit about how to do the job of someone building internet applications and consumable services.
The following has played out at more than one company:
App: "Hey api, I need some data"
API: "200 (plain text response message, content-type application/json, 'internal server error')"
App: *blows the fuck up
*msg service team*
Me: "Getting a 200 with a plaintext response containing an internal server exception"
Team: "Yeah, what's the problem?"
Me: "...200 means success, the message suggests 500. Either way, it should be one of the error codes. We use the status code to determine how the application processes the request. What do the logs say?"
Team: "Log says that the user wasn't signed in. Can you not read the response message and make a decision?"
Me: "That status for that is 401. And no, that would require us to know every message you have verbatim, in this case, it doesn't even deserialize and causes an exception because it's not actually json."
Team: "Why 401?"
Me: "It's the code for unauthorized. It tells us to redirect the user to the sign in experience"
Team: "We can't authorize until the user signs in"
Me: *angermatopoeia* "Just, trust me. If a user isn't logged in, return 401, if they don't have permissions you send 403"
Team: *googles SO* "Internet says we can use 500"
Me: "That's server error, it says something blew up with an unhandled exception on your end. You've already established it was an auth issue in the logs."
Team: "But there's an error, why doesn't that work?"
Me: "It's generic. It's like me messaging you and saying, "your service is broken". It doesn't give us any insight into what went wrong or *how* we should attempt to troubleshoot the error or where it occurred. You already know what's wrong, so just tell me with the status code."
Team: "But it's ok, right, 500? It's an error?"
Me: "It puts all the troubleshooting responsibility on your consumer to investigate the error at every level. A precise error code could potentially prevent us from bothering you at all."
Team: "How so?"
Me: "Send 401, we know that it's a login issue, 403, something is wrong with the request, 404 we're hitting an endpoint that doesn't exist, 503 we know that the service can't be reached for some reason, 504 means the service exists, but timed out at the gateway or service. In the worst case we're able to triage who needs to be involved to solve the issue, make sense?"
Team: "Oh, sounds cool, so how do we do that?"
Me: "That's down to your technology, your team will need to implement it. Most frameworks handle it out of the box for many cases."
Team: "Ah, ok. We'll send a 500, that sound easiest"
Me: *..l.. -__- ..l..* "Ok, let's get into the other 5 problems with this situation..."
Moral of the story: If this is you: learn the protocol you're utilizing, provide metadata, and stop treating your customers like shit.23
Oh the joys of working with an Enterprise customer.
Discussion about service architecture with me, development architect (ArchDev) and integration architect (ArchInt). The topic arises of needing to access int. segment systems for a public facing cloud application.
Me: so we'll just need a s2s vpn and then we can just create a route and call the services normally.
ArchDev: sounds good to me, it will take a few months to get that set up
ArchInt: we done need that, we can just use the gateway and then route all the requests through the ESB.
Me: 😕 do you mean the service gateway?
ArchInt: (drops bomb) no, we decide that all API should be implement in ESB, so ESB will handle traffic
Me: *pauses, steps up to the whiteboard, does latency math* setting aside the fact that isn't how ESB's work, that will add at least 700ms latency to each request.
ArchInt: well that is fine for enterprise, things not usually as fast in enterprise you must expect slowdown to be safe
ArchDev: *starts updating resume on the ladders
New twist on an old favorite.
- TeamA provides a service internal to the company.
- That service is made accessible to a cloud environment, also has a requirement to be made available to machines on the local network so you can develop against it.
- Company is too cheap/stupid to get a s2s vpn to their cloud provider.
- Company also only hosts production in the cloud, so all other dev is done locally, or on production non-similar infra, local dev is podman.
- They accomplish service connectivity by use of an inordinately complicated edge gateway/router/firewall/message translator/ouija board/julienne fry maker, also controlled by said service team.
Me: "Hey, we're cool with signing requests using an x509 cert. That said, doing so requires different code than connecting to an unsecured endpoint. Please make this service accessible to developer machines and lower environments on the internal network so we can, you know, develop."
TeamA: "The service should be accessible to [cloud ip range]"
Me: "Yes, that's a production range. We need to be able to test the signing code without testing in production"
TeamA: "Can you mock the data?"
Me: "The code we are testing is relating to auth, not business logic"
TeamA: "What are you trying to do?"
Me: "We are trying to test the code that uses the x509 you provide to connect to the service"
TeamA: "Can you deploy to the cloud"
Me: "Again, no, the cloud is only production per policy, all lower environments are in the local data center"
TeamA: "can you try connecting to the gateway?"
Me: "Yes, we have, it's not accessible, it only has public DNS, and only allows [cloud ip range]"
TeamA: "it work when we try it"
Me: "Can you please supply repro steps so we can adjust our process"
TeamA: "Yes, log into the gateway and try issuing the call from there"
Me: (╯°□°)╯︵ ┻━┻
tl;dr: Works on my server
It's easy for a job you love to become something you hate. We have to do our jobs, and build influence at the same time lest we fall victim to this.
You either shape or you get shaped.1
Finished porting clientX to Linux, including dev laptops and desktops. The best part was getting to personally enjoy the moment I got to delete all IIS/Windows integration logic from the .Net core services.
"Windows... where we're going we don't need Windows. #CLD-15"2