Joined devRant on 10/29/2016
Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple APILearn More
07.25 early ringing of phone.
I'm usually dead asleep till 9 o clock.
Went to bed at 01 o clock.
Something crushed at work, needed to be fixed ASAP.
No coffee. Tired. Stinking peace of garbage
08.45 o clock - super market delivery came earlier than expected, while la me was still trying to communicate necessary steps to resolve the issue.
Forgot to pre pay online.
Still had no coffee, still a piece of stinking shit, still tired.
Took me nearly 20 mins to get my PIN right.
Poor delivery guy had unpleasant call from chef, I needed to deescalate.
Back to work, people angry for me being 20 mins away in midst of chaos.
Me back to fixing stuff.
Done at 09.30.
One of these days where everything you touch becomes a large pile of poop and no matter what you do it's wrong.
Yep. The rest of the day went pretty much as bonkers as it started.
At least no work on weekend.
DNS is everywhere.
I hate DNS.
I hate DNS migrations.
I hate having a hundred plus DNS names inside my brain.
I hate resolving issues.
I hate DNSSEC.
I hate CNAMES.
I hate services which cannot be persuaded to stop trying AAAA resolves first.
I hate the fucking stupid braindead idea to use TXT as a configuration store inside DNS... And thus the necessity to blow up DNS query size aka EDNS.
I really really really really really want to burn this whole mfucking shit down...6
More a call for discussion...
How can it be that devs constantly whine about technical debt, how everything is "ancient" bla bla bla...
Yet don't want to update libraries / stuff unless one explicitly rams an klingon pain stick up their arse because one is very very very very tired of lame excuses.
Even better example - and reason for the rant - new microservice.
They honestly started with JDK 8.
Looking at the dependencies is like walking in a museum...
OWasp Dependency check?
Lot's of 7.5s and greater (NVD score).
How brain fucked ignorant can one team be?!!!
Let alone that that thing - despite being just a skeleton project - has already 178 dependencies.
I don't want to look at the build files, I'll guess I'd turn to Freddy Krueger otherwise...
But really - why whining all the time like you have a clit / arsehole full of sand and then starting a new project with an obviously copy pasted graveyard skeleton?!5
I think I executed more than 250 build plans in > 12 build plans.
I don't want to click anymore.
More than drunk.
Unclogging a workflow that stretches > dozen porjects, from building to analysis of build (static / lint check etc.)… Deployment.
Touching one thing and everything falls apart.
Thus small changes, fixing the rest of workflow, testing ...
Going now since 1 1/2 weeks. Possibly another or two weeks more.
It is soooooooo boring tedious annoyingly frustrating slowmotion shit
Funniest game after a week of torture:
How many cocktails per hour are needed to forgot partially the mayhem?
3 and going strong currently.
Seeeex on the beach, my saviour, my Adonis, my strong dicked butt pecker.4
Someone: Let's use this XY.
Where XY is a new service, like Slack / Jira / ...
Yeah. Let's just add another workflow without any planning, thinking etc..
Not that we did that a thousand times already and it always ended in an ever growing pile of shit.
These pointless discussions...2
Had 2 days of vacation. Theoretically (plus weekend, plus 2 days) 6 days.
Worked today… At Saturday.
Some administrators forgot to properly check bandwidth limitations....
We had a major version upgrade of some server software at Monday.
Guess why I got called...
Of course it MUST be the software upgrade.
It couldn't be the new hardware that was setup 2 weeks ago and on which a lot of "important" VMs were migrated.
*eyes roll inside till only white is visible*
The even more annoying thing is that it wasn't that hard to figure out.
Looking at monitoring, we had spikes on 20 Gbit/s (roughly 2.x Gigabyte/sec - Ethernet) connection of some server at roughly 1.9 plus Gigabyte/sec.
IO latency spikes that made the graph look like a heartbeat EKG with severe tachycardia...
*additionally to white eyes starts cursing in reverse latin*
Incompetent admin answer: Booboo that can only be your fault - the developers must investigate.
Me (just a tad more polite): Meep Meep mother fucker, get your shit together. If the software would eat that much, the network would be a niece chunk of charcoal. Plus the time (sending instead of links to monitoring pictures… guess the lazy fucktard who's brain is a vacuum didn't even bother to check it)...
Incompetent admin: It starts at the same time. Always.
After wasting roughly another hour of time discussing with him, I just hanged up the video call.
Called someone I knew from the admin department and turns out that - drumrolls please - the incompetent admin was someone who got recruited 3 months ago…
*turning into antichrist*
I then had a not so polite discussion about how the only competent people could take days off (all except incompetent admin were on vacation) and the seemingly incompetent fresh recruit - who by the way NEVER mentioned this - was the only one left of the admin department. Which would be bad alone, but no - he even got the 24/7 emergency support role for the whole weekend.
Sometimes this company and HR especially notoriously drive me insane...
Guess next week there will be some HR barbecue.
But yeah. After a lot of raging around we nailed it down to the traffic of backups and could fix it.
Roughly 4 hours of analysis, communication, raging and hatred.
Just one hour implementing shit.
To give my own old rant here a bit more background:
Cloud solutions offline for some customers, a bit longer than a week...
Bamboo release fucked up...
They've withdrawn the new release 8.2.0 and instead went for 8.2.1 ...
Well just because you had to give everyone admin rights in an minor release as they fucked up permissions, who cares.
Dear Atlassian Support,
In my life I had a lot of experiences...
But your software manages to replace all these experiences with a unique feeling of depression, hatred, anger... Only negative emotions.
Not once have I said anything good about your software - not once in > 5 years.
Whenever your chum bucket of mismanagement and misanthropy stops working, it's never the fault of the end user, the administrator or someone else.
It's entirely your fault.
Fucked up upgrades, lack of documentation, catastrophic handling of logging, lack of support of current database systems, lack of proper migration and clean up of plugins, ....
I could go on. But it's really just and endless tirade.
I wish I could stop management for even giving you money for the pile of poo you call software, but sadly they don't listen.
But there's hope on the horizon.
Thanks for making people go cloud only.
No one wants that.
It would mean entrusting that pile of poo to the craptastic hands of your irresponsible people.
No one really wants that.
Not even management who blindly paid the license fees all the times.
Thank you for your cloud only movement.
Maybe we can finally find an alternative and I can finally start a therapy for the PTSD I have thx to your software.3
For frigging fucking fucks sake, universe put the cactii away.
I HATE THE UNIVERSE.
I FUCKING HATE IT.6
I'd like to ask: What's trending at the moment instead....
Either I'm old and senile and missing something, or there is not really sth new.
Okay, JS might be crapping out new frameworks in their common "Not invented here" diarrhea....
But otherwise? What's really new?
I don't really know. I'm not only thinking about languages and stuff, but even in hardware there ain't really a big thing going on in my opinion.
Hab ich wat verpennt?
(Have I overslept?)
We had an interesting and frightening discussion regarding NGINX, as it is russian software today and that a new trend of a true, actively developed webserver is severely lacking... Apache looks semi dead and most other niche webservers, too.
That's all I've seen as a "trend" discussion in the latest time4
Today in the land of cactii.
A broken update, a sever that dies for unknown reasons violently, an HAPRoxy bug which fix uncovered another bug in an application which needed another bugfix, an meeting at 12 where I was already drinking beer, yet another dev who doesn't know how to solve problems on their own...
Nah. It's fine. I guess my next vacation will be to the looney bin.
The thought of pure uninterrupted quietness gives me a raging hardon.1
Putting mobile phone to DnD.
Putting on Bluetooth headsets with ANC, blasting some good shit music.
Violently cooking or desperately ordering food at the good restaurants.
While waiting, grinding some coffee beans, making fresh coffee or some nice tea with milk.
Laying on the sofa in a food induced coma, turning on any streaming service with the real bad shitty movies.
Hentai Kamen, The Machine Girl, ...
Anything thats either pure groteske bullshit or that doesn't require more than a braincell cause it appeals either to the violent or complete nonsense side of my brain.
Last but not least, a few cold beers.
ANC headset stays on, just switching from music to tv - shutting out all the outside noise.1
I've read the docs but my tired brain overrided an important detail.
"By default, HAProxy Enterprise will serve these pages only if it initiated the error itself. For example, it will return the page for a 503 Service Unavailable error if it can't reach any backend servers."
I had _the_ return part for interception of the error page from the backend added, not the default override for the error page of HAPRoxy itself.
Took me 4 hours, crying, madness and screaming to realize it.
This week is really wringing the last bits of the gooey slime what should be my brain out...
Another fun part is that I mistakenly thought the delimiter for multiple strings to an ACL comparison is a comma... It's a whitespace.
acl is_evil hdr(host) -i one,two is wrong.
acl is_evil hdr(host) -i one two is right.
I used to write HAPRoxy configurations blindly, today it was more like writing two lines of codes 100000000 times and still doing it wrong TM.
I need new brain.
Anyone got an offer?3
The universe has taken a cactus.
It proceeded to gift the cactus with a toxin that greatly enhances the stimulus of pain.
After the universe watched it's miraculous creation it decided to shove it up so far my arse that my gag reflex turned on and I puked a lot of cactus.
Didn't sleep well, weekend hardware migration finish, today an old server got moved.
Some part, most likely the redundant PSU, had a short circuit - decided to take the switches out... Which are the only non redundant hardware...
There was only one critical system in the whole rack, that was one redundant firewall.
Guess what happened..... Naaaa?
For whatever reason, the second firewall didn't kick in, so large part of internal network unreachable as VPN was on the firewall.
That's not cactus level yet.
Spontaneously a large part of the work at home crew decided to call, cause getting an email wasn't enough.
So while all the phones were ringing and we had the joyful fun to carefully take apart a whole rack to check for possible faulty wiring / electric burns / hardware damage and getting firewall up and running again...
Some dev decided to run a deployment (doable as one of the few working at the company at the moment -.-).
I work from home, but we had a conference phone call running the whole time so I could "deescalate" and keep others up-to-date. So me on headphone with conference call, regular phone for calls, while typing mails / sms for de-escalation.
Now we're reaching cactus level, cause being tortured by being annoyed out of hell by all telephone ringing, the beeping of UPS (uninterruptible power supplies), the screaming of admins from the server room and the roaring of air coolers…
Suddenly said dev must have stood in the midst of the chaos… and asked for help cause "the deployment broke, project XY is offline"...
I think it was the first time since years that I screamed at the top of my lungs.
Bad idea (health issues)… but oh boy was it a pleasure to hear my own voice echo through the conference speaker and creating an echoic sound effect.
It was definitely worth coughing out my loungs for the next hour and I think it was the best emotional outburst ever.
I feel a bit sorry for the dev, but only a tiny bit.
After the whole rack thing, the broken deployment fixing and the "my ears are bleeding and I think I will never be able to talk again" action...
We had to roll out several emergency deployments to fix CVEs (eg libexpat).
This day was a marvelous shit show.
I will now cry myself to sleep with some codein.1
A becomes B
B becomes C
C becomes A
D becomes B
E becomes A
Now add real hostnames... Make this list longer (roughly 15-18)
Add resource calculation, migration of VMs, organizing new hardware, removing and rebuilding hosts, etc.
I think my brain is permanently damaged and cannot be repaired.
Hardware migration finally over tomorrow.
I really won't miss the fuckton of Excel lists, constant speaking mistakes, having sore fingers from mutilating the desk calculator etc.
I'm too tired to be happy. But... It's over.1
Someone's guts will be torn out tomorrow and put up on a nice clean razor barbed wire ...
I was wondering what the fucking fuck messed up my brain - till I realized that some dev mixed up the timezone on one of our servers. Dunno how the dev managed it - but the end result was not funny.
Due to the difference in time strings the newer backup had an older timestamp - and vice versa.
Which - when you want to do mass clean up and migration - is a very fucked up thing.
I had to manually check dozens of backups to make sure I got the right ones...
-.- knife goes in, gut goes out. Thx Bart Simpson.8
TM: Hey, do you have a moment?
Me: not really, I'm already overtime and have enough work for the whole year.
TM: Yeah, we know. Just a quick meeting to discuss something awkward.
Later that day:
TM: Yeah. To make it quick - we're confused and bit dissatisfied with how project X turned out. The staging server is blazing fast, but the devs machines seem to be extremely slow... Some devs complained.
Me: No wonder. I said from the beginning that the devs shouldn't do X and Y, and that the dev machines need to be redone after staging is done - as we need to gather hands on experience first, cause no one could explain to me what resources the project actually needed.
TM: Oh. I wasn't aware of that.
Me: I guessed so. You were on vacation at the beginning and I didn't had the time to lead another team...
TM: Yeah... So the dev machines get replaced?
Me: They _could_ be replaced, but the devs would need to reset up their environment, as I and won't transfer the environment of the dev user.
TM: Ah... So they would have to retransfer their personal modifications, if they made any?
Me: Yes. As always, the basic setup just provides the necessary services, settings etc. - stuff like remote IDE settings on the machine, configuration etc is left out and we don't transfer it as it is usually too much of a hassle and risky, as every dev does have his / her own preferences, and we don't want to support every possible configuration out there.
TM: Just out of curiosity... Staging was ready like... Last year?
Me: Beginning of December, yes.
Me: The jolly of having a kinder garten full of toys that no kid wants to clean up...
TM: No comment. The kinder garten Kids might make me a Pinata otherwise.
Me: If only they'd fill us with chocolate first instead of just beating us.
Tales of lazy devs, to be continued...3
Bring the fun and curiosity back.
School education? Mostly rinse and repeat, learn from heart and do as you are told.
First job? Take these bread crumbs, shit out gold ingots, please.
There are few who had either very kind and gifted teachers / persons in their life or had a strong will / desire to learn by / for themselves - but it's hard to combine fun and curiosity with the - most of the time - very harsh reality and environment we live in.
I'd really wish that it would get back to fun and curiosity and not the endless myriad of bitching, hissing and fighting it usually is.
What I find most tiresome in education is the overflow of information with no value - most content is outdated, wrong, harmful, not precise and especially not helpful.
Thinking about good education I've got very fond memories of hanging out in IRC chats, talking with people who were "ancient" (la me 15-20, them 40 plus ;) ) and not being "shood" away, but rather getting fed by book recommendations, hints, appointments when they had more spare time to explain in private IRC sessions etc.
The atmosphere was always a "we might not have time for it, but we'll try and don't worry if you don't understand it".
When I'm trying to find information today... It's really 90 - 95 % filtering, 4 % try and error, 1 % finding what I need.3
Few days off cause the month was like an motor race on drugs while having an heart attack.
I slept two days sitting either on the couch drooling or actually sleeping.
Yesterday I managed to wash and hang up a full cellar room of clothing... And cooked enough for a 5 head family with 3 teenagers.
Today parents drove over, brought even more food.
I started an telephone chain and now roughly 3-4 friends come over tomorrow to take the food and distribute it among their parents.
It's ... Irritating... How I need to have stress to reduce my stress level and feel more relaxed.
I'm glad I'm having a few more days off... I think I'm now in a near coma state due to eating.
Maybe I should go shopping tomorrow.... 🤔😆
So. Here's a cooking thread.
Please behave nicely, get ya favorite recipes out or your "my brain is melting and I randomly chop sueyed hot cooked this stuff together and it was awesome" creations.7
I watched today one of our devs working in Windows with a Docker Environment.
I think I'm pretty insensitive regarding pain, horror and morbid stuff.
But damn. I really needed to turn off the stream or else I'd walk to the company and rip his fucking workstation out of the server rack to put it out of his misery...
Errors? ignore them....
Weird python messages? Ignore them...
wild copy pasta between notepad++ containing shell commands and a git bash... Per mouse context. Yes. Move the cursor, mark the text, right click, copy, go to terminal, right click, paste.
Understanding of whats happening. Zero. Like literal zero.
He was wondering why there were strange characters when he pasted log output in a text file...
My question: How do you think colored text works in a terminal environment?
was answered by : "Don't know, never thought about it. But don't think this has something to do with the weird characters?"
I don't wanna talk about the rest.
Retarded humanity can please kindly kill itself so the intelligent above average nice people can live in peace...
The meeting was 2 hours. I drank 5 bottles of beer after it in1 hour and I'm please to announce I'm forgetting large parts of what has happened.
Okidoky you lil fucker where you've been hiding...
*streaming tcpdump via SSH to other box, feeding tshark with input filters*
Finally finding a request with an ominous dissector warning about headers...
Not finding anything with silversearcher / ag in the project...
*getting even more pissed causr I've been looking for lil fucker since 2 days*
*generating possible splits of the header name, piping to silversearcher*
*I/O looks like clusterfuck*
Common, it are just dozen gigabytes of text, don't choke just because you have to suck on all the sucking projects this company owns... Don't drown now, lil bukkake princess.
*half an hour later*
Oh... Interesting. Bukkake princess survived and even spilled the tea.
Someone was trying to be overly "eager" to avoid magic numbers...
They concatenated a header name out of several const vars which stem from a static class with like... 300? 400? vars of which I can make no fucking sense at all.
Class literally looks like the most braindamaged thing one could imagine.
And yes... Coming back to the network error I'm debugging since 2 days as it is occuring at erratic intervals and noone knew of course why...
One of the devs changed the const value of one of the variables to have UTF 8 characters. For "cleaner meaning".
Sometimes I just want to electrocute people ...
The reason this didn't pop up all the time was because the test system triggered one call with the header - whenever said dev pushed changes...
And yeah. Test failures can be ignored.
Why bother? Just continue meddling in shit.
I'm glad for the dev that I'm in home office... :@
TLDR: Dev changed const value without thinking, ignoring test failures and I had the fun of debunking for 2 days a mysterious HAProxy failure due to HTTP header validation...
Fought fire with fire.
I used regexes with quoting necessary regex bits inside to replace the regexes that tortured me.
I think I need a lot of pain meds.
Or why do I feel pain and numb at the same time?6
A question or more discussion / looking for feedback.
Let's say we have
- Application Routing
-- API Gateway
Then lots of containers.
Would you use host based routing in the containers?
It seems wrong to me, as in my opinion it breaks the boundary the API gateway provides.
Maybe someone can share his experience, I'd be delighted.5
Debugging a request that got lost in a myriad of containers of a scaled application....
It wouldn't be worth a rant if there wasn't some kinky SM stuff in it, wouldn't it?
Regexes. The fucker who wrote a lot of the NGINX (🤢) configuration decided to use the Perl Regexes with named group matching. A lot.
So now I have to fight wild variables supposedly coming from nowhere (as they stem from the named groups)… fucking single location redirects instead of maps.... A d have to write an explanatory documentation while going down the rabbit hole of trying to find out where the fuck that shitty frigging bastard redirected wrong.
I really wish I could eradicate the person who wrote this shit....1
Reverse engineering an applications internal object model and creating an database model for it...
The reason: Several versions of application exist, each deliver flat data by rest. The data is a complete potpourri of several different entities. *yaaaay*
Eg. an example fictional call (real call and data would get me in trouble I think....)
get_fiscal_report returning the fiscal data for _several_ companies, the companies _subsidiaries_ and the respective _segments_ for a _year_ with a key value enumeration.
So it's an happy fuck up of N:N associative data that usually would be a hierarchical relationship...
Year - Company
Each Company has subsidiaries
Each Company subsidiary has segments
Each segment has a fixed enumeration of keys
Each key has then the monetary value (e.g. 'operating_income' - 155_000 US-$)
Example is made up, but my data contains exactly such a lovely nested hierarchical data flattened and misnamed to a point where it's close to garbage.
I had now 6 days of untucking this mess to a usable database representation...
Sprinkling Unique Keys everywhere...
Running persist script...
Running persist script...
Running persist script.
Starting sacrificing interns...
I need a new brain and a format of my soul.
Reverse engineering proprietary software is really an morbid adventure.1
Debugging TLS failures.
With the funny certstore cause "we need to do this by ourselves".
Fucking shitty broken pile of cunt code.
At least the debugging output is good.
As much as I love TLS, debugging it is a nightmare and when a programming language like Java decides to wrap it, it becomes Ctulhu.
- TLS Library
-- TLS Certificate Chain
-- JDK SSL Handler
--- JDK Certstore
---- Java Library Abstraction, eg. WS SSL
Joyfully fingering of a tentacle arsehole.2
Going through the list of projects:
Absolutely nope ...
Burn it Nope.
Don't touch this.
It's so fluffy, I will die.
Wow. If this was a fungus, it could talk by now.
What the hell was that...?
Uh. Ehm. So old it's already done. Oopsie.
2 hours later...
Well. Backlog is sorted.
Conclusion: I just don't work anymore.
I think it's time to add a field "pain level" to Jira.
And I doubt that the backlog contains any ticket less than pain level 8 / 10. -.-
But yey. I ordered beer.2