Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple APILearn More
Search - "haproxy"
Hi lil puppies what's your problem?
Have you eaten something wrong....
*proxy happily eats requests and answers correctly*
Hm... Seems like you are...
*proxy vomits dozen of requests at once*
... Not okay.
Ok.... What did u you get fed you lil hellspawn.
TLS handshake error.
Thousands. Of. TLS. Handshake. Errors.
*checking autonomous system information*
Yeah... Requests come from same IP or AS. Someone is actively bombing TLS requests on the TLS terminator.
Wrong / outdated TLS requests.
Let's block the IP addresses....
*Pats HAProxy on the head*
*Gets more vomit as a thank you no sir*
I've now added a list of roughly 320 IP adresses in 4 h to an actively running HAProxy in INet as some Chinese fuckers seemingly find it funny to DDOS with TLS 1.0... or Invalid HTTP Requests... Or Upgrade Headers...
Seriously. I want a fucking weekend you bastards. Shove your communism up your arse if you wanna have some illegal fun. ;)11
You mother fucking piece of shit.
Whoever taught you programming should be removed from history.
And whatever form of intelligence you claim to possess, let me assure you: breathing is the limit of it.
Some of the projects I'm working on are really the epitome of "YOLO let's turn the poopomat machine on in diarrhea mode".
The worst: I cannot really give examples.
I've seen the last days everything.
(bash scripting, docker, services like nginx /haproxy/...)
Eval as an template generator in bash...
Declaring an whole environment in an Dockerfile, that should never be used as it is only necessary for building... But not checking if an env file is provided, so the whole thing can blow up spectacularly.
A nearly 1k long bash calculator for system limits, reading out all kinds of stuff from /proc and /sys, seemingly partially stolen from NGINX Docker.
Declaring and starting an own DNS Server to bypass the Docker DNS service inside an docker container.
Mkfifo fun for creating several stdout and stderrs for seemingly no reason...
Actively not using bash, instead of creating shell only functions to emulate bash...
I could go on.
But really. I'm getting too old for this shit.3
>Client complains about a 30 minute downtime around midnight
>Client also pays only for a single VM on a HV that they don't even own themselves
>Replies with an offer of how to make the setup more resilient, going from 1 VM to 2 LBs/FE loadbalanced through BGP, and distributing traffic through HaProxy onto 2 BE machines that in turn talk to a Postgres Cluster with RepMgr for dynamic failover.
>No reply so far
Is it just my random madness...
Or do you sometimes picture yourself in a fictional comic / movie / whateva...
Had this feeling today.
Burned a database down, grilled 2 terabyte of data, deleted ~ 500 elasticsearch indices.
Then I chopped an haproxy loadbalancer into 6 seperate machines, because noone likes to read ~ 2.5 to 3 k of lines.
And I guess now I'm doing some backups of elasticsearch before the second round of flamethrower madness starts.
It's somehow very satisfying to just destroy everything.3
With a recent HAProxy update on our reverse proxy VM I decided to enable http/2, disable TLS 1.0 and drop support for non forward-secrecy ciphers.
Tested our sites in Chrome and Firefox, all was well, went to bed.
Next morning a medium-critical havock went loose. Our ERP system couldn't create tickets in our ticket system anymore, the ticket systems Outlook AddIn refused to connect, the mobile app we use to access our anti-spam appliance wouldn't connect although our internal blackboard app still connected over the same load balancer without any issues.
So i declared a 10min maintenance window and disabled HTTP/2, thinking that this was the culprit.
Nope. No dice.
Okay, i thought, enable TLS 1.0 again.
Suddenly the ticket system related stuff starts to work again.
So since both the ERP system and the AddIn run on .NET i dug through the .NET documentation and found out that for some fucking reason even in the newest .NET framework version (4.7.2) you have to explicitly enable TLS 1.1 and 1.2 or else you just get a 'socket reset' error. Why the fuck?!
Okay, now that i had the ticket system out of the way i enabled HTTP/2 and verified that everything still works.
It did, nice.
The anti-spam appliance app still did not work however, so i enabled one non-pfs cipher in the OpenSSL config and tested the app.
Behold, it worked.
I'm currently creating a ticket with them asking politely why the fuck their app has pfs-ciphers disabled.
And I thought disabling DEPRECEATED tech wouldn't be an issue... Wrong...
Change haproxy config on the remote machine so that every developer gets rerouted to an older version of the development app.
Tried out linkerd recently, it's pretty cool when you want to get dynamic routing based on entries in a service registry ^^ originally planned to just dynamically generate configs for haproxy but this makes it way easier ^^
The jolly of unriddling multiple DNS zone overrides to a static, single IP of a HAProxy loadbalancer which acts as a router and has domain based backend association rules, but frontend based CORS overrides.
My eyes are bleeding, my brain is defeated and I think I need more gaffa type to put together the pieces of what some puny humans call a soul.
Nooooo.... Why on earth do you do that.
Looking at the sysctl settings someone took a road trip to Google and stackoverflow and just copy pasted every mother fucking stupid bullshit bingo inside it.
Half of this doesn't apply as the kernel version doesn't even support it anymore (for good reasons) or makes sense as these settings have NOTHING not even REMOTELY to do with the servers hw setup.
If you have no fucking clue what you do, ram the keyboard up your arse till you enjoy it.
But stay the fuck away from administration and the fuck away from anything that carries responsibilities.
Joyful task today: unclogging old failing Haproxy setups while being busy with 3 other tasks.
And if you wanna know why they're failing and it needed to happen today... Weeeell....
They restarted. And today they decided to restart so fast people finally noticed it.
Cause yeah. They did that the last fucking years every few hours. Now every 5 minutes.
:@ :@ :@ :@ :@ :@
Another part of messy network gone.
Caching fucked me hard....
Isn't it just lovely that nowadays you need to nearly wipe a machine to get it from claiming stale data....
And thanks to DNS, HAProxy -/ service names / ... I think I know now why the curse of babel is so powerful.
When you have to think for 2 mins to make sure you've set the zone's right, cause otherwise you need to ProxyJump with SSH through more tunnels than imaginable (VPN/HO) to fix possible caching on several DNS servers.... You'll realize that it's russian roulette with too much bullets. :(
And If a monitoring service asks another monitoring service for status information which asks the first monitoring service which then asks the second monitoring cause you were too late...
You'll get very funky monitoring statistics.
Too slow, had to nuke it (mismatched a DNS name, the second monitoring service should have been a service node).
I think I've had more near death scenarios in the last 2 weeks than I like.
Hopefully I'll never have to do that again.
(Splitting and reordering a few dozen VLANs, assigning proper DNS names, loadbalancer migration....)