Another part of messy network gone.

Caching fucked me hard....

Isn't it just lovely that nowadays you need to nearly wipe a machine to get it from claiming stale data....

And thanks to DNS, HAProxy -/ service names / ... I think I know now why the curse of babel is so powerful.

When you have to think for 2 mins to make sure you've set the zone's right, cause otherwise you need to ProxyJump with SSH through more tunnels than imaginable (VPN/HO) to fix possible caching on several DNS servers.... You'll realize that it's russian roulette with too much bullets. :(

And If a monitoring service asks another monitoring service for status information which asks the first monitoring service which then asks the second monitoring cause you were too late...

You'll get very funky monitoring statistics.

Too slow, had to nuke it (mismatched a DNS name, the second monitoring service should have been a service node).

I think I've had more near death scenarios in the last 2 weeks than I like.

Hopefully I'll never have to do that again.

(Splitting and reordering a few dozen VLANs, assigning proper DNS names, loadbalancer migration....)

Add Comment