Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
I feel you on that one.
But in all fairness: Poking with network issues is an ungrateful and shitty task.
Especially if there are multiple HTTP stacks involved and one has to do the task of literally finding the needle in a haystack.
Been there, done that, never want to do it again.
Rather get 3 kicks in the balls and testicular torsion than sifting through gigabytes of logs looking for the reason. -
@IntrusionCM I had an IT guy telling me maybe openssh is broken and not their cert setup.
I had to find a completely convoluted way to prove to them that it's them -
-
@IntrusionCM he probably meant openssl
-
Im a software guy working on a network team. I have a little insight.
1) Everything is a network issue first. Web page is slow? "Sometimes wrong with the network today". They don't believe you anymore.
2) network engineering does not have the o11y that software and system admins have now. It's harder to verify what's going on, so they truly don't know.
3) if it is their problem, it's a tonne of work to investigate and fix, and they're biased to avoid it at all costs.
4) they're overworked with compliance nonsense that they hate themselves. -
@IntrusionCM Sure, it's a shitty task, and it's really tough - but it's a shitty task that's their responsibility, and one they're paid to do. We all have crap we have to diagnose and debug at some point - it's part of the industry.
I'd happily wade through the logs myself at this point and do their job for them - but they won't give me the required access. -
@AlmondSauce
Then you're a rare sight for sore eyes.
Most of "my" devs need to get a battery clamped to their ears to get them to take a look at logs.
I absolutely hate it when people have no access to "telemetry" (be it logs, metrics, whatever)... But even more when the devs don't use it. -
@lungdart Totally get and appreciate that - but I feel I've exhausted everything I can possibly do this side, and I've got nothing back. I've even got to the point where I've shown TCP packets over a certain size are more likely to be dropped when going through a certain network path, and anecdotally found a bunch of people having the same issue with buggy firmware in a switch that they're using - but I just get told "can you try from different IP address and do tracert on the same and give us result".
-
@IntrusionCM I've learnt the hard way, many times, that if I want a job done properly I often need to just do it myself - even if it's not my job role.
-
@IntrusionCM @AlmondSauce
speaking of telemetry, I've been nagging our infra folks to provide me access to ANY infra telemetry to have a useful correlation point.
No luck yet. The request has been frozen for months now. I feel like it's gonna get closed after another year due to inactivity...
See, it's a two way street. We can't help to relieve the infra guys' w/load if they don't bother to give us tools to do it ourselves. -
@IntrusionCM yeah. openssl. I used openssl to show our it guy the difference between the environment that's working and one that isn't and he said it's openssl that's broken o.o
I think the problem was missing an intermediate cert or something of that sort.
That's the kind of IT folks I work with. Everyone else praise them for fixing their basic MS office issues while I had to fight around permissions and accesses to figure out the IT issues with our software without the tools they had. -
@netikras yeah.
That's a part I really cannot understand.
Unless we're talking about Windows Server OS where permissions are a nut job.
Cause with Linux, permission handling makes it really trivial to add a dedicated user with limited access who could read the logs.
Not much work, not hard work.
Luckily I never had to persuade admins to do this... I'd guess if an admin would start whining about how much work it is, I'd really get angry and whack some sense in them. -
Fck this one hits home. It is always everything BUT the fucking network ain't it?
Some net tech sent a user a while ago because an internal application was not working for said user. Dude didn't even bother to test anything out, he just heard "I can't access the <name of internal app> and sent the user my way.......shit like that happens all the time. It is never the network according to them.
Switch monkeys
What is it with networking guys refusing to do any kind of fault finding? Pretty much everywhere I've worked they seem to be overpaid address hogs who occasionally want everyone to be proud of them for installing a new switch.
Currently seeing a production issue that's clearly due to spikes in packet loss on a certain part of the network - but oh no, it's always "our tests are fine", "we can establish a route no problem", "this is an application level issue", etc.
No you morons, when a dozen unrelated applications hosted on different cloud services fail because none of them can contact anything in your particular subnet in your data center at the same time, it's a damn networking issue. Sort it out.
rant