Guess I need a break from JS

So for those of you keeping track, I've become a bit of a data munger of late, something that is both interesting and somewhat frustrating.

I work with a variety of enterprise data sources. Those of you who have done enterprise work will know what I mean. Forget lovely Web APIs with proper authentication and JSON fed by well-known open source libraries. No, I've got the output from an AS/400 to deal with (For the youngsters amongst you, AS/400 is a 1980s IBM mainframe-ish operating system that oriiganlly ran on 48-bit computers). I've got EDIFACT to deal with (for the youngsters amongst you: EDIFACT is the 1980s precursor to XML. It's all cryptic codes, + delimited fields and ' delimited lines) and I've got legacy databases to massage into newer formats, all for what is laughably called my "data warehouse".

But of course, the one system that actually gives me serious problems is the most modern one. It's web-based, on internal servers. It's got all the late-naughties buzzowrds in web development, such as AJAX and JQuery. And it now has a "Web Service" interface at the request of the bosses, that I have to use.

The programmers of this system have based it on that very well-known database: Intersystems Caché. This is an Object Database, and doesn't have an SQL driver by default, so I'm basically required to use this "Web Service".

Let's put aside the poor security. I basically pass a hard-coded human readable string as password in a password field in the GET parameters. This is a step up from no security, to be fair, though not much.

It's the fact that the thing lies. All the files it spits out start with that fateful string: '<?xml version="1.0" encoding="ISO-8859-1"?>' and it lies.

It's all UTF-8, which has made some of my parsers choke, when they're expecting latin-1.

But no, the real lie is the fact that IT IS NOT WELL-FORMED XML. Let alone Valid.

THERE IS NO ROOT ELEMENT!

So now, I have to waste my time writing a proxy for this "web service" that rewrites the XML encoding string on these files, and adds a root element, just so I can spit it at an XML parser. This means added infrastructure for my data munging, and more potential bugs introduced or points of failure.

Let's just say that the developers of this system don't really cope with people wanting to integrate with them. It's amazing that they manage to integrate with third parties at all...

Imagine if a structural engineer whose bridge has collapsed and killed several people calls it a feature.

Imagine if that structural engineer made a mistake in the tensile strength of this or that type of bolt and shoved it under the rug as "won't fix".

Imagine that it's you who's relying on that bridge to commute every day. Would you use it, knowing that its QA might not have been very rigorous and could fail at any point in time?

Seriously, you developers have all kinds of fancy stuff like Continuous Integration, Agile development, pipelines, unit testing and some more buzzwords. So why is it that the bridges don't collapse, yet new critical security vulnerabilities caused by bad design, unfixed bugs etc appear every day?

Your actions have consequences. Maybe not for yourself but likely it will have on someone else who's relying on your software. And good QA instead of that whole stupid "move fast and break things" is imperative.

Software developers call themselves the same engineers as the structural engineer and the electrical engineer whose mistakes can kill people. I can't help but be utterly disappointed with the status quo in software development. Don't you carry the title of the engineer with pride? The pride that comes from the responsibility that your application creates?

I wish I'd taken the blue pill. I didn't want to know that software "engineering" was this bad, this insanity-inducing.

But more than anything, it surprises me that the world that relies so much on software hasn't collapsed in some incredible way yet, despite the quality of what's driving it.