The API they gave me.

I am working on a project that integrated with Microsoft Dynamics. The D365 team changes the data and API too often. I keep munging the data over and over so that it fits view models.

Yesterday I had a technical discussion with someone that the company installed to coordinate building technical solutions that get shared between projects. This is a problem across all of our Dynamics projects and is a technical one than needs coordinated approach.

The management heard that I am “doing repeated work because the data isn’t stable.” So they decided that it must be a problem with the project manager. The executive decides to use it as a reason to fire him. Which shows me that I can’t talk to even a technical person without risk of project chaos.

After the conversation but before the firing I get an offer letter from another company. I plan on taking it to get away from this crazy company. The project is going to lose their key web developer and also the project manager.

This executive seems to love firing project managers. My other project is on its third PM. My trust for the upper management is basically gone. I can’t even discuss the technical hurdles with a technical person without someone getting fired. I’m so ready to be out of this zoo.

The only downside of leaving is that I won’t have as many stories to tell on devRant.

So for those of you keeping track, I've become a bit of a data munger of late, something that is both interesting and somewhat frustrating.

I work with a variety of enterprise data sources. Those of you who have done enterprise work will know what I mean. Forget lovely Web APIs with proper authentication and JSON fed by well-known open source libraries. No, I've got the output from an AS/400 to deal with (For the youngsters amongst you, AS/400 is a 1980s IBM mainframe-ish operating system that oriiganlly ran on 48-bit computers). I've got EDIFACT to deal with (for the youngsters amongst you: EDIFACT is the 1980s precursor to XML. It's all cryptic codes, + delimited fields and ' delimited lines) and I've got legacy databases to massage into newer formats, all for what is laughably called my "data warehouse".

But of course, the one system that actually gives me serious problems is the most modern one. It's web-based, on internal servers. It's got all the late-naughties buzzowrds in web development, such as AJAX and JQuery. And it now has a "Web Service" interface at the request of the bosses, that I have to use.

The programmers of this system have based it on that very well-known database: Intersystems Caché. This is an Object Database, and doesn't have an SQL driver by default, so I'm basically required to use this "Web Service".

Let's put aside the poor security. I basically pass a hard-coded human readable string as password in a password field in the GET parameters. This is a step up from no security, to be fair, though not much.

It's the fact that the thing lies. All the files it spits out start with that fateful string: '<?xml version="1.0" encoding="ISO-8859-1"?>' and it lies.

It's all UTF-8, which has made some of my parsers choke, when they're expecting latin-1.

But no, the real lie is the fact that IT IS NOT WELL-FORMED XML. Let alone Valid.

THERE IS NO ROOT ELEMENT!

So now, I have to waste my time writing a proxy for this "web service" that rewrites the XML encoding string on these files, and adds a root element, just so I can spit it at an XML parser. This means added infrastructure for my data munging, and more potential bugs introduced or points of failure.

Let's just say that the developers of this system don't really cope with people wanting to integrate with them. It's amazing that they manage to integrate with third parties at all...

Power BI: wonderful tool, pretty graphics, and can do a lot of powerful stuff.

But it’s also quite frustrating when you want to do advanced things, as it’s such a closed platform.

* No way to run powerquery scripts in a command line
* Unit testing is a major pain, and doesn’t really test all the data munging capabilities
* The various layers (offline/online, visualisation, DAX, Powerquery, Dataset, Dataflow) are a bit too seamless: locating where an issue is happening when debugging can be pain, especially as filtering works differently in Query Editing mode than Query Visualisation mode.

And my number 1 pet peeve:
* No version control

It’s seriously disconcerting to go back to a no version control system, especially as you need to modify “live code” sometimes in order to debug a visual.

At best, I’ve been looking into extracting the code from the file, and then checking that into git, but it’s still a one-way street that means a lot of copying and pasting back into the program in order to roll back, and makes forking quite difficult.

It’s rewarding to work with the system, but these frustrations can really get to me sometimes