Spent 10 minutes starting to implement XML serialization in my old app... and then realize i already implemented it.

Transaction isolation levels can suck a dick.

Also,
serialization
doesn't
fucking
work.
(╯°□°）╯︵ ┻━┻

PHP arrays.

The built-in array is also an hashmap. Actually, it's always a hashmap, but you can append to it without specifying indexes and PHP will use consecutive integers. Its performance characteristics? Who knows. Oh, and only strings, ints and null are valid keys.

What's the iteration order for arrays if you use them as hashmaps (string keys)? Well, they have their internal order. So it's actually an ordered hashmap that's being called an array. And you can produce an array which has only integer keys starting with 0, but with non-sequential internal (iteration) order.

This array weirdness has some non-trivial implications. `json_encode` (serializes argument to JSON) assumes an array corresponds to a JSON array if its keys are consecutive integers in increasing order starting with 0, otherwise the array becomes a JSON object. `array_filter` (filters arrays/hashmaps using callback predicate) preserves keys, so it will punch holes in the int key sequence if non-last items are removed, thus turning arrays into hashmaps and changing your JSON structure if you forget to discard keys before serialization.

You may wonder how JSON deserialization works, then? There's a special class for deserialized JSON objects, `stdClass`. It's basically a hashmap too, but it's an object, not an array, and all functions that would normally accept arrays won't work with it. So basically its only use is JSON (de)serialization. You can even cast arrays to objects, producing `stdClass`.

Bonus PHP trivia:

Many functions return nonsensical values. `preg_match`, the regex matching function, returns 1 for success, 0 for no matches and false for malformed regular expression. PHP supports exceptions, so it could just throw one on errors. It would even make more sense to return true, false and null for these three cases. But no, 1, 0 and false. And actual matches are returned by output arg.

`array_walk_recursive`, a function supposed to recursively apply callback to each element of an array. That's what docs say. It actually applies it to leafs only. It will also silently accept object instead of array and "walk" it, but without recursing into deeper objects.

Runtime type enforcing is supported for function arguments and returned values. You can use scalar types, classes, array, null and a few special keywords. There's also a `mixed` keyword, which is used in docs and means "anything". It's syntactically valid, the parser will accept it, but it matches no values in runtime. Calling such function will always cause a runtime error.

Strings can be indexed with negative integers. Arrays can't.

ReflectionClass::newInstanceWithoutConstructor: "Creates a new class instance without invoking the constructor". This one needs no commentary.

`array_map` is pretty self-explanatory if you call it with a callback and an array. Or if you provide more arrays of equal length via varargs, callback will be called with more arguments, one from each array. Makes sense so far. Now, you can also call `array_map` with null instead of callback. In that case it treats provided arrays as rows of a matrix and returns that matrix, transposed.

So yeah XML is still not solved in year 2018. Or so did I realize the last days.
I use jackson to serialize generic data to JSON.

Now I also want to provide serialization to XML. Easy right? Jackson also provides XML serialization facitlity similar to JAXB.

Works out of the box (more or less). Wait what? *rubbing eyes*

<User>
<pk>234235</pk>
<groups typeCode="usergroup">
<pk>6356679041773291286</pk>
</groups>
<groups typeCode="usergroup">
<pk>1095682275514732543</pk>
</groups>
</User>

Why is my groups property (java.util.Set) rendered as two separate elements? Who the fuck every though this is the way to go?

So OK *reading the docs* there is a way to create a collection wrapper. That must be it, I thought ...

<User typeCode="user">
<pk>2540591810712846915</pk>
<groups>
<groups typeCode="usergroup">
<pk>6356679041773291286</pk>
</groups>
<groups typeCode="usergroup">
<pk>1095682275514732543</pk>
</groups>
</groups>
</User>

What the fuck is this now? This is still not right!!!

I know XML offers a lot of flexibility on how to represent your data. But this is just wrong ...

The only logical way to display that data is:
<User typeCode="user">
<pk>2540591810712846915</pk>
<groups>
<groupsEntry typeCode="usergroup">
<pk>6356679041773291286</pk>
</groupsEntry>
<groupsEntry typeCode="usergroup">
<pk>1095682275514732543</pk>
</groupsEntry>
</groups>
</User>

It would be better if the individual entries would be just called "group" but I guess implementing such a logic would be pretty hard (finding a singular of an arbitrary word?).

So yeah theres a way for that * implementing a custom collection serializer* ... wait is that really the way to go? I mean common, am I the only one who just whants this fucking shit just work as expected, with the least amount of suprise?
Why do I have to customize that ...

So ok it renders fine now ... *writes test for it+
FUCK FUCK FUCK. why can't jackson not deserialize it properly anymore? The two groups are just not being picked up anymore ...

SO WHY, WHY WHY are you guys over at jackson, JAXB and the like not able to implement that in the right manner. AND NOT THERE IS ONLY ONE RIGHT WAY TO DO IT!

*looks at an apple PLIST file* *scratches head* OK, gues I'll stick to the jackson defaults, at least it's not as broken as the fucking apple XML:

<plist version="1.0">
<dict>
<key>PayloadOrganization</key>
<string>Example Inc.</string>
<key>PayloadDisplayName</key>
<string>Profile Service</string>
<key>PayloadVersion</key>
<integer>1</integer>
</dict>
</plist

I really wonder who at apple has this briliant idea ...

So I've created this account specifically for this rant. I usually just browse anonymously.

I've recently been hired in a big company that is one of the biggest Microsoft users in the world and my essentially revolves on making it easier for our collaborators to work with SharePoint (and other ms software)

Never in my life have I hit that much of a roadblock. So for the past week I've been trying to integrate what Ms calls webparts. And to modify the default webparts Ms provides you need to their properties (or Metadata). Except here's the big problem these are NOT documented anywhere (unless I failed to find it, if you do know where it is documented please HMU), so I've found myself trying to reverse engineer the js scripts that are served with SharePoint to figure out what the webpart properties are called and what type of data they are! I've been going through endless github repos using the CSOM nuget package (it's the library everyone uses to interact with SharePoint) and I finally found out about this other library called PnP which is a wrapper around CSOM that makes it easier to use. That wrapper has a way for me to load existing page and look at the properties of existing webparts. So here I thought it was the end of my suffering and I could finally get an idea of what it should be. Turns out this method doesn't work because one of the dependencies it has has had breaking changes and they still updated it even though it breaks their code! So for the past two days I've been trying random combinations of key values with different data types and json serialization methods.

Oh and yeah I've also looked at all the http calls via the chrome network tab, the metadata is not served as an individual file but is computed by Ms servers when they're serving you their html files.

So uh yeah run from CSOM if you can..

Ah, the little subtle things we have to iron out as we progress from Junior Developer to Medior Developer.. things like:

- knowing the difference between a carriage return and a line feed (although having worked with analog typewriters helps) and later knowing that Unix-based systems and Windows NT-based systems implement it differently..

- knowing that serialization is important because not all computers interpret data the same way and some computers allocate 4 Bytes for a construct, others 16 Bytes.. and then we get the funkiness of transferring character sets between machines..

- knowing that a whitespace character is not only an actual space (as is known in ASCII as code 32). This one can cause even medior developers a headache, as in: why the fuck does this string function say that "hello I am a duck" and "hello I am a duck" are not the same?! Turns out then in the debugger that when you expand every character in the string you see that string1 contains 32 32 32 32 as usual.. but then string2 contains -96 -96 -96 -96 and you're like.. what the fuck..? Then you know you have to throw \\h regex at it. Haha.

- finalizing our objects and streams (although modern languages do that for us).. otherwise we have to do funky shit like trying to find what's locking a file, which is not so easy to figure out.

- figuring out why something won't work often requires you to not only break down the problem in smaller steps, to use a debugger, but sometimes it's even better to just create a proof of concept, slap some minimal code in there and debug that.. much easier.

- etc.

:)

JSON de/serializer for C++, recommendations?

I used boost serialization until now, and it's fine.
But I will need to send some stuff over REST protocol to my future-web-GUI.
I would like something that is easy to build, not bloated, and could handle class serialization.
Shoot!

The feature was to parse a set of fairly complex xml files following a legacy schema. Problem was, the way this was done previously did not conform to the schema so it was a guideline at best, which over the course of many years snowballed into an anarchy where clients would send in whatever and it was continuously updated per case as needed. They wanted to start enforcing their new schema while phasing out the old method.

The good news is that parsing and serialization is very testable, so I rounded up what I could find of example files and got to work. Around the same time I asked our client if they had any more examples of typical cases we need to deal with, and sure enough a couple of days later I receive a zip with hundreds of files. They also point out that I should just disregard the entire old set since they decided to outright cut support for it after all if it makes things simpler. Nice.

I finish the feature in a decent amount of time. All my local tests pass, and the CD tests pass when I push my branches. Once we push to our QA env though and the integration tests run, we get a pass rate of less than 10%.

I spend a couple of days trying to figure out what's going on, and eventually narrow it down to some wires being crossed with the new vs. old xml formats. I'm at a loss. I keep trying to chip away at it until I'm left with a minimal example, and I have one of those lean-back moments where you're just "I don't get it". My tests pass locally, but in the QA environment they fail on the same files.

We're now 3 people around my workstation including the system architect, and I'm demonstrating to the others how baffling and black magic this is. I postulate that maybe something is cached in my local environment and it's not actually testing the new files. I even deleted the old ones.

"Are you sure you deleted the right files?"

"Duh of course -- but let me check..."

Fucking garbage piece of shit microsoft httpclient

identical request works in node!

identical request works in postman!

but noooooooo httpclient, you have to add the content length on the content itself, can't add authorization header except through special way, serialization is wrong bunch of shit pile of shit no working shit

I'm such a fucking idiot

I'm setting up an api and to prevent unwanted fields or circular dependencies from showing up I define what fields should appear in a few serialization yaml files.

These files define what fields should appear in a given context. The default context for every field is to always show the id, and only a call to /posts will give you all the fields of the posts for example. This means that if you retrieve a comment with a linked post, the post will only show up as an id, but the comment will have all its fields.

I've been struggling with a stupid problem for 2 hours, I could verify that the yaml files were loaded in, all entities had such a file and the configuration was exactly according to the docs.

Guess why my api calls still caused circular errors?

Because I forgot to do the $view->setContext$this->defaultContext); call that determines what context should be used for the response.

FUCK ME WHY DID IT TAKE SO FUCKING LONG TO FIGURE THAT OUT OMG

Google you say? Ofcourse I hunted google results! But I was unknowingly part of an XY problem and was looking for what the problem wasn't >:(

At least it works now, ugh

When I was an apprentice in a small company, ...

my boss told me that his company would never ship release builds, because the "evil optimization option" is responsible for breaking his code.
My first thought was that it wouldn't make any sense at all. The default option for code optimization is always set to zero.
After investigating his code, I found out that he didn't care to properly initialize his variables. The default compiler option for debug builds did implicitly initialize all variables to zero. After that I've confronted him with the fact that implicit null initialization does not conform to the standard of C and C++. He didn't believe me what I was saying and he was questioning my knowledge about C and C++. He refused to fix his code to this day, so he keeps building his libraries and applications always in debug mode.
Bonus fact: He would never build 64-bit applications, because his serialization functions do get incompatible with exisiting file formats.

the irony appears to be that JavaScript is more consistent than rust

so let's say you want to create some enums to represent some potential values in a REST JSON payload

well you can implement Display trait but that won't determine the JSON output
you can make a as_str() method and that doesn't even make sense frankly, I guess it's not even a trait even though it's everywhere in the std library? (traits being rust's version of interfaces, so you'd think they should be consistent)

I have a halfway urge to say rust was a beloved language but then the foundations' drama made everyone escape the ship, leaving behind a mess

well evidently the answer is you use the stupid annotations:
enum Lang {
#[serde(rename = "en-US")]
EnUS,
}
well then this only works in serialization with serde. way to go.

how about if I have some JSON data that starts with numbers? I have an interval field in the REST that expects things like 1m, 15m denoting time scale
well no deal
because rust doesn't want enums starting with numbers

and here I thought rust was superior with its static typing. but I am having to rename things all the way down and nothing is consistent. this would be so trivial in JavaScript. and there's only one toString() method! and no interfaces people say you should use while nobody uses them!

Some dev makes a claim that a Serialization library won't serialize a field unless it's public

I don't know the library details after but first reaction was "that has to be BS...."

Then I just do a Google search and yes I'm right.

And of course, in my head I go...why the fuck do these ppl not know how to use Google or just RTFM.....

Or use their brains.... If need for serialization required making all fields public that violates the most basic OOP principles...

It's sad because Django is a really great framework, but I can't understand how their serializers work.
I finally tricked to make my own using JsonResponse and alternatives methods, but I can't see why there is "serializers", "DjangoJSONEncoder", "JsonResponse", "json.dumps" and so on...

The documentation doesn't explain much about it :/

You know what I'm CONSTANTLY trying to improve on? Time. With my career out ahead of me, I feel like I gotta get a stopwatch going for myself on my personal projects. It took me hours throughout the weekend to get a stupid table UI and JSON serialization system working on a small project. Meanwhile I remember two guys created a Pokémon Go client in a couple days (back when I had a Windows Phone that didn't have one). Like, how important is time in the industry? A lot according to the memes.

Converting javascript/ typescript Map to json
or python date to json

or anything complicated to json is mostly ending with implementing serialization patterns

With date it’s so annoying cause we have iso standards that every language implemented or have libraries

so typescript doesn’t recognize Map<string, string> so you have to convert it to array and then to object

with python you need to make your own serializer / deserializer

So much waste of power usage that if only Greta know it she would say ‘how dare you!’
It can stop global warming.

The documentation of scala akka http may be just gibberish as far as I am concerned. You would think that hooking into the marshalling process (aka de/serialization) would be straight forward, I've dealt with similar problems before and solved it.

I have an object, it should be transformed into a Json and vice versa. Should be easy as pie.

Not with scala and akka-http. The docs tell you how to achieve something in dozen different ways yet lack a complete example. My first custom marshaller I created in a "marshall" package in my! namespace, but it was breaking scala compilation due to some black magic.

It's not clear how when and why marshallers are added, they just somehow are. Why do I have to deal with entity marshallers vs response marshallers. I just want each instance of a certain type to be transformed into a specific Json presentation.

Asking on stackoverflow also only yields in incomplete hints of "just do boargh" presupposing certain knowledge while sounding borderline condescending.

Currently, I just want to burn the project and rebuild it with fucking PHP. Flame all you want, at least I would get things done and the JMS serializer library has decent documentation and it works in an expected way.

Akka-http, combined with Scala, looks from my current rage-driven perspective like a solution worse than the problem.

The project I have been working on was growing and growing and growing... It reached it a point where the front-end was really hard to maintain. The worst part was the communication protocol, we were using JSON to serialize really complex objects.

I took some initiative and suggested that we use protobuf instead of JSON. Long story short, data usage is 10% of what it used to be, serialization and deserialzation is much faster, and the best of all, everything is strongly typed, with auto generated classes. Fucking awesome!

I keep forgetting what a massive pain in the ass it is to use dynamic libraries in anything other than C. I'm genuinely contemplating defining a serialization-based API and talking to plugins with a blob->blob function pair.

When you're using openapi generators and stuff for generating SDK code and let "the architect" handle the data structure and nomenclature, don't you hate having to add 33 (I counted) models, most of which are just the same class with different name or one property apart from each other, serialization of which gives request body overhead 56-132x (actual calculated results depending on the model complexity) the size of actual data you want to send, just to add support for one endpoint that needs just one model that started this whole madness?
I just had to add this one top level model reference and this happened to me. Those 33 models are not including the ones I already had included in my project so they didn't have to import them again.
For the love of <your_belief_here /> and all that's holy, never ever agree on generating code based on openapi if the person responsible for that is unexperienced. It will do more harm than good, trust me.
Before we decided to go with generated SDK my compiled product was a bit over 30KB, and worked just fine, but required a bit of work on each breaking API change. Every change in the API requires now 75% of that work and the compiled package is now over 8MB (750KB of which is probably my code and actually needed dependencies).
Adding an endpoint handler before? Add url, set method and construct the body with the bare minimum accepted by the server
Now? Add 33 models (or more), run full-project find&replace and hope it will work with the method supplied by the generated code, because it's not a mature tech and it's not always guaranteed it will work.

Spent a whole day building custom OnDeserialize events for a complex import project... Find out Microsoft XmlDeserializer completely ignores all serialization callbacks.

Fucking cunt.

This is yet another rant about php.

But I'll put my hands on first: I'm less than a junior and I'm looking for a backend language to learn.
So far I've been looking at php with Symfony because it's been used where I work.

Is it my impression or Symfony somehow overcomplicates everything? Like I don't know, for any stupid thing I get stucked (like yesterday, spent two hours on a circular reference problem with serialization).

Also, I don't like it's documentation. I am a book person, meaning that I need pages of text explaining how the framework (or whatever) works in a precise order.

Symfony's docs are like a graph: you often have no idea where you are or "what comes next".

Also, I feel like every page makes you just copy-paste everything without explaining very much what's happening under the hood.

I know there is a cookbook, but it's pretty outdated (like it's at version 3 or 2.7, I don't remember).

Is it just me? Do other Symfony developers experienced the same?

I think the Golang serialization API is utter garbage.
By convention it usually looks like this,

MarshalFormat() ([]byte,error)
UnmarshalFormat([]byte) error

This means that all serialization and deserialization is done on, heap allocated, in memory buffers ( technically also mmaped files, but thats not cross platform).
As the Go GC is ruthlessly optimized for minimum latency, this can potentially cause memory leaks in long running applications.

I think we need a new serialization API that looks more like

SerializeFormat(io.Writer) error
DeserializeFormat(io.Reader) error

TLDR: RTFM...

My dad (taught me how to code when I was a kid) was stuck serializing a Java enum/class to XML.... The enum wasn't just a list of string values but more like a Map(String,Object>.

He tried to annotate it with XMLEnum but the moment I saw this enum, I'm thinking that's unlikely to work.... Mapping all that to just a string?

He tried annotating the Fields in it using XMLAttribute but clearly wasnt working...

Also he use XMLEnumValue but from his test run I could clearly see it just replaced whatever the enum value would've been with some fixed String...

Me: Did you read the documentation or when the javadocs?

Dad: no, I don't like reading documentation and the samples didn't work.

I haven't done XML Serialization for years thought did use JSON and my first instinct was... You need a TypeAdapter to convert the enum to a serializable class.

So I do some Googling, read the docs then just played around with the code, figured out how to serialize a class and also how to implement XmlTypeAdapter.... 20 mins ...

Text him back with screenshots and basically:

See it's not that hard if you actually read up on the javadocs and realized ur enum is more like a class so probably the simple way won't work...

A year ago I built my first todo, not from a tutorial, but using basic libraries and nw.js, and doing basic dom manipulations.

It had drag n drop, icons, and basic saving and loading. And I was satisfied.

Since then I've been working odd jobs.

And today I've decided to stretch out a bit, and build a basic airtable clone, because I think I can.
And also because I hate anything without an offline option.

First thing I realized was I wasn't about to duplicate all the features of a spreadsheet from scratch. I'd need a base to work from.

I spent about an hour looking.
Core features needed would be trivial serialization or saving/loading.

Proper event support for when a cell, row, or column changed, or was selected. Necessary for triggering validation and serialization/saving.

Custom column types.

Embedding html in cells.

Reorderable columns

Optional but nice to have:

Changeable column width and row height.

Drag and drop on rows and columns.

Right click menu support out of the box.

After that hour I had a few I wanted to test.

And started looking at frameworks to support the SPA aspects.

Both mithril and riot have minimal router support. But theres also a ton of other leightweight frameworks and libraries worthy of prototyping in, solid, marko, svelte, etc.
I didn't want to futz with lots of overhead, babeling/gulping/grunting/webpacking or any complex configuration-over-convention.

Didn't care for dom vs shadow dom. Its a prototype not a startup.

And I didn't care to do it the "right way". Learning curve here was antithesis to experimenting. I was trying to get away from plugin, configuration-over-convention, astronaut architecture, monolithic frameworks, the works.

Could I import the library without five dozen dependancies and learning four different tools before getting to hello world?
"But if you know IJK then its quick to get started!", except I don't, so it won't. I didn't want that.

Could I get cheap component-oriented designs?

Was I managing complex state embedded in a monolith that took over the entire layout and conventions of my code, like the world balanced on the back of a turtle?

Did it obscure the dom and state, and the standard way of doing things or *compliment* those?

As for validation, theres a number of vanilla libraries, one of which treats validation similar to unit testing, which seems kinda novel.

For presentation and backend I could do NW.JS, which would remove some of the complications, by putting everything in one script. Or if I wanted to make it a web backend, and avoid writing it in something that ran like a potato strapped to a nuclear rocket (visual studio), I could skip TS and go with python and quart, an async variation of flask.
This has the advantage that using something thats *not* JS, namely python, for interacting with a proper database, and would allow self-hosting or putting it online so people can share data and access in real time with others.

And because I'm horrible, and do things the wrong way for convenience, I could use tailwind.
Because it pisses people off.

How easy (or hard) would it be to recreate a basic functional clone of the core of airtable?

I don't know, but I have feeling I'm going to find out!

I always thought J2EE was like a completely different thing from general Java. It has features like Enterprise management beans, special packages and has it's on Project type, structure in IDEs.

But it seems like it's not actually that different? If you use Spring, Hibernate, custom library annotations and build a Servlet, web server with @POST, @GET or use Serialization... And maybe use JDK instead of JRE, Maven, Ant... apparently that's considered J2EE?

I guess I'll just die.

Using unity for a commission project:
Have a CCG-like setup, the cards inherit from Scriptable object, need to serialize a card inventory for the sake of persistence.
Attempt 1: XML serialization: get fucked, can't serialize dictionaries (what the hell)
Attempt 2: using data representation of the dictionary contents: get fucked, can't serialize Scriptable objects because they have to be handled by the engine...
Well okay, what if I use a Scriptable object to keep a persistent dictionary?
Attempt 3: Scriptable object with dictionary: get fucked, the dictionary didn't persist
Well now I'm starting to lose it, I've tried so many things, XML, Binary and JSon serialization, Scriptable objects, data representations, I'm really running out of ideas. I can only think of one more option: throw the Card objects into a Resources folder, an build a set of comma delimited strings to serialize. This is stupid.

Fuck Unity. Shit like this is why I'm making my own engine. Every week I find some new peeve, some new way that unity is full of redundancy and poor design, architectural flaws and workflow deficiencies. I don't know how much more of this I can take.

Frustration is starting a very large data object with multiple child objects while a project wide refactor is underway changing the MVC architecture and introducing serialization.

Okay, so I had an object consisting of tables (basically classes) and structs (classes with only scalars as their properties).

I was about to serialize the object with vectors of classes and structs and wrote some nice tests for it wondering why they fail to validate the data after deserialization and why I only got garbage for the vectors of structs whilst the tables worked just fine.

Turns out there is an undocumented function called CreateVectorOfStructs which shall be used for structs instead of the regular CreateVector ...

There go three hours of blaming memory issues and running Valgrind over and over again ...

Loopback 4 really is pulling me out of the expressjs shithole... It really feels like I'm avoiding a lot of tedious work.

"B-b-bbut you have to use typescript and its a slower"

I've preferred typescript ever since angular 2 came out, and the 'slower' comment is invalidated by the fact that, when working on projects that are distributed between multiple devs, are marked by silent errors and serialization issues, and can change datasource types between customers, then the benefits of typescript and loopback's CLI far outweigh any potential slowdowns that may be caused. If you can find me an alternative that does this better, please let me know.