Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple APILearn More
Search - "serialization"
Spent 10 minutes starting to implement XML serialization in my old app... and then realize i already implemented it.1
Transaction isolation levels can suck a dick.
The built-in array is also an hashmap. Actually, it's always a hashmap, but you can append to it without specifying indexes and PHP will use consecutive integers. Its performance characteristics? Who knows. Oh, and only strings, ints and null are valid keys.
What's the iteration order for arrays if you use them as hashmaps (string keys)? Well, they have their internal order. So it's actually an ordered hashmap that's being called an array. And you can produce an array which has only integer keys starting with 0, but with non-sequential internal (iteration) order.
This array weirdness has some non-trivial implications. `json_encode` (serializes argument to JSON) assumes an array corresponds to a JSON array if its keys are consecutive integers in increasing order starting with 0, otherwise the array becomes a JSON object. `array_filter` (filters arrays/hashmaps using callback predicate) preserves keys, so it will punch holes in the int key sequence if non-last items are removed, thus turning arrays into hashmaps and changing your JSON structure if you forget to discard keys before serialization.
You may wonder how JSON deserialization works, then? There's a special class for deserialized JSON objects, `stdClass`. It's basically a hashmap too, but it's an object, not an array, and all functions that would normally accept arrays won't work with it. So basically its only use is JSON (de)serialization. You can even cast arrays to objects, producing `stdClass`.
Bonus PHP trivia:
Many functions return nonsensical values. `preg_match`, the regex matching function, returns 1 for success, 0 for no matches and false for malformed regular expression. PHP supports exceptions, so it could just throw one on errors. It would even make more sense to return true, false and null for these three cases. But no, 1, 0 and false. And actual matches are returned by output arg.
`array_walk_recursive`, a function supposed to recursively apply callback to each element of an array. That's what docs say. It actually applies it to leafs only. It will also silently accept object instead of array and "walk" it, but without recursing into deeper objects.
Runtime type enforcing is supported for function arguments and returned values. You can use scalar types, classes, array, null and a few special keywords. There's also a `mixed` keyword, which is used in docs and means "anything". It's syntactically valid, the parser will accept it, but it matches no values in runtime. Calling such function will always cause a runtime error.
Strings can be indexed with negative integers. Arrays can't.
ReflectionClass::newInstanceWithoutConstructor: "Creates a new class instance without invoking the constructor". This one needs no commentary.
`array_map` is pretty self-explanatory if you call it with a callback and an array. Or if you provide more arrays of equal length via varargs, callback will be called with more arguments, one from each array. Makes sense so far. Now, you can also call `array_map` with null instead of callback. In that case it treats provided arrays as rows of a matrix and returns that matrix, transposed.5
So yeah XML is still not solved in year 2018. Or so did I realize the last days.
I use jackson to serialize generic data to JSON.
Now I also want to provide serialization to XML. Easy right? Jackson also provides XML serialization facitlity similar to JAXB.
Works out of the box (more or less). Wait what? *rubbing eyes*
Why is my groups property (java.util.Set) rendered as two separate elements? Who the fuck every though this is the way to go?
So OK *reading the docs* there is a way to create a collection wrapper. That must be it, I thought ...
What the fuck is this now? This is still not right!!!
I know XML offers a lot of flexibility on how to represent your data. But this is just wrong ...
The only logical way to display that data is:
It would be better if the individual entries would be just called "group" but I guess implementing such a logic would be pretty hard (finding a singular of an arbitrary word?).
So yeah theres a way for that * implementing a custom collection serializer* ... wait is that really the way to go? I mean common, am I the only one who just whants this fucking shit just work as expected, with the least amount of suprise?
Why do I have to customize that ...
So ok it renders fine now ... *writes test for it+
FUCK FUCK FUCK. why can't jackson not deserialize it properly anymore? The two groups are just not being picked up anymore ...
SO WHY, WHY WHY are you guys over at jackson, JAXB and the like not able to implement that in the right manner. AND NOT THERE IS ONLY ONE RIGHT WAY TO DO IT!
*looks at an apple PLIST file* *scratches head* OK, gues I'll stick to the jackson defaults, at least it's not as broken as the fucking apple XML:
I really wonder who at apple has this briliant idea ...2
So I've created this account specifically for this rant. I usually just browse anonymously.
I've recently been hired in a big company that is one of the biggest Microsoft users in the world and my essentially revolves on making it easier for our collaborators to work with SharePoint (and other ms software)
Never in my life have I hit that much of a roadblock. So for the past week I've been trying to integrate what Ms calls webparts. And to modify the default webparts Ms provides you need to their properties (or Metadata). Except here's the big problem these are NOT documented anywhere (unless I failed to find it, if you do know where it is documented please HMU), so I've found myself trying to reverse engineer the js scripts that are served with SharePoint to figure out what the webpart properties are called and what type of data they are! I've been going through endless github repos using the CSOM nuget package (it's the library everyone uses to interact with SharePoint) and I finally found out about this other library called PnP which is a wrapper around CSOM that makes it easier to use. That wrapper has a way for me to load existing page and look at the properties of existing webparts. So here I thought it was the end of my suffering and I could finally get an idea of what it should be. Turns out this method doesn't work because one of the dependencies it has has had breaking changes and they still updated it even though it breaks their code! So for the past two days I've been trying random combinations of key values with different data types and json serialization methods.
Oh and yeah I've also looked at all the http calls via the chrome network tab, the metadata is not served as an individual file but is computed by Ms servers when they're serving you their html files.
So uh yeah run from CSOM if you can..3
Ah, the little subtle things we have to iron out as we progress from Junior Developer to Medior Developer.. things like:
- knowing the difference between a carriage return and a line feed (although having worked with analog typewriters helps) and later knowing that Unix-based systems and Windows NT-based systems implement it differently..
- knowing that serialization is important because not all computers interpret data the same way and some computers allocate 4 Bytes for a construct, others 16 Bytes.. and then we get the funkiness of transferring character sets between machines..
- knowing that a whitespace character is not only an actual space (as is known in ASCII as code 32). This one can cause even medior developers a headache, as in: why the fuck does this string function say that "hello I am a duck" and "hello I am a duck" are not the same?! Turns out then in the debugger that when you expand every character in the string you see that string1 contains 32 32 32 32 as usual.. but then string2 contains -96 -96 -96 -96 and you're like.. what the fuck..? Then you know you have to throw \\h regex at it. Haha.
- finalizing our objects and streams (although modern languages do that for us).. otherwise we have to do funky shit like trying to find what's locking a file, which is not so easy to figure out.
- figuring out why something won't work often requires you to not only break down the problem in smaller steps, to use a debugger, but sometimes it's even better to just create a proof of concept, slap some minimal code in there and debug that.. much easier.
The feature was to parse a set of fairly complex xml files following a legacy schema. Problem was, the way this was done previously did not conform to the schema so it was a guideline at best, which over the course of many years snowballed into an anarchy where clients would send in whatever and it was continuously updated per case as needed. They wanted to start enforcing their new schema while phasing out the old method.
The good news is that parsing and serialization is very testable, so I rounded up what I could find of example files and got to work. Around the same time I asked our client if they had any more examples of typical cases we need to deal with, and sure enough a couple of days later I receive a zip with hundreds of files. They also point out that I should just disregard the entire old set since they decided to outright cut support for it after all if it makes things simpler. Nice.
I finish the feature in a decent amount of time. All my local tests pass, and the CD tests pass when I push my branches. Once we push to our QA env though and the integration tests run, we get a pass rate of less than 10%.
I spend a couple of days trying to figure out what's going on, and eventually narrow it down to some wires being crossed with the new vs. old xml formats. I'm at a loss. I keep trying to chip away at it until I'm left with a minimal example, and I have one of those lean-back moments where you're just "I don't get it". My tests pass locally, but in the QA environment they fail on the same files.
We're now 3 people around my workstation including the system architect, and I'm demonstrating to the others how baffling and black magic this is. I postulate that maybe something is cached in my local environment and it's not actually testing the new files. I even deleted the old ones.
"Are you sure you deleted the right files?"
"Duh of course -- but let me check..."1
I really hate people that don't document well their 'inventions'.
Let's take for example kotlinx.serialization library. It's a self-obvious fact that there are at least two things that will make anyone suffer: generics and polymorphism. So, they must be on the front page. Solved. With a recipe. Ready to use.
And what do we have? One mention somewhere in docs and an amazing test that 'explains' 'everything':
'What we got here is a failure to communicate.' (c)9
I'm such a fucking idiot
I'm setting up an api and to prevent unwanted fields or circular dependencies from showing up I define what fields should appear in a few serialization yaml files.
These files define what fields should appear in a given context. The default context for every field is to always show the id, and only a call to /posts will give you all the fields of the posts for example. This means that if you retrieve a comment with a linked post, the post will only show up as an id, but the comment will have all its fields.
I've been struggling with a stupid problem for 2 hours, I could verify that the yaml files were loaded in, all entities had such a file and the configuration was exactly according to the docs.
Guess why my api calls still caused circular errors?
Because I forgot to do the $view->setContext$this->defaultContext); call that determines what context should be used for the response.
FUCK ME WHY DID IT TAKE SO FUCKING LONG TO FIGURE THAT OUT OMG
Google you say? Ofcourse I hunted google results! But I was unknowingly part of an XY problem and was looking for what the problem wasn't >:(
At least it works now, ugh1
When I was an apprentice in a small company, ...
my boss told me that his company would never ship release builds, because the "evil optimization option" is responsible for breaking his code.
My first thought was that it wouldn't make any sense at all. The default option for code optimization is always set to zero.
After investigating his code, I found out that he didn't care to properly initialize his variables. The default compiler option for debug builds did implicitly initialize all variables to zero. After that I've confronted him with the fact that implicit null initialization does not conform to the standard of C and C++. He didn't believe me what I was saying and he was questioning my knowledge about C and C++. He refused to fix his code to this day, so he keeps building his libraries and applications always in debug mode.
Bonus fact: He would never build 64-bit applications, because his serialization functions do get incompatible with exisiting file formats.
Some dev makes a claim that a Serialization library won't serialize a field unless it's public
I don't know the library details after but first reaction was "that has to be BS...."
Then I just do a Google search and yes I'm right.
And of course, in my head I go...why the fuck do these ppl not know how to use Google or just RTFM.....
Or use their brains.... If need for serialization required making all fields public that violates the most basic OOP principles...19
It's sad because Django is a really great framework, but I can't understand how their serializers work.
I finally tricked to make my own using JsonResponse and alternatives methods, but I can't see why there is "serializers", "DjangoJSONEncoder", "JsonResponse", "json.dumps" and so on...
The documentation doesn't explain much about it :/6
You know what I'm CONSTANTLY trying to improve on? Time. With my career out ahead of me, I feel like I gotta get a stopwatch going for myself on my personal projects. It took me hours throughout the weekend to get a stupid table UI and JSON serialization system working on a small project. Meanwhile I remember two guys created a Pokémon Go client in a couple days (back when I had a Windows Phone that didn't have one). Like, how important is time in the industry? A lot according to the memes.2
or python date to json
or anything complicated to json is mostly ending with implementing serialization patterns
With date it’s so annoying cause we have iso standards that every language implemented or have libraries
so typescript doesn’t recognize Map<string, string> so you have to convert it to array and then to object
with python you need to make your own serializer / deserializer
So much waste of power usage that if only Greta know it she would say ‘how dare you!’
It can stop global warming.5
My dad (taught me how to code when I was a kid) was stuck serializing a Java enum/class to XML.... The enum wasn't just a list of string values but more like a Map(String,Object>.
He tried to annotate it with XMLEnum but the moment I saw this enum, I'm thinking that's unlikely to work.... Mapping all that to just a string?
He tried annotating the Fields in it using XMLAttribute but clearly wasnt working...
Also he use XMLEnumValue but from his test run I could clearly see it just replaced whatever the enum value would've been with some fixed String...
Me: Did you read the documentation or when the javadocs?
Dad: no, I don't like reading documentation and the samples didn't work.
I haven't done XML Serialization for years thought did use JSON and my first instinct was... You need a TypeAdapter to convert the enum to a serializable class.
So I do some Googling, read the docs then just played around with the code, figured out how to serialize a class and also how to implement XmlTypeAdapter.... 20 mins ...
Text him back with screenshots and basically:
See it's not that hard if you actually read up on the javadocs and realized ur enum is more like a class so probably the simple way won't work...2
The documentation of scala akka http may be just gibberish as far as I am concerned. You would think that hooking into the marshalling process (aka de/serialization) would be straight forward, I've dealt with similar problems before and solved it.
I have an object, it should be transformed into a Json and vice versa. Should be easy as pie.
Not with scala and akka-http. The docs tell you how to achieve something in dozen different ways yet lack a complete example. My first custom marshaller I created in a "marshall" package in my! namespace, but it was breaking scala compilation due to some black magic.
It's not clear how when and why marshallers are added, they just somehow are. Why do I have to deal with entity marshallers vs response marshallers. I just want each instance of a certain type to be transformed into a specific Json presentation.
Asking on stackoverflow also only yields in incomplete hints of "just do boargh" presupposing certain knowledge while sounding borderline condescending.
Currently, I just want to burn the project and rebuild it with fucking PHP. Flame all you want, at least I would get things done and the JMS serializer library has decent documentation and it works in an expected way.
Akka-http, combined with Scala, looks from my current rage-driven perspective like a solution worse than the problem.
The project I have been working on was growing and growing and growing... It reached it a point where the front-end was really hard to maintain. The worst part was the communication protocol, we were using JSON to serialize really complex objects.
I took some initiative and suggested that we use protobuf instead of JSON. Long story short, data usage is 10% of what it used to be, serialization and deserialzation is much faster, and the best of all, everything is strongly typed, with auto generated classes. Fucking awesome!1
I think the Golang serialization API is utter garbage.
By convention it usually looks like this,
This means that all serialization and deserialization is done on, heap allocated, in memory buffers ( technically also mmaped files, but thats not cross platform).
As the Go GC is ruthlessly optimized for minimum latency, this can potentially cause memory leaks in long running applications.
I think we need a new serialization API that looks more like
Spent a whole day building custom OnDeserialize events for a complex import project... Find out Microsoft XmlDeserializer completely ignores all serialization callbacks.
This is yet another rant about php.
But I'll put my hands on first: I'm less than a junior and I'm looking for a backend language to learn.
So far I've been looking at php with Symfony because it's been used where I work.
Is it my impression or Symfony somehow overcomplicates everything? Like I don't know, for any stupid thing I get stucked (like yesterday, spent two hours on a circular reference problem with serialization).
Also, I don't like it's documentation. I am a book person, meaning that I need pages of text explaining how the framework (or whatever) works in a precise order.
Symfony's docs are like a graph: you often have no idea where you are or "what comes next".
Also, I feel like every page makes you just copy-paste everything without explaining very much what's happening under the hood.
I know there is a cookbook, but it's pretty outdated (like it's at version 3 or 2.7, I don't remember).
Is it just me? Do other Symfony developers experienced the same?10
I always thought J2EE was like a completely different thing from general Java. It has features like Enterprise management beans, special packages and has it's on Project type, structure in IDEs.
But it seems like it's not actually that different? If you use Spring, Hibernate, custom library annotations and build a Servlet, web server with @POST, @GET or use Serialization... And maybe use JDK instead of JRE, Maven, Ant... apparently that's considered J2EE?9
I guess I'll just die.
Using unity for a commission project:
Have a CCG-like setup, the cards inherit from Scriptable object, need to serialize a card inventory for the sake of persistence.
Attempt 1: XML serialization: get fucked, can't serialize dictionaries (what the hell)
Attempt 2: using data representation of the dictionary contents: get fucked, can't serialize Scriptable objects because they have to be handled by the engine...
Well okay, what if I use a Scriptable object to keep a persistent dictionary?
Attempt 3: Scriptable object with dictionary: get fucked, the dictionary didn't persist
Well now I'm starting to lose it, I've tried so many things, XML, Binary and JSon serialization, Scriptable objects, data representations, I'm really running out of ideas. I can only think of one more option: throw the Card objects into a Resources folder, an build a set of comma delimited strings to serialize. This is stupid.
Fuck Unity. Shit like this is why I'm making my own engine. Every week I find some new peeve, some new way that unity is full of redundancy and poor design, architectural flaws and workflow deficiencies. I don't know how much more of this I can take.2
Frustration is starting a very large data object with multiple child objects while a project wide refactor is underway changing the MVC architecture and introducing serialization.
Hey, anyone ever worked on Avro or Serialization?
Please have a look.
I don't know why whenever I ask a decent question on StackOverflow it gets ignored. :/
And a stupid ones gets downvoted lol5
Okay, so I had an object consisting of tables (basically classes) and structs (classes with only scalars as their properties).
I was about to serialize the object with vectors of classes and structs and wrote some nice tests for it wondering why they fail to validate the data after deserialization and why I only got garbage for the vectors of structs whilst the tables worked just fine.
Turns out there is an undocumented function called CreateVectorOfStructs which shall be used for structs instead of the regular CreateVector ...
There go three hours of blaming memory issues and running Valgrind over and over again ...
Loopback 4 really is pulling me out of the expressjs shithole... It really feels like I'm avoiding a lot of tedious work.
"B-b-bbut you have to use typescript and its a slower"
I've preferred typescript ever since angular 2 came out, and the 'slower' comment is invalidated by the fact that, when working on projects that are distributed between multiple devs, are marked by silent errors and serialization issues, and can change datasource types between customers, then the benefits of typescript and loopback's CLI far outweigh any potential slowdowns that may be caused. If you can find me an alternative that does this better, please let me know.1
I love steem-"I don't know how to serialise JSON so lets throw an error when you provide a dictionary"-js. I know it's a convinience, but don't tell me one of my characters is invalid as it's just not serialised.
Should probably mention:
The rest of the libraries accept JSON text and dictionaries/arrays so this is non-standard and also this lib doesn't provide type hints.
Learning node has been a big pain in the ass, when you come from everywhere else where asynchronicity is an afterthought, with node it's not a choice LOL.