I've optimised so many things in my time I can't remember most of them.

Most recently, something had to be the equivalent off `"literal" LIKE column` with a million rows to compare. It would take around a second average each literal to lookup for a service that needs to be high load and low latency. This isn't an easy case to optimise, many people would consider it impossible.

It took my a couple of hours to reverse engineer the data and implement a few hundred line implementation that would look it up in 1ms average with the worst possible case being very rare and not too distant from this.

In another case there was a lookup of arbitrary time spans that most people would not bother to cache because the input parameters are too short lived and variable to make a difference. I replaced the 50000+ line application acting as a middle man between the application and database with 500 lines of code that did the look up faster and was able to implement a reasonable caching strategy. This dropped resource consumption by a minimum of factor of ten at least. Misses were cheaper and it was able to cache most cases. It also involved modifying the client library in C to stop it unnecessarily wrapping primitives in objects to the high level language which was causing it to consume excessive amounts of memory when processing huge data streams.

Another system would download a huge data set for every point of sale constantly, then parse and apply it. It had to reflect changes quickly but would download the whole dataset each time containing hundreds of thousands of rows. I whipped up a system so that a single server (barring redundancy) would download it in a loop, parse it using C which was much faster than the traditional interpreted language, then use a custom data differential format, TCP data streaming protocol, binary serialisation and LZMA compression to pipe it down to points of sale. This protocol also used versioning for catchup and differential combination for additional reduction in size. It went from being 30 seconds to a few minutes behind to using able to keep up to with in a second of changes. It was also using so much bandwidth that it would reach the limit on ADSL connections then get throttled. I looked at the traffic stats after and it dropped from dozens of terabytes a month to around a gigabyte or so a month for several hundred machines. The drop in the graphs you'd think all the machines had been turned off as that's what it looked like. It could now happily run over GPRS or 56K.

I was working on a project with a lot of data and noticed these huge tables and horrible queries. The tables were all the results of queries. Someone wrote terrible SQL then to optimise it ran it in the background with all possible variable values then store the results of joins and aggregates into new tables. On top of those tables they wrote more SQL. I wrote some new queries and query generation that wiped out thousands of lines of code immediately and operated on the original tables taking things down from 30GB and rapidly climbing to a couple GB.

Another time a piece of mathematics had to generate all possible permutations and the existing solution was factorial. I worked out how to optimise it to run n*n which believe it or not made the world of difference. Went from hardly handling anything to handling anything thrown at it. It was nice trying to get people to "freeze the system now".

I build my own frontend systems (admittedly rushed) that do what angular/react/vue aim for but with higher (maximum) performance including an in memory data base to back the UI that had layered event driven indexes and could handle referential integrity (overlay on the database only revealing items with valid integrity) or reordering and reposition events very rapidly using a custom AVL tree. You could layer indexes over it (data inheritance) that could be partial and dynamic.

So many times have I optimised things on automatic just cleaning up code normally. Hundreds, thousands of optimisations. It's what makes my clock tick.

"I'm almost done, I'll just need to add tests!"

Booom! You did it, that was a nuke going off in my head.

No, you shouldn't just need to add tests. The tests should have been written from the get go! You most likely won't cover all the cases. You won't know if adding the tests will break your feature, as you had none, as you refactor your untested mess in order to make your code testable.

When reading your mess of a test case and the painful mocking process you went through, I silently cry out into the void: "Why oh why!? All of this suffering could have been avoided!"

Since most of the time, your mocking pain boils down to not understanding what your "unit" in your "unit test" should be.

So let it be said:

- If you want to build a parser for an XML file, then just write a function / class whose *only* purpose is: parse the XML file, return a value object. That's it. Nothing more, nothing less.
- If you want to build a parser for an XML file, it MUST NOT: download a zip, extract that zip, merge all those files to one big file, parse that big file, talk to some other random APIs as a side-effect, and then return a value object.

Because then you suddenly have to mock away a http service and deal with zip files in your test cases.

The http util of your programming language will most likely work. Your unzip library will most likely work. So just assume it working. There are valid use cases where you want to make sure you acutally send a request and get a response, yet I am talking unit test here only.

In the scope of a class, keep the public methods to a reasonable minimum. As for each public method you shall at least create one test case. If you ever have the feeling "I want to test that private method" replace that statement in your head with: "I should extract that functionality to a new class where that method public. I then can create a unit test case a for that." That new service then becomes a dependency in your current service. Problem solved.

Also, mocking away dependencies should a simple process. If your mocking process fills half the screen, your test setup is overly complicated and your class is doing too much.

That's why I currently dig functional programming so much. When you build pure functions without side effects, unit tests are easy to write. Yet you can apply pure functions to OOP as well (to a degree). Embrace immutability.

Sidenote:

It's really not helpful that a lot of developers don't understand the difference between unit, functional acceptance, integration testing. Then they wonder why they can't test something easily, write overly complex test cases, until someone points out to them: No, in the scope of unit tests, we don't need to test our persistance layer. We just assume that it works. We should only test our businsess logic. You know: "Assuming that I get that response from the database, I expect that to happen." You don't need a test db, make a real query against that, in order to test that. (That still is a valid thing to do. Yet not in the scope of unit tests.)

PHP arrays.

The built-in array is also an hashmap. Actually, it's always a hashmap, but you can append to it without specifying indexes and PHP will use consecutive integers. Its performance characteristics? Who knows. Oh, and only strings, ints and null are valid keys.

What's the iteration order for arrays if you use them as hashmaps (string keys)? Well, they have their internal order. So it's actually an ordered hashmap that's being called an array. And you can produce an array which has only integer keys starting with 0, but with non-sequential internal (iteration) order.

This array weirdness has some non-trivial implications. `json_encode` (serializes argument to JSON) assumes an array corresponds to a JSON array if its keys are consecutive integers in increasing order starting with 0, otherwise the array becomes a JSON object. `array_filter` (filters arrays/hashmaps using callback predicate) preserves keys, so it will punch holes in the int key sequence if non-last items are removed, thus turning arrays into hashmaps and changing your JSON structure if you forget to discard keys before serialization.

You may wonder how JSON deserialization works, then? There's a special class for deserialized JSON objects, `stdClass`. It's basically a hashmap too, but it's an object, not an array, and all functions that would normally accept arrays won't work with it. So basically its only use is JSON (de)serialization. You can even cast arrays to objects, producing `stdClass`.

Bonus PHP trivia:

Many functions return nonsensical values. `preg_match`, the regex matching function, returns 1 for success, 0 for no matches and false for malformed regular expression. PHP supports exceptions, so it could just throw one on errors. It would even make more sense to return true, false and null for these three cases. But no, 1, 0 and false. And actual matches are returned by output arg.

`array_walk_recursive`, a function supposed to recursively apply callback to each element of an array. That's what docs say. It actually applies it to leafs only. It will also silently accept object instead of array and "walk" it, but without recursing into deeper objects.

Runtime type enforcing is supported for function arguments and returned values. You can use scalar types, classes, array, null and a few special keywords. There's also a `mixed` keyword, which is used in docs and means "anything". It's syntactically valid, the parser will accept it, but it matches no values in runtime. Calling such function will always cause a runtime error.

Strings can be indexed with negative integers. Arrays can't.

ReflectionClass::newInstanceWithoutConstructor: "Creates a new class instance without invoking the constructor". This one needs no commentary.

`array_map` is pretty self-explanatory if you call it with a callback and an array. Or if you provide more arrays of equal length via varargs, callback will be called with more arguments, one from each array. Makes sense so far. Now, you can also call `array_map` with null instead of callback. In that case it treats provided arrays as rows of a matrix and returns that matrix, transposed.

Regarding Article 13 (or 17 or wherever it moved to now)… Let's say that the UK politicians decide to be dicks and approve the law. After that, we need to get it engineered in, right? Let's talk a bit about how.. well, I'd maybe go over it. Been thinking about it a bit in the shower earlier, so.. yeah.

So, fancy image recognition or text recognition from articles scattered all over the internet, I think we can all agree.. that's infeasible. Even more so, during this lobby with GitHub and OpenForum Europe, guy from GitHub actually made a very valid point. Now for starters, copyright infringement isn't an issue on the platform GitHub that pretty much breathes collaboration. But in the case of I-Boot for example, that thing from Apple that got leaked earlier. If that would get preemptively blocked.. well there's no public source code for it to get compared against to begin with, right? So it's not just "scattered all over the internet, good luck crawling it", it's nowhere to be found *at all*.

So content filtering.. yeah. Nope, ain't gonna happen. Keep trying with that, EU politicians.

But let's say that I am a content creator who hates the cancer of joke/meme because more often than not it manifests itself as a clone of r/programmerhumor.. someone decides to freeboot my content. So I go out, look for it, find it. Facebook and the likes, make it easier to find it in the first place, you dicks. It's extremely hard to find your content there.

So Facebook implements a way to find that content a bit easier maybe. Me being the content creator finds it.. oh blimey! It can't be.. it's the king of freebooting on Facebook, SoFlo! Who would've thought?! So at that point.. I'd like to get it removed of course. Report it as copyright infringement? Of course. Again Facebook you dicks, don't make it so tedious to fill in that bloody report. And look into it quickly! The videos those SoFlo dicks post is only relevant in the first 48h or so. That's where they make the most money. So act more quickly.

So the report is filled, video's taken down.. what else? Maybe temporarily make them unable to post as a bit of a punishment so that they won't do it again? And put in a limit to the amount of reports they can receive. Finally, maybe reroute the revenue stream to the original content creator instead. That way stolen content suddenly becomes free exposure! Awesome!

*suddenly realizes that I've been talking about the YouTube copyright strike system all along*

… Well.. maybe something like that then? That shouldn't be too hard to implement, and on YouTube at least it seems to be quite effective. Just imagine SoFlo and the likes that are repeat offenders, every 3 posts they get their account and page shut down. Good luck growing an audience that way. And good luck making new accounts all the time to start with.. account verification technology is pretty good these days. Speaking of experience here, tried bypassing Facebook's signup hoops a fair bit and learned a bit about some of the things they have red flags on, hehe.

But yeah, something like that maybe for social media in general. And.. let's face it, the biggest one that would get hurt by something like this would be Facebook. And personally I think it's about time for that bastard company to get a couple of blows already.

What are your thoughts on this?

Developers who think complex code is good.

"Oh, lookie here, I can swizzle methods and inject dependencies in the runtime!"

"Although we have no valid use case, let's use dependency injection and follow the commandory stateor patterns because I watched a video."

Just because you learn something new that looks cool does not make it practical, you tosser.

I know I’ll get mixed views for this one...

So I’ll state my claim. I agree with the philosophy of uncle bob, I also feel like he is the high level language - older version of myself personality wise.. (when I learned about uncle bob I was like this guy is just like me but not low level haha).

Anyway.. I don’t agree with everything because I think he thinks or atleast I get the vibe he thinks everything can be solved by OOP, and high level languages. This is probably where Bob and I disagree. Personally I don’t touch ruby, python and java and “those” with a 10 foot pole.

Does he make valid arguments, yes, is agile the solve all solution no.. but agile ideas do come natural and respond faster the feedback loop of product development is much smaller and the managers and clients and customers can “see things” sooner than purly waterfall.. I mean agile is the natural approach of disciplined engineers....waterfall is and was developed because the market was flooded with undisciplined engineers and continues to flood, agile is great for them but only if they are skilled in what they are doing and see the bigger picture of the forest thru the trees.. which is the entire point of waterfall, to see the forest.. the end goal... now I’m not saying agile you only see a branch of a single tree of the forest.. but too often young engineers, and beginners jump on agile because it’s “trendy” or “everyone’s doing it” or whatever the fuck reason. The point is they do it but only focus on the immediate use case, needs and deliverables due next week.

What’s wrong with that?? Well an undisciplined engineer doing agile (no I’m not talking damn scrum shit and all that marketing bullshit).. pure true agile.
They will write code for the need due next week, but they won’t realize that hmm I will have the need 3 months from now for some feature that needs to connect to this, so I better design this code with that future feature in mind...

The disciplined engineer would do that. That is why waterfall exists so ideally the big picture is painted before hand.

The undisciplined engineer will then be frustrated in the future when he has to act like the cool aid man thru the hard pre mature architectural boundaries he created and now needs links or connections that are now needed.

Does moving to agile fix that hell no.. because the undisciplined engineer is still undisciplined.

One could argue the project manager or scrum secretary... (yes scrum secretary I said that right).. is suppose to organize and create and order the features with the future in mind etc...

Bullshit ..soo basically your saying the scrum kid is suppose to be the disciplined engineer to have foresight into realizing future features and making requirements and task now that cover those things? No!
1 scrum bitch focuses too much on pleasing “stake holders” especially taken literally in start ups where the non technical idiots are too involved with the engineering team and the scrum bastard tries to ass kiss and get everything organized and tasks working so the non technical person can see pretty things work.

Scrum master is a gate keeper and is not needed and actually hinders the whole process of making a undisciplined engineer into a disciplined engineer, makes the undisciplined engineer into a “forever” code grunt... filling weekly orders of story points unable to see the forest until it’s over because the forest isn’t show to the grunt only the scrum keeper knows the big picture..... this is bad this is why waterfall is needed.

Waterfall has its own problems, But that’s another story for another day..

ANYWAY... soooo where were we ....

Ahh yess....

Clean code..

Is it a good book, yes.. does uncle bobs personality show thru the book .. yes lol.
If you know uncle bob you will understand what I just did with this post lol. I had to tangent ( at least mine was related to the topic) ...

I agree with the principles of the book, I don’t agree with the extreme view point. It’s like religion there’s the modest folks and then there are the extremists. Well he’s the preacher of the cult and he’s on the extreme side.. but that doesn’t mean he’s wrong.. many things he nails... he just hits the nail thru the wall just a bit.

OOP languages are not the solution... high level languages do not solve everything.. pininciples and concepts can be used across the board and prove valuable.. just don’t hold everything up like the 10 commandments of which you cannot deviate from.. that’s the difference here I think..

Good book, just don’t take it as the Bible as a beginner, actually infact DONT read this book as a beginner. Wait a bit learn then reflect by reading this.

Sus!
yesterday I bought a cool domain in namecheap, I was very lucky to find short and good one for my case.

Today (at weekends!!!!) I receive a letter:
>Hello **redacted name**,
>
>We are contacting you from the Namecheap Risk Management Team regarding your '**redacted name account**' account.
>
>Unfortunately, your Namecheap account was flagged by our fraud screening system as requiring verification and was locked.
>
>Please follow the instructions below to get your account verified:
>
>- take a color photo of the credit card used for the payment at **redacted link**
>
>Please make sure all of the edges of the credit card are visible, and that we can clearly see the card holder's name, expiration, and last four digits of the card number. The screenshots or images of the card cannot be accepted for verification. >If the submission does not meet these requirements, we can either request to submit the details again or permanently suspend your account.
>
>- provide a valid phone number and the best time to call you (within normal business hours, US Pacific time).
>
>If we do not hear back from you within 24 hours, we will be forced to cancel your orders.
>
>We apologize for any inconvenience that may result from this process. This extra verification is done for your security and to ensure that orders are legitimate. This industry, unfortunately, has a high rate of fraudulent orders, and this sort of >verification helps us drastically reduce fraud and ensure our customers remain secure. Such documents are used for verification only and are not provided to third parties in any way. Account verification is a one-time procedure, after your account >is verified, you will never face this issue again.
>
>Looking forward to your reply.
>
>---------------
>Dmitriy K.
>Risk Management
> Namecheap, Inc.

what if I did not notice it in 24 hours? It is the weekend for god's sake! People usually rest until monday.
They would what, cancel order and scalpel it to super high price?!
I have some doubts if the request is trully having anti fraudulent origins.

What if I used digital visa card? How was I supposed to photo it?

And the service they provided for photoing accepts only photos from web camera. I was lucky that I bought recently web camera with high enough amount of pixel power and manual focus. What if I did not?

That's all really SUS!
The person can not notice the letter within 24 hours time frame until the morning, when it would be already too late.

I recently got into an argument with a random person on internet about the new Corsair XENEON FLEX OLED, the new fancy one that you can make curved or flat…
In my opinion it doesn't make any sense, curved is better, in particular with a 45" display, so it's a cool technology but useless in this case.

Apparently this guy thinks "for work is better flat, for gaming curved".
It made me thinking… really?
There is someone out there (and maybe here) that uses huge flat monitors or when have 2 puts them parallels to each other and not turned towards himself at an angle?

It seemed a random bullshit, but maybe I could find some valid arguments why "flat is better for work" or not. 🤔

What's the minimal feature set that can make a language as ornamented as JS into a comfortable REPL?
Should I write a full parser or should I try to patch my way around with regex?
It will have to interface a lot with JS so it has to be able to manage JS datastructures in some fashion, which means that I can't just make a whole new command line with its own programs.

My current plan:
Some delimiter (probably a semicolon) will take the output of a command and inject it in the next in case you decide halfway through a line to do some more processing, It also awaits promises and does some other nice stuff to make controlling such pipelines easy. I have an elaborate system in mind to decide where a value must be injected to make the line valid so in most cases you don't even have to indicate it. JS has beautifully simple syntax rules so I have a lot of technical balance to burn before I start building technical debt.

I have some ideas for automatic parentheses and commas in function calls. I realize while using a command line you do not want to tap shift often. My main idea here is that two names or values in js are always joined by an operator so the first missing operator is a call and following missing operators are commas until the end of line. This has lots of nasty edge cases though, like that no argument expression can begin with a unary operator or a bracket of any shape. You can always prepend a comma but it's cognitive load.

Anyway, do you have any suggestion or warning besides "js bad" which I know but it's the most popular sandboxable language and has a massive existing set of libraries which I kinda need.