Optimising code

Android is upgrading...

Optimising app 1 of 93563

Fuck you you stupid fucking cock sucking power company.

I was fucking in the middle of optimising a class and the power goes out.

FUCK FUCK FUCK!!!
Why didn’t I press save 😂

Now to go find me a UPS for the rare times this occurs, before my phone dies.

Oh I have a good one.

A dev once was added to the team -he deleted all of the whitespace from the backend code (minified) and called that optimising. We had a lot of back and forth and me reverting the things back to non minified and him back to minified on live environement. I shit you not i had to endure that shit and he kept on insisting that I'm the shitty developer because I don't know how to optimise. I'm starting to rage even thinking about it....

“One of the biggest mistakes engineers make is spending time on optimising something (a component) that shouldn’t exist in the first place” - Elon

I’ve been guilty of this a few times 😅

Just rememberes a collegue i had for a short period, i was remote and he was on location randomly added, I was told he was a php developer.
What he did was delete ALL the whitespace in the php code and called it optimising, he told the director it’ll make the code run faster. You can imagine how fun that was...

That feeling when a feature is estimated to take 4 hours but, thanks to your well structured code and good use of object oriented programming you implement it in 30 minutes and can spend the rest optimising something else.

Is your code green?

I've been thinking a lot about this for the past year. There was recently an article on this on slashdot.

I like optimising things to a reasonable degree and avoid bloat. What are some signs of code that isn't green?

* Use of technology that says its fast without real expert review and measurement. Lots of tech out their claims to be fast but actually isn't or is doing so by saturation resources while being inefficient.
* It uses caching. Many might find that counter intuitive. In technology it is surprisingly common to see people scale or cache rather than directly fixing the thing that's watt expensive which is compounded when the cache has weak coverage.
* It uses scaling. Originally scaling was a last resort. The reason is simple, it introduces excessive complexity. Today it's common to see people scale things rather than make them efficient. You end up needing ten instances when a bit of skill could bring you down to one which could scale as well but likely wont need to.
* It uses a non-trivial framework. Frameworks are rarely fast. Most will fall in the range of ten to a thousand times slower in terms of CPU usage. Memory bloat may also force the need for more instances. Frameworks written on already slow high level languages may be especially bad.
* Lacks optimisations for obvious bottlenecks.
* It runs slowly.
* It lacks even basic resource usage measurement.

Unfortunately smells are not enough on their own but are a start. Real measurement and expert review is always the only way to get an idea of if your code is reasonably green.

I find it not uncommon to see things require tens to hundreds to thousands of resources than needed if not more.

In terms of cycles that can be the difference between needing a single core and a thousand cores.

This is common in the industry but it's not because people didn't write everything in assembly. It's usually leaning toward the extreme opposite.

Optimisations are often easy and don't require writing code in binary. In fact the resulting code is often simpler. Excess complexity and inefficient code tend to go hand in hand. Sometimes a code cleaning service is all you need to enhance your green.

I once rewrote a data parsing library that had to parse a hundred MB and was a performance hotspot into C from an interpreted language. I measured it and the results were good. It had been optimised as much as possible in the interpreted version but way still 50 times faster minimum in C.

I recently stumbled upon someone's attempt to do the same and I was able to optimise the interpreted version in five minutes to be twice as fast as the C++ version.

I see opportunity to optimise everywhere in software. A billion KG CO2 could be saved easy if a few green code shops popped up. It's also often a net win. Faster software, lower costs, lower management burden... I'm thinking of starting a consultancy.

The problem is after witnessing the likes of Greta Thunberg then if that's what the next generation has in store then as far as I'm concerned the world can fucking burn and her generation along with it.

It's been a while DevRant!

Straight back into it with a rant that no doubt many of us have experienced.

I've been in my current job for a year and a half & accepted the role on lower pay than I normally would as it's in my home town, and jobs in development are scarce.

My background is in Full Stack Development & have a wealth of AWS experience, secure SaaS stacks etc.

My current role is a PHP Systems Developer, a step down from a senior role I was in, but a much bigger company, closer to home, with seemingly a lot more career progression.

My job role/descriptions states the following as desired:

PHP, T-SQL, MySQL, HTML, CSS, JavaScript, Jquery, XML

I am also well versed in various JS frameworks, PHP Frameworks, JAVA, C# as well as other things such as:

Xamarin, Unity3D, Vue, React, Ionic, S3, Cognito, ECS, EBS, EC2, RDS, DynamoDB etc etc.

A couple of months in, I took on all of the external web sites/apps, which historically sit with our Marketing department.

This was all over the place, and I brought it into some sort of control. The previous marketing developer hadn't left and AWS access key, so our GitLabs instance was buggered... that's one example of many many many that I had to work out and piece together, above and beyond my job role.

Done with a smile.

Did a handover to the new Marketing Dev, who still avoid certain work, meaning it gets put onto me. I have had a many a conversation with my line manager about how this is above and beyond what I was hired for and he agrees.

For the last 9 months, I have been working on a JAVA application with ML on the back end, completely separate from what the colleagues in my team do daily (tickets, reports, BI, MI etc.) and in a multi-threaded languages doing much more complicated work.

This is a prototype, been in development for 2 years before I go my hands on it. I needed to redo the entire UI, as well as add in soo many new features it was untrue (in 2 years there was no proper requirements gathering).

I was tasked initially with optimising the original code which utilised a single model & controller :o then after the first discussion with the product owner, it was clear they wanted a lot more features adding in, and that no requirement gathering had every been done effectively.

Throughout the last 9 month, arbitrary deadlines have been set, and I have pulled out all the stops, often doing work in my own time without compensation to meet deadlines set by our director (who is under the C-Suite, CEO, CTO etc.)

During this time, it became apparent that they want to take this product to market, and make it as a SaaS solution, so, given my experience, I was excited for this, and have developed quite a robust but high level view of the infrastructure we need, the Lambda / serverless functions/services we would want to set up, how we would use an API gateway and Cognito with custom claims etc etc etc.

Tomorrow, I go to London to speak with a major cloud company (one of the big ones) to discuss potential approaches & ways to stream the data we require etc.

I love this type of work, however, it is 100% so far above my current job role, and the current level (junior/mid level PHP dev at best) of pay we are given is no where near suitable for what I am doing, and have been doing for all this time, proven, consistent work.

Every conversation I have had with my line manager he tells me how I'm his best employee and how he doesn't want to lose me, and how I am worth the pay rise, (carrot dangling maybe?).

Generally I do believe him, as I too have lived in the culture of this company and there is ALOT of technical debt. Especially so with our Director who has no technical background at all.

Appraisal/review time comes around, I put in a request for a pay rise, along with market rates, lots of details, rates sources from multiple places.

As well that, I also had a job offer, and I rejected it despite it being on a lot more money for the same role as my job description (I rejected due to certain things that didn't sit well with me during the interview).

I used this in my review, and stated I had already rejected it as this is where I want to be, but wanted to use this offer as part of my research for market rates for the role I am employed to do, not the one I am doing.

My pay rise, which was only a small one really (5k, we bring in millions) to bring me in line with what is more suitable for my skills in the job I was employed to do alone.

This was rejected due to a period of sickness, despite, having made up ALL that time without compensation as mentioned.

I'm now unsure what to do, as this was rejected by my director, after my line manager agreed it, before it got to the COO etc.

Even though he sits behind me, sees all the work I put in, creates the arbitrary deadlines that I do work without compensation for, because I was sick, I'm not allowed a pay rise (doctors notes etc supplied).

What would you do in this situation?

TL;DR: TIL for heavy queries use PDO and not some frameworks DB class

ffs I was trying to save 300k+ lines at once with Laravel for weeks. Mind you from a text file. 1gb ram on the vps so while trying to prepare the text to save: Fatal Error: Allowed Memory Size of bla bla Bytes Exhausted

ok so lets put it in a loop: Fatal error: Maximum execution time of 30 seconds exceeded (set_time_limit(0); lol)

optimising, varying the code got me into a situation when the content got saved in the BD but inconsistent (duplicates) and the table had often more than 1,5M rows. That was what told me its not a performance issue, my code is the issue. (dah)

I was starting to think it would be easier to export a prepared query to a sql file and load the file into the db as thats the fastest possible option...I even started to think about switching to python, then it hit me, Laravel has a shitload of routes to the DB so I switched to PDO

benchmark on 1vCPU, 1GB RAM VPS with SSD
379k lines with 11 columns in less than 10 sec with a loop of saving every ~6000 rows (if i tried choking it to save the whole thing at once it went up to 16-17sec)

Our production server has huge memory shortage so I have to jump trough a lot of mod_rewrite optimising hoops to keep it running because no time to configure a new server...

Why the industry jumped on photoshop as a web design and layout tool is beyond me. It's like trying to stir coffee with your thumb. I'm a descent photoshop user but have always used inDesign in web mode. Far quicker for chucking around layouts and options (as page). It also exports as rgb png's either full pages or selections with or without transparency (at any resolution). Which are perfect for then optimising in Photoshop (Pixelmator these days) or any other less costly image editor. I hand code my sites then in Coda, love it.

I hate optimising in Unity3D... It makes me feel like a even worse Dev than I am

Making me happy is like optimising an image for the web.

Just strip me, interlace me and give me a sampling factor of 420 😂

The real web development is optimising the shitty front end code.
The task assigned to me is optimisation of dashboard page of website which was developed by freenlancers.(end of contract from their side)
The front end is mess. Individual js files (bootstrap, popper, jQuery, jQuery ui, loader and main) loading in production inside head tag of html file
No text compression.
Every template has random number of their own js files in any block of template. Nothing structured. There will be fantastic waste of time figuring out file dependencies.
Same with css files. Some are scss, some plain css. No compression. No proper modules.
Basically, I have to go through 25-30 html files. Then understand, which template is extending which one. Go through all js and css files in each html file and again understand dependencies between them
This is gonna be real fun.

I am having an introspective moment as a junior dev.

I am working in my 3rd company now and have spent the avg amount of time i would spent in a company ( 1- 1.5 years)

I find myself in similar problems and trajectories:

1. The companies i worked for were startups of various scales : an edtech platform, an insurance company (branch of an mnc) and a b2b analytics company

2. These people hire developers based on domain knowledge and not innovative thinking , and expect them to build anything that the PMs deem as growth/engagement worthy ( For eg, i am bad at those memory time optimising programming/ ds/algo, but i can make any kind of android screen/component, so me and people like me get hired here)

3. These people hire new PMs based on expertise in revenue generation and again , not on the basis of innovative thinking, coz most of the time these folks make tickets to experiment with buttons and text colors to increase engagement/growth

4. The system goes into chaos mode soon since their are so many cross operating teams and the PMs running around trying to boss every dev , qa and designer to add their changes in the app.

5. meanwhile due to multiple different teams working on different aspects, their is no common data center with up to date info of all flows, products and features. the product soon becomes a Frankenstein monster.

6. Thus these companies require more and more devs and QAs which are cogs in the system then innovative thinkers . the cogs in the system will simply come, dimwittingly add whatever feature is needed and goto home.

7. the cogs in system which also start taking the pain of tracking the changes and learning about the product itself becomes "load bearing cogs" : i.e the devs with so much knowledge of the product that they can be helpful in every aspect of feature lifecycle .

8. such devs find themselves in no need for proving themselves , in no need for doing innovative work and are simply promoted based on their domain knowledge and impact.

My question is simply this : are we as a dev just destined to be load bearing cogs?

we are doing the work which ideally a manager should be doing, ie maintaining confluence docs with end to end technical as well as business logic info of every feature/flow.

So is that the only definition of a Software Engineer in a technical product?

then how come innovations happen in companies like meta Microsoft google open ai etc?

if i have to guess as a far observer, i would say their diversity in different fields helps them mix and match stuff and lead to innovative stuff.

For eg, the android os team in google has helped add many innovative things in google cloud product and vice versa.

same is with azure and windows . windows is now optomissed to run in cloud machines when at one point it was just a horrible memory hogging and slow pc OS

for small companies, 1 ideology/product/domain is their hero ideology/product/domain .

an insurance company tries to experiment with stuff related to insurances,health,vehicles,and the best innovations they come up with is "lets give user a discount in premium if they do 5000 steps a day for an year".

edtech would say "lets do live streaming for children apart from static videos"

but Android team at google said , "since ai team is doing so well, lets include ai in various system apps and support device level models" ~ a much larger innovation as 2 domains combined to make a product

The small companies are not aiming to be an innovative product, they are just aiming to be a monopoly product. and this is kinda sad

Spent a few hours optimising a procedure...
It's almost quitting time now, my procedure is returning wrong results in more time...
I hate "productivity" on Friday...

Started optimising everything. For example last week while cooking, instead of taking another spoon to stir the food, I reused the same knife which I used to cut

tell me guys what would you prefer:

function a(){
..
b(..)
..
b(..)
..
}
function b(p1,p2,p3,p4,p5,p6){.
...
}

or

function a(){
..
b(..)
..
b(..)
..
}

function b(
p1,
p2,
p3,
p4,
p5,
p6
){
...
}

if you read this rant before expanding, you got a complete context on how what function a is, its calling b 2 times and how function b looks.

if instead of the first option, i had used 2nd block, you wouldn't even know the 2nd param of b function without expanding this rant.

my point?
i prefer to keeping unnecessary info on one line. and w lot of linters disagree by splitting up the code. and most importantly , my arrogant tl disagree by saying he prefers the splitted code "for readability" and becaue "he likes code this way, old-eng1 likes this and old-eng2 likes this" .

why tf does an ide have horizontal a scrolling option available when you are too stupid to use it?

ok, i know some smartass is going to point that i too can use vertical scrolling, but hear me out: i am optimising this!

case 1 : a function with 7 params is NOT split into 7 lines. lets calculate the effort to remember it
- since all params could have similar charactersticks ( they will be of some type, might have defaults, might be a suspendable/async function etc), each param will take similar memory-efforts points. say 5sp each.
- total memory-efforts= 5sp *7 = 35 sp.
- say a human has 100 sp of fast memory storage, he can use the remaining 65 sp for loading say 5 small lines above or below.
- but since 5 lines above are already read and still visible on screen, they won't be needed to be loaded again nd again, nd we can just check the lines below.
- thus we are able to store 65+35+65 = 165 sp or about 11 lines of code in out fast memory for just a 100sp brain storage

case 2 function with 7 params IS split into 7 lines.
- in this case all lines are somewhat similar. 5sp for param lines as they are still similar which implies same 35sp for storing current function and params
- remaining 65sp can only be used to store next 5 lines of 13sp as the previous code is no longer visible.
- plus if you wanna refresh the code above, you gotta scroll, which will result in removing bottom code from screen , and now your 65sp from bottom code is overwritten by 65sp of top code.
- thus at a time, you are storing only 6 lines worth of code info. this makes you slow.

this is some imaginary math, but i believe it works

Hi everyone! I'm in need of some help regarding the approach to my bachelor thesis.

The practical stuff is basically clickstream/task and usability analysis on an existing platform and creating mockup improvements for some processes. I was thinking about using a spider to generate a tree (or another datastructure) regarding all the different tasks available and then trying to optimise said tree, thus automatically optimising the processes within. I'm having however issues imagining how this optimisation might be generalised for more than this one platform.

Basically, I'm a bit lost and grasping for any pointers in any direction regarding these ideas.

i was watching a video on how whatsapp can't make enough profits coz its free and even though its a clear lie (the cartel money made by selling user data will obviously not show up in legal books), i had a thought. can any good consumer software be ever kept free for usage?

Say i made a very awesome chat app. it has 0 bugs, it does the basic tasks of sending /receiving data and media correctly and do not require any maintenence .It also is optimising a lot of cloud cost by keeping user data in their own devices and only transmitting data on triggers.

i still would require a server to keep the trigger architecture alive. and all the servers in the world are maintained by for profit corporates which will charge a premium for their services. so free products are a fallacy as someone is paying for it. it will be an investor, a different business or we the consumer (either directly as subscription , or indirectly via ads or personal data)

So i guess this realisation is going to hit soon to a lot of tiktok and insta influenza kids

The moment when you are optimising too much of an infrastructure cost that makes your fundraiser made less than your competitors to start with.