Boy do I hate office politics...

A client asked our company to fix perf issues on their product. Our coleagues had been picked for the job [being led by another 3rd-party, as per client's request]. Aaand they dropped the ball. The deadline is in 2 weeks, nothing is working.

Mgmt engaged us to put out the fire, but strictly at the scope the other guys were working in.

On the first day of testing we've revealed an elephant-sized perf issue that's as easy to fix as brainlessly changing 4 values in config. And that elephant is masking all the other perf issues.

We got a firm NO for config changes as that is out of the defined scope. And we're asked to continue testing.

I mean, the elephant is THAT huge that any further testing is moot - all other bottlenecks are hidden behind it. And just changing those 4 values would reduce the resources required by a magnitude of ~10.

But that's out of scope...

Client is desperate, lost and honestly asking us, pros in the field, for help.. We know how to help.. It takes 10 seconds to apply the fix..

But our mgmt forbids us to step out of the scope :/

as a result we have to pretend to be dummies hardly knowing what to do and hide the truth from the customer they so desperately want.

This is frustrating. And wrong. And imo unprofessional

Biggest challenge I overcame as dev? One of many.

Avoiding a life sentence when the 'powers that be' targeted one of my libraries for the root cause of system performance issues and I didn't correct that accusation with a flame thrower.

What the accusation? What I named the library. Yep. The *name* was causing every single problem in the system.

Panorama (very, very expensive APM system at the time) identified my library in it's analysis, the calls to/from SQLServer was the bottleneck

We had one of Panorama's engineers on-site and he asked what (not the actual name) MyLibrary was and (I'll preface I did not know or involved in any of the so-called 'research') a crack team of developers+managers researched the system thoroughly and found MyLibrary was used in just about every project. I wrote the .Net 1.1 MyLibrary as a mini-ORM to simplify the execution of database code (stored procs, etc) and gracefully handle+log database exceptions (auto-logged details such as the target db, stored procedure name, parameter values, etc, everything you'd need to troubleshoot database errors). This was before Dapper and the other fancy tools used by kids these days.

By the time the news got to me, there was a team cobbled together who's only focus was to remove any/every trace of MyLibrary from the code base. Using Waterfall, they calculated it would take at least a year to remove+replace MyLibrary with the equivalent ADO.Net plumbing.

In a department wide meeting:

DeptMgr: "This day forward, no one is to use MyLibrary to access the database! It's slow, unprofessionally named, and the root cause of all the database issues."
Me: "What about MyLibrary is slow? It's excecuting standard the ADO.Net code. Only extra bit of code is the exception handling to capture the details when the exception is logged."
DeptMgr: "We've spent the last 6 weeks with the Panorama engineer and he's identified MyLibrary as the cause. Company has spent over $100,000 on this software and we have to make fact based decisions. Look at this slide ... "
<DeptMgr shows a histogram of the stacktrace, showing MyLibrary as the slowest>
Me: "You do realize that the execution time is the database call itself, not the code. In that example, the invoice call, it's the stored procedure that taking 5 seconds, not MyLibrary."
<at this point, DeptMgr is getting red-face mad>
AreaMgr: "Yes...yes...but if we stopped using MyLibrary, removing the unnecessary layers, will make the code run faster."
<typical headknodd-ers knod their heads in agreement>
Dev01: "The loading of MyLibrary takes CPU cycles away from code that supports our customers. Every CPU cycle counts."
<headknod-ding continues>
Me: "I'm really confused. Maybe I'm looking at the data wrong. On the slide where you highlighted all the bottlenecks, the histogram shows the latency is the database, I mean...it's right there, in red. Am I looking at it wrong?"
<this was meeting with 20+ other devs, mgrs, a VP, the Panorama engineer>
DeptMgr: "Yes you are! I know MyLibrary is your baby. You need to check your ego at the door and face the facts. Your MyLibrary is a failed experiment and needs to be exterminated from this system!"

Fast forward 9 months, maybe 50% of the projects updated, come across the documentation left from the Panorama. Even after the removal of MyLibrary, there was zero increases in performance. The engineer recommended DBAs start optimizing their indexes and other N+1 problems discovered. I decide to ask the developer who lead the re-write.

Me: "I see that removing MyLibrary did nothing to improve performance."
Dev: "Yes, DeptMgr was pissed. He was ready to throw the Panorama engineer out a window when he said the problems were in the database all along. Didn't you say that?"
Me: "Um, so is this re-write project dead?"
Dev: "No. Removing MyLibrary introduced all kinds of bugs. All the boilerplate ADO.Net code caused a lot of unhandled exceptions, then we had to go back and write exception handling code."
Me: "What a failure. What dipshit would think writing more code leads to less bugs?"
Dev: "I know, I know. We're so far behind schedule. We had to come up with something. I ended up writing a library to make replacing MyLibrary easier. I called it KnightRider. Like the TV show. Everyone is excited to speed up their code with KnightRider. Same method names, same exception handling. All we have to do is replace MyLibrary with KnightRider and we're done."
Me: "Won't the bottlenecks then point to KnightRider?"
Dev: "Meh, not my problem. Panorama meets primarily with the DBAs and the networking team now. I doubt we ever use Panorama to look at our C# code."

Needless to say, I was (still) pissed that they had used MyLibrary as dirty word and a scapegoat for months when they *knew* where the problems were. Pissed enough for a flamethrower? Maybe.

I worked once in a company which had this tourist app which should show places on map of the city. Unfortunately it slowed the App down to load more then a couple of places. Their solution was to limit the number of loaded places to teb and prohibited zomming out. I made it handle thousands of places at the same time. Main reason for the Performance issue was, that they sent all data they had about places big, big json objects with large text blobs) to the frontend. This part was easy, I instead sent only the data needed for the map like coordinates and icon type obviously. But still the backend struggeled hard with many objects from the DB, because they built a really shitty orm or what ever this was supposed to be: every line of data retrieved from the DB was immideatly wrapped in some class wich direved from another class which had some magic methods in it which caused some absurd loops over all other obejcts and even more DB queries in unexpected moments and also in the fucking constructor. So it turrned out that the map issue was only the top of the iceberg, since using any data from the DB was extremely expensive. The hard part was to understand the insaness of this abnormination and find the bottlenecks.

"CTO" here.
Two week ago the CEO informs me that the "investor" want to put me in contact urgently with an external software house to help me with my "bottlenecks".
The investor goes immediately on holiday, so it's not available for explanations. The CEO doesn't know much.

Today I meet the software house CTO and CEO.
They tell me that I should do a transfer of knowledge with them. That they will respect my requirements, my schedule and that they want to help me.

During the meeting the business consultant explains "his" vision. Some new development nobody understand. Not even the CEO. The other cofounder is probably in disagreement but stay silent.

I agree to cooperate with them in due time and with due scope and planning.

It appears they already signed a contract with the investor. The investor is offering to us 40 days of a senior developer, for "free".
The CEO doesn't even know the economical details of the contract and he is surprised that has been signed.He also didn't know that a person will come over for 40 (?) days and that we will have to pay the transfer expenses.

I try to be friendly. I explain to them the issues I need to solve. I say specifically that I need help on certain tasks and that my wish is that nothing "new" will start until we fix some obvious problems.

After leaving, in the evening I receive an email from the software house guy, telling me that next week I MUST allocate a slot for technical transfer and the 2 weeks after for on site training. Like that. He also mention we "agreed" on that which is false. We agreed on me deciding the timing.

We are only 2 developers, at the moment and the other one will be on holiday next week, so I'm trying to get from him a lot of things I don't know because I don't know everything.

I'm not even sure I'll be able to explain how to prepare all the environment.
Worst thing is that I don't know what will be the scope of the project.

I really don't know how to behave.
I wrote back setting my conditions. I have holiday too. I have to prepare "documentation", explanation, etc.

I don't want the "senior dev" coming when I'm not present.
Maybe I was too weak answering and I should have started a fight immediately. Because he actually AGREED to let me decide and after that he set conditions on me immediately.

I don't know.
My stomach is burning, I had a very bad digestion with fever and headache, feel like puking, plus I spent several evening hours fixing the fucking Linux kernel bug.

I want to survive. I don't want to let them oust me in this stupid way. I want to fight.
I know that if I will explode, scream or whatever I will be at fault and I'll accelerate my demise.
When I try to be "diplomatic" actually I end up being weak.
When I try to be assertive I'm in fact rude and hysterical.
I can't think anything else.

This is what burnout looks like.

Typical code life?

1. Write rough comments
2. Write more detailed comments
3. Write pseudo code
4. Write semi-working but definitely ugly code
5. Write working but very ugly code
6. Refactor the code to be nicer, check for patterns, bottlenecks and other bits and pieces
7. Push to git the "final" code
8. After few months blame whoever wrote the code
9. Refactor all the things!

---

This happened in my career more than once and still - it seems like the best option out there to get things done. What do you guys think? Should something be added/removed from this? Is this over-complicated or what?

My boss's SQL schema has no foreign keys and he said he left them out intentionally because they should be handled in the application layer and they're a large performance impact.

This is a fresh greenfield project and he's already pre-optimizing for problems we don't have yet, on things that may or not be bottlenecks using ideas (e.g. foreign keys have huge performance costs on mariadb/auora) with no hard data or facts to back them up.

Let's start a new project with some technical debt!

When you, being severely understaffed, ask your boss and the president of the company to hire another embedded dev and a technical QA to work with you at the minimum, with hard proof of the need, because you work alone on a project that was initially poorly designed even though it is the core business of the company and that it bottlenecks literally almost every other sphere of the company...

and instead, he propose you to find by yourself 4 offshore unpaid interns to help you develop an EMBEDDED SOFTWARE that requires specific EXPENSIVE HARDWARE to run and test...

because paying new employees or even an intern is too expensive even though we had record sales the last few months and that the government pays 50% of salaries until the pandemic is gone.

Fuck this shit I'm out! I'm tired of not being paid enough to fix other people's mistakes. Have fun with your failing business. Btw, this is all your fault.

Is your code green?

I've been thinking a lot about this for the past year. There was recently an article on this on slashdot.

I like optimising things to a reasonable degree and avoid bloat. What are some signs of code that isn't green?

* Use of technology that says its fast without real expert review and measurement. Lots of tech out their claims to be fast but actually isn't or is doing so by saturation resources while being inefficient.
* It uses caching. Many might find that counter intuitive. In technology it is surprisingly common to see people scale or cache rather than directly fixing the thing that's watt expensive which is compounded when the cache has weak coverage.
* It uses scaling. Originally scaling was a last resort. The reason is simple, it introduces excessive complexity. Today it's common to see people scale things rather than make them efficient. You end up needing ten instances when a bit of skill could bring you down to one which could scale as well but likely wont need to.
* It uses a non-trivial framework. Frameworks are rarely fast. Most will fall in the range of ten to a thousand times slower in terms of CPU usage. Memory bloat may also force the need for more instances. Frameworks written on already slow high level languages may be especially bad.
* Lacks optimisations for obvious bottlenecks.
* It runs slowly.
* It lacks even basic resource usage measurement.

Unfortunately smells are not enough on their own but are a start. Real measurement and expert review is always the only way to get an idea of if your code is reasonably green.

I find it not uncommon to see things require tens to hundreds to thousands of resources than needed if not more.

In terms of cycles that can be the difference between needing a single core and a thousand cores.

This is common in the industry but it's not because people didn't write everything in assembly. It's usually leaning toward the extreme opposite.

Optimisations are often easy and don't require writing code in binary. In fact the resulting code is often simpler. Excess complexity and inefficient code tend to go hand in hand. Sometimes a code cleaning service is all you need to enhance your green.

I once rewrote a data parsing library that had to parse a hundred MB and was a performance hotspot into C from an interpreted language. I measured it and the results were good. It had been optimised as much as possible in the interpreted version but way still 50 times faster minimum in C.

I recently stumbled upon someone's attempt to do the same and I was able to optimise the interpreted version in five minutes to be twice as fast as the C++ version.

I see opportunity to optimise everywhere in software. A billion KG CO2 could be saved easy if a few green code shops popped up. It's also often a net win. Faster software, lower costs, lower management burden... I'm thinking of starting a consultancy.

The problem is after witnessing the likes of Greta Thunberg then if that's what the next generation has in store then as far as I'm concerned the world can fucking burn and her generation along with it.

Fucking cant solve productivity problem. Since I was working as programmer, about 8-9 years experience, constant complaints about my productivity and some jobs even fired me for this. Only one job did not complain and I worked in it longest time but still I was worrying very often about my productivity. It is fucking annoying. Why others are productive and I am not. How the fuck to find biggest bottlenecks to know which to work on.
I know I am not knowing technology perfectly, and from time to time I get stuck and so I ask other people help, or somehow manage to find solution myself but it takes more time. But dont know if that is the biggest issue. Should I intecify my learning? I am regularly studying, and working with symfony about 2 years, so I think I should know enough to be productive even with those strugles from time to time. But maybe they are too often?
I have listened book "deep work" and basic thing I think from this is - to minimize disctractions and learn to focus very well. But to minimize like in this book, I should work alone in my room. And even then I would like each hour for few minutes to read some new or smth, which this book says is bad, but a lot of people do that and they somehow get away with it. Plus if I work alone in my room, my social skills might get worse, and we all need social skills, even programmers.
I so envy to others who know how to be productive. I would hate if the only thing to be more productive is to reduce quality of the code, make more bugs. THats fucking cheating system.

Freelance Headhunters...

No idea where they got my name and the number of my work phone...

- they do not give a damn and call me 1045 am to discuss "EXCITING BUSINESS OPPORTUNITIES™"

- they are sure I am having a case of stockholm syndrome, because I could NEVER be satisfied in my current employment

- they call every other day with a new number and start the same process again

How fucking dense can people get?

Not dev, but a perf-eng confidence boost.

Our company was hired by a client to onboard perf-testing process and do some perf-related go-live stuff. Basically, make sure the app meets the SLAs.

Our company mobilized some internal resources for the task. The had 3-4 months. 2 months later they realized they won't pull it off. What a shame...

When the threat of dropping the ball and losing the client and recommendations became very real, they engaged us. Half the time, half the resources, a worried and annoyed client who now wants to control the whole initiative.

During the first 2 meetings we get the general idea of what they have, what they want. We take some time to prepare a plan to make it on time. The client argues our plan, mostly because one of the main points was mocking downstream dependencies [integrations]. He asks, then demands to do it all with live integrations. We explain why this is an incredible risk and why we should do it the proposed way. He disagrees.

Alright then... Maybe he knows smth we don't. Let's do it the risky way...

A month later test results are far from the target. I did my best with app de-bottlenecking and fine-tuning. But since the live integrations do not deliver, they hide other bottlenecks. The initiative is stuck.

Finally, the client agrees to do it with mocking. But now there's no time left as it will take almost a month to prepare mocks...

The client agrees we should have done it our way from the start. They postpone the go-live and we carry out our testing and tuning the right way.

That was one expensive and long "I told you so". But it boosted our [perf team's] confidence to the top and beyond :)

don't tell us how to do our job, unless you do want extra expenses

Sometimes I feel like my brain has a really fast CPU. But the RAM frequency really bottlenecks the whole system.
Any suggestions to upgrade the RAM?

A "portal" built on Drupal 7. Started by someone who cannot do anything outside Drupal, and overseen by someone who believes JS to be "low level programming" (he literally said that).

What normally would be a table with 7 columns is instead 7 tables joined together. That goes for each data structure.

Each page, built in a separate module, either manually includes the same css files, or simply copy/pastes them.

Old, legacy modules have been hacked, and now depend on newer modules... Which, in turn, depend on the same old ones.

The theme contains huge, hardcoded parts of logic, so it can never be updated.

Worst part of it? It's only 3 years old. And there are people buying it as SaS. Already hitting bottlenecks at 2k users.

When the CTO/CEO of your "startup" is always AFK and it takes weeks to get anything approved by them (or even secure a meeting with them) and they have almost-exclusive access to production and the admin account for all third party services.

Want to create a new messaging channel? Too bad! What about a new repository for that cool idea you had, or that new microservice you're expected to build. Expect to be blocked for at least a week.

When they also hold themselves solely responsible for security and operations, they've built their own proprietary framework that handles all the authentication, database models and microservice communications.

Speaking of which, there's more than six microservices per developer!

Oh there's a bug or limitation in the framework? Too bad. It's a black box that nobody else in the company can touch. Good luck with the two week lead time on getting anything changed there. Oh and there's no dedicated issue tracker. Have you heard of email?

When the systems and processes in place were designed for "consistency" and "scalability" in mind you can be certain that everything is consistently broken at scale. Each microservice offers:

1. Anemic & non-idempotent CRUD APIs (Can't believe it's not a Database Table™) because the consumer should do all the work.
2. Race Conditions, because transactions are "not portable" (but not to worry, all the code is written as if it were running single threaded on a single machine).
3. Fault Intolerance, just a single failure in a chain of layered microservice calls will leave the requested operation in a partially applied and corrupted state. Ger ready for manual intervention.
4. Completely Redundant Documentation, our web documentation is automatically generated and is always of the form //[FieldName] of the [ObjectName].
5. Happy Path Support, only the intended use cases and fields work, we added a bunch of others because YouAreGoingToNeedIt™ but it won't work when you do need it. The only record of this happy path is the code itself.

Consider this, you're been building a new microservice, you've carefully followed all the unwritten highly specific technical implementation standards enforced by the CTO/CEO (that your aware of). You've decided to write some unit tests, well um.. didn't you know? There's nothing scalable and consistent about running the system locally! That's not built-in to the framework. So just use curl to test your service whilst it is deployed or connected to the development environment. Then you can open a PR and once it has been approved it will be included in the next full deployment (at least a week later).

Most new 'services' feel like the are about one to five days of writing straightforward code followed by weeks to months of integration hell, testing and blocked dependencies.

When confronted/advised about these issues the response from the CTO/CEO
varies:

(A) "yes but it's an edge case, the cloud is highly available and reliable, our software doesn't crash frequently".
(B) "yes, that's why I'm thinking about adding [idempotency] to the framework to address that when I'm not so busy" two weeks go by...
(C) "yes, but we are still doing better than all of our competitors".
(D) "oh, but you can just [highly specific sequence of undocumented steps, that probably won't work when you try it].
(E) "yes, let's setup a meeting to go through this in more detail" *doesn't show up to the meeting*.
(F) "oh, but our customers are really happy with our level of [Documentation]".

Sometimes it can feel like a bit of a cult, as all of the project managers (and some of the developers) see the CTO/CEO as a sort of 'programming god' because they are never blocked on anything they work on, they're able to bypass all the limitations and obstacles they've placed in front of the 'ordinary' developers.

There's been several instances where the CTO/CEO will suddenly make widespread changes to the codebase (to enforce some 'standard') without having to go through the same review process as everybody else, these changes will usually break something like the automatic build process or something in the dev environment and its up to the developers to pick up the pieces. I think developers find it intimidating to identify issues in the CTO/CEO's code because it's implicitly defined due to their status as the "gold standard".

It's certainly frustrating but I hope this story serves as a bit of a foil to those who wish they had a more technical CTO/CEO in their organisation. Does anybody else have a similar experience or is this situation an absolute one of a kind?

Leetcode.

It doesn't matter if you've done multiple projects with different tools, languages, team sizes and requirements for ANY company / org etc.

You will feel fucking stupid while taking too long on some of these questions.

I know interview questions are mostly to test your critical thinking skills but fuck I feel so bad after 2 evenings of doing this shit.

It is addictive though...

Need advice about switching to contracting.

TL;DR;

So I had 2 years of exp as an android dev, then I had a 1.5 year gap from doing android and now for the past 6 months Ive been doing android again fulltime. Im thinking of switching to contracting due to my debts and boring project and life crushing slow corporate processes in my current fulltime job, so I need tips and advices as to where should I start looking for new contracting gigs and in general what should I pay attention to. If it helps, I am based in EU, but am open to any EU/US gigs.

Now the full story:

Initially when I joined my current fulltime job after a break I had zero confidence, lowered my and employers expectations, joined as a junior but quickly picked up the latest standards and crushed it. Im doing better than half devs in my scrum team right now and would consider myself to be a mid level right now.

Asked for a 50% bump, manager kinda okayed it but the HQ overseas is taking a very long time to give me the actual bump. I have been waiting for 10 weeks already (lots of people in the decision chain were on and off vacations due to summer, also I guess manager sent this request to HQ too late, go figure). Anyways its becoming unnaceptable and I feel like its time for a change.

Now since I have mortgage and bills to pay, even with the bump that I requested that would leave me with like maximum 700-800 bucks a month after all expenses. I have debts of around 20k and paying them back at this rate would take 3 years at least and sounds like a not viable plan at all.

Also it does not help that the project Im working on is full of legacy and Im not learning anything new here. Corporate life seems to be very slow, lots of red tape kills creativity and so on. I remember in startups I was cooking features left and right each sprint, in here deploying a simple popup feature sometimes takes weeks due to incompetence in the chain. I miss the times where I worked in startups, did my job learned nre skills and after 6 months could jump on another exciting gig. Im not growing here anymore.

So because my ADD brain seems to be suited much better for working in startups, and also I need to make more money quick and I dont see a future in current company, I am thinking of going back to contracting. All I need right now is to build a few side apps, get them reviewed by seniors and fill my knowledge gaps. Then I plan of starting interviewing as a mid level or even a senior for that matter, since I worked with actual seniors and to be honest I dont think getting up to their level would be rocket science.

Only difference between mid and senior devs that I see atleast in my current company is that seniors are taking on responsibility more often, and they also take care of our tools, such as CD/CI, pipeline scripts, linters and etc. Usually seniors are the ones who do the research/investigations and then come up with actual tasks/stories for mids/juniors. Also seniors introduce new dependencies and update our stack, solve some performance issues and address bottlenecks and technical debt. I dont think its rocket science, also Ive been the sole dev responsible for apps in the past and always did decent work. Turns out all I needed was to test myself in an environment full of other devs, thats it. My only bottleneck was the imposter syndrome because I was a self taught dev who worked most of my career alone.

Anyways I posted here asking for some tips and advices on how to begin my search for new contract opportunities. I am living in EU, can you give me some decent sites where I could just start applying? Also I would appreciate any other tips opinions and feedback. Thanks!

The timelines at my workplace are too short that it's impossible to actually build anything or observe procedures like testing, software techniques for maintaining oop code, telemetry and other things I may have learnt along the way

So application templates are the order of the day. They pull solutions off the shelf, edit the interface, hand over to clients at an alarming rate (sometimes, within a matter of days!). So yesterday, the cto asked for ways I can recommend that the team is made more efficient. He takes what I say very seriously, owing to Suphle's appendix chapter as well as the issues its blueprint set out to solve

Like I said, those do not apply here. I mean, the developers I've met are making do and winging it. I'm the one struggling to adapt to rummaging through templates and customise shit

Maybe I'm over thinking it cuz there's no sense in fixing something that's not broken. So far, only flaw I've observed (because the product designer has complained to me bitterly that the devs hardly ever translates his prototypes verbatim), is the need for a dedicated mobile developer (not that multifunctional, confused portfolio called "fullstack). But I didn't raise this since the time frames hardly even afford time for writing apis or writing mobile code. You'd be surprised to realise that everything a client can possibly ask for is already somewhere, built at a higher standard than you can replicate

My question now is, what other positive novelty can I bring aboard? How can this process be further optimised? If it can't, what suggestions outside regular software development or this work flow can I bring to the table?

Personally, I'm considering asking him to tell me bottlenecks if he has identified any. But it's very likely that he would already have begun working towards it if he knew them. I suspect he needs someone outside the system to see what is lacking or a new addition that could even be a distant, outlandish branch of the tech market, but drive the company towards more profit