I hired a woman for senior quality assurance two weeks ago. Impressive resume, great interview, but I was met with some pseudo-sexist puzzled looks in the dev team.

Meeting today. Boss: "Why is the database cluster not working properly?"

Team devs: "We've tried diagnosing the problem, but we can't really find it. It keeps being under high load."

New QA: "It might have something to do with the way you developers write queries".

She pulls up a bunch of code examples with dozens of joins and orderings on unindexed columns, explains that you shouldn't call queries from within looping constructs, that it's smart to limit the data with constraints and aggregations, hints at where to actually place indexes, how not to drag the whole DB to the frontend and process it in VueJS, etc...

New QA: "I've already put the tasks for refactoring the queries in Asana"

I'm grinning, because finally... finally I'm not alone in my crusade anymore.

Boss: "Yeah but that's just that code quality nonsense Bittersweet always keeps nagging about. Why is the database not working? Can't we just add more thingies to the cluster? That would be easier than rewriting the code, right?"

Dev team: "Yes... yes. We could try a few more of these aws rds db.m4.10xlarge thingies. That will solve it."

QA looks pissed off, stands up: "No. These queries... they touch the database in so many places, and so violently, that it has to go to therapy. That's why it's down. It just can't take the abuse anymore. You could add more little brothers and sisters to the equation, but damn that would be cruel right? Not to mention that therapy isn't exactly cheap!"

Dev team looks annoyed at me. My boss looks even more annoyed at me. "You hired this one?"

I keep grinning, and I nod.

"I might have offered her a permanent contract"

Oh, man, I just realized I haven't ranted one of my best stories on here!

So, here goes!

A few years back the company I work for was contacted by an older client regarding a new project.

The guy was now pitching to build the website for the Parliament of another country (not gonna name it, NDAs and stuff), and was planning on outsourcing the development, as he had no team and he was only aiming on taking care of the client service/project management side of the project.

Out of principle (and also to preserve our mental integrity), we have purposely avoided working with government bodies of any kind, in any country, but he was a friend of our CEO and pleaded until we singed on board.

Now, the project itself was way bigger than we expected, as the wanted more of an internal CRM, centralized document archive, event management, internal planning, multiple interfaced, role based access restricted monster of an administration interface, complete with regular user website, also packed with all kind of features, dashboards and so on.

Long story short, a lot bigger than what we were expecting based on the initial brief.

The development period was hell. New features were coming in on a weekly basis. Already implemented functionality was constantly being changed or redefined. No requests we ever made about clarifications and/or materials or information were ever answered on time.

They also somehow bullied the guy that brought us the project into also including the data migration from the old website into the new one we were building and we somehow ended up having to extract meaningful, formatted, sanitized content parsing static HTML files and connecting them to download-able files (almost every page in the old website had files available to download) we needed to also include in a sane way.

Now, don't think the files were simple URL paths we can trace to a folder/file path, oh no!!! The links were some form of hash combination that had to be exploded and tested against some king of database relationship tables that only had hashed indexes relating to other tables, that also only had hashed indexes relating to some other tables that kept a database of the website pages HTML file naming. So what we had to do is identify the files based on a combination of hashed indexes and re-hashed HTML file names that in the end would give us a filename for a real file that we had to then search for inside a list of over 20 folders not related to one another.

So we did this. Created a script that processed the hell out of over 10000 HTML files, database entries and files and re-indexed and re-named all this shit into a meaningful database of sane data and well organized files.

So, with this we were nearing the finish line for the project, which by now exceeded the estimated time by over to times.

We test everything, retest it all again for good measure, pack everything up for deployment, simulate on a staging environment, give the final client access to the staging version, get them to accept that all requirements are met, finish writing the documentation for the codebase, write detailed deployment procedure, include some automation and testing tools also for good measure, recommend production setup, hardware specs, software versions, server side optimization like caching, load balancing and all that we could think would ever be useful, all with more documentation and instructions.

As the project was built on PHP/MySQL (as requested), we recommended a Linux environment for production. Oh, I forgot to tell you that over the development period they kept asking us to also include steps for Windows procedures along with our regular documentation. Was a bit strange, but we added it in there just so we can finish and close the damn project.

So, we send them all the above and go get drunk as fuck in celebration of getting rid of them once and for all...

Next day: hung over, I get to the office, open my laptop and see on new email. I only had the one new mail, so I open it to see what it's about.

Lo and behold! The fuckers over in the other country that called themselves "IT guys", and were the ones making all the changes and additions to our requirements, were not capable enough to follow step by step instructions in order to deploy the project on their servers!!!

[Continues in the comments]

So, you start with a PHP website.

Nah, no hating on PHP here, this is not about language design or performance or strict type systems...

This is about architecture.

No backend web framework, just "plain PHP".

Well, I can deal with that. As long as there is some consistency, I wouldn't even mind maintaining a PHP4 site with Y2K-era HTML4 and zero Javascript.

That sounds like fucking paradise to me right now. 😍

But no, of course it was updated to PHP7, using Laravel, and a main.js file was created. GREAT.... right? Yes. Sure. Totally cool. Gotta stay with the times. But there's still remnants of that ancient framework-less website underneath. So we enter an era of Laravel + Blade templates, with a little sprinkle of raw imported PHP files here and there.

Fine. Ancient PHP + Laravel + Blade + main.js + bootstrap.css. Whatever. I can still handle this. 🤨

But then the Frontend hipsters swoosh back their shawls, sip from their caramel lattes, and start whining: "We want React! We want SPA! No more BootstrapCSS, we're going to launch our own suite of SASS styles! IT'S BETTER".

OK, so we create REST endpoints, and the little monkeys who spend their time animating spinners to cover up all the XHR fuckups are satisfied. But they only care about the top most visited pages, so we ALSO need to keep our Blade templated HTML. We now have about 200 SPA/REST routes, and about 350 classic PHP/Blade pages.

So we enter the Era of Ancient PHP + Laravel + Blade + main.js + bootstrap.css + hipster.sass + REST + React + SPA 😑

Now the Backend grizzlies wake from their hibernation, growling: We have nearly 25 million lines of PHP! Monoliths are evil! Did you know Netflix uses microservices? If we break everything into tiny chunks of code, all our problems will be solved! Let's use DDD! Let's use messaging pipelines! Let's use caching! Let's use big data! Let's use search indexes!... Good right? Sure. Whatever.

OK, so we enter the Era of Ancient PHP + Laravel + Blade + main.js + bootstrap.css + hipster.sass + REST + React + SPA + Redis + RabbitMQ + Cassandra + Elastic 😫

Our monolith starts pooping out little microservices. Some polished pieces turn into pretty little gems... but the obese monolith keeps swelling as well, while simultaneously pooping out more and more little ugly turds at an ever faster rate.

Management rushes in: "Forget about frontend and microservices! We need a desktop app! We need mobile apps! I read in a magazine that the era of the web is over!"

OK, so we enter the Era of Ancient PHP + Laravel + Blade + main.js + bootstrap.css + hipster.sass + REST + GraphQL + React + SPA + Redis + RabbitMQ + Google pub/sub + Neo4J + Cassandra + Elastic + UWP + Android + iOS 😠

"Do you have a monolith or microservices" -- "Yes"

"Which database do you use" -- "Yes"

"Which API standard do you follow" -- "Yes"

"Do you use a CI/building service?" -- "Yes, 3"

"Which Laravel version do you use?" -- "Nine" -- "What, Laravel 9, that isn't even out yet?" -- "No, nine different versions, depends on the services"

"Besides PHP, do you use any Python, Ruby, NodeJS, C#, Golang, or Java?" -- "Not OR, AND. So that's a yes. And bash. Oh and Perl. Oh... and a bit of LUA I think?"

2% of pages are still served by raw, framework-less PHP.

I’m a senior dev at a small company that does some consulting. This past October, some really heavy personal situation came up and my job suffered for it. I raised the flag and was very open with my boss about it and both him and my team of 3 understood and were pretty cool with me taking on a smaller load of work while I moved on with some stuff in my life. For a week.

    Right after that, I got sent to a client. “One month only, we just want some presence there since it’s such a big client” alright, I guess I can do that. “You’ll be in charge of a team of a few people and help them technically.” Sounds good, I like leading!

    So I get here. Let’s talk technical first: from being in a small but interesting project using Xamarin, I’m now looking at Visual Basic code, using Visual Studio 2010. Windows fucking Forms.

    The project was made by a single dev for this huge company. She did what she could but as the requirements grew this thing became a behemoth of spaghetti code and User Controls. The other two guys working on the project have been here for a few months and they have very basic experience at the job anyways. The woman that worked on the project for 5 years is now leaving because she can’t take it anymore.

    And that’s not the worse of it. It took from October to December for me to get a machine. I literally spent two months reading on my cellphone and just going over my shitty personal situation for 8 hours a day. I complained to everyone I could and nothing really worked.

    Then I got a PC! But wait… no domain user. Queue an extra month in which I could see the Windows 7 (yep) log in screen and nothing else. Then, finally! A domain user! I can log in! Just wait 2 extra weeks for us to give your user access to the subversion rep and you’re good to go!

    While all of this went on, I didn’t get an access card until a week ago. Every day I had to walk to the reception desk, show my ID and request they call my boss so he could grant me access. 5 months of this, both at the start of the day and after lunch. There was one day in particular, between two holidays, in which no one that could grant me access was at the office. I literally stood there until 11am in which I called my company and told them I was going home.

    Now I’ve been actually working for a while, mostly fixing stuff that works like crap and trying to implement functions that should have been finished but aren’t even started. Did I mention this App is in production and being used by the people here? Because it is. Imagine if you will the amount of problems that an application that’s connecting to the production DB can create when it doesn’t even validate if the field should receive numeric values only. Did I mention the DB itself is also a complete mess? Because it is. There’s an “INDEXES” tables in which, I shit you not, the IDs of every other table is stored. There are no Identity fields anywhere, and instead every insert has to go to this INDEXES table, check the last ID of the table we’re working on, then create a new registry in order to give you your new ID. It’s insane.

   

    And, to boot, the new order from above is: We want to split this app in two. You guys will stick with the maintenance of half of it, some other dudes with the other. Still both targeting the same DB and using the same starting point, but each only working on the module that we want them to work in. PostmodernJerk, it’s your job now to prepare the app so that this can work. How? We dunno. Why? Fuck if we care. Kill you? You don’t deserve the swift release of death.

    Also I’m starting to get a bit tired of comments that go  ‘THIS DOESN’T WORK and ‘I DON’T KNOW WHY WE DO THIS BUT IT HELPS and my personal favorite  ‘??????????????????????

So my manager (a 29 y/o, who hardly can use a mac) walks towards me with a hint of panic in his eyes.

Manager: Hey commander keen, do you know how to use vertical look up in excel, I've tried, and looked at tutorials.

Me: yeah I really don't know excel (and not willing to learn, especially on the fly), I don't even have excel installed, I can write I script that does what you want.

Manager: No you have enough on your plate

3 hours later

Manager: hey I still can't figure it out, could you solve it with a script, won't that take to long

Me: no send me the files, Ill do it with a script.

I start writing 2 for loops and wait for the file, 10 ish minutes later its basicly done, just need to put in the column indexes.

I send a message on both slack channels (hey are you going to email or slack me the file)

After a hour I walk to his desk and again ask him for the file.

Manager a good 2 hours later on slack: Hey I just send you the file, I hope its not to much work, it has to happen asap.

So if you have kids, and they are not that bright by some kind of birth defect, don't worry, they can always become a manager.

But you can't get me down today. I hit 2000 upvotes and the employer is unknowingly a proud sponsor for reading and writing all these rants and comments :-) thnx devrant

I stare through the blueish black backgrounds and blurry colorful syntax into a somewhat familiar office within a mirrored world. That damned reflective glass layer covering these meaningless pixels is certainly not on my side.

The rushing sound of transactions flowing through cables is silenced today. Some blood cloth in the invoicing system is zeroing out everything after the currency mark.

While sighing I spin a one-and-a-half pirouette on my desk chair — even when desperate, you shouldn't give up on style — I take three steps away from my screen and try to harmonize my thoughts.

So much noise, everywhere... Noise from within?

I have been stuck at the apogee of an inhale for a while now. Locked into some masochistic constriction, self-punishment for the blindness which stings my ego.

Just fucking take a deep breath you asshole...

I freeze in place, and fall backwards.

Patterns on the creamy drywall rapidly vibrate and synchronize on vivid rhythms of respiration and resonating basslines. Deep indigo rainbows ripple through tiny veins, in-between chalky grains, raining as fine magenta dust through the ceiling frames.

My bare feet slide over soft oscillating concrete, fine flows of unsievable sand surrounded by toes, toes surrounded by streaming variables veiled in obscure vile abstractions.

A jadegreen field of vectored compressions resiliently rumbles and bounces through the clearances and corners of the vibrant concrete office cave, whispering in tongues. I try to voice my woes in little blips and bleeps but I seem to be missing an asymmetric key to their shrouded sequenced speech.

Suddenly, a wild turbulence breaks up all signals.

Joanna floats by in her tipsy effervescent cloud of disordered black hair and alcohol perfume, one hand grasping grapes, her other waving at me.

With every finger she moves a thousand tensors propagating paradoxically flawed but perfect pieces of an intricate surreal picture, sketching whole constellations of possible paths throughout the leafs of the giant Ficus next to her desk.

She stops dead in her tracks, and asks somewhat hypocritically: "Are you high?"

I can not discern the meaning of her words, and respond stoically.

"Joanna! Check out those branches!".

"Pun intended?", she giggles.

I'm focused on her grapeless hand, her fingers stretching to reach the lush little tree.

On touch, the plant shivers, grappled in the tight net of the puppet master. She pulls her strings, applying measured weights, all nodes normalize, and Joanna speaks in an oddly soft tone:

"Isn't it beautiful, how so many models emulate nature"

Her cheek buried in foliage she babbles on about unbalanced search trees and machine learning models... but from the tips of her fingers tables and indexes flow into the plant. Users, payments, tariffs, invoices and taxes crawl over the bark, joining at thicker branches, joining at the stem....

Joining. JOINING. A JOIN.

"IF THERE'S NO FUCKING TAX MULTIPLIER IN THIS LEFT JOIN, EVERYTHING COALESCES TO ZERO" I shout at a perplexed Joanna who squeezes grape juice over her desk. I hop on the beat to my keyboard. She looks puzzled, hugs her Ficus tightly, and reaches for the whiskey bottle behind her monitor.

Attracted by my exclamation, Tom from finance swings open the door, while I push my branch.

I look at Joanna still half hiding between the leaves, and I laugh at her: "Branches! Oh, lame, I finally got it!"

Tom's heavy voice interrupts me: "Does this mean... does this mean that the invoicing bug is resolved?".

I smile at Tom with his tailored suit and waxed hair. "The money is flowing once more. All debts are being settled."

He releases his breath in relief, which he seems to have held since that morning as well.

Joanna adds: "Although I think he is forever indebted to my Ficus".

I nod.

Working on a database priorly designed and maintained by some private agency.

The fuck I'm dealing with!

Boolean values stored as 'TRUE'/'FALSE'. It's varchar, my dudes.

There are no FK relations. Just the values of IDs in a column.

There are no indexes, all on just the PKs, nothing else. Nothing.

Null, what's that? I'm dealing with 'N/A', my dudes.

Unique key, what's that? The table which stores users has all the fields nullable. Email is not unique ( even though that's the required behaviour).

ALL the numeric values are stored as varchar. Varchar, my dudes. Varchar. '1', '1.1'

And finally, the good ole, 1 table to rule them all. Normalisation, fuck that.

And what's the root cause of all this? My PM used to hand them Excel sheets she maintains on her local system. FTW. I don't have a enough explanations.

So I'm back from vacation! It's my first day back, and I'm feeling refreshed and chipper, and motivated to get a bunch of things done quickly so I can slack off a bit later. It's a great plan.

First up: I need to finish up tiny thing from my previous ticket -- I had overlooked it in the description before. (I couldn't test this feature [push notifications] locally so I left it to QA to test while I was gone.)

It amounted to changing how we pull a due date out of the DB; some merchants use X, a couple use Y. Instead of hardcoding them, it would use a setting that admins can update on the fly.

Several methods deep, the current due date gets pulled indirectly from another class, so it's non-trivial to update; I start working through it.

But wait, if we're displaying a due date that differs from the date we're actually using internally, that's legit bad. So I investigate if I need to update the internals, too.

After awhile, I start to make lunch. I ask my boss if it's display-only (best case) and... no response. More investigating.

I start to make a late lunch. A wild sickness appears! Rush to bathroom; lose two turns.

I come back and get distracted by more investigating. I start to make an early dinner... and end up making dinner for my monster instead.

Boss responds, tells me it's just for display (yay!) and that we should use <macro resource feature> instead.

I talk to Mr. Product about which macros I should add; he doesn't respond.

I go back to making lunch-turn-dinner for myself; monster comes back and he's still hungry (as he never asks for more), so I make him dinner.

I check Slack again; Mr. Product still hasn't responded. I go back to making dinner.

Most of the way through cooking, I get a notification! Product says he's talking it through with my boss, who will update me on it. Okay fine. I finish making dinner and go eat.

No response from boss; I start looking through my next ticket.

No response from boss. I ping him and ask for an update, and he says "What are you talking about?" Apparently product never talked to bossmang =/ I ask him about the resources, and he says there's no need to create any more as the one I need already exists! Yay!

So my feature went from a large, complex refactor all the way down to a -1+2 diff. That's freaking amazing, and it only took the entire day!

I run the related specs, which take forever, then commit and push.

Push rejected; pull first! Fair, I have been gone for two weeks. I pull, and git complains about my .gitignore and some local changes. fine, whatever. Except I forgot I had my .gitignore ignored (skipped worktree). Finally figure that out, clean up my tree, and merge.

Time to run the specs again! Gems are out of date. Okay, I go run `bundle install` and ... Ruby is no longer installed? Turns out one of the changes was an upgrade to Ruby 2.5.8.

Alright, I run `rvm use ruby-2.5.8` and.... rvm: command not found. What. I inspect the errors from before and... ah! Someone's brain fell out and they installed rbenv instead of the expected rvm on my mac. Fine, time to figure it out. `rbenv which ruby`; error. `rbenv install --list`; skyscraper-long list that contains bloody everything EXCEPT 2.5.8! Literally 2.5 through 2.5.7 and then 2.6.0-dev. asjdfklasdjf

Then I remember before I left people on Slack made a big deal about upgrading Ruby, so I go looking. Dummy me forgot about the search feature for a painful ten minutes. :( Search found the upgrade instructions right away, ofc. I follow them, and... each step takes freaking forever. Meanwhile my children are having a yelling duet in the immediate background, punctuated with screams and banging toys on furniture.

Eventually (seriously like twenty-five minutes later) I make it through the list. I cd into my project directory and... I get an error message and I'm not in the project directory? what. Oh, it's a zsh thing. k, I work around that, and try to run my specs. Fail.

I need to update my gems; k. `bundle install` and... twenty minutes later... all done.

I go to run my specs and... RubyMine reports I'm using 2.5.4 instead of 2.5.8? That can't be right. `ruby --version` reports 2.5.8; `rbenv version` reports 2.5.8? Fuck it, I've fought with this long enough. Restarting fixes everything, right? So I restart. when my mac comes back to life, I try again; same issue. After fighting for another ten minutes, I find a version toggle in RubyMine's settings, and update it to 2.5.8. It indexes for five minutes. ugh.

Also! After the restart, this company-installed surveillance "security" runs and lags my computer to hell. Highest spec MacBook Pro and it takes 2-5 seconds just to switch between desktops!

I run specs again. Hey look! Missing dependency: no execjs. I can't run the specs.

Fuck. This. I'll just push and let the CI run specs for me.

I just don't care anymore. It's now 8pm and I've spent the past 11 hours on a -1+2 diff!

What a great first day back! Everything is just the way I left it.

TLDR - you shouldn't expect common sense from idiots who have access to databases.

I joined a startup recently. I know startups are not known for their stable architecture, but this was next level stuff.
There is one prod mongodb server.
The db has 300 collections.
200 of those 300 collections are backups/test collections.
25 collections are used to store LOGS!! They decided to store millions of logs in a nosql db because setting up a mysql server requires effort, why do that when you've already set up mongodb. Lol 😂
Each field is indexed separately in the log.
1 collection is of 2 tb and has more than 1 billion records.
Out of the 1 billion records, 1 million records are required, the rest are obsolete. Each field has an index. Apparently the asshole DBA never knew there's something called capped collection or partial indexes.

Trying to get approval to clean up the db since 3 months, but fucking bureaucracy. Extremely high server costs plus every week the db goes down since some idiot runs a query on this mammoth collection. There's one single set of credentials for everything. Everyone from applications to interns use the same creds.
And the asshole DBA left, making me in charge of handling this shit now. I am trying to fix this but am stuck to get approval from business management. Devs like these make me feel sad that they have zero respect for their work and inability to listen to people trying to improve the system.

Going to leave this place really soon. No point in working somewhere where you are expected to show up for 8 hours, irrespective of whether you even switch on your laptop.
Wish me luck folks.

Biggest challenge I overcame as dev? One of many.

Avoiding a life sentence when the 'powers that be' targeted one of my libraries for the root cause of system performance issues and I didn't correct that accusation with a flame thrower.

What the accusation? What I named the library. Yep. The *name* was causing every single problem in the system.

Panorama (very, very expensive APM system at the time) identified my library in it's analysis, the calls to/from SQLServer was the bottleneck

We had one of Panorama's engineers on-site and he asked what (not the actual name) MyLibrary was and (I'll preface I did not know or involved in any of the so-called 'research') a crack team of developers+managers researched the system thoroughly and found MyLibrary was used in just about every project. I wrote the .Net 1.1 MyLibrary as a mini-ORM to simplify the execution of database code (stored procs, etc) and gracefully handle+log database exceptions (auto-logged details such as the target db, stored procedure name, parameter values, etc, everything you'd need to troubleshoot database errors). This was before Dapper and the other fancy tools used by kids these days.

By the time the news got to me, there was a team cobbled together who's only focus was to remove any/every trace of MyLibrary from the code base. Using Waterfall, they calculated it would take at least a year to remove+replace MyLibrary with the equivalent ADO.Net plumbing.

In a department wide meeting:

DeptMgr: "This day forward, no one is to use MyLibrary to access the database! It's slow, unprofessionally named, and the root cause of all the database issues."
Me: "What about MyLibrary is slow? It's excecuting standard the ADO.Net code. Only extra bit of code is the exception handling to capture the details when the exception is logged."
DeptMgr: "We've spent the last 6 weeks with the Panorama engineer and he's identified MyLibrary as the cause. Company has spent over $100,000 on this software and we have to make fact based decisions. Look at this slide ... "
<DeptMgr shows a histogram of the stacktrace, showing MyLibrary as the slowest>
Me: "You do realize that the execution time is the database call itself, not the code. In that example, the invoice call, it's the stored procedure that taking 5 seconds, not MyLibrary."
<at this point, DeptMgr is getting red-face mad>
AreaMgr: "Yes...yes...but if we stopped using MyLibrary, removing the unnecessary layers, will make the code run faster."
<typical headknodd-ers knod their heads in agreement>
Dev01: "The loading of MyLibrary takes CPU cycles away from code that supports our customers. Every CPU cycle counts."
<headknod-ding continues>
Me: "I'm really confused. Maybe I'm looking at the data wrong. On the slide where you highlighted all the bottlenecks, the histogram shows the latency is the database, I mean...it's right there, in red. Am I looking at it wrong?"
<this was meeting with 20+ other devs, mgrs, a VP, the Panorama engineer>
DeptMgr: "Yes you are! I know MyLibrary is your baby. You need to check your ego at the door and face the facts. Your MyLibrary is a failed experiment and needs to be exterminated from this system!"

Fast forward 9 months, maybe 50% of the projects updated, come across the documentation left from the Panorama. Even after the removal of MyLibrary, there was zero increases in performance. The engineer recommended DBAs start optimizing their indexes and other N+1 problems discovered. I decide to ask the developer who lead the re-write.

Me: "I see that removing MyLibrary did nothing to improve performance."
Dev: "Yes, DeptMgr was pissed. He was ready to throw the Panorama engineer out a window when he said the problems were in the database all along. Didn't you say that?"
Me: "Um, so is this re-write project dead?"
Dev: "No. Removing MyLibrary introduced all kinds of bugs. All the boilerplate ADO.Net code caused a lot of unhandled exceptions, then we had to go back and write exception handling code."
Me: "What a failure. What dipshit would think writing more code leads to less bugs?"
Dev: "I know, I know. We're so far behind schedule. We had to come up with something. I ended up writing a library to make replacing MyLibrary easier. I called it KnightRider. Like the TV show. Everyone is excited to speed up their code with KnightRider. Same method names, same exception handling. All we have to do is replace MyLibrary with KnightRider and we're done."
Me: "Won't the bottlenecks then point to KnightRider?"
Dev: "Meh, not my problem. Panorama meets primarily with the DBAs and the networking team now. I doubt we ever use Panorama to look at our C# code."

Needless to say, I was (still) pissed that they had used MyLibrary as dirty word and a scapegoat for months when they *knew* where the problems were. Pissed enough for a flamethrower? Maybe.

Checking database table with 100m+ transactions ... no indexes or primary key :/

Is obsidian a fucking joke?

Seriously, is it a joke? Why would you ever care so much about indexing literally everything, if the entire thing crashes and/or takes >5min to LITERALLY just open the fucking directory and/or (so help you) if that directory is full of projects/repos or whatever the fuck and the total size of said directory is like >5GB.

WHY THE FUCK WOULD YOU INDEX EVERYTHING? -- "Ohh obsidian's not supposed to be used a fully fledged IDE, ohh obsidian should just handle MD files and normal sized projects, ohh the plugins and ease-of-use" -- Fuck.

There's no fucking real reason to index everything, BY DEFAULT. You open a directory with Obsidian? Doesn't matter, it's 1 byte, it's 100GB, you get indexed. Deal with it. It will use LITERALLY every resource your computer has. I'm surprised it doesn't go galaxy brain and ping if any other computers/devices are on the network and then attempt to connect and use their hardware (obsidian can be like a node!).

How shit can you be at understanding basic data structures and algorithms, where you just revert to based google-chrome brain and let the FUCKING TEXT EDITOR -- OBSIDIAN IS A FUCKING TEXT EDITOR HOLY SHIT -- hog all conceivable memory.

I swear to <some-deity> if anyone fucking says "Ohhhhhhhh actually, it's not a text editor, it has plugins and features and shit, it does all dis cool stff", OR, "Ohhhhh actually, obsidian indexes things for a very specific/rationale/apt/pragmatic/academic reason" OR "ohhhh, I have 100 iphones, 1000 ipads and a trillion desktop computers that each have 256GB of memory, why you hating on obsidian?" then go kick rocks. The fucking lot of you. Are you fucking kidding me.

Why dont people trust you?

I was hired to be an SQL developer, I don't actually get to do much development, normally doing something involving copying and pasting in Excel.

Some of our databases were running slow and we noticed some (a few hundred) indexes were in shit state.

I knocked up a couple of scripts, one to reorganise indexes that were up to a certain amount of fragmentation and one to rebuild the indexes

My boss wants them tested (they were several times in dev) we've had these for over 3 weeks, but she doesn't want to run them.

Instead of fixing hundred of indexes she decided I should contrate on fixing some historic data issues that are preventing 10 indexes from being rebuilt.

Now there are serious issues and the CTO is asking why the indexes haven't been fixed.

I could have done this nearly a month ago, but now it's turned into a huge fucki g deal, and no doubt they'll try and push it back on me

PHP arrays.

The built-in array is also an hashmap. Actually, it's always a hashmap, but you can append to it without specifying indexes and PHP will use consecutive integers. Its performance characteristics? Who knows. Oh, and only strings, ints and null are valid keys.

What's the iteration order for arrays if you use them as hashmaps (string keys)? Well, they have their internal order. So it's actually an ordered hashmap that's being called an array. And you can produce an array which has only integer keys starting with 0, but with non-sequential internal (iteration) order.

This array weirdness has some non-trivial implications. `json_encode` (serializes argument to JSON) assumes an array corresponds to a JSON array if its keys are consecutive integers in increasing order starting with 0, otherwise the array becomes a JSON object. `array_filter` (filters arrays/hashmaps using callback predicate) preserves keys, so it will punch holes in the int key sequence if non-last items are removed, thus turning arrays into hashmaps and changing your JSON structure if you forget to discard keys before serialization.

You may wonder how JSON deserialization works, then? There's a special class for deserialized JSON objects, `stdClass`. It's basically a hashmap too, but it's an object, not an array, and all functions that would normally accept arrays won't work with it. So basically its only use is JSON (de)serialization. You can even cast arrays to objects, producing `stdClass`.

Bonus PHP trivia:

Many functions return nonsensical values. `preg_match`, the regex matching function, returns 1 for success, 0 for no matches and false for malformed regular expression. PHP supports exceptions, so it could just throw one on errors. It would even make more sense to return true, false and null for these three cases. But no, 1, 0 and false. And actual matches are returned by output arg.

`array_walk_recursive`, a function supposed to recursively apply callback to each element of an array. That's what docs say. It actually applies it to leafs only. It will also silently accept object instead of array and "walk" it, but without recursing into deeper objects.

Runtime type enforcing is supported for function arguments and returned values. You can use scalar types, classes, array, null and a few special keywords. There's also a `mixed` keyword, which is used in docs and means "anything". It's syntactically valid, the parser will accept it, but it matches no values in runtime. Calling such function will always cause a runtime error.

Strings can be indexed with negative integers. Arrays can't.

ReflectionClass::newInstanceWithoutConstructor: "Creates a new class instance without invoking the constructor". This one needs no commentary.

`array_map` is pretty self-explanatory if you call it with a callback and an array. Or if you provide more arrays of equal length via varargs, callback will be called with more arguments, one from each array. Makes sense so far. Now, you can also call `array_map` with null instead of callback. In that case it treats provided arrays as rows of a matrix and returns that matrix, transposed.

Rant!

Been working on 'MVP' features of a new product for the past 14 months. Customer has no f**king clue on how to design for performance. An uncomfortable amount of faith was placed on the ORM (ORMs are not bad as long as you know what you are doing) and the magic that the current framework provides. (Again, magic is good so long as you understand what happens behind the smoke and mirrors - but f**k all that... coz hey, productivity, right?). Customer was so focussed on features that no one ever thought of giving any attention to subtler things like 'hey, my transaction is doing a gazillion joins across trizillion tables while making a million calls to the db - maybe I should put more f**king thought into my design.' We foresaw performance and concurrency issues and raised them way ahead of the release. How did the customer respond? By hiring a performance tester. Fair enough - but what did that translate into? Nothing. Nada. Zilch. Hiring a perf tester doesn't automagically fix issues. The perf tester did not have a stable environment, a stable build or anything that is required to do a test with meaningful results. As the release date approached, the customer launched a pilot and things started failing spectacularly with the system not able to support more than 15 concurrent users. WTF! (My 'I told you so' moment) Emails started flying in all directions and the hunt for the scapegoat was on (I'm a sucker for CYA so I was covered). People started pointing in all directions but no one bothered to take a step back and understand what was causing the issues. Numero uno reason for transaction failure was deadlocks. We were using a proprietary DB with kickass tooling. No one bothered to use the tooling to understand what was the resource in contention let alone how to fix the contention. Absolute panic - its like they just froze. Debugging shit and doing the same thing again and again just so that management knew they were upto something. Most of the indexes had a fragmentation of 99.8% - I shit you not. Anywho, we now have a 'war room' where the perf tester needs to script the entire project by tonight and come up with some numbers that will amount to nothing while we stay up and keep profiling the shit out of the application under load.

Lessons learnt - When you foresee a problem make a LOT of noise to get people to act upon it and not wait till it comes back and bites you in the ass. Better yet, try not to get into a team where people can't understand the implications of shitty design choices. War room my ass!

Much obliged if you stop reloading the folder and searching it every five fucking seconds you fucking cunts.

Good god damn this fucking 'feature' of windows 10 grinds my fucking gears. I hit 'x' to stop seeing the visual distraction of the fucking green loading bar when the folders already loaded. Same thing with music. All I want it to do is open and play my fucking song.

Does it do that?

No instead it spends precious cycles updating fucking indexes or sprinkling crack rocks on the corpse of my cpu or whatever cycle fairies at fucking microsoft programmed it to do while wasting my fucking time.

I wish I had a brick and a microsoft programmer within throwing distance, I'd be sorely tempted to nail the motherfucker square in his fucking big fat melon.

Cunts.

fuck count: 8

I don't understand how my managers suddenly forgot that my "down weeks" we're due to technical debt I inherited. The whole on boarding hasn't been in my favor. I've stayed at work everyday til long after work hours, digging through code, trying to get JIRA tickets done, encountering issues specific to our code base that no one would ever discover on their own without docs/help from the original dev. The whole time, I was told that they know what's going on and apologize. I constantly expressed that plenty of what we were doing was building on antipatterns. They acknowledged. When a ticket wasn't done, they always knew the very specific reason and I wasn't faulted. 6 months in, I receive a great annual review. 7 months in? I receive an email titled "Performance Discussion," detailing 4 of those incidents where a ticket was pushed back -- with inaccurate depictions of what actually went down. They actually wrote that I didn't communicate. One part of the report expressed that there were "bugs found in production due to inadequate test coverage." WTF!! Everything made it past code review and QA. What are you talking about?? In fact, the person who wrote that merged my code in each time!!!! Insane!! Anyway, Q2 is partly about cleaning up technical debt, which is a responsibility I have been vested (fantastic). I've deleted about 800 lines of code in the last 2 weeks and added plenty of doc strings. Two of the most important modules our application works from are about 1000 lines of JavaScript each without any comments/docs. I'm changing that, but I don't know if my managers truly know the significance. Someone was recently promoted to my position but manually wrote out a sorting algorithm (specified numeric indexes and all); didn't do shit to earn it but breathe. And while they get more and more praise and responsibility, I'm over here stuck trying to prove myself and live up to why I assume they hired me. It's ridiculous. I love the company, but I'm not getting any sleep and I'm stressed out. It's only been about 7 months and I've been doing everything I can. Why is this happening? What am I doing wrong? I've been developing a recurring (physical) headache and ticks. My heart/chest area sometimes feels like it's lifting weights. I sound like an idiot, pushing so hard for a company that isn't mine, but I take so much pride in being in this position, and I'm so set on proving myself this early in my career (I'm 25).

Found a clever little algorithm for computing the product of all primes between n-m without recomputing them.

We'll start with the product of all primes up to some n.

so [2, 2*3, 2*3*5, 2*3*5*,7..] etc

prods = []
i = 0
total = 1
while i < 100:
....total = total*primes[i]
....prods.append(total)
....i = i + 1

Terrible variable names, can't be arsed at the moment.
The result is a list with the values

2, 6, 30, 210, 2310, 30030, etc.

Now assume you have two factors,with indexes i, and j, where j>i

You can calculate the gap between the two corresponding primes easily.
A gap is defined at the product of all primes that fall between the prime indexes i and j.

To calculate the gap between any two primes, merely look up their index, and then do..
prods[j-1]/prods[i]

That is the product of all primes between the J'th prime and the I'th prime

To get the product of all primes *under* i, you can simply look it up like so:

prods[i-1]

Incidentally, finding a number n that is equivalent to (prods[j+i]/prods[j-i]) for any *possible* value of j and i (regardless of whether you precomputed n from the list generator for prods, or simply iterated n=n+1 fashion), is equivalent to finding an algorithm for generating all prime numbers under n.

Hypothetically you could pick a number N out of a hat, thats a thousand digits long, and it happens to be the product of all primes underneath it.

You could then start generating primes by doing
i = 3
while i < N:
....if (N/k)%1 == 0:
........factors.append(N/k)
....i=i+1

The only caveat is that there should be more false solutions as real ones. In otherwords theres no telling if you found a solution N corresponding to some value of (prods[j+i]/prods[j-i]) without testing the primality of *all* values of k under N.

Waiting 15+ minutes while Windows "indexes" a folder just so I can see the contents reminds me of why I dislike Windows as an OS so much.

This is a senseless operation that is of no benefit to the user.

Newer Dev here. Just recently started in a position as a developer. I'm tasked with consolidating our monitoring systems into one cohesive display. After lumping together all the indexes and helping build a custom API I'm now working on front end. Front end is easy, I've done it before. Should be no problem. I was wrong. I spent a whole day fiddling with a React dynamic table and the CSS to format it. Today, I stumble upon the react-table component. Got the results I was looking for in less than 2 hours. I'm convinced that this was a lesson better learned early on.

The saddest and funniest side of our industry is (atleast in India): someone works hard and makes it to the best colleges, do great projects on AI, ML; get a good score on Leetcode, codechef; gets a job in FAANG-like companies...

Changes colors in CSS and texts in HTML.

And, why is there so much emphasis on Data Structures and Algorithms? I mean, a little bit is fine, but why get obsessed with it when you never write algorithms in production code?

Now, don't tell me that, we use libraries and we should know what we are doing, no, we don't use algorithms even in libraries.

Now, before you tell me that MySQL uses B-tree for maintaining indexes, you really don't need to solve tricky questions to be able to understand how a B-tree works.

It's just absurd.

I know how to little bit on how design scalable systems.
I know how to write good code that is both modular and extensible.
I know how to mentor interns and turn them into employees.
I know how to mentor junior engineers (freshers) and help them get started.
Heck I can even invert a binary tree.

But some FAANG company would reject me because I cannot solve a very tricky dynamic programming question.

my work drives me crazy sometimes, our production tables dont have primary keys or indexes. There are several tables who are basically the same, most scripts/reports are hacked together with no common agreement on dates/values and as a result, it is almost impossible to check whether values are correct or not.

So this web company i joined had a page load time in minutes. The free text search (inverted index search, based on elasticsearch) queries would return results in 10-45 seconds (should be milliseconds always). The indexes had no schema. And they would crawl data and feed into mssql db, which had a 2 gb/db limit on the free version. So everytime the db hit the limit, a new db was created and the name was incremented by one.

Had a very tough time cleaning up that mess. Plus the architect who had made this architecture was on his way out and unhelpful to the core.

What was worse was that most of the changes i did were very simple changes that should have been done long back. Basic sanity changes.

Working on an application - everywhere an enum should be is a database table instead.

Me: What happens when someone changes "rejected" to "approved" in your status table?

Me: What happens when you re-seed your database and the indexes for your types are different?

All problems, with no time or scope to fix!
USE A FUCKING ENUM

Tldr:
Can't fucking figure out why I'm the only one who can't solve a DP problem in code, when me and friends use the same idea and no one knows why only mine doesn't work...

We are given a task to solve a problem using DP. My friends write their code with the same idea as a solution. Copying the code is not allowed. I follow the same idea but my code won't work. Others look into it, in case they find errors. They can't find any.

The problem (for reference):
Given a fixed list of int's a = (a_1,a_2,...,a_n) and b = (b_1,b_2,...,b_n), a_i and b_i >= 0, a.length == b.length
We want to maximize the sum of a_i's chosen. Every a_i is connected with the b_i at the same index. b_i tells us how many indexes of a we have to skip if choosing the corresponding a_i, so list index of b_i + b_i's value + 1 would be the position of the next a_i available.

The idea:
Create a new list c with same length as a (or b).
Begin at the end of c and save a_n at the same position in c. Iterate backwards through c and at each position add the max value of all previously saved values of c (with regards to the b_i-restriction) with the current a_i, else a_i + 0 if the b_i-resctriction goes beyond the list.
Return the max value of c.

How does that not work for me but for the others?? Funny enough, a few given samples work with my code. I'm questioning my coding ability...

I hate the elasticsearch backup api.

From beginning to end it's an painful experience.

I try to explain it, but I don't think I will be able to cover it all.

The core concept is:
- repository (storage for snapshots)
- snapshots (actual backup)

The first design flaw is that every backup in an repository is incremental. ES creates an incremental filesystem tree.

Some reasons why this is a bad idea:
- deletion of (older) backups is slow, as newer backups need to be checked for integrity
- you simply have to trust ES that it does the right thing (given the bugs it has... It seems like a very bad idea TM)
- you have no possibility of verification of snapshots

Workaround... Create many repositories as each new repository forces an full backup.........

The second thing: ES scales. Many nodes / es instances form a cluster.

Usually backup APIs incorporate these in their design. ES does not.

If an index spans 12 nodes and u use an network storage, yes: a maximum of 12 nodes will open an eg NFS connection and start backuping.

It might sound not so bad with 12 nodes and one index...

But it get's pretty bad with 100s of indexes and several dozen nodes...

And there is no real limiting in ES. You can plug a few holes, but all in all, when you don't plan carefully your backups, you'll get a pretty f*cked up network congestion.

So traffic shaping must be manually added. Yay...

The last thing is the API itself.

It's a... very fragile thing.

Especially in older ES releases, the documentation is like handing you a flex instead of toilet paper for a wipe.

Documentation != API != Reality.

Especially the fault handling left me more than once speechless...

Eg:
/_snapshot/storage/backup
gives you a state PARTIAL
/_snapshot/storage/backup/_status
gives you a state SUCCESS

Why? The first one is blocking and refers to the backup status itself. The second one shouldn't be blocking and refers to the backup operation.

And yes. The backup operation state is SUCCESS, while the backup state might be PARTIAL (hence no full backup was made, there were errors).

So we have now an additional API that we query that then wraps the API of elasticsearch. With all these shiny scary workarounds like polling, since some APIs are blocking which might lead to a gateway timeout...

Gateway timeout? Yes. Since some operations can run a LONG (multiple hours) time and you don't want to have a ton of open connections hogging resources... You let the loadbalancer kill it. Most operations simply run in ES in the background, while the connection was killed.

So much joy and fun, isn't it?

Now add the latest SMR scandal and a few faulty (as in SMR instead of CMD) hdds in a hundred terabyte ZFS pool and you'll get my frustration level.

PS: The cluster has several dozen terabyte and a lot od nodes. If you have good advice, you're welcome - but please think carefully about this fact.

I might have accidentially vaporized people sending me links with solutions that don't work on large scale TM.

In the past, apps I've written have used a flat file backend. It's very fast, but obviously clunky to have a big structure of flat files for an app. It ran circles around framework-based RDBMS backends, as performance is concerned, but again, it was clunky. Managing backups and permissions on tens or hundreds of thousands of small files was no fun. Optimizing code for scaling was fun- generating indexes, making shortcuts -but something was still missing. Early in 2017 I discovered redis. A nosql backend that just stores variables and lives almost entirely in memory. Excellent modules and frameworks for every language. It was EXACTLY what I'd needed, even though I didn't know I did. I spent a good deal of time in 2017 converting apps from flat files to redis, and cackled with glee as they became the apps I wanted them to be. Earlier this week, I started building my first app that started with redis, instead of flat files, and I can't stop gushing to anyone who will listen. Redis for president!

Manjaro has some quirks that annoy me(no MST timezone, spotty support for my WD NVME), so I decided that since I'm not interested in any pre-configured graphical desktop of any kind, I should just dive into Arch, since it increasingly felt like that's what I was doing anyway but with Manjaro to dull the blow. So I did, and I am over the moon for doing so. Lots of gnashed teeth, but DDG indexes an answer to every question I've had, and it always makes sense when I find it. I've enjoyed having to dive into systemd in a much more low-level way than ever before-- to actually LEARN what it's doing, how, and why.

But one by one, I have been faced with some issue that I need to resolve, and one by one, I've knocked them off. The result now is the best work and gaming desktop I have ever used.

Arch is not for geniuses or wizards. Just patient people who are willing to read. The payoff is staggering, and many times over worth the effort.

How do you feel about not creating database tables for objects that only exist in relations?

For example, I have made a wiki engine. Because nothing on wiki pages can actually change, they aren't an entity. Revisions are an entity, and they refer to the title of the page which was changed. The same application also includes two non-version-controlled directed graphs between the pages (element of category and navigation log), which are represented by tables that link two titles. Of course the indexes are all set up so that it works like a foreign key would, but there is no Page or Article table.

I've been freelancing lately with an agency to develop an android app for their client and at the same time another person is developing the website .

The story begins when I first contacted the web dev to give me access to the database (because he started before me ).It turns out that this guy purchased an almost ready cms template with a shitty data structure that has no relations between object .This database has no primary keys , no foreign keys , no indexes ... no nothing . Adding to that the web dev refused that I rewrite a new data structure claiming that he has done a good progress on the website .
Forward couple of weeks , I managed to create the api and develop an alpha for the app and sent it to the agency manager .
This bastard told me that the website and design have changed and the app shouldn't be like that .He told me to contact the other bastard the web dev to seen what the changes are . I'm waiting for the response about the new updates and I'm praying that they'll be just minor colors updates or something not a whole concept update .

My problem here is I'm stuck with this fucking agency cuz they paid half of the payment when I started .

Damn I must learn to say no to people .