Hey, Root? How do you test your slow query ticket, again? I didn't bother reading the giant green "Testing notes:" box on the ticket. Yeah, could you explain it while I don't bother to listen and talk over you? Thanks.

And later:
Hey Root. I'm the DBA. Could you explain exactly what you're doing in this ticket, because i can't understand it. What are these new columns? Where is the new query? What are you doing? And why? Oh, the ticket? Yeah, I didn't bother to read it. There was too much text filled with things like implementation details, query optimization findings, overall benchmarking results, the purpose of the new columns, and i just couldn't care enough to read any of that. Yeah, I also don't know how to find the query it's running now. Yep, have complete access to the console and DB and query log. Still can't figure it out.

And later:

Hey Root. We pulled your urgent fix ticket from the release. You know, the one that SysOps and Data and even execs have been demanding? The one you finished three months ago? Yep, the problem is still taking down production every week or so, but we just can't verify that your fix is good enough. Even though the changes are pretty minimal, you've said it's 8x faster, and provided benchmark findings, we just ... don't know how to get the query it's running out of the code. or how check the query logs to find it. So. we just don't know if it's good enough.

Also, we goofed up when deploying and the testing database is gone, so now we can't test it since there are no records. Nevermind that you provided snippets to remedy exactly scenario in the ticket description you wrote three months ago.

And later:

Hey Root: Why did you take so long on this ticket? It has sat for so long now that someone else filed a ticket for it, with investigation findings. You know it's bringing down production, and it's kind of urgent. Maybe you should have prioritized it more, or written up better notes. You really need to communicate better. This is why we can't trust you to get things out.

*twitchy smile*

Me 5 years ago : "Guys we are gonna have a perf..."

CEO : "Not now, we need to deliver that functionality asap"

Me 3 years ago : "Guys, performances/scalibiluity will hit us like a trucK'

CEO : "Not nowem a new functionality needs to be done ASAP"

Me 1 year ago : "We are gonna be hit by a tank. We won't even understand what happens"

CEO : "I'm sure we can manage"

2 Days ago : Plateform quasiment down, response time in MIONUTES instead of milliseconds, database on fire.

CEO : "WHAT THE FUCK !!! GO FIXC I ASAP WE CANNOT HAVE THAT SHIT HAPPEN".

This is a brief summaru of working in a startup.

Fuck the memes.

Fuck the framework battles.

Fuck the language battles.

Fuck the titles.

Anybody who has been in this field long enough knows that it doesn't matter if your linus fucking torvalds, there is no human who has lived or ever will live that simultaneously understands, knows, and remembers how to implement, in multiple languages, the following:

- jest mocks for complex React components (partial mocks, full mocks, no mocks at all!)
- token cancellation for asynchronous Tasks in C#
- fullstack CRUD, REST, and websocket communication (throw in gRPC for bonus points)
- database query optimization, seeding, and design
- nginx routing, https redirection
- build automation with full test coverage and environment consideration
- docker container versioning, restoration, and cleanup
- internationalization on both the front AND backends
- secret storage, security audits
- package management, maintenence, and deprecation reviews
- integrating with dozens of APIs
- fucking how to center a div

and that's a _comically_ incomplete list; barely scratches the surface of the full range of what a dev can encounter in a given day of writing software

have many of us probably done one or even all of these at different times? surely.

but does that mean we are supposed to draw that up at a moment's notice some cookie-cutter solution like a fucking robot and spit out an answer on a fax sheet?

recruiters, if you read this site (perhaps only the good ones do anyway so its wasted oxygen), just know that whoever you hire its literally the luck of the draw of how well they perform during the interview. sure, perhaps some perform better, but you can never know how good someone is until they literally start working at your org, so... have fun with that.

Oh and I almost forgot, again for you recruiters, on top of that list which you probably won't ever understand for the entirety of your lives, you can also add writing documentation, backup scripts, and orchestrating / administrating fucking JIRA or actually any somewhat technical dashboard like a CMS or website, because once again, the devs are the only truly competent ones - and i don't even mean in a technical sense, i mean in a HUMAN sense of GETTING SHIT DONE IN GENERAL.

There's literally 2 types of people in the world: those who sit around drawing flow charts and talking on the phone all day, and those WHO LITERALLY FUCKING BUILD THE WORLD

why don't i just run the whole fucking company at this point? you guys are "celebrating" that you made literally $5 dollars from a single customer and i'm just sitting here coding 12 hours a day like all is fine and well

i'm so ANGRY its always the same no matter where i go, non-technical people have just no clue, even when you implore them how long things take, they just nod and smile and say "we'll do it the MVP way". sure, fine, you can do that like 2 or 3 times, but not for 6 fucking months until you have a stack of "MVPs" that come toppling down like the garbage they are.

How do expect to keep the "momentum" of your customers and sales (I hope you can hear the hatred of each of these market words as I type them) if the entire system is glued together with ducktape because YOU wanted to expedite the feature by doing it the EASY way instead of the RIGHT way. god, just forget it, nobody is going to listen anyway, its like the 5th time a row in my life

we NEED tests!
we NEED to know our code coverage!
we NEED to design our system to handle large amounts of traffic!
we NEED detailed logging!
we NEED to start building an exception database!

BILBO BAGGINS! I'm not trying to hurt you! I'm trying to help you!

Don't really know what this rant was, I'm just raging and all over the place at the universe. I'm going to bed.

ARGH. I wrote a long rant containing a bunch of gems from the codebase at @work, and lost it.

I'll summarize the few I remember.

First, the cliche:
if (x == true) { return true; } else { return false; };
Seriously written (more than once) by the "legendary" devs themselves.

Then, lots of typos in constants (and methods, and comments, and ...) like:
SMD_AGENT_SHCEDULE_XYZ = '5-year-old-typo'

and gems like:

def hot_garbage
magic = [nil, '']
magic = [0, nil] if something_something
success = other_method_that_returns_nothing(magic)
if success == true
return true # signal success
end
end

^ That one is from our glorious self-proclaimed leader / "engineering director" / the junior dev thundercunt on a power trip. Good stuff.

Next up are a few of my personal favorites:

Report.run_every 4.hours # Every 6 hours
Daemon.run_at_hour 6 # Daily at 8am

LANG_ENGLISH = :en
LANG_SPANISH = :sp # because fuck standards, right?

And for design decisions...

The code was supposed to support multiple currencies, but just disregards them and sets a hardcoded 'usd' instead -- and the system stores that string on literally hundreds of millions of records, often multiple times too (e.g. for payment, display fees, etc). and! AND! IT'S ALWAYS A FUCKING VARCHAR(255)! So a single payment record uses 768 bytes to store 'usd' 'usd' 'usd'

I'd mention the design decisions that led to the 35 second minimum pay API response time (often 55 sec), but i don't remember the details well enough.

Also:
The senior devs can get pretty much anything through code review. So can the dev accountants. and ... well, pretty much everyone else. Seriously, i have absolutely no idea how all of this shit managed to get published.

But speaking of code reviews: Some security holes are allowed through because (and i quote) "they already exist elsewhere in the codebase." You can't make this up.

Oh, and another!
In a feature that merges two user objects and all their data, there's a method to generate a unique ID. It concatenates 12 random numbers (one at a time, ofc) then checks the database to see if that id already exists. It tries this 20 times, and uses the first unique one... or falls through and uses its last attempt. This ofc leads to collisions, and those collisions are messy and require a db rollback to fix. gg. This was written by the "legendary" dev himself, replete with his signature single-letter variable names. I brought it up and he laughed it off, saying the collisions have been rare enough it doesn't really matter so he won't fix it.

Yep, it's garbage all the way down.

I was on vacation when my employer’s new fiscal year started. My manager let me take vacation because it’s not like anything critical was going to happen. Well, joke was on us because we didn’t foresee the stupidity of others…

I had to update a few product codes in the website’s web config and deploy those changes. I was only going to be logged in for 30 minutes to complete that.

I get messaged by one of our database admins. He was doing testing and was unable to complete a payment on the website. That was strange. There was a change pushed by our offsite dev agency, but that was all frontend changes (just updating text) and wouldn’t affect payments.

We don’t want to enlist the dev agency for debugging work, especially when it’s not likely that it’s a code issue. But I was on vacation and I couldn’t stay online past the time I had budgeted for. So my employer enlists the dev agency for help. It’s going to be costly because the agency is in Lithuania, it was past their business hours, and it was emergency support.

Dev agency looks at error logs. There are Apple Pay errors, but that doesn’t explain why non Apple Pay transactions aren’t going through. They roll back my deployment and theirs, but no change. They tell my employer to contact our payment processor.

My manager and the Product Manager contact Payroll, who is the stakeholder for our payment gateways. Payroll contacts our payment gateway and finds out a service called Decision Manager was recently configured for our account. Decision Manager was declining all payments. Payroll was not the person who had Decision Manager installed and our account using this service was news to her.

Payroll works with our payment processor to get payments working again. The damage is pretty severe. Online payments were down for at least 12 hours. Our call center had logged reports from customers the night before.

At our post mortem, we had to find out who ok’d Decision Manager without telling anyone. Luckily, it was quick work. The first stakeholder up was for the Fundraising Dept. She said it wasn’t her or anyone on her team. Our VP of Analytics broke it to her that our payment processor gave us the name of the person who ok’d Decision Manager and it was someone on the Fundraising team. Fundraising then starts backtracking and says that oh yes she knew about it but transactions were still working after the Decision Manager had been configured. WTAF.

Everyone is dumbfounded by this. How could you make a big change to our payment processor and not tell anyone? How did our payment processor allow you to make this change when you’re not the account admin (you’re just a user)?

Our company head had to give an awkward speech about communication and how it’s important. The web team can’t figure out issues if you don’t tell us what you did. The company head was pissed because it was a shitty way to start off the new fiscal year. Our bill for the dev agency must have been over $1000 for debugging work that wasn’t helpful.

Amazingly, no one was fired.

I was pressued to shift the blame.

We received an angry email from a customer that some of their data had disappeared. The boss assigns me to this task. This feature is relatively new and we've found some bugs in the past in here. I go through request logs, search the database, run some diagnostics, etc. for about 5 hours and I cannot find the problem. I focus on the bugs that we've had before but they don't seem to be the problem.

I tell the boss "sorry but I checked XYZ and I can't find the problem. I'm out of ideas." But the boss wanted answers by the end of the day. They did not want to admit to the client that we couldn't figure out what's wrong.

By now I was more pressured to find an answer, find something or someone to blame it on, not exactly to find the real solution. So I made up some BS:

"Sometimes, in HTML forms, the number inputs allow you to change the number by scrolling. We have some long forms where the user has to scroll. Perhaps the focus remained on the number input, so when they scrolled down they accidentally changed the number they meant to input."

The boss was happy with that. We explained this to the customer, and there's now a ticket to change type="number" to type="text" in our HTML forms and to validate it in th backend.

A week later another customer shows us a different error. This one is more clear because it had a stack trace, but I realise that this error is what caused our last error. It was pretty obscure, mind you, the unit tests didn't detect it.

I didn't tell the boss that they were connected tho.

With two angry clients in two weeks, I finally convinced the boss to give us more time to write more unit tests with full coverage.

A personal memo to all developers on devRant:

* Assume every external line of code, (including every service you consume) is an unreliable crock of flaming shit. These services can and will fail in the most glorious ways. Write your code to be resilient, and ASSUME FAILURE of dependencies. Even if it's your own team writing the other service.

Heard in a meeting today: "Your team's service outage is going to cause my service to corrupt the database!"

Response I wanted to give: "No, you asshat, my service outage is a normal part of living with microservices. Your app should have been smart enough to recognize the failure."

I can't figure out shit..
To be honest I created this profile just so I can write down somewhere what I am going through.
So, once upon a time I had graduated from college and went right into a corporate (has only been 2 years since). I was fortunate enough that I got assigned a project that was just starting, and even though I had no clue what was going on, I started doing whatever was assigned.
I initially worked in java and then finished all my tasks earlier than expected, so they switched me to another C++ project that builds on top of it.
Fast forward 2.5 years, I'm now the team lead of the CPP project and all my friends who were in the core team have left the company.
As usual, the reason behind it is shitty management. These mfs won't hire competent people and WILL ABSOLUTELY NOT retain the ones that are. I can feel it in my bones that it is time for me to leave, but fuck me if I understand what I am good at.
I have been able to handle all the tasks that they threw at me, be it java or c++ - just because I love logic and algorithms. I have been dabbling in ML and AI since 4-5 years now, but could never go into it full time.
Now I'm looking at the job postings and Jesus Christ these bitches do not understand what they want. I have to be expert in 34567389 technologies, mastering each of whom (by mastering I mean become proficient in) would need at least 6-8 months if not more, all with 82146867+ years of experience in them.
I don't know if I am supposed to learn on Java (so spring boot and stuff) or I'm supposed to do c++ or I'm gonna go with Python or should I learn web dev or database management or what.
I like all of these things, and would likely enjoy working in each of these, but for fucks sake my cv doesn't show this and most of the bitch ass recruiter portals keep putting my cv in the bin.
Yeah...
If you have read so far, here's a picture of a cat and a dog.

Following on from yesterday's catastrophe...
It happened again overnight!
And THEN another server failed when backing up the DBs...turns out 2 websites have activated logging .. each of around 20GB in the database!

So this morning both of our servers are down for different but similar reasons.

It's absolutely fine everybody.
I'm fine,
we're fine,
it's FINE!

"And in a stunning turn of events, he got it to work!"

But seriously... I've literally been throwing shit at a wall and seeing what would stick.
Fucking DTOs and getting shit out of a database. I need better resources on how to do this properly!

Anyways, I found that just using 'object' and letting the compiler deal with the rest of the bullshit actually allowed my code to work and run. I'm still a little in shock.

I'm over here trying to keep things in a nice one-to-one because that's what my PM recommended... and instead I just get slammed by Type casting nonsense and more errors than I can begin to understand. And unfortunately, Stackoverflow is of no help because everyone's issues are very nuanced and unrelated to my problem... Maybe I'm the problem? 🤷

But here it is working without all that bullshit. I don't know man... This code base is not the rager I was expecting. I'm getting my ass kicked with code that doesn't fall in line with the book I'm learning from.

You know how they say, "forget everything you've read and learned"? I'm feeling that really hard right now.

Constantly fighting the urge to rip everything down and do it based on what my book is recommending, but then the logical natured side of me is like "you ain't got that kind of time to be unfucking someone's work, only to get caught in more trouble. Your ego is not worth it"

Anyways, it's fucking late here and I'm glad enough to not have to think about this issue anymore. Bye.

Boss needs certain stats pulled from database once a year for board meeting. This time I delegate it to a junior dba/sysadmin. He looks at my 3-year-old docs that I hastily jotted down and pasted and included my rambling notes with results from way back then. Mostly they were just to jog my own memory, not to be a really neat, clean instruction guide. He does the queries correctly, but in ticket for boss he pastes also all my notes from the docs. boss gets confused, "what is this other number, I don't get it?!" We have to have a meeting of the 3 of us and waste an hour or so just to figure out what went wrong, finally I realize what junior guy accidentally did. Moral of story: to avoid baffling the nontechs, always simplify, simplify, simplify. Alternate moral of story: before delegating a task that seems old hat to you, always review your notes/docs and make sure they're ready for someone else to use them.

Hello all,

I am an apprentice, 19. I joined this software developer apprenticeship to leave college as it was not particularly great for my mental health, and programming is the only thing I can do reasonably well.

The company that I find myself in is a strange one. It has about twenty or so employees, but we all instructed to operate as if we are a giant company—our sales person, for example, will tell our clients that we have hundreds.

The development team is a collection of software developers. There is no database administrator, network administrator, software engineer (not in name only), test engineer, requirements engineer, etc. There are just several software developers. Of these developers, one has left by now. When he joined, he was promised to be working on a new system: he left after spending seven years on an old system. A new developer has just arrived to replace him: he was told he would be working with Raspberry Pis; it was interesting to see his face after we informed him that we do not use Raspberry Pis.

The codebase is fourty-years-old and written in Delphi, which is some kind of cousin of pascal, from what I understand. Code is not peer-reviewed. Instead, it is self-reviewed, and you just push whatever changes you make. The code is very much spaghetti, and there is a whole array of bugs that, at least to me, look impossible to track down and fix. I have a bug assigned to me at the moment were someone appears somewhere when they are not supposed to. After asking seniors about this, I learn of this huge checking mechanism and all of its flaws: a huge, flawed checking mechanism... for toggling a single boolean value. This isn't a complicated boolean value, by the way, this is just a value to say whether someone has clocked in or clocked out of a building, via a button.

In terms of versioning, we have several releases, and we often do development work in older releases (or new releases and then write them into older releases) because our clients are larger than us and often refuse to upgrade, and the boss does not want to lose any contracts. We also essentially have multiple master branches.

With the lack of testers, bizarre version control, what appears to be unfiffled promises to staff, etc. I must ask that, since this is my first gig as a software developer, is any of this normal?

Let's say you're working on a web application, and you notice that one of the pages is not displaying the correct data. You investigate further and realize that the data is being retrieved from an API endpoint, but for some reason, the API is returning the wrong data.

You start looking into the code that calls the API and notice that it's passing in the correct parameters, so you dig deeper into the API code itself. After hours of poring over the code, you finally discover that the bug is caused by a typo in the database query that the API is using to retrieve the data.

You fix the typo and think the problem is solved, but then you realize that the data is still not displaying correctly on the page. After even more investigation, you discover that the bug is actually being caused by a caching issue on the client side.

At this point, you're feeling incredibly frustrated and overwhelmed. You've spent hours trying to track down this bug, and it feels like every time you think you've found the root cause, another issue pops up. This is just one example of the many challenges that developers face on a daily basis.

I went down a rabbit hole of code changes to try and delete a stupid for loop with a break in it.

It was super stupid and I gave up and submitted to the fact that some battles are not worth the time and stress.

OK... But seriously, It was returning multiple entities from the database, but we only always want the first one. My logic is that we should just go in there and fix the LINQ so we are explicitly getting one entity out.

But fuck that logic. No I'll have to change fucking everything that's tied to that method and expects a list from it. Every fucking thing. That includes error handling, parsing, for loops..... Nevermind...

You can have your foreach and your break. I'm taking mine, now.

Lessions I learned so far from my first big node/npm project with tons of users:
1) If you didn't build something for a while, expect 3 hours of resolving version conflicts for every two weeks since the last build.
2) Even if the tests pass, run the containers on your own machine and make sure that the app doesn't randomly crash before deploying
3) Even if the app seemed to work on your own machine, run the tests again in an environment mimicking prod at most 15 minutes before replacing the running containers.
4) Even if all else indicates that the app will work, only ever deploy if you expect to be available within the 4 hours following a deployment.
5) Don't use shrinkwrap for anything other than locking every version down completely. A partial shrinkwrap will produce bugs that are dependent on the exact hour you built the app _and_ the shrinkwrap file, and therefore no one will ever have seen them other than you.
6) Avoid gyp, and generally try not to interface too much with anything that doesn't run on node. If parts of your solution use very different toolchains, your problems will be approximately proportional to the amount of code. And you'd be surprised just how much code you're running. (otherwise it's more logarithmic because the more code the less likely a new assumption is unique)
7) Do not update webpack or its plugins or anything they might call unless you absolutely need to
8) Containers are cool but the alpine ones are pretty much useless if you have even just one gyp module.
9) There's always another cache. To save yourself a lot of pain, include the build time in every file or its name that the browser can download, and compare these to a fresh build while debugging to assert that the bug is still present in the code you're reading
+1) Although it may look like it, SQLite is far from a simple solution because the code and the bindings aren't maintained. In fact, it'll probably be more time consuming than using a proper database.

When I first started down the path to becoming a developer, I was a "business analyst" where I managed our departments reports and ended up migrating all the reports from daily query run in MS Access with Task manager and emailed out to all the managers including the VP of the entire business unit, I created
Views in the database and sent out the same spreadsheet with the view in excel daily since management didn't want "change". Granted this was at a large health care company in the US and didn't want to invest in a real dashboard for their reports. The only thing that was changed in the email and file was the file name with the current date. I left the company a while ago and recently applied for a similar position for the shits and gigs. Interviewed with the It manager and they're still using the same excel macro I wrote 3 years later.

Not sure if it should be a joke or a rant, but something rather funny (at least to me) happened today.

TL;DR; Someone's outlook was crippled by 100k+ of warning notifications

So we have developed a server that has an internal database that wraps around an elastic search instance, that is managed by a POS vault/storage solution, that we have to use for legal reasons. The elastic search is "provided" by the software, but we keep this internal database just to be sure and totally not because it's unreliable POS.

Anyways, they take data integrity very seriously, so every warning our server produces is emailed to someone in charge to review it and if necessary forward it to us. This will be important later on.

Couple of days ago we got error forward when trying to write an entry into the POS software we get an error, because an object we tried to write already existed. After some investigation we concluded an entry was missing when the internal database was created, so we asked them to repopulate elastic search to solve this problem.

When start the server we always sync the internal database to the elastic search and emit a warning when an entry is missing in internal database or vice versa. And well... almost all of them were missing, which caused our server to emit ~40 warnings/ms. Poor outlook. Still investigating for the cause, but damn, I never expected I'd take down someone email account by accident