When you watch hunger games and wonder how the database works

If all you have is a hammer, everything looks like a nail!

This was something which my tech lead used to tell me when I was so obsessed with nosql databases a few years back. I would try to find problems to solve that has a use case for nosql databases or even try to convince me(I didn’t realise it back then) that I need to use nosql db for this new idea that I have, without really thinking deep enough whether the data in question is better represented using an sql schema or not.

Now, leading a team of young developers, I come across similar suggestions from few of my team members who just discovered this new and shiny tech and want to use it in production projects.

While I am not against new and shiny, it’s not a good practice to jump right in to it without exploring it deep enough or considering all the shortcomings. The most important question to ask is, whether some of the problems you are trying to solve can be solved with the current stack.

Modifying your stack requires more than just a week’s experience of playing around with the getting started guide and stack overflow replies. This is something which need to be carefully considered after taking inputs from the people who would be supporting it, that include operations, sysadmins and teams that are gonna interface with your stack indirectly.

I am not talking about delaying adoption by waiting for long list of approvals to get some thing that would bring immediate value, but a carefully orchestrated plan for why and how to migrate to a new stack.

Just because one of the tech giants made a move to a new stack and wrote about it in their engineering blog doesn’t mean that you need to make a switch in the same direction. Take a moment to analyse the possible reasons that motivated them to do it, ask yourself if your organisation is struggling with the exact same problems, observe how others facing the same issue are addressing it, and then make an informed decision.
Collect enough data to support your proposal.
Ask yourself again if you are the one holding the hammer.
If the answer is no, forge ahead!

Google, you can't do that to me

YOU CANNOT DO THAT TO ME

I AM NOT A THING

Quick recap of my last two weeks: 15 year old production server is basically dead, boss has taken over calls and claims credit for "resolving" outages (even though my coworker and I did the work, but ultimately the traffic died down enough to where it wasn't an issue anymore).

I go to a meeting to plan migration to a better server, boss bitches about not getting invited, I tell him I invited myself, and then he lectures about how that's not our job.

Different boss says we're migrating a schema for an application that should have been decommissioned 5+ years ago to use as a baseline. I explain what's going on, he says he understands, and proceeds to tell higher bosses it's perfect because there will be no user impact. OF COURSE THERE'S NO FRICKING IMPACT, YA DUNCE! there are no users!!!!

I merge two email threads together, since they discuss the same thing, but with different insight, and get yelled at, even though they requested it.

The two bosses I like are OOO for the next week, too, so I'm just sitting here hoping I don't say something that'll get me fired or sent to sensitivity training.

I'm just starting my on call rotation and don't know that I can do this. I cry when my phone rings, now, because I experience physical pain with how hard I cringe.

I got yelled at today by a guy because SOMEONE I DON'T KNOW assigned a ticket to him directly, rather than to the proper team (not his team). So I had to look into that, which at least had the benefit of preventing a catastrophic outage to our customers world wide, but no one will know because I don't brag at work; I'm too busy doing my job as well as most of my division/section/larger team, whatever the hell it's called. I saved us probably 25+ hours of continuous troubleshooting call from noticing something tiny that the people "smarter" than me missed.

**edit: sorry for typos; got my nails done yesterday but they feel like they're a mile long and I have to relearn how to type**

🍿🍿 pull up a chair and get comfy. This was a few years ago and anger has filled some details, so bear with me...

One day, during one of rare afternoons off of work, I was in the library to work on a group project for school. This was maybe a month before it was due, so we were tracking for decent progress and one less stressor over finals. It was about 80° F out, with the perfect breeze for the beach, but school comes first.

I'm team lead (which is terrifying, but less important) and my bro C shows up early to be ready to go on time because he's professional. I'M SO BAD I FORGOT DOUCHEBAGS NAME, so he's A (for asshole), shows up AN HOUR AND 15 MINUTES LATE. But it's not the end of the world, C and I worked around our database schema (which A sent us and we approved), so we could iron out kinks as we went.

A gets there... Fucking finally.

Fucker didn't have the database built (had 2 months to do it, we all agreed on schema a month prior. We're trying to be the adults our ages claim is to be).

*breathe in, count to 10* not a problem, A, just go ahead and start it now so we can at least check what we have.

Ok, my queen, I'll have it done in 10 minutes...

🤔🤔

We needed an id (sku... Which, in 99.9999% of companies is numeric), a short name (xBox one, Macbook, don't smart tv), a description and a price (with 2 decimals). All approved by all 3 of us.

His sku ranges from 3 to 9 ALPHA NUMERIC CHARACTERS, the names were even more generic than expected (item1, item 2, Item_3), no description, and he somehow thought US currency had 5 decimal places!!! (it's more accurate...)

There was an epic, royal, and expensive fight scene in the library (may have been during the Lenten season I decided to give up caffeine AND fast for 40 days to prove a point to an ass wipe of a history teacher, don't recall). I made him cry, he failed the class because C and I wound up fixing everything he touched (graded by commits, because it was also an intro to git, but also, a classmate saw it all), and I had to buy multiple people coffee for yelling in the library.

A tried making out buttons work (I was fed up and done thinking for the day, so moved to documentation), but he fucked those up. I then made those worse by having nested buttons, but I deleted all his shit and started over and fixed it.

I then cried, but C and I survived and have each others backs still.

What is it with clients who have never even seen a single line of code in their entire lives telling me how long they believe a certain change request should take to code.

I mean, what if I told you that your "seemingly simple enough" change needed an update to 36 class files, 9 oracle stored procedures AND several database schema changes. Also, who the fuck is going to perform a regression test to make sure I didn't break anything? Your Uncle Bob??

So I took on a fairly big project and poured my heart and soul into it, was the biggest thing I did yet. I kept on sending beta's to the customer after each change for review! Kept on insisting that they review it, the answer was always "this looks amazing keep doing what you're doing"! After I finished and pushed everything to production.

They didn't use it for nearly 6 months! And then out of the blue they call me saying that half of the app is wrong.. WTF? Where was this information during testing! I informed them that the changes would take some time since I need to do migrations and change the whole database schema.

In which they replied "but you already finished it once won't changing things make it easier? We shouldn't pay for your mistakes"
I don't know how I handled that but they should be thankful they were half way across the country 😠😠😠😠

I've seen several rants about dumb/useless teachers, college and the CS degree studies; today is a good day to vent out some "old" memories.

Around two semesters ago I enrolled in a Database seminar with this guy, a tall geek from the 80's with a squeaky voice, so squeaky mice could had an aneurysm if they listened to him.

Either way this guy was a mess, he said he was an awesome coder, that we were still "peasants" when it came to coding, that relational databases had nothing on him since he was an awesome freelancer and did databases every day, that we had to redo the programming course with him and with his shitty, pulled out of the ass own C++ style guide with over 64 different redacted rules.

He gave us sample code of "how it should be done" in Java...it ain't my favorite language but fuck me a fucking donkey could have written better code with his ass!! He even rewrote Java's standard input function and made it highly inefficient. He still wrote in a structural paradigm in OOP languages! And he dared to make this code reviews were he would proyect someone's code and mock it in front of the class as he took off points, sometimes going to the negative realm (3,2,1,0,-1...)

But you know what's shittier? That he actually didn't even attend, 90% of the time, it was literally this:
> Good morning class
> Checks attendance. . .
> I'll be back, I'm going to check in...
> 1 hour 45 minutes later (class was 2 hrs long) - comes back
> do you have any doubts?
> O.o no...? I'm ok.
> We're done

Not only that, he scheduled from 4 to 17 homeworks throughout the week, I did the math, that was around 354 files from everyone; of course he didn't check them, other students from higher semesters did and they gained each point taken from students making students from lower semesters get the short end of the stick.

How did I pass? He didn't understood my code or database schema and he knew he couldn't fail me as he had no ground to stand on.

Thanks for listening, if you got to the end of this long ass post and had a similar experience I'd love to read it.

Well, it wasn't fun, but I switched jobs this month. And sadly, it was mostly because my old company started building custom applications for our larger customers. Now, normally that wouldn't be too bad (other than the fact that it distracts us form working on our main product...) but... it was decided that we would use the back end of our user-generated forms module as the data storage layer. Someone outside of my department thought it would be a great idea, and my boss kinda just rolled over without a fight because he always just figures he can "make it work" if he works hard enough...

You shoulda seen the database and SQL code...

Because of that decision, everything took at least 3x as long to write and there was always the looming possibility that the user could change the schema on a whim and break the app.

I think the reasoning behind it was to try and keep the customers tied to the aging flagship product (with a pricy subscription model), but IMO, it was not with it. Our efforts could've had much greater impact somewhere else. Nobody seemed to care what I thought about it though...

I had to start over as a front-end dev, but I'm trying to look on the bright side and seeing it as an opportunity to sharpen my skills in that area. I'm already learning a lot. And although it's a little scary at times, it's also so refreshing to work at a place where I know I'm not the smartest guy in the room.

To the future!

The proprietary API I have to use which uses XML but is totally dynamic and without a schema. They basically managed to successfully merge JSON and XML to get the disadvantages of both JSON and XML.

निम्न (Nimn) schema less data. 80% smaller than XML, 60% smaller than JSON.

Imagine being so full of yourself that you use your own URL schema.

Coworker: so once the algorithm is done I will append new columns in the sql database and insert the output there

Me: I don't like that, can we put the output in a separate table and link it using a foreign key. Just to avoid touching the original data, you know, to avoid potential corruption.

C: Yes sure.

< Two days later - over text >

C: I finished the algo, i decided to append it to the original data in order to avoid redundancy and save on space. I think this makes more sense.

Me: ahdhxjdjsisudhdhdbdbkekdh

No. Learn this principal:
" The original data generated by the client, should be treated like the god damn Bible! DO NOT EVER CHANGE ITS SCHEMA FOR A 3RD PARTY CALCULATION! "

Put simply: D.F.T.T.O
Don't. Fucking. Touch. The. Origin!

The cleaning lady

Yeah, I should not be at work this late, but it happenes from time to time..
Anyhow, she still doesn't know where the switches for each light are, so whenever she comes, lights start to flicker on off on off... Yeah, lightshow baby!! //read this in austin powers voice
I fear for people with epilepsy.. Yeah, I get it, push and toggle switches are mixed in pannel, but we do have a schema for each of them..ofc she doesn't use it..

And every goddamn time if I work later than when she leaves, she disturbes me to ask if I will turn off the lights when I leave.. NOoOooo, I will leave them on like an asshole.. :/ I once even wrote on the whiteboard in bigg letters 'yes, I will turn of the lights when I leave' and left it there for some days... coworkers were all puzzled/amused when they saw it.. //I know, shitty attitude on my part too, but I've repeated myself a dozen times already.. + I always turn off lights she actually manages to switch on and leave them like that, even the ones not in my part of office

And when she vacuums..well.. I usually get up and go on a cigarette break to give her space so she can do her job around my desk.. but she insists that I am not in her way and I don't need to leave... Well I prefer not to be proded and poked with the hose, thank you..

So yeah, fun times... O.o especially if working late due to some production fuckups I need to fix ASAP.. :/

So, there was some project going around universities for online lab and connecting students to learn together. My school saw it as opportunity and I was appointed to make a simple web app doing just that.

T - teacher

First there was the database.
T1: "I think that you can design the tables better"
Me: "This is the most logical way, but sure, I can modify it. Although, T2 told me to do it this way."
T1: "Just do what I said"

And I did.

T2: "Why did you change the design, I gave you the schema, didn't I?"

Oh I was so fucking pissed. I'm skipping classes so I can work on this fucking piece of shit and this is what I get. Two self fucking centered idiots.

I resolved situation somehow, then came the frontend. I was told that the T3 will do it I just have to connect it to my backend, great.

So this so called "Teacher of Web design and development" downloaded fucking bloated bootstrap template with scripts longer than my future. It took 2 minutes to just load the html page from local drive.

I told T2 that I will make design it from scratch and I will try to make it look the same. I finished it somehow. Loaded in seconds, worked superb. First thing T1 told me: "Just roll back the design"
He didn't even listen to what I had to say.

Ohh boii, I fucking lost two months on your piece of shit garbage app, and what I have from it? Fucking bag of dicks. The thing is that I have another year with those pricks.
If this is what the industry is like, you guys deserve a medal.

Who ever prefixes their table names like this should be sentenced to death.

tbltable_name or tbltablename

Is that some kind of sadistic fetish or something.

My eye can’t fucking pinpoint the first letter, thus I can’t easily find the table. I have seen it so many times!!! This trend has to STOOOP.

Prefix it tbl_table_name for all i care ffs.

OMFG I don't even know where to start..
Probably should start with last week (as this is the first time I had to deal with this problem directly)..

Also please note that all packages, procedure/function names, tables etc have fictional names, so every similarity between this story and reality is just a coincidence!!

Here it goes..

Lat week we implemented a new feature for the customer on production, everything was working fine.. After a day or two, the customer notices the audit logs are not complete aka missing user_id or have the wrong user_id inserted.
Hm.. ok.. I check logs (disk + database).. WTF, parameters are being sent in as they should, meaning they are there, so no idea what is with the missing ids.

OK, logs look fine, but I notice user_id have some weird values (I already memorized most frequent users and their ids). So I go check what is happening in the code, as the procedures/functions are called ok.

Wow, boy was I surprised.. many many times..
In the code, we actually check for user in this apps db or in case of using SSO (which we were) in the main db schema..
The user gets returned & logged ok, but that is it. Used only for authentication. When sending stuff to the db to log, old user Id is used, meaning that ofc userid was missing or wrong.

Anyhow, I fix that crap, take care of some other audit logs, so that proper user id was sent in. Test locally, cool. Works. Update customer's test servers. Works. Cool..

I still notice something off.. even though I fixed the audit_dbtable_2, audit_dbtable_1 still doesn't show proper user ids.. This was last week. I left it as is, as I had more urgent tasks waiting for me..

Anyhow, now it came the time for this fuckup to be fixed. Ok, I think to myself I can do this with a bit more hacking, but it leaves the original database and all other apps as is, so they won't break.
I crate another pck for api alone copy the calls, add user_id as param and from that on, I call other standard functions like usual, just leave out the user_id I am now explicitly sending with every call.
Ok this might work.

I prepare package, add user_id param to the calls.. great, time to test this code and my knowledge..

I made changes for api to incude the current user id (+ log it in the disk logs + audit_dbtable_1), test it, and check db..
Disk logs fine, debugging fine (user_id has proper value) but audit_dbtable_1 still userid = 0.

WTF?! I go check the code, where I forgot to include user id.. noup, it's all there. OK, I go check the logging, maybe I fucked up some parameters on db level. Nope, user is there in the friggin description ON THE SAME FUCKING TABLE!!
Just not in the column user_id...

WTF..Ok, cig break to let me think..

I come back and check the original auditing procedure on the db.. It is usually used/called with null as the user id. OK, I have replaced those with actual user ids I sent in the procedures/functions. Recheck every call!! TWICE!! Great.. no fuckups. Let's test it again!
OFC nothing changes, value in the db is still 0. WTF?! HOW!?

So I open the auditing pck, to look the insides of that bloody procedure.. WHAT THE ACTUAL FUCK?!
Instead of logging the p_user_sth_sth that is sent to that procedure, it just inserts the variable declared in the main package..
WHAT THE ACTUAL FUCK?! Did the 'new guy' made changes to this because he couldn't figure out what is wrong?! Nope, not him. I asked the CEO if he knows anything.. Noup.. I checked all customers dbs (different customers).. ALL HAD THIS HARDOCED IN!!! FORM THE FREAKING YEAR 2016!!! O.o

Unfuckin believable.. How did this ever work?!

Looks like at the begining, someone tried to implement this, but gave up mid implementation.. Decided it is enough to log current user id into BLABLA variable on some pck..
Which might have been ok 10+ years ago, but not today, not when you use connection pooling.. FFS!!

So yeah, I found easter eggs from years ago.. Almost went crazy when trying to figure out where I fucked this up. It was such a plan, simple, straight-forward solution to auditing..

If only the original procedure was working as it should.. bloddy hell!!

Confession: sometimes when im too lazy to look at the db schema, i just select *, and filter the stuff i need from the ResultSet.

I was asked to look into a site I haven't actively developed since about 3-4 years. It should be a simple side-gig.
I was told this site has been actively developed by the person who came after me, and this person had a few other people help out as well.
The most daunting task in my head was to go through their changes and see why stuff is broken (I was told functionality had been removed, things were changed for the worse, etc etc).
I ssh into the machine and it works. For SOME reason I still have access, which is a good thing since there's literally nobody to ask for access at the moment.
I cd into the project, do a git remote get-url origin to see if they've changed the repo location. Doesn't work. There is no origin. It's "upstream" now. Ok, no biggie. git remote get-url upstream. Repo is still there. Good.
Just to check, see if there's anything untracked with git status. Nothing. Good.
What was the last thing that was worked on? git log --all --decorate --oneline --graph. Wait... Something about the commit message seems familiar. git log. .... This is *my* last commit message. The hell?
I open the repo in the browser, login with some credentials my browser had saved (again, good because I have no clue about the password). Repo hasn't gotten a commit since mine. That can't be right.
Check branches. Oh....Like a dozen new branches. Lots of commits with text that is really not helpful at all. Looks like they were trying to set up a pipeline and testing it out over and over again.
A lot of other changes including the deletion of a database config and schema changes. 0 tests. Doesn't seem like these changes were ever in production.
...
At least I don't have to rack my head trying to understand someone else's code but.... I might just have to throw everything that was done into the garbage. I'm not gonna be the one to push all these changes I don't know about to prod and see what breaks and what doesn't break
.
I feel bad for whoever worked on the codebase after me, because all their changes are now just a waste of time and space that will never be used.

TFW you realize you're the first dev at your company to ever run a database schema comparison between dev and live.

We just got into a malicious bots database with root access.

So guard duty gave us some warnings for our tableau server, after investigating we found an ip that was spamming us trying all sorts. After trying some stuff we managed to access their MySQL database, root root logged us in. Anyway the database we just broke into seems to have schemas for not only the bot but also a few Chinese gambling websites. There are lots of payment details on here.

Big question, who do we report this to, and what's the best way to do so anonymously? I'm assuming the malicious bot has just hyjacked the server for these gambling sites so we won't touch those but dropping the schema the bot is using is also viable. However it has a list of other ips, trying those we found more compromised servers which we could also log in to with root root.

This is kinda ongoing, writing this as my coworker is digging through this more.

Today's project was answering the question: "Can I update tables in a Microsoft Word document programmatically?"

(spoiler: YES)

My coworker got the ball rolling by showing that the docx file is just a zip archive with a bunch of XML in it.

The thing I needed to update were a pair of tables. Not knowing anything about Word's XML schema, I investigated things like:
- what tag is the table declared with?
- is the table paginated within the table?
- where is the cell background color specified?

Fortunately this wasn't too cumbersome.

For the data, CSV was the obvious choice. And I quickly confirmed that I could use OpenCSV easily within gradle.

The Word XML segments were far too verbose to put into constants, so I made a series of templates with tokens to use for replacement.

In creating the templates, I had to analyze the word xml to see what changed between cells (thankfully, very little). This then informed the design of the CSV parsing loops to make sure the dynamic stuff got injected properly.

I got my proof of concept working in less than a day. Have some more polishing to do, but I'm pretty happy with the initial results!

Just got an email with a new really nice douche-bag move from Postman to raise their prices again (this time for almost double) on their paid plans with excuse "it will help us deliver more of what our customers need from us."

Even though I've decided to look other way around for years on their electron-based garbage of bloated app, have not been a fan of their pricing 8$/month/user just for a simple feature such as sharing request schema and environment data.

This simply needs to stop and I'm seriously thinking about doing something about it. 🙄

Fuck uninspired jr devs that are simply collecting a pay check.

I have been handed a project that a jr dev was allowed to wallow on for over two fucking years. This lazy mother fucker managed to create 5 functions, a whole fucking mess of bullshit that I now have to straighten out on top of the 8 other things that I have to deliver on in the next month.

They never followed requirements. Not-a-one. The API is fully broken. The DB schema is BEYOND fucked. There's ZERO validation/sanitation on I/O. The deployments only work half the fucking time. Their code is so spaghetti I'm getting triggered from when I worked at Olive Garden with Eminem. But hey, at least they were able to demo it to the client to say "it works".

I don't condone violence, but every time I find malformed if statements, linter exceptions, broken deploy configurations in this project -- I just want to kick them in their stupid fucking face.

Wherever you ended up you piece of shit, I hope your dreams of becoming a rich asshole only bring you unending despair. I believe you can make it though, because you're already halfway there.

10 year anniversary 'celebration' for a couple of employees (one dev, one a DBA) and the VP of the department was saying kind words about them, talking about the 'good old days'.

VP to the DBA: "I apologize, when you started, you walked into my database architecture. I didn't know that much back then and never thought about the architecture much beyond a few years. Its amazing my design has lasted over 20 years and triple digit business growth..blah blah blah"

Inner voice: "Mother F-er!...My database was designed IN SPITE of your meddling and demanding to create 1,500 field tables. Shut the F up you egotistical bastard!"

I can't even count how many times I had to stop him from, for example, adding a 'ProductID' field to a Customer table.

Me: "Why did you add a product id field?"
VP: "How else will we know what product the customer wants to buy?"
Me: "You mean like a wish list? What if the customer wants more than one product?"
VP: "Oh, that’s easy, we'll create more fields when that happens. ProductID2. Microsoft made it really easy to add fields."
Me: "We already have a wish list table schema. Customer can have as many wish lists and as many products as they want."
VP: "I don't understand. All I want is a field for me to store the product I'm buying. I don't know why you make this so hard, its just one more field."

Now the VP is bragging all the success was due to his expertise?! Gaaaaahhhh!

I quelled my rage with ample quantities of donuts, juice, and chocolate milk.

3rd time the DB schema has been completely redone since starting this project.

3rd time I'm rewriting the back-end of this login.

But... 3rd time I'm getting paid for the same thing so ¯\_(ツ)_/¯

Having to do database schema changes manually is HELL. Management seems to not be taking seriously enough the need to stop and implement migrations, my boss/lead dev suggested me to look up how to adopt/implement them in my own free time, instead of stopping all the feature implementing to do this VERY IMPORTANT, CRUCIAL CHANGE. Now I am the retard who takes too much time to do simple changes in the database.

Maybe I am retarded after all.

"So Alecx, how did you solve the issues with the data provided to you by hr for <X> application?"

Said the VP of my institution in charge of my department.

"It was complex sir, I could not figure out much of the general ideas of the data schema since it came from a bunch of people not trained in I.T (HR) and as such I had to do some experiments in the data to find the relationships with the data, this brought about 4 different relations in the data, the program determined them for me based on the most common type of data, the model deemed it a "user", from that I just extracted the information that I needed, and generated the tables through Golang's gorm"

VP nodding and listening intently...."how did you make those relationships?" me "I started a simple pattern recognition module through supervised mach..." VP: Machine learning, that sounds like A.I

Me: "Yes sir, it was, but the problem was fairly easy for the schema to determ.." VP: A.I, at our institution, back in my day it was a dream to have such technology, you are the director of web tech, what is it to you to know of this?"

Me: "I just like to experiment with new stuff, it was the easiest rout to determine these things, I just felt that i should use it if I can"

VP: "This is amazing, I'll go by your office later"

Dude speaks wonders of me. The idea was simple, read through the CSV that was provided to me, have the parsing done in a notebook, make it determine the relationships in the data and spout out a bunch of JSON that I could use. Hook it up to a simple gorm golang script and generate the tables for that. Much simpler than the bullshit that we have in php. I used this to create a new database since the previous application had issues. The app will still have a php frontend and backend, but now I don't leave the parsing of the data to php, which quite frankly, php sucks for imho. The Python codebase will then create the json files through the predictive modeling (98% accuaracy) and then the go program will populate the db for me.

There are also some node scripts that help test the data since the data is json.

All in all a good day of work. The VP seems scared since he knows no one on this side of town knows about this kind of tech. Me? I am just happy I get to experiment. Y'all should have seen his face when I showed him a rather large app written in Clojure, the man just went 0.0 when he saw Lisp code.

I think I scare him.

Really now?
AWS, #1 cloud provider with their #1 cloud database DynamoDB, all shiny, highly dynamic NoSQL, your data schema could change any second...

then DynamoDB errors out when one of your values is an empty string? {"foo":""} is impossible to store?

Like nobody ever saw or used empty strings a a value or what? There are tons of upvotes to fix this.

I just have to imagine the Product owner standing there: "No,no,no. They are just using it wrong if their data has empty strings as value. Won't fix!"

Inherited a simple marketplace website that matches job seekers and hospitals in healthcare. Typically, all you need for this sort of thing is a web server, a database with search

But the precious devs decided to go micro-services in a container and db per service fashion. They ended up with over 50 docker containers with 50ish databases. It was a nightmare to scale or maintain!

With 50 database for for a simple web application that clearly needs to share data, integration testing was impossible, data loss became common, very hard to pin down, debugging was a nightmare, and also dangerous to change a service’s schema as dependencies were all tangled up.

The obvious thing was to scale down the infrastructure, so we could scale up properly, in a resource driven manner, rather than following the trend.

We made plans, but the CTO seemed worried about yet another architectural changes, so he invested in more infrastructure services, kubernetes, zipkin, prometheus etc without any idea what problems those infra services would solve.

Today was a manic-depressive kind of day. Spent the morning helping some developers with getting their code to run a stored procedure to drop old partitions, but it wasn't working on their end. It was a fairly simple proc. But working with partitions is a little like working with an array. I figured out that they were passing the wrong timestamp, and needed to add +1 to delete the right partition. Got that sorted out, and things were good. Lunch time.

After lunch I did some busy work, and then the PO comes up at about 2PM and says he's assigned some requests to me. The first was just attaching some scripts. Easy. The second, the user wants a couple of schemas exported ... at 6PM. I've been in the office since 6:45AM.

While I'm setting up some commands to run for the data export, a BA walks up and asks if I'm filling in for another DBA who is out for a few weeks. Yep. There's a change request that hasn't been assigned, and he normally does the work. I ask when it's due. Well, the pre-implementation was supposed to be done in the morning, but it wasn't, and we're in the implementation window ... half way through. I bring up the change task, and look at. Create new schema and users. That's all it says. The BA laughs. I tell I need more to go on. 10 minutes later he sends an email with the information. There's only two hours left in the window, and I can only use half of it, because the production guys have to their stuff, and we're in their window. Now I'm irritated, because I'm new to Oracle, and it's an unforgiving mistress. Fortunately, another DBA says he'll do it, so that we can get it done in time. But can't work it either, because Dev DBAs don't have access to QA, and the process required access for this task. Gets shelved until the access issue is resolved. It's now after 4:15PM. I'm going to in traffic with that 6PM deadline.

I manage to get home and to the computer by 5:45PM. Log in. Start VPN. Box pops on screen. Java needs to update. I chose skip update. Box pops up again. It won't let me log in until Java is current. Passed.

I finally get logged in, and it's 6:10PM. I'm late getting the job started. I pull up Putty and log into the first box, and paste my pre-prepared command in the command line and hit error. Command not found. I'm tired, so it's a moment to sink in. I don't have time for this.

I log into DBArtisan and pull up the first data base, use the wizard to set the job, and off it goes. Yay. Bring up the second database, and have enter the connect info. Host not found. Wut? Examine host name. Yep, it's correct. Try a different method. Host not found. Go back to Putty. Log in. Past string. Launch. Command not found. Now my brain is quitting on me. Why now? It's after 6:30PM. Fiddle with some settings, reset $Oracle home. Try again. Yay. It works. I'm done. It's after 7PM.

There is nothing like technology to snatch the euphoria of a success away from you. It's a love-hate thing, but I wouldn't trade it for anything else. I'm done. Good night.

When you go at your first IT internship and you realize that a SQL database could be way bigger that anything you could had imagined.

Then I showed the schema to one of my database engineer friend and he laughed at me. :(

At this point, I just feel bad for my coworker.

No, I am not frustrated or angry, just feeling terribly bad for her as how difficult life must be for someone so dumb.

We are introducing a new method to track some data in our product, like total number of sign-ups, DAU, etc.

Now the implementation is already WIP and this is known to all.

The dev has documented the approach where he has mapped the screen name, a screenshot, and a snippet of the schema that tracks that particular screen.

I kid you not guys, this coworker somehow landed on that document and started some scientific study to try and extract data.

Yes, she looked at the schema screenshot and spent like few hours trying to decode it to figure out the sign-ups and DAU.

Data via a screenshot in a document. I can't even express it.

And then texts me in panic mode that she isn't able to access the data because the file is . jpeg within a document.

I asked where is she executing the schema, because I thought she is joking initially. She said she doesn't know and asked me where she should execute it.

My mind is numb. Life must be real hard when you are so fucking dumb.

Ok. FUCK MySQL Workbench.
Most of our products built on MySQL and we just had enough of the tools that we are using for our mysql databases...
We decided to make our own tool :)

If it goes well, we plan to open source it. Would you guys be interested in it?
We planned the following features:
1. Schema editing
2. Schema versioning
3. Update/downgrade script generation to move easily between schema versions
4. Manual/auto sync
5. Might include our own replication solution too...

What do you think?

This is a short tale that can be summed up as "oh fuck meee".

After finishing an API the night before I settled in for a day of bug fixes and tidy ups. Until slack went off.

The front end dev was getting an error, a code breaking error. After doing the standard process of request checking i went okay must be me. I find the script that is has the error and the line that it is failing at.

Que 2 hours of the full cycle of anger, sadness, pleading, and finally acepting that it had finally happened I had gone insane. The code was to documentation best practise correct and it still had the same error.

I the cheaked the DB on a whim and I found that my code was not wrong and it was doing exactly what I wanted the data however had a single record that was old and the schema had change juuussstt enoigh to break everything at that record. One 3 secound deletion later code ran perfectly.

Fucking hate my job 😡
I joined as nodejs dev at a mnc 3months ago involved in banking software in which i dont have any domain knowledge.. first 10 days I was told to go through fucking udemy nodejs and graphql tutorial (wtf) which i already have experience with before joining.. after that my reporting manager gives me task to resolve fields and gave me shitty jira story link to read.. that shit story link had no explanation about the fields and what the database it is, then she says to use some shitty sdk which is built internally by shiity devloper which had no documentation and have to follow other module which was again written by that sr. Dev... They hav fucked up the graphql and nodejs and entire stack and also till date no one has ever given any explanation about the domain and the fields and database schema.. this manager refuses to share knowledge about the domain now how the fuck i resolve the graphql schema which was again written by non technical b.a.. all they have used is latest technology in a shitty way with no standards to to follow .. no dataloading no caching no batching.. use shitty sdk which does not give access to dbconn and fucking tightly coupling expressjs which when i start consumes crazy 400Mb of memory .. these fucking seniors devs + the fucking b.a having 12+. Yrs exp each have fucked the entire codebase... Each day killing my passion for app development.. fuckkk ... Dunno what to do now

My day:
5:30AM - 2yo son wakes me up, I send him back to his bed
6AM - wakes me up again, gotta grab a coffee
7:30AM - leaving towards the office
8:30AM - finally arriving to the office, after horrible traffic.
*continue working on major schema change I started yesterday*
12:30PM - Lunch + Beer
1:30PM - Tequila time!
*back to work*
7:30PM - Finally done with coding, leaving the office
8PM - home at last
9:30PM - Beer time
9:31PM - "I'll just write a couple of more lines"
12:30AM - "That's it, no more code for today"
12:31AM - "I'll just scroll through devRant"

Yep, today was a Monday. Had a request to truncate some tables. Set up a script to disable constraints, truncate, and re-enable constraint. It ran flawlessly. Too bad I ran against the wrong schema, though. Ran it against a dev schema instead of the requested itest schema, because the dev schema was stuck in my head. Well, I guess I tested it in dev. Against some developers schema. But, hey, it worked flawlessly. Because, you know, when you are pointing at the wrong place, it will always work. Every time.

*Releases software for State project comprising of a solution with 3 WinServices and 2 WCF Services*

*Literally 15 min after release*

Email from State: "Here is an updated wsdl schema that is going literally fuck up everything you have released, enjoy!"

Me: *starts convulsing*

You know you have chosen the wrong JS library when you read a function definition like this:

validateSync(schema, options, callback)

Me: I finished the tests! Almost 1000 lines of testing data. They're actually pretty thorough. We're ready to push the changed.

Manager: That's great! But the requirements changed. We're gonna change the schema and queries quite a bit.

Me: *asks boss for the id of his test store so I can apply experimental schema changes to test out a new dashboard app*

Boss: *gives me production store id and doesn't say anything*

Fate: … "You got lucky this time."

This is the CEO of the company btw. Startups. *sigh*

Best:
- optimized a lot of queries and pieces of code
- graduated from the dutch equivalent of community college
- started a new education
- updated our password schema from a shameful algorithm to bcrypt

Worst:
- haven't been able to convince my colleague and bosses to automate stuff
- still no tests
- still a php dev
- still alone

2018:
Come at me with your c++ and robots! I'll fucking master you!

Promised to stay late to run a large deploy.

Now waiting for a large schema migration on a 7M row table.

My boss's SQL schema has no foreign keys and he said he left them out intentionally because they should be handled in the application layer and they're a large performance impact.

This is a fresh greenfield project and he's already pre-optimizing for problems we don't have yet, on things that may or not be bottlenecks using ideas (e.g. foreign keys have huge performance costs on mariadb/auora) with no hard data or facts to back them up.

Let's start a new project with some technical debt!

I just love PMs that wait until an hour before I'm supposed to leave and come and say, "Hey, we need you to do this thing right now. We're doing performance testing in QA this evening." And then take 45 minutes to get the information to me, without telling me which database or schema. Like, did you just find this out, or have you known this for the past week, and are just now telling me?

So we had this legacy Objective-C codebase for a mobile app that was actually pretty good: I'd inherited the codebase and spent the past several years gradually improving it and I was actually quite proud of the work I put into it. So of course management decides to scrap it (with NO consultation from the engineers) and outsource a complete rewrite of the app in C# for Windows Universal.

Let me tell you. That code was without a doubt and without exaggeration the *worst* code I've seen in my close to 30 years of experience as a developer. I mean they broke every rule in the book, I'm talking rookie mistakes. Copypasta everywhere, no consistent separation of concerns, and yet way too many layers. Unnecessary layers. Layers for the sake of layers. There was en entire abstraction layer complete with a replicated version of every single data class *just* to map properties in pascal case to the same property in camel case. Adding a new field to a payload in the API amounted to hours of work and about eight different files that needed to be modified. It was a complete nightmare. This was supposed to be a thin client, yet it had a complete client-side Sqlite database with its own custom schema (oh and of course a layer for that!) completely unrelated to the serverside schema, just for kicks. The project was broken up into about eight or nine different subprojects, each having their own specific dependencies on various of the other subprojects in such a tightly-knit way that it made gradual refactoring almost impossible. This architecture was so impressively bad, it was actually self-preserving!

Suffice it to say it was a complete nightmare, and was one of the main reasons I ended up leaving that company. So just sayin', legacy code isn't always bad. :)

Dear API vendor,

Please get off your arse and learn about REST, OpenAPI, JSON Schema, XSD and basic documentation so that I don't have to guess how to use your shitty, inconsistent, RPC over HTTP service.

With Love,

Platypus

I am sitting on a fucking vibrator!!

PoV: Writing my code, minding my own business, listening to lofigirl.. private Schema<?> asSc
<rrrrrrrRrRRrRRRRRRRRHHHHHHRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRHHHHHRRRRRRRRRRRRRRRRRHHRHRHRHRHRHRHRHRHRRRRRRRRGGGGGRRRGGRGRGGRGRRRRRRRRrrrrrr.><KNOCK KNOCK KNOCK KNOCK KNOCK KNOCK><3.5sec pause><rrRRRRRRRHHHRHRHRHRRRRRRRHRRHHRHHRRRRrrrrr>
hema(Type ty
<hhRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRrrRRRrRRRRRRRrRRRRhhRHHrHRHHRRRrrrrr>
pe) {

How the hell am I supposed to concentrate when my neighbours are trying to make me cum with a drill.

Swear the God, my ass is on some serious vibro.

endor's first magical adventures with PostgreSQL

"Alright, got the docker image up and running, and I'm connected to the db, both from console and from Datagrip! Cool, let's get started with the tutorial!"
*cue montage of me using Datagrip to create my first schema, then the first table, then insert a bunch of data to try things out*

"Cool, now let's see if I can view my data from the console"
db1-# select * from my_schema.table1
db1-# [nothing]

"*Ahem*, I said:"
db1-# select * from my_schema.table1
db1-# [nothing]

db1-# select * from my_schema.table1
db1-# [cricket noises]

"Wut, why can't I see the data that I inserted? Wtf is going on?"

*30 minutes later*
"Alright, I have no idea what's going on, so let's try inserting the data from console and see if Datagrip can see it"

db1-# insert into my_schema.table1(id, name, field2, field3) values (1, 'Mike', null, 123), (2, 'Jake', 0, 456);
ERROR: syntax error at or near "SELECT"
LINE 2: SELECT
^

"Wait, what?"

db1-# insert into my_schema.table1(id, name, field2, field3) values (1, 'Mike', null, 123), (2, 'Jake', 0, 456);
INSERT 0 2

"Wtf? Haaang on... "

db1-# select * from my_schema.table1;
id | name | field2 | field3
----+------+--------+--------
1 | Mike | | 123
2 | Jake | 0 | 456
1 | Mike | | 123
2 | Jake | 0 | 456
(4 rows)

*eye twitches*

procrastinating by getting drunk since 11:00 AM, and writing specs for my (hypothetical) language/os/platform.

feeling righteous retribution because the client made me be stressed for 3 hours due to an issue that THEY caused but for 3 hours the only info I had was "there's a critical blocker issue and we're convinced YOU caused it"

well... no... i did NOT cause the fact that you UPGRADED PHP DURING THE WEEKEND BEFORE MONDAY'S PRESENTATION TO CLIENT (while waiting for an urgent commit from me).

seriously.

also, germans. i've heard many times from other people that they're... basically racist towards us (slavic nations), thinking of us as untermensch, coal-miner peons, but I didn't realize their passive-aggressive covertly smug demanding attitude is due to this, I just assumed it's a reaction to me being incompetent.

so yesterday when we finished the call (in preparation for which I tried to switch to their "client demonstration" branch since that's where the error was, and I wanted a headstart on fixing it, ended up in a place that my today's whole-day task should be "rebuilding the DB into working condition", because there's about 10 "core" sql scripts in two different folders, which need to be run (in a very specific order, of course, which readme tells you, but what it tells you has been outdated at least for 3 months, of course), and

...THE MAIN CORE SCRIPT THAT IS THE FIRST TO RUN, THAT CREATES THE DB schema, HAS THREE SYNTAX-LEVEL typos which fail it mid-way...

...the joys of continuous deployment via scripts, I guess? I would love to challenge any person from them to screenshare to me, manual deployment of the current version from zero, and I would be willing to give the person 20% of my monthly salary if they would be able to do it within 20 minutes.

but... well...
the point is, i should be doing not entirely bullshit stuff.
but yesterday's 6 hours of being in "at full attention because it seems we fucked up" totally convinced me, that today I'm taking a break.

So I'm gonna go buy another 3 beers and continue writing the specs of my dream language/os/platform.

Come on guys, use those JSON schemas properly. The number of times I see people going "err, few strings here, any other properties ok, no properties required, job done." Dahhh, that's pointless. Lock that bloody thing down as much as you possibly can.

I mean, the damn things can be used to fail fast whenever you misspell properties, miss required properties, format dates wrong - heck, even when you want to validate the set format of an array - and then libraries will throw back an error to your client (or logs if you're just on backend) and tell you *exactly what's wrong.* It's immensely powerful, and all you have to do is craft a decent schema to get it for free.

If I see one more person trying to validate their JSON manually in 500 lines of buggy code and throwing ambiguous error messages when it could have been trivially handled by a schema, I'm going to scream.

Yes its completely necessary to have a spring server with a mysql database with docker containers all over your ass for 3 fucking endpoints and a (url, varchar, varchar) schema. Fuck you. How the fuck do i run all this shit and how do you expect me to create a frontend for something that has no documented endpoints?? Fuck you.

In other news, im now a senior.

*Spends the day writing script to back up production nosql databases before applying sweeping schema changes.*

*Accidentally runs the script to apply sweeping changes instead of backup.*

😭

Bloody bitch. Asks me to analyze pretty big schema and write my concerns. Sits with me for 30 minutes, gets the details and sends a mail to the lead and claims 12 fucking hours of work. It's me who did all the work bitch not you. I am supposed to report to her.

You may know I love to hate tests. Well not the tests actually, what I hate is the TDD culture.

DBMS schema in my app dictates a key can either have a value, or be omitted - it can't be null, and all queries are written with that in mind (also they're checked compile-time against schema). But tester failed to mock schema validation, inserted a bunch of null keys with mock data, actually wrote assertions to check those keys are null (even though they never should be), and wanted me to add "or null" to my "exists" queries.

No, we don't need more tests, and you're not smart with your "edge cases" argument. DBMS and compiler ensure those null values can never exists in our DB, and they're already well tested by their developers. We need you to stop relying on TDD so much you forget about the practical purpose of the code, and to occasionally break from the whole theoretical independent tests to make sure your testing actually aligns with third-party services some code uses.

And no, we don't need more tests to test your mocks, and tests to test those test, and yo dawg, I heard ...

There is a commercially sold ERP solution that has it's DB schema in excel and Other documentations in MS Word. And its not even properly structured, no schema diagrams, last updated for a 4 year old major release 😒😫.
I have to develop a custom module for it and that requires building an ActivexDLL Project in VB fucking 6 😭😭 .

VB6
Unstructured Documentation
Legacy code
Incomplete documentation
FML

Tell me if you want ss in comments.

I worked with this hack of a backend dev that was too lazy to add a complex(ish) object to our CMS tool. His solution?

One giant-ass text box with the label "put JSON here".

If tech people were using it I wouldn't mind, but our poor content managers have no idea what json is. Plus like... no examples, no schema... they would have to change shit then go look at the website to see if it worked. Fucking asshole.

Plus.. I mean SHIT, MAN! This was in a Node.js tool... if you have the Json parser you could just GENERATE the respective form fields. DO YOUR JOB

TLDR: There's some days where the Gods of IT are not with you. Just lost a whole day of work.

So this morning, we (me and my team) big performance issues with our web app. Lot's of requests time out, big latency, etc

Try to ssh to VPS, latency of 10 seconds between user input and output.

Usual checks: RAM ok, Proc ok, hard drive ok, reboot server (20 minutes), update/upgrade

We decide to call OVH. After 15 minutes call, we try to reboot in rescue mode. Reboot fails at 60% + everything freezes.

After an hour, OVH opens an incident ticket on +200 vps instances (including mine) everything is down during +1h

Finally everything is okay ! Even had time to migrate my new database schema.

Still, quick heavy on the mind but feels good to go home with everything working out correctly

Today I spent like 30 mins trying to pass one of FreeCodeCamps test.
Found the problem.
I added 'u' to "favorite" when creating the field for a schema. 😂😂😭

The state of digital comic book metadata is a mess. There is not really a standard format if the metadata even exists at all. All digital comic books consist of is a zip or rar with ordered images and potentially some type of file to store metadata. The closest thing to a standard is the Comic Rack format of metadata and even that is not very widespread. There exists a project called comictagger(on github) that attempts to assign metadata in Comic Rack format but it is somewhat unpolished yet provides a solid feature set.

I am planning on making a program to organize comics based on metadata attributes and am frustrated with the lack of consistency in this department. This isn't really a problem because of any developers but I would argue more so due to the organization of comic books themselves. For example, the term volume can have a different meaning based on who is asked or what context is used. The redundancy between issues and trade paperbacks can also lead to confusion and logistical problems. I just wish we already had a widespread schema in place for comic books metadata already.

The feature was to parse a set of fairly complex xml files following a legacy schema. Problem was, the way this was done previously did not conform to the schema so it was a guideline at best, which over the course of many years snowballed into an anarchy where clients would send in whatever and it was continuously updated per case as needed. They wanted to start enforcing their new schema while phasing out the old method.

The good news is that parsing and serialization is very testable, so I rounded up what I could find of example files and got to work. Around the same time I asked our client if they had any more examples of typical cases we need to deal with, and sure enough a couple of days later I receive a zip with hundreds of files. They also point out that I should just disregard the entire old set since they decided to outright cut support for it after all if it makes things simpler. Nice.

I finish the feature in a decent amount of time. All my local tests pass, and the CD tests pass when I push my branches. Once we push to our QA env though and the integration tests run, we get a pass rate of less than 10%.

I spend a couple of days trying to figure out what's going on, and eventually narrow it down to some wires being crossed with the new vs. old xml formats. I'm at a loss. I keep trying to chip away at it until I'm left with a minimal example, and I have one of those lean-back moments where you're just "I don't get it". My tests pass locally, but in the QA environment they fail on the same files.

We're now 3 people around my workstation including the system architect, and I'm demonstrating to the others how baffling and black magic this is. I postulate that maybe something is cached in my local environment and it's not actually testing the new files. I even deleted the old ones.

"Are you sure you deleted the right files?"

"Duh of course -- but let me check..."

Fuck, really FUCK the fucking MySQLWorkbench on Mac.
Useless piece of shit.
I fucking touched some fucking buttons and now I can't have my view back with query editor, output results, and schema view.

A fucking hour wasted restarting this shit of a tool touching things, nothing. All to execute a fucking stupid query.
AHAHAHAAHAHAHAHAH FUUUUUUUUUUCKKKKK
I NEED to work, not to understand how your stupid GUI works, designed by a cripple mind with poor IQ and developed by retarded

I really hate PHP frameworks.

I also often write my own frameworks but propriety. I have two decades experience doing without frameworks, writing frameworks and using frameworks.

Virtually every PHP framework I've ever used has causes more headaches than if I had simply written the code.

Let me give you an example. I want a tinyint in my database.

> Unknown column type "tinyint" requested.

Oh, doctrine doesn't support it and wont fix. Doctrine is a library that takes a perfectly good feature rich powerful enough database system and nerfs it to the capabilities of mysql 1.0.0 for portability and because the devs don't actually have the time to create a full ORM library. Sadly it's also the defacto for certain filthy disgusting frameworks whose name I shan't speak.

So I add my own type class. Annoying but what can you do.

I have to try to use it and to do so I have to register it in two places like this (pseudo)...

Types::add(Tinyint::class);
Doctrine::add(Tinyint::class);

Seems simply enough so I run it and see...

> Type tinyint already exists.

So I assume it's doing some magic loading it based on the directory and commend out the Type::add line to see.

> Type to be overwritten tinyint does not exist.

Are you fucking kidding me?

At this point I figure out it must be running twice. It's booting twice. Do I get a stack trace by default from a CLI command? Of course not because who would ever need that?

I take a quick look at parent::boot(). HttpKernel is the standard for Cli Commands?

I notice it has state, uses a protected booted property but I'm curious why it tries to boot so many times. I assume it's user error.

After some fiddling around I get a stack trace but only one boot. How is it possible?

It's not user error, the program flow of the framework is just sub par and it just calls boot all over the place.

I use the state variable and I have to do it in a weird way...

> $booted = $this->booted;parent::boot();if (!$booted) {doStuffOnceThatDependsOnParentBootage();}

A bit awkward but not life and death. I could probably just return but believe or not the parent is doing some crap if already booted. A common ugly practice but one that works is to usually call doSomething and have something only work around the state.

The thing is, doctrine does use TINYINT for bool and it gets all super confused now running commands like updates. It keeps trying to push changes when nothing changed. I'm building my own schema differential system for another project and it doesn't have these problems out of the box. It's not clever enough to handle ambiguous reverse mappings when single types are defined and it should be possible to match the right one or heck both are fine in this case. I'd expect ambiguity to be a problem with reverse engineer, not compare schema to an exact schema.

This is numpty country. Changing TINYINT UNSIGNED to TINYINT UNSIGNED. IT can't even compare two before and after strings.

There's a few other boots I could use but who cares. The internet seems to want to use that boot function. There's also init stages missing. Believe it or not there's a shutdown and reboot for the kernel. It might not be obvious but the Type::add line wants to go not in the boot method but in the top level scope along with the class definition. The top level scope is run only once.

I think people using OOP frameworks forget that there's a scope outside of the object in PHP. It's not ideal but does the trick given the functionality is confined to static only. The register command appears to have it's own check and noop or simply overwrite if the command is issued twice making things more confusing as it was working with register type before to merely alias a type to an existing type so that it could detect it from SQL when reverse engineering.

I start to wonder if I should just use columnDefinition.

It's this. Constantly on a daily basis using these pretentious stuck up frameworks and libraries.

It's not just the palava which in this case is relatively mild compared to some of the headaches that arise. It's that if you use a framework you expect basic things out of the box like oh I don't know support for the byte/char/tinyint/int8 type and a differential command that's able to compare two strings to see if they're different.

Some people might say you're using it wrong. There is such a thing as a learning curve and this one goes down, learning all the things it can't do. It's cripplesauce.

So this web company i joined had a page load time in minutes. The free text search (inverted index search, based on elasticsearch) queries would return results in 10-45 seconds (should be milliseconds always). The indexes had no schema. And they would crawl data and feed into mssql db, which had a 2 gb/db limit on the free version. So everytime the db hit the limit, a new db was created and the name was incremented by one.

Had a very tough time cleaning up that mess. Plus the architect who had made this architecture was on his way out and unhelpful to the core.

What was worse was that most of the changes i did were very simple changes that should have been done long back. Basic sanity changes.

Print("Hello World")

When people design a brand new Postgresql schema (case sensitive) using a mix of upper and lower case letters.
Only to then proceed and escape every single table and column name in every single query.

I can't convince my team that a good database model promotes a good API design and a good UI/UX experience.

Instead, I have to work with a ridiculous table setup.

Imagine, if you will, a table (table B) that references another table (table A) via a foreign key. The FK is a string in both tables. And table A only has one column, which is labeled as "name".

The schema i have to look at it kind of like this:

Table parent
Name varchar(10) primary key

Table child
Name varchar (10),
parent_name varchar(10),
Foreign key (parent_name) references parent (name)

Sorry if the syntax is wrong, a little frustrated having to look at it...

Am I crazy to want to change this table design? Am I missing something? I feel like I'm taking crazy pills, because this is just scratching the surface of the problems I have to deal with.

We've got a big legacy app which we have to rewrite. The current client applications are only working on XP(!). We have to move the clients to the browser so we can finally get rid of all XP vm-s. The db schema is complex but still 1000+ stored procedures and functions and about a hundred tables with 13 years of data.
So I ask the guy responsible for maintaining the DB code. (he is ~25 years older than me)

me - Where is the source of the database. Which project?
he - Where would it be? It's in the db.
me - So we've got a huge db without VCS, upgrade/downgrade scripts, etc?
he - Yes. I don't get why young developers always want to use shiny new tech like git just because it is cool. It has nothing that an external usb backup drive can't do.
me - VCS has been around since the early 1980's...
he - If you really want, you can put it under git or whatever, so you can sleep better, but I still think it is stupid and a waste of time.

I get that it's hard to keep up, but getting personal...

How do you feel when in the middle of tight deadline, your client calls you and tells you that he has changed some little detailing in the requirements, like naming, color schema etc. This needs to be done within the deadline. You tell yes and tell him to send email.
After receiving the email, you find out it's not little it's bag of new features listing.
Now either you have to overwork or let the client go....

So here I am trying to understand the database schema in order to write a REST interface. Then I find that one of the tables contains an id and a name columns but the name is Integer not a string!
I contacted the developer who gladly explained that it was easier for them to store the ids in the database but the actual names strings were hard coded in the source files so they can handle translations!

our team are responsible to build backend restful API for other team to look up data in DB.
the consumer team just sit beside us.
the interface definition came from our pm in a different time zone. btw he did not have any programming background.
and he insisted that just build what he said and ignore the noise from the consumer team. because each interface change should be considered as new features and need him to prioritize and create user story and he will review the schema with the pm from consumer team and so called architecture who did not coding real shit for years.
we ended up with building shit code not useable by our real consumer.
yes he do manage to keep our team busy building worthless shit and accomplishmented lots of jira items to show we have value to change a useless shit into very hard to use shit

I talked to the client how functionality should look like on UI, draw a mockup, designed and made changes to db schema, created REST api, made documentation how to use it, told frontend developer to make changes on frontend application according to the documentation and mockups. Still no one have fucking clue how to do it. Fucking testers can’t write anything, only clicking.
So I sent curl code how the fucking request should look like exactly then resolved bugs they reported as won’t fucking fix because I will not be also making fucking frontend. Probably they even don’t know what curl is. What a fucking fuck.
And that’s what I am mostly doing from Monday till Friday to keep this project going.
It’s cause client are nice guys and we are doing something good, not some fucking ai, blockchain, big data, financial scam everyone is wanking around.
And friends are asking, why I drink.

To me this is one of the most interesting topics. I always dream about creating the perfect programming class (not aimed at absolute beginners though, in the end there should be some usable software artifact), because I had to teach myself at least half of the skills I need everyday.

The goal of the class, which has at least to be a semester long, is to be able to create industry-ready software projects with a distributed architecture (i.e. client-server).

The important thing is to have a central theme over the whole class. Which means you should go through the software lifecycle at least once.

Let's say the class consists of 10 Units à ~3 hours (with breaks ofc) and takes place once a week, because that is the absolute minimum time to enable the students to do their homework.

1. Project setup, explanation of the whole toolchain. Init repositories, create SSH keys for github/bitbucket, git crash course (provide a cheat sheet).
Create a hello world web app with $framework. Run the web server, let the students poke around with it. Let them push their projects to their repositories.
The remainder of the lesson is for Q&A, technical problems and so on.

Homework: Read the docs of $framework. Do some commits, just alter the HTML & CSS a bit, give them your personal touch.
For the homework, provide a $chat channel/forum/mailing list or whatever for questions where not only the the teacher should help, but also the students help each other.

2. Setup of CI/Build automation. This is one of the hardest parts for the teacher/uni because the university must provide the necessary hardware for it, which costs money. But the students faces when they see that a push to master automatically triggers a build and deploys it to the right place where they can reach it from the web is priceless.
This is one recurring point over the whole course, as there will be more software artifacts beside the web app, which need to be added to the build process. I do not want to go deeper here, whether you use Jenkins, or Travis or whatev and Ansible or Puppet or whatev for automation. You probably have some docker container set up for this, because this is a very tedious task for initial setup, probably way out of proportion. But in the end there needs to be a running web service for every student which they can reach over a personal URL. Depending on the students interest on the topic it may be also better to setup this already before the first class starts and only introduce them to all the concepts in a theory block and do some more coding in the second half.

Homework: Use $framework to extend your web app. Make it a bit more user interactive with buttons, forms or the like. As we still have no backend here, you can output to alert or something.

3. Create a minimal backend with $backendFramework. Only to have something which speaks with the frontend so you can create API calls going back and forth. Also create a DB, relational or not. Discuss DB schema/model and answer student questions.

Homework: Create a form which gets transformed into JSON and sent to the backend, backend stores the user information in the DB and should also provide a query to view the entry.

4. Introduce mobile apps. As it would probably too much to introduce them both to iOS and Android, something like React Native (or whatever the most popular platform-agnostic framework is then) may come in handy. Do the same as with the minimal web app and add the build artifacts to CI. Also talk about getting software to the app/play store (a common question) and signing apps.

Homework: Use the view API call from the backend to show the data on the mobile. Play around with the mobile project to display it in a nice way.

5. Introduction to refactoring (yes, really), if we are really talking about JS here, mention things like typescript, flow, elm, reason and everything with types which compiles to JS. Types make it so much easier to refactor growing codebases and imho everybody should use it.
Flowtype would make it probably easier to get gradually introduced in the already existing codebase (and it plays nice with react native) but I want to be abstract here, so that is just a suggestion (and 100% typed languages such as ELM or Reason have so much nicer errors).
Also discuss other helpful tools like linters, formatters.

Homework: Introduce types to all your API calls and some important functions.

6. Introduction to (unit) tests. Similar as above.
Homework: Write a unit test for your form.

(TBC)

"Could not complete cursor operation because the table schema changed after the cursor was declared"

So, our customer is updating their side of the database, while their own users continue to use our software. Would've been nice if we got word of the maintenance window too.

Current directory:

upstream
potatoecode

find ./upstream -maxdepth 1 -type d -ls >> potatoecode/.gitignore

pushd potatoecode
git add .gitignore
git commit -m 'Updated gitignore" .gitignore
git rm -r --cached .
git add .
git commit -am "Purgatory"
popd

*watching with a big smile the burning CI*

--

Story of how I made some devs today very sad. They now have the joyful task to think of a better way to code than to create a nightmare blob of modified source code from upstream - where upstream has ...

- a rest API
- an extension / plugin system
- an system to even modify db schema via an API.

But nooooo.... That would be too good.

Instead one just creates an potatohead of upstream source code with modifications without any version tracking or stuff like that.

Sometimes I really wonder if the devs at our company are masochists and want to be punished....

PM: *mocking me if I'll JUST wait for feedback*
Me: *thinking* I can't produce everything from an INCOMPLETE DB schema and incomplete App Scribble, can I?

any thought about the best way to start a solo project? I think going for the db schema first is easier, curious to see your opinion 😁

Client be like:
Pls, could you give the new Postgres user the same perms as this one other user?

Me:
Uh... Sure.

Then I find out that, for whatever reason, all of their user accounts have disabled inheritance... So, wtf.

Postgres doesn't really allow you to *copy* perms of a role A to role B. You can only grant role A to role B, but for the perms of A to carry over, B has to have inheritance allowed... Which... It doesn't.

So... After a bit of manual GRANT bla ON DATABASE foo TO user, I ping back that it is done and breath a sigh of relief.

Oooooonly... They ping back like -- Could you also copy the perms of A on all the existing objects in the schema to B???

Ugh. More work. Lets see... List all permissions in a schema and... Holy shit! That's thousands of tables and sequences, how tf am I ever gonna copy over all that???

Maybe I could... Disable the pager of psql, and pipe the list into a file, parse it by the magic of regex... And somehow generate a fuckload of GRANT statements? Uuuugh, but that'd kill so much time. Not to mention I'd need to find out what the individual permission letters in the output mean... And... Ugh, ye, no, too much work. Lets see if SO knows a solution!

And, surprise surprise, it did! The easiest, simplest to understand way, was to make a schema-only dump of the database, grep it for user A, substitute their name with B, and then input it back.

What I didn't expect is for the resulting filtered and altered grant list to be over 6800 LINES LONG. WHAT THE FUCK.

...And, shortly after I apply the insane number of grants... I get another ping. Turns out the customer's already figured out a way to grant all the necessary perms themselves, and I... No longer have to do anything :|

Joy. Utter, indescribable joy.

Is there any actual security reason for disabling inheritance in Postgres? (14.x) I'd think that if an account got compromised, it doesn't matter if it has the perms inherited or not, cuz you can just SET ROLE yourself to the granted role with the actual perms and go ham...

Well, this happens time to time...

I'm freelancing as a backend guy. I like to take care of all infrastructure before really starting to build anything, this mostly includes dev/staging/prod environments with some linear promotion strategy. So.. I did this API. Still on staging, proceeding with the development as planned, everything goes according the timeline.

And then.. this happens... At some point PM told frontend guy that it's time for production (without notifying me), so the frontend guy does what "anyone" would do in this case - tells PM to create DNS record for production to point to staging app.

Time passes, I'm still unaware of this. But I'm starting to see some quality entries in the DB, not the usual QA crap. I write to them that they're doing good job and continue with my tasks.

One of the tasks required some major DB change. I could've written migrations script, but since we're not in "production" yet, I just wipe the DB and recreate schema as I need it.

In 10 minutes the furious PM starts shouting that "production" is down and I need to fix it ASAP.

I'm lost, I'm asking questions, I'm slowly understanding what's happening...

So I want to grab some coffee, sat back down, wrote politely that they suck, added a finger emoji and terminated the contract.

Felt like the right thing to do as I definitely don't want to continue within the same "team".

When you look at a client's current database schema and you're like (whatIWasGoingToChargeYou * numberOfUnnormalizedTables)

I wonder what is going on in the minds of inexperienced developers. It must be very interesting.

I just read this bit of code
The task was to implement a certain schema into a database. They were given an ORM library, and several tutorials explaining what an ORM was and how it did it.

The result were these 3 models:
- A defaultUser with all of the defaults values for the User model. It wasn't even supposed to be instantiated, just accessed by `defaultUser.fields.username.default`
- another default table for another model.
- The "main" model, containing all other models in the form of JSON fields that would contain an array with other JSON objects that would represent the instances themselves.

I will say though, they made a home-brew ORM with (most of) all the logic a normal ORM has by parsing through the "main model", except, of course, common sense

I think UPS' Api documentation and service must be the worst documented and build API I have ever seen from a corporate.

1. The developer website is a mess. A total mess. You can barely find the API type you are looking for.

2. When you get the API and download the documentation, the files, .pdf etc is still a mess. Pages long that most are craps.

3. Each request returns Status Code 200. Even if it is an error. This blew my mind.

4. Each request, based on error type or based on tracking activity returns different JSON schema.

For example, the JSON Schema for a shipment in transit is different from JSON schema for a shipment that has been delivered. A shipment that has been returned, a shipment that required signature etc. They are different from each other.

5. And the worst. They do not provide with test tracking codes. I have found some on internet, but they do not work in development and production environment.

It's a shame that people don't want to use F# but prise C# for how cool it became and continue becoming. At the same time, little do they know that many of the features were simply drawn from F#.

It's just rediculous how far this OO and C-Style syntax crap has progressed. They keep copying things from functional langugages, making the initial language to be a monstrocity like C++ is now, insted of just using languages like C#. I mean, it was right there before C#: async/task, immutablility, records, indexes, lambdas, non-null by default, who the hell knows what else.
Besides, many people (in my company at least) are just blindly overengineering with patterns and shit, where a simple function would be just enogh.

Watch some some NDC talks about F#, in particular those of Scott Wlaschin. It's just better in so many ways: less noice (I'm looking at you, brackets, commas and semicolons), the whole LOT of type inference and less duplication (just look at the C# signatures of linq methods - it's difficult to read them), immutability by default, non-nullable by default, ADTs and pattern matching, some neat features like type providers (how many times have used "paste special" or an online tool to create C# classes from a JSON/XML file, and how many times have your regenrated it because of schema changes?) and units of measure.

Of course, in some cases it's not optimal, in some cases mutable datastructures of C# are better for performance. But dude, how many performance critical systems have you wrote in C#? I mean, if it comes to performance you should use Rust or C++ or C after all.

*sighs*

Just spent half an hour helping someone install my open source side project only to see more GitHub issues have been raised because the underlying application changed its schema without warning...

Please remind me why I do this...

Why does MySQL call databases schemes? Makes absolutely no sense.

Schema is the metadata around your tables or documents. What is Json Schema? Metadata around a json object.

MySQL, databases contain data. Schemes do not! Bad DB technology. Bad...

I was presented with a different backend schema today after finishing the front end, not before not even during, it just had to be after.

It's too early to be asking these questions today:

Are your DB schema changes checked into source control?

What branch are they checked into?

Why are the schema changes checked into one branch, but deployed to a completely different database?

Is my CI pipeline deploying incorrectly? Oh, you manually deployed changes.

Are your DB changes in source control an accurate reflection of what you actually put in the staging database?

Why not?

Can I just cherry-pick update my schema with your changes from the staging database?

Why is there a typo in your field name?

Oh. Why is there a typo in the customer data set? Don't they know how to spell that word?

Why is the fucking staging database schema missing three critical tables?

Is the coffee ready? I need coffee.

Why is the coffee not ready yet?

What's going on in DevRant this morning?

What project am I working on now anyway?

Did my schema update finish yet?

Yup, it finished. Crap. Where the hell do I keep those backup files?

What's the command line to restore the file again?

Why doesn't our CLI tool support automated database restores?

I can fix that. What branch name should I check the CLI tool into?

What project was I working on this morning again?

It's so annoying to set up a vm and oracle DB. Just when you think that you are done with recreating a schema from schema dump, four packages won't compile. And of course they are wrapped and it's the most generic malformed or corrupted wrapper unit error...

Have to create a social network type app for a project. Anyone ever used firebase for the purpose?
Any sort of help in defining the DB schema would help :D

DynamoDbMapper ISSUE

There were multiple pojos which maps with one of our DynamoDb table with slightly different schema (leveraging nosql).

For one of the pojos, while populating one of the attributes, it was always throwing some weird exception and no one had any idea about it.
An intern was assigned to fix it in case some new pair of eyes can observe something weird about the pojo.

Later, I realized that the way DynamoDbMapper behaves inside a pojo is very particular and hidden.
A method was declared as public instead of private in the pojo, and DynamoDbMapper while mapping the pojo to the table with reflection, it said that this attribute (a substring of the method name) cannot be converted.

Finally, it was just a single word change from PUBLIC TO PRIVATE.

SQL was never meant to be used in web applications. It is inherently insecure in its c&c schema. Can we please fucking stop using databases that are not designed for the web in the web? Please?

(I know, I know, we’re stuck with what we have. But for fok’s sake, I want to strangle that muthafucka who thought it’s a good idea…)

This is the link that has brought me the most joy acceleration in my life.
Go see what it does yourself.
I love adminer so much.
Hyper recommended.
Screw MySQL Workbench.

(this rant was saved since that day images stopped working, good job guys)

ANYONE USING MONGOOSE HERE?!!

TypeError: User is not a function.

That User is a Schema. I've been searching for so long now and I can't seem to find a way to make this work. Sorry for being desperate that I had to ask on devRant.

Can there be a happy rant?

This is going to be a bit of a rambling semi coherent story here:

So this customer who just doesn't know what their data schema is or how they use it (they're a conglomeration of companies so maybe you get how that works out in a database). For every record there's like a ton of reference number type things mapped all over the DB to fit each companies needs needs.

To each company the data means something different, they use the data differently, and despite their claims otherwise, I think there are some logical conflicts in there regarding things like "This widget is owned by company A, division B, user C.". I'm also pretty sure different companies actually don't agree on who owns what... but when I show them they just sort of dance around what they've said in the past...

So I write a report (just an SQL query that outputs ... somewhere ... I mean what isn't that?) that tells them about all the things that happened given X, Y, Z.

Then every damn morning they'd get all up in arms about how some things are 'missing' but sometimes they don't know what or why because they've no clue what the underlying data actually is / their own people don't enter the data in a consistent way. (garbage in garbage out man...)

So I've struggled with this for a few weeks and been really frustrated. Every morning when I'm trying to do something else ... emails about how something isn't working / missing.

In the meantime I'm also frustrated by inquiries about "hey this is just a simple report right?" (to be clear folks asking that aren't being jerks, and they're not wrong ... it really should be simple)

Anyway my boss being the good guy he is offers to take it over, so I can do some things. Also sometimes it helps just to have someone else own something / not just look it over.

So a few days into this.... yup, emails coming in about things 'missing' or 'wrong' every day.

Like it sucks, but it's nice to see it suck for someone else too as validation.

So yesterday at a client location, our support guy called me and said this thing is trimming the characters whenever I save it. It was a ckeditor in our application, so basic troubleshooting was to check the system configuration for that page and the ckeditor configuration.
Checked the system configuration, ckeditor configuration, found nothing.
Out of curiosity, checked the schema for the table in which the data is stored, so one of the idiots took the backup of original table and appended it with the date time on which it was backed up. And created a new table with field data type of varchar with a 255 limit.
This was in UAT server as well as Production server. Changed the field type to text again in UAT. Asked to team to get the same thing done on Production server as well.

Ok, this question is related to mysql and php.,
Let me state the current situation
I have a db, say "gd".
That db has several tables all with same columns.(i.e same fields for different manufacturers such as product name, cost , stock etc.)
Now i want to know how many tables have the product 'a' in them and what's the cost of 'a' in each of those table.(tables are generated dynamically so I'll never now how many tables are there, well ofcourse i can refer information schema, but just wanted to highlight this fact). So is there a way to achieve this?.. excluding "dump the whole data and then search it" solution.
Plz help, .. sorry for my bad English, .

I'm working with an xml schema that effectively emulates xml... inside xml...
It looks like this
<child>
<tag>ABC</tag>
<value>DEF</value>
</child>
I don't event want to get started on how it handles child elements.
This is some next level abstraction hell.
And it's not like it can't use normal XML tags. In other parts of the project it uses a slightly more sane schema.

Am I in developer hell already? A shitty project is about to come to an end (hopefully), or should I rather say: It needs to come to an end. But I am still quite lost in how to deal with it, hence procrastinating on it - making the deadline come closer and with it the realization that I'll probably have to rewrite almost everything. I'm not sure how, but I do know that the current code is a dumpster fire.

Basically what I need to do is dealing with the APIs of different payment providers/gateways (like PayPal, AmazonPay). For most cases I'll get a payment ID from the shop and need to act on it later, e.g. capture the authorized money in the case of a credit card transaction or do refunds (without user interaction, unless there is an error). Now at first I put something together where I try to abstract the payment information into two tables:
orders{1}<->{0..n}payments
payments{1}<->{1..n}paymentDetails

Unfortunately trying to abstract the different payment methods and to squeeze them (and their different possible stati and functions) in these tables was not very successful, it's a total mess with magic numbers, half-broken behavior and without any consideration for partial payments/captures or unfinished requests (i.e. if there is an exception before the response is dealt with, there is no indication that anything has ever been sent). Also the current amount is calculated through the history of the paymentDetails table, which basically works differently for each payment type.

How to fix this mess in a way that I'll still have a job by next week?

I'm trying to improve the db schema first, as I think my biggest problems are lying there. Through some research I've come across a recommendation for making payment type specific subtables (with a magic number/string in the main table to prevent having to look up all subtables). That way I can record what I send and receive without having to abstract it too much, so I'll have an acceptable transaction log. The paymentDetails table can be removed (necessary fields go to the payments table). The payments table gets multiple fields for the amount (differentiating between open, authorized, captured, processing and refunded values) and always reflects the current status.

Tables:
payments
paymentRequestsPaypal
paymentRequestsAmazonpay
paymentRequestsXyz

I think I'm going in the right direction here. hm. Maybe there's some light at the end of this long, dark tunnel. Or a train. I'll have two days to find out.

Did it ever happen to you that you were needed to map a mental and overly complex for no reason db schema to a simple json, and you hate yourself more every key you press because rewriting it more efficiently and simply is never an option?
How do you cope with managers and legacy code?
If it works does not necessarily mean that we should keep it, jesus christ.

Random thought while upgrading schema and manually recreating 100+ rows table; when looking for a job, make sure you look for one where company focuses on employing brainpower, not manpower.. 🤦‍♂️ 🙄

God, these people...

Little backstory. I'm making an training application and we have a MySQL database set up where some elements of the training are configured. This is so learning experts can easily change some aspects of the training without programmer's help.

Meanwhile, I'm also in the middle of a server migration, because our current server is running a lot of deprecated software and is in dire need of replacement.
This is going pretty slowly, though, because of other, high-priority, work that keeps being shoved my way.

Now, someone accidentally deletes a bunch of data from one of the schemas. No big deal in my book, the training is still in development and we have nightly backups of the database.
So I shoot a support ticket to the hosting provider and ask them to restore a specific schema, telling them to restore the image to some other machine and dump the tables in an MySQL file so I can restore it that way.
I also told them to get the backup of the OLD server, not the NEW one we're still migrating to.

About an hour later, I get a message that they dumped the schema's files in a Temp folder on the D drive. So I RDP to the server to check and... The files aren't there. Just before writing a response asking where the file is, I remembered the server I was migrating to and checked that server, and there were the files.

I had already migrated part of our databases and was testing compatibility before I moved to something else.
The hosting provider just dumped the files of the wrong server, despite me telling them exactly which server to use.

This is not the first time this hosting provider has let me down...
I'm really considering jumping to another if they keep doing this...

!rant

Rant from my previous work as a consultant Data Engineer (wish I had known this site back then).

During my stay at the place, we have a big client whose contact with us was an incompetent stressful fellow.

I single-handedly build a humongous automated data pipeline using Airflow. I am very proud of my baby as my first massive project and check it obsessively for every possible flaw, especially when writing down documentation for the poor soul that would take my place.

Luckily for me, everything is working as intended, until of course on my last day of work, shit hits the fan, and everything breaks down.

After a moment of initial panic: it was Thursday morning, we had a Machine Learning model to run over the weekend, predictions to make and reports to write and a very lovely next week deadline, I calm down.

"I won't be dealing with this shit anymore, starting from 18:00 PM and anyway Fear Is The Mind Killer."

Quite sure that it couldn't have been my code, I start looking at various logs when the culprit was clear. The B(ig) S(tupid) C(lient) changed the whole schema of the data he was feeding to us.

I call him: he has no idea of what was done to the data. Hell, at first he doesn't seem to remember what the deal with schema, data, and SQL is (the guy was supposed to be a big shot in the IT department). It turns out he hired one of our competitors to do his side of the collection pipeline. He tries to get mad at me, but everything he throws bounces back to him. I am calm yet ruthless pointing out how every major hiccup had been his fault and that I could quickly reach to his board of directors explaining why their Machine Learning model was late.

Result: he apologizes, extends our deadline, and I get a round of applause from other juniors who would have to deal with me had I failed.

Never am I happier to not work as an underpaid cannon fodder apprentice in a shitty consultant firm.

Luckily for me, everything is working as intended, until of course on my last day of work, shit hits the fan, and everything breaks down.

After a moment of initial panic: it was Thursday morning, we had a Machine Learning model to run over the weekend, predictions to make and reports to write and a very lovely next week deadline, I calm down.

"I won't be dealing with this shit anymore, starting from 18:00 PM and anyway Fear Is The Mind Killer."

Quite sure that it couldn't have been my code, I start looking at various logs when the culprit was clear. The B(ig) S(tupid) C(lient) changed the whole schema of the data he was feeding to us.

I call him: he has no idea of what was done to the data. Hell, at first he doesn't seem to remember what the deal with schema, data, and SQL is (the guy was supposed to be a big shot in the IT department). It turns out he hired one of our competitors to do his side of the collection pipeline. He tries to get mad at me, but everything he throws bounces back to him. I am calm yet ruthless pointing out how every major hiccup had been his fault and that I could quickly reach to his board of directors explaining why their Machine Learning model was late.

Result: he apologizes, extends our deadline, and I get a round of applause from other juniors who would have to deal with me had I failed.

Never am I happier to not work as an underpaid cannon fodder apprentice in a shitty consultant firm.

So my boss wants this mssql reference table editor. Where you point it at references table containing mapping references and it will build a pretty gui for editing the table.... and they want it in like 3 days... so I now either hack something together which is so tightly coupled to the scheme of the table it will require redevelopment each time a new reference table needs adding to be edited or I don't deliver on time and give them a solution which will understand the schema and build the exit view dynamically... I'm starting to hate these stupid deadlines! And to top it all off they justify it with stuff like "it's just an edit view!?!?" Or recently on a basic form I created it was "why do you need to write c#? It's just a HTML form!?"

If i could estimate a schema issue. I looked at this site once in my time here. Don’t know shit about schema’s.
Estimating haha.

Im not asking you an estimate for example building a rocket. Right?

I don’t know this shit.

Creating logical schema and converting to physical. Are there better tools than embarcadero?

I work in a small team. As the senior dev I tens to focus on important tasks that shape the core of the product but some times I can’t divide my self when there are multiple tasks at hand, so I pass some tasks to the an other mid level dev.

So the task was to create an automation in order to CD (continuously deliver) an order from WHMCS of the (git versioned) product to customers UAT, PROD envs.

To get a background this is an old guy with “constricted” experience in PHP/jQuery/Joomla/Wordpress.

So when we were breaking up the tasks he told me he would like to implement this so i gave him the task as i was busy with core features.

I was like what could go wrong? I know he doesn’t know much about CI/CD but he can read right? He will google right? He will search for CI/CD solutions that do this out of the box right? He will design on paper or what ever and do small POCs right? He will design the flow first before starting the implementation right? RIGHT?

So fast forward to today I had a call with him this morning about some DB staff. And he wanted to show me his progress…

His solution is:
(parentheses is my brain)
1. Customer completes WHMCS order (perfect)
2. Web Hook 🪝 action (YES)
3. cpanel gets source and “automatic!” Init, all using pure PHP code ignoring the usage of the current framework (ok… something is missing)
4. cpanel web hooks(?) WHMCS to send email to customer with the envs initial setup page(?)
5. Customer opens link and adds setup info (ok fuck, fuck, fuck)

(Ok stay cool composed, lets ask some questions maybe he thought it all in a cool way I can’t get my mind around)

Me: So how are you gonna get the correct version from the repo to the env and init the correct schema?

Dev: I haven’t thought about it yet.

Me: Are we gonna save each version to a file system then your code is going to fetch them?

Dev: I haven’t really thought about it we will see. But look on customer init user setup I implemented a password strength validation and it also checks if the password is the same.

So after this Pokémon encounter I politely closed teams. Stood up drank some (a lot) coffee ☕️. Put out the washed laundry while reflecting on life’s good things, while listening to classical music 🎼 .

Then I sat on my office chair drank some more coffee, put some linking park starting with in that order:
“Numb” then “What I’ve Done” and ended with “In the end, it does really fucking matter”

Grr.
I have been struggling with schema compare and editing anything SQL related in Visual Studio for a couple of weeks now. Cursor constantly flickering, can't click on anything, and actually running the compare results in Visual Studio disappearing in puff of smoke.
I'd put it down to an update, but then this week's update didn't fix it.
Turns out it is a "feature" that Microsoft has been refusing to fix for over a year now, where none of the SQL stuff in VS works properly on a 4k external monitor. And of course I only started trying to use that stuff on my 4k monitor since I have been working from home.
Dragging VS across to the laptop screen solves the problem (and is the Microsoft fix), but I think I now need to visit the optician to get new glasses to see WTF is going on.

Leave it to an investing company 'dUe DiLigAnCe' document to list the following requirement:

"Schema of computing infrastructure setups for development, testing, and production"

Ah yes, the highly technical and well-known term of "schema of computing infrastructure"

God I hate business people, so clueless

BRB going to start my own business and make real money. if these neanderthals are top investors, i can be too

I recently built an automated payout functionality for bank-to-bank transfers, and we initially looked at using the pain.001 XML schema to do it. Luckily, we ended up finding a service that has a simple REST API to do this instead. (Thank god we didn't go with the XML method, I know how much of a headache that could become, I can imagine the treasure trove of memes with naming an XML schema with the name PAIN)

Anyway, for one of our big-brained product managers, this will forever be the infamous "XML Task" that he continues to ask about and bring up. I've already clarified a few times that we have long chosen a solution process that has nothing to do with XML, but to no avail, it will forever be his "XML Task". Wonder what name he'll pick next time we need XML in a solution? "Second XML Task?" Let's just keep the mental overhead idiot train going!

I spent last week trying to figure out estimates about how much time we'll spend in this quarter's project - like, 3 days on the schema, 4 days on the API, 5 days on the GUI etc. Why does management think that each part of the project is separate? Oh I forgot, they're management. I wonder how much work I could've done if management wasn't so obsessed in knowing metadata about the project instead of doing the project itself 🤔

Implement a rest API for elasticsearch.
Follow the client's index's mapping.
Generate json document from Java pojos, given by the client.
Jsons don't match the schema mapping, one (at least) field, for geographic coordinates, is in another format.
Ask the client for explanation.
Client response, after 6 hours:
"We build it in this shape so you have to convert them to another format before posting into ES".
What the hell is wrong with you?!

Writing DAL code that I am 99% sure will be replaced really soon along with the database schema.
Angry coding is almost as good as tipsy coding - AMAZING.

When I was doing my onboarding training for work, we had to do a group exercise. We had to build a small app using Spring MVC connecting to a MySQL database.
We had a team of 4 people, and I think I was the only person who wrote a single line of Java the whole day.
One person decided that she would build the DB schema, so I thought ok fair enough I will make a start on hooking up Spring. But the other 2 decided that they would “focus on making it look pretty”.
Several hours later what they had basically managed to do was import Bootstrap.
We ended up with only one screen to demo while other groups had 3-4.
Thats not the only story I have where Im in a group project and basically end up writing all the code. I’ll post the other one later.

Finally got my multi-page dialog with stepper to have validation on every field based on a schema in a way that doesn’t totally mess up styling and that still collects information from fields from both pages to send the Graphql mutation.

So happy I could cry.

I dont get it. Please give me one good reason to use mongoose with a mongoDB.
Once upon a time it might have made sense to use a schema for the db. Today the native driver supports schemas and can check them on inserting. Nevermind one should validate the data before its hitting the db. I listened to an 1hr podcast last week where one of the maintainers tried to give reasons why its might be a good idea to use mogoose, and he failed miserably.
It introduces dependencies that are useless, it doesnt really abstract anything useful from the native driver, its TS support is shit and I dont like the API.
Every time I see someone use it he either fails or doesnt explain at all why to use it. Its so redundant it makes me angry. We have enough abstraction already. We really dont need more code that doesnt provide value. Please just use mongo the way the people of mongoDB intended it to be used.

When RoundhousE throws an error saying specified schema doesn't exist and after 2 hours debugging realise it's actually an ambiguous foreign key name. What the fuck?

New Project

M: Hey, check these two processes. Both took different paths for the same input. Here are the logs. Both are the same though.
Me: Ok... do we have a debugger?
M: No this product doesn't have a debugger
Me: Any unit tests i should know of?
M: We don't do unit testing. Everything is done in Integration Testing.
Me: Ok. So how can i check the db for this?
M: You can't, the access is restricted. You'll have to raise a ticket to other team with the sql output you need.
Me: Ok. So I hope you have the schema at least.
M: Yes we have the schema. But there was some issue last week so the values might not be there in the correct column. They may or may not be present where they are supposed to be.

Wtf am i supposed to do... fucking play football on ticketing system with the other team 😐

# ./symfony test:unit

Propel-Exception: Unable to execute DELTE ALL statement [...] Integrity constraint violation: 1451 Cannot delete or update a parent row: a foreign key constraint fails.

WHY ist a UNIT TEST reaching out to a REAL data base?
And who in their right mind would create a different data base schema for the tests?
This was with a clone of the real thing. Removing the FK results in double PK-errors...

Why does Microsoft make entity migration default and every tutorial show them? I have other ways of migrating my db schema changed. Newbies come to me with this cluster already in motion and I have to help straighten it out. Then the newbie says "why do all the tutorials show it". I say because the people making the tutorials are trying to quickly create content for a tutorial.

I was wondering how I can go about with the development of an Android app.
I've made a few small apps, but I want to make something a little more complex.

I've made the use case diagrams, high-fidelity prototypes and the database schema is almost ready.

I'm just stuck with how to model the app. Which architecture to use? How to actually implement a complex project?

Okay, I'm learning XSD (I wish i didn't, but yeah)

Why the fuck does it require so much code to use the simplest regex?

<xsd:simpleType>
<xsd:restriction base="xsd:integer">
<xsd:pattern value="([0-9])*" />
</xsd:restriction>
</xsd:simpleType>

wasting time trying to write good code and minimize duplication , but it probably won't work and there are no working examples

duplicate code and do the stupid thing, which will work, with extra dumbass boilerplate code to convert between swagger2 & 3 schema

fuck you swagger code gen

We ended up finding ourselves with a bunch of tables that have mostly the same columns, but differ by a few. Every time we consume a REST API, we store the `access_token`s and expiration dates and the other OAuth data. However, each provider has slightly different requirements. For example, we store email addresses for email api's, other providers require us to store some additional information, etc. etc.. I'm tempted by the flexibility and lack of schema brought by document databases, but not enough to use one since they're generally slower and we already have everything in SQL. So I got the idea of using JSON columns to alleviate this issue: have a single table for all REST integrations (be it outlook or facebook), and then store the unique integration data inside of this JSON column for "additional data". This data is mostly just read, not filtered by (but ocasionally so). Has anyone had experience with this? How's the performance of JSON fields? Is this a good practice or will it get harder with more integrations?

Does anyone know of an API/JSON schema designer for iPhone?

Hey we need to implement a schema to the entire website...

Me: NOOOOOOOOOOOO
*starts crying*

Why does rails have validation at the model layer? Isn't that what the table schema/migration is for?

A genuine question

How can one achieve the following with ld+json schema, it really looks cool or is it done with something else?

Hear me out:

Since keras and tf are pretty much schema design rn what if someone made a no-code solution where you drag and drop layers and tweak things in a UI so those data scientists can design it in a UI instead of writing shitty code?

Follow up sorta...

So I got pulled into a support issue on a day off. Some system was facing timeouts on our servers so had to investigate.

Over the weekend as part of the release, I released the ELK stack I built and today I used that to help.

Pretty much immediately pinpointed which machine was hanging though still had to investigate and confirm so split between KQL and checking the server logs.

One thing I've always griped about is how no one created schema docs for it mongo collections so can't easily figure out what they do or your to get the document needed.

Well guess it's my turn.... Because only I know the schema :)

What if, the newly added JSON datatype in mysql is a way to provide mysql with no-sql-like capabilities.
I mean, some would prefer no-sql cuz they beleive that the tables schema will evolve a lot.
An extra column in mysql table with json datatype called "custom_fields" would do the trick.
What do u think ?

WTF?
W_T_F?
W-T-F?
W$T$F?

how the fuck do you brand codebases?

say you've got a brand called dicks,

and you've got a website, api server, some SDK's, and whatever else. How do you format the codebase names so it's A, usable as a variable name, B clickable at the variable part of the name, C, that's fucking it

dicks-website can't be used in js because

dicks-website = require('dicks-website') won't work

dicks_website can't be used in url's and in general because double clicking any part will select the whole word, so you can't replace dicks_X easily.

dicks$api seems to work, dicks and api are seperately clickable, and usable as a variable in javascript. I already know that the $ sign probably fucks up many many systems so that's probably a no go

has this question been addressed already?

Creating some sort of relational schema in mongodb is dumb or am i overthinking?

The worst tech I've been working on is not related to a programming language, is more about the codebase itself.
One of them was in .net, the guy reinvented the wheel creating a custom mvc framework and a custom entity framework, copying from cakephp models, was disgusting and felt terribly wrong to work with.
Then I moved to an old cms written in php on top of an old version of cakephp, that was a nightmare too. Fat controllers and a disgusting db schema, no coding standards whatsoever. Everything so deeply convoluted and connected that was impossible to change something without breaking something else.
The technology itself is never the worst thing, people who thinks they are the best ninja developers, are the real problem imho, and the code they leave behind speaks for them. Yuck

I can't tell if I'm being clever or a dick here. When I can't be arsed writing a dB schema I switch hibernate to create mode, let it build the database, I then just dump the schema as SQL...

So, I am in the last stages of development of a really big project and I need to figure out a way to package future patches and updates for the client in order for them to manually update the project on prod server.

For reasons I cannot specify here, they will not use any automated process, and we need to provide regular patches and updates for the next year.

So I was thinking of using git archive to package changed files from our repo for every new commit, or series of commits, and just give them that, along with any database schema updates as sql files (again, no automation can be used).

We are talking about a large PHP + MySQL app, and cannot use automated deployment strategies.

I feel there must be a better way to do this, but this is the best I could come up with so far.

What do you people think?

Any ideeas?

I am trying to extract data from the PubSub subscription and finally, once the data is extracted I want to do some transformation. Currently, it's in bytes format. I have tried multiple ways to extract the data in JSON format using custom schema it fails with an error

TypeError: __main__.MySchema() argument after ** must be a mapping, not str [while running 'Map to MySchema']

**readPubSub.py**

import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions
import json
import typing

class MySchema(typing.NamedTuple):
user_id:str
event_ts:str
create_ts:str
event_id:str
ifa:str
ifv:str
country:str
chip_balance:str
game:str
user_group:str
user_condition:str
device_type:str
device_model:str
user_name:str
fb_connect:bool
is_active_event:bool
event_payload:str

TOPIC_PATH = "projects/nectar-259905/topics/events"

def run(pubsub_topic):
options = PipelineOptions(
streaming=True
)
runner = 'DirectRunner'

print("I reached before pipeline")

with beam.Pipeline(runner, options=options) as pipeline:
message=(
pipeline
| "Read from Pub/Sub topic" >> beam.io.ReadFromPubSub(subscription='projects/triple-nectar-259905/subscriptions/bq_subscribe')#.with_output_types(bytes)
| 'UTF-8 bytes to string' >> beam.Map(lambda msg: msg.decode('utf-8'))
| 'Map to MySchema' >> beam.Map(lambda msg: MySchema(**msg)).with_output_types(MySchema)
| "Writing to console" >> beam.Map(print))

print("I reached after pipeline")
result = message.run()
result.wait_until_finish()

run(TOPIC_PATH)

If I use it directly below

message=(
pipeline
| "Read from Pub/Sub topic" >> beam.io.ReadFromPubSub(subscription='projects/triple-nectar-259905/subscriptions/bq_subscribe')#.with_output_types(bytes)
| 'UTF-8 bytes to string' >> beam.Map(lambda msg: msg.decode('utf-8'))
| "Writing to console" >> beam.Map(print))

I get output as

{
'user_id': '102105290400258488',
'event_ts': '2021-05-29 20:42:52.283 UTC',
'event_id': 'Game_Request_Declined',
'ifa': '6090a6c7-4422-49b5-8757-ccfdbad',
'ifv': '3fc6eb8b4d0cf096c47e2252f41',
'country': 'US',
'chip_balance': '9140',
'game': 'gru',
'user_group': '[1, 36, 529702]',
'user_condition': '[1, 36]',
'device_type': 'phone',
'device_model': 'TCL 5007Z',
'user_name': 'Minnie',
'fb_connect': True,
'event_payload': '{"competition_type":"normal","game_started_from":"result_flow_rematch","variant":"target"}',
'is_active_event': True
}

{
'user_id': '102105290400258488',
'event_ts': '2021-05-29 20:54:38.297 UTC',
'event_id': 'Decline_Game_Request',
'ifa': '6090a6c7-4422-49b5-8757-ccfdbad',
'ifv': '3fc6eb8b4d0cf096c47e2252f41',
'country': 'US',
'chip_balance': '9905',
'game': 'gru',
'user_group': '[1, 36, 529702]',
'user_condition': '[1, 36]',
'device_type': 'phone',
'device_model': 'TCL 5007Z',
'user_name': 'Minnie',
'fb_connect': True,
'event_payload': '{"competition_type":"normal","game_started_from":"result_flow_rematch","variant":"target"}',
'is_active_event': True
}

Please let me know if I m doing something wrong while parsing the data to JSON. Also, I am looking for examples to do data masking and run some SQL within Apache Beam