9

Guys, what the fuck.

Today i was doing some consistancy checks accross the board after update made for one of our core systems that manages money. Yeah, real, live money.

I have hidden from public payment processor with simple API etc. So one of my checks, gate has same balances as gate's internal account on core blinked red. Okay well, fuck, thats really really shitty situation to be in. I guess my gate is fucked up some way.

Okay, debug mode on, maintainence mode on, quick look at DB, oh shit, client payed 4 times 15k eur without any txn on core system... SHIT! postman... Fuck, postman ofc wont start, quick google, fixing postman, tention in me grows, becouse its really rough and tough fuckup on my side, and got call. That moment when you know someone already knows is for me apogeum of stress that just skyrocketed from calm morning to mad morning.. Okay, i pick up phone, and I hear that one client payed (using core system app) and got strange message, YES I KNOW, im working on it.. Wait, you say that core system gave them odd message??? I will check it out. Finally fixed postman, 3 requests and I know its bug on core system.

Why, why in the motherfucking blody world anyone would push critically bugged update to system that just sends api callbacks "yes, he payed" when someone didnt pay...

Fuck im stressed and pissed, but at same time reliefed its not my personal fuckup (yeah, I solo wrote that gate, but externally audited code and all they had to say that some cosmetic linting should be done)

Comments
  • 1
    @-red

    Luckly client is one of these who I personally don't describe "hostile" but this caused a lotta mess all over the place.

    Was stressfull as hell for a second (Im one of these who try-hard to manual test own code just to be FOR FU**NG SURE it's OK and everything at first glance looked like it's my fuckup, also that could be loses I might end up responsible for sooo yeeee....)

    If there is any takeaway from this, tests, tests, tests, manual, automatic, edge, non-edge, all the f**ng tests. just do them, it's not that hard... ;-;
  • 0
    @-red

    We got officiall aplology from software house that made this fuckup. And as bonus they suggested very simple to implement yet quite robust improvement (done it in like 1h including testing)

    It all ended up just OKish and left bad taste about us in mounth of our quite important partner but other than that, everyone is fine, noone was fired

    Edit:

    Also, thx for pointing out typos. Some of them (most of them) are just typos becouse smartphone keyboard sucks for faster writing and i had to go for cig pause to rant about it, and sime of words I legit have problems with. So +1, man.
Add Comment