Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API

From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "emergency update"
-
Received "emergency update" code from internal enterprise security team. Wasn't given time to do code review; was assured code was reviewed and solid.
Pushed code to over 6k lower-level servers before finding this gem buried deep within:
...
cd /foo; rm -rf *; cd /
...
(This ran as root, and yes, the cwd was / from earlier in the code).
/foo, of course, did not exist on some servers.
Now, it is those servers which do not exist.
FMLundefined security root linux file not found directory structure rm -rf / directory not found fml rm15 -
If you are a salesperson, you can just go straight to hell. You're all a bunch of cocksucking twats and I'm amazed you manage to get yourselves dressed each day. You're a no good fucking waste of oxygen and you need to put your fork in a socket the next time you're eating.
I'm working on building a crm and ticket management system for use in the office to handle client passwords. Since I'm building from scratch I wanted to make sure I had properly planned my classes and functions before opening the code editor so I put a message on my door that says "Don't interrupt, thanks" followed by the date so people knew it was a fresh message and not something left from the previous day.
I'm deep in the zone, the psuedo code and logic is flowing, I'm getting classes planned and feeling really productive for an hour or so when suddenly my door flies open and in comes a sales person.
SP: "Hey, do you have any extra phones lying around? Mine's being slow and keeps hanging up on people."
Me: "Do you see the sign on my door right there at eye level which says not to bother me?"
SP: "oh, do you want me to come back later?"
Me: "You've already interrupted me now, let's go see what's going on before I spent an hour setting up a new phone for you." While we are walking across the office I asked him when the last time the phone rebooted.
SP: "idk, Salesperson#2 suggested that as I was headed over here but I figured I'd just ask you."
We get over to his desk and I see he has two phones sitting on his desk. "Where did this one come from?"
SP: "Oh that was on the desk over here but I figured I could use it."
Me: "Well aside from the fact that the phones are assigned to specific people for a reason, you took the time to unhook your phone to set this one up and you didn't think to reboot your phone first. Plug your phone back in."
He plugs the old phone, which is assigned to him, and while booting it does a quick firmware update and boots up fine. He tests a few things and decides it's all better now.
So someone suggested a fix for you and you decided, instead, you would break company IT policy by moving equipment from one station to another without notifying the IT department. You entered a room which had a closed door without knocking, and you disobeyed the sign on the actual door itself which politely requests that you go away. All because you couldn't be bothered to take 2 minutes and reboot your phone, which you had to do anyways.
You completely broke my train of thought and managed to waste 2 hours of effecient workflow because you had an emergency.9 -
I was on vacation when my employer’s new fiscal year started. My manager let me take vacation because it’s not like anything critical was going to happen. Well, joke was on us because we didn’t foresee the stupidity of others…
I had to update a few product codes in the website’s web config and deploy those changes. I was only going to be logged in for 30 minutes to complete that.
I get messaged by one of our database admins. He was doing testing and was unable to complete a payment on the website. That was strange. There was a change pushed by our offsite dev agency, but that was all frontend changes (just updating text) and wouldn’t affect payments.
We don’t want to enlist the dev agency for debugging work, especially when it’s not likely that it’s a code issue. But I was on vacation and I couldn’t stay online past the time I had budgeted for. So my employer enlists the dev agency for help. It’s going to be costly because the agency is in Lithuania, it was past their business hours, and it was emergency support.
Dev agency looks at error logs. There are Apple Pay errors, but that doesn’t explain why non Apple Pay transactions aren’t going through. They roll back my deployment and theirs, but no change. They tell my employer to contact our payment processor.
My manager and the Product Manager contact Payroll, who is the stakeholder for our payment gateways. Payroll contacts our payment gateway and finds out a service called Decision Manager was recently configured for our account. Decision Manager was declining all payments. Payroll was not the person who had Decision Manager installed and our account using this service was news to her.
Payroll works with our payment processor to get payments working again. The damage is pretty severe. Online payments were down for at least 12 hours. Our call center had logged reports from customers the night before.
At our post mortem, we had to find out who ok’d Decision Manager without telling anyone. Luckily, it was quick work. The first stakeholder up was for the Fundraising Dept. She said it wasn’t her or anyone on her team. Our VP of Analytics broke it to her that our payment processor gave us the name of the person who ok’d Decision Manager and it was someone on the Fundraising team. Fundraising then starts backtracking and says that oh yes she knew about it but transactions were still working after the Decision Manager had been configured. WTAF.
Everyone is dumbfounded by this. How could you make a big change to our payment processor and not tell anyone? How did our payment processor allow you to make this change when you’re not the account admin (you’re just a user)?
Our company head had to give an awkward speech about communication and how it’s important. The web team can’t figure out issues if you don’t tell us what you did. The company head was pissed because it was a shitty way to start off the new fiscal year. Our bill for the dev agency must have been over $1000 for debugging work that wasn’t helpful.
Amazingly, no one was fired.4 -
(Part 2/2?)
THE RAT-RACE ARC:
I get a mail 2 months into this fiasco telling me to register on their website and take up another test. I was already over with my emergency and was working my full-time default. (Fortunately I found another internship during this time which was one of the best initiatives I've worked with).
It asks me to register as a new user, take up the test and "share" my results. Not pushing it on insta/fb but legitimately share my test results link to my friends manually like a referral code. The more shares the more marks I'll get in the test. Why the test you ask. Of course to sign you up for the same Whatsapp trickery bullshit.
Luckily these nutcases didn't know they could be bypassed. I simply opened the link in incognito and logged in with my own account and that counted as a point. So I automated that shit.
Surprise surprise. The same fucking "Hello everyone" message into my mail. To my surprise I was relatively lucky to get ghosted after my attempt. This story is quite depressing in general cases. You're supposed to do this assignment shit for 2 months and then they ask for 2000 INR for a training period, past which you are paid between 1000/- and 7000/-. Though I didn't get the chance but I'm willing to bet you get 1000/- per month in a 2-MONTH INTERNSHIP. WTF.
You also have the other option of ranking first in their 3 consecutive competition that they hold. The theme is again to create chunks of their actual outsourced work.
WHY NOW:
The reason why this rant sparked is because I recently received an email with my results of the aptitude exam that I first took before the Whatsapp fiasco. I imagine they just pushed out a new update to their test thingy and forgot to set it's limit.
THE CORRECTION ARC:
I pushed this message to Internshala. They were kind enough to remove them from their website. I also shot down their Angel and Indeed listings. I sent a strongly worded email counting their con-artist operations and how I've alerted authorities (obviously a bluff but I was enjoying it). They most probably are not affected by this though. They might still be continuing their operations on their website.
I'm sharing the story here with the moral of:
Don't do jackshit if they're not compensating you for it
Always check for reviews before you start working at a place.
Be cautious of bulk messages (and the infamous HEY GUYS!! opening)
Don't do anything outside your work specification at least while doing an assignment.
You're free to question and inquire respectfully about the proceedings.
If you're good at your job you'll get good working place. No need to crush yourself with an oppressive job due to external restrictions.
And if you manage a company, please don't take advantage of helplessness.
There's no good ending to this tale as I have not received a follow-up. Though I want to see scumbags of their calibre shot down without remorse.
Good bye and thank you for listening.2 -
Update on this rant: https://devrant.com/rants/1322051/...
The server only boots into emergency mode now, and all data is lost.
Moral of the story: Its NOT okay to trash the home partition xD -
Windows 10 updating, decides it would be cool to install gigabytes of sdk, edge, and other bloatware without asking first, on a metered connection i use for work.
Guys, between you there and those fanboy demons in cupertino, one wants to just shut it all off and return to monke.
Sidenote this, because all of this nonsense started on that crap called Windows 8, which was in the end caused to copy that Unholy crap (sold as gold) that is Apple's range of products. It's a company that sells designs nowadays, like Prada, to say, Jobs era is long gone. Everything related to Apple, Mac, Safari, Development, Gaming, UI/UX, productivity and whatever is a
f***ing Nightmare.
We alreay have a global plague, and Apple exists, we dont' need you too making another catastrophe.
All this said,
Use your goddamn trillions to create your own customizable environment that is stable, fast, and WITHOUT BULLSHIT.
I don't give a mindflying F**k of the blurs, i know how to place them with a shell, if i need those. I want control, the shit i decide is going to happen, to happen fast.
This is of Critical importance, because it defines my productivity. And considering we're all sealed indoors since 2019, i want to get away from my pc asap and live my life, instead of spending time(and money, in this moment of emergency) fixing your F**kfests, or else seeing my pc slowing down to death.
First: IF i want stuff on my pc, I know how to install something, thank you.
Second: You can take it, all your Useless - Bugged as Hell - Nonsensical - and of no practical use Bloatware, and shove it deep in your Backdoor.
I'll debloat my pc with batches again, and there's nothing you can do to stop me doing that at every update you force me into.
So please, stop wasting my time, and yours.4 -
🚨 EMERGENCY ALTRANT UPDATE 🚨
Release Notes:
- Fixed critical UI hangs when scrolling up a rant's comments on slow networks
- Fixed critical UI hangs when loading the profile screen on slow networks
Today, I discovered that there is a huge issue with UI responsiveness when the device is connected to a slow (or subpar) network connection. I deemed this absolutely unacceptable and not in the standard I strive to achieve and scrambled to make a fix. The fix is now *live* and available.
In a week from now, I will expire the update I released yesterday (build 2070) in favor of this new one (build 2084). The schedule for expiring the build before yesterday's update (build 1607) is still scheduled to be expired on Wednesday, 11/23/2022, 6 days from the upload of this post.8 -
A "safe" Ubuntu update decided to fuck with my AMD drivers and after rebooting, a nostalgic black emergency terminal greeted me with some cryptic message containing hex code as if any mortal user could make sense of it.
To add insult to injury, local mirrors don't have 18.04 which makes apt vomit errors during software installation.
How the hell does the most well known distro out there manage to have problems like that?6 -
I am the responsible for the atlassian Suite at work, as I maintain the systems, set them up, and stuff.
One day, our crowd (the authentication and authorization application) just went crazy. At like lunch time it could not connect to the AD anymore. No reasons. Throwing XSRF errors (cross site scripting), because http would connect to https. "won't do it, fuck you" it told me. Out of the blue. Noone changed anything. And yea, seriously. Noone did.
It just refused to connect (as connecting to AD is connecting yourself with you own api. And refusing yourself talking to yourself). It runs behind a proxy. Therefore http/https. Well, this worked for years. But out of sudden not anymore.
Yea. Fuck you.
It was reported some hours later, at like 3pm, as people could not login to the applications using crowd as authentication and authorization server.
Tried to debug the system, where nothing was did, to make it work. At best time to fail.
First workaround: if you are logged into one of the other applications of atlassian, just refresh the site, so your SSO token gets a refresh and you are signed on again.
Then I searched more and more. And more.
But nothing worked, nothing helped.
So I addressed an emergency maintenance, take down the whole Suite, restart crowd, to apply some changes to it's settings, not knowing what happening then, because all connections of SSO will then be released. Sent out the mail like 30 minutes beforehands.
While waiting for the window, I just typed my credentials... And redid, and redid, so to type and being bored.
Three minutes before the window...
It just worked again.
Well. Wtf. Serioudl
Just came back.
No Intrusion, no changes at all. Just came back, as nothing has happened.
Kind of best part of this story... A headhunter messaged me on my way home to offer me a job as an Atlassian Suite SysAdmin for a company, at kinda the double of my salary.
At first I was thinking to go there, and when someone then asked me sth about Atlassian just start to laugh and then leave still laughing...
But then I very nicely respond that I dont want to cry at work. And wished him best luck.
I am doing some bad upgrades now on our Suite. Very painful.
And I looked into the start scripts. Some Look like the untalented intern tells another one to write scripts. Seriously wtf.
Today I followed the guide to Update a confluence and change database to Postgres. Didnt work, Postgres error.
Try it again, jquery won't load. Next try, tomcat not starting anymore. Did same thing. Every fucking time.
Yea. Maintenance window to get a nice new export soon. Will only take an hour.
To switch database in confluence, you need to set it up very fresh. And then Import your export.
Export takes an hour at our system.
Importing maybe the same time. Hope it will work (hint: Nope).
Oh, can be nice also. Just tell the Bitbucket to migrate databases, there is a fucking setting for it. Enter new database, ready, go, finished.
At least they don't raise costs very much every kinda year.
Oh sorry, yes, they do.4 -
I hate when clients think they're always right.
Yesterday this client showed up saying that he had an emergency on computer, because after an update all icons turned white. When we told he the situation and that all data was lost, he begged to retrieve all the info because he needs it to work.
He was affected by ransomware, something like Fantom, and not only all the files are encrypted but also the backups.9 -
I don't often have reasons to rant, but today is the one.
We had a deadline to finish a project, because today people are being trained on it. I've been working my ass off on it for a year now.
I "finished" about 2 weeks ago, meaning QA could start for real 2 weeks ago. As you can imagine for a project this long, there was bugs. Lots of them.
We did our best to fix most of them, or find work-arounds we could use during the demo.
Let's just say it isn't going great so far. We have several known bugs, which at some point may crash the app, a very low confidence in the fact that it's going to work well.
Oh and obviously the client is one who already use heavily the solution. Today we figured we never tested on a device with 0% disk space. Files are cut partway because of that, and obviously things crash.
I have a feeling there will be yelling sometime soon.
Right now I'm enjoying the calm before the storm, with coffee in hand.
Why do people still continue to promise dates to clients, after me telling them for 5 years not to do that?
We are a 2 devs team, with 11 apps on 2 platforms, 2 back-ends (one is legacy) and obviously our marketing site, which doubles up as e-commerce. We just can't promise anything, because any emergency reduce our development bandwith for new features either to 50% or 0%. There are so much known bugs it's not funny anymore, and we don't even have time to solve those.
To add insult to injury, at the beginning of the month, the SaaS provider for our legacy back-end (which have not been maintained for 2 years now) decided we had to update to PHP7.1 before 1st October. If we don't do anything, on monday this thing is broken. I hate that thing, and I hate having to maintain it even though I was promised I wouldn't have to ever have anything to do on it.
Monday will be "fun"...2 -
Only last month I removed a file called 'statuspage.aspx', this file has been sat there for years on our customer sites.
The file did one simple query on the database to ensure connectivity, this query dumped the admin username and password to the page, no encryption.
Needless to say we rolled out ab emergency update... Not quite sure how that made it through QA! -
My life at the moment is summed up quite well by https://xkcd.com/1586/ . My phone completely died and wouldn't boot in the first week of my five week holiday. When I get home and try to restore it, the computer with the backup fails to boot. I go and try a different computer, wait an hour or so for iTunes to update and Windows decides to update too. I try to update the phone so that it keeps it's data but no, it needs a full reset. I lose all the stuff on there: contacts, photos, texts, 2FA codes. Let's hope I have the emergency codes somewhere. I can't install gmail or google drive because I don't have ios 10 because it doesn't work on an iPhone 4S. The only advantage of this is that it removed obsolete apps that I never used. Which device will fail me next!1
-
Got an emergency update to my iOS app blocked because the iPad screenshot wasn't pretty enough. While I admit the iPad screenshot was crap, it has been there for over 7 months with many updates not being held back until this update