Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
Me : So how's the deadlines here?
Coworker : There are none.
Me : ?
Coworker : if they are unrealistic, we push non-working code. Prod comes up with bugs, and we get a new sprint to resolve those bugs.
Me: ╰[ ⁰﹏⁰ ]╯10 -
She: I like people who takes risks.
He : I develop in a prod server.
She: My place or you place? 😉3 -
Fucked up an sql join once and accidentally deleted myself from the employee table in the prod database. So I kinda fired myself... Good thing we had backups. 😂2
-
Why is this CSS not updating???
Ctrl + F5
Still!!!
Let's check in incognito..
F*CK, really!!!!
.... Ooooooh right I'm checking prod not dev.9 -
Expectation: arrive at work, everyone in awe of devrant swag, social status++
Reality: arrive at work, prod failing, get blamed, employment status--
7 -
Hi client,
I am not able to login to your prod server. Can you please verify the following:
Host: x.y.x.y
Port: 1234
Username: ABCD
Password: password1234
Thanks,
My idiotic coworker8 -
my team: "lets get rid of materialize it doesnt work too well w/ react"
my team: *deletes materialize*
my team: *accidentally pushes to prod*
me: "guys why are we back in 2003"
19 -
Jingle bells,
Batman smells,
Don't push to prod today!
But if you do,
Make sure you
Test it all the way!
Happy holidays, y'all!4 -
Someone called me an incapable, arrogant bitch today because I didn't let them test in prod. We had words.
The monologue from Full Metal Jacket went through my head at one point, and I almost didn't mute my phone in time before I said it rather than just thinking it. I'm not sure I would still have a job if I did, but I kind of wish I didn't mute my phone.13 -
🎉 As of today, I can proudly count myself among the members of the "Killed Prod on A Friday Afternoon" club. 🎉16
-
5th June is World "Environment" Day..
Reaction of Software Engineer -
Which one? Dev, Test or Prod?
😂😂2 -
Don’t you just love it when you do the following in a prod database 😅
UPDATE table SET someImportantField= NULL; WHERE abc = 123;
If I don’t laugh right now I’m going to cry.5 -
Got some customers not paying their bills?
Play russian roulette with these projects on the prod. server:
# [ $[ $RANDOM % 6 ] == 0 ] && rm -rf /<path_to_project> || echo "*click*"9 -
Devs: We need access to PROD DB in order to provide support you're asking us for.
Mgmt: No, we cannot trust you with PROD DB accesses. That DB contains live data and is too sensitive for you to fuck things up
Mgmt: We'll only grant PROD DB access to DBAs and app support guys
Mgmt: <hire newbies to app support>
App_supp: `update USER set invoice_directory = 54376; commit;`
----------------
I have nothing left to say....7 -
My coworker complained about an way to big Angular build (20 MB in prod!).
We looked over his code.
... Well deleting that 200 unused base64 encoded images shrinked it down to 500kB
DELETE the Code you replace14 -
Writing customer passwords fulltext into the prod database because "it's easier to associate them with the user"5
-
Root: Fleshes out missing data in some factories. Tests affected code and finds the change breaks some specs (but shouldn’t).
Root: Reaches out to spec author.
Root: Messages thundercunt (the ticket’s code reviewer) on slack about the specs and the reaching out. No response.
Root: Works on another ticket while blocked.
Root: Logs off.
Root: Talks with spec author chick in the morning. Decide to pair on specs later.
TC: Still no slack response.
Root: Gives update in standup. Mentions factories and broken specs. Mentions pairing with spec chick.
TC: Still no slack response.
Root: Pulled off tickets in favor of prod issue. Gets ignored by everyone else diagnosing prod issue. Investigates prod issue by herself. Discovers prod issue isn’t from bad code, but bad requirements — code works as requested. Communicates this with details. Gets ignored by people still diagnosing prod issue. Tries again. Gets ignored. Gives up. Works on non-blocked tickets instead.
TC: Still no slack response.
Hours later:
TC: Comments on PR telling me I broke specs (how did I not notice?), that I need to reach out to spec chick and work with her, and that I can’t resolve the ticket until it’s fixed and passes code review.
TC: Still no slack response. (21 hours later at this point)
TC: Logs off. Still no response (25 hours at this point)
———
Ignoring the prod issue for the moment…
I broke specs. No shit.
I need to talk with spec chick. No shit.
I can’t resolve the ticket. No shit!
Bitch, I told you all of this 21 fucking hours prior, and again 3 hours prior during standup. But no, I clearly “don’t communicate” and obviously have no bloody clue what I’m doing, either, so I need everything spelled out for me.
And no, I didn’t resolve the fucking ticket. Why the fuck would I if it still has pending changes? Do you even check? Ugh!
And what the fuck with that prod issue? I’m literally giving you the answer. fucking listen! Stupid cunts.
Why is it all of the women I work with are useless or freaking awful people? Don’t get me wrong, many of the men are, too, but I swear it’s every single one of the women. (Am I awful, too?)
Just. Ugh.
I can’t wait to leave this sewer of a company.
Oddly still a good day, though. Probably because I talked to recruiters and sent out my resume again.15 -
Hello! I am Johnny Knoxville and tonight on Jackass...we test in prod with no back out plan in place...and then I will glue Bam's hand to his dick while he's sleeping.6
-
MOTHER FUCKER IDIOTS!!!
SO I HAVE TO ROLLBACK OUR PROD DEPLOYMENT BECAUSE IM THE ONLY FUCKER WHO UNDERSTANDS THE SIMPLE CONCEPT OF "DONT PUSH UNFINISHED CHANGES"?!?
DAMN!!!! FUCK YOU ALL...10 -
Colleagues sharing passwords.That was a big fat NO when I was a sysadmin - and for a good reason. But now, since I'm closer to development, it feels like no one really cares about the passwords. If I tell my colleague I'll take 10 minutes more because I can't log in, he OFFERS me his credentials. And sends them over saying "in case you need it". [the next day the same colleague was complaining his account is locked out. Oh, wonders! How on Earth...!]
But seriously, password sharing is a serious problem. I would fire the person on spot if I caught him sharing his credentials! This is the 8th deadly sin! IDC if they are for non-prod. Most people reuse their passwords in multiple systems, and even non-prod envs can bring the prod down! Or worse - install a trojan.15 -
Kudos to the devrant team for epic response time!
<50min from bug report on Twitter to workaround on prod!2 -
PM (on slack): "we’re about to deploy to production".
Me: "ok"
… I keep on working on a task / remain available for any post deployment issues …
PM (5 minutes later on slack): "deployment broke production! We need to handle this NOW!"
My dev colleague has already called it a day, but I’m still online
Me: "ok I don’t have access to prod, can you describe what’s going on? I can’t reproduce on any other environment"
PM: …
10 minutes go by
Me: "anybody there?"
PM: …
45 minutes later, I realize PM is offline
The following day:
PM: "ok we got prod running again" (turns out it was client’s fault for not updating a config we as devs can’t access)
PM: "but we’re REALLY UPSET! You guys need to be available to intervene for any issues following deployment to production! At least one of you should be available!"
Me: "but, but…" 🫠14 -
The riskiest dev choice...
How about "The riskiest thing you've done as a dev"? I have a great entry for that. and I suppose it was my choice to build the feature afterall.
I was working on an instance of a small MMO at a game company I worked for. The MMO boasted multiple servers, each of them a vastly different take on the base game. We could use, extend, or outright replace anything we wanted to, leading to everything from Zelda to pokemon to an RP haven to a top-down futuristic counterstrike. The server in this particular instance was a fantasy RPG, and I was building it a new leveling and experience system with most of the trimmings. (Talents, feats/perks, etc. were in a future update.)
A bit of background, first: the game's dev setup did not have the now-standard dev/staging/prod servers; everything ran on prod, devs worked on prod, players connected and played on prod, etc. Worse yet, there was no backup system implemented -- or not really. The CTO was really the only person with sufficient access. The techy CEO did as well, but he rarely dealt with anything technical except server hardware, occasionally. And usually just to troll/punish us devs (as in "Oops ! I pulled the cat5 ! ;)"). Neither of them were the most reliable of people, either. The CTO would occasionally remote in and make backups of each server -- we assumed whenever he happened to think of it -- and would also occasionally do it when asked, but it could take him a week, sometimes even up to a month to get around to it. So the backups were only really useful for retreiving lost code and assets, not so much for player data.
The lack of reliable backups and the lack of proper testing grounds (among the plethora of other issues at the company) made for an absolutely terrible dev setup, but that's just how it was, and that's what we dealt with. We were game devs, afterall. Terrible or not, we got to make games! What more could you ask for!? It was amazing and terrible and wonderful and the worst thing ever, all at the same time. (and no, I'm not sharing the company name, but it isn't EA or Nexon, surprisingly 😅)
Anyway, back to the story! My new leveling system also needed to migrate players' existing data, so... you can see where this is going.
I did as much testing and inspection of my code as I could, copied it from a personal dev script to the server's xp system, ... and debated if I really wanted to click [Apply]. Every time I considered it, I went back to check another part or do yet more testing. I ended up taking like 40 minutes to finally click it.
And when I did... that was the scariest button press of my life. And the scariest three seconds' wait afterwards. That one click could have ruined every single player's account, permanently lost us players ...
After applying it, I immediately checked my character to see if she was broken, checked the account data for corruption or botched flags, checked for broken interactions with the other systems....
Everything ended up working out perfectly, and the players loved all of the new features. They had no idea what went into building them, and certainly had no idea of what went into applying them, or what could have gone wrong -- which is probably a good thing.
Looking back, that entire environment was so fragile, it's a wonder things didn't go horribly wrong all the time. Really, they almost never did. Apocalypses did happen, but were exceedingly rare, and were ususally fixed quickly. I guess we were all super careful simply because everything was so fragile? or the decent devs were, at least. We never trusted the lessers with access 😅 at least on the main servers where it mattered. Some of the smaller servers... well, we never really cared about those.
But I'm honestly more surprised to realize I've never had nightmares of that button click. It was certainly terrifying enough.
But yay! Complete system overhaul and migration of stored and realtime player data! on prod! With no issues! And lots of happy players! Woooooo!
Thinking back on it makes me happy 😊1 -
It's really fascinating how managers can act surprised over a critical prod issue especially after the whole dev department warned them repeatedly.3
-
166 lines of if else hell just to decide between prod and sandbox.
Run the same query multiple times to get the same data.
And that's all written by our code review team lead.
What a mess.2 -
Trying to exit a bash script with 'halt'
on a friday afternoon
ran w/ root
on a internal prod-vm
which I did not had the permissions to turn on again4 -
any hardcore peeps like this? :P
gotta commit b4 I do anything..
gotta push b4 I do anything..
gotta deploy to prod b4 I do anything..
God! kill me already 😭😭😭😭
5 -
That moment when you make a bug fix on your local dev deployment and keep refreshing PROD web page hoping to reflect the change.1
-
Guys, when you have to manually edit records in the prod DB, please at least make sure that your overweight cat isn't rolling around on the keyboard...8
-
# 3 weeks ago
customer informed us that the app will get quite some load since the beginning of July
# this monday
last spare hdd on database's SAN died.
I told everyone another hdd is to follow it very soon eithet whether nothing is done at all or if anyone attempts to touch the san array [cuz that's what redundant raids do...]. No fucks given by anyone, no attempts to have maintenance and planned outage within 24 hrs....
# this morning
another hdd has failed and now 1tb of data is lost. No way to restore backups cuz there is no database to restore them to....
# 20 minutes later I head out to get some popcorn. It's gonna be a fun week!
So... The planned heavy load [and revenue along with it] is not gonna happen... I guess they are gonna ha a week long outage.
That's what happens when you ignore warning shots fired at your face.2 -
Me: I prefer to make sure dev is fully separated from prod, that way if we fuck up, it doesn't matter.
Manager: It's fine, the service isn't fully deployed and dev/prod are named differently.
Me: runs ansible script to stop dev
*prod breaks*
Me: Huh. Wonder why that happened.2 -
Making an infinite loop of cat /dev/urandom for anyone trying to login to server via ssh as an April Fool...
But on prod server instead of dev server...3 -
I bypassed SSL certificate verification because that goddam certificate had some flags which my JVM did not understand and threw errors. Still in prod after 10+ years 🤐1
-
Someone tried to push a patch in prod while demo was going on in front of CEO.
Good thing they locked the repo before the demo. 😏3 -
WHY THE FUCK ARE YOU ASSIGNING PROD BUGS WHEN I'M ON A FUCKING VACATION ?!?
Oh wait I wrote that code...
Welp6 -
That time I thought I crashed a prod server which I had no access to physically turn it back on. Turned out it was doing its weekly reboot. Scary 3 minutes.
-
"Eh, just push it to prod and see if anything breaks"
The entire system went down for hours, estimated loss north of $1M.5 -
Half way through a 2 hour deployment, and a fucking test fails.
Yes, it takes 2+hours to run the tests.
Rerun in test environment: pass
Rerun in prod: pass
Rerun changed test in prod: fail..
Why, why you got to hate me for?
I love it when production is the place where config gets changed.5 -
— Filezilla *open prod ftp*
— right click a file *edit*
— edit the file, save,
— filezilla *save new file*1 -
One day after the release of the website of a medium sized travel company, I made a big mistake by accidentally taking it offline for 1 hour during peak usage (~150 simultaneous visitors).
Turns out deleting the wrong image transformation cache folder in production can hang up the PHP process for taking too much load on regenerating image transformations.
The designer of PHP probably took a big load too while creating the first draft.9 -
Me: eh probably shouldn’t push to prod on Friday, maybe not even before the first...
Co worker: no balls
Me:
-
Never update prod stuff on a Friday afternoon.
After the team lunch.
With cheese fondue and white wine.1 -
Programming music:
scarlxrd - HeartAttack
https://soundcloud.com/scarlxrd/...
scarlxrd - BANE
https://soundcloud.com/scarlxrd/...
xxxtentacion - look at me! https://soundcloud.com/rojasonthebe...
xxxtentacion - #IMSippinTeaInYoHood
https://soundcloud.com/veronica-mar...
These 4 songs suffice me.
Sub-genre is TRAPMETAL11 -
I created our login system to be secure and reliable.
One coworker hardcoded the roles a person who is logging in receives and built a backend to just assign roles you want. He pushed this to prod...
Yeah...2 -
Deadline : Friday evening
Testing : There is only prod
Deploy : Friday evening
Why : Boss says customer wants it
Me : . . .11 -
4:55: Everything looks good in prod.
4:56: Deploy new feature after all is well in Dev.
4:57: Prod goes to shit!
4:58: Call wife to tell her I'm not coming home at 5.
4:59: Prod looks fine w/o anyone doing anything.
5:00: Leave work.
5:25: Get yelled at by wife for leaving at 5 after telling her I couldn't.3 -
Did a nasty thing in prod...
Very nasty.
So nasty even default graph tells me something about going yolo in prod.
Maybe it's weed. But feels like that graph tells me not to trry this shit ever again.
10 -
Happiness is when manager appreciates your work and says there are very few crashes on prod now... That feeling 😎2
-
It’s Friday, YAY my week is over....Oh fuck someone wants to do a prod release today, I am so not looking forward to today.
-
Discovered yesterday that my boss does tests with production database... and I'm responsible for the fucking backups and he doesn't even care to let me now when so I can at least schedule one at slave. Come on... it's not that hard to let others know or test on your own machine...1
-
This is deployed on PROD(!) from my Senior Dev's app. Have I told you devs how much I hate this guy already?
7 -
Instead of coffee to start your morning, try prod going down on Saturday! It will really get your blood pumping!2
-
Leading a team of 10 people, 5 environments (3 non prod 2 prod) to support, 25 formal deployments per week, and all I have is one fucking repository in fucking svn.
-
I don't know how managers are planning deadlines and counting December as a full working month!
Most companies that I worked with, count either half a month or push the deadline until the end of January when the workforce is back but not here.
Our division manager has promised the customer that the production environment will be ready on the first week of January, without even consulting the team or checking the schedule like WTF!
The person responsible for setting the infrastructure was on vacation for 2 weeks and he didn't hand over the access to production or share the progress done.
Fast forward, the manager went to slack and pinged the whole company with full caps message that the production should be done today.
Fun times :/7 -
Accidentally killed the ssh deamon on the prod server. I meant to kill my local sshd but confused the terminal tabs :#
-
Merry Christmas to all the dev ranters out there. My thoughts go out to those on call. May prod never fail and your phone never ring.
-
Me: Why are we spending time building reports for Support? AFAIK they never read or use them.
Boss: Seems they expect you to do it.
Me: Then what exactly are they supposed to be doing? All the issues seem to just escalate back to us.... We should just make sure issues never get into PROD
Boss: I agree, they're always firefighting... we gave them more funding so eventually they should catch up **Me: I highly doubt it, you should just stop hiring monkeys** esp. if we prevent PROD issues.
Me: Yea... we should prevent production issues because someone always has to pay off the (technical) debt and interest rates in PROD are very high6 -
Been a long ass day, then some later afternoon prepush testing that went long and wrong which led into night - and after all was said and done we had to throw our hands up and say...
https://youtu.be/Qy-Y3HJNU_s
-
Returning from vacation tmr... But apparently we just had a "someone at github deleted all the repos and no backup"-like prod issue....
Well guess I say least can say "¯\_(ツ)_/¯ not my area, can't help u"6 -
So I was changing some CSS, but the changes weren't showing.
Was it being cached? Nope
Was the selector wrong? Nope
Well it was the right file yeah? Yup
So after like 10min of scratching my head, restarting the server, etc it turns out I was checking prod instead of dev.
This isn't even the first time this has happened 😑
Guys just remember to keep your dev tab and your prod tab away from each other, like way away.8 -
Found this on prod
Don't know who did this but wwhhhyyyy do you need to add foreach loop😅
Just add directly
9 -
My first Project is finally ready to be deployed on prod server.... Nervous.... How u guys cope with this stress?10
-
I had 2 features I coded pushed to prod today at work! It's my first week at my new job. It made me realize how toxic and detrimental my last job was to both my self esteem and career. I'm so happy!!2
-
Departure: 13.00 Train: test Destination: A1 Delay 18888min
Now it is 15:43 and it is still on...
Finaly poland joins the testing in the prod club!
Station is Wroclaw if anybody is interested.
2 -
Right now, I'm tracing a bug that causes some of our data to be lost.
and I found a table with 'xxx' field on it. ohhh. what is it for?17 -
Solution for all the problems... Even global warming
Have you tried turning it off and on again...!?? 😌
2 -
Management has bequeathed that it shall hit prod tomorrow, the developers cry 'but it's not ready`. Hush little dev for tomorrow we hit another payment milestone
-
Holy shit, my first freelance project will be field-tested for the first time tomorrow!
After a few more rounds of fixing bugs, some serious improvements, and some new features additions, I can actually say that I'm proud of the code I've written! It's not perfect, but I definitely like the way it works - AND IT ACTUALLY WORKS!
Knock on wood, hopefully it won't shit the bed tomorrow.1 -
I got terminal white-space autism because of the CMS Of Doom.
Forgetting one or adding one too many leads to major problems on prod.4 -
Update Table_Name Set Column_Name = ‘New Value’;
Commit;
I did this on prod and my manager started screaming on me.
Is there any issue in it?5 -
1 Week ago I took down the development version of an internal demo, leaving only the prod one (temporary space issue). Colleagues were told n+15 times via email, face to face, comments on tickets etc. The prod one has also been live for weeks, and again, they were told when it went up.
This just happened:
Colleague 1: practiseSafeHex can you help me, the demo doesn't seem to be working.
Me: *logs in*, *click around* ... seems fine dude. Are you using the correct URL: <prod-url>.
Colleague 1: let me try again and see.
Colleague 2: practiseSafeHex i'm with Colleague 1, were trying to use the demo, and its not working. Can you have a look, we need it.
Me: I just told him, its fine, I think you have the wrong URL <prod-url>. Can you try again.
Colleague 2: No I have the right one, can you check it.
Me: *does nothing*, yep i've had a look at it, can you try again: <prod-url>.
Colleague 2: Ah its back now, cheers.
They are with a customer now, so I won't say anything, when they get back, one of them is being castrated. -
36 in previous job (sysadmin) - had to run 3 shifts in a row. Boy was I a vegetable in the end....
18 in current job (java dev), when I messed up PROD db -
Pressing ctrl+s in sql dev when checking packages.. you accidentally press space or sth and later ctrl s it compiles.. then shit gets real when you accidentaly lock up everything on prod db..2
-
Ideal dev job would be teaching kids code. Probably a side-gig at a local school.
Main gig would be writing code to exploit the "push to prod" Internet of Things things. Security on that is garbage. 🚮5 -
That moment when you realize your team-mate pushes code to PROD on Friday evening
but you are the one on pager rotation over the weekend...
2 -
The panic when I overhear my colleague in a meeting "sure I can test it in prod but we normally don't do that..."5
-
Dealing with stubborn devs who don't understand why non prod apps shouldn't talk to production apps.2
-
If 2020 were a software, everything happened because that one dev keeps pushing untested code into prod3
-
Admin home page secured well, but every CRUD page available without admin authentication on prod environment... for at least 3 weeks
-
Found this line in code that's been in prod for 10 years
"int kak = object.getValue()"
There is no contextual reason for it to be called kak, which means shit in Afrikaans15 -
Who the hell deploys PROD environments in a STAGING cluster.....?
Who the hell deploys PROD environments in a PROD cluster and (deliberately) configures them to use a STAGING environment's DB cluster....?
5 -
When a microservice you created does not perform on prod like it performed on the load stage. There goes my week and my self-esteem 😥2
-
When the company puts a 2 week freeze on all prod updates for the holidays , but still makes you show up for work (with nothing to do). 😭7
-
Worst prod scenario experienced - on site in small African country working on CRM/billing system my colleague was testing some new SQL and after finishing decided to drop and recreate the DB. She thinks the process is very slow and suddenly realizes she is dropping the prod DB. In a panic she shuts down the system and starts doing a restore from tape, but is so stressed out she writes "tar cv" instead of "tar xv" and overwrites the backup with the broken DB. Took a while to clean that one up...2
-
When you raise a ticket with your support team that your prod server is currently down and they mark it as a medium request, like dude... seriously?! Get it sorted ASAP paleaaaaase4
-
How come something works absolutely perfectly in dev but not in prod?
I was making a desktop app in election js and everything is working perfectly. No problem at all. But then I create the installer/distributable and nothing shows on the screen. And out of curiosity, I wanted to see the error log and it shows an unknown error, I didn't even know from what thing the error is being generated. And after I fixed that, another problem came with Asana Api. I mean, if it's a public API, why do you have to block it with cors? I hate cors!
And after all of it, there's more to it. I mean, why can't you just show the errors in dev?4 -
Setupwars. Show off your epic coding zone setup.
Your biggest oh fuck moment as developer. "I dropped the prod DB"1 -
So far? Not realizing my load balancer was not set up for sticky sessions... and since this load balancer only existed in prod not in qa or CAT o found out the night we went to install it into prod...1
-
I broke prod for 10 seconds, and nobody noticed. Also, why the fuck did the alarms not go off? 🤷♂️ 🚨 🙈5
-
Biggest sin? Easy. Pushed directly to master branch and deployed to prod. I should mention it was something very small before I get crucified😂
-
Today I had my first ever prod db fuckup due to too wide where clause in delete query. Spent 8 hours restoring service. It's 02.09 now and I'm finally home, going to bed.
Interestingly enough I've enjoyed this experience... -
Goodbye, imposter syndrome.
Today I patched a StackOverflowException bug (that I myself introduced a few months ago) which caused the prod application to crash the other day.
Now I truly feel like I belong! 😂2 -
https://devops.com/dear-staging-wer...
What the F#*@&$# %#@$!?!?!
This person has decided to skip using staging, because it doesn't correctly reflect prod!
If that's your problem, than why don't you try to fix it? Create a DB with fake data, make one based on anonymised customer data, or even do it on non-anonymised data (with permission of course), but fix the staging env so that it reflects prod!
This is a devops site (it's literally the name!), and instead of teaching you how to make staging exactly like prod, they tell you to do what caused the creation of the staging->prod system IN THE FIRST PLACE!!!
There's all these stuff like Vagrant that are literally designed to help you as a dev mimic prod, and you just throw it all out!?
"With feature flags, I can safely test in production without fear of breaking something or negatively affecting the customer experience."
Famous last words.12 -
In appraisal discussion,
Boss : Give me good reason why should we give you 30% appraisal!!!
Employee ( After logging into prod machine) : I am going to quit this job 😅😅 -
Just delayed a push to Prod. because of this motherfucking shit:
https://gitlab.com/gitlab-org/...
It never occurred to me to even look. FUCK.15 -
Working with LEGAL is the worst!
They say yes to all reviews and once it's in prod they want to completely change the whole thing cause they think every user reads the Terms & Condition!
To be honest I deployed 5 times. I have not read that shit once!1 -
Realized I could of prevented an entire day of our team dealing with a prod issue that could've been quickly resolved if I had noticed the Support guy actually restarted the wrong server...
Prod Issue + Friday == BAD... My brain has already logged off as of this morning...
attention to details == 0
Assuming someone else will fix it/do things correctly == 100 -
Let me check Slack
Just before I go to bed
Just in case — OH NO
It’s not what you think
It isn’t like I broke prod
… request makes me cringe.3 -
Angular2 rant..
It's in rc they said, write prod ready apps they said, it will be fun they said.. Arjziglfns.
just started rewriting for rc5 and modules, multiple declarations everywhere fml.
-_-7 -
Does anyone here uses Docker for all the environments (dev, qual, prod)?
If yes, how do you feel about it? How it is?8 -
Identical configuration.
Application won't find proper base URL locally.
Application will find proper base URL on test and prod instances.
Excuse me, what the FUCK?3 -
This condition over here which is never supposed to be true sometimes becomes true, no one know how or why, just change it manually in prod whenever that happens...
-
5am now and trying to do a production deployment. It just failed because boss loves making un-approved changes to prod that change entire database schemas. Argh!!
-
That awesome feeling when you launch your application on prod server....
*after 5 mins*
Customers start calling about dashboard not working.... -
Last day at work: goodbye overnight sessions breeding over some arcane legacy code that blew up in prod or manually restoring dozens of backups because the customer continued to work while systems went down due to power failure.
Colleagues last commit message at that place: It just works (Friday 8pm then shipping the code to prod)2 -
Got through 4 hours of merge conflicts before pushing to prod... I imagine this is how doctors feel during an operation... #ef
-
yay!!! Another PROD issue on something that we've had PROD issue on and well should be an easy fix... Cuz the root cause is prolly bc the idiot that implemented the logger had no idea what the fuck they were doing... And well the whole team didn't bother to ask me (and never listen to me anyway).... WHO WOULD PROBABLY SHOW THEM HOW THEY FCKED UP...
BUT ....
¯\_(ツ)_/¯1 -
Other dev in group chat: we need to stop syncing so much data from prod to uat because it's crashing the uat db...
Me thinking: no really... U just realized it's not a good idea to dump most of the prod db into uat every week?
¯\_(ツ)_/¯4 -
Assigned mandatory online training... Finally got some time to do it from my busy week fixing a prod issue... But the training website is completely broken....
Just another reminder of why things always blow up around here
-
Just before deploying this to prod found this bug... discover this bug with bunch of print statements
7 -
Work: yeah we just all use the prod credentials for s3. Just use the dev bucket.
Me: ummmmm..............2 -
"Okay, we'll try to explain again why you being able to execute your test cases against prod is a bad thing..."
-
Not sure what's better on Friday:
Fix the bugs and push it to prod so manager will be happy.
Or sit there like a dumb fuck and wait for Monday before touching the code2 -
Moving my baby into prod this morning. I find that the louder my headphones are, the less I can hear myself freaking out.
-
When you need input from your seniors to run a script in prod and none of them are comming in to work that day :(2
-
For a back-end, Java engineer, It's really sad when you are so bored at work that you get excited at the thought of designing SQL views/procs to make prod support easier.
-
Yesterday I had a user reachout via email because his account became inaccessible when he updated it.
This was something I didn't realize could happen... so it was fun recreating this scenario locally.
Basically he somehow bypassed the lowercase requirement for the updateUsername endpoint. So in the DB, his name was uppercase. And the SQL query was searching for his exact username without doing LIKE (by design).
It was failing on that getProfile endpoint and showing a Whoops screen. 😬 Should be a 404 screen, I know. I'm working on it.
Since then, I added Sentry logging to the backend and frontend.
And this showed me there were endpoints still being triggered nightly to remove stale data. (I forgot to remove). And these endpoints were getting 401's. I'm thinking who TF is triggering these endpoints. I soon realized those specific endpoints needed keys that expired monthly. 🤦♂️ -
People who freak out and think they have been hacked when you send a message to them on the prod server using the wall command should not be allowed SSH access.
-
After a long development and you do a live testing you but you forget to update the prod repo. AFkdkskdkdkskf errors is coming. My boss try to kick me 😂😂
-
Confession: a very important feature of the website I'm developping wasn't working for a certain time. The boss wasn't aware because he doesn't go on the site, and I only found out last week because I needed to implement a new feature that used the previous one. Problem: the bug was only on production, not on local (and of course we don't have test server).
I took advantage of the absence of my boss today to clear the situation by making all of my tests on prod. I hope no customer tried to pass a command today, but it's finally repaired. I am both proud and shameful.4 -
I have to convert a very complex algorithm form matlab to C , and push to prod in next 7 days 😭
[FYI can’t use matlab to c coverter]4 -
Ran a query on a prod server for 50 minutes before realizing I forgot to put my exclusions in the results... Oops.... New query runs in 1.5min. 4500 rows from the potential 340k rows.2
-
You know that feeling you get when there is a bug in Prod and to identify the cause, you setup your dev environment with the exact same codebase as Prod. But the bug won't show it's ugly little mug in devo...
Yeah, Fucks me up too... -_-2 -
Every time we have a release, "Release Engineering" stops building our test environments from master. I've been preaching Continuous Deployment for months and here we are with the most broken system. The build actually builds all test environments AND prod from the same branch because "it's easier" for "Release Engineering". So now I have to wait for the code freeze on release and hope RE doesn't fuck up and deploy an untested branch to prod... AND hope we're given enough time to test and debug the next release since we can't right now...1
-
did on my last project:
1 .Using QA env as dev env
2. Deploy in production not completely tested stuff (90% tested)
3. Run with errors in prod
4. Manual fix in prod
5. Git versioning1 -
I feel there aren't enough tutorials on "best practices" when it comes to combining server/client tooling in a monorepo.
Having done so this weekend, the tasks involved were:
* using graphql w/ express to serve requests and expose a "graphiql" ide instance
* differentiating build steps in prod vs. dev
* applying middleware in prod vs. dev
* working with a single heroku dyno
Still missing:
* hot reloading
* my general sense of direction -
There is a serious possibility that our team will need to deploy into prod on a Friday because of a regulatory deadline and 3rd parties not being ready.
God help us -
My company misses the word agile so much, now we are deploying 3-4 times to prod in a day 🤦🏽♂️🤷🏽♂️5
-
Ahhh... Waking up Monday morning.... To see a bunch of text messages... and new ones coming in every second... on my phone because another prod issue that was caused by a weekend release... That was supposed to fix a prod issue...
O well not my code, not my problem... It's nice I guess I'm getting less involved and sorta have my own domain of work. -
Believe me or not but there are some companies that do test their software on prod and ignoring users complaining.
What you think of this?14 -
i have seen bugs getting into prod but not the local language getting into prod. what language is this?
15 -
ITS FRIDAY!!!!!! such a long week..... fcking prod issue and new (anti) agile process team wants to implement2
-
Ops teams that are "assuring" stability of production by being the only ones who can click to deploy to Prod add no value.1
-
Should I be even be testing if no one cares.
I keep asking Devs regarding the functionalities of their system for testing and then, realize it's already on Prod even before I test in beta or enable to test in beta.
Do I exist!!!
Now, for the nth time, I have started to test when things are almost there or already there in Prod.
I should keep myself in adrenaline mode always on from now on.
Need to do something worthy.
The worst way to start a new job.2 -
Pushing a project in production without as much problems as expected is a very, very good feeling.
Have a great Friday, fellow developers.1 -
Today's winner of the characters that made me waste hours debugging for too... Dumb roll please...
/
For being forgotten to be added to the webpack config so that my docker builds the prod image correctly...3 -
Ok peeps!
A newbie to nodejs and linux VPS (ubuntu) would appreciate any tips you could throw my way on how to push locally developed nodejs webapp to prod in vps server 😊
Thanks!5 -
Tfw you find a comment saying something needs to be tested when the system is in prod.
Narrator: they did in fact not test it when the system was in production -
Some little piece of shit fairy messed with my code.
Made some modifications in a tamplate file, tested in dev everything worked. Pushed it to prod and in one of its cases it displayed the array key of each element. Checked the code, an that was what written in there in prod and in dev too. However that shit in dev (and previous versions in prod) displayed key[0]->name as intended.
So I thought git blame and check where it went wrong... Guess what... That fucking line is the same age as the file itself....
How that motherfucking shit worked, and what had I had in mind when writing that shitcode is a mistery... -
not me, but my co-workers left del logging active for about 6 months, then one day it became severely slow for finding free filenames to save into.
mine is DROP DATABASE in prod, after that i have been like I will never keep open console to prod -
To ensure everything goes well on prod, the chief QA set an env called 'stable' after prod deployment to stimulate true prod situation. Then we found that some apis on 'stable' crashed more often then those on prod.
-
It takes days to get a review. 3rd notice unanswered. So today it will be merged without review of 5 MRs. We’ll see what breaks in prod anyways.4
-
Realize the migrations of the technical base of application,
Go to the master branch of the git repository,
Open the pom.xml to get the last version of the app (running on prod) and see 0.0.1-SNAPSHOT
Lateral position of security and crying...1 -
I've having issues trying to form a proper branching strategy for my mean stack app deployment.
Heroku creates staging and prod branches for my web app so I'm a bit confused if I need my own staging branch?
Currently I have this: feature -> dev -> staging -> heroku staging (the staging branch seems useless)
Also, Heroku allows you to promote heroku staging to heroku prod, so there's no point in making master push to heroku prod.
I'm thinking of making my strategy to the following, but wasn't sure of any pitfalls or anything I'm overlooking long term.
feature -> dev -> master -> heroku staging -> manual promote to heroku prod.
Any suggestions?5 -
This Old Fart looks at webpack prod ver of my code and be like change the way you write code because I can't read it.1
-
What can you do if your boss is basically refusing to give you access to a server that you need access to to do your job? It's really weird to me. Only he has access to any prod systems we have. Is this realistic? Is this more of an attempt at staying necessary and needed? I'm pretty fed up with it as I've had prod access at every other company I've worked for and have nearly ten years of web experience .. what gives?11
-
I've just spent 4 hours trying to fix a bug on prod. that can be fixed in 30 secs. At the end I remembered that I should check the error log. FML (error reporting turned off, logs only)
-
development progress is like chinese whispers; "we have encountered issues and will be delayed" becomes "we'll be ready to push to prod by COB" when it reaches the CTO. /facepalm
-
Looks like someone has an itchy trigger finger when pushing to prod https://techcrunch.com/2016/09/...
-
How reliable or better freebsd is? I was thinking to use it in production server instead of ubuntu 16 LTS. I've heard it has pretty good networking stack and whatsapp uses it in their prod machines.6
-
Wish me luck.
Deploying Blazor app for the first time in prod in about a month.
Did tests. Curent infrastructure can hold around 70 concurent sessions with no problems. (probably more, 70 was the limit in my browser).
I tested each sessiuon with 70.000 line table. (Yep whole 70.000 lines for each session with a virtual scroll).
Shit is fast. Too fast even. I'm waiting for other shoe to drop, but so far in simulated tests it's amazing.
Let's talk in 2 month AFTER prod deploy.4 -
me to myself:
stick with one already
sometimes it's npm run build, npm run prod, npm run production, npm run dev, npm run watch, npm run serve, npm run hot, npm run start.
:D5 -
Whats the fucking purpose of our companys dev test and prod env. Dev always only has a single instance. Sometimes clustered services run as cluster on test. Producing headaches because the clustering behaviour couldnt be seen on a single instance and Prod lacks all the nice deployment tools off dev/test. Fuck thinking you could dev then test and prod without any major reconfiguration and headaches. And all because the Storage costs is RETARDEDLY expensive because the backup EVERYTHING with ridiculess overkill. That results in headaches when requesting new servers. Took an old Workstation from the shelves and made it my vm slave so at least i could reliably deploy to test.. Fuck this process
-
Do you use rust for production apps? if yes:
1. which framework do you use to build the server?
2. how do you work with mongodb?
3. how do you handle authorizations?
4. any beginner friendly project idea?1 -
I'm doing it !
I WILL run current PROD projects in a preview 1. (Well, at least I will try).
Wish me luck !
8 -
I just thought of an elegant solution to a problem in prod. Basically need to run 2 different versions of a app.
Just deploy it to uat and point it to prod db for the data.
I guess if we were following microservices architecture though we could deploy both versions to prod but we don't....3 -
Guys, about Continuous Integration/Delivery using SVN as repo... What is the best approach? It would be nice if it could trigger the files that should come to prod by commit message or something like that...
Thank you.3 -
Ask a client to populate Dev and QA environments with Prod data. They also populated prod with the data two more times.
-
Pro tip:
Don't make stateful singletons. Just spent an entire fucking morning debugging because one of those fuckers was trying to use prod, and not the demo environment.10 -
On pager duty. Getting alerts about New Relic synthetic monitors failing in prod. Freak out. Go to check them out...all of them are pointed at the uat sandbox, not prod.
I don't know how to feel about this. -
runbook: go manually dig and check some information, then run some sql on the prod database
Does manually running sql on prod database make anybody else feel nervous and gross?7 -
String good =
"I'll never give up type systems!!"
+ " Do you hear me?!"
+ " Never!";
bad =
"Can't believe it made it all the way into prod."
% NaN + 3 / '11' -
Has anyone else noticed a trend this summer of os updates completely shafting servers? Just saved my 5th prod server and its getting fucking old
-
One nightmarish project that was doomed from the beginning, had me as the sole developer. I could hardly sleep when we began testing on a separate test system, but with (nearly) all the config stored in shared memory and copied from the production system, I dreaded, half awake, that the production server data base connection was still configured in the test system and that it was shooting all it's test data repeatedly to prod.
Finally drove to company in middle of the night at 4 o'clock. Checked everything was OK, tried to sleep 3 hours before the start of the work day.
This system also had the most hideous memory corruption in some shared memory that was used across several processes and should have been thoroughly protected by a mutex, but somehow, sometimes this crucial map, that was used to speed up the access to all the customer data just contained garbage.
Still haunts me to that day. (Like xkcd's unresolved tension of a non-matching parenthesis - an unresolved bug. -
Random error that was occurring in prod discord authentication for new users (only in prod):
It wasn't returning the discord ID of the user logging in (only in prod). This should not even be possible if the user authenticates. So the null value from the ID was used and returned the first user in the DB (I know, weird mongodb edge cases). It should have returned null for the user searched not the first user in the DB wtfff.
Anyways, new users logging in would be logged in as me. Bahahah
Pushed the fix. All is good in the hood.1 -
PUT method works in dev but not in prod for code ignitor. Stuck on this one from a week. Someone HELP 🥲
(Checked htaccess and PUT is enabled)12
Top Tags
Weekly Rant
View