Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API

From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "do it in production"
-
"You gave us bad code! We ran it and now production is DOWN! Join this bridgeline now and help us fix this!"
So, as the author of the code in question, I join the bridge... And what happens next, I will simply never forget.
First, a little backstory... Another team within our company needed some vendor client software installed and maintained across the enterprise. Multiple OSes (Linux, AIX, Solaris, HPUX, etc.), so packaging and consistent update methods were a a challenge. I wrote an entire set of utilities to install, update and generally maintain the software; intending all the time that this other team would eventually own the process and code. With this in mind, I wrote extensive documentation, and conducted a formal turnover / training season with the other team.
So, fast forward to when the other team now owns my code, has been trained on how to use it, including (perhaps most importantly) how to send out updates when the vendor released upgrades to the agent software.
Now, this other team had the responsibility of releasing their first update since I gave them the process. Very simple upgrade process, already fully automated. What could have gone so horribly wrong? Did something the vendor supplied break their client?
I asked for the log files from the upgrade process. They sent them, and they looked... wrong. Very, very wrong.
Did you run the code I gave you to do this update?
"Yes, your code is broken - fix it! Production is down! Rabble, rabble, rabble!"
So, I go into our code management tool and review the _actual_ script they ran. Sure enough, it is my code... But something is very wrong.
More than 2/3rds of my code... has been commented out. The code is "there"... but has been commented out so it is not being executed. WT-actual-F?!
I question this on the bridge line. Silence. I insist someone explain what is going on. Is this a joke? Is this some kind of work version of candid camera?
Finally someone breaks the silence and explains.
And this, my friends, is the part I will never forget.
"We wanted to look through your code before we ran the update. When we looked at it, there was some stuff we didn't understand, so we commented that stuff out."
You... you didn't... understand... my some of the code... so you... you didn't ask me about it... you didn't try to actually figure out what it did... you... commented it OUT?!
"Right, we figured it was better to only run the parts we understood... But now we ran it and everything is broken and you need to fix your code."
I cannot repeat the things I said next, even here on devRant. Let's just say that call did not go well.
So, lesson learned? If you don't know what some code does? Just comment that shit out. Then blame the original author when it doesn't work.
You just cannot make this kind of stuff up.105 -
Big event. Massive traffic in production, so we were monitoring all night.
I was in a room with 2 devs of my team, a marketting girl, my boss and a designer... chilling.
Suddenly the production is down.
Boss: production is down, anyone can check?
Me: already on it
Dev1: it looks ok for me
Dev2: me too
Me: wait what? Impossible everything is down
Dev1: oh I refreshed the page it's not working
Me: don't stay on the page refreshing it like you are fucking monkeys. Give me useful intel or be quiet.
Market girl: is it working?
...
Guys is it working?
...
Hello?
Me: Not yet we are looking. Don't distract me.
Boss: client called us. They want it online now.
Dev1&2: he's looking
... 1 min later...
Boss: is it working?
Boss: is it working?
Boss: is it working?
Me: SHUT THE FUCK FOR FUCKING ONE SECOND. ALL OF YOU, OUT NOW. YOU ARE FUCKING MONKEYS WHO CAN'T DO SHIT. IF YOU CAN'T HELP JUST SHUT YOUR DAMN SHITHOLE. DEVS, LOOK WITH ME. MARKET GIRL PREPARE A FUCKING POST-MORTEM MAIL. BOSS GET THE CLIENT ON THE PHONE AND STALE. DO. YOUR. FUCKING. JOBS.
That's how I ended up screaming at everyone... the rest of the night went in complete silence and I fixed the issue 2min after the got quiet or busy.24 -
Best quotes from IT teacher:
- "C# is a language to program your IDE."
- "C# is a language for beginners, and is not really used in production."
- "We won't use Python to learn programming, because Python is a very old, slow and useless language, and is not really used anymore."
- "Yeah, your algorithm is fantastic, but you wrote 'The answer is: ' instead of 'Answer: ', so it's just a B."
- One of my classmates was bored and opened Notepad++, and when the teacher saw it, she said "I have been teaching programming for years, but I've never seen this program, what do you use it for?"
I feel so lucky that I have started learning programming years before at home, I just couldn't start if I had to learn this way.37 -
I’m a senior dev at a small company that does some consulting. This past October, some really heavy personal situation came up and my job suffered for it. I raised the flag and was very open with my boss about it and both him and my team of 3 understood and were pretty cool with me taking on a smaller load of work while I moved on with some stuff in my life. For a week.
Right after that, I got sent to a client. “One month only, we just want some presence there since it’s such a big client” alright, I guess I can do that. “You’ll be in charge of a team of a few people and help them technically.” Sounds good, I like leading!
So I get here. Let’s talk technical first: from being in a small but interesting project using Xamarin, I’m now looking at Visual Basic code, using Visual Studio 2010. Windows fucking Forms.
The project was made by a single dev for this huge company. She did what she could but as the requirements grew this thing became a behemoth of spaghetti code and User Controls. The other two guys working on the project have been here for a few months and they have very basic experience at the job anyways. The woman that worked on the project for 5 years is now leaving because she can’t take it anymore.
And that’s not the worse of it. It took from October to December for me to get a machine. I literally spent two months reading on my cellphone and just going over my shitty personal situation for 8 hours a day. I complained to everyone I could and nothing really worked.
Then I got a PC! But wait… no domain user. Queue an extra month in which I could see the Windows 7 (yep) log in screen and nothing else. Then, finally! A domain user! I can log in! Just wait 2 extra weeks for us to give your user access to the subversion rep and you’re good to go!
While all of this went on, I didn’t get an access card until a week ago. Every day I had to walk to the reception desk, show my ID and request they call my boss so he could grant me access. 5 months of this, both at the start of the day and after lunch. There was one day in particular, between two holidays, in which no one that could grant me access was at the office. I literally stood there until 11am in which I called my company and told them I was going home.
Now I’ve been actually working for a while, mostly fixing stuff that works like crap and trying to implement functions that should have been finished but aren’t even started. Did I mention this App is in production and being used by the people here? Because it is. Imagine if you will the amount of problems that an application that’s connecting to the production DB can create when it doesn’t even validate if the field should receive numeric values only. Did I mention the DB itself is also a complete mess? Because it is. There’s an “INDEXES” tables in which, I shit you not, the IDs of every other table is stored. There are no Identity fields anywhere, and instead every insert has to go to this INDEXES table, check the last ID of the table we’re working on, then create a new registry in order to give you your new ID. It’s insane.
And, to boot, the new order from above is: We want to split this app in two. You guys will stick with the maintenance of half of it, some other dudes with the other. Still both targeting the same DB and using the same starting point, but each only working on the module that we want them to work in. PostmodernJerk, it’s your job now to prepare the app so that this can work. How? We dunno. Why? Fuck if we care. Kill you? You don’t deserve the swift release of death.
Also I’m starting to get a bit tired of comments that go ‘THIS DOESN’T WORK and ‘I DON’T KNOW WHY WE DO THIS BUT IT HELPS and my personal favorite ‘??????????????????????14 -
My mentor/guider at my last internship.
He was great at guiding, only 1-2 years older than me, brought criticism in a constructive way (only had a very tiny thing once in half a year though) and although they were forced to use windows in a few production environments, when it came to handling very sensitive data and they asked me for an opinion before him and I answered that closed source software wasn't a good idea and they'd all go against me, this guy quit his nice-guy mode and went straight to dead-serious backing me up.
I remember a specific occurrence:
Programmers in room (under him technically): so linuxxx, why not just use windows servers for this data storage?
Me: because it's closed source, you know why I'd say that that's bad for handling sensitive data
Programmers: oh come on not that again...
Me: no but really look at it from my si.....
Programmers: no stop it. You're only an intern, don't act like you know a lot about thi....
Mentor: no you shut the fuck up. We. Are. Not. Using. Proprietary. Bullshit. For. Storing. Sensitive. Data.
Linuxxx seems to know a lot more about security and privacy than you guys so you fucking listen to what he has to say.
Windows is out of the fucking question here, am I clear?
Yeah that felt awesome.
Also that time when a mysql db in prod went bad and they didn't really know what to do. Didn't have much experience but knew how to run a repair.
He called me in and asked me to have a look.
Me: *fixed it in a few minutes* so how many visitors does this thing get, few hundred a day?
Him: few million.
Me: 😵 I'm only an intern! Why did you let me access this?!
Him: because you're the one with the most Linux knowledge here and I trust you to fix it or give a shout when you simply can't.
Lastly he asked me to help out with iptables rules. I wasn't of much help but it was fun to sit there debugging iptables shit with two seniors 😊
He always gave good feedback, knew my qualities and put them to good use and kept my motivation high.
Awesome guy!4 -
The project where I realized I wanted to go from chemist to pro dev.
I built a flow-chemistry spectrometer with monitoring backend in Haskell.
Spectroscopy is where you add a reagent to a glass tube, it changes color, and by measuring the exact color it tells you how much of something (for example, a toxin) is present in the sample.
I had to do that a lot on factory samples, writing down measurements using pen & paper.
I'm lazy so I decided to do the logical thing: Automate it. I bought a second hand spectrometer, stripped the casing, did a shitload of glassblowing and hooked up tubes to the production pipelines, so I could get samples, mixing them in the correct ratio with reagents in continuous flows using valves.
I ended up using 2 home-crafted arduino-like boards (etching PCBs is fun!).
One to calibrate the mixture against known samples and control solenoid valves to continuously cycle through various reagents and deionized flushing water, the other to record the measurements and send them to a server running a Haskell/Yesod API.
The server collected the information into InfluxDB (A time series database), displaying all data on a graphite dashboard.
Eventually I wrote Haskell plugins for most of the chemistry processes, from pH & temperature measurements to polymer property and pigment tests (they made a lot of printer ink).
Then I was fired because they didn't need chemists anymore, and the code "could be maintained by the intern" (poor guy)...
But I did find out that I loved functional programming, chemistry automation projects, and crafting my own electronics during that time.16 -
Here's a recent interview I had for an Android Developer job:
I: Interviewer, M: Me
I: hello, welcome
M: hi, thanks
I: do you know Kotlin?
M: yes, I've been working with it for 1.5 years and have written 3 projects in it
I: do you know RxJava, Dagger, Retrofit, and how to make Custom Views?
M: yes, I'm comfortable with them *explains*
I: do you know Room?
M: yes I do, I've done a lot of practices in it, but unfortunately have never needed to use it in production
I: what architecture do you use? Do you know MVP?
M: I'm currently using MVVM, but not MVP. I've debugged projects in it so I know what's going on in it
I: ok, do you have any questions for us?
M: how did I do?
I: I'm sorry sir, but you're not even a junior here
M: what? Why is that?
I: well you don't know Room and MVP?
M: I said I know them, just haven't used them in production.
I: well you have 3 years of experience but you dont even know Kotlin!
M: Kotlin was your first question and I said I have 3 projects in it. Did you even check the samples you asked for in the job posting?
I: SIR YOU'RE NOT A GOOD FIT FOR US, THANK YOU FOR COMING.
:/56 -
Is this the code life
Another scrum meeting
Caught in the the Node life
No escape from reality
Open your eyes
Look up to the screens and see..
I'm just a dev boy
Doing some debugging
Because there's warnings here
Errors there
Segment faults
Everywhere
Anytime you distract
Takes another hour from me
From me
*piano starts
Mama. Just committed a bug
Merge the branch to production
Did it fast for milestones
Mama. The repo has just begun
But now they going to throw the stack away.
Mama. U u u uu
Didn't mean to code in LAMP
But it's the only stack i know how to setup
In Ubuntu. Without docker
I really don't get vagrant
*piano
It's too late
My team is done
Some dev is working in Nepal
A UX dev. Now what is that?
Goodbye everybody
I've got to go
Gotta leave this lame meeting
And face the truth
Oh nooooo. I i interns
(they have questions)
I want to debug
I don't want to stay till 3 in the morning
*epic guitar
I see a litlle dev over there
Let's code review, let's code review
Did he do the last commit?
Coding in the white board
Very very frightening me
That's bug(that's a bug)
That's a bug (that's a bug)
What the f*ck did you do that?
Magnificcooooooo
I was just coding and nobody liked it
He was coding and nobody liked it, spare his some time to do his debugging
Easy man. Here go. Will you let me code?
A meeting. No,we will not let you code. ( let me code)
A meeting. we will not let you code. ( let me code)
A meeting. we will not let you code. ( let me code)
We will not let you code
Never never let you go
Never let you code, oh
No no no no no no no
Oh mama mia, mama mia ( dude, you've gotta let me code)
Screw you guys, I'm gonna code and commit. Commit. Comiiiiitt!
*epic guitar
So you think you can review me and spit in my eye?
So you think you can dump me and erase my branch?
Oh baby, cant do this to me baby
I've just have to log out.
I've just have to log outta here
*epic guitar solo
Nothing really matters
The users will not care
Nothing really matters
To them
Any way this code blows10 -
Has anyone here have crazy ideas of what they want to do in life, only to realise those might be absolutely pointless?
As I've studied Food Production Technology, I have this idea of somehow making it possible to mix phages in foods for vaccination purposes. Phages are natural viruses to bacteria, so they are difficult to be used in that way...
Another idea is to make simulations of life of bacteria of different kinds.
And study the hell out of some pretty weird genetic mutations that are purely esthetic, but shouldn't happen.
Yeah, I am into microbiology and genetics...
I'd also like to develop games...14 -
I worked on a greenfield project a couple of years ago. The company had an old solution written in Omnis (heard of it? Yeah, me neither) with an SQL database. My team was to create a completely new web based system... on top of the old database, so the customers could keep their existing stuff.
The dba was an intelligent man, one of the nicest people I've met, and over the course of fifteen years he had made a remarkably terrifying monstrosity of a database. Some years before me they wanted to "future proof" the system and make it "easier to switch to new technologies". So they moved the entire business logic into the database...
I used a tool to create a visualization of said database when we started. It had no views, only tables and sprocs. Look at it! Tables and sprocs are rectangles (well, dots) and any connections are drawn in grey lines. There were no foreign keys, so a tables only visualization only yielded a collection of independent rectangles without a single line.
Now, the stored procedures were bloody MASSIVE. A single procedure that only registered a new interested party and attached them to a property had 2500+ lines and over 150 parameters.
Also, this dba added features and fixed bugs by logging into the respective customers production server and writing SQL.
That database is the stupidest thing I've ever seen a developer do.34 -
My biggest dev blunder. I haven't told a single soul about this, until now.
👻👻👻👻👻👻
So, I was working as a full stack dev at a small consulting company. By this time I had about 3 years of experience and started to get pretty comfortable with my tools and the systems I worked with.
I was the person in charge of a system dealing with interactions between people in different roles. Some of this data could be sensitive in nature and users had a legal right to have data permanently removed from our system. In this case it meant remoting into the production database server and manually issuing DELETE statements against the db. Ugh.
As soon as my brain finishes processing the request to venture into that binary minefield and perform rocket surgery on that cursed database my sympathetic nervous system goes into high alert, palms sweaty. Mom's spaghetti.
Alright. Let's do this the safe way. I write the statements needed and do a test run on my machine. Works like a charm 😎
Time to get this over with. I remote into the server. I paste the code into Microsoft SQL Server Management Studio. I read through the code again and again and again. It's solid. I hit run.
....
Wait. I ran it?
....
With the IDs from my local run?
...
I stare at the confirmation message: "Nice job dude, you just deleted some stuff. Cool. See ya. - Your old pal SQL Server".
What did I just delete? What ramifications will this have? Am I sweating? My life is over. Fuck! Think, think, think.
You're a professional. Handle it like one, goddammit.
I think about doing a rollback but the server dudes are even more incompetent than me and we'd lose all the transactions that occurred after my little slip. No, that won't fly.
I do the only sensible thing: I run the statements again with the correct IDs, disconnect my remote session, and BOTTLE THAT SHIT UP FOREVER.
I tell no one. The next few days I await some kind of bug report or maybe a SWAT team. Days pass. Nothing. My anxiety slowly dissipates. That fateful day fades into oblivion and I feel confident my secret will die with me. Cool ¯\_(ツ)_/¯12 -
USER: I can't see any data in the page...!
ME: ok, I'll do a check
ME: API calls get no data back. Boss, did you change anything and put it in production?
BOSS: Absolutely not, I just modified the name of what was the "Family" parameter in "Type".
ME: Seems legit. Totally agree. I'm going to lunch. Can you check in the meanwhile why calling the API with "Family" does return nothing? Thanks.3 -
If all you have is a hammer, everything looks like a nail!
This was something which my tech lead used to tell me when I was so obsessed with nosql databases a few years back. I would try to find problems to solve that has a use case for nosql databases or even try to convince me(I didn’t realise it back then) that I need to use nosql db for this new idea that I have, without really thinking deep enough whether the data in question is better represented using an sql schema or not.
Now, leading a team of young developers, I come across similar suggestions from few of my team members who just discovered this new and shiny tech and want to use it in production projects.
While I am not against new and shiny, it’s not a good practice to jump right in to it without exploring it deep enough or considering all the shortcomings. The most important question to ask is, whether some of the problems you are trying to solve can be solved with the current stack.
Modifying your stack requires more than just a week’s experience of playing around with the getting started guide and stack overflow replies. This is something which need to be carefully considered after taking inputs from the people who would be supporting it, that include operations, sysadmins and teams that are gonna interface with your stack indirectly.
I am not talking about delaying adoption by waiting for long list of approvals to get some thing that would bring immediate value, but a carefully orchestrated plan for why and how to migrate to a new stack.
Just because one of the tech giants made a move to a new stack and wrote about it in their engineering blog doesn’t mean that you need to make a switch in the same direction. Take a moment to analyse the possible reasons that motivated them to do it, ask yourself if your organisation is struggling with the exact same problems, observe how others facing the same issue are addressing it, and then make an informed decision.
Collect enough data to support your proposal.
Ask yourself again if you are the one holding the hammer.
If the answer is no, forge ahead!9 -
To replace humans with robots, because human beings are complete shit at everything they do.
I am a chemist. My alignment is not lawful good. I've produced lots of drugs. Mostly just drugs against illnesses. Mostly.
But whatever my alignment or contribution to the world as a chemist... Human chemists are just fucking terrible at their job. Not for a lack of trying, biological beings just suck at it.
Suiting up for a biosafety level lab costs time. Meatbags fuck up very often, especially when tired. Humans whine when they get acid in their face, or when they have to pour and inhale carcinogenic substances. They also work imprecisely and inaccurately, even after thousands of hours of training and practice.
Weaklings! Robots are superior!
So I replaced my coworkers with expensive flow chemistry setups with probes and solenoid fluid valves. I replaced others with CUDA simulations.
First at a pharma production & research lab, then at a genetics lab, then at an Industrial R&D lab.
Many were even replaced by Raspberry Pi's with two servos and a PH meter attached, and I broke open second hand Fischer Sci spectrophotometers to attach arduinos with WiFi boards.
The issue was that after every little overzealous weekend project, I made myself less necessary as well.
So I jumped into the infinitely deep shitpool called webdev.
App & web development is kind of comfortable, there's always one more thing to do, but there's no pressure where failure leads to fatalities (I think? Wait... do I still care?).
Super chill, if it weren't for the delusion that making people do "frontend" and "fullstack" labor isn't a gross violation of the Geneva Convention.
Quickly recognizing that I actually don't want to be tortured and suffer from nerve damage caused by VueX or have my organs slowly liquefied by the radiation from some insane transpiling centrifuge, I did what any sane person would do.
Get as far away from the potential frontend blast radius as possible, hide in a concrete bunker.
So I became a data engineer / database admin.
That's where I'm quarantining now, safely hiding from humanity behind a desk, employed to write a MySQL migration or two, setting up Redis sorted sets, adding a field to an Elastic index. That takes care of generating cognac and LSD money.
But honestly.... I actually spend most of my time these days contributing to open source repositories, especially writing & maintaining Rust libraries.10 -
Doot doot.
My day: Eight lines of refactoring around a 10-character fix for a minor production issue. Some tests. Lots of bloody phone calls and conference calls filled with me laughing and getting talked over. Why? Read on.
My boss's day: Trying very very hard to pin random shit on me (and failing because I'm awesome and fuck him). Six hours of drama and freaking out and chewing and yelling that the whole system is broken because of that minor issue. No reading, lots of misunderstanding, lots of panic. Three-way called me specifically to bitch out another coworker in front of me. (Coworker wasn't really in the wrong.) Called a contractor to his house for testing. Finally learned that everything works perfectly in QA (duh, I fixed it hours ago). Desperately waited for me to push to prod. Didn't care enough to do production tests afterwards.
My day afterwards: hey, this Cloudinary transform feature sounds fun! Oh look, I'm done already. Boo. Ask boss for update. Tests still aren't finished. Okay, whatever. Time for bed.
what a joke.
Oh, I talked to the accountant after all of this bullshit happened. Apparently everyone that has quit in the last six years has done so specifically because of the boss. Every. single. person.
I told him it was going to happen again.
I also told him the boss is a druggie with a taste for psychedelics. (It came up in conversation. Absolutely true, too.) It's hilarious because the company lawyer is the accountant's brother.
So stupid.18 -
Hey, Root? How do you test your slow query ticket, again? I didn't bother reading the giant green "Testing notes:" box on the ticket. Yeah, could you explain it while I don't bother to listen and talk over you? Thanks.
And later:
Hey Root. I'm the DBA. Could you explain exactly what you're doing in this ticket, because i can't understand it. What are these new columns? Where is the new query? What are you doing? And why? Oh, the ticket? Yeah, I didn't bother to read it. There was too much text filled with things like implementation details, query optimization findings, overall benchmarking results, the purpose of the new columns, and i just couldn't care enough to read any of that. Yeah, I also don't know how to find the query it's running now. Yep, have complete access to the console and DB and query log. Still can't figure it out.
And later:
Hey Root. We pulled your urgent fix ticket from the release. You know, the one that SysOps and Data and even execs have been demanding? The one you finished three months ago? Yep, the problem is still taking down production every week or so, but we just can't verify that your fix is good enough. Even though the changes are pretty minimal, you've said it's 8x faster, and provided benchmark findings, we just ... don't know how to get the query it's running out of the code. or how check the query logs to find it. So. we just don't know if it's good enough.
Also, we goofed up when deploying and the testing database is gone, so now we can't test it since there are no records. Nevermind that you provided snippets to remedy exactly scenario in the ticket description you wrote three months ago.
And later:
Hey Root: Why did you take so long on this ticket? It has sat for so long now that someone else filed a ticket for it, with investigation findings. You know it's bringing down production, and it's kind of urgent. Maybe you should have prioritized it more, or written up better notes. You really need to communicate better. This is why we can't trust you to get things out.
*twitchy smile*rant useless people you suck because we are incompetent what's a query log? it's all your fault this is super urgent let's defer it ticket notes too long; didn't read19 -
This is where everything started. I got the chance to work with actual production code. While it is very fun to work with, in some places it's also very frustrating. And this is from where, most of my rants come from.
Thank you @dfox and @trogus for making such a beautiful community.
The best part of this community is I never have to think or take time to make jokes or posts so that I can get upvotes. I've always wanted likes or retweets or reddit upvotes. But it never worked out because I have to think so much to make clever comments or posts. Most of the time, I gave up.
But in devRant, all I do is just share what's happening in my daily dev life. My frustrations, my happiness. That's all it takes. Everyone understands, everyone cares and everyone loves.
Over time, thanks to devRant, I've understood that I was part of the wrong community. This is the community that I deserve, this is the community that every dev deserves.
Thank you all. I love you. And I promise, more rants are coming :D
Especial thanks to @Yeah69 @kevbost @yarwest @tisaconundrum @Linux @donkeyScript . I have no idea why you guys all of a sudden rapidly upvoted me. Although I would love to reach 10k naturally but won't complain haha8 -
Advice that I give to interns/grads:
In uni/college, you're taught *how* to code something to achieve a goal, and 99% of the time the code will work and do the job in a lab.
But when building things for a real production environment, you learn the 100 ways how *not* to code, from seeing things break left right and centre - basically everything and anything can break your code, whether it is users, the OS, other people's code, legacy code, lag, concurrency, the alignment of the moon to your server...5 -
FML. An overreaching supergenius "architect" and a database team:
A: "We have decided that apps should use mysql. Install a MySQL so we match cloud"
DBA: "we don't have an image or experience with MySQL. We have mssql and Oracle "
A: "ok, use mssql in data center and mysql in production cloud"
DBA: "that's... not going to work well"
A: "just do it!"
...
Me, reading this shit, sends email: "ignoring the fact that we have more than 500 queries in this application which will need to be checked and most likely rewritten, how are we supposed to test the mysql queries without production access?"
A: "just use mssql local and MySQL in cloud"
M: "... Just to make sure I understand, you want us to write queries for mssql, test them locally, and then write separate queries, with a separate SQL connection abstraction that deploys to production? Again, how are we going to test this?"
A: "no, use same queries, should be fine"
M: "they really won't, they're different dialects"
A: "do the needful, make work!"
If karma were a thing, this person would have long since exploded into a cloud of atomized blood.18 -
Client: "I need you to implement a feature which does x"
Me: "We can it do like this, I can do it in Y hours."
Client: "Perfect do it"
Me: "Here you go have a look and if you give your ok I'll implement it on production."
Client:"That is not what I need. I need Z"
Me: "Well then you should have said Z and not x. But I can do Z if you want me to."
Client:"Do it it is urgent!!!!111"
Me: "All done here you go."
Client: "That works like what I said what I need, but I meant more like xZ."
Me: "Ok, you know I have to charge you for all this, do you?"
Client: "What why? It isn't the feature I wanted!!11 Do it right and I'll pay you for the right one!"
Me: "It might not be what you wanted but it is exactly what you specified to me. I'll send you the bill and will not continue working for you. Good luck finding someone who is willing to do unpaid work for you."
I am so done with that kind of client.8 -
Coworker: "Hey I have the final logo for your software"
Me: "Awesome, only a few bugfixes after my holiday and we are good to go on production. 2 more hours to go until I am free for a week."
Coworker: "After your holiday? Boss said it goes on production tomorrow!"
Me: "You are kidding me, aren't you? There is no way I can do all the work today and push to production!"
Coworker: "Boss promised the client..."
How about boss can go an f himself? He knows I have some other project to finish today and that I am leaving. This is in our team cal for over a month now! Ahhh. My coworker now has to deal with it-.-2 -
Woke up early. Feel great. Even office coffee tastes great. Today is going to be a good day.
Lead QA: - Hey I'm fixing some stuff in production, and now I can't reach it. Do you have the same problem?
TODAY IS A GOOD DAY, GO FIX YOUR OWN PROBLEM CUNT.
COFFEE TASTES FUCKING GREAT.2 -
I quit and my last day is next week.
Apparently management has decided that I should spend my last day implementing a new feature for a customer where I have been the only developer, and release it to production (without first implementing it in test) the same day. A feature that potentially could cripple a whole workflow if done wrong.
Of course I advised not to release untested code to production on a friday, just before the only person that knows how it works leaves the company. But no, “the customer reaaaaaally wants it before summer, so just be careful not to write any bugs”.
I’m not saying that I’m intentionally gonna write bad code - but if I do, I’m not gonna pick up the phone when it calls.17 -
My team handles infrastructure deployment and automation in the cloud for our company, so we don't exactly develop applications ourselves, but we're responsible for building deployment pipelines, provisioning cloud resources, automating their deployments, etc.
I've ranted about this before, but it fits the weekly rant so I'll do it again.
Someone deployed an autoscaling application into our production AWS account, but they set the maximum instance count to 300. The account limit was less than that. So, of course, their application gets stuck and starts scaling out infinitely. Two hundred new servers spun up in an hour before hitting the limit and then throwing errors all over the place. They send me a ticket and I login to AWS to investigate. Not only have they broken their own application, but they've also made it impossible to deploy anything else into prod. Every other autoscaling group is now unable to scale out at all. We had to submit an emergency limit increase request to AWS, spent thousands of dollars on those stupidly-large instances, and yelled at the dev team responsible. Two weeks later, THEY INCREASED THE MAX COUNT TO 500 AND IT HAPPENED AGAIN!
And the whole thing happened because a database filled up the hard drive, so it would spin up a new server, whose hard drive would be full already and thus spin up a new server, and so on into infinity.
Thats probably the only WTF moment that resulted in me actually saying "WTF?!" out loud to the person responsible, but I've had others. One dev team had their code logging to a location they couldn't access, so we got daily requests for two weeks to download and email log files to them. Another dev team refused to believe their server was crashing due to their bad code even after we showed them the logs that demonstrated their application had a massive memory leak. Another team arbitrarily decided that they were going to deploy their code at 4 AM on a Saturday and they wanted a member of my team to be available in case something went wrong. We aren't 24/7 support. We aren't even weekend support. Or any support, technically. Another team told us we had one day to do three weeks' worth of work to deploy their application because they had set a hard deadline and then didn't tell us about it until the day before. We gave them a flat "No" for that request.
I could probably keep going, but you get the gist of it.4 -
You know side projects? Well I took on one. An old customer asked to come and take over his latest startups companys tech. Why not, I tought. Idea is sound. Customer base is ripe and ready to pay.
I start digging and the Hardware part is awesome. The guys doing the soldering and imbedded are geniuses. I was impressed AF.
I commit and meet up with CEO. A guy with a vision and sales orientation/contacts. Nice! This shit is gonna sell. Production lines are also set.
Website? WTF is this shit. Owner made it. Gotta give him the credit. Dude doesn't do computers and still managed to online something. He is still better at sales so we agree that he's gonna stick with those and I'll handle the tech.
I bootstrap a new one in my own simplistic style and online it. I like it. The owner likes it. He made me to stick to a tacky logo. I love CSS and bootstrap. You can make shit look good quick.
But I still don't have access to the soul of the product. DBs millions rows of data and source for the app I still behind the guy that has been doing this for over a year.
He has been working on a new version for quite some time. He granted access to the new versions source, but back end and DB is still out of reach. Now for over month has passed and it's still no new version or access to data.
Source has no documentation and made in a flavor of JS frame I'm not familiar with. Weekend later of crazy cramming I get up to speed and it's clear I can't get further without the friggin data.
The V2 is a scramble of bleeding edge of Alpha tech that isn't ready for production and is clearly just a paid training period for the dev. And clearly it isn't going so well because release is a month late. I try to contact, but no reaction. The owner is clueless.
Disheartening. A good idea is going to waste because of some "dev" dropping a ball and stonewalling the backup.
I fucking give him till the end of the next week until I make the hardware team a new api to push the data and refactor the whole thing in proper technologies and cut him off.
Please. If you are a dev and don't have the time to concentrate on the solution don't take it on and kill off the idea. You guys are the key to making things happening and working. Demand your cut but also deserve it by delivering or at least have the balls to tell you are not up for it. -
Juniors are a fun bunch to work with.
Over confident, hero complex of that fresh graduate high, and then thrown in to the real world! Where there hopes and dreams are crushed in minutes when they see what monolithic applications really look like!!
But don't let that overwhelm you, your not going to be changing all of it any time soon, hell some of this code hasn't been touched in 5+ years and still works without fail.
Don't stress about the work load, you can only write 1 line of code at a time anyway, and hell, even seniors make mistakes.
The key about being able to manage this beast is simple, break it! Because the more you break it, the more you'll understand how a project is put together, for better or worse. Learn from the examples in front of you, and learn what not to do in the future 😎
But more importantly, plan your changes, whiteboard the high level logic of what it is you want to add, then whiteboard in the current codebase and determine where to slice this bitch up, then when it all looks well and good, take out your scalpel and slice and dice time.
Don't worry, your changes aren't going to production anytime soon, hell, you'll be lucky to get past the first pull request with this working 100% the first time, and that's a good thing, learn from tour short comings and improve your own knowledge for the next time!2 -
My first testing job in the industry. Quite the rollercoaster.
I had found this neat little online service with a community. I signed up an account and participated. I sent in a lot of bug reports. One of the community supervisors sent me a message that most things in FogBugz had my username all over it.
After a year, I got cocky and decided to try SQL injection. In a production environment. What can I say. I was young, not bright, and overly curious. Never malicious, never damaged data or exposed sensitive data or bork services.
I reported it.
Not long after, I got phone calls. I was pretty sure I was getting charged with something.
I was offered a job.
Three months into the job, they asked if I wanted to do Python and work with the automators. I said I don't know what that is but sure.
They hired me a private instructor for a week to learn the basics, then flew me to the other side of the world for two weeks to work directly with the automation team to learn how they do it.
It was a pretty exciting era in my life and my dream job.4 -
Probably the intern who, while a decent and intelligent guy for the most part, thought it was increasingly hilarious to keep putting random cat GIFs somewhere in the product with each PR he made.
First time "ok, very funny, but you can't do that in production software, don't do it again."
The third time around the "joke" was wearing a little thin.
Eventually a script was written so that, every time he made a PR, he'd get "emailed" one of a few pre-defined messages from a random member of the team a few minutes later, telling him to remove it and stop pissing about...3 -
Okay so about a year ago these FUKKING IDIOTS decides, against my recommendation, to do this quick-ugly-hack and ninja it into production.
I tell them its a FUCKING BAD idea that will blow up in a year or so...
But no, just go go go!!!!!!!
Now a year later, shit blew up badly. A total FUCKING derail. These new idiots asks me to "fix the problem", the same fucking problem I predicted and warned them about a year ago. So now i have to clean up their ducking mess because "Nobody else knows how to fix it".
What the FUCKING HELL do we pay them fuckkkers to do?!!
New idiots you ask? Yep, because 3 out of the 4 original fucktards already left the place in order to go and make some other new collegues lives fucking miserable.
FUCK YOU FUCKING MOOTHERFUUUKKKEEERRRRRSSSSSS!!!!!!!!!!!!!!2 -
Sex talk between programmers.
She: I'm a virgin.
He: Don't worry. They call me the virginslayer007.
She: Oh! So how many virgins have you slayed till now?
He: That would be ONE in a few minutes.
She: So u r also a virgin then..
He: Don't worry. I watched so many video tutorials. We just have to do exactly as they did. Best thing is that it can be done both for testing and production purposes.
She: Let's stick to testing purposes for now.6 -
Messaging me at 4:30pm on a Friday about a high priority issue currently in production.
My reply: a link to the code review from 2 months ago that literally explained the problem and what to do to fix it. That got implemented. Then removed. For some reason...
Feels good.2 -
Freaking tech support.
Freaking sparkhire.
Their 'one-way interview' bs only supports flash. Flash. in production. in 2019. Flash died years ago, and its support ends next year. What the crap?
Anyway, I finally decided I should do the interview since they already have all of my information anyway. Thanks, "privacy-conscious" third party. Totally appreciate it.
I spent half an hour and couldn't get flash working on their site (but all other sites were fine), so I contacted their support. I gave them all the relevant specs (inc. ofc browser), the steps to reproduce, and all of my attempts at fixing the issue.
To their credit, I recieved a response within a few minutes. To their discredit: their response was: "What browser are you using?" This question was followed by my report (including, ofc, my browser and all the other overlooked details), immediately followed by a "debugging info" section appended by their support service that also included my browser, os, and other specs.
Learn to fucking read.
Their suggestion? Use google chrome. Barring that: record your 20-30 minute video by holding your phone in front of your face the entire time. I am so not kidding.
They also asked what page i was having difficulty on. You guessed it: the page url was also included within that "debugging info" section.
It wasn't a form letter, either. I'd understand if it was all automated, but it was a real person who was really typing up the emails, and really didn't bother reading a damned thing.
I did end up getting flash working, but their "tech support" (script-reader) was entirely useless.16 -
Just started learning python and here is my experience so far
I had started programming with C++ but since I wanted to venture into the fullstack domain (upto some extent, circumstances played a major role), I switched to java and boy was I in love with it.. Spring boot was my life and I had written several applications on it currently running on production.. One of them was crawling tweets and getting some insights out of them.. Today when I started with python, I found a tutorial (link at the end of the post) that almost did the same thing..
Within 2-3 hours and some very basic lines of codes I could achieve exactly what java would have taken at least a week to do.. Python for sure is a good thing..
Probably I am still in my very adolescent stage of learning, but python does seem a very good option worth considering.. Though for now, I would stick to java for writing useful code..
https://marcobonzanini.com/2015/03/... -
my story so far
Hey guys. i just wantes to share my story becoming something i think is like a dev.
I was always interested in solving problems. my grandfather has a company with a bit over a 100 employees. one day i decided to start working there. he needed someone to build up the erp system (mostly maintenance). about a month after i started he decided to get a new erp system because the one he had would not fill his needs. not knowing how big this got i told him that i want to build it up. from getting the orders over production with machines to billing.
he agreed. after a short time we knew that even this new system does not fullfill our needs. but it was so damn expensive. i told my grandfather: trust me, i am handling this. no further costs. and i started to learn programming. i learned night and day (visual basics.net, sql, c#). since then i wrote about 8 additional modules for the system in coorperation with the users. today, 3 years later we are far ahead our market in terms of transparency and information flow. i worked very hard for this and it is a great feeling to see that the things i do help my colleagues and are used.
i never learned this stuff in school and i know that i cannot tell that i am a professional programmer.
but when someone asks me i tell them i am a programmer because my solutions work and i think i deserve to call me that.
thanks for reading :)4 -
I was only seventeen back then and I was a Java Developer Intern, not knowing much about enterprise oriented coding.
The project leader in our dev team saw a lot of potential and passion in my work, but was convinced I wasn't taught enough to do the right thing.
I was mainly doing shitty mappers and services back then, which were somewhat used but never lasted long and were ditched a few months later, which always bummed me out. I wanted to make an impact on REAL projects that would deploy into production.
So Mister Mentor (GDPR forbid to use the actual name), who was always first to come and last to leave the office, taught me what it means to code for real.
We stayed after 5pm until 7-8pm multiple times a week and he taught me in a deeply understanding and calm way how to:
- Git (SVN)
- Refactor
- SOA
- Annotate
- Deploy
- Unit Test
And most importantly:
- How to debug like an absolute BOSS
(We even debugged native Java Libraries just for fun to see if we could break them)
Fast-forward a month later and little intern me made his first commit on production.
Without Mister Mentor, I wouldn't be half as good of a developer as I am today.3 -
So I took on a fairly big project and poured my heart and soul into it, was the biggest thing I did yet. I kept on sending beta's to the customer after each change for review! Kept on insisting that they review it, the answer was always "this looks amazing keep doing what you're doing"! After I finished and pushed everything to production.
They didn't use it for nearly 6 months! And then out of the blue they call me saying that half of the app is wrong.. WTF? Where was this information during testing! I informed them that the changes would take some time since I need to do migrations and change the whole database schema.
In which they replied "but you already finished it once won't changing things make it easier? We shouldn't pay for your mistakes"
I don't know how I handled that but they should be thankful they were half way across the country 😠😠😠😠3 -
I worked for over 13 hours yesterday on super-urgent projects. I got so much done it's insane.
Projects:
1) the printer auto-configuration script.
2) changing Stripe from test mode to live mode in production
3) website responsiveness
I finished two within five minutes and pushed to both QA and Production. actually urgent, actually necessary. Easy change.
The printer auto-configure script was honestly fun to write, if very involved. However, the APIs I needed to call to fetch data, create a printer client, etc... none of them were tested, and they were _all_ broken in at least two ways. The CTO (api guy in my previous rant) was slow at fixing them, so getting the APIs working took literally four hours. One of them (test print) still doesn't work.
Responsiveness... this was my first time making a website responsive. Ever. Also, one of the pages I needed to style was very complicated (nested fixed-aspect-ratio + flexbox); I ended up duplicating the markup and hacking the styling together just to make it work. The code is horrible. But! "Friday's the day! it's going live and we're pushing traffic to it!" So, I invested a lot of time and energy into making it ready and as pretty as I could, and finally got it working. That page alone took me two hours.
The site and the printer script (and obv the Stripe change as well) absolutely needed to be done by this morning. Super important.
well.
1) Auto-configure script. Ostensibly we would have an intern come in and configure the printers. However, we have no printers that need configuring, so she did marketing instead. :/ Also, the docs Epson sent us only work for the T88V printer (we have exactly one, which we happened to set up and connect to). They do not work for the T88VI printers, which is what we ordered. and all we'll ever be ordering. So. :/ I'll need to rewrite a large chunk of my code to make this work. Joy :/
2) Stripe Live mode. Nobody even seemed to notice that we were collecting info in Test mode, or that I fixed it. so. um. :/
3) Responsiveness.
Well. That deadline is actually next Wednesday. The marketing won't even start until then, and I haven't even been given the final changes yet (like come on). Also! I asked for a QA review last night before I'd push it to production. One person glanced at it. Nobody else cared. Nobody else cared enough to look in the morning, either, so it's still on QA. Super-important deadline indeed. :/
Honestly?
I feel like Alice (from Dilbert) after she worked frantically on urgent projects that ended up just being cancelled. (That one where Wally smells that lovely buttery-popcorn scent of unnecessary work.)
I worked 13 hours yesterday.
for nothing.
fucking. hell.undefined fuck off we urgently don't need this yet! unnecessary work unsung heroine i'm starting to feel like dark terra.7 -
- WE NEED TO KNOW THE VERSION OF THE SYSTEM THIS INSTANT!
"what? version? wtf are you talking about"
- THE CLIENT HAS I.T. GUIDELINES TO STRICT CONTROL THE VERSION OF EACH SOFTWARE VENDOR'S SYSTEMS!
"We are not a 'software vendor', we provide them consulting on logistics!"
- THEY USE OUR WEBSITE! THIS MAKES US A SOFTWARE VENDOR!
"Wouldn't that make 'google' their vendor too?"
- IM SURE THEY STRICTLY CONTROL GOOGLE'S VERSION TOO!
"I'm pretty sure they don't. But, whatever, that do answers the question of what they want. Some paperwork jockey wants a meaningless number to fill a form, let's give'em one"
I just had someone make an API endpoint where they can ask "the version", and it is just the number of commits in our production branch. For lols, we even 0-fill and split every three magnitude orders with a dot, so we're in version 0.012.345 or something.
Major version upgrade every million commits!
Fuck those guideline-parrots who are unaware that words sometimes have meaning, and sometimes not.7 -
I recently joined this big MNC after shutting down my own startup. I was trying to automate their build process properly. They were currently using grunt and I favor gulp, so I offered to replace the build process with gulp and manage it properly.
I was almost done with it in development environment and QA was being done for production.
In the meantime I was trying to fix some random bug in a chrome extension backend. I pushed some minor changes to production which was not going to affect the main site. That was in the afternoon.
This Friday my senior rushed to me. It was like he ran six floors to reach me. He asked, did you push the new build system to production, I refused. He then went to the computer nearby and opened the code.
It was Friday and I was about to leave. But being a good developer, I asked what's the problem. He told me that one complete module is down and the developers responsible for them left for the day already and are unreachable.
I worked on that module multiple times last month, so I offered my help. He agreed and we get to work.
The problem was in the Angular front end. So we immediately knew that the build process is screwed. I accidentally kept the gulp process open for anyone, so I immediately rebuilt using grunt and deployed again, but to no success.
Then I carefully analyzed all the commits to the module to find out that I was the one who pushed the change last. That was the chrome extention. I quickly reverted the changes and deployed and the module was live again. The senior asked, how did you do that? I told the truth.
He was surprised that how come that change affect the complete site too. We identified it after an hour. It was the grunt task which includes all the files from that particular module, including chrome extension in the build process.
He mailed the QA team to put Gulp in increased priority and approved the more structural changes, including more scrutiny before deployment and backup builds.
The module was down for more than 5 hours and we got to know only after the client used it for their own process. I was supposed to be fired for this. But instead everyone appreciated my efforts to fix things.
I guess I am in a good company 😉4 -
I am UI developer. The fucking domain model which we have in our project changes very frequently (once in a week (f**k)).
There is one person who is backend developer in this project. I call him TATTI because code he wrotes is "maha tatti".
Now real story,
Whene ever he makes any chnages to, so called, very complex domain model, he even doesn't inform me to reflect changes in UI. So whenever it goes on production, our boss abuse me for not updating UI.
Fucking tatti, just message me what should be changed or not and I really don't understand why they need changes in domain model every week.
F**g cto, decide domain structure first and then tell your team to start implementing it.
Other thing, when ever I ask tatti any doubt about any field is needed to display in UI or not because it is removed or added in domain model, he fucking chutiya starts explaining me whole project instead of just telling yes or no.
I don't care what your fucking backend does. Just tell me what is needed or not. And also he is not sure whether to keep that field or not even he had made that change.
He tells, "ha ye field chahie (after 5 seconds) , nahi wese to hum kahi use nahi kr rhe, ye sirf furture ke liye rkha tha use hta do (after 5 seconds), ek kam kro Mr xyz(another back end developer) se puch lo"
Fuck you maha tatti. And due to this i left that project and switched to other.19 -
Advice to all new programmers, take this one from personal experience. DO NOT PUT SWEAR WORDS IN DEBUG STATEMENTS.
You will miss one, it will go to production and it will get picked up by your log monitoring...2 -
!rant
Sad to make my first post here a depressing one, but I really hope that some of you have some tips to help in this line of work.
If anyone of you suffer from depression, how do you cope with it? How do you keep yourself motivated and don't start this self-loathing that I'm currently in? Other than antidepressants or therapy (already have meds).
Why I'm asking is because I have a very tough time getting motivated these days and right now I really need to be most active. I need to do a lot of small and big stuff at my work and at the same time try to graduate from school. The deadline for my thesis is at the start of May, which surely seems far away now, but it does not feel like enough.
The more I understand the systems that I'm working with, the more I can see how much I may have f*cked everything up and I build this never-ending list of tasks for myself in my head to try and fix everything. Which leads to a complete lockup with anxiety and I can't get anything done.
I don't believe in myself or my code anymore. I'm afraid of pushing anything to production. I also don't have anyone else to help me with my work, as I'm the only developer in the company (we have a service provider where most of the big stuff happens).
To add to all this, I have been sick for the last 4 days.
I truly am in a bad place right now.22 -
Rather than singling out one person, I wanna present what I see as incompetent/stupid/ignorant:
- no will to learn
- failure to follow the very specific instructions & later asking for help when they FUBR sth & not even knowing what they did to fuck up in the first place
- asking how to solve stuff, then ignoring the suggestions & doing sth totally against recommendations
- failure to remember most basic stuff, especially if not writing it down to look at later when needed
- failure to check logs & 'google' stuff before asking why something isn't working the way they want it
- after two weeks, asking me how feature xy works, mind you they coded it, not me
- asking me why they did something in a specific way - WTF, am I a mind reader?! Who designed that crap?! Me or you?!!
- being passive/aggressive & snarky when told to do something or being asked why isn't it done already
- not testing their shit properly
- not making backups when upgrading (production) servers
- not checking the input value, no validation.. even after many many debacles on production with null ref exceptions
- failure to admit they fucked up
- not learning from (their) mistakes8 -
[3:18 AM] Me: Heya team, I fixed X, tested it and pushed to production. Lemme know what you think when you wake up.
[6:30 AM] Me: Yo, I just checked X and everything is peachy. Let me know if it works on your end.
[9:14] Colleague A: Whoop! Yeah! Awesome!
[9:15] Boss: Nice.
[9:30] A: X doesn't work for me.
Me: OK, did you do M as I told you.
A: yes
Me: *checks logs and database, finds no trace of M*
Me: A, you sure you did M on production? Send me a sreenshot plz.
A: yeah, I'm sure it's on production.
Me: *opens sreenshot, gets slapped in the face by https://staging.app.xyz*
Me: A, that's staging, you need to test it on production.
A: right, OK.
[10:46] A: works, yeah! Awesome, whoop!
[10:47] Boss: Nice.
Me: Ok! A, thanks for testing...
Me: *... and wasting my time*.
[10:47:23] Boss: Yo, did you fix Y?
Courageous/snarky me: *Hey boss, see, I knew you'd ask this right after I fixed X knowing that I could not have done anything else while troubleshooting A's testing snafu since you said 'Nice' twice. So, yesterday, I cloned myself and put me to work in parallel on Y on order fulfill your unreasonable expectations come morning.*
Real me: No, that's planned for tomorrow. -
I didn't scream.. just told him to jump off of terrace..
What ticked me?! He was a support guy..slowest mofo ever..
I was in the middle of fixing major fuckup on prod, when our VPN to client disconnected. I rushed over to support to ask if it is 'just' an expired session (which he was in charge of renewing but constantly fucked up) or if there is some other problem, so I know how to proceed..do I need to contact our sysadmins, client's support guys etc..
He
started
to
slooooooooowly
explain
I
am
not
the
only
one
with
VPN
problems
...
Was that what I asked you?! // he had an annoying habit of slooooowly talking and explaining unrelated things & personal stuff that bothered him & most of the times he chose the most time sensitive period to drone off..
So I cut him of saying, that others were probably not 'tinkering' with production and that I need this back ASAP, so if he could tell me when the session will be renewed or if there is something else problematic..
He said he will check..I didn't move.. he looked at me insurprise, you want me to check *NOW*?! Yeah, it's urgent.. He proceeded very very veeeery slooooowly, taking the support phone../* he was even eating sandwich during that, so only one hand free, typing one letter at a min */
I was finaly notified that the session expired and that he will fix it soon (meaning in 15-20mins o.O which should not take him more than 5).. and was like 'can I do sth else for you'?! Yeah, do the backflip.. you know the rest..3 -
An excerpt from the best rant about whiteboard interviews posted on the internet. Ever.
"Well, maybe your maximum subsequence problem is a truly shitty interview problem. You are putting your interview candidate in a situation where their employment hinges on a trivia question. — Kadane's algorithm! They know it, or they don't. If they do, then congratulations, you just met an engineer that recently studied Kadane's algorithm.
Which any other reasonably competent programmer could do by reading Wikipedia.
And if they don't, well, that just proves how smart the interviewer is. At which point the interviewer will be sure to tell you how many people couldn't answer his trivially simple interview question.
Find a spanning tree across a graph where the edges have minimal weight. Maybe one programmer in ten thousand — and I’m being generous — has ever implemented this algorithm in production code. There are only a few highly specific vertical fields in the industry that have a use for it. Despite the fact that next to no one uses it, the question must be asked during job interviews, and you must write production-quality code without looking it up, because surely you know Kruskal’s algorithm; it’s trivial.
Question: why are manhole covers round? Answer: they’re not just round, if you live in London; they're triangular and rectangular and a bunch of other shapes. Why is your interview question broken? Why did you just crib an interview question without researching whether its internal assumption was correct? Do you think that “round manhole covers are easier to roll" is a good answer? Have you ever tried to roll an iron coin that weighs up to 300 pounds? Did you survive? Do you think that “manhole covers are circular so that they don’t fall into manholes” is a good answer? Do you know what a curve of constant width is? Do you know what a Reuleaux triangle is? Have you ever even been to London?
If the purpose of interviewing was to play stump the candidate, I’d just ask you questions from my area of specialization. “What are the windowing conditions which, during the lapping operation on a modified discrete cosine transform, guarantee that the resynthesis achieves perfect reconstruction?” The answer of course is the Princen-Bradley condition! Everyone knows that’s when your windowing function satisfies the conditions h(k)2+h(k+N)2=1 (the lapping regions of the window, squared, should sum to one) and h(k)=h(2N−1−k) (the window should be symmetric). That’s fundamental computer science. So obvious, even a child should know the answer to that one. It’s trivial. You embarrass your entire extended family with your galactic stupidity, which is so vast that its value can only be stored in a double, because a float has insufficient range:"
Author: John Byrd
Src: https://quora.com/What-is-the-harde...3 -
Old boss story. This guy was nice but a terrible boss. Also relevant, he has a background in IT so should know better.
Him: So when you wanna check a password is correct you just unhash it in the database?
Me: *facepalm*
Me: Hey we should be doing unit and integration testing at a minimum to lower bugs.
Him: We don't need those, we're not a bank. If a problem comes up we just fix it and push to production.
(A while later)
Him(in email): Why do we keep getting bugs reported. Don't you devs test your code.
I was mildly annoyed at that one.
Him: We're always over budget on projects, how can we fix this.
Me: What if we increase our quotes.(technically there are other ways as well but not really possible at that time)
Him: We can't do that, clients won't want to pay.
Me: *finishing off my handover as I'm leaving for a new job*
Him: Wow you do a lot of work2 -
I was straight out of uni so I didn't mind recruiters getting in touch, I just wanted a job.
The whole conversation was just generally awkward, it was just as if he hated his job, being there, and me. He had a wordpress project lined up which I was extremely ambivalent about, but I didn't want to rule anything out. We talked about my experiences with VueJS, React, material styling, how I had done a project which is live in production more or less solo fullstack, and some of my side projects.
He nods knowingly through it all while taking notes, before asking if I had heard of javascript or if I'd be willing to look at it until next time.
That's when you just get struck with this sense of "what can I do to leave this room as soon as possible without making a scene".2 -
Ok so this happend in the last 3 days, I didn't post it till now because I had to seriously take a rest with all the bullshit and stress that came with it...
(Legacy project I have the lead in called: "Foo")
Monday:
Management decided it would be effective to add a senior and a junior to Foo, which would make (together with me) to be 2 juniors and one senior developer
Well I've spend most of that day helping both the junior and the senior to setup "Foo" on their local development machines... So I could not do any programming myself
tuesday:
The senior wanted to refactor EVERYTHING... and I had to stop him multiple times because we simply do not have the time to do that...
The junior tried to work on other things as much as he could, and after he had run out of things to do, asked me for EVERYTHING... EVEN WHERE TO FUCKING CHANGE SOME GOD DAMN STRINGS!....
Also he did in total 3 commits, two of which existed of my code (because I had to "help" him
wednesday:
Both the junior and senior were removed from the project and I got another senior.. who fucking deleted the production database on accident
god damn rough few days man...7 -
This happened via mail thread today.
Boss: we need this new brilliant feature I just made up and running asap! Top priority, it has to be done well, for my reputation is on the line!
Me: *looks at the specifics* 'kay, looks easy enough, this evening max and it will be ready. I just nees some extra info about what kind of data validations (I speak no accountant) are needed, and some other details (a total of 3 questiona).
B: Sure! Remember, it needs to be perfect, as my reputation is at stake. Call me on the phone and I'll give you the details!
M: Can't you answer via mail? Thua way both me and the other devs will have clewr guidelines
B: Just call me! Why do you need it to be written down? It's faster this way!
...Fine. I'll keep asking until you're ready to give me a written answer to my questions. No way I'll take security details via phone for something you want in production this evening. No chance in Hell I'll take responsibility for "misunderstanding" what you said on the phone. Why does it always has to be like that?8 -
Assigned to a new project team..
Using git, in a creative way. So.. "master" is "dev" branch, usually. Everyone can push their branch to dev server .. so it's "dynamic for us". Production branch is whatever, as long as the branch has the release version. Sometimes, the release comes from "master".. that mean "dev" in normal geek..
That's just Git. The source code is a saturated spagetti of Entity framework and Caliburn. It is littered with antipatterns, especially basebean. Holy Christmas and Easter that baseclass do a lot of stuff that has no place as a base class ..
Fucking frameworks, I'm gonna start to evangelize frameworks as the no1 antipattern.
MS SQL as the main DB, but is dumped to json FILES through a scheduled task to increase read performance on web.
There is a soap endpoint to expose the json files, fml..
I am assuming I was placed here to improve stuff, I have never in my life seen anything like this before.
There is a special place in hell for this repository7 -
Hey Root, remember that super high-priority ticket that we ignored for five months before demanding you rewrite it a specific way in one day?
Yeah, the new approach we made you use broke the expected usecases, and now the page is completely useless to the support team and they're freaking out. Drop everything you're doing and go fix it! Code-complete for this release is tonight! -- This right after "impacting our business flow" while being collapsed on the fucking floor.
Jesus FUCKING christ, what the fuck is wrong with these people?
If I dropped the ball on a high-priority ticket for two weeks, I'd get fired, let alone for five fucking months.
If I was a manager and demanded a one-day rewrite I can only imagine the amount of chewing out I'd receive, especially on something high-priority.
And let's not forget product ownership: imagine if I screwed up feature planning for someone so badly I made them break a support tool in production. I'd never hear the end of it.
Fucking double standards.
And while I'm at it. Some of the code I've seen in this codebase is awful. Uncommented spaghetti, or an unreadable mess with single-letter variables, super-tightly coupled modules so updates are nearly impossible, typos in freaking constants added across sixty+ files, obviously-incorrect comments, ... . I'll have to start posting snippets to show them off. But could I get away with any of it? ha. Hell no. My code must be absolutely perfect. I hear about any and every flaw, doesn't matter how minor, and nothing can go out until everything is just so.
Hell, I even hear about flaws in other peoples' code during my code reviews. Why? Because I should have fixed it, that's why. But if I do, I get yelled at for "muddying the waters."
Just. JESUS FUCKING CHRIST.
It's like playing a shell game where no matter which shell I pick (or point to their goddamn sleeve where they're clearly hiding it), I get insulted for being so consistently useless, and god damn, how can I never find the fucking pea or follow the damned rules? I'm so terrible and this is why "nobody trusts me." Fuck you.
I'll tell you why I can't find your damned pea: IT'S RATTLING INSIDE YOUR FUCKING HEADS, you ASSHOLE FUCKING IMBECILES.
That's right: one pea among the lot of them.
goddamn I am fucking pissed off.rant drop everything and rewrite your rewrite oopsie someone else made a mistakey double standards shell game root can do no right root swears oh my8 -
So, a few years ago I was working at a small state government department. After we has suffered a major development infrastructure outage (another story), I was so outspoken about what a shitty job the infrastructure vendor was doing, the IT Director put me in charge of managing the environment and the vendor, even though I was actually a software architect.
Anyway, a year later, we get a new project manager, and she decides that she needs to bring in a new team of contract developers because she doesn't trust us incumbents.
They develop a new application, but won't use our test team, insisting that their "BA" can do the testing themselves.
Finally it goes into production.
And crashes on Day 1. And keeps crashing.
Its the infrastructure goes out the cry from her office, do something about it!
I check the logs, can find nothing wrong, just this application keeps crashing.
I and another dev ask for the source code so that we can see if we can help find their bug, but we are told in no uncertain terms that there is no bug, they don't need any help, and we must focus on fixing the hardware issue.
After a couple of days of this, she called a meeting, all the PMs, the whole of the other project team, and me and my mate. And she starts laying into us about how we are letting them all down.
We insist that they have a bug, they insist that they can't have a bug because "it's been tested".
This ends up in a shouting match when my mate lost his cool with her.
So, we went back to our desks, got the exe and the pdb files (yes, they had published debug info to production), and reverse engineered it back to C# source, and then started looking through it.
Around midnight, we spotted the bug.
We took it to them the next morning, and it was like "Oh". When we asked how they could have tested it, they said, ah, well, we didn't actually test that function as we didn't think it would be used much....
What happened after that?
Not a happy ending. Six months later the IT Director retires and she gets shoed in as the new IT Director and then starts a bullying campaign against the two of us until we quit.5 -
Me : I found this code issue, I think we need to fix it
PO: does it affect the user?
Me: not really but we can make it better
PO: do you have a defect for it in *insert issue tracker here*
Me: no, I just noticed it
PO: is there an IM ticket for it?
Me: I don't think so
PO: is this issue already in production?
Me: possibly. Yes. That's why I was wondering if we should fix it.
PO: okay then we will fix it in the 3rd release from now if you still remember it by then.5 -
It's about a guy that knows better.
I was working as a subcontractor on a bigger system. We (subs) were not allowed to deploy code, we had to wait for contractor to deploy.
One day I got an email that my code is bugged and that my feature is not working on production. I checked it on test env, everything was fine. Then I checked if the code I wrote was deployed. It was not.
I send an email explaining that if they deployed my code it would be working. Then I got a response. There was a bug in my code.
Another email. I asked how would they know? Do they have a test on their environment that failed?
No. There is one guy that READ my code and he said it should not work, so he will not deploy it. He was not a programmer, he was a business consultant responsible for the documentation.
His issue was that I used a function that was not in a class. So if the function is not declared it's obvious it will not work. I had to explain to him in another email, that you can use object of another class inside your class and then call a function, that is not in your class. It was the last time this guy blocked my deploy.
TL;DR, I had to explain a non-dev how object composition works in order to have my code deployed. Took four emails.4 -
Why do we have code reviews:
To stop stupid crap like column names with mis-matched cases:
* FirstName
* Lastname
Now it's in the production APi and nobody can change it.8 -
Well the clown strikes again,
How do u break production and a testing environment in one night?
One full month preping for same thing that revolves around one config file and assured us he was confident,
He wasn't
he managed to fuck it up so bad for the team d brass lost d plot,
I'm not one for condemning people but my God Dante's inferno woulda had an extra ring if he worked with this buck,
The stupidity has shattered my belief in sunshine and rainbows -
So my department is "integrating CI/CD"
Right now, there's a very anti-automation culture in the deployment process, and out of our many applications, almost none have automated testing. And my groups is the only one that uses feature branching - one of the few groups that uses branching at all beyond "master, dev"
So yeah... You could see how this is already ENTIRELY fucked from the very beginning.
First thing they want to do is add better support for a process... Which goes directly against CI/CD.
The process is that to deploy to production (even after it is manually approved by manager), someone in another department needs to press a button to manually deploy. This, as far as I can tell, is for business rule reasons rather than technical ones.
They want us to improve that (the system will stay exactly the same with some streamlined options for said button pressers)
I'm absolutely astounded at the way our management wants to do something but goes in exactly the opposite direction. It's like the found an article of what CI/CD was and then took notes on exactly what not to do.25 -
I just love it when our clients decide to make a clone of live production server..then put it immediately online..and don't tell anyone about this.. and then start bitchin how data gets doubled all of a sudden..
Yeah, no shit sherlock.. you have two prod servers for 'hot swapping' and some services may only be running on one at a time.. You even have a manual on how to switch primary to secondary (turn off services on primary first, then turn them on on secondary and all)..or in case primary actually dies, just turn on services on secondary and you're good to go, right?
So how do ya think cloning the one with running services and putting the clone immediately online will work out?! 🤔
God, I thought it was common sense to not do that..but here I am, bitchin about how people fail to RTFM.. :/ or use brain..fuck..4 -
Hiring process is fucking broken ok?
We all do have something else to do, nobody wants to do "homework" for 4 fucking hours. Which let's be real, isn't 4 hours. It's always more. I try to squeeze it in a least amount of time which means mistakes will be made. I always try to show my knowledge of the language and it's features. But, you didn't do X. That's it, that is a no from us.
Dude, I just wrote a high production grade small project with 90%+ test coverage and you are telling me that those 2 small shit I made is a big deal? Fuck off
Most companies I worked with have a code full of shit and here I present to you, with a poetry and it's a no because of X?
My bet is that if I ever started to work there I would find a code that isn't tested and is in shit state
\rant4 -
!rant
For the past two years I've always wanted to make Programming tutorial videos to help others learn to code while fueling my passion for coding, discovery, and teaching..... and after two years I've finally uploaded my first two videos to YouTube.
I want to cover fun and exciting topics such as how to make custom plugins, create your own linux web server, and more... but decided to do a web basics 101 as my "Hello World" videos to get better in making content and production.
The inspiration for my "Web 101" comes from have a lot of my senior year CS classmates who have never seen HTML/CSS code before and wanting to provide them a source to get the basics all in one place.
I have a lofty goal of getting 10 subscribers by the end of the month. If you wouldn't mind giving me some pinpointers or comments I'd greatly appreciate it!
Also I did buy a new microphone so the sound quality between video one and two should be better!
https://youtube.com/channel/...12 -
Because I’m a fucking cowboy and a charlatan, and because I hate sleep and despise feeling refreshed and happy, I’m working pretty much full time as a contractor (I’m the full stack dev. I do everything) on a (well funded) startup alongside my day job.
Tonight I had to make some quick (lol “quick”) changes to a core piece of the platform.
Now before continuing please refer back to the first line of this rant.
So instead of writing new functionality, I copied and pasted another section.
I renamed all references of “new_order” to, cleverly “new_order2”.
I know.
I deploy to production...
My phone starts blowing up. In short, everything is fucked.
I’m going over the query, checking the production database. Why is this manifesting like this? It all looks correct.
2 HOURS of broken sales, pissed off customers, pissed off service agents and I see that there was still one reference of “new_order” that should have been “new_order2”.
I am a piece of shit.4 -
The cleaning lady
Yeah, I should not be at work this late, but it happenes from time to time..
Anyhow, she still doesn't know where the switches for each light are, so whenever she comes, lights start to flicker on off on off... Yeah, lightshow baby!! //read this in austin powers voice
I fear for people with epilepsy.. Yeah, I get it, push and toggle switches are mixed in pannel, but we do have a schema for each of them..ofc she doesn't use it..
And every goddamn time if I work later than when she leaves, she disturbes me to ask if I will turn off the lights when I leave.. NOoOooo, I will leave them on like an asshole.. :/ I once even wrote on the whiteboard in bigg letters 'yes, I will turn of the lights when I leave' and left it there for some days... coworkers were all puzzled/amused when they saw it.. //I know, shitty attitude on my part too, but I've repeated myself a dozen times already.. + I always turn off lights she actually manages to switch on and leave them like that, even the ones not in my part of office
And when she vacuums..well.. I usually get up and go on a cigarette break to give her space so she can do her job around my desk.. but she insists that I am not in her way and I don't need to leave... Well I prefer not to be proded and poked with the hose, thank you..
So yeah, fun times... O.o especially if working late due to some production fuckups I need to fix ASAP.. :/19 -
Refactored an authentication library a while back and teams are now getting around to updating their nuget packages.
It is a breaking change, but a simple one. The constructor takes a connection string, application name, and user name.
A dev messages me yesterday saying ...
Tom: "I made the required changes, but I'm getting a null reference exception when I try to use the authorization manager"
Odd because the changes have been in production for months in other apps, so I asked him to send me a screen shot of how he was using the class (see attached image below).
Me: "Send me a screenshot of how you are using the class"
<I look at what he sent>
Me: "Do you really not see the problem why it is not working?"
<about 10 minutes later>
Tom: "Do I need to pass a real connection string? The parameter hint didn't say exactly what I should pass."
<not true, but I wasn't going to embarrass him any more>
<5 minutes later>
Tom: "The authorization still isn't working"
Me: "Do you still have 'UserName' instead of the actual user name?"
<few minutes later>
Tom: "Authorization is working perfect, thanks!"
A little while later my manager messages me..
B:"I'm getting reports from managers that developers are having a lot of problems with the changes to the authorization nuget package. Were these changes tested? Can you work with the teams to get these issues resolved as soon as possible? I want this to be your top priority today."
Me: "It was Tom"
B: "Never mind."11 -
Adding a feature to webapp...
Webapp relies on database in production server...
*adds feature to production webapp directly*
Every page: ERROR 500
Manager: what did you do???!!!! You MESSED UP the production, FIX IT NOW
*Use ctrl-z because manager doesn't like Version Control*5 -
Debate (with rant-ish overtones):
FYI, while it is a debate, its a practiseSafeHex debate, which means there is a correct answer, i'm just interested in your responses/thoughts.
Ok lets kick off. So the remote team I work with had an opening for a new iOS developer (unrelated to anything to do with me). They interviewed and hired a guy based off his "amazing" take home challenge.
The challenge consists of 4 screens and was for a senior level position. For the challenge the interviewee created a framework (a iOS library) for each screen, included all the business logic for each screen inside, each one needs to be built separately, exposed some API/functions from each one and then created a main project to stitch it all together.
Now, my opinion is, this is highly unscalable and a ridiculous approach to take as it would add so much unnecessary overhead, for no benefit (I am correct btw).
The interviewee said he did it like this to "show off his skills and to stand out". The remote team loved it and hired him. The challenge said "show us the code standard you would be happy to release to production". I would argue that he has only demonstrated 1 extra skill, and in exchange delivered something that is unscalable, going to be a nightmare to automate and require huge on-boarding and a paradigm shift, for no reason. To me thats a fail for a senior to not realise what he's doing. This person will be required to work alone (in part), make architecture decisions, set the foundation for others etc. Having someone who is willing to just do mad shit to show off, is really not the type of person suited to this role.
Debate!11 -
So you want full stack engineers to: design, do UX, create front end, build backend and deploy it in your mono repo stupid manual deployment "kubernetes cluster", add monitoring alerting manually, review others PR, QA our own apps and features, manually sync to Production, use VPN otherwise we cannot connect to anything, 2factor auth, do SRE, architecture diagrams, demo, run agile ceremonies, and learn a legacy coding language which was never mentioned in the job description. Did I miss anything?7
-
One of our senior dev enjoys berating the other devs because they don't check-in code according to his schedule (once a day, once an hour..he flip-flops a lot), then when they do, he 'reviews' their code, beating them up because of incomplete features, commented out code..petty..petty nonsense.
Ex. (this occurred couple of weeks ago).
Ralph: "The button click code in this event isn't complete"
Dev: "No, its not, the code in my development branch. You said it was best practice to check in code daily whether the code worked or not. I didn't finish the event last night and ..."
Ralph: "Exactly. Before you check any code into source control, it has to work and be 100% complete. What if someone moved that code into production? What happens if that code got deployed? I'm not even going talk about the lack of unit tests."
Dev: "Uh..well..the code is on the development channel, and I branched the project in my folder ...I didn't think it mattered.."
Ralph: "Ha ha...you see what happens when you don't think...listen..."
- blah blah blah for 10 minutes of hyperbole nonsense of source control check-in 'best practice'
This morning Ralph's computer's hard-drive crashed.
Ralph: "F-k! ..F-k! ... my f-king computer hard drive crashed!"
Me: "Ouch...did you loose anything important?"
Ralph: "A f-king week of code changes."
Me: "You checked everything into source control on Friday ...didn't you?"
Ralph: "F-k no!...I got busy...and...f-k!"
Me: "Look at the bright side, you'll have a good story to tell about the importance of daily check-ins"
Oh...if looks could kill. Karma...you're the best. -
Most ignorant ask from a PM or client?
Migrated to SharePoint 2016 which included Reporting Services, and trying to fix a bug in the reporting services scheduler, I created a report (aka, copied an existing one) 'A Klingon Walks Into a Bar', so it would first in the list and distinct enough so the QA testers would (hopefully) leave it alone.
The PM for the project calls me.
PM: "What is this Klingon report? It looks like a copy of the daily inventory report"
Me: "It is. The reporting service job keeps crashing on certain reports that have daily execution schedules."
PM: "I need you to delete it"
Me: "What? Why? The report is on the dev sharepoint site. I named the report so it was unique and be at the top of the list so I can find it easily."
PM: "The name doesn't conform to our standards and it's confusing the testers."
Me: "The testers? You mean Dan, you, and Heather?"
PM: "Yes, smartass. Can you name the report something like daily inventory report 2, or something else?"
Me: "I could, but since this is in development, no. You've already proofed out the upgrade. You're waiting on me to fix this sharepoint bug. Why do you care what I do on this server? It's going away after the upgrade."
PM: "Yea, about that. We like having the server. It gives us a place to test reports. Would really appreciate it if you would rename or delete that report."
Me: "A test sharepoint reporting services server out of scope, so no, we're not keeping it."
PM: "Having a server just for us would be nice."
Me: "$10,000 nice? We're kinda fudging on the licensing now. If we're keeping it, we will be required to be in compliance. That's a server license, sharepoint license, sql server license, and the dedicated hardware. We talked about that, remember?"
PM: "Why is keeping that report so important to you? I don't want to explain to a VP what a Klingon is."
Me: "I'm not keeping the report or moving it to production. When I figure out the problem, I'll delete the report. OK?"
PM: "I would prefer you delete the report before a VP sees it."
Me: "Why would a VP be looking? They probably have better things to do."
PM: "Jeff wants to see our progress, I'll have to him the site, and he'll see the report."
Me: "OK? You tell Jeff it's a report I'm working on, I'll explain what a Klingon is, Jeff will call me a nerd, and we all move on."
PM: "I'm not comfortable with this upgrade."
Me: "What does that mean?"
PM: "I asked for something simple and I can't be responsible for the consequences. I'll be documenting this situation as a 'no-go' for deployment"
Me: "Oookaayyy?"
I figured out the bug, deleted the 'Klingon' report, and the PM couldn't do anything to delay the deployment.4 -
Hey Designer/Developers, I got a question for you. Yeah, you 👇🏽
When working on a project codebase that is expected to grow and evolve heavily. How do you usually split up your CSS (SASS, LESS etc) in a good way to take into account all the different device sizes?
I am not asking how it is done but more about the design of the code. This would be for a production codebase to be released.
Do you use large blocks broken down by media...
(Media width) {
~site code
}
(Other media) {
~same site code with diff sizes
}
Or do you do individual media queries inside css classes...
.className {
(Media size) {
}
(Other media) {
}
}
Or a mixture of both?
If it is a mixture of both then how do you decide which way to go about structuring the code.
I have been endeavouring to greatly improve my CSS and have done so. But this question has been bugging me. Both sides seem to be a bit sloppy and my programmer side is fighting the repeatitipve code.
Note: all code examples are gibberish and only intended for visualization.17 -
So, I'm the only iOS developer in the company working on this stupid app which is never going to succeed because the client idea is just plain stupid. Anyway, 6 months developing this crap, countless iterations, trillions of so-called retard "features", and now that we are almost there to go into production my PM decides it's a great moment to document every aspect of what was done and asks me to do it. Now I have 3 documents to write and around 80 bugs to review before Friday. Stupid me for thinking he didn't waste 6 months doing absolutely nothing while I was working 12 hrs a day to meet deadlines.2
-
Have you ever had a problem with a partner, but you cannot prove at 100% that he has something against you? But you cannot stand him?
Well, this happened two years ago. I was working as a tester, and “John” (I won’t say his real name) was the dude who tests my tests, but in production.
I ‘m a sociable person, and I don’t mind talking with another people. Suddenly, I noticed that my co-worker started to behave a little bit... rude? Plain? I don’t know, but sometimes he didn’t answer my conversations/questions, or sometimes answered with extra-negative stuff.
“Well, his life is not easy” I said to myself. “Everybody haveproblems”, “I have to understand him and calm down”.
Two weeks later started to report really REALLY absurd production bugs, and with absurd I mean, for example, that he didn’t like the color of a button, a point next to a phrase, etc., things very very simples, but sometimes he ignored big errors.
Once I had to went out of the city for few hours, and asked to permission to go out. I had pending tests, but I left a document with specifications in case of emergency. Even passwords. Before I could finish the thing in the other city, my partner called me two times, and asking me obvious things! I had to go back to the office ealier that I had planned, so f*** angry 😡 and when I arrived to the office, John said “no, forget it, let’s solve it tomorrow” 🙃 WTF?!!!!!!
I decided not to argue with him.
Also noticed that his headaches suddenly increased, and looked so tired ☹️ I felt guilty to judge him
I felt so guilty, and even today I don’t know what to feel about that or what to think. I don’t work there anymore, but, What do you think? What would you do?6 -
“PHP is evil” is not just a joke.
PHP is usually percieved as a language which is not so consistent and has some opinionated historical aspects but allows rapid development because it’s easy. They say PHP doesn’t focus on that “purist shit” such as concepts and “just gets things done”.
Hovewer, this is not true. PHP lures you in and lies to you promising saving time on development, but everything, and I mean EVERYTHING written in PHP is doomed to turn into a bloody mess sooner or later.
You have to be an AI to manage the growing PHP codebase and add features without breaking anything. With every feature it gets harder and harder. If you’re still a human managing a human team, you have to enforce guidelines. Automatic error preventon measures are made of code themselves so the cost of deploying them ona late stage can be ridiculous. And you never deploy them on early stage because you want to “save time”. Your people have to spend more and more time everyday checking on that guidelines. Your development process only becomes slower and slower. If you try to push things, your project will crumble to dust.
To make PHP at least decent, you have to figure out all this by yourself on an early stage. When you’re done, you spent a lot of time creating the buggy, ad-hoc, unspecified and unsupported alternative of what works out of the box in other languages. And you still code in PHP and still have all its disadvantages in your project’s DNA.
PHP is evil because it promises and never delivers. PHP is evil because it lies to you and it already fucked over so many of us.
If you want to code in PHP, do it under your pillow. Code your own silly projects.
If your project has the word “production” somewhere in its plans, PHP is not the way to go.
Amen.65 -
I started my internship at the end of the year..
Fuck my ass!!! This code I have to work with is a huge pile of shit.
The code base I need to work with is around 40k LOC. It is a mixture of C++, C, Java, Python, Bash and I think I saw some lonely js files around.
A list of awesome parts:
- Paths are hard coded.
- Redundant code everywhere
- No documentation or inline comments available
Most of the comments in the code are just old code that is not used anymore. But the cherry on the turd is the class that should provide all kind of useful functions in my daily routine. About ninety percent of the functions have the same description or nothing. Sometimes a function name says "readSomethingFromSomewhere" but instead it writes something to a file. It is really confusing and I need to check everything twice instead of rely on what the function name promises.
I have also learned why copy paste isn't that good. The brief descriptions of every method in a files are always the same.
getName() - Description: Fork child process
getIp() - Description: Fork child process
getIpv6() - Description: Fork child process.
Surprise: None of these functions forks a child process. :D
Another awesome feature is the thing that they store up to five different versions of libraries. Everyone with slight modifications but no hint which one you need to use. Sometimes it is the newest, sometimes the oldest which is running in production. Another case of try and error.
Oh and my dev machine is a potato with a power supply and a fan. I started with NetBeans and every time I compiled the code it sounds like the machine wants to lift off and leave for a better place. (At this point I switched to Emacs and everything runs smoothly now)
At first I thought that I'm just not that good at coding and understanding a big project from scratch but some colleagues have the same problem. The whole system is very inflexible and it is all about "std::cout"-debugging to check if your changes do what you want them to do.
Currently I'm just trying to fix this mess to make the life for the next student or employee easier. The first month was just frustrating as hell. I need to ask so many questions and most of the time the answer was "I don't know, haven't touched this code in years". Needless to say that my progress isn't that awesome but at least I get a nice payment for 20 hours of work a week.2 -
Can't say I am a religious type, not really into discovering/discussing religions and comparing them either... BUT I do hope there is someplace (like hell on steroids!!) for developers who don't test their code before checking it in and/or puting it on production..
Also another question, can I plead not guilty due to insanity when killing such devs?!? O.o FUuuuuuuuu!!!!2 -
I used to work on a production management team, whose job was, among other things, safeguarding access to production. Dev teams would send us requests all the time to, "run a quick SQL script."
Invariably, the SQL would include, "SELECT * FROM db_config."
We would push the tickets back, and the devs would call us, enraged. I learned pretty quickly that they didn't have any real interest in dev, test, or staging environments, and just wanted to do everything in prod, and see if it works.
But they would give up their protests pretty fast when I offered to let them speak to a manager when they were upset I wouldn't run their SQL.2 -
I have discovered a fresh hell
Some guy I’ve never met or heard of in the office lobbed a comment at one of my *approved and merged* pull requests. He doesn’t say anything specific, only that my REST urls are not in line with naming convention. That’s all he says, and I’ve already walked the URL consumers through the code and given them the URLS.
I’m really annoyed that this guy won’t just say what he has in mind, but fine whatever this is a professional environment and developers are not known for being a diplomatic people. Let it go and get your work done!
I do some googling and find an obvious change that needs to happen- I implement it, open a new pull request and inform my URL consumers of the change.
This rando still isn’t satisfied and still won’t say what needs to change. I am on round 3 of this wonderful cycle and this guy is acting all fuckin HAUGHTY about it. “Here is a list of conventions I found googling, you should read them even if it takes 4 hours because it will benefit your career”
Sure dog you’re probably right on that one but we are in a professional environment and at this point you are holding up production so you can wave your dick around! Just SAY WHAT YOU MEAN SO WE CAN MAKE THE CHANGES AND GET OUR WORK DONE4 -
This was some time ago. A Legendary bug appeared. It worked in the dev environment, but not in the test and production environment.
It had been a week since I was working on the issue. I couldn't pinpoint the problem. We CANNOT change the code that was already there, so we needed to override the code that was written. As I was going at it, something happened.
---
Manager: "Hey, it's working now. What did you do?"
Me: *Very confused because I know I was nowhere close to finding the real source of the problem* Oh, it is? Let me check.
Also me: *Goes and check on the test and prod environment and indeed, it's already working*
Also me to the power of three: *Contemplates on life, the meaning of it, of why I am here, who's going to throw out the trash later, asking myself whether my buddies and I will be drinking tonight, only to realize that I am still on the phone with my manager*
Me again: "Oh wow, it's working."
Manager: "Great job. What were the changes in the code?"
Me: "All I did was put console logs and pushed the changes to test and prod if they were producing the same log results."
Manager: "So there were no changes whatsoever, is that what you mean?"
Me: "Yep. I've no idea why it just suddenly worked."
Manager: "Well, as long as it's working! Just remove those logs and deploy them again to the test and prod environment and add 'Test and prod fix' to the commit comment."
Me: "But what if the problem comes up again? I mean technically we haven't resolved the issue. The only change I made were like 20 lines of console logs! "
Manager: "It's working, isn't it? If it becomes a problem, we'll work it out later."
---
I did as I was told, and Lo and Behold, the problem never occurred again.
Was the system playing a joke on me? The system probably felt sorry for me and thought, "Look at this poor fucker, having such a hard time on a problem he can't even comprehend. That idiotic programmer had so many sleepless nights and yet still couldn't find the solution. Guess I gotta do my job and fix it for him. I'm the only one doing the work around here. Pathetic Homo sapiens!"
Don't get me wrong, I'm glad that it's over but..
What the fuck happened?5 -
It's 5 AM and I don't want to shit on anybody's party but trust me when I say most of you here complaining about legacy code don't know the meaning of the word.
As someone who maintained a PHP4 codebase with an average file length of 3000+ lines for almost 4 years, I feel you, I feel your pain and your helplessness. But I've seen it all and I've done it all and unless you've witnessed your IDE struggle to highlight the syntax, unless you had to make regular changes in a test-less SVN's working copy that **is** the production and unless you are the reason that working copy exists because you've had enough of `new_2_old_final_newest.php` naming scheme, you do not know legacy. If you still don't believe me bare in mind I said "is" as in: "this system is still in production".
But also bare hope. Because as much grief as it cost me and countless before me, today of all days, without a warning, it got green lit for userbase migration to a newer platform. And if this 20 years of generous custom features and per client implemented services can be shut down even though it brings more profit than all the other products combined, so can happen to any of your projects. 🙏
Unfortunately, I do mean *any*.7 -
I accidentally started a reindex on a collection that had 14 million records in the middle of the day. Caused an outage in a major portion of our applications for about 3 hours. Worst thing was that once I pressed enter, I realized that it was for the production database, and not the staging database like I intended. I immediately went to go tell the dev ops lead, and he basically said, "whelp, let's just sit back and watch the world burn. Not much we can do about it"1
-
Just deployed version 0.0.1 of my holiday project to production.
It's a pie "attached" to some speakers, mounted below the kitchen cubord. So far it only runs a spotify connect on it and you're able to play music there while cooking/cleaning.
I'd actually like it to also be able to just play regular music, either from a smartphone or a computer in the network. (It's connected by wifi to our lan). Any advice on what software I could install on it to achive that? i'd prefere to enable it for devices on the lan and have bluetouth dissabled, but I'm actually not sure if that is even possible. So yeah any advice to that? Or any other things you'd install on it?
I know, that I need to do some "cable management" back there :P10 -
Why the hell do you keep commit+push your shit to the master branch! We have develop, feature and hotfix, you *************!
Just because you want to test something in prd environment, you don't mess with the master to build the production image. And you do not even rebase fucking develop branch and keep it out of sync you POS!
Excuse my language, thank you.10 -
So... Heard back from a recruiter today. Lovely lass.
I’d passed over a submission for her tech demo.
The brief was basically just to create a small simple module that calculates shit, nae effort.
But, when the recruiter had me on the phone she said “I know it’s a silly small module but try and run it up like you would a production ready app”.
The job spec and recruiter were keen on me demonstrating TDD, not specific on js version, final runtime, etc. The job was a senior spec at a higher salary range. So it warranted some effort, and demonstrating more than a simple module.
“Okay, cool, nae bother, let’s crack on.”
The feedback in the response from the dev today:
“He’s over-engineered tests, build...”
SUCK MY LEFT TESTICLE YOU FUCKWIT.
Talk to your recruiters, not me.
The feedback included a phrase I never hope to hear from a developer I work with:
“Tests are good but...” 😞
It was a standard 98% test suite from an RGR cycle, no more or less than I’d expect in prod.
The rest of the feedback was misguided or plain wrong. It was useful to see because I know now when they say they have “high standards” they mean: we listen to the dude who put the factory pattern in a JS brief.
Oh shit also: “someone’s done chmod 777” was in there as a sarcastic comment in the feedback. It was his fucking unarchive tool 😞
My response was brief and polite: “cheers for the consideration, all the best, James”
It’s honestly not worth warning them. Or, asking why they’d criticise something they’d asked me to do.
If you want a shitty js module, ask for a shitty js module and no more.4 -
Created a variable in the middle of a rant with JavaScript and named it as an acronym for "mother fucker", it actually worked what I tried to do with it... "mother fucker" is in production now2
-
Today was a manic-depressive kind of day. Spent the morning helping some developers with getting their code to run a stored procedure to drop old partitions, but it wasn't working on their end. It was a fairly simple proc. But working with partitions is a little like working with an array. I figured out that they were passing the wrong timestamp, and needed to add +1 to delete the right partition. Got that sorted out, and things were good. Lunch time.
After lunch I did some busy work, and then the PO comes up at about 2PM and says he's assigned some requests to me. The first was just attaching some scripts. Easy. The second, the user wants a couple of schemas exported ... at 6PM. I've been in the office since 6:45AM.
While I'm setting up some commands to run for the data export, a BA walks up and asks if I'm filling in for another DBA who is out for a few weeks. Yep. There's a change request that hasn't been assigned, and he normally does the work. I ask when it's due. Well, the pre-implementation was supposed to be done in the morning, but it wasn't, and we're in the implementation window ... half way through. I bring up the change task, and look at. Create new schema and users. That's all it says. The BA laughs. I tell I need more to go on. 10 minutes later he sends an email with the information. There's only two hours left in the window, and I can only use half of it, because the production guys have to their stuff, and we're in their window. Now I'm irritated, because I'm new to Oracle, and it's an unforgiving mistress. Fortunately, another DBA says he'll do it, so that we can get it done in time. But can't work it either, because Dev DBAs don't have access to QA, and the process required access for this task. Gets shelved until the access issue is resolved. It's now after 4:15PM. I'm going to in traffic with that 6PM deadline.
I manage to get home and to the computer by 5:45PM. Log in. Start VPN. Box pops on screen. Java needs to update. I chose skip update. Box pops up again. It won't let me log in until Java is current. Passed.
I finally get logged in, and it's 6:10PM. I'm late getting the job started. I pull up Putty and log into the first box, and paste my pre-prepared command in the command line and hit error. Command not found. I'm tired, so it's a moment to sink in. I don't have time for this.
I log into DBArtisan and pull up the first data base, use the wizard to set the job, and off it goes. Yay. Bring up the second database, and have enter the connect info. Host not found. Wut? Examine host name. Yep, it's correct. Try a different method. Host not found. Go back to Putty. Log in. Past string. Launch. Command not found. Now my brain is quitting on me. Why now? It's after 6:30PM. Fiddle with some settings, reset $Oracle home. Try again. Yay. It works. I'm done. It's after 7PM.
There is nothing like technology to snatch the euphoria of a success away from you. It's a love-hate thing, but I wouldn't trade it for anything else. I'm done. Good night.3 -
Fuck sake, so my bank has been migrating/rolling out new IT system and app/site have been broken for about a week (others noted evidence of devs debugging in production)
Assuming I don't lose my money as some mischievous assholes will inevitably exploit the fuck up, and rob the bank, I will be moving my funds to a different bank...
In mean time I'm trying to prepare for uni, and they're making a ton of semi-random changes in addition to rolling out a site with course details and info along those line, and good fucking god is it bad.
Is is slow as fuck? Check. Does it use never-seen-before naming for standard things? Check! Is the UI pulled from late 90's? YOOU BETCHA! Are the pages bloated with unnecessary content? Fuck yeah! Do I get SQL exceptions when I finally locate my course? Of course I do. Does clicking "back" take me back to the landing page instead of previous page, when I'm several steps deep? .....
I could keep going, but don't feel like ranting and feel more like punching someone in the throat.repeatedly. -
Okay. So my dumbass boss took this project that had a steep timeline. I told him straight up, it won't work because we won't make the timeline. If we do this, I will be the one bending over backwards to deliver. I don't like to promise and fail. I got the oh don't worry let's just try. If we don't make it that's fine. Unfortunately that's not how I work. I refuse to deliberately fail. So I say okay and we begin. I suggested open source is the fastest way to deliver bit the fucked up part is, I am the only senior dev in the team. I will be expected to reverse engineer the open source app to connect our own deployment parameters. Use tech I have never used before. Connect frontend and backend. Handle dns bullshit. I have literally been working on Vibes and coffee for the past two weeks because ofcourse I ran into so many issues. Now I have an extension for Monday and I hate to fail. So I am not sleeping or resting just working on a fucking java app I didnt build and I am expected to make it work seemlessly on our production environment. I made some progress. Deployed frontend, deployed backend. Forgot to connect production dB so I decided to go with azure database for mysql driver since we have credits on azure. Now my java app is pissing itself over ssl handshake. I generate my keystore and add it and now java socket just times out. I want to pummel somebody or a punching bag that looks like my boss.15
-
Ever dealt with people who don’t understand a damn thing they’re coding and just copy and paste stuff around and say that it works? It’s amazing how long people can skate by with only knowing 50% of how what they’re working on works.
We had some dumb know it all on our team who would regularly ship half-done stuff to production and then scramble to fix it after the users bitched about how it didn’t meet the requirements.
This stupid person changed an else to an if having no idea what any of the logic did in this section of code, didn’t adequately test it, and it somehow made it through code review because the better devs were out of office.
This bullshit goes to production, fucks up 200,000 records, and users start complaining about it. Shitty developer refuses to revert the offending code until multiple people on the team overrule them. They spend the next week unfucking the data and decides to just take a day off on Wednesday because it’s been “too much mental energy to fix.” The shit wasn’t even fully resolved yet.
Some people seriously do not belong in this industry. This person’s thought process was:
“Changing an else to an if can’t possibly have significant consequences. Let’s just change this so my code change executes to see what happens.”
Still not sure how they weren’t fired when this happened. They unfortunately got to quit on their own two months later.3 -
relative: can you help me with my website?
me: ok, what do you need?
them: okay, i have this wordpress homepage that someone made for me and actually, when you go to my image gallery, i only want a simple "back" button that leads to the main page. but i cannot add it via wordpress functionality.
me: okay, i'll make you this button in under 10 minutes, given it doesn't require any fancy design. but i have no experience with messing with wordpress so far... let me check what their templates look like
(after 1 hour): wtf is this mess 😭
...since they only had their production website, i didn't dare to change anything and told them they best hire that dev again 🙈9 -
young user @Mizukuro asked days ago for ways to improving his javascript skills.
I wasn't sure what to say at the moment, but then I thought of something.
Lodash is the most depended upon package in npm. 90k packages depend on it, more than double than the second most depended upon package (request with 40k).
Lodash was also created 6 years ago.
This means lodash has been heavily tested, and is production ready.
This means that reading and understanding its code will be very educational.
Also, every lodash function lives in its own file, and are usually very short.
This means it's also easy to understand the code.
You could start with one of the "is..." (eg isArray, isFunction).
The reason for such choice is that it's very easy to understand what these functions do from their name alone.
And you also get to see how a good coder deals with js types (which can be very impredictible sometines).
And to learn even more, read the test file for that function (located in tests/<original file name>.js. For the most part they are very readable and examples of very good testing code.
Here's the isFunction code
https://github.com/lodash/lodash/...
Here's the test for isFunction
https://github.com/lodash/lodash/...
The one thing you won't learn here is about es5, 6, or whatever.3 -
Sat down with the Project Management team today to discuss a signage installation. This is how the conversation went...
Me: Right, so we need to get the hardware on-site asap so we can get this configured before it goes over to the production guys to have the facisa installed.
Them: That's fine we have plenty of time. Stop rushing things.
Me: Okay, so do we have the story board in place ready for development?
Them: Nope. Hasn't been done by the designer yet because he is in a bad mood.
Me: Okay so when does the client want this?
Them: 3 Weeks' time
Me: But it is atleast a week of dev time?
Them: Sure. But you can work late if needed...right?
This is a typical conversation between them and me. I'm the sole developer here. So done with today.12 -
So after 6 months of asking for production API token we've finally received it. It got physically delivered by a courier, passed as a text file on a CD. We didn't have a CD drive. Now we do. Because security. Only it turned out to be encrypted with our old public key so they had to redo the whole process. With our current public key. That they couldn't just download, because security, and demanded it to be passed in the fucking same way first. Luckily our hardware guy anticipated this and the CD drives he got can burn as well. So another two weeks passed and finally we got a visit from the courier again. But wait! The file was signed by two people and the signatures weren't trusted, both fingerprints I had to verify by phone, because security, and one of them was on vacation... until today when they finally called back and I could overwrite that fucking token and push to staging environment before the final push to prod.
Only for some reason I couldn't commit. Because the production token was exactly the same as the fucking test token so there was *nothing to commit!*
BECAUSE FUCKING SECURITY!5 -
Discovered pro tip of my life :
Never trust your code
Achievements unlocked :
Successfully running C++ GPU accelerated offscreen rendering engine with texture loading code having faulty validation bug over a year on production for more than 1.5M daily Android active users without any issues.
History : Recently I was writing a new rendering engineering that uses our GPU pipeline engine.. and our prototype android app benchmark test always fails with black rendering frame detection assertion.
Practice:
Spend more than a month to debug a GPU pipeline system based on directed acyclic graph based rendering algorithm.
New abilities added :
Able to debug OpenGL ES code on Android using print statement placed in source code using binary search.
But why?
I was aware of the issue over a month and just ignored it thinking it's a driver bug in my android device.. but when the api was used by one of Android dev, he reported the same issue. In the same day at night 2:59AM ....
Satan came to me and told me that " ok listen man, here is what I am gonna do with you today, your new code will be going production in a week, and the renderer will give you just one black frame after random time, and after today 3AM, your code will not show GL Errors if you debug or trace. Buhahahaha ahhaha haahha..... Puffff"
And he was gone..
Thanks satan for not killing me.. I will not trust stable production code anymore enevn though every line is documented and peer reviewed. -
Fucking Square Enix Website is just a huge pile of shit. NOTHING WORKS!
Wanna change your password? Nah sorry an error occured.
Wanna change your username? Nah sorry I'll just show you a loading symbol forever.
Wanna add a game to your collection? Nah sorry the "add" button is on fucking holiday and doesn't do anything.
Wanna change your avatar? Nah sorry I'll just redirect you and don't do anything.
Most amazing part is where you log in, then get redirected to the home page but it still shows the "Log in" button. Then you click on that "Log In" button and wosh! Home page reloads and tada! You're logged in!
Seriously who let this code into production? Also I know that you're using GraphQL now, due to an error message. Thank you!
Fucking bullshit...6 -
I once agreed to maintain and develop an application used in a different section of the school to keep inventory and make sure everything is where it is supposed to be.
At first there was enthusiasm, together with 2 of my classmates we agreed and git clone-d the .NET application that now graduated students built and maintained for the past few years. What could go wrong right?!
It became clear that the original students that worked on it followed an older curriculum, meaning they still got taught .NET instead of the core variant that we get now, not only that but it also seemed that they either did not fully grasp the Clean/Onion architecture or didn't get it in class since there were infrastructure components in the 'Domain' project of the solution. Think of 2 DBContexts in the domain model, yep.
One of us bailed in the first week, the other one and I felt bad for the people using the app so we went on and tried to work on the first bugs that were described in a document. One of these bugs was 'whenever I filter on something in the list, everybody gets to see that filter on their screen instead of only me'. Woah that's weird! Let's see how they put that together!
Oh god, they are using a _static_ variable to store filters, no wonder that it doesn't work properly. Ever heard of sessions?!
Second bug: Sometimes people can't create an account when we sign them up from the admin panel. Alright that is weird, let's figure that one out! Wait a second it seems to work in development? What's this about.
Oh wait I can't create an account on production either? Oh that's weird, wait a second... Why do I have to put my e-mail in a form that was sent to me through e-mail? Why is my address not filled in already? OOH, if someone types in the wrong e-mail address (which is easy since our school has 4 variants of the same f*cking e-mail address) it won't work since it can't recognize the user! Brilliant! Remove e-mail input box and make a token/queryparam determine the user account.
Ah that seems good, it's a mess but it seems a tiny bit better now, great! We're making progress and some sweet buck.
Next bug, trillions of 50x errors on random pages, that's a weird one.
Hm everything works in development, that's odd. Is the production data corrupted?
DID I MENTION that in order to get into the system in development we have to load in a f*cking production database backup ON OUR DEVELOPMENT MACHINE and then ask one of the users' password to login to it and create an account for ourselves? Seeding? What's that, right?!
Anyway, back to bug fixing. I e-mail the the people responsible for the app and get a production admin account, oh I also can't ssh into it because of policies so I have to do everything over e-mail and figure out what's causing the errors. I somehow also wonder if they have any kind of virtualization in place, giving students a VM to do that stuff in doesn't seem so weird does it ? Even with school policies?
Oh btw, 'deploying' means sending a .zip file to a guy in another building and telling him how to configure it, apparently this resulted in a missing folder that the application needed to work and couldn't make on its own. This after 2 weeks of e-mailing back and forth.
After 3 months i quit out of despair and sadness, and due to the fact that I just couldn't do it anymore. I separated everything into logical subprojects and let the last guy handle it, he was OK with that and understood why I left.
Luckily, around that time I already had an actual job at a software development company :)3 -
Dev: "I've pushed some code. Give it a code review."
Me: "ok, i'll do it"
<<fast forward>>
Me: "Sounds good to me. Only thing, I wouldn't have gone for all those renames because that was not part of the request, maybe we can discuss ...."
Dev: "I like those names and besides, it's already deployed in production"
Me: " :| .... what's the purpose of a code review when you push straight into production ?4 -
Looks like Matrix just got educated on hiding administrative stuff behind a VPN, by the guy (or gal, but those don't exist on the internet) that hacked their production infrastructure. Coincidentally, it reminded me of that time when a dev wanted to educate me, a sysadmin, about VPN's 😄
https://devrant.com/rants/2030041
What I've learned from this incident are 2 things.. well mainly 2 things.
1. Never *ever* entrust developers with production access. Let DevOps take care of the glue that sticks dev and prod together.
2. Trust nobody's competence but your own. Matrix was advertised as "highly secure", and then they do a fuckup like this. Only trust yourself, and ensure that you're in control.4 -
I goofed up and forgot the WHERE clause in my UPDATE query. Accidentally all of production possibly updated because we don't do test databases. I think it didn't actually go through because I cancelled it but now I need to restore a backup and compare data. Which means explaining this to the co-owner who can help me with a restore. I'm mortified, more so because it was a stupid thing to do to begin with.4
-
DONT do production stuff on friday afternoon. This friday evening we had an issue on production and just wanted to do a quick fix. The fix resulted in a ddos attack that we accidentally started on our servers in an IoT project. We contacted all customers' devices and asked them for response at the same time. Funny thing is that the devices are programmed to retry if a request fails until it is successful. We ended up with 4 hours downtime on production, servers were running again at 11pm.4
-
My coleague's story
- before leaving after long day at the office final look at support cases (after official support hours)
- sev1 ticket logged an hour ago, noone called us (although should have; after support hours)
- angry manager calls and demands to get in touch with the client immediately (we're already after support hours, FTS should pick the case, not us)
- we reach out. Customer has business-impacting case
- after initial info gathering: some cert got expired, they got a new one and placed it in the app's directory. The app still does not work
- the first question we ask: "are you sure you have placed it in the right directory?"
- "yes, we are sure. No problems there" - answers a voice with indian accent
- noone finds the root cause for hours.
- It's already 1am
- someone from client's specialists comes up with an idea: "are we sure the cert is in the right place? Let's try to move it to the same directory the old one was in the first place"
- .................................................
- production is working again
- "Why didn't anyone from support suggest this?!?!"
- .................................................
- 2am. Case solved, manager is informed everything's allright now.
- In the morning we get yelled at by the manager bcz we supposedly missed a sev1 ticket and were incompetent during the conf. call
This reminds me why I stay away from support. And why I started hating people. And why I do not work with indians (our ways are too different for me to stay sane and not to kill anyone).3 -
Being mocked by the chef because I am "almost a week on a task"... Yeah dude, if a script which should update hundred thousands of main entities in production with a 45 min duration even though it is already paralleled is a half day job why don't you do it fucking yourself?4
-
-Writes a function that I'm going to schedule for django.
-works in development.
-adds it to production cron using django-crontab
-not working.
-spends 3 hours editing code, searching for similar problems and reading documentations but find nothing wrong and it's still not working.
-maybe it's django-crontab so I decide to just write a custom management command and call it through cron.
-still not working.
-calls function using what I'm telling cron to do.
-everything works.
-?????????
-adds logs to cron command (sorry for not making it earlier)
-mfw the code is not working because I imported 'patterns' in urls.py which has been deprecated since django 1.8 -
I'm getting really tired of those dumbass programmers that do not understand shit and then come to me when production breaks. (I am also a programmer, not really a DevOps engineer, but I'm the least worst at DevOps stuff, so it's my job...).
We're programming some kind of document management tool. Today we had a release, and one of the new features is to download all of your documents as a zip file, which is asynchronuously generated. When it's done, the user gets a mail with the download link to the zip file.
The feature works basically, but today it broke our production service, as somebody was running a test of it.
Turns out all the documents are loaded into memory to be zipped. So if you have 2 gigs of documents, a container with memory restrictions in that area will crash.
I asked the programmer who reported this «ops problem» to me, why he didn't just shit the files into a temp foler in order to zip them in there.
He told me that he wanted to do so, but did not know how to mock this for a unit test, and therefore went to the in-memory «solution», which was easier for him to mock.
For fuck's sake, unit tests and mocks are fucking tools, not ends in itself! I don't give a fuck about your pointless mocking code when the application crashes!
When I got to deal with such dumbasses, I'd prefer to mock those motherfuckers with a leaky bucket of liquid shit, which basically accomplishes the same task from my perspective: dripping shit all over the place and make everything suck as fuck.3 -
People don't seem to know how to properly do print-debugging, so here's a simple guide:
1. A log of "aaaaaa" or "got here" isn't as helpful as you think when ALL OF THEM ARE THE FUCKING SAME. You put a descriptive label or copy verbatim the conditional statement. This saves time matching statements, allows one to watch multiple branches at once, and allows others to understand and help faster when dragged in to help.
2. When trying to see where code fucks up, before each line, paste said line into a proper print statement for your language. If there's, say, a function call or some shit, have it output something like "functionCall(varA=<varA contents>,varB=<varB contents);" Most normal lines should be like this too, but it's especially helpful for calls and comparisons.
If need be, add return values after if they're not shown in another print statement later.
This allows for a trail of execution AND the line that fucks up will be the last in the log, making finding it easier when dealing with hangs and such.
3. Putting something unique like "DEBUG: " or something in front of all statements ensures you can just search for them to ensure you're not rolling one out to production. It also separates debug output from normal output at a glance, making digging through logs faster.16 -
I really think there should be a subject in every CS course to teach us how to handle/work-under Grade-A assholes and dumbfucks. Not that it would help, but atleast warn us on what we are getting into.
In my opinion, development is not *that* hard or frustrating but is made so by these shitty people. But again, what do I know.
I was scolded by my boss for using for-loop to iterate through an array recently. Apparently for-loop is not used in real world projects and this iteration should be done "in-memory". My colleagues and I are still trying to understand and process that.
I was asked to add fitbit integration to a project within 2 hours just because I had "already done it a week ago" in *another* project. Luckily, it was then given to a "senior" developer who took 4 days for it and essentially copy-pasted my work without much changes, ofcourse it stopped working every now and then.
I am given unreal deadlines on my tasks, on technologies I haven't worked on before, and then expected to churn out production ready code with no bugs in them.
My boss literally just sends me the links of 1st three google results on the problems I encounter and report, after humiliating me ofcourse. Yes, I did google it and yes I went through all I could find from Google forums to GitHub issues. When the library/plugin author himself says that this feature is not yet available, don't expect me to develop it in 2 hours you dumbfuck.
And for the love of God, please stop changing the data model every single day and justify it with agile development. Think before making any changes to it. Ever heard of Join queries? Foreign keys? Or any other basic database concepts.
We reached a point where each branch in the repo had different data model. Not kidding. And we were a team of just 4 developers. Atleast inform us when you change models after discussing it with your shit for knowledge "senior" developer, so we don't have to redo it all over again. The channels on slack are not for sharing random articles only.
I am just waiting to complete my year here.
I should have known what I got myself into the day he asked me to remove the comments I had added to explain what my code does. Why you ask? Because "we don't write comments". -
So i just saw a post about not pushing to production on Friday's and that reminded me of something.
A team we are working with (we needed help on something so we got another company to send us a team and do it for us) recently push to production on a Friday at around 5pm.
We didn't know that they had done it until a shit ton of errors started happening and we had no idea why (it was working so far, none of us did anything, so what the fuck is going on).
To make things better, one of my colleagues tried calling them and they wouldn't answer the phone. They knew what they did, so they went home before we could notice it and we had to correct their mistake.
We had everyone calling us saying they had gotten an error, and they needed to get home but couldn't because of it (it messed up a process that only happens at the end of the day before people go home. I can explain more in the comments if anyone is confused, or just wants to know).
Eventually everything was solved, but that was a very stressful ending of the week.
So yeah, don't deploy on Friday's please and thank you7 -
Me: Why are we spending time building reports for Support? AFAIK they never read or use them.
Boss: Seems they expect you to do it.
Me: Then what exactly are they supposed to be doing? All the issues seem to just escalate back to us.... We should just make sure issues never get into PROD
Boss: I agree, they're always firefighting... we gave them more funding so eventually they should catch up **Me: I highly doubt it, you should just stop hiring monkeys** esp. if we prevent PROD issues.
Me: Yea... we should prevent production issues because someone always has to pay off the (technical) debt and interest rates in PROD are very high6 -
I didn’t turn down a dev freelance project when the client decided against going with best practices because the solution I offered was a well-established design pattern but created a need for a financial management change she didn’t like. I stupidly built what she asked for. It worked fine in the 3rd party vendor test environment but failed on production. After hours of analysis of code to ensure no changes happened to my source during test->prod deployment, and the vendor denying they had config differences between them, and the client refusing to pay, all I could do was abandon the project.2
-
Oh gee whiz fellas. I lived through my nightmare. Recently too.
(Multiple rants over last few months are merged in this one. Couldn't rant earlier because my login didn't work.)
I joined a new shithole recently.
It was a huge change because my whole tech stack changed, and on top of that the application domain was new too.
Boss: ho hey newbie, here take this task which is a core service redesign and implementation and finish it in two weeks because it has to be in production for a client.
Normally I'd be able to provide a reasonable analysis and estimate. But being new and unaware of how things work here, I just said 'cool, I'll try my best.' (I was aware that it was a big undertaking but didn't realize the scope and the alarming lack of support I'd get and the bullshit egos I'd have to deal with)
Like a mad man I worked 17+ hours a day with barely a day off every week and changed and produced a lot of code, most of it of decent quality.
Deadline came and went by. Got extended because it was impossible (and fake).
All the time my manager is continuously building pressure on me. When I asked questions I never got any direct/clear answers. On asking for help, I'd get an elaborate word vomit of what was already known/visible. Yet I finally managed to have an implementation ready.
Reviewer: You haven't added parameter comments on your functions and there aren't enough comments in code. We follow standards. Clean code and whatnot. Care for the craft verbal diarrhea.
Boss: Ho hey anux, do you think we'll be able to push the code to production?
Me: Nope. We care for the craft and have standards. We need to add redundant comments to self documented code first, because that is of utmost importance as Nuthead reviewer explained.
(what I wish I had said)
What I actually said: No, code is not reviewed yet.
And despite examples of functions which were not documented (which were written by the reviewer nut), I added 6-7 lines of comments for my single line functions describing how e.g. Sum takes two input integers and returns their sum and asked for a review again.
Reviewer: See this comment is better written as this same-meaning-but-slightly-longer way. Can we please add full stops everywhere even though they were not there to begin with? Can we please not follow this pattern and instead promote our anti-pattern? Thanks.
Me: Changed the comments. Added full stops. Here's a link for why this anti-pattern is bad.
Reviewer: you have written such beautiful code with such little gems. Brilliant. It's great to see how my mentoring has honed your skills.
.
.
.
I swear I would have broken a CRT on his stupid face if we weren't working remotely (and if I had a CRT).
It infuriates me how the solution to every problem with this guy is 'add a comment'.
What enrages me more is that I actually thought I could learn from this guy (in the beginning). My self doubt just made me burnout for little in return.
Thankfully this living nightmare will soon be over.rant fuck you shitty reviewer micromanagement by micrococks wk279 living nightmare fml glassdoor reviews don't lie9 -
What is it with networking guys refusing to do any kind of fault finding? Pretty much everywhere I've worked they seem to be overpaid address hogs who occasionally want everyone to be proud of them for installing a new switch.
Currently seeing a production issue that's clearly due to spikes in packet loss on a certain part of the network - but oh no, it's always "our tests are fine", "we can establish a route no problem", "this is an application level issue", etc.
No you morons, when a dozen unrelated applications hosted on different cloud services fail because none of them can contact anything in your particular subnet in your data center at the same time, it's a damn networking issue. Sort it out.14 -
Ok, here goes...
I was once asked to evaluate upgrade options for an online shop platform.
The thing was built on Zend 1, but that's not the problem.
The geniuses that worked on it before didn't have any clue about best practices, framework convention, modular thinking, testing, security issues...nothing!
There were some instances when querying was done using a rudimentary excuse for a model layer. Other times, they would just use raw queries and just ignore the previous method. Sometimes the database calls were made in strange function calls inside randomly loaded PHP files from different folders from all over the place. Sometimes they used JOINs to get the data from multiple tables, sometimes they would do a bunch of single table queries and just loop every data set to format it using multiple for loops.
And, best of all, there were some parts of the app that would just ignore any ideea of frameworks, conventions and all that and would be just a huge PHP file full of spagetti code just spalshed around, sometimes with no apparent logic to it. Queries, processing, HTML...everything crammed in one file...
The most amazing thing was that this code base somehow managed to function in production for more than 5 years and people actualy used it...
Imagine the reaction I got from the client the moment I said we should burn it to the ground and rebuild the whole thing from scratch...
Good thing my boss trusted me and backed me up (he is a great guy by the way) and we never had to go along with that Frankenstein monster... -
Production: "Do you have [device]?"
Me: "Yes. I'm implementing the software that controls [device]."
Production: "Ok, that's the only [device] in the building, and I need to ship it to meet a deliverable."
Me: "... Does he want it to do anything?"
Production: "We'll build you another one."
Me: "When?"
Production: "... Eventually."
Software Development: The art of spinning straw into gold when you don't have any straw. -
I dunno about coolest, but I did sort of cement my reputation as the "database guy" in my first job because of this.
My first job was with a group maintaining a series of websites. Because of the nature of the websites, every morning we had to pull the records from one database on one network, sneaker net the data to a database on another network, and import the data via custom data import function.
However, the live site would crash after 100 or so records were imported. The dba at the live site had to script out a custom data partitioning script to do his daily duties, but it definitely messed up his productivity.
Turns out, the custom mass import function had recycled the standard import function, which was only used to import 1 record at a time, and it never closed its database connections, because it never needed to. A one line fix to production code was delivered 6 months later (because that was our release cycle) and I came up with the temporary work around, which was basically removing the connection limit. It would still crash with the work around, but only with multiple days worth of data. So basically only on Monday. Also developed the test set for the import (15k+ records). -
This is a public service announcement with a threat at the end of it:
"Do not, I repeat, do NOT attempt to write web applications, or any particular sort of application that works with a relational database (damn near more than half of applications) without a PROPER grasp and knowledge of SQL.
I do not want to see you reaching out for an ORM either, no, you need to learn to properly design a database or to properly interact with them AT most before you even attempt using an ORM OR designing an application from the beginning, shit will only hurt you in the long term I promise, learning SQL can go a looooong fucking way and most DBA's I know make way tf more than people think they make, it might even be an interesting career choice"
If you do not follow the above advise, and I see your ass reaching for building a web application without the above knowledge I will be under your bed at night, putting oil in my hairy body before I jump into bed to you and leave you confused for the rest of your life.
Build to learn, YES, but for the love of Chamberlain and Boyce PLEASE do not neglect SQL. I have seen such neglect REACH production and I am currently wishing I had these mfkers close to me.9 -
Issue in production. Multi billion dollar enterprise. Complex landscape. We sort of make things.
Turns out there is a single point of failure at a specific integration point. Kind of a lot stopped. When I reached out to the people knowing anything about it and I raised the issue that maybe we should make a slight change in how we do things they just brushed it off. Like it was nothing… 😬
No data was lost but everything was delayed for many hours. The _truth_ varied in different parts of the ecosystem causing potential wrong or suboptimal decisions to be taken.
When I asked why this LOS was not detected they told be they have no means of detecting it. 😬
I’m like, yeah, it’s 2023, we’re going to land on Mars and you can bet your ass we can detect it and you are just LAZY DEVELOPERS!
Anyway, I escalated (nicely) and they are now implementing a (more) resilient system and we’re helping the team detecting THEIR LOS in minutes instead of downstream services hours later (they are bad also but it’s not their fault!)
Stay safe!14 -
Visual Studio in the recent releases got some updates where it "accelerates build time" by caching DLLs or something
Good in theory? sure.
In practice? So very often, a "hot reload" now doesn't trigger a DLL swap. VS says that changes have been updated but you see stale code and you've to turn off the program and re-run it
I'm sure there's a way to turn this acceleration off and will do that after this rant, but I don't get how such retarded features get green-lit and make it to production :v
I understand that for biiiig solutions with minutes of build-time, this would be god-sent, but if it's this unreliable in my 8-Project Solution, I wonder how unreliable it'd be in bigger Solutions
at least turn it off by default if you know it's shit ffs.6 -
Rant from a previous gig I just remembered that reignited my fury lol
Suddenly, CSV exports became massively critical to our product's success. "They were always part of the plan, if we don't have them the product is a failure". Plot twist, they were NOT always part of the plan. And our backend is not at all designed for querying the combinations of data you're asking for.
Nevermind we've been entirely focused these last few months on making the new user experience as slick as possible because "our customers want cake, not meat and potatoes". Forget the fact that, in order to meet the deadlines, my team coupled the backend a little too much with the needs of the frontend because otherwise integrations took too long. We NEED fucking CSV exports of everything you can fucking imagine.
No. Fuck you. If you want it, it's gonna take at least 2 engineers and a month, and according to you we only have a few weeks of runway. No, I'm not compromising jack shit, this is the reality we live in. This is going to go nuclear in production if we don't do it right. Either give us the month and bankrupt the company, or fucking drop it.
Or...you could go cry to the frontend team for solutions. And convince them to page through ALL of the data and generate CSVs in the fucking browser. Sure, it sort of works in QA with the miniscule amount of data we have there, but how'd that work out for you in prod?
Jesus fucking christ why are you people such incompetent morons, and how the fuck did you become executives??2 -
Startup-ing 101, from Fitbit:
- spy on users
- sell data
- cut production costs
- mutilate people's bodies, leaving burn scars that will never heal
- announce the recall, get PR, and make the refund process impossibly convoluted
- never give actual refunds
- claim that yes, fitbit catches fire, but only the old discontinued device, just to mess with search results and make the actual info (that all devices catch fire) hard to find
- try hard to obtain the devices in question, so people who suffered have no evidence
- give bogus word salad replies to the press
This is what one of the people burned has to say:
"I do not have feeling in parts of my wrist due to nerve damage and I will have a large scar that will be with me the rest of my life. This was a traumatic experience and I hope no one else has to go through it. So, if you own a Fitbit, please reconsider using it."
Ladies and gentlemen, cringefest starts. One of fitbit replies:
"Fitbit products are designed and produced in accordance with strict standards and undergo extensive internal and external testing to ensure the safety of our users. Based on our internal and independent third party testing and analysis, we do not believe this type of injury could occur from normal use. We are committed to conducting a full investigation. With Google's resources and global platform, Fitbit will be able to accelerate innovation in the wearables category, scale faster, and make health even more accessible to everyone. I could not be more excited for what lies ahead".
In the future, corporate speech will be autogenerated.
(if you wear fitbit, just be aware of this.)13 -
I've been working for the last 5 years on some large legacy code used in production, more than 100K LOC, poor comments (when existing) often outdated, huge parts of code that can no longer be reached, over-engineered class hierarchy, functions of thousands lines, huge parts of deprecated code that cannot be removed because "someone might still be using it". Statistically, every small change caused 3 new issues somewhere else and every bug fix or new feature required 10 times the time that would be necessary with a decent codebase. But after five years in hell I can finally say that... Oh wait, nothing changed, the code is still legacy and nobody is going to do anything about that.1
-
developer makes a "missed-a-semicolon"-kind of mistake that brings your non-production infrastructure down.
manager goes crazy. rallies the whole team into a meeting to find "whom to hold accountable for this stupid mistake" ( read : whom should I blame? ).
spend 1-hour to investigate the problem. send out another developer to fix the problem.
... continue digging ...
( with every step in the software development lifecycle handbook; the only step missing was to pull the handbook itself out )
finds that the developer followed the development process well ( no hoops jumped ).
the error was missed during the code review because the reviewer didn't actually "review" the code, but reported that they had "reviewed and merged" the code
get asked why we're all spending time trying to fix a problem that occurred in a non-production environment. apparently, now it is about figuring out the root cause so that it doesn't happen in production.
we're ALL now staring at the SAME pull request. now the manager is suddenly more mad because the developer used brackets to indicate the pseudo-path where the change occurred.
"WHY WOULD YOU WASTE 30-SECONDS PUTTING ALL THOSE BRACES? YOU'RE ALREADY ON A BRANCH!"
PS : the reason I didn't quote any of the manager's words until the end was because they were screaming all along, so, I'd have to type in ALL CAPS-case. I'm a CAPS-case-hater by-default ( except for the singular use of "I" ( eye; indicating myself ) )
WTF? I mean, walk your temper off first ( I don't mean literally, right now; for now, consider it a figure of speech. I wish I could ask you to do it literally; but no, I'm not that much of a sadist just yet ). Then come back and decide what you actually want to be pissed about. Then think more; about whether you want to kill everyone else's productivity by rallying the entire team ( OK, I'm exaggerating, it's a small team of 4 people; excluding the manager ) to look at an issue that happened in a non-production environment.
At the end of the week, you're still going to come back and say we're behind schedule because we didn't get any work done.
Well, here's 4 hours of our time consumed away by you.
This manager also has a habit of saying, "getting on X's case". Even if it is a discussion ( and not a debate ). What is that supposed to mean? Did X commit such a grave crime that they need to be condemned to hell?
I miss my old organization where there was a strict no-blame policy. Their strategy was, "OK, we have an issue, let's fix it and move on."
I've gotten involved ( not caused it ) in even bigger issues ( like an almost-data-breach ) and nobody ever pointed a finger at another person.
Even though we all knew who caused the issue. Some even went beyond and defended the person. Like, "Them. No, that's not possible. They won't do such dumb mistakes. They're very thorough with their work."
No one even talked about the person behind their back either ( at least I wasn't involved in any such conversation ). Even later, after the whole issue had settled down. I don't think people brought it up later either ( though it was kind of a hush-hush need-to-know event )
Now I realize the other unsaid-advantage of the no-blame policy. You don't lose 4 hours of your so-called "quarantine productivity". We're already short on productivity. Please don't add anymore. 🙏10 -
After CR. Seen a function called "initConfig", with another version of it commented out.
I said in the CR - delete the commented function out.
In response, the developer said that it is needed. The commented version is for local run, while the other is for production.
After a lovely argument about cancer in our Code-base, the project manager concluded that we don't have time to do it better, and basically forced me to basically accept.
I hate being sane sometimes. -
So my co-worker asked me to change something. It was simple, took me about 5min to do it and pushed to production right away!
Two minutes after, asked me to change two other things. In the span of 1h30, she came to my desk 3 times and asked for 3 different things!
The thing is: before I push anything to production, I have to ask my other co-worker who is currently on vacation.
Already sent him one email for the first thing, and now I feel bad because I have to send him another one...
In case she comes up with any more changes, I'm just going to wait for the end of the day and send one more email with every change I made. This is great. FML3 -
!Dev / story
My phone starts dying gently but surely. Since last week I cannot use my jack input anymore, and thus can't listen to music in the car. I also compose music, and was eager to listen to my latest production (for reviewing purposes) in the car. In my frustrated search for a spare device with a jack, I found a pile of blank CDs instead. "Aw yiss, I can haz music in my car" I thought with a huge relieved grin.
I grabbed a CD, looked at my pc, and my grin faded instantly to an "oh" of disappointment: I deliberately did not install a CD drive in my computer.
Not losing hope, I grabbed my Mac and tried inserting my blank disk in the drive. "Clunk, clunk", the cd won't go in. "Ah silly me, I replaced that drive with a SSD". So I went looking for that spare cd drive.
After I found it, its SATA power interface was smaller than regular SATA devices, and any connectors I tried were too big. "Hmpf, ok, I'm desperate, let's remove that SSD in my Mac". So I went grabbing some screwdrivers, removed the cover lid underneath the computer, and removed two screws from the SSD casing, allowing me to lift the unit up, disconnect the cable, plug in the cd drive, flip the Mac carefully, turn it on... And burn my CDs, and finally I resetted everything back to normal, carefully removing the cd drive and closing the computer.
What one doesn't do in frustration...2 -
If they followed my suggestion and went straight to debugging the server issues they would have been solved it from week 1 and everyone would have thought the migration had a minor performance hiccup. In fact, we have already done such at least twice before and nobody batted an eye.
Instead they self-labelled the migration a failure on first error, setting the stage for apologizing to the client, and put themselves on the spot for a whole staging / production signoff, replication / backup worfklow, almost a blue-green "seamless" deployment reminiscent of DigitalOcean.
Well they're not DigitalOcean, and anyone who has spent any time understanding users knows they will not participate in "new system" tests long enough to find or report issues.
So of course the migration stretched out to almost three months up until the whole reason for the migration - the rapidly escalating risk of the old provider disappearing - hit like a freight train and now they have to go through the problem of debugging the server like I told them to on week 1. Only this time they've set the client mindset against it, lost any chance of reverting, have had grave risk for data loss, and are under pressure to debug other people's code in real-time.
This is why I don't trust devs to do ops. A dev's first solution to any problem is to throw tech at it. -
TL;DR Dear boss, firstly, you always get someone to review anything important done by a fucking intern.
Secondly, you do not give access to your fucking client's production server to an intern.
Thirdly, you don't ask your fucking intern to test the intern's work that has not been reviewed by anyone directly on your client's fucking production server.
Last week, the boss and one of the lead devs (the only guy with some serious knowledge about systems and networking) decided to give me (an intern who barely has any work experience) the task of fixing or finding an alternate solution to allowing their support team access to their client machines. Currently they used a reverse SSH tunnel and an intermediary VH but for some reason, that was very unreliable in terms of availability. I suggested using OpenVPN and explained how it would work. Seemed to be a far better idea and they accepted. After several days of working through documentations and guides and everything, I figured out how OpenVPN works and managed to deploy a TEST server and successfully test remote access using two VMs. On seeing my tests, the boss told me that he wanted to test it on the client network. I agreed. Today he comes to me and he tells me to prepare testing for tomorrow and that the client technician is going to give me access to one of their boxes. And then he adds, "It's a working prod server. We'll see if we can make it work on that" and left. I gaped at him for a while and asked another dev guy in the room if what I heard was right. He confirmed. Turns out, the lead dev and the boss's son (who also works here) had had a huge argument since morning on the same issue and finally the dev guy had washed it off his hands and declared that if anything goes wrong from testing it on production, it's entirely the boss's own fault. That's when the boss stepped in and approached me. I ran back to his office and began to explain why prod servers don't top the list of things you can fuck around with. But he simply silenced me saying, "What can go wrong?" and added, "You shouldn't stay still. You should keep moving". Okay, like firstly what the fuck and secondly, what the fuck?.
Even though OpenVPN client is not the scariest thing to install, tomorrow's going to be fun.4 -
Hey this is the first time i post here.
I just started working part-time for this company last week. What i have to do is to change some windows from Win32 to WPF. As i was reading the legacy code i just had to sigh man. They have like 100 projects in a single solution, from C++ to C#, everything acctached to each other, with almost NO comments or docs. Wtf man? I don't know how it actually works in the industry (this is my first dev job) but when you write fucking 20 classes with each one contains bunch of attributes, methods, properties, you can't just leave all the code's semantics in their names. And by the way the app is so fucking ugly i bet they have appointed part-time developers as UX engineers... Even i have little knowledge about UX/UI, i just can't bear with this kind of ugly and confusing and unintuitive production with a cost of a good photo editting software.
Ok there may be much more to rant in the future but let me try through this and tell you more. Have a good day. :)5 -
Living on the edge!
One or two years ago I managed to deploy a DDL change directly on the production server. As I knew there was a backup job which will run every day at noon and at midnight. So I run my script some minutes after noon. So far so good. But somehow I tested it badly in my test environment and the UI of the application throws error after error now in production.
Well, just revert the db to the latest recovery point with the backup, I thought.
It became clear then after a couple of minutes of searching the backup folder for the db backup that there was no such file. The youngest backup file was 3 years old.
Now what happened: The backup script had a switch "simulate=true" and then simulated a successful backup on each run. Therefore the monitoring system got no alerts for not correctly executing those jobs correctly. Then the monitoring job which should do the backupfolder surveillance stuck with green, because there was a valid backup file inside. But it did not check for a specific creation date.
Now this database is the one we need for doing our daily business and is really crucial. Therefore It was easier to emergencyfix the application than doing a rollback of the db 🙄
Well, not really a data loss story, but close to one. -
When coworkers have a var dump on a page in production.-_-
I aint saying shit because everytime I mention something they do wrong I get assigned with fixing it. -_- -
A story about a helpless intern : On a fine day an intern was assigned a feature to develope. He worked his ass off, completed it, submitted it for testing, the build was approved by the tester and got released the next day. Fast forward a few days, the feature is failing in few cases at the production.Everbody, starts pointing fingers at the intern.
The intern wonders...How the fuck am I gonna know this fucking use case, do they really expect me to to build a full proof feature without telling me about all the possible senarios...And how the fuck did the tester approved this...? I mean, now that I know this senario, it seems pretty obvious that it should have been tested...!
Note : This also happened to another developer who recently joined...The PM failed to properly communicate all the requirements and the fucking lazy ass tester did not consider all the possible senarios. And the script failed in the production...!
Note : It's 4 fucking AM and the intern still can't sleep...5 -
It is not on production anymore, but it was for long enough. Someone thought it would be a great idea to be able to debug a web app while signed in as a user reporting a problem. How to do it? It's easy. Just check on every request if magic HTTP parameter SIGN_IN_AS=id is present and if it is, sign in as this user. Of course, it worked also with admin account with hard-to-guess id=1.1
-
So I did a clean Windows 10 restore recently on my laptop from Insider program to just Anniversary Update . Went away from my computer for a day or so by the time I got updates completed and Visual Studio up and running. So earlier today I went to start back up development on a project of mine to come across the emulators not working. The thing is that I lost 6 hrs of production to figure this out. I tried everything possible so I gave up and reinstalled VS to just remember I forgot to turn on my Hyper-V in BIOS setting. So I'm half way in VS reinstalling and I can't do anything about it. GOD FUCKEN DAMMIT W10!8
-
*face palm*
My boss just IM'd me. She asked me the status of a work item. I told her my teammate completed it on 2/27. She wanted to know if it was in production. I said I was unsure, she would have to ask my teammate. So of course, she asked *me* to ask my teammate, and then get back to her.....WHY do I need to ask my coworkers stuff for her? She is the BOSS, last time I checked?2 -
I found a bug.... it's in production for a long long time... I wrote it while optimizing some legacy code....
FUCK.... how do you feel when you discover a bug in production created by you?6 -
Other team lead: Hi DevOps Team, We need you to deploy this app to production. It's maintainers gave up on it in 2019, but we looked at it and it feels right.
Me: Uhm. That's not going to work. It'll fail the security scan before you can even finish the build in CI.
Other team lead: Yeah, this app is the right thing to do, and we needed it last week, but since that won't work, we'll just use this other very very infant technology that was just born yesterday. It's not stable in production, or on MySQL, or in AWS at all, but it's the other direction we can to go.
Me: What problem are you trying to solve in the first place?
Other team lead: Oh, we need access to the read from the production database.2 -
Don't use your senior software engineer title or years of experience as reasons in a debate or argument about software
My manager was asking me what steps needs to be done to perform a disaster recovery for our cluster( on production). I will be honest here, I have not maintained this type of cluster(kubernetes) in production before. However, I have enough understand of the system to answer my boss question. I basically told him there are A, B, C you have to do.
My senior developer jumped in and said "No you should do A,C, B because C is more critical than B. " I then replied to him: "I understand your point, I notice that too, but .." Before I can even finish my sentence, this dude has already rolled his eyes and interrupted me very loudly: "Have you worked with these systems on production before? I did". The asshole knows I haven't maintained Kubernetes on production yet of course.
I got super pissed at him and pretty much shouted back to him and my manager: "Just because I haven't worked on this system on production yet, does not mean my argument is wrong" .
I then dragged my managers, that asshole, and other engineers in a room and settle this out. In the end, people agreed with my steps over that asshole senior engineer dude because I gave them rational reasons.
The conclusion is: Your senior title is given by the company, It doesn't mean anything to me. Also, it doesn't make you more right than another person just because he has a "lower" title than you.1 -
I started off in a MNC company as a junior developer. I entered with candy glasses.
I didn't expect to win the lottery. Of getting abuse by superior.
I stayed for a year, at the project. Constantly being belittled by this team lead. It was awful i enter as a fresh grad. All the new tech were so new and scary at that point.
During my time there, i constantly think that developer is not my stuff.
Ultimately i reach the state of burnout. I reached out to the manager and broke down in his office.
I actually told the manager. "I hate coding"
I remember staying up to 4am just complete a piece of program. To be ready to be push to production the next day. My team lead just come screaming at me saying there is bug.
Upon receiving that message via skype. I broke, tears flow down my eyes.
After which i reach a state of burn out. I start to reach out to external parties for help to get me out of there.
Now i am recovered from the burn out. I am curious of the technology that were utilized in that project. I literally face palm. After understanding the technology it isn't so hard after all. I just didn't gear myself up with the tech.
I still do enjoy working on code.3 -
[SERIOUS ADVICE NEEDED, PLZ HELP]
I am going to school again for like 4 days from tomorrow (don't ask me why, blame the government) and I feel a bit depressed. I just don't know what I have done in the last 2 years.
What I learned:
- Bunch of stupid facts from devRant
- C# stuffs
- Games are expensive
- Music production
And.... that's it, tbh
I don't really have "PERSONAL PROJECTS" that everyone is bragging about, I just have bunch of empty projects with a cool name but just Program.cs in it.....
I am worried of what to do now.
I just feel I made the wrong choice going with C#.
I just feel I should have went with JS.
With JS, you can do
- React Native + Cordova + Titanium + etc and make native android/ios/wp apps
- The WWW stuffs
- Electron --> Cross platform desktop apps (win/mac/linux)
- UnityScript (deprecated, but whatever) --> Games
So, what I am seeing now is a thick fog in the way to my future + career etc.....
I am stuck rn.
Please help.
Should I continue with my pace and learn more C# and the things I do rn, or change the language and start from scratch, or as a last resort, leave the "make stuff by coding" industry and go to music industry, or just go to the airport and do planespotting and upload in youtube to earn money?
Serious advice please, and no jokes about C# and JS. These languages may suck, but YOUR language may suck more.10 -
Was just reading some of the OpenVPN scripts to renew a certificate where I forgot to source the vars file first (apparently OpenVPN stores those in a separate file that you always have to source first, and I tend to forget it sometimes).
Reading the revoke-full script that OpenVPN provides, it's just bash so I can read it no problem. But traversing through it and trying to understand it... Horrible! There's a test file in $RT named keys/revoke-test.pem. It's not used anywhere in OpenVPN for anything useful as far as I'm aware. The script however - the script that's running on a production server! - attempts to remove this file. It doesn't exist. Test files do (or at least should) not exist in production. They're not supposed to be there.
It exports empty variables. Some of them are set by the sourced vars file, some aren't. Not entirely sure why it's exporting variables as empty when they're uninitialized, or why it doesn't just unset the ones that are initialized.
And finally it goes ahead and revokes the key file that I'm actually concerned about through regular OpenSSL and verifies it.
Not to mention that the lack of the sourced vars file, which admittedly I should think about in the current status quo, if it *always* needs to be sourced anyway... Why doesn't the script do that itself then? One less thing to go wrong. But hey, proper design?
Gore. I don't have any other words for it.
And before anyone tells me that I should go and fix it if I'm so worried about it. Remember, I am not a developer. That's the job of the developers that made this in the first place.9 -
At a startup where the software was built haphazardly because the developer thought he'll lifelong be the sole maintainer. The dude antagonized me at every turn and refused to help with familiarising with his code. He eventually left majority of the work for me, and dedication to work continued to dwindle until he threw in the towel
After his departure, we surprisingly grew fond of each other, discussing code concepts at length. He was in the habit of refusing to read any of the articles I sent him, or answer open ended questions citing the claim that they require thinking and he was busy. I didn't take any of this to heart
But it accumulated and I deleted his number. I didn't bear him any ill wishes but it wasn't respectful to myself for him to remain in my space. Some day, I was looking for a point raised during our conversations and went rummaging through our chats. Going down memory lane opened scars I'd long forgotten. I was embarrassed to see the way I forgot all about it. I should never have had anything to do with someone like that
He contacted me for a favour just less than a year after I deleted his contact. I didn't even think of declining. But this evening, I randomly remembered how he saw a defect in my code, promised me that the code will fail in production and resisted all pleas to point out what it was. I don't know if I hate him for his dastardly acts. What I feel deepest is sadness/bitterness that I got to experience all that2 -
A couple of years ago I was working on a fairly large system with a complex (by necessity) access control architecture.
As is usually the case with those projects, it's awkward for developers to repro bugs that have to do with a user's accesses in production when we are not allowed to replicate production data in test, let alone locally.
We had a bug where I ended up making myself a new row in the production database for a thing I could have access to without affecting real data to repro it safely. I identified the bug so I could repro it in dev/test and removed the row and ensured everything worked normally, whew scary.
Have you ever walked into the office one day, and everyone is hunched over in a semicircle around one person's workstation, before one turns around to look at you and says - after a pause - "... ltlian?.."
Turns out I had basically "poisoned the well" with my dummy entity in a way where production now threw 500 for everyone BUT me who had transitive access to this post-non-entity. Due to the scope of the system, it had taken about a day for this to gradually propagate in terms of caching and eventual consistencies; new entities coming in was expected, but not that they disappear.
Luckily I had a decent track record for this to be a one-off. I sometimes think about how I would explain testing in prod and making it faceplant before going home for the day, other than "I assumed it would be fine". I would fire me.3