Worst dev team failure I've experienced?

One of several.

Around 2012, a team of devs were tasked to convert a ASPX service to WCF that had one responsibility, returning product data (description, price, availability, etc...simple stuff)
No complex searching, just pass the ID, you get the response.

I was the original developer of the ASPX service, which API was an XML request and returned an XML response. The 'powers-that-be' decided anything XML was evil and had to be purged from the planet. If this thought bubble popped up over your head "Wait a sec...doesn't WCF transmit everything via SOAP, which is XML?", yes, but in their minds SOAP wasn't XML. That's not the worst WTF of this story.

The team, 3 developers, 2 DBAs, network administrators, several web developers, worked on the conversion for about 9 months using the Waterfall method (3~5 months was mostly in meetings and very basic prototyping) and using a test-first approach (their own flavor of TDD). The 'go live' day was to occur at 3:00AM and mandatory that nearly the entire department be on-sight (including the department VP) and available to help troubleshoot any system issues.

3:00AM - Teams start their deployments
3:05AM - Thousands and thousands of errors from all kinds of sources (web exceptions, database exceptions, server exceptions, etc), site goes down, teams roll everything back.

3:30AM - The primary developer remembered he made a last minute change to a stored procedure parameter that hadn't been pushed to production, which caused a side-affect across several layers of their stack.

4:00AM - The developer found his bug, but the manager decided it would be better if everyone went home and get a fresh look at the problem at 8:00AM (yes, he expected everyone to be back in the office at 8:00AM).

About a month later, the team scheduled another 3:00AM deployment (VP was present again), confident that introducing mocking into their testing pipeline would fix any database related errors.

3:00AM - Team starts their deployments.
3:30AM - No major errors, things seem to be going well. High fives, cheers..manager tells everyone to head home.
3:35AM - Site crashes, like white page, no response from the servers kind of crash. Resetting IIS on the servers works, but only for around 10 minutes or so.
4:00AM - Team rolls back, manager is clearly pissed at this point, "Nobody is going fucking home until we figure this out!!"
6:00AM - Diagnostics found the WCF client was causing the server to run out of resources, with a mix of clogging up server bandwidth, and a sprinkle of N+1 scaling problem. Manager lets everyone go home, but be back in the office at 8:00AM to develop a plan so this *never* happens again.

About 2 months later, a 'real' development+integration environment (previously, any+all integration tests were on the developer's machine) and the team scheduled a 6:00AM deployment, but at a much, much smaller scale with just the 3 development team members.

Why? Because the manager 'froze' changes to the ASPX service, the web team still needed various enhancements, so they bypassed the service (not using the ASPX service at all) and wrote their own SQL scripts that hit the database directly and utilized AppFabric/Velocity caching to allow the site to scale. There were only a couple client application using the ASPX service that needed to be converted, so deploying at 6:00AM gave everyone a couple of hours before users got into the office. Service deployed, worked like a champ.

A week later the VP schedules a celebration for the successful migration to WCF. Pizza, cake, the works. The 3 team members received awards (and a envelope, which probably equaled some $$$) and the entire team received a custom Benchmade pocket knife to remember this project's success. Myself and several others just stared at each other, not knowing what to say.

Later, my manager pulls several of us into a conference room
Me: "What the hell? This is one of the biggest failures I've been apart of. We got rewarded for thousands and thousands of dollars of wasted time."
<others expressed the same and expletive sediments>
Mgr: "I know..I know...but that's the story we have to stick with. If the company realizes what a fucking mess this is, we could all be fired."
Me: "What?!! All of us?!"
Mgr: "Well, shit rolls downhill. Dept-Mgr-John is ready to fire anyone he felt could make him look bad, which is why I pulled you guys in here. The other sheep out there will go along with anything he says and more than happy to throw you under the bus. Keep your head down until this blows over. Say nothing."

Fresh out of college?
Entry-level?
Apply Here!

Junior Web Intern: 16k A Year!
Basic Requirements: HTML, CSS, JavaScript, Node, Angular, JQuery, Bootstrap, Backbone, Handlebars, D3, p3, CMS (WordPress, Wixx), PHP, Java (Android), C++(iOS), openframeworks, openGLSL, Cinder, failed at least two startups, 8086 assembly language.

Recommended requirements:
Git version control
Agile development

Must be able to display example of each requirement.

At age of 20, I got hired as junior dev at a mobile gaming company. We were 2 junior devs hired at the same time and one of our senior colleagues made a prank: he came in the office before us and rearranged our offices in a "funny" manner.

Two days later I waited for him to go home. I opened his PC case, removed the power button cable from the motherboard and then re-arranged everything back to normal. Well, I couldn't resist...

Next day he came into the office and, well, surprise... the PC was not starting. He went to the IT department and they spent 4 hours trying to figure out why it was not working. They replaced the CPU, RAM memory, including the PSU.

I had to go and tell them: "maybe it's the power button jack?!".

I got into some problems for that prank. Indeed I crossed a line, but what the hell... that was a bad IT department.

I have been a mobile developer working with Android for about 6 years now. In that time, I have endured countless annoyances in the Android development space. I will endure them no more.
My complaints are:

1. Ridiculous build times. In what universe is it acceptable for us to wait 30 seconds for a build to complete. Yes, I've done all the optimisations mentioned on this page and then some. Don't even mention hot reload as it doesn't work fast enough or just does not work at all. Also, buying better hardware should not be a requirement to build a simple Android app, Xcode builds in 2 seconds with a 8GB Macbook Air. A Macbook Air!

2. IDE. Android Studio is a memory hog even if you throw 32GB of RAM at it. The visual editors are janky as hell. If you use Eclipse, you may as well just chop off your fingers right now because you will have no use for them after you try and build an app from afresh. I mean, just look at some of the posts in this subreddit where the common response is to invalidate caches and restart. That should only be used as a last resort, but it's thrown about like as if it solves everything. Truth be told, it's Gradle's fault. Gradle is so annoying I've dedicated the next point to it.

3. Gradle. I am convinced that Gradle causes 50% of an Android developer's pain. From the build times to the integration into various IDEs to its insane package management system. Why do I need to manually exclude dependencies from other dependencies, the build tool should just handle it for me. C'mon it's 2019. Gradle is so bad that it requires approx 54GB of RAM to work out that I have removed a dependency from the list of dependencies. Also I cannot work out what properties I need to put in what block.
4. API. Android API is over-bloated and hellish. How do I schedule a recurring notification? Oh use an AlarmManager. Yes you heard right, an AlarmManager... Not a NotificationManager because that would be too easy. Also has anyone ever tried running a long running task? Or done an asynchronous task? Or dealt with closing/opening a keyboard? Or handling clicks from a RecyclerView? Yes, I know Android Jetpack aims to solve these issues but over the years I have become so jaded by things that have meant to solve other broken things, that there isn't much hope for Jetpack in my mind 😤

5. API 2. A non-insignificant number of Android users are still on Jelly Bean or KitKat! That means we, as developers, have to support some of your shitty API decisions (Fragments, Activities, ListView) from all the way back then!

6. Not reactive enough. Android has support for Databinding recently but this kind of stuff should have been introduced from the very start. Look at React or Flutter as to how easy it is to make shit happen without any effort.

7. Layouts. What the actual hell is going on here. MDPI, XHDPI, XXHDPI, mipmap, drawable. Fuck it, just chuck it all in the drawable folder. Seriously, Android should handle this for me. If I am designing for a larger screen then it should be responsive. I don't want to deal with 50 different layouts spread over 6 different folders.

8. Permission system. Why was this not included from the very start? Rogue apps have abused this and abused your user's privacy and security. Yet you ban us and not them from the Play Store. What's going on? We need answers.

9. In Android, building an app took me 3 months and I had a lot of work left to do but I got so sick of Android dev I dropped it in favour of Flutter. I built the same app in Flutter and it took me around a month and I completed it all.
10. XML.

If you're a new dev, for the love of all that is good in this world, do NOT get into Android development. Start with Flutter or even iOS. On Flutter and build times are insanely fast and the hot reload is under 500ms constantly. It's a breath of fresh air and will save you a lot of headaches AND it builds for iOS flawlessly.

To the people who build Android, advocate it and work on it, sorry to swear, but fuck you! You have created a mess that we have to work with on a day-to-day basis only for us to get banned from the app store! You have sold us a lie that Android development is amazing with all the sweet treat names and conferences that look bubbly and fun. You have allowed to get it so bad that we can't target an API higher than 18 because some Android users are still using devices that support that!

End this misery. End our pain. End our suffering. Throw this abomination away like you do with some of your other projects and migrate your efforts over to Flutter. Please!

#NoToGoogleIO #AndroidSummitBoycott #FlutterDev #ReactNative

I. FUCKING. HATE. MOBILE. DEVELOPMENT.

I already manage the data, devops, infra, and most of the backend dev.

We had a mobile guy. He was great. I never had to think about it and kept moving quickly on my work. #SpecializationOfLaborFTW

He left. Why? Because they wouldn't give him a small raise despite being one of the best mobile engineers in the firm. WTF.

I made the mistake of picking up just enough slack on this workflow in the interim such that I'm, apparently, the fucking god-damned release manager, fixer of pipelines, fixer of build configs, fixer of anything where someone just needs to RTFM for a half-hour to not fucking break things.

Now, 8 months later...and, apparently, Fortune 500 companies are too fucking god-damned cheap to pay for someone who actually knows WTF they're doing for a very reasonable thing to have at least one dedicated set of eyes for.

I never wanted to be a mobile dev.
I never will want to be a mobile dev.
And I certainly don't want to manage your HALF-FACE-FUCKED detached expo configs.

There's a reason I never intentionally involved myself in mobile. All the way down, it's just shitty cross-compilation, transpilation, dependency-hell, brittle-as-fuck build processes so we can foot-gun and mouth-gun react-native and expo and babel and whatever the fuck else cargo-culted horseshit into the wild.

And why? What's the actual fucking root cause? The biggest white elephant that ever fucking elephant-ed? It's because Apple and Google decided to never collaborate on a truly-native cross-platform SDK--where engineers could write native code that compiles to native binaries that's simply write-once, run-everywhere. They know they could have done that, and they didn't. So what'd they get back? Expo--a too-cleverly-designed backdoor/hack--more-or-less a way to circumvent the sane release process software has usually followed: code -> executable -> deploy. Or code -> deploy (for interpreted langs). Expo's like "keep your same executable, we're just gonna to do updates by injecting new code into it whenever we want". Didn't we learn anything with web? Shit gets messy real quick? Not to mention: HEY EXPO, WE WERE ALREADY BUILDING NATIVE APPS, YOU SHORT-SIGHTED FUCKS. THANKS FOR LURING OUR CTOs INTO FORCING EXPO DOWN OUR THROATS W/ THE IMPLICIT (BUT INCORRECT) TOO-GOOD-TO-BE-TRUE PROMISE THAT WE CAN HAVE WRITE-ONCE, RUN-ANYWHERE WITHOUT ANY BUY-IN OR COOPERATION FROM THE ACTUAL TARGET PLATFORMS.

And, we just, like, accept this? We all know it's garbage engineering. The principles we learned in the classroom aren't just academic abstractions--they actually yield real-world results--and eschewing them yields real-world failures. Expo is tightly-coupled to high-heaven, with leaky abstractions six-ways-to-christmas, chock-full of foot-guns, and fails the most basic test of quality: does it, "just work?"

Expo is fucking shameful and it should fucking die. Its promises are too bold, its land-mines too many, its future-proof-ness is alway, always, always questionable as fuck and a risk to every project that uses it.

You want a rant? This is my fucking venue, 'tis not? Well, then this is a piss and vinegar rant straight from my blood-red, beating fucking heart:

EXPO FUCKING SUCKS. AND IF YOU'RE A FAN, YOU FUCKING SUCK TOO.

Me: *Applies for entry level full-stack job*

Recruiter: "Sorry, I can't hire you because you don't have the years of experience we're looking for. We can take you on as an intern! Unpaid of course, while we train you."🙂

Clueless Me: "Sure, why not."

*second day into the internship*

Boss: "I have this really big project, and I want you to be the lead. I'm going to be very vague about what I want, so you'll constantly have to make changes to user stories, wireframes, & database designs until I'm satisfied. Don't ask me any questions for clarity, because I'm busy 🙂"

Silly Me: "okay"

Boss: "Also, can you train all the other interns? You're so lucky! You'll get to pick the best to join your team" 🙂

Stupid Me: "okay"

Boss: *emails me a spreadsheet of 80 Front-End interns (freshmen and sophomores)*
"Did you start building the app yet?" 🙂

Me (Dummy): "You haven't approved the final wireframes ye-"

Boss: "And for the other interns' training, what did you have in mind?" 🙂

Me (Dumbass): "I made a training guide, they're already followi-"

Boss: "My project manager for this other project left, guess he couldn't handle the pressure of a real job... HAHAHAHA! You're gonna take the lead of that project, too!"
*Adds me to the slack group* 😁

Me (Imbecile): "Wha-"

Boss: "And we've been having trouble with keeping track of everyone's code. Is there something we can do instead of slacking code snippets back and forth?" 🤔😮

Me (Fucking Imbecile): "Wait, you guys are working on a project and you don't have any form of version control? Maybe we should take a few steps back and plan thi-"

Boss: "Are you gonna take initiative or not!?" 😡

Me (Enlightened): "I quit." 😑

Former Boss: "Too bad... I was going to offer you a paid role tomorrow morning. Oh well!" 😔

Debugging native Android app is just a pain in the ass -_-

For years in college I saw the memes about entry level jobs basically requiring you to be god himself but always thought they were over exaggerated. Well now that I've been on the job hunt for 4 months, they could not have been more right...

Using Ubuntu for the first time on my new XPS-13 Dev. Ed.

I'm loving the OS but it sure is frustrating. Can't even get Spotify to download -.-

Bought Macbook pro. Because React native iOS apps don't compile on Linux machine.. :/:/

Had my first official coding interview for a "real" developer position! Really is a different experience. So much to do and so little time.

Overall I feel like it went well. Hope I get some good news soon 🤞

Here's a real tip for people new to the industry.

It's one of those things that's been said over and over again but very few can really seem to employ. I suggest you learn it /well/.

You are not your code. Criticisms of your code, ideas, or your thought processes, is not a criticism of YOU. You absolutely cannot take criticisms of your work personally.

We are engineers. We strive to seek the best solution at all times.

If someone has found a problem with your code or with an idea or whatnot, it is coming from a place of "this is not the best solution", NOT "you're an idiot".

It's coming from a place of "I'm closing this PR because it is not a change I feel suits this project", NOT "I'm closing this PR because it's coming from a woman".

It's coming from a place of "This feature request is ridiculous/this bug is not actually a bug", NOT "you're a fucking idiot, fuck you".

It's coming from a place of "I've already had to address this in a number of issues before and it's eaten up a considerable amount of my time already", NOT "I don't even know you and this I don't have time for a nobody".

You do not get to be bitchy to maintainers because they denied your request. It's not a reflection of you at all. But if you're arguing with someone who has maintained a piece of code for almost a decade, and they're telling you something authoritative, believe them. They're probably smarter than you on this subject. They've probably thought about it more. They've probably seen their code used in many different places. They have more experience than you with that codebase in almost all cases.

Believe me, if we cared about who was behind all of the issues, pull requests, etc. we get, we'd get NOTHING done. Stop taking shit personally. It's a skill, not a defense mechanism. Nobody has the time to sugar coat every little thing.

Let's normalize directness and stop wasting time during technical discussions into opportunities for ego-stroking and circle-jerking and back-patting.

A young guy I work with burst into tears today, I had no idea what happened so I tried to comfort him and ask what was up.

It appears his main client had gone nuts with him because they wanted him to make an internet toolbar (think Ask.com) and he politely informed them toolbars doesn't really exist anymore and it wouldn't work on things like modern browsers or mobile devices.

Being given a polite but honest opinion was obviously something the client wasn't used to and knowing the guy was a young and fairly inexperienced, they started throwing very personal insults and asking him exactly what he knows about things (a lot more than them).

So being the big, bold, handsome senior developer I am, I immediately phoned the client back and told them to either come speak to me face-to-face and apologise to him in person or we'd terminate there contract with immediate effect. They're coming down tomorrow...

So part my rant, part a rant on behalf of a young developer who did nothing wrong and was treated like shit, I think we've all been there.

We'll see how this goes! Who the hell wants a toolbar anyway?!

The way 90% of the population wears their face masks really explains a lot about their approach to using software, apps & websites as well.

I feel like giving up.

I am not a developer for the salary, or just to solve analytical puzzles. Those are motivators, but my main drive is to make the world more comfortable and enjoyable, better optimized, build ethical services which bring happiness into people's lives. I want to improve society, even if it's just a tiny bit.

But if users invest absolutely zero percent of their limited brain capacity into understanding a product that already has a super-clean design and responds with helpful validation messages...

...why the fuck bother.

I used to think of the gap between technology and tech-incompetent people as an optimization problem.

As something which could be fixed by spending a fortune on UX research. Write tests, hire QA employees, decrease tech debt, create a bold but unified & simple design.

But the technologically incompetent just get more entitled with every small thing you simplify.

It's never fucking fool-proof enough.

Why can't I upload a 220MB PDF as profile picture? Why doesn't the app install on my 9 year old Android Froyo phone? Why can't I sign up if my phone number contains a  U+FFFC? Why does this page load so slowly from my rural concrete bunker in East Ukraine? WHY DO I HAVE PNEUMONIA, HOW DID I GET INFECTED EVEN THOUGH I WAS WEARING A MOUTH MASK ON MY FOREHEAD?

This is why I ran away from Frontend, to Backend, to DBA.

If I could remove myself further from the end user, I would.

At least I still have a full glass of tawny port and a huge database which needs to be normalized & migrated.

Fuck humans, I'm going to hug a server.

I just gave a 20 minute presentation in front of fifty people, and apparently did well enough that I got five private compliments afterwards, including one from the vice president. 🥳

And all of that without a single drop of rum!

Most memorable coworker? Definitely one of our devs in the first company I worked at. He was around fifty, quirky as fuck but damn knowledgeable about pretty much everything. Think some kind of uncle Iroh who could build his own compiler.
I haven't learned as much from university as I learned from our talks during smoking breaks. He never judged anyone for not knowing something (even really basic stuff) and was actually happy if he could help. Now, a few years later I still find myself applying techniques for conceptualizing software he explained to me on the balcony and I have to say I wouldn't be half the dev I am today if I'd have never met him so I guess that counts as memorable.

The king is dead!
Long live the kind!

Bro : What’s the difference between Git & Github?

Me : Its just like the difference between Porn & Porn Hub. 😎

People refusing to learn an improved programming language or technology, because it might contradict to what they "already know" and you're just there mastering it and appreciating its advantages that you will never learn if you had a closed mindset.

It’s true

made my day

Craziest prep for interview :

Step 1 : Given sufficient time for the scheduled interview by any company, start by searching "How to prepare for Google interviews". Awe at the information before you and get all pumped up to jump in.

Step 2 : Starting with Algorithms, study each one and try not to mix any of them in confusion. In case you are stuck in whiteboard coding, close your eyes, take deep breath and visualize Don Kunth. If that doesn't help, well you are ruined anyway.

Step 3: Practice coding without internet connection, till you are able to write code while you talk about how the weather is really great today. Libraries and methods should flow like poetry. SO is sin.

Step 4: The X programming language which you added to your resume because you can write Hello World, head over to Wikipedia and read more about it just in case.

Step 5: Read some xkcd comics so you can impress the interviewer with some humor. You can try Dilbert too.

I hate interviewing with these FAANG or FAACK companies. I just finished 3 rounds of mind torturing coding sessions involving some obscure algorithms that I was expected to come up with on the fly! Thinking that the hard part is over and next is the behavioral round, I just received an email from the recruiter telling me that the next round is some obscure Linux kernel internals I need to be ready for.
How do people get jobs at these companies?!!!

DO NOT LIE ON YOUR RESUMÉ!

I don't understand why people do this. I understand that some shady recruiters like to "gin up" the occasional resumé, but I'm talking about the people who write that they're familiar with MySQL and can't even write a SELECT, or the people who write that they're familiar with Python and can't describe the differences between v2 and v3.

And the interviews are awkward as *fuck*.

I: "So it says you're good at MySQL, could you answer a few questions about it?"

C: "Uhh... okay"

I (sensing danger): "Why would you add an index to table that already exists?"

C: "I.. don't know"

I (oh jesus I see where this is going): "Okay, we'll skip that. How would you query across a couple of tables?"

C: "Uh...."

I ([internally screaming]): "How about a single query on a single table?"

C: "I don't know that, sorry..."

I (desperately wanting to ask why the FUCK is MySQL on your resumé?): "Thank you for your time, we'll call you."

You almost feel sorry for the guy, but come the fuck on, did you think nobody would check?

According to my CTO, this is how I look like when I explain our applications to him:

Bug 🐛

Twitter doing test on production... Interesting. At least it worked 😂

When I first joined the profession, I had a mentor who refused to give me straight-forward answers to my questions / queries. He always had the same answer, "Google it. Find the solution yourself." I hated him for that. Sometimes he used to explain that it was for my own good (blah, blah, the usual stuff) and not because he didn't know or couldn't give me the answer straight-away. I still thought it was just that I was too smart to ask all the right (complicated) questions and he didn't have the answers.
(Of course, that is a bit too exaggerated; he used to help me out with complicated stuff when he knew I was blocked and couldn't move further; he wasn't a sore mentor; he was a good one, in his own way.)

Several years later, I find myself giving the same answers and advice to juniors I mentor. It turns out that push to figure things out on my own did me a lot of good. I'm able to approach any problem head-on and not freak out even if the specs or the deadlines seem surreal. I know how to "figure" answers to problems that I come across for the first time. In the process you learn a lot of stuff that "keep you ahead of the curve and not grow old".

juniors

Tech Industry: “We need more developers!”

NewDev: “Hire me”

Tech Industry: “only experienced developers please! We don’t have time to train juniors ”

Older Dev: “Hire me”

Tech Industry: “no, you want too much money and too much time off“

Mid dev: “Hire me”

Tech Industry: “only experienced devs who are a culture fit!”

Robot dev: “Hire me”

Tech Industry: “You are Hired”