Worst thing you've seen another dev do? Long one, but has a happy ending.

Classic 'Dev deploys to production at 5:00PM on a Friday, and goes home.' story.

The web department was managed under the the Marketing department, so they were not required to adhere to any type of coding standards and for months we fought with them on logging. Pre-Splunk, we rolled our own logging/alerting solution and they hated being the #1 reason for phone calls/texts/emails every night.

Wanting to "get it done", 'Tony' decided to bypass the default logging and send himself an email if an exception occurred in his code.

At 5:00PM on a Friday, deploys, goes home.

Around 11:00AM on Sunday (a lot folks are still in church at this time), the VP of IS gets a call from the CEO (who does not go to church) about unable to log into his email. VP has to leave church..drive home and find out he cannot remote access the exchange server. He starts making other phone calls..forcing the entire networking department to drive in and get email back up (you can imagine not a group of happy people)

After some network-admin voodoo, by 12:00, they discover/fix the issue (know it was Tony's email that was the problem)

We find out Monday that not only did Tony deploy at 5:00 on a Friday, the deployment wasn't approved, had features no one asked for, wasn't checked into version control, and the exception during checkout cost the company over $50,000 in lost sales.

Was Tony fired? Noooo. The web is our cash cow and Tony was considered a top web developer (and he knew that), Tony decided to blame logging. While in the discovery meeting, Tony told the bosses that it wasn't his fault logging was so buggy and caused so many phone calls/texts/emails every night, if he had been trained properly, this problem could have been avoided.

Well, since I was responsible for logging, I was next in the hot seat.

For almost 30 minutes I listened to every terrible thing I had done to Tony ever since he started. I was a terrible mentor, I was mean, I was degrading, etc..etc.
Me: "Where is this coming from? I barely know Tony. We're not even in the same building. I met him once when he started, maybe saw him a couple of times in meetings."
Andrew: "Aren't you responsible for this logging fiasco?"
Me: "Good Lord no, why am I here?"
Andrew: "I'll rephrase so you'll understand, aren't you are responsible for the proper training of how developers log errors in their code? This disaster is clearly a consequence of your failure. What do you have to say for yourself?"
Me: "Nothing. Developers are responsible for their own choices. Tony made the choice to bypass our logging and send errors to himself, causing Exchange to lockup and losing sales."
Andrew: "A choice he made because he was not properly informed of the consequences? Again, that is a failure in the proper use of logging, and why you are here."
Me: "I'm done with this. Does John know I'm in here? How about you get John and you talk to him like that."
'John' was the department head at the time.
Andrew:"John, have you spoken to Tony?"
John: "Yes, and I'm very sorry and very disappointed. This won't happen again."
Me: "Um...What?"
John: "You know what. Did you even fucking talk to Tony? You just sit in your ivory tower and think your actions don't matter?"
Me: "Whoa!! What are you talking about!? My responsibility for logging stops with the work instructions. After that if Tony decides to do something else, that is on him."
John: "That is not how Tony tells it. He said he's been struggling with your logging system everyday since he's started and you've done nothing to help. This behavior ends today. We're a fucking team. Get off your damn high horse and help the little guy every once in a while."
Me: "I don't know what Tony has been telling you, but I barely know the guy. If he has been having trouble with the one line of code to log, this is the first I've heard of it."
John: "Like I said, this ends today. You are going to come up with a proper training class and learn to get out and talk to other people."

Over the next couple of weeks I become a powerpoint wizard and 'train' anyone/everyone on the proper use of logging. The one line of code to log. One line of code.

A friend 'Scott' sits close to Tony (I mean I do get out and know people) told me that Tony poured out the crocodile tears. Like cried and cried, apologizing, calling me everything but a kitchen sink,...etc. It was so bad, his manager 'Sally' was crying, her boss 'Andrew', was red in the face, when 'John' heard 'Sally' was crying, you can imagine the high levels of alpha-male 'gotta look like I'm protecting the females' hormones flowing.

Took almost another year, Tony released a change on a Friday, went home, web site crashed (losses were in the thousands of $ per minute this time), and Tony was not let back into the building on Monday (one of the best days of my life).

One comment from @Fast-Nop made me remember something I had promised myself not to. Specifically the USB thing.
So there I was, Lieutenant Jr at a warship (not the one my previous rants refer to), my main duties as navigation officer, and secondary (and unofficial) tech support and all-around "computer guy".
Those of you who don't know what horrors this demonic brand pertains to, I envy you. But I digress. In the ship, we had Ethernet cabling and switches, but no DHCP, no server, not a thing. My proposition was shot down by the CO within 2 minutes. Yet, we had a curious "network". As my fellow... colleagues had invented, we had something akin to token ring, but instead of tokens, we had low-rank personnel running around with USB sticks, and as for "rings", well, anyone could snatch up a USB-carrier and load his data and instructions to the "token". What on earth could go wrong with that system?
What indeed.
We got 1 USB infected with a malware from a nearby ship - I still don't know how. Said malware did the following observable actions(yes, I did some malware analysis - As I said before, I am not paid enough):
- Move the contents on any writeable media to a folder with empty (or space) name on that medium. Windows didn't show that folder, so it became "invisible" - linux/mac showed it just fine
- It created a shortcut on the root folder of said medium, right to the malware. Executing the shortcut executed the malware and opened a new window with the "hidden" folder.
Childishly simple, right? If only you knew. If only you knew the horrors, the loss of faith in humanity (which is really bad when you have access to munitions, explosives and heavy weaponry).
People executed the malware ON PURPOSE. Some actually DISABLED their AV to "access their files". I ran amok for an entire WEEK to try to keep this contained. But... I underestimated the USB-token-ring-whatever protocol's speed and the strength of a user's stupidity. PCs that I cleaned got infected AGAIN within HOURS.
I had to address the CO to order total shutdown, USB and PC turnover to me. I spent the most fun weekend cleaning 20-30 PCs and 9 USBs. What fun!
What fun, morons. Now I'll have nightmares of those days again.

Background: I'm not drunk yet, BUT I'M WORKING ON IT.

okay.

I just finished a second sprint on my React app. The first was to build a merchant onboarding flow. The second was to do substantial cleanup as I learned more about react/redux, and to create a "supply order" flow -- basically purchasing marketing materials and services. I finished that in a week, and I'm pretty proud. api-guy wanted it done in a day. i laughed. he probably could have, but it would have been a copy of the code in a new repo with some lines changed.

ANYWAY. it's all done and It's super pretty and works amazingly well. It has both the onboarding flow and the ordering flow, with a nice pop-out sidebar for navigation, namespaced actions, etc. Everything is pretty clean. I even added a cart to the ordering (despite everyone telling me not to) because wtf, what if someone wants to order TWO items? dumbasses. So I made that. it's sexy.

Anyway, it's all done and shiny and fancy and wonderful and I'd *love* to share screenshots if only it didn't give away where I worked. :<

... but the point of the rant!

After the first sprint, I made a copy of the repo so I could rework it and add more functionality without touching the original. (Hey! That's what a branch is for, right? Why didn't I branch it up?
well, read on)

I knew we were going to have multiple separate flows for this app: onboard, ordering, merchant tools, admin tools, support, etc. So, I wrote its server portion (the webpack builder + http server) so it would serve the same app at whatever url the user hit, and set a cookie containing that host+url. This allows the app to serve different content (basically showing/hiding content) based on the URL and future login roles. If someone hits /order, it would hide everything but the order flow. If they're a merchant, it would show all the merchant views plus ordering, etc.

tl;dr This way I can use the same codebase for multiple sites, drastically simplifying development, branding, and what have you. This new app could obv also be a drop-in replacement for the original onboarding project because of the above.

HOWEVER. this apparently isn't good enough for api-guy. He's terrified that adding/updating future components will affect all the existing content somehow.

so.

now we have three repos for basically the same codebase. 1) onboard aka "surfboard", 2) ordering, 3) merchant tools, aka "ferrari" (the "future" app).

Except.
1) "surfboard" is a very old version of the code. 3) "ferrari" is also old, since 2) "ordering" has newer content in it now.

... and somehow this is better?
fuck if i can figure out how.

His reasoning is "well, you won't be touching surfboard or ordering for 6 months, so now you don't have to worry about it." Sure, except, you know, it'll be a pain in the ass in 6 months now when I have a crapton of code and branding to redo. ffs.

Oh. We also have three Heroku pipelines for these three repos. for the same codebase.

and now you know why i'm drinking.

me: the source code is currently store on GitHub and we use GitHub Actions after each updates to compile your code into binary before deploying to your servers
client: storing source code on GitHub (external server) is insecure and breaks compliance
me: so i guess you will need to have a copy of the source code on all your servers and build them directly there (too cheap to have a separate build server) instead of using GitHub Actions
client: yeah
me: keep in mind that all your certificates and tokens are going to be store as plain text in all your servers so if a hacker gain access to anyone of your servers, they will have access to everything.
client: yeah, this is in compliance to our security policy

A bot just made 519 pull requests with malicious Makefile code to get a github actions server to send a curl to a random host.

It's gonna be one of those days

Finally finished the screwdriver followup ticket. I think.

I spent almost two full days (14 hours) on a seemingly simple bug on Friday, and then another four hours yesterday. Worse yet: I can’t test this locally due to how Apple notifications work, so I can only debug this on one particular server that lives outside of our VPN — which is ofc in high demand. And the servers are unreliable, often have incorrect configuration, missing data, random 504s, and ssh likes to disconnect. Especially while running setup scripts, hence the above. So it’s difficult to know if things are failing because there’s a bug or the server is just a piece of shit, or just doesn’t like you that day.

But the worst fucking part of all? The bug appeared different on Monday than it did on Friday. Like, significantly different.

On Friday, a particular event killed all notifications for all subsequent events thereafter, even unrelated ones, and nothing would cause them to work again. This had me diving through the bowels of several systems, scouring the application logs, replicating the issue across multiple devices, etc. I verified the exact same behavior several times over, and it made absolutely no sense. I wrote specs to verify the screwdriver code worked as expected, and it always did. But an integration test that used consumer-facing controller actions exhibited the behavior, so it wasn’t in my code.

On Monday while someone else was watching: That particular event killed all notifications but ONLY FOR RELATED EVENTS, AND THEY RESUMED AFTER ANOTHER EVENT. All other events and their notifications worked perfectly.

AKL;SJF;LSF

I think I fixed it — waiting on verification — and if it is indeed fixed, it was because two fucking push event records were treated as unique and silently failing to save, run callbacks, etc.

BUT THIS DOESN’T MATCH WHAT I VERIFIED MULTIPLE TIMES! ASDFJ;AKLSDF

I’m so fucking done with this bs.

Today in development: discovered that it's possible via combination of keys to rename a database in SQL Server Management Studio without as much as a dialog box to confirm.

Shout out to the 2000ish users in production that discovered this delightful nugget of info with me.

Lessons learned:

A) Don't trust Microsoft to create software that makes you confirm potentially catastrophic actions

B) Make sure your user hasn't been granted ALTER DATABASE permissions without your knowledge before you start using it.

Our project at work goes live in 3 weeks.

The code base has no automated tests, breaks very often, has never had any level of manual testing

will not be releasing with any form of enforced roles or permissions in our first release now due to no time to enforce, however there is a whole admin api where you can literally change anything in our database including roles.

We also have teams in various countries all working separately on the same solution using microservices with shared nuget packages and they aren't using them properly.

Our pull requests are so big - as much as, 75 file changes - in our fe app that I can't keep up with it and I honestly have no idea if it even works or not due to no automated tests and no time to manually test.

We have no testing team, or qa team of any sort.

Every request into the system has to hit a minimum of 3 different databases via 3 different microservices so 1 request = 4 requests with the load on the servers.

We don't use any file streams so everything is just shoved in the buffer on the server.

Most of the people working on the angular apps cba to learn angular, no one across 2 teams cba to learn git. We use git so they constantly face problems. The guy in charge has 0 experience in angular but makes me do things how he wants architecturally so half the patterns make no sense.

No one looks at the pull requests, they just click approve so they may as well push directly to master.

Unfinished work gets put in for pull request so we don't know if the app is in a release state since aall teams are working independently, but on the same code base.

I sat down and tested the app myself for an hour and found 25 fe only issues, and 5 breaking cross browser issues.

Most of our databases are not normalised. Most of our databases make no sense. 99% of our tables have no indexing since there is no expertise with free time to do it.

No one there understands css properly. Or javascript.

Our. Net core microservices all directly use ef in the controller actions so there is no shared code there.

Our customer facing fe app is not dry because no tests so it was decided it was better this way.

Management has no idea on code state, it seems team lead is lieing to them about things like having any level of tests.

Management hire devs that claim to be experts but then it turns out they have basically no knowledge of what they were hired to do, even don't know what json is or the framework or language they are hired for, but we just leave them to get on with it and again make prs too big to review.

Honestly I have no hope that this will go well now but I am morbidly curious to watch. I've never seen anything like the train wreck that we are about to get experience.

Me: Oh, man, there are hooks for react-redux now? I don’t have to wrap components in a higher order component to get information from the redux store and dispatch actions? Could this solve the problem I’m having with data fetching and consistency in the app I’m working on?!

Spends entire Saturday writing a basic server, connecting to an mLab instance, filling said instance with dummy data, starting a create-react-app, writing a reducer, action creators, components, etc. just to test how useSelector(), useEffect(), and useDispatch() would work in an application that isn’t just a simple counter (why is it that like every example is always the counter example?!). Bonus, react-router now ALSO has hooks, so got to play with useHistory() and useParams()

Conclusion: Maybe. It does appeal to me to not have the cascade of virtual DOM that you always get nesting HOCs, but I’m also wary of appearing too willing to jump on it just because it’s the new thing.

Has anybody else played around with react-redux hooks? Your thoughts?

Also, yes, I know, not every app needs redux. It had it when I was brought on and I don’t really have the ability to change that implementation detail now.

Oh no AI can destroy hummanity in the future! It is like skynet and such... Bad! It will be the end! FEAR THE AI!

Yeah so i cant sleep now so im writting a rant about that.

What a load of bullshit.

AI is just a bunch of if elses, and im not joking, they might not be binary and some architectures of ML are more complex but in general they are a lot of little neurons that decide that to output depending on the input. Even humans work that way. It is complicated to analyse it yes. But it is not going to end humanity. Why? Because by itself it is useless. Just like human without arms and legs.

But but but... internet.... nukes... robots! Yeah... So maybe DONT FUCKING GIVE IT BLOODY WEAPONS?! Would you wire a fucking random number generator to a bomb? If you cant predict actions of a black box dont give it fucking influence over anything! This is why goverment isnt giving away nukes to everybody!

Also if you think that your skynet will take control of the internet remember how flawless our infrastructure is and how that infrastructure is so fast that it will be able to accomodate terabytes per second or more throughput needed by the AI to operate. If you connect it to the internet using USB 2.0 it wont be able to do anything bloody dangerous because it cant overcome laws of physics... If the connection isnt the issue just imagine the AI struggle to hack every possible server without knowing about those 1 000 000 errors and "features" that those servers were equiped with by their master programmers... We cant make them work propely yet alone modify them to do something sinister!

AI is a tool just like a nuclear power. You can use it safely but if you are a idiot then... No matter what is the technology you are going to fuck shit up.

Making a reactor that can go prompt critical? Giving AI weapons or controls over something important? Making nukes without proper antitamper measures? Building a chemical plant without the means to contain potential chemical leak? Just doing something stupid? Yeah that is the cause of the damage, not the technology itself.

And that is true for everything in life not only AI.

Gift that keeps on giving

Three days ago my focus was shifted from a development role to a support role. I was shifted to replace another support guy who had used fraud to get the position. I have no experience with this role but there was decent KT and I'm catching on fine. During onboarding and KT I'm serving as the first contact for new tickets and whatnot...

Today I got a ticket with an error on our production instance that no one had ever seen before. It prevented the guy from using our service entirely. I tried to reproduce it and... I couldn't use the service either. No one could. Everything was down. I could see the sweat building on my manager's forehead.

Thankfully another member on my team has done a bit of support before, so we collaborated with each other and other teams throughout the day to figure out what's wrong and how to fix it. I'm listening to them chat remotely as we speak - so far I've been working on it 9 hours straight.

This service is used by everyone - it's a business critical service with due dates on actions and escalations to managers... Imagine if the support ticketing service for your company crashed. That means a lot of people are asking what's wrong, requiring extensions, etc. I've been answering to managers and seniors in the business throughout the day.

The best part? We figured out why the server went down, and the reason is fantastic: someone updated the server's code without telling anyone, and all they had done was remove critical parsing code. Just took it right out, pushed, redeployed. We don't know who did it or who even has access to do that. I guess I have some detective work cut out for me after we've fixed everything that was broken by that.

I miss coding already.

Dear Webmin,

how is it that you fail to update and fuck up every Apache config file existing on the server.

Why can't I just be a lazy dev tonight, instead of fixing your moronic actions upon those files, one by one.

Why is it that you frigging forget to close Directory tags properly.

Why is it that you show a Forbidden page when everything seems to be finally ok.

And why is it that I can not re-generate that shit with one button.

Fuck this shit.

sudo rm -rf /

This has been bothering me for a while. I have an old freelance client of mine I’ve created an web site for (his company) it was small one so I took the complete payment before deployment and I needed no contract. I deployed the complete version of the site on my server, bought the domain for his company under my name and it has been running for a year now.
Lately he had asked me to give admin privileges to his son (cs student 1y) to upload some photos of their new building. I noticed he ruined several functions on the site in doing so, but I was never paid to support that just the hosting for a year.
When I was making the design I made a simple but pretty logo as a placeholder for the site which went in production since they never gave me company logo. All good, no contract small cash all delivered, everyone happy.
Up until few days when I saw my f**king logo cut out from the site as 250px jpeg and made as a huge banner on the company building..
From my pov I would’ve never given permission to use that since its not something i’m proud of and would suggest to make a better one for a fee. I see this as stolen/unauthorized use of intellectual property. But the laws are super shitty in our country so at this point I am stuck at taking their site, domain a hostage until they pay for the logo they used or take it down or taking legal actions.. we never signed anything about that logo.

I am slowly turning my home into an automated smart home, however. I have found a lot of responding devices (media players, sockets, etc..) but no trigger devices (buttons, sensors) I can work with.

Am I looking in the wrong place or do I really have to build something myself using arduino?

My setup is the following: I have a central server in my home that hosts a bunch of docker services that all server a certain purpose. All smart devices have static ips so that server can address them quickly. So it is capable of controlling many things. However, now I want to trigger certain actions through a hardware button. It seems I cannot find such a device....

Any other hads on here?

Omg nextjs 14 is so good. I cant believe this. Server actions are so powerful. This shit makes you prototype and move RAPIDLY FAST. And the framework itself is fast as cum! Unbelievable. No wonder every website lately is built in nextjs. This framework is definitely the future of web. It made working with databases blazingly simple. Prisma ORM is unbelievably flexible. The shit you can build with this framework has no fucking limits! It has /api folder to just add restful apis and just reuse the same prisma methods from shared lib functions and boom you can now scale the project to a mobile app!

All of this bullshit took me YEARS to learn how to do properly in a regular frontend-backend separated type of project. While I learned this nextjs shit blazing fast. Am i missing something or is this framework too good to be true?

I'll bend over for nextjs

For the people experienced with Stripe and C#, given that I'm in a Server+WebAssembly context, what's your experience setting up the payments?
What would look like the flow of actions to pursue a charge?

Nextjs 14 just came out and they added a new server actions syntax which is the same bullshit syntax like php where you insert server side code in the middle of html div! And not only is that ridiculous enough but also vulnerable to sql injection 😂😂😂