Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "friday is deploy day"
-
Currently on an internship, PHP mostly, little bit of Python and the usual web stuff, and I just had the BEST FUCKING DAY EVER.
Wake up and find out I'm out of coffee, oh boy here we go.
Bus leaves 10 minutes late, great gonna miss my train.
Trains just don't wanna ride today, back in a bus I go, what's normally a 10 minute train travel is now a 90 minute bus ride.
Arrive at internship, coffee machine is broke, non problem, I'll just lose it slowly.
NOW HERE COMES THE FUCKING GOOD PART!!
Alright, so I'm working on a CMS that can be used just about on any device you want, mobile or desktop, it's huge, billion's of rows of scientific data. Very specific requirements and low error margins. Now, yesterday I was really enjoying myself here until today, Project manager walks in, comes to my desk and hands me a Samsung Gear S3, an Apple watch and some cheap knockoff. He tells me that before the Friday deploy, THE ENTIRE CMS SHOULD WORK ON THOSE WATCHES!
I mean, don't get me wrong, I like a challenge but it's just not right, I mean, I'm still not sure what the right way to handle tables on phones is, but smart watches, just no. Besides that, I've never worked with any Apple devices, let alone WatchOs, nor have I worked with Android Wear.
Also, Project Manager is a total dickhead, he's the kinda guy that prefers a light theme, doesn't clean up his code, writes 0 documentation for an API, 1 space = tab, pure horror.
So after almost flipping my desk, I just called my school coach to announce I'm leaving this internship. After a brief explanation he decides to come over, and guess what, according to the Project Manager I wasn't supposed to do that, I was supposed to test if it would be possible.
FUCKING ASSFUCKFACE9 -
I had a huge epiphany on Friday... not all developers enjoy coding.
Discovered when they brought down 2 of our environments, well told them what was wrong with the changes in their code that caused the environments to break, gave them links directly to the file in the gitlab repo that needed to be updated, and...
They fucking went home. The change would’ve taken all of about 30-45 seconds to update and they fucking left.
This person’s team lead come storming in pissed off because her manager is furious about 2 environments going down and preventing everyone else from being able to deploy their changes.
We provide the exact same details to the team lead about what needs to be changed, and advise that her team member took off....
30 mins later, her manager is storming up to us (devops/sre) livid as hell.
Explain the situation for a third time... manager is like, why can’t you guys fix it?
Look here you dense motherfuckers, we can fix the code. We can be the plumbers that clean up your shit. But what value do you gain as a developer if you don’t understand how the systems work and you keep pushing shit in?
Made the changes, fixed the environments, done right? Wrong.
The original developer made more changes not knowing what would happen and thoroughly fucked the environments again.
This dumb-fucking dumpster fire of a dude then sends us a slack message. “It’s down again, can you fix it?”
Our manager steps in and tells us to send him a link to the logs and have him fix it himself!
Thank goodness we have a badass manager.
Send logs, send repo file links (again), and send line numbers in the logs to try and help just a bit more. Dude goes almost the whole day without fixing it, environments are down, other devs are pissed, we throw this dude to the wolves. His manager starts to head over and was about to talk with my team lead when our manager steps out of his office and tells him the in’s and out’s of the situation and that our job isn’t to play log parser/error fixer for the developers. This dude that’s breaking the environments needs to be the one to fix the issue and his team lead should be aware of the problems and should have been able to correct his errors before it ever came to us.
The amount of hand-holding we do is ridiculous.
(Disclaimer, this one guy making some mistakes doesn’t sound too bad, but this is actually a common occurrence for like 40% of all of our developers)
We literally have interns still in college running circles around some of our full time devs. I know I’m not a developer, but for anyone that’s new-ish to developing, when you see shit like that please don’t lose hope. Those ass-hats got into programming purely for a paycheck, not because of passion.
Stick with it and your greatness will know no bounds 👍
As for you craptastic dipstick lickers, FUCK YOU!!! Go back to school and learn how to give a damn.4 -
Long rant ahead.. 5k characters pretty much completely used. So feel free to have another cup of coffee and have a seat 🙂
So.. a while back this flash drive was stolen from me, right. Well it turns out that other than me, the other guy in that incident also got to the police 😃
Now, let me explain the smiley face. At the time of the incident I was completely at fault. I had no real reason to throw a punch at this guy and my only "excuse" would be that I was drunk as fuck - I've never drank so much as I did that day. Needless to say, not a very good excuse and I don't treat it as such.
But that guy and whoever else it was that he was with, that was the guy (or at least part of the group that did) that stole that flash drive from me.
Context: https://devrant.com/rants/2049733 and https://devrant.com/rants/2088970
So that's great! I thought that I'd lost this flash drive and most importantly the data on it forever. But just this Friday evening as I was meeting with my friend to buy some illicit electronics (high voltage, low frequency arc generators if you catch my drift), a policeman came along and told me about that other guy filing a report as well, with apparently much of the blame now lying on his side due to him having punched me right into the hospital.
So I told the cop, well most of the blame is on me really, I shouldn't have started that fight to begin with, and for that matter not have drunk that much, yada yada yada.. anyway he walked away (good grief, as I was having that friend on visit to purchase those electronics at that exact time!) and he said that this case could just be classified then. Maybe just come along next week to the police office to file a proper explanation but maybe even that won't be needed.
So yeah, great. But for me there's more in it of course - that other guy knows more about that flash drive and the data on it that I care about. So I figured, let's go to the police office and arrange an appointment with this guy. And I got thinking about the technicalities for if I see that drive back and want to recover its data.
So I've got 2 phones, 1 rooted but reliant on the other one that's unrooted for a data connection to my home (because Android Q, and no bootable TWRP available for it yet). And theoretically a laptop that I can put Arch on it no problem but its display backlight is cooked. So if I want to bring that one I'd have to rely on a display from them. Good luck getting that done. No option. And then there's a flash drive that I can bake up with a portable Arch install that I can sideload from one of their machines but on that.. even more so - good luck getting that done. So my phones are my only option.
Just to be clear, the technical challenge is to read that flash drive and get as much data off of it as possible. The drive is 32GB large and has about 16GB used. So I'll need at least that much on whatever I decide to store a copy on, assuming unchanged contents (unlikely). My Nexus 6P with a VPN profile to connect to my home network has 32GB of storage. So theoretically I could use dd and pipe it to gzip to compress the zeroes. That'd give me a resulting file that's close to the actual usage on the flash drive in size. But just in case.. my OnePlus 6T has 256GB of storage but it's got no root access.. so I don't have block access to an attached flash drive from it. Worst case I'd have to open a WiFi hotspot to it and get an sshd going for the Nexus to connect to.
And there we have it! A large storage device, no root access, that nonetheless can make use of something else that doesn't have the storage but satisfies the other requirements.
And then we have things like parted to read out the partition table (and if unchanged, cryptsetup to read out LUKS). Now, I don't know if Termux has these and frankly I don't care. What I need for that is a chroot. But I can't just install Arch x86_64 on a flash drive and plug it into my phone. Linux Deploy to the rescue! 😁
It can make chrooted installations of common distributions on arm64, and it comes extremely close to actual Linux. With some Linux magic I could make that able to read the block device from Android and do all the required sorcery with it. Just a USB-C to 3x USB-A hub required (which I have), with the target flash drive and one to store my chroot on, connected to my Nexus. And fixed!
Let's see if I can get that flash drive back!
P.S.: if you're into electronics and worried about getting stuff like this stolen, customize it. I happen to know one particular property of that flash drive that I can use for verification, although it wasn't explicitly customized. But for instance in that flash drive there was a decorative LED. Those are current limited by a resistor. Factory default can be say 200 ohm - replace it with one with a higher value. That way you can without any doubt verify it to be yours. Along with other extra security additions, this is one of the things I'll be adding to my "keychain v2".11 -
So i just saw a post about not pushing to production on Friday's and that reminded me of something.
A team we are working with (we needed help on something so we got another company to send us a team and do it for us) recently push to production on a Friday at around 5pm.
We didn't know that they had done it until a shit ton of errors started happening and we had no idea why (it was working so far, none of us did anything, so what the fuck is going on).
To make things better, one of my colleagues tried calling them and they wouldn't answer the phone. They knew what they did, so they went home before we could notice it and we had to correct their mistake.
We had everyone calling us saying they had gotten an error, and they needed to get home but couldn't because of it (it messed up a process that only happens at the end of the day before people go home. I can explain more in the comments if anyone is confused, or just wants to know).
Eventually everything was solved, but that was a very stressful ending of the week.
So yeah, don't deploy on Friday's please and thank you7 -
I was thinking about the problems one of our clients faced with the launch of their project the other day, because things were rushed, stuff was omitted and in the end they could not meet the launch date, and I started making a list of hard lessons I learned over the years that would have helped them avoid this situation.
Feel free to add yours in the comments.
- Never deploy on Friday
- Never make infrastructure changes right before a launch
- Always have backups. Always!
- Version control is never optional
- A missed deadline is better than a failed launch
- If everything is urgent, nothing is important
- Fast and cheap, cheap and quality, quality and fast. Only one pair at a time can be achieved
- Never rush the start or the end of a project
- Stability is always better that speed
- Make technical decisions based on the needs of the project two years from now
- Code like you will be the only maintainor of the project two years from now. You probably will...
- Always test before you deploy
- You can never have too many backups (see above)
- Code without documentation is a tool without instructions
- Free or famous does not necessarily mean useful or good
- If you need multiple sentences to explain a method, you should probably refactor
- If your logic is checked beforehand, writing the code becomes way easier
- Never assume you understand a request the first time around. Always follow up and confirm
There are many more that should be on this list, but this is what came to mind now.2 -
I've just been told that I'll be alone in the office this friday, with only a handfull of easy tasks.
It's rather tempting to bring a box of beer and have some fun on the keyboard. 🥳4 -
A top food chain client wants a feature Fx
and has a deadline on Friday.
We are still working on it and already estimated hours and set deployment on Monday.
(No deployments on Friday)
And the business/sales guy comes up with new deadline to submit it at Friday morning.
And was only discussing with one of my team member already working on it. And i knew there is more hours required for testing and need to deployment pre deployment phase (staging of dev)
I was over hearing the conversation between them and I got pissed off and jumped in and said Not Possible at all.
He tries to argues about giving something to him. I said we can give it to you but will not garauntee anything. Now project manager jumps in. PM and my team already know that we will be delivering on Monday.
He arguing that if the Fx is not ready then I will call client developer to office to test it directly on my team members laptop.
I said, No way. We are not ready yet and havent finished yet. Major work will be on Thursday and on Friday we will be testing till end of the day.
PM explains him blah blah stuff.
He calms down and says no worries we will check the status on Friday afternoon amd roll out something to Client.
PM, developer and I looked each other and I said, sure will deploy but will not garauntee anything. He goes back to his desk.
Seriously.
WE ALREADY ESTIMATED F* MAN HOURS AND WILL BE READY ON MONDAY MEANS MONDAY DONT F* BUILD MORE PRESSURE ON US. F* SALES2 -
!Rant
Tldr: great spike to solve deployment problem may be a wasted effort.
Deployments of an ancient electron application need to be done in CodeDeploy to deploy the latest build. Customer hour restrictions cause this to be done only after midnight, and manually checked.
The whole team knows this is the wrong method of deployment and that there are many other operational problems with the project.
A few other senior team members get together and decide to spike out a way to use electron auto-deployment to accomplish this without using code-deploy at all.
After a shallow dive into this subject, we all get pulled aside to handle a change in another part of the software ecosystem. It happens. We leave the spike behind.
A junior-intermediate developer on the team pics the project up and gets a good spike going in a day and a half! We are all high fives and beers. This is Friday.
By Monday there is a pull request in for code review and it looks solid. Seems like it will make deployments a lot better.
Preparing the last deployment (hopefully) with CodeDeploy ever...
Marketing team members inform us that they are running an add system on the customer devices and to do it they are using Linux.
The current application being deployed is using Windows 10 (yeah, another problem).
They say they have made plans to move our application over to Linux. This means we may not be able to launch the junior devs great spike and the old deployment method may stay for the time being.
Meetings soon to find out how all of this will hash out.
End of rant. I hope I'm doing this right -
Soo, lets talk about deadlines and how often I come in trouble when there is one near. This week I have (had) three. So I had a lot of problems this week.
But, after all today was a really good day.
I deleted an production database, fucked up my laravel migrations and tomorrow we have to deploy another application. The app is still in development and the site isn´t finished yet.
Friday I need to launch another webapplication. Because I don't want to deploy webapps and apps on friday and I go snowboarding next week, I somehow convinced my client to launch it next week. He agreed with me, so that's a burden off my shoulder. And I'll have an extra day for fixing the last bugs.
Today I worked really hard and almost finished the last application. So I think after all I'll somehow manage it.1