Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
Get a devDuck
Rubber duck debugging has never been so cute! Get your favorite coding language devDuckBuy Now
Search - "cron jobs"
So that high level prank from yesterday.
Senior Linux engineer, the fucker.
He somehow installed shitloads of cron jobs onto my system.
Every few minutes it would create a new user with a freaking complicated password. Then it would install openssh server in case it wasn't installed yet. After that it'd set all iptables rules to allow incoming AND outgoing connections on port 22.
That was one badass ansible script though!
I'm not sure what more there's to it because sometimes when i removed crons, they'd magically appear again later AND i forgot to check the boot scripts so i might be fucked again when I get to work today!
Plus side, i finally fully understand cron 😅19
***Interviewing potential sys admins so us devs don't have to build everything and run everything***
Coworker: Do you know how to use cron and cron jobs?
Candidate: Yes I'm familiar with setting up users and permissions.
Boss: We will give you a call have a good day.
If you had just admitted you didn't know but we thought you could learn we might have been open to teaching you but brazenly acting like you know something when you don't is dangerous if you're running a multi thousand user production system.3
If the java programming language were a human I would lock it in my basement and videotape a timelapse of it starving to death. I hate java. I'm irretrievably revolted by its existence. I actually just threw up in my mouth a little bit thinking about the possibility that there's a chance that I might have to write three lines of it before I die by stabbing myself to death as a result of having to write three lines of code in java. I think I would rather wrestle ten alligators than be within 100m of a computer containing a storage device that has binary code for the jvm stored on it, running or otherwise. I have cron jobs that search my system for any code that is stored in java bytecode format on my hard drives, and purges it with a 41 pass government wipe no questions asked once every hour just because it leaves behind a disgusting taste if it's not properly purified. I turn over and over in my bed when I think that some high school or college student somewhere is learning java and not wearing the appropriate hazmat suit while handling it and their children will be born with horrible deformities because of it. And they don't even know. I'm angry that president trump has not built a wall around oracle corporation and ordered the banishment of all of their employees that have contributed to java for a permanent vacation on Guantanamo bay.
Please say with me the prayer of the gnu:
Oh bless thee gnu that though might purge my computer of lesser proprietary softwares and software specifications, and deliver us to holy lands of free coreutils and runtime environments where our children's children may live without sin and curse the dark age of fucking bullshit languages, in thy mercy; and they will raise thy holy license up on high and say the great gnu prayer as you taught us to say: our father, who art Richard stallman, hallowed be thy name. Thy kingdom come. Thy bug tickets be resolved. Give us this day our daily blog entry, and forgive us our sins, as we consider but ultimately pass on forgiving jvm engineers for ruining the world, and lead us not into temptation, but deliver us from windows, for thine is the kingdom, and the power, and the glory forever amen.
So let's get back to java. It fucking sucks, I despise the fact that anybody believes it holds any advantages over c++ as it's memory model is fucked up in plenty of ways, and many critical bugs have been uncovered and fixed in the garbage collector over the years. It's hated by many. It's father was an investment banker, and it's mother won the Darby.55
So I looked at our dashboard and noticed a banner mentioning scheduled maintenance set for 7:00 AM. And I thought to myself, "I never released an update, and even if I had, the maintenance would be performed 15 minutes after the build finished, not at 7:00 AM." So I emailed my coworkers, asking if they had put up the banner, no, no. I started pulling my hair out trying to figure out what caused this banner to be created. Was there some old job that was just now running? I combed through the server logs, thousands of entries later, and I found the banner was installed by some user with the IP 172.18.0.1...which was the local machine. I went through all the users on the system, running atq to see if anyone had jobs scheduled. And there was one job scheduled, under the root user. At that moment, I legit thought to myself, "have we been hacked? How is that possible?" It's wasn't! Then I looked under /var/spool/atjobs to see what the job actually was. And then I saw it. My weekly updater cron job had installed updates and had scheduled a maintenance window to reboot the system. And I smiled, realizing that my code was now sentient.1
My coworker doesn't know how to use a terminal. He talked himself into his position and instead of taking the time to learn about the basic commands he keeps asking someone else (including the teammanager, who's actually a software engineer) to do things for him.
For reference; we need the terminal to tail log files, keep track of processes, cron jobs, manipulate file structures, use scp (I use sshfs) to move things between other workstations and servers etc. Being able to use a terminal is one of the basic requirements for our job.
Why do people do this?2
Going to create a couple of cron jobs that delete stuff from database.
What could possibly go wrong? ;)6
I hate time.
Yes, that dimension which unidirectionally rushes by and makes us miss deadlines.
Also yes, that object in most programming languages which chokes to death on formatting conversions, timezones, DST transitions and leap seconds.
But above all, I hate doing chronological things from the point of view of code, because it always involves scheduling and polling of some kind, through cron jobs and queues with workers.
When the web of actions dependent on predicted future and passed past events becomes complicated, the queries become heavy... and with slow queries, queues might lock or get delayed just a little bit...
So you start caching things in faster places, figure out ways to predict worker/thread priorities and improve scheduling algorithms.
But then you start worrying about cache warming and cascading, about hashing results and flushing data, about keeping all those truths in sync...
I had a nightmare last night.
I was a watchmaker, and I had to fix a giant ticking watch, forced to run like a mouse while poking at gears.
I fucking need a break. But time ticks on...2
This was about 3 years ago. I’m on vacation and just getting off the plane, when my boss calls me on his cellphone. Apparently the crontab on our main file upload server had gotten nuked, and he was asking if there were any backups.
A word about this server. I work with video, so this thing is doing about a few gigabits of traffic incoming at any moment. The cron jobs are necessary to move and organize these massive files into a sane scheme for processing. Hundreds of drop folders receiving thousands of files resulting in terabytes of data every single day. Our storage vendor tells us we have the third largest deployment they know about.
No cron jobs mean all of this content is just sitting around piling up. I tell him sorry, try contacting $otherAdmin since he’s more familiar with that system.
A few days later, after the vacation, I come back in. $boss and $otherAdmin have reconstructed the crontab from scratch after an all nighter.
I ask how it got deleted.
$boss was training some people how to set up new customers on this file server, and he told the trainees to open the crontab in read-only mode. One of them ran:
Yes, we back up our crontabs now.3
Saturday 9.00 AM. I was sleeping, my colleague (on holiday) sent me a text: "We got a problem on our system, probably we ran out of space". I checked the log and found out that several cron jobs failed due to not enough space on the disk. I started deleting some unnecessary logs (we're paranoid) and ended up to squeeze the vm like a lemon to save some space. Sent an email to the sysadmin, "We got to add more space ASAP, users are getting 500 errror for almost everything". Silence. I thought to myself: "Until monday we're safe..". I did a df (96%) and sent a screen to the sysadmin, just to be sure that we understood each other. Finally monday comes, nobody worries about the issue. At noon I literally takled the guy of IT dept. "Yeah, we read your email. I think the sysadmin didn't take you seriously". "Why? Which part of 'we're running out of space' isn't serious?!!!". "He just told me that we have unlimited space on that vm". Unlimited space...sure.... "Right.....the disk is at 96%, buuuuut if he said so No news to worry. Don't call me if everything burns. Have a good day!!!"4
6 different package managers to keep up to date.
Good thing bash scripts and cron jobs exist5
Work has been inefficiently using multiple cron jobs to run php scripts to generate pre-baked data.
The last two days I took the steps needed to internalize all those scripts and run them from an individual php controller which is ran from Jenkins. My script keeps track of scheduling and error tracking.
I'd say I'm pretty proud of what I came up with.1
I hate cron jobs. Hours of googling and double checking. My job is perfect. Still doesn't run according to the logs3
So I was instructed today, after lunch, to spend an hour teaching a member of my team how to SSH, store keys, basic io routines, and create CRON jobs to auth our ECR registry by my team lead.. Why am I wasting dev time teaching someone how to use an operating system? Need I add, our primary Dev workspace is a spun up using vagrant using xubuntu. I just can't comprehend how this person has been using xubuntu as their primary OS for two months and doesn't know the SSH protocol. Much less how they landed a dev job without any prior experience with a *NIX based OS.2
Manual EC2 instances + Elastic load balancer or Elastic beanstalk for a PHP 7 application? I might have some cron jobs to be run too...
CRON JOBS SUCK. @LINUX YEAH YOU HEARD ME
MY PROGRAM WRITES INFO TO A DATABASE, SENDS EMAILS AND OUTPUT IS PIPED TO A LOG FILE. NONE OF THESE THINGS HAVE OCCURRED DURING THE CRON RUN SO I DON'T KNOW WHAT IS OR ISN'T WORKING.5