Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple APILearn More
Search - "sysadmin"
What a stupid configuration of firewall at my work:
devrant -> blocked because of entertainment category.
xvideos -> no problem at all.
Conclusion: sysadmin likes watching porn.13
Me: so, ifconfig, what is my gateway?
ifconfig: [ip address]
Me: nmap, what is this IP address?
nmap: it's a network switch with an open telnet port.
Me: what happens if I connect to it?
switch: WHAT IS THE PASSWORD?!?!
Me: is it blank?
switch: correct. what do you want to do?
Me: can I look at all the IP and MAC addresses on the network?
switch: WHAT IS THE ADMIN PASSWORD?!?!
Me: is it... admin?
switch: correct. Here's everyone that's connected to the network: [400+ IPs and MACs]
Me: ok python, would you filter through these and tell me what manufacturer each one belongs to?
[~50 manufacturer lookups later]
python: there's a bunch of apple product, a bunch of miscellaneous laptop and printer manufacturers, and some raspberry pis.
Me: raspberry pis?
python: yep. about 20 of them.
Me: What happens if I connect to one?
rpi: WHAT IS THE PASSWORD?!?!
rpi: correct. what do you want to do?
Me: can I make you do my bidding in the background when you aren't being used?
rpi: sure, sounds fine.
I love ignorant sysadmins.8
"You should use Windows server!"
It was a high security project which needed to run very stable. Even the windows sysadmin looked at that guy like 'dude what the actual fuck'.27
Happy SysAdmin day ... even if I’m wondering if sysadmins can be happy.
Source : xkcd (of course...)3
Senior IT engineer enters the room and quietly talks to a coworker about a job related issue.
Another coworker decided to troll the sysadmin.
CW: *yells* "Open a ticket!" (That's the sysadmin's regular reply)
CW: *trying to get his attention* "Open a ticket first! Then come back"
IT: *gives him the stare of death*
CW: "Go away and open a ticket!"
IT: *silently leaves the room*
After no more than a minute CW gets a reject from all networks outside the company's VPN.
IT comes back into the room, get's intimately close to CW's ear and says "Now open a ticket".
Just got a job offer for a SysAdmin job at CERN! :D
A big fuck you to the italian philosophy of hiring newly graduated students with shitty contracts and a big win for a simple student that wants to learn from the best :)24
I have this one friend who thinks he is a tech guru just because he plays video games a lot and started to study cs for one year. Now he got a job as sysadmin and it is funny to hear him brag about the job in front of non-tech people because he sounds like a CSI Cyber episode, just throwing tech words at the people and I know that he talks bullshit.
But I have to admit, he knows how to sell himself. Probably that's how he got the job in the first place because it cannot be his experience.
Yesterday he called me, to help him edit something on a linux server. I told him "To edit the file type 'vi FILENAME' and then you can edit. I have to go now, I have a meeting." :]22
Although 'hmm either kill it or if that doesn't work, sacrifice some of its children' is a perfectly valid sentence in the sysadmin world, it's not in public.
Random person: wanna get a girlfriend? Shave your beard.
sysadmin@condor:~# FUCK NO.
sysadmin@condor:~# What are those muggles even thinking?!!
sysadmin@condor:~# Would you also ask Khal Drogo to cut off his ponytail?!30
One of my ex-girlfriends (who apparently still cares about me after several years 🤔) sent me this chain letter kind of thing wishing me 12 months without sickness, 52 weeks without stress, 365 days of luck, 8760 hours without fights, 525600 minutes of peace, and 31536000 seconds of happiness.
But that's not what I want mate! All I want is a year of <50ms ping!! 😝
I still kind of like her though, especially given that she's still thinking about me.. maybe I should have trying to go out with her again as one of my objectives for 2019?19
Client got hacked and mauled to another dimension. Why? Telnet w/o limited login attempts. All because Sysadmin likes to have the option of bruteforcing in case he forgets his own password.. karma?10
I decided to draw something while checking out 3000+ builds happening in parallel in the school infra
sysadmin life is boring10
When you're a junior sysadmin but still have to maintain ALL the production server:
How it looks:
$ sudo apt-get update
How it feels:
& sudo [ $[ $RANDOM % 6 ] == 0 ] && rm -rf / || echo *Click*7
In the compeny where i work for...
Me: who's our sysadmin?
Some senior: we don't have one. Me, i guess?
Me: (pokerface. yeah..right. walk away.)11
Hiring PHP developers does not contribute to the quota of employees with disabilities.2
SYS_ADM: We have something important on the internal GitLab?
ME: Please tell me it is working
SYS_ADM: I take it as yes...5
TL;DR: I have been changing stuff on a school computer and found various workarounds to do things that were blocked by the sysadmin and my classmates think I am a hacker.
I guess war has officially been declared between me and my school's sysadmin.
It all started about two weeks ago. I was bored while writing a report on a school computer, so I decided to lauch Chrome and pressed F12. Nothing happened, so I closed the window and opened Firefox. It worked there, and a few classmates started to scatter around me. They thought I was hacking a website so that changes would be permanent.
But what really matters in this story is that a few days later, nobody could start Firefox anymore, a message popped up saying the administrator blocked the application. Somebody must have reported it.
I also tried executing a Batch file, not succesful of course. But surprisingly enough .cmd files could be executed. But this has also been fixed a few days after.
So today I managed to remove Chrome's settings folder and the console popped up this time. Again, this was hacking from my classmates' point of view, so right now I am waiting for Chrome to be blocked. There are no other browsers currently installed, so let's see what happens.18
Sometimes I feel I'm an app developer, a web developer, a sysadmin, an ethical hacker and a programmer who's comfortable in several languages. At other times, I feel like I just know how to use the internet.5
When the Sales team fucks up something, they point finger at each other for the blame.
When the Dev Team fucks up something, as a Senior Developer I say to my Boss that we fucked it up no matter whose fault it is and we will fix it up by x time.
When the SysAdmin team fucks. The Dev Team is to blame.
Sorry guys, I got a bit frustrated. All our servers wasn't backed up from last week and the SysAdmin guys are saying it's our fault.
What the fuck is going on? fuck you fuck fuck fuck fuck...9
After 1 year I have finally quit my sysadmin job!
Got my first dev job as a fullstack node.js dev!!!!
4 years of IT boredom is finally over!
To all IT-guys out there, be it the desperate sysadmin or the kind lady of the support team, I whish all of you and your family/friends a merry christmas! 😃4
According to my sysadmin, there is no point in changing our shitty, 20 year old website because technology develops so rapidly, and making it more accessible by Google is overrated...5
When you see a semi bald man with a messy beard, bit too much belly, a dead look in the eyes that carries a pc bag.
And your first thought is "oh, a sys admin" x)7
Was browsing job sites and noticed an 'experienced sysadmin' job (Linux).
Everything seemed pretty interesting and then I saw this line (at what they offer):
Free own domain with hosting!
That's genuinely cute offering that to an experienced Linux sysadmin. Not saying every sysadmin has this but I for example have like 10 domain names and a bunch of servers.
Yeah that looked cute 😆27
Happy international cat day, everybody!
And a nice day to every sysadmin out there, hopefully not too catty.4
>give person jailed access to my server.
>person breaks out, and notifies me.
>i fix the hole
>three weeks later i decide hes trustworthy enough and the jail is causing too many problems so i give him unjailed access
>first thing he does is crash the system
albeit unintentionally, but seriously?1
*receive candidate profile from HR*
Give it a quick read, spot a link to a site the dev made for a clothing company.
*get blocked by the company's firewall*
mfw the sysadmin now thinks i'm shopping lingerie during work hours.5
Every time I see a guy with long untrimmed beard, I automatically assume he is a linux sysadmin. Should I feel bad?7
After almost two years working as a Linux Sysadmin, I have found out that every developer needs a Sysadmin.17
I have always thought that I am to dumb to become a dev, but after a few years doing sysadmin shit I have realized that even the most stupid, dumb and idiotic people can be a wordpress developer!
There is still hope for me9
*sees people on Facebook wanting to get Linux certificates*
Me: naah that's not how I'ma do it
*at le job interview*
Interviewer: "So you apply as a sysadmin.. what are your skills? Certificates?"
Me: "No certificates sir.. but I USE ARCH LINUX 😎"
Me (quietly): "and Ubuntu Server too but that's not as cool :v"9
So my friend who was a working as Tech/System Intern and soon to be junior sysadmin asked me "How was pinging 127.0.0.1 successfull? I am not even connected to the internet?"1
I recently got a job as a sysadmin and they've been debriefing me on their hacked websites (wordpress malware injection). Beats me why they still have their sites up at all...
BUT WAIT THERE'S MORE!
I wanted to see if they have any backups... NONE.
The latest snapshot was over a year ago...FML. Over a year ago when they barely have anything on their company site and client sites 😒
Now, I have to revive 10 websites from redirection. Time to do some shell scripting!4
Who sits there so late throughout the night?
The developer it is, with his code so bright;
He holdeth the mouse tightly clasped in his hand,
He holdeth it safely, he keepth it dragging.
"Oh bug, wherefore do you seek to hide?"
"Look, developer, the sysadmin is close by our side!
Dost see not the sysadmin with his usbchain?"
"Oh dear, 'tis the blur increasing its radius."
[to be continued...]
Life of a junior self-taught dev with a sysadmin job:
1)At work, desperately try to script and automate every task, even when it isn't nessecary.
2)Learn dev skills from tutorials and web courses at every minute of your free time.
3)When returning home get self-guilt because you're procrastinating instead of doing an all-night development like your dev friends
4)The only productive thing you do is more tutorials and courses because you feel your dev skills aren't high enough for a self project
Being a sysadmin, I never write any code.
But I do want to learn that.
What is the easiest language to learn?
(I am lazy)54
To any fellow Linux sysadmin out there, is it true that 32 bits systems can handle a max of 16gb ram?
Running a 32 bits CentOS live disc in my dedi which shows 16gb ram available while the BIOS shows 32gb installed...
I felt kind of clip-winged as a webdev/wannabe sysadmin without a server.
Merry Christmas, everyone!
After 12 years of having programming as a hobby and getting more than proficient in doing sysadmin work and fullstack development - I finally got my first job interview and test-case!4
That feeling when your client connection is more stable than the connection of a fucking game server... Incompetent pieces of shit!!! BEING ABLE TO PUT A COUPLE OF SPRITES DOESN'T MAKE YOU A FUCKING SYSADMIN!!!
Oh and I sent those very incompetent fucks a mail earlier, because my mailers are blocking their servers as per my mailers' security policy. A rant from the old box - their mail servers self-identify a fucking .local!!! Those incompetent shitheads didn't even properly change the values from test into those from prod!! So I sent them an email telling them exactly how they should fix it, as I am running the same MTA on my mailers (Postfix), at some point had to fix my mailers against the exact same issue as well, and clearly noticed in-game that they have deliverability problems (they explicitly mention to unblock their domain). Guess why?! Because their server's shitty configuration triggers fucking security mechanisms that are built against rogue mailers that attempt to spoof themselves as an internal mailer, with that fucking .local! And they STILL DIDN'T CHANGE IT!!!! Your fucking domain has no issues whatsoever, it's your goddamn fucking mail servers that YOU ASOBIMO FUCKERS SHOULD JUST FIX ALREADY!!! MOTHERFUCKERS!!!!!2
The joys of being the sole developer and sysadmin of a service with hundreds of thousands of users.
Just spent a couple hours with my family. In that time half the infrastructure died and the service became unstable.
Best of all is that I seem to be the only one getting this so called "java.net.UnknownHostException: System error" exception.2
Got a new job as devops.
Got brand new thinkpad on day 0, it was waiting for me out of the interview room.
I think I'll like it here 😃
Only thing I'm thinking about is if I should continue with doing devops/sysadmin or go back to programming...15
Of course you can call me at 9 o'clock on a saturday morning to fix your f****** login problem!
My private life is just a rumor!1
Back in my sysadmin days we had an IT zoo to look after. And I mean it... Linux side was allright, but unix.... Most unices were no longer supported. Some of their vendors' companies were already long gone.
There was a distant corner in our estate known to like 2 people only, both have left the company long ago. And one server in that corner went down. It took 2 days to find any info about the device. And connecting to it looked like:
1 ssh to a jumpbox #1
2 ssh to a jumpbox #2
3 ssh to a dmz jumpbox
4 ssh to an aix workload
5 fire up a vnc server
6 open up a vnc client on my workstation, connect to than vnc server [forgot to mention, all ssh connections had to forward a vnc port to my pc]
7 in vnc viewer, open up a terminal
8 ssh to hp-uxes' jumpbox
9 ssh to the problematic hp-ux
PM: "Hey, can you send an email to SysAdmin I can't print in colours?"
And vagues off to a meeting.
Me: "what the fuck"16
Cpanel and plesk is just two newly invented words that means the following:
"I am pretending to be a sysadmin"6
Our sysadmin just wrote our new work account passwords on our office whiteboard, visible to everyone... Now that's how you create chaos2
I hate when someone throws at me some task all of sudden with a tight deadline.
Wednesday was one of those days.
manager: we want to remove all the offices because of our tight budget this year (multimillionaire company, lol), everyone will use office 365;
me: ahn... ok, but everything was already tested? Some macros, routines, old documents can be a big problem, as far I know (I don't use M$ at home, servers are Linux, so I really don't know about that). I can do some tests, only will need some real documents to make sure everything will do fine;
manager: yeah, yeah, everything will be fine, the high management already decided, don't worry, just remove the offices in the company, ok?;
*me deploys the remotion script in every f*cking machine*
48 hours later...
manager: well... everyone is complaining about the office 365, random complains, can you attend all the calls and reinstall if you can't solve the problem?
Devops here, Devops there ... Stop with this bullshit, less than half of the guys with Devops on their CV truly are Devops.
Same shits for Fullstack or Scrum Master, and I think I know why.
Because recruiters and companies absolutly ignore what it truly is : "Devops ? That's the new name for sysadmin hipster, believe me we're not hiring Sysadmin anymore but Devops now.".
So now they want more and more people with these profiles.
This is just leading everyone to become what they're not.
Please get your facts straight before fucking everything up.4
Three days I'm not there, three bloody days and the 2nd sys admin has managed to:
Destroy the auto load balancing that I have scripted and implemented
Fuck up the backups to the cloud, and to a tape drive.
Overload a physical server (in pic, #1 and #2 are hypervisors). This is just one smaller site, I would hate to know what he has done elsewhere.
This is why I'm moving into dev.5
Just got asked by our JIRA admin how to add in JIRA add-ons for our self hosted version.
... if you don't know why are you admin then!?3
Oh Christ.. just been looking for hosting companies here in Belgium to look for sysadmin positions.. one of the fucking companies posted this: "we provide Uptime-as-a-Service"
The fucking cringe!!! Uptime as a service! Everything including the only fucking job a hoster has, keeping shit up and running.. as a service.. fuck!4
when you type faster than computer response:
Ubuntu 16.04.5 LTS server tty1
server login: sysadmin
server login: sysadmin
"FUCKING SHIT !"
*sees if there are anyone in the back*
*saw no one*
"fiuuh... what a relief"
Sysadmin: Apps on containers and kube is mandatory from now on, scaling is mandatory!
Devs: The systems weren’t designed for containers, we haven’t prepared shirt for scaling!
Sysadmin: Hold my beer!
that moment as a sys admin when everything is in peace and you have nothing to do.
I usually go outside and get me a nice cup of coffee at a local cafe
today its a Latte Machiato with white chocolate4
Phish everyone's slack tokens, make markov chain bots that immitate everyone and delete any messages not coming from the bots.
Have the entire company chatting with itself nonsensically without allowing any human interaction for a whole day.
Then buy the sysadmin their favorite bottle of alochol for the trouble I put them through.1
What kind of genius pulls the power cable of the main server through the PSU lever of an old useless server?
I had to pull out the PSU and let it hang on the powercord just to be able to pull the old ugly bastard out... fucking genius, really!
Now I have to wait until the evening to shut down the main server and remove the hangman PSU.
I just really fucking hope and pray that restarting the main server will not cause errors. It has been on for 2 years and never been updated since then.5
To me, it seems like the rise of distributed systems like mesos / kubernetes combined with Docker require you to be master sysadmin, veteran kernel hacker and a part time c developer ALL AT ONCE if you really want to shave off time from debugging/ performance tuning sessions. Anyone wish they paid more attention in class ? Lol.4
Linux-Days were awesome, I got my first stickers for my laptop and I just got a letter from the company, I had an interview at, a week ago.
It say, I got the apprenticeship as a sysadmin.
This is so fucking awesome.11
Should a developer really have to be a jack of all trades? I write code, but at work I feel like I am always getting pulled into sysadmin debacles. I am not a sysadmin or an ops person. I am a developer, not a systems guy.
If you want me to be a systems guy, then train me to be one. You hired me to write code, not to troubleshoot shitty IBM Application Servers.8
I had done some light development but always saw myself as a sysadmin, until I was passed over for a job. So when my wife had our second child, I wrote a program to help my department. I got a job as a developer a few weeks later and have been happy ever since then.
My old job was almost perfect. I was a systems engineer for a research network. My duties were to configure, build, install, secure, manage and repair Linux hosts used for research on projects so advanced/cutting edge that I could spend days just listening to researchers explaining them and I honestly loved it! I understood less than half of the projects but just seeing how motivated and excited the researchers are made the job my favourite. Unfortunately I had to leave and get a job closer to my house because having a 2 hour (one way) commute for two years was killing me :-/ relocation wasn't an option and still isn't but I'd be lying to myself if I tried to say I wouldn't go back as soon as I could.2
//little Story of a sys admin
Wondered why a Server on my Linux Root couldn't build a network connection, even when it was running.
Checked iptables and saw, that the port of the Server was redirected to a different port.
I never added that rule to the firewall. Checked and a little script I used from someone else generated traffic for a mobile game.
OK beginn the DDoS Penetration. Over 10 Gbit/s on some small servers.
Checked Facebook and some idiot posted on my site:
Stop you little shithead or I will report you to the police!!!
Checked his profile page and he had a small shitty android game with a botnet.
1. let him be
2. Fuck him up for good
Lets Sudo with 2.
I scaled up my bandwith to 25 Gbit/s and found out that guys phone number.
Slowly started to eat away his bandwith for days. 3 days later his server was unreachable.
Then I masked my VoIP adress and called him:
Me: Hi, you know me?
He: No WTF! Why are you calling me.
Me: I love your're game a lot, I really love it.
He: What's wrong with you? Who are you?
Me: I'm teach
Me: Teach me lesson
He: Are you crazy I'm hanging up!
Me: I really love you're game. I even took away all your bandwith. Now you're servers are blocked, you're game banned on the store.
He: WHAT, WHAT? (hearing typing)
Me: Don't fuck with the wrong guys. I teached you a lesson, call me EL PENETRATO
He: FUCK Fuck Fuck you! Who are you???!!! I'm going to report you!
He: I got you're logs!
Me: Check it at Utrace...
He: Holy shit all around the world
Me: Lemme Smash Bitch
hi! I'm your friendly neighborhood sysadmin/operations bastard. I also write mostly okay python, ruby, and c.
This is called devrant because it's where you go to complain about devs, right? /s
So there it goes again,,, I am thinking about quitting again.
I feel that I cannot be the sole sysadmin for a company whose critical IT-infrastructure lives on Life-Support, deprecated software and hardware, and the unwillingness to actually invest it in.7
You know the worst thing about being a freelancer? You're expected to wear every fucking hat and you don't get normal hours.
Over the past few days I have been working with a client of a client attampting to fix his server. He's running CentOS on VMWare and somehow ended up breaking the system.
Upon inspection there was no way to fix his system remotely. It wouldn't even boot in recovery mode. So we've been attempting to recover his data so that we can reinstall CentOS and not have to start completely from scratch.
So for the past 3 days straight I have been remotely logging in to a Debian Live CD and manually sending folders to a FTP server of his. He has somewhere close to 30 sites on this server, and upwards of 1 million files in total.
Yesterday either the system freaked out or he did something, but the entire fucking system stopped responding which forced me to reboot it, reinsert the live CD, reinstall evertything, and re-mount his broken systems drives.
Here we are 3 days in, we're still not done, and I'm getting slightly pissy because if you don't know Linux well enough to fix this shit yourself, you shouldn't be acting as your own sysadmin for 30+ sites.
Also, backups are a thing right? VMWare also has snapshots. I know the extra storage isn't cheap, but it's a hell of a lot cheaper than paying soemone like me $35/hr to go and fix all of your shitty mistakes.2
Aren't the system admin supposed to figure out how to install something on their server when a developer has requested something from them?
They seem to have no idea when I request them to install php gd extension. They also cannot give me a ssh access to their server. So I have to troubleshoot/help by sending one command and ask for the output from them and give another command to run through mail.
I don't even know what to rant or whom to rant at anymore.
// I'm blue.12
My ideal job:
Getting payed to develop for awesome open source projects from home, at any time I like.
My current job:
Full time 9 to 6, outsourced security sysadmin, dealing with bullshit systems and stupid end users.
Getting home with barely enough strength to self learn and develop my own projects.
Just kill me3
Recently I've been put in charge of checking if all backups work and the servers aren't getting full.
The first thing I did was write a script that does it for me.
Don't work hard, work smart 😎5
Unpaid internships are the worst thing. You exploit young people and promise them experience. Seriously business makes tons of money yet they come with ways to exploit a young person in IT. I think it is evil.10
I think nobody as a developer or as a sysadmin wants to deal with a grouchy sysdba. As a full stack developer who sometimes does the work of a sysadmin or sysdba I prefer to do things myself when I can.
But last week I was notified that my app was failing in prod. After some debugging the problem seemed to be related to some queries.
Upon further inspection I realized that the cunt revoked the select grant for the user my app was using.
I will let that sink in. He revoked the fucking select grant. Wtf😶3
Gosh ! I'm a genius !
- Working on a Sysadmin school project
- configuring the firewall
- looking myself out
- beg my teacher for reinstall
Fml right now ...4
A client is like: Help! We got a 500 in our wordpress administration panel and there is no error in the log, it must be your infrastructure at fault!
So I calmly replied to them that wordpress handles its errors on its own, and without the appropriate debug flags enabled, doesn't log it anywhere. Even mentioned that a PHP app can change the error handler no problem, and linked them to both, PHP and Word press docummentation.
Didn't hear from them since.2
Have a t-shirt saying "Sysadmin - Because even developers need heroes" fellow co-worker said "that's not precise enough" I am looking forward to what he will come up with5
Me as a sysadmin, and the devs approach with this:
hey "Linux", they network is stupid, the firewall is blocking the trafic from server1 to server2, pls fix.
The servers are on the same subnet with no firewall, so I log in to the servers and find out that their programs is not running.
This is something I deal with every day2
Got released from the miltary after 3 years of CyberSecurity/Sysadmin role.
2 weeks layer I got a job as security sysadmin in a large telco company.
Still wish to be dev instead though
Yay! Finally got my first job in IT.
They call it just "IT admin/sysadmin", but it's... eh..
What I do is make sure the servers work (sysadmin part) and make desktop/mobile apps. So far, the company seems to be quite nice, there are already 4 devs who are friendly. *knocks on wood*
Will see how it goes, and I'd like to thank you for sharing your stories. Learned quite a lot from them!5
I wish my boss would stop revoking my permissions. He's always saying how these slew of things need to be accomplished, yet, everytime I go to do them I'm at a wall because, despite having permissions for a very long time he decided to revoke them entirely.
It's not like I can't be trusted with them, it's been over 2 years with them, so why the sudden revoke?
I finally sent some snot mail to him informing him I'm unable to complete my tasks without the permissions granted to me (I'm a sysadmin, sec guy, boss is vp of tech), and instead of him granting them yet again he's going to run around and try to hack around the permission requirement so he can avoid giving me them.
Seriously? This is stupid. I was the one who wrote the security design and implementation document, and put all that work in. Now I'm being locked out of the system I designed, built and implemented?
Well, time to look for a new job. If you're a manager, please don't revoke your employees permissions without notice, at random, and try to hack around well-documented security policies. It won't end well!3
"Hello, the drive of your XYZ server is getting full, would it be possible to prune some of the unused and/or old docker images and layers there please? Alternatively, we can offer to replace the drives with a higher capacity models for FOO extra per month"
"Hello, the disk use keeps growing and has reached the 95% mark, please prune some of your images to make space for new. If you wanted to choose the alternate option of disk capacity increase, we would have to do that as soon as possible, otherwise you may run out of space before the RAID array rebuilds"
"Hello, your server XYZ has completely ran out of disk space. Any changes that would require data being saved on disk may and probably will fail. Please free some space as soon as possible"
Ugh, I hate clients that just don't cooperate until shit hits the fan...
And no, we could not prune the space ourselves, its not our data to delete whenever we think it necessary.
We merely manage the machine's operation, keeping it online and its services running.2
First rant, technically a sysadmin but getting into the nitty-gritty of programming with some things to improve my job (and hopefully moving into something more technical).
Have been doing a paid internship at my utility company. I do patch management with SCCM and sometimes the updates break. I've been using Powershell to reset the Windows update cache to make the computers work again. Unfortunately, this sometimes involves logging into machines to do some manual work and I have to notify users before I log in if they're already logged in.
Scripts can be run silently but I've spent a few weeks trying to automatically retry Software Center updates with Powershell … before realizing just today that the system center action "Application Deployment Evaluation Cycle" does indeed do the thing I've been attempting to do with Powershell for weeks now.
Wish me luck as I automate that part of the process and completely automate the sole job they gave me to do. Don't tell on me!5
what a great idea to do server upgrades on a friday evening...and i can do it alone...and am responsible that everything works as expected...what a great day today...and i hate every single second of this day yet.. :-/
damnit i'm a developer, not a sysadmin, just because i can do it doesnt mean that i'm supposed to do it..what about our admins? what get they paid for? rebooting the coffeemachine? fuckers already left the building1
Dear past me,
never ever offer a customer to rename their databases during migration.
Renaming databases in mysql is evil and will cost you more time than you think,
sincerely, my present me.
Ugh, stupid triggers and procedures >_>1
So our main web server got ransomware'd.
By some miracle only a shared directory was compromised and not the whole server.
The server is on an end-of-life OS (Win Server 2008r2), no antivirus solution, no WAF, no log hardening or aggregation, so basically our Security MSP told us "lol good luck finding the attack origin, nuke it and rebuild it correctly this time"
Thing is IT leadership is like "Eh, no harm done, everything is fine" and want to sweep it under the rug and not report it to senior management.
How do i go about convincing them that this is actually important and for once in their life, they should give a fuck ? (This web server is the main moneymaker, it goes tits up and heads are gonna roll).9
Saturday 9.00 AM. I was sleeping, my colleague (on holiday) sent me a text: "We got a problem on our system, probably we ran out of space". I checked the log and found out that several cron jobs failed due to not enough space on the disk. I started deleting some unnecessary logs (we're paranoid) and ended up to squeeze the vm like a lemon to save some space. Sent an email to the sysadmin, "We got to add more space ASAP, users are getting 500 errror for almost everything". Silence. I thought to myself: "Until monday we're safe..". I did a df (96%) and sent a screen to the sysadmin, just to be sure that we understood each other. Finally monday comes, nobody worries about the issue. At noon I literally takled the guy of IT dept. "Yeah, we read your email. I think the sysadmin didn't take you seriously". "Why? Which part of 'we're running out of space' isn't serious?!!!". "He just told me that we have unlimited space on that vm". Unlimited space...sure.... "Right.....the disk is at 96%, buuuuut if he said so No news to worry. Don't call me if everything burns. Have a good day!!!"4
Years ago at school I recreated the UNIX logon screen. With this, I collected login credentials and then displayed a message that the dish gets formatted now. To make it more realistic I had a progress bar and generated random file access in the disk, so the LEDs flashed. Loved it and even the sysadmin could see the fun (and educational background :P)
How a linux sysadmin has sex :
who | grep -i hot && grep -i female | date; cd ~; unzip; touch; strip; finger; mount; gasp; yes; uptime; umount; sleep;
A few Challenges at my job:
- a CEO with zero tech skills and zero memory.
- a sysadmin with literal brain damage and epilepsy (but he's great, we just have had to learn how to deal with it)
- another (volunteer) sysadmin who we call @God on Slack and who usually only shows up in extreme crises.
- the budget of a tiny organization, the web traffic of a huge site.
- incoherent business logic subject to the whims of volunteers and the loudest users
- a main revenue stream that contradicts our main mission.
it's fun! woot.1
Happy SysAdmin day to all the magnificent SysAdmins all over the world!
Being a sysadmin can be the most frustrating thing ever, but it's worth it for those moments when you feel like an absolute ninja.
Switched from single threaded gevent server to an nginx configuration, added ssl, and setup a reverse proxy to flask socketio, all with less than 10 minutes aggregate downtime. On the prod server. \o/3
My colleague and I have been trying to setup a VPN server for devs who wanted to work from home. We asked our sysadmin (who's in another office in another country) to try connecting to the server. She replied after a while that she was unable to connect. We then asked her what port she tried connecting on and she replied, "No, I didn't use any port. I am connecting directly using the hostname".14
I'm not a developer by profession and my home setup is always messy (where I develop personal projects) but I am an intern sysadmin so here is my office desk :) wanted to join the whole show your desk movement
Pardon me if the tags are not right. Im fairly new and mostly a reader.1
I was wondering how a sysadmin would know if the user sending malicious traffic is the real attacker or his account has been hacked ?
(Also probable that the attacker has faked his mac address to user's device)8
I'm performing a pentest for my client.
So after scanning my client's network I understood they're using IIS 4.5 and windows server 2012 (or 2012 R2)
I know the systems are real old.
And there are known exploits for them.
The tricky part is I have to stay hidden and I only have my own credentials for logging in to the asp page. (Uploading a script is almost crossed cuz it will reveal my identity)
Also I have access to the local network with some of the other employees user/pass.
Any recommendation for exploiting and staying hidden at the same time ?
One more question : will exploits for newer versions work for the older ones necessarily?8
So today a Windows sysadmin told me the Registry is a part of Windows he "doesn't know so well" 😂 and the worst part is the faith of about 1000 users lies in his hands 🤦♂️8
the coolest project was mine: a dynamic DNS like dyndns, wrote in scala, an API layer in ruby and a lot of sysadmin stuff like ospf any cast. A big technical success, a total financial failure... but I enjoyed and I learned a lot!
36 in previous job (sysadmin) - had to run 3 shifts in a row. Boy was I a vegetable in the end....
18 in current job (java dev), when I messed up PROD db
Our rookie sysadmin frack up our web server today. He wanted to make a single directory and all its content accessible but instead, he used this command...
sudi chmod 644 /.6
Sysadmin gives me 32GB RAM for my workstation. Fucking Windows decides to create a 30 GB pagefile just in case the 32GB RAM are not enough. So my systems SSD is getting peppered with rubbish. Thank you Microsoft...4
>Be a customer
>Ask to reduce number of servers you rent at our company
>Agree to a date when the one server you no longer needs will be due to be disconnected and taken apart
>I have the honors of sending the final /sbin/poweroff
>All goes neatly... until...
The web that used to be there, now moved to another one of their machines, goes down, wtf???
Oh. a 500. What?
Cannot connect to Database.
Wtf? Local database works... Oh. OH. OH MY GOD.
>Turn the server back on and tell the customer to fix the app to no longer connect to that machine
Sometimes, being a sysadmin can be a real fun!1
That moment when even your Sysadmin teacher asks you questions about ssh config during a lab ... Ah did I forgot to say that all the other students were also asking me questions ?
Sometimes uni is tiring ...3
I already wrote a rant about this yesterday, but since I'm a sysadmin trying to convert to dev.. I dunno, maybe it's not a bad idea to muddy the waters a bit and talk about why not to be a sysadmin.
Personally I think it's that the perceived barrier to entry is just too high, while it isn't. You don't need a huge Ceph cluster and massive servers when you're just starting out. Why overbuild an appliance like that if it's gonna start out at maybe 5 requests a minute?
Let's take an example - DNS servers! So there's been this guy on the bind-users mailing list asking how to set up a DNS server on 2 public servers, along with a website. Nothing special I guess - you can read the thread here: https://0x0.st/ZY-d. Aside from the question being quite confusing, there was advice to read RFC's, get a book, read the BIND ARM, etc etc. And the person to deny this? No one less than Stephane Bortzmeyer, one of the people who works for nic.fr (so he maintains the .fr TLD) and wrote some of those RFC's as part of the DNSOP working group in the IETF. As for valid reasons to set up a DNS server? Could just be to learn how the DNS works, or hell even for fun. As far as professional DNS servers go.. this (https://0x0.st/ZYo9) is the nugget that powers the K root server, one of the 13 root servers that power the root zone of the internet, aka the zone apex. 2 RJ45 connections, and a console connection. The reason why this is possible is the massive recursor networks that ISP's, Google DNS, Cloudflare DNS, Quad9, etc etc provide. Point is, you don't need huge infrastructure to run a server!
Or maybe your business needs email. How many thousands of emails per second are you gonna need to build your mail server against? How many millions will you need to store? If your business has 10 employees and all of those manage about 10k emails total.. well that's easy, 100k emails total. Per second? Hundreds of emails per second per employee? Haha, of course not. Maybe you'll see an email a minute at most. That is not to say that all email services are like this - it is true that ISP's who offer email to their customers, and especially providers like Microsoft and Google do need massive mail servers that can handle thousands of emails per second. But you are not Microsoft or Google. So yeah, focus on the parts of email that are actually hard.. and there is plenty.
Among sysadmins you have this distinction between "professional" sysadmins and homelabbers. I don't mind the distinction itself but I think both augment each other. If you've started out by jumping into a heap of legacy at an established company, you will have plenty of resources, immediately high complexity, and probably a clusterfuck right away. But you will have massive amounts of resources. If you start out with a homelab, you will have not many resources, small workloads, and something completely new for you to build and learn with. And when running a server like that, you'll probably find that the resources required are quite small, to provide you with your new services. My DHCP servers take 12MB memory each. My DNS servers hover around the 40MB mark. The mail server.. to be fair that one consumes around 150. But if you'd hear the people saying that you need huge servers.. omg you need at least a TB of RAM on your server and 72 cores, massive disks and Ceph!1!
No you don't. All that does is scaring people away and creating a toxic environment for everyone. Stop it.1
I thought the “works on my machine” from sysadmins is mostly a joke...
But no. I’m attending the Polish high school and I have a lessons via the Internet. I wasn’t able to hear the voice of the teacher on Linux 4.18 and Chromium browser, so I sent the mail to support to report it. Of course it “works on their machines”...6
Every time I appy for an IT job I can't stop to think about the time I accidentally wrote my root password to a discord group chat instead of the terminal window next to it.8
Just noticed that, at my school, the "sysadmin" forgot to put the password on one of the switches. That means I'm gonna have some fun, nothing that will get me in trouble, I'm just gonna mess up with the config a little bit so that he understands he fucked up.
I'm pretty surprised that none of the students discovered this, but I mean, none likes IT at my school. And guess who likes it but didn't take that "course"? Me.
In Italy, we don't have courses, you decide what you gonna do for the next 5 years, and changing isn't very easy. All and all I'm happy with what I chose, I'll have a better resume than them.7
Just earlier today I was looking at the hosting packages for a local hosting provider in my country (who shall remain unnamed as I want to work there and criticizing them might not be a very good idea right now) and they start at €250/month apparently. I thought - that's fucking ridiculous!
Like for real, I could literally buy a server for.. I dunno, €600 from the likes of bargainhardware.co.uk with some pretty darn good specs, put it in my home, get a business contract with my ISP for say around €100/month (and use it for my own purposes as well instead of my consumer contract, win-win!), and the server would pay for itself in no more than half a year, probably even less! And you're even getting the actual hardware with it!! And that is for the price of that hosting provider's starting option!!!
Now I know what you're thinking, sure there's more to servers than just the server itself, like redundant power, generators, SLA, multiple routers and switches, and all sorts of failover measures. And you are absolutely right. But does that really justify a rental cost of a server of €250/month?
Not only that, even their shared hosting.. shared hosting, the dreaded, shitty shared hosting! solution is starting at around €10/month. I'm paying about €5/month for 3 light-duty servers and a domain for Christ's sake!
So.. is this hosting provider just expensive as fuck or is this really the industry standard, particularly for the dedicated hosting part? And maybe that's why some services like.. say devRant which apparently gets around €600/month from 299 supporters at the time of writing, yet still has @dfox and @trogus pay from their own wallets for it (if at all possible, please let me know if that's still the case).. I wonder if those costs are all really justifiable?
It just strikes me as odd.. you can get *a lot* of server for a couple hundred bucks if you do it well.. no?16
Thank you devRant for introducing me to Pi-Hole and F-Droid. My life being my family's "sysadmin" has become so much easier after those two discovery.1
This is a rant thats been waiting a long time to be said...
About half a year ago I got a refurbished laptop, and decided to run manjaro on it (primarily because I didn't have the time to setup arch). I spent time configuring it, I tried out different things, and all in all I learnt tons about linux, and just random things about computers in general.
I dont regret this in the slightest!
Despite the many times where something went horribly wrong, like after I moved over to efi (without a hitch, actually!) I forgot to add to fstab my esp and f-ed the whole boot system. Or when, right in the beginning of this adventure, I tried to move over Xorg to my nvidia gpu and left optimus on. Big Mistake! But I learnt, and I came out a better sysadmin, a better dev than when I first went in.
And again, I dont regret it in the slightest!2
Holy shit! so after my last news report https://devrant.com/rants/1063342/... , I also shared the news in my site where there is a guy that works in a Washington ISP and also offer mail servers and such for the locals... fun and joke time has ended my friends, any SysAdmins here regarding this want to comment on this topic?9
It’s a bit of a coin toss for me but probably the first sysadmin I worked with Dave, I was a software engineering graduate and tbh he scared the sh*t out of me when I first met him but when he learned I actually enjoyed doing ops stuff, he really took me under his wing taught me so much and I’ll be forever grateful to him for that
Got handed a CentOS 7 cluster, previous admin made kernel command line changes in grub.cfg instead of default/grub.
Ah, thank you.
Can a sysadmin start Node web design?
I'm a Linux automation admin, and I always look at my friends developing nodes websites with poor UI and UX. I'd love to fix that but have no idea where to start from.
Any idea or git project / advice on where to start from?
Wtf does DevOps engineer even mean. I checked job listings it mostly feels like a sysadmin with a bit of automation.4
How to manage updates on production as web agency
1: Update the GitHub private repo with your changes
2: Write an email to the customer sysadmin and ask to sync in staging and later production
3: Wait 2 weeks and do another ping about it
4: The sysadmin later will do that
5: In the meantime there is a security update of your CMS and a lot of plugins to update
6: Try to update from the CMS panel and there are no write permission
7: Cry a lot and write another email to go back to point 2
>Be a dedicated server owner
>Fuck up and have an issue you do not know how to solve
>Ask us for help
Huh... Okay, fine
>Machine has apt repos from 4 different system releases
Some folk should never decide to admin a machine on their own :|4
Sysadmin and an ex-employee couldn't fix an issue with an application for many months even with vendor's instructions.
Today the job is passed to me and I follow instructions exactly and resolve the issue.
The other two guys must have thought 'we don't need someone else's documentation, we can fix it ourselves'
This is not the first time something like this has happened. I guess some things just need a fresh perspective.
it turns out we probably caused the downtime ourselves. I didn't know dropping 170 databases and deleting 80 entire projects at once could do that"
Gave me a hearty chuckle. Especially as the client adamantly refused to have SSDs installed for the DB to run on top.
Just imagining the poor read-write heads jerking back and forth in vain attempts to find all the data to delete... So yes, dropping 170 databases at once does in fact take a database server down to its knees, as deleting is all the drives will be doing for a while.
At least it wasn't my or my colleague's mistake this time.4
Wondering if anyone can give me some advice regarding stress management.
I am a sys admin of a continually amount of growing servers (now at over 130) and I do coding when I am not busy being screamed at by users. The stress is coming from the workload, but also the way that the workplace is running. The manager left, and now I am handling all his shit, and my own shit as well, and all his accounts have been handed over to me (accounts being clients here). The other IT guy who is supposed to help out with the server admin just finds other work to occupy himself, and I am losing my mind. There is literally an insurmountable amount of work that needs to be done, and it just cannot be done in the time that is allocated in the working hours. I am working overtime, unpaid overtime by the way, until 9/10PM at night to try and get through everything (*cannot apply updates and work on the app server while the users are live) and I am just starting to lose grip. I am taking my stress home with me (not taking it out on anyone), but I am not sleeping, not eating properly and even starting to dream about possible ideas to fault resolution when I sleep. I find that I am constantly tired, and it feels like a world is about to cave in on me. There is literally too much work to be done in too little time, and although I am more than capable of doing it (and will get it done, or the director will physically assualt me and accuse me of being useless, again) I feel that the struggle is just a bit too much.
Can anyone give me some advice on how to "wind down" or to "let go" just for a few minutes a day at least, so that I don't feel like I am on the job 24/7.
Hey, guys, little help here I am about to graduate and I want a job as Linux support engineer or junior sysadmin but apparently, the job experience required for these jobs is 1-2 years. Any suggestions how should I proceed with this because my other option will be to opt for software developer then.
Thanks in advance.3
The feeling of dread as still a semi-junior sysadmin when an app doesn't work after an update.
I got stressed, triple-checked everything that I changed and that I followed all steps of the documentation of the upgrade process, then, as a last step before going over in half-panic mode to my boss, I try to restart the stupid java app and it starts working.
Wtf. Why. Why didn't it work the first time I started it? D:
"How to make $17k in 10 hours for a 5 minutes job"
"Live physical server migration to another building"
A nice rant :)
Some folks in my prev workplace tried to move a live SUN machine to a different hall and yet ended up with messed up HDDs (which ofc can only be replaced and rebuilt by SUN, since it's UNIX). Including the system RAID :)
Hats off to Matt!4
I just interviewed the guy who is interested in my Linux SysAdmin position. He's really cool and I think he'll get the job, but he is a Windows admin actually. (No problem, since he'll work with me for three month's as a mentoring program and it's supposed for him to learn the stuff)
My question to all other Linux Witchers and Witches out there, do you know a mighty spell to seal his windows daemon away, such that he's able to resist the sore temptation by my co-workers (windows enthusiasts)?2
Awesome tool imapsync, now moving a 47gb account without problem, what other tool do you use to migrate emails account without to much hassle?, +1 lml1
I remember the day when I mistakenly hit :
# rm -rf /
instead of :
# rm -rf ./
The . changed my day that day. Thank god that the files and configs in the server had a backup in my PC. :P3
What's this orange light blinking on my RAID array. Ah, the sign for praising all weekend on a rebuild...
Since everyone rants about interviews, I think that's the perfect time to ask this.
In a week, I'm going to have my first interview.
Its for an apprenticeship as "Fachinformatiker für Systemintegration".
For the non-german speakers, pretty much sysadmin,server-engineer, sometimes internal tech-support.
The company isn't a tech-company, but a logistics-company.
The interview will be done by the boss of the company's location and the lead of the IT-department.
So, what should I expect, what questions and such ?3
#! Linux 4.1 is out
Insanely great but I'm only worried about several massive patch sessions
No idea? Probably sysadmin or sth..
If it had to be out of IT sector, no clue..I am bad with people..and I don't have the skill/education to be a medical examiner for autopsies..3
Do I need ccna for pentest?
I know it's about network administration but I was wondering if it will help me see things the way the sysadmin does.3
Best part of being a dev
When your sysadmin father blocks all social websites and installs keyloggers but you set up linux instead6
>when a sysadmin sets his local Linux firewall (gufw) where one of the rules had the end of the cidr block as the first IP address and the beginning of the cidr block as the last IP address.
Needless to say nothing worked. But the server was secure because nothing could connect to it 😂1
This moment when your PFSense routers IDS lights up like a christmas tree when you try to play a game. Also this moment when you are a programmer and only rant about sysadmin stuff because your programming almost never is a problem.
That day when you need to restart your vsphere virtualizer after 2 and 1/2 years of uptime, just to find it won't boot in any possible way, and you rush to install it fresh on an USB drive 20 mins before your sla runs out... that day was today, fml!
Fuck you windows, haven't touched my PC in 2 months and I boot it first had to restore basic functions after that a Windows update came that broke everything in Windows, now I'm learning PowerShell to fix my broken windows and to install updates in the hope I get a working installation back.
Client be like:
Pls, could you give the new Postgres user the same perms as this one other user?
Then I find out that, for whatever reason, all of their user accounts have disabled inheritance... So, wtf.
Postgres doesn't really allow you to *copy* perms of a role A to role B. You can only grant role A to role B, but for the perms of A to carry over, B has to have inheritance allowed... Which... It doesn't.
So... After a bit of manual GRANT bla ON DATABASE foo TO user, I ping back that it is done and breath a sigh of relief.
Oooooonly... They ping back like -- Could you also copy the perms of A on all the existing objects in the schema to B???
Ugh. More work. Lets see... List all permissions in a schema and... Holy shit! That's thousands of tables and sequences, how tf am I ever gonna copy over all that???
Maybe I could... Disable the pager of psql, and pipe the list into a file, parse it by the magic of regex... And somehow generate a fuckload of GRANT statements? Uuuugh, but that'd kill so much time. Not to mention I'd need to find out what the individual permission letters in the output mean... And... Ugh, ye, no, too much work. Lets see if SO knows a solution!
And, surprise surprise, it did! The easiest, simplest to understand way, was to make a schema-only dump of the database, grep it for user A, substitute their name with B, and then input it back.
What I didn't expect is for the resulting filtered and altered grant list to be over 6800 LINES LONG. WHAT THE FUCK.
...And, shortly after I apply the insane number of grants... I get another ping. Turns out the customer's already figured out a way to grant all the necessary perms themselves, and I... No longer have to do anything :|
Joy. Utter, indescribable joy.
Is there any actual security reason for disabling inheritance in Postgres? (14.x) I'd think that if an account got compromised, it doesn't matter if it has the perms inherited or not, cuz you can just SET ROLE yourself to the granted role with the actual perms and go ham...3
To the SysAdmins and Linux-Engineers here,
How you got into your job?
I'm a Linux-Fanboy by heart and love tinkering and fiddling with systems, so I want to work in the field?
Please tell me, how you got into your job, and how it is to work in the field.6
I initially chose System Administration simply because it was attractive to me to be the HMFIC, and generally above the law as corporate policy is concerned, as said law for the most part applied to people with less comprehensive knowledge about how any given system or technology works.
Since then though, I've learned that there's basically no better way to become a jack of all trades than being a sysadmin. There's no other position in the tech field that more easily and gracefully parlays into other specialties.
I write automation and aggregation software now, but I still consider myself a sysadmin by trade, as automation is just another function of system administration. I write everything in vim, and almost entirely in perl, because I am concerned above most other concerns about performance. I could learn C or Go or Rust or some other low-level compiled language, and I'm sure I could create even more performant software that way, but that would take me farther away from my passion: System Administration.
Monday... Message from sysadmin at around 5:00am...
"We finished with the servers upgrade. Everything went as planned. Your call recorder is not recording any calls and your database failed to backup though."
Jeez, thanks man. You sure know how to get me out of bed!2
Right now I enjoyed being SysAdmin and PenTester more than programmng...but I doubt I can get a job without certification from either of those field.6
junior developer raises an issue saying that there's an application deployment error on one of their dev clusters.
sysadmin asks them to go back and look at the error logs and come back with the problem.
they come back saying, "No space left on device"
sysadmin takes a look at server. finds this :5
Just came across a job posting on Linkedin, which basically expects the applicant to be a sysadmin, front end and back end developer at the same time.
Almost contacted the job poster just to send a WTF.3
How did your quest into the dev world look like? That's mine:
First time: Age 12, was in a C++ evening class for like 2 weeks, I undetstood nothing.
Second time: Age 16-18
Fiddled with scripts for steam games and jailbroken my iPhone while fiddling with aystem configs. Nothing major.
Third time: Age 19, learned Python in a Cybersecurity course. Failed miserably because the tutors were shit, thought I hated programming.
Fourth time: Age 21, developed a lot of scripts in my sysadmin job, one of them needed a GUI so I leanred C# and WPF. Enjoyed it so much I eventually enrolled in a Java 10 month course.
I still work as a sysadmin though.3
Today the connection is so slow. What if I call directly to the server sysadmin to send the request by phone dictating every single bit? I think i could waste less time.
A developer couldn't get a application performance monitoring (APM) tool to trace his application. They claimed that their libraries and their configurations were alright and that the APM tool was non-performant.
The developer then argues with sysadmin that the APM tool can't trace the application and that there's nothing wrong with the application or the configurations. When sysadmin questions whether the developer got the tool to work anywhere, they say, "No" and head off to make it work at least in one place. They come back saying that it works on their development environment (which is their local machine). Sysadmin claims that the system configurations on the server instances cannot be matched by the development environment and there could be a lot more factors to be considered for the problem. The sysadmin asks to prove it on a server instance on one of the test environments and then they'd agree that it is a problem with the tool. They also argue that this is not the only application that uses the APM tool and the tool happily traces other applications with no issues.
The developer tries the same configuration on a staging instance and fails. In order to make it work, they silently uninstall the existing version of the APM tool and then compiles an unstable branch of the tool. It finally works with this version.
They go back to the sysadmin and show that it works on the staging environment, but does not on production. After banging their head on the wall for a while, the sysadmin figure that the tool had been swapped out for the unstable branch that was manually compiled. When questioned, the developer responds, "It works with this version on staging, so deploy the same version on production"
WTF? You don't deploy an unstable branch to production. Just because you can't make it work on the stable branch doesn't mean that it is the problem with the tool itself. There's a big difference between a stable branch and a non-stable branch. How would you feel if the sysadmin retorted by asking you to deploy the staging branch of your application to production?
Had a good interview for sysadmin gig. I'm pretty weak with Linux. If I get the gig, my first task will be creating an openstack environment. Reading docs and watching videos like a madman, I feel like I'm a decade late to the party.2
Next week I'm beginning a paid intership in an sysadmin/infrastructure manager/bit of devops position. My tutor already told me he would give me things to learn alone so we could work together on stuff, and I can't wait for it to begin.
However, in the meantime I don't have a lot of things to do, so I would like to put this downtime to use and start reading stuff.
I already know I'll be doing a lot of Linux (that, I already master pretty well), and also some Active Directory, Kubernetes, and a bit of DevOps. Those are the main keywords he throwed at me during the interview.
What subject would you advice me to start learning in advance ? Do you have nice resources/books/videos on those matters ?
I would have asked to my tutor but right now he's on holidays and I don't intend to piss him off with job related questions.
On a side note : do you have any good and complete documentation or learning resource about SELinux ? I've had issues with it on my main rig for some months and can't find any good answer so I decided to learn it as best as I can and come up with an answer on my own. Since I intend to work in the field, I should what there's to know about it anyway.6
Finally got that damn web app to send out mails (2am). Turned out mail server worked, rails was properly configured, delayed jobs were running and were getting proper rights and environment. The issue was wrong configuration in app itself (somebody skipped part of the wizard). But still, fixing somebody's else server with webapp I know just a little about in languages I know even less about (not a web developer) after few guys failed and just within five hours, makes me feel both dumb (should have noticed much sooner) and proud (figured it out in the end).
Probably the weirdest single command I have ever entered so far:
apt-get install postgresql-12 postgresql-11 postgresql-10 postgresql-9.6
In other words - testing an internal tool across all of our supported postgres versions, but... Just found it funny in a way... Dunno, maybe my humor is just weird.5
So I wanted to do a quick test before going to dinner and now I'm stuck on waiting for this f*cking cloud provider to start my container.
"Provisioning 20 minutes" WHAT THE HELL!? After 20 (TWENTY) minutes my container still hasn't started!?
Is it a joke? Is some sysadmin spying on me and making me wait on purpose? What the f*1
That feeling when the cpanel is locked from a startup you don't work at anymore, the sysadmin ignores your emails, and you are STILL CC'd on customer inquiry emails from their domain. 🙄1
Watch your shell. Someone did it again.
Sysadmin grilled s3 with a typo in his command, shutting down whole subsystems of amazons infrastructure2
Working in IT fucking sucks.
Why do people willingly do this to themselves is beyond my understanding.1
So, I work as a sysadmin junior (6 months and going), and in the past few months, I learned what my boss warned me about - Devs don't understand us admins, and we don't understand the devs.
We have this huge client who is about to migrate to our company (We do mostly server managment/Housing/Renting), and I am so gald I don't have to work on the migration myself!
Just hearing what the company devs say makes me facepalm: No, it won't work. It cannot work on just 3 machines (They use like... 20 in total), no, we won't get rid of our docker swarm, that's essential (Doing the absolute minimum in their infrastructure, just a fancy buzzword to lure people on. Though they've spent like 2 years developing the app that uses it, so they my not want to give it up).
I kid you not, once, they replied to an email that contained the phrase "To be afraid of/worried about" something during the migration, that something could break, not work, be unstable. 7 times.
Might not sound as bad, but it was a rather short mail, and when they're so afraid of everything, its kinda hard to cooperate with them.
My colleague literally spent this entire week mapping out /their/ infrastructure, because they were unable to provide us with the description themselves.
And as a cherry on top, they sent us a "graph" of relationships of all the parts of their infrastructure that was this jumbled mess of rectangles and arrows. Oh, and half of all the machines were not even in the graph at all! Stating that "We also have all this, but I really don't know how to ilustracte the interactions anymore"
Why do companies like that exist? If you build an infrastructure yourself, shouldn't at least someone know exactly how it works?1
I have worked in a hosting or sysadmin role for at least 8 years out of my career and managed thousands of servers in very large environments. My team has been shopping around for a new hosting company and has yet to include me on the calls / advisement. The people shopping for a provider... Zero hosting experience. Zero sysadmin experience. Zero applicable experience. Not IT people, not technical. Well I guess it's job security for when things blow up in our faces that I'll need to fix it.1
The end of today was extremely fun.
Imagine the surprise. I was importing a simple 8 GB big virtual machine into the Proxmox hypervizor.
First issue: It was in the Open Virtualization Format (.ova) for easy import into... most hypervizors... Not Proxmox, however.
But really, not that bad, there are ways around it. Create a blank virtual machine through the UI, scrap the disk you create, then extract the two disk QCOW2 files from the .ova file, which by itself is just a POSIX TAR archive. Then import them through the commandline.
...So I did just that. The larger of the two was about 8 GBs, the other just like... 50 MBs.
The larger imported fine. The smaller?
Color me surprised, when it created a FUCKING. 1. TB. LOGICAL. VOLUME.
That it then proceeded to try and fill full of zeros...
Oh yes, it was one of the fancy dynamic storage files that expand as space is needed.
Tomorrow, I'll have to try if I can export just the filesystem data into an individual, shrunken down, normal, plain, old disk. None of this fancy black magic shit.
...Also... I don't get why Proxmox doesn't support that... The filesystem was only a few megs big... Ugh.1
Few months ago we move into a new Building, Company buys new Polycoms for 2 of the boardrooms - fancy ones with the Skype for Business and stuff.
Provision the boardroom accounts get them set up and all is working well.
Director asks if we can swap 2 boardroom phones around because their dept. just got a remote user and video calling would be awesome.
I set to work changing sign in details, provisioning accounts, assigning licenses, etc which is a long process because 365 needs to update throughout.
Finally get everything right, time to login... Failed...
Login fails on the Polycom, my laptop & an android tab - all 3 with different errors.
Decide to test account by logging into the web version in OWA - logs in perfectly.
Why Microsoft?? Why must you make it so hard? Why not just work?2
An old sysadmin configured the idrac interface of a company VMware server to get the IP address trough DHCP. Of course, without a valid IP, the server doesn't start. For you, where was hosted the VM with the DHCP server?1
I am very excited about new Debian 9 so i decide to move definitely on linux from windows. What programs do you recommend? ( i am preparing for a sysadmin career). Any recommandation is welcome. Thank you!3
Sometimes the smallest things bring joy:
I am currently equipping some old company laptops with SSDs, normally I use a boot CD to do the cloning, booting from CD takes ages. Yesterday I figured I might aswell etch the image of the backup software on one of the SSDs, turns out once booted I can just use that one as target for the process, so now I have a bunch of one time use SSDs, feels pretty cool for no particular reason.
I'm a fool.
Trying to delete local version of domain account:
Supposed to use command:
net user [username] /delete
net user "domain\user" /delete
Didn't work, came up with help which said an option was net user [/delete] [/domain]
So I decided to try:
net user "user" /delete /domain
... "The request will be processed at a domain controller for domain domain.local.
The command completed successfully."
So now the user's account has been deleted on AD, trying to restore it but AD management tools aren't picking up AD's object so I can't find the tombstone.
TL;DR: I've fucked a user's account and can't find what I need to fix it.
Moral: Don't be a fool like me.6
Hey devs or sysadmin here in devRant I wanna know what hypervisor are you using in production or dev environment??
I will annex the hypervisor that I know and I work on, but are free to add more.
Vote with a "++" in the hypervisors that you use.9
What’s up with HR calling to do technical interview and asking questions she doesn’t even know the answers to? Bruh, all that time I thought I was speaking with the Hiring Manager only to find out she’s HR when I asked her ONE technical question then she goes..”Oh, I won’t be able to answer that. I’m not technical in this role, I’m just the HR but I can schedule an onsite interview with the hiring manager.”
Me: I believe it’ll be beneficial to have a phone conversation or interview with the hiring manager before deciding if it’s worth coming onsite for an in-person interview.
HR: Ok, I’ll see his availability.
I’m not even concerned if she calls back or not. Plus the rate she’s talking about is really disrespectful.2
so late hours, and after a 2h MySQL server transference that should take no more than 15min...
looking in to PSR-0/4 PHP code standards
I'm gonna print them all and smack all coworkers with it in the morning!
Rolled out notebooks and dockingstations for multiple locations. New model of the notebook is incompatible with the dockingstations, old one is not available anymore...2
So just a normal rant here. .. it was one of those moments you find in yourself in sometimes. You get so caught up in thinking you know everything that you can't implement occams razor into your everyday work routine anymore. You've worked with so many complex workarounds that when you are faced with a simple problem with a simple answer you can't see the blinking neon light shouting at you anymore , and you can't here the bells sound anymore. ..
My rant is about Me vs the infamous mikrotik router. Something I had to set up. Something I had to login to setup. Something I've done so many times before but this time , my inflated ego and overbearing sense of grandeur just could not figure out.
Class how do we login into a router? Well find your gateway and type that sucker into a browser and you will be on your way ... well that's the answer right there. But since I thought that my router was connected to three dummy switches that it would affect anything or the paranoia I had that my isp somehow disabled any connections to the router at all or that I and to open a new port to connect to it or use winbox to connect to it using only the mac address or ssh into it ..would work ...I didn't try using the tried and tested way of doing it.
I wanted it to be an adventure. I wanted it to be a problem to solve so I shoved the ordinary answer out of the way and used other methods to try and connect to it...
All I had to do was used Nmap to scan the gateway for open ports and realise to view it in the Browser on port 8080 instead and finish my journey ...
I was looking for a dragon to slay , a maze to conquer, glory at the end of my mission ... when all I felt was a sheer sense of idiocy.
I have a small NUC-like machine in my home with an old external hdd connected to it. I use it to run my local gitlab, nextcloud and to test a few websites I build for the lolz.
If you too have a homelab, whether it's a single raspberry or an entire room full or racks, you know damn well that everything you have running locally as a web service keeps going until it doesn't, for whatever fucking reason. This time, it was the turn of my nextcloud.
The machine has arch linux running, I chose it since I already use it on my coding laptop and being a rolling release means I don't have to manually upgrade to a newer version, risking various fuck-ups and consequent screaming of profanity.
The downside is that arch is a bleeding-edge distro, so, despite being pretty good for what concerns security, as updates are pushed out some packages may still require legacy software to work as intended, since obviously not all developers for all packages can release simultaneously.
The problem was that php reached 8.2.x but nextcloud couldn't use anything beyond 8.1, so the highlighted solution was to download php-legacy, a package with a set of utilities which the cloud could use instead of mainline php.
Pretty easy, right? fuck my life, here we go.
I edited apache-httpd's configurations to link the new libraries, updated every reference in every virtual host that could possibly screw up the web server.
Then I went on and disabled the php-fpm mainline, creating a new systemd unit that would instead run the legacy executable and afterwards I edited nextcloud's additional configs so they use that instead.
Done, getting a bit dizzy, but I reboot everything and breathe.
At this point the migration should be complete, but wait, the server returns an error saying that the application is still trying to use php 8.2+...wait, what in the sysadmin Christ?
Back to nextcloud config, everything is set, everything else in every other fucking php-legacy and web server is fine, the old fpm service is disabled, I am confused, and why in the FUCKING FUCK is the new php-fpm unit failing to start at boot with "error 78/config - directory not found"? Hello? Am I being trolled by a shitty dual-core amazon fake NUC?
Maybe yes, cause it turns out that the unit was referencing a directory in the external hdd, which gets mounted at boot time after the unit itself starts, so nothing much, just a matter of tinkering with cron jobs, a reboot and at least this one is off my balls.
But why still isn't the server responding correctly? why? WHY?
After slamming my cock on the keyboard here and there scrolling back through all the config files I think to myself, hmmm, my gitlab is working flawlessly, well yeah, I didn't need to install the whole web stack, everything was nice and easy wrapped in a docker container...so why am I even here, why the fuck am I bothering with all this layered web-app bullshit, why don't I just run the up-to-date docker image that someone else has already set up for me, back up all the data and reupload them on the application?
Oh joy, you can't imagine, after 3...almost 4 hours of pure computer-touching the relief I had from seeing the blue web page with the "welcome to nextcloud" title.
Right now it's copying back all the files, and the external hdd is now linked to include the data folder.
Like really, everything was solved in two lines of bash.
I am still fuming, but at least I learned a valuable lesson, if you want a service up for yourself, implement it and deploy it as fucking easy straight-forward as you can, giving MAXIMUM priority to already fully-working options that are out there just waiting to be downloaded and used. I swing my scrotal sack on web-apps elegance as long as it's MY homelab in MY place.
Eat a fat dick php.
sudo pacman -Rns nextcloud
sudo systemctl disable --now php-fpm-legacy
sudo pacman -Rns php-legacy
sudo pacman -Rns $(sudo pacman -Qdtq)2
The day when you Plan an Alarm clock which will awake you when one of your server have trouble.
Hm I think that should look like this:
Fucking AWS Elastic Beanstalk took a week of work to get fucking mounts set up. They invented their own version of docker compose that is missing half the features so I had to work my way from hacking their scripts to include options they don't support, to restarting the whole fucking docker service on every fucking deployment and now the shit finally works. How can most StackOverflow answers just say restart docker, this shit is not ok! I fucking hate sysadmin work. I want to code :(
I run two servers, one that runs WordPress multisite and one that runs a vpn. As a self taught sysadmin I learn best through projects. I’m also interested in databases and backends for iOS (swift) apps. Do you have any suggestions for what I should make my servers do next? Thank you.2
I am busy with multiple projects simultaneously, and this... very nice business guy that I've been working with for the past year just decides to send me a message telling me that I should send our sysadmin a message and ask why he can't login to our ticketing system... You have him in your contacts, I'm not your fucking butler holy shit...
For context, I rarely ever work together with the sysadmin, and he's quite easy to reach1
As SysAdmin I did set up a server with WordPress, so that the external webdesigner create the templates and stuff and everything.
And they can't change the language of the backend.
Well. now it seems I am a WordPress Admin? 😑3
Some days I just want to move to a bigger city, to get in touch with other kind of people, to get insights of my ideas, to hear about unsuccessful projects.
I know the Saint Internet is there to bring people together, but it's not the same thing.3
Our sysadmin is leaving in a few weeks and I'm sure he's going to leave us in the shit.
He's not the type who would format all the drives on the server on the last day, he's more subtle, like resetting our passwords or leave a backdoor into the system so he can keep an eye on us.
Unfortunately our line manager doesn't agree with me1
Doing network security and infrastructure(im a dev but our sysadmin resigned so the next person they thought capable is me)...like seriously yesterday Im fixing c# bugs then the next day I find myself having a meeting with system/security admins...i feel so noob :)1
I need some help, does someone know what is a good way to get a legit linux sysadmin certification? I've been using linux for a couple years now and i'd like to have it on my (baren) resumee :D
I freakin love this community btw, its hilarious
I have done some experiments on my server in the past. It's a great way to learn new things. However, I am bound to make some mistakes and over time the sever becomes messier and messier.
A week ago I installed UNRAID on my machine and I love it! I can now have my critical infrastructure live and working in docker containers and vms.
Then if I want to do an experiment I spin up a VM in a couple of minutes to do my thing and remove it when I am done. No traces left!
As my first post, i wanna ask a question :)
How is Gnu/Linux Sysadmin's possition in your country? :)
(If you want, you can say your country.)
When the SysAdmin recommends to rewrite the whole Java (Spring) web app in Clojure. I mean, let's each of us do our assigned jobs.
Gitlab as a product is awesome, the real wtf is the processes (manual, automated or otherwise) and people supporting their cloud offering.3
It's those days when you get to spend 10 hours moving your servers from one room to another, that make you think "why didn't I choose to become a podologist"...
So I started out in 2010ish as an intern, entirely remote. It let me attend school in my home state while working for a company elsewhere. Fast forward to 2017, I leave that company to work at a college, as a hybrid model. Found I was more productive on days that I was home/working from the lab versus days I was in the office. Skip to 2018, I get a job working for the Air Force which is ALSO hybrid. It wasn't bad, but it wasn't as good for me as when I was remote. In 2019 I started working for NYS and had to report to an office full time. YIKES was that not for me. My mental health started slipping, my physical health went out the window, and I barely got anything done. Along comes COVID and I'm back to 100% remote! Well, NYS Gov Cathy GoKill is trying to push state employees back to the office, and I really don't want to deal with that unnecessary stress again. Ever. Does anyone have any tips for starting out or looking for employment in the private sector, when my priority would be staying remote?
Bribing sysadmin with an ice cold Coke hoping he will deploy my latest tag sort of in a hurry since I fudged up last one :/1
Which one of these job types is the easiest to get into(interview and HR-wise, not professionally)?
- Full Stack Developer
- Android Developer
- Backend Developer
- Devops Engineer
I'm a security/sysadmin who wants to move to dev and have some skills and experience in each of them. Do you think they will take that into account?5
8 hours of coding later and Im back where I began, and Im not even a dev, Im a sysadmin with a little PHP background tasked to write a Sku generating bundle for a PIM running on symphony.
<Insert I have no idea what Im doing dog meme here>1
We have 2 sites. I, dev, set the https system for one in 10 minutes. The sys admin has been working on it for two hours already and he doesn't want help2
So basically a client's website still works on PHP 5.4. I manage the site's content which works on an old server that takes around 5-6 seconds each time I work on the backend (WordPress).
Asked the client's sysadmin to upgrade but he doesn't want in case some old and non updated plugin would break the site. (Which did about a year ago and I had to fix it).
Feels like working on a minefileld.
I'll fucking kill you bitch who the fuck made you the system administrator of this fucking college? The fuckery you do... god damn some students know more than you and you should agree to this fact. The previous sysadmin was knowledgeable and you are just a fucking worker in the finance department. You fucking dare not call yourself a sysadmin when you can't even read the fucking docs provided by the G Suite. At least you didn't make the mail server yourself otherwise god knows what the shit you'd have spread around.
Hello, i am new here and I saw a lot of great people with good advices, so here I am. I am new in sysadmin field and i need some advice from you. Now I watch CCNA videos and practice in packet tracer. What do you recomand? Videos/ programs/ OS . ( i am a computer science student).4
In my case, the only way to stay productive is to task switch often. Suffering of adhd, I get bored of researching / developing a specific solution rather quickly, and then have a huge issue staying focused.
That's also why I can't imagine being a programmer. Being a sysadmin, however, is great! Dealing with many different tickets a day.
This is a repost of an original rant posted on a request for "Community Feedback" from Atlassian. You know, Atlassian? Those beloved people behind such products as :
• Thing I Love™
• Other Thing You Used One Time™
• Platform Often Mentioned in Suicide Notes, Probably™*
Now this rant was written in early 2022 while I was working in an Azure Cloud Engineer role that transformed into me being the company's main Sysadmin/Project Manager/Hiring Manager/Network Admin/Graphic Designer.
While trying to simultaneously put out over 9000 fires with one hand, and jangling keys in the face of the Owner/Arsonist with the other, I was also desperately implementing Jira Service Desk. Normally this wouldn't have been as much of a priority as it was, but the software our support team was using had gone past 15 years old, then past extended support, then the lone developer died, then it didn't work on Windows 10, then only functioned thanks to a dev cohort long past creating a keygen....which was now broken. So we needed a solution *now*.
The previous solution was shit of a different tier. The sight of it would make a walking talking anthropomorphised sentient puddle of dogshit (who both eats and produces further dookie derivatives) blush with embarrassment. The CD-ROM/Cereal Box this software came in probably listed features like "Stores Your Customer's First AND (or) Last Name!" or "Windows ME Downgrade Disk Included!" and "NEW: Less(-ish) Genocide(s)"!
Despite this, our brain/fearless leader decided this would be a great time to have me test, implement, deploy, and train everyone up on a new solution that would suck your toes, sound your shaft, and that he hadn't reminded me that I was a lazy sack enough lately.
One day, during preliminary user testing I received an email letting me know that the support team was having issues with a Customer's profile on our new support desk. Thanks to our Owner/Firestarter/Real World Micheal Scott being deep in his latest project (fixing our "All 5 devs quit in the last 12 months and I can't seem to hire any new ones" issue (by buying a ping pong table)), I had a bit of fortuitous time on my hands to investigate this issue. I had spent many hours of overtime working on this project, writing custom integrations and automations, so what I found out was crushing.
Below is the (digitally) physical manifestation of my rage after realising I would have to create / find / deal with a whole new method for support to manage customer contacts.
I'm linking to the original forum thread because you kind of need to have the pictures embedded in said reply to get really inhale the "Jira-Rant" ambiance. The part where I use several consecutive words as anchor links to tickets with other people screaming into the void gets a bit sweet n' savoury too - having those hyperlinks does improve the je ne say what of it all.
bit.ly/JIRANT (Case Sensitive)
There is some good news at the end of this brown n' squirty rainbow though!
Nice try silly little Jira button, you can't ruin *my* 2022!
• I was able to forget all about Jira a month later when I received a surprise vacation home! (To be there while my Mom passed away).
• Eventually work stress did catch up to me - but my boss thoughtfully gave me a nice long vacation! (By assaulting *while* firing me (for emailing in a vacation request while he was a having a bad (see:normal) day))5
I'm from Tunisia and I'm looking for opportunities in the USA. One point bothers me. What's the average salary of an entry level DevOps/SysAdmin engineer?!1
Where do you guys get your daily news about Tech, IT, Dev Related things?
I want something with push notifications and just serious sources without bullshit infos.9
What is the best alternative to cronjobs, guaranteeing high availability and jobs not being duplicated?6
Does anyone from here working by GMX? I am specially looking for a sysadmin.
The story is the following. We can't send emails to GMX addresses in general. I've contacted my provider, and they said, that they've contacted GMX several times but no solution has been made so far. This was almost a month ago and the problem still persists.
If anyone from here willing to help me clear this mess, or just give some explanation, I would be grateful. We are loosing reputation as a company having to answer from a different email address.
If it is a sensitive info please give me a channel where we can speak about the details.
Please note I am not a sysadmin by the hosting company, i am simply a customer of theirs.14
Suggest me one of the best sysadmin course online which is really worthy.(paid as well as free)
I'm Linux user from for years but have always been inclined towards scripts.3
I had this dell server lying around and finally got to make a virtualisation server out of it.
It is now running xcp-ng as its hypervisor, with a CoreOS VM in it, containing a docker container serving xen-orchestra for managing the server.
Enterprise grade hardware really is a thing of its own. Also sysadmin type of stuff is quite fun. I look forward to be playing with it some more. :)1
Not my fav because I'm not proficient in anything and don't consider myself a programmer. But, I use many languages day to day as a sysadmin.
I come across Perl enough to know I don't like how their modules and dependencies work. I have the most difficulty when dealing with this. PHP, Python, Ruby, and GoLang never give me as much trouble as Perl.
Also, coming across more Python3 dependencies, dealing with older Python2 environments, as stated by many others as well, is becoming more and more painful.
Maybe all of this can be solved with some unifying virtualenv for all popular languages/environments, supported fully by the underlying OS.
I've been lurking for a bit and had a question what dev/coding skills should I be looking at to be able to move up? I currently do support for large cluster machines but not full admin work. I want to move to a more sysadmin type position but my coding/scripting is not the strongest and wanted to hear your thoughts
I am a front-end developer with 2 years of experience and I want to shift over to devops role. How can I get started and what are my chances to interview for any devops role especially since I haven't worked as one or as a sysadmin? Any help is greatly appreciated.1
Am I the only one who's hands start shaking when about to send "CHANGE MASTER TO" on a dev server?
Happened to me yesterday, replication got stuck after corrupting a relay log file when the database segfaulted under my hands.
I could check and recheck the positions I was about to reset it to a bilion times and I was still nervous!
I am the only Backend developer in my team, so I have to do sysAdmin tasks, deployment and configurations myself....
I HATE it!