Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
What a stupid configuration of firewall at my work:
devrant -> blocked because of entertainment category.
xvideos -> no problem at all.
Conclusion: sysadmin likes watching porn.14 -
"You should use Windows server!"
It was a high security project which needed to run very stable. Even the windows sysadmin looked at that guy like 'dude what the actual fuck'.27 -
Happy SysAdmin day ... even if I’m wondering if sysadmins can be happy.
Source : xkcd (of course...)
3 -
Senior IT engineer enters the room and quietly talks to a coworker about a job related issue.
Another coworker decided to troll the sysadmin.
CW: *yells* "Open a ticket!" (That's the sysadmin's regular reply)
IT: *ignores*
CW: *trying to get his attention* "Open a ticket first! Then come back"
IT: *gives him the stare of death*
CW: "Go away and open a ticket!"
IT: *silently leaves the room*
After no more than a minute CW gets a reject from all networks outside the company's VPN.
IT comes back into the room, get's intimately close to CW's ear and says "Now open a ticket".
👋
🎤9 -
I have this one friend who thinks he is a tech guru just because he plays video games a lot and started to study cs for one year. Now he got a job as sysadmin and it is funny to hear him brag about the job in front of non-tech people because he sounds like a CSI Cyber episode, just throwing tech words at the people and I know that he talks bullshit.
But I have to admit, he knows how to sell himself. Probably that's how he got the job in the first place because it cannot be his experience.
Yesterday he called me, to help him edit something on a linux server. I told him "To edit the file type 'vi FILENAME' and then you can edit. I have to go now, I have a meeting." :]23 -
Pro tip:
Although 'hmm either kill it or if that doesn't work, sacrifice some of its children' is a perfectly valid sentence in the sysadmin world, it's not in public.
😅10 -
Random person: wanna get a girlfriend? Shave your beard.
sysadmin@condor:~# no.
sysadmin@condor:~# FUCK NO.
sysadmin@condor:~# What are those muggles even thinking?!!
sysadmin@condor:~# Would you also ask Khal Drogo to cut off his ponytail?!30 -
One of my ex-girlfriends (who apparently still cares about me after several years 🤔) sent me this chain letter kind of thing wishing me 12 months without sickness, 52 weeks without stress, 365 days of luck, 8760 hours without fights, 525600 minutes of peace, and 31536000 seconds of happiness.
But that's not what I want mate! All I want is a year of <50ms ping!! 😝
I still kind of like her though, especially given that she's still thinking about me.. maybe I should have trying to go out with her again as one of my objectives for 2019?19 -
I decided to draw something while checking out 3000+ builds happening in parallel in the school infra
sysadmin life is boring
10 -
Client got hacked and mauled to another dimension. Why? Telnet w/o limited login attempts. All because Sysadmin likes to have the option of bruteforcing in case he forgets his own password.. karma?10
-
When you're a junior sysadmin but still have to maintain ALL the production server:
How it looks:
$ sudo apt-get update
How it feels:
& sudo [ $[ $RANDOM % 6 ] == 0 ] && rm -rf / || echo *Click*7 -
Sysadmin TIL:
Hiring PHP developers does not contribute to the quota of employees with disabilities.2 -
In the compeny where i work for...
Me: who's our sysadmin?
Some senior: we don't have one. Me, i guess?
Me: (pokerface. yeah..right. walk away.)11 -
SYS_ADM: We have something important on the internal GitLab?
ME: Please tell me it is working
SYS_ADM: I take it as yes...5 -
Sometimes I feel I'm an app developer, a web developer, a sysadmin, an ethical hacker and a programmer who's comfortable in several languages. At other times, I feel like I just know how to use the internet.5
-
TL;DR: I have been changing stuff on a school computer and found various workarounds to do things that were blocked by the sysadmin and my classmates think I am a hacker.
I guess war has officially been declared between me and my school's sysadmin.
It all started about two weeks ago. I was bored while writing a report on a school computer, so I decided to lauch Chrome and pressed F12. Nothing happened, so I closed the window and opened Firefox. It worked there, and a few classmates started to scatter around me. They thought I was hacking a website so that changes would be permanent.
But what really matters in this story is that a few days later, nobody could start Firefox anymore, a message popped up saying the administrator blocked the application. Somebody must have reported it.
I also tried executing a Batch file, not succesful of course. But surprisingly enough .cmd files could be executed. But this has also been fixed a few days after.
So today I managed to remove Chrome's settings folder and the console popped up this time. Again, this was hacking from my classmates' point of view, so right now I am waiting for Chrome to be blocked. There are no other browsers currently installed, so let's see what happens.18 -
When the Sales team fucks up something, they point finger at each other for the blame.
When the Dev Team fucks up something, as a Senior Developer I say to my Boss that we fucked it up no matter whose fault it is and we will fix it up by x time.
When the SysAdmin team fucks. The Dev Team is to blame.
Sorry guys, I got a bit frustrated. All our servers wasn't backed up from last week and the SysAdmin guys are saying it's our fault.
What the fuck is going on? fuck you fuck fuck fuck fuck...9 -
According to my sysadmin, there is no point in changing our shitty, 20 year old website because technology develops so rapidly, and making it more accessible by Google is overrated...
6 -
After 1 year I have finally quit my sysadmin job!
Got my first dev job as a fullstack node.js dev!!!!
4 years of IT boredom is finally over!
WOOHOOO!
😎😁9 -
To all IT-guys out there, be it the desperate sysadmin or the kind lady of the support team, I whish all of you and your family/friends a merry christmas! 😃
4 -
When you see a semi bald man with a messy beard, bit too much belly, a dead look in the eyes that carries a pc bag.
And your first thought is "oh, a sys admin" x)7 -
Was browsing job sites and noticed an 'experienced sysadmin' job (Linux).
Everything seemed pretty interesting and then I saw this line (at what they offer):
Free own domain with hosting!
That's genuinely cute offering that to an experienced Linux sysadmin. Not saying every sysadmin has this but I for example have like 10 domain names and a bunch of servers.
Yeah that looked cute 😆27 -
Happy international cat day, everybody!
And a nice day to every sysadmin out there, hopefully not too catty.
4 -
>give person jailed access to my server.
>person breaks out, and notifies me.
>i fix the hole
>three weeks later i decide hes trustworthy enough and the jail is causing too many problems so i give him unjailed access
>first thing he does is crash the system
albeit unintentionally, but seriously?1 -
Every time I see a guy with long untrimmed beard, I automatically assume he is a linux sysadmin. Should I feel bad?7
-
After almost two years working as a Linux Sysadmin, I have found out that every developer needs a Sysadmin.17
-
I recently got a job as a sysadmin and they've been debriefing me on their hacked websites (wordpress malware injection). Beats me why they still have their sites up at all...
BUT WAIT THERE'S MORE!
I wanted to see if they have any backups... NONE.
The latest snapshot was over a year ago...FML. Over a year ago when they barely have anything on their company site and client sites 😒
Now, I have to revive 10 websites from redirection. Time to do some shell scripting!
4 -
Life of a junior self-taught dev with a sysadmin job:
1)At work, desperately try to script and automate every task, even when it isn't nessecary.
2)Learn dev skills from tutorials and web courses at every minute of your free time.
3)When returning home get self-guilt because you're procrastinating instead of doing an all-night development like your dev friends
4)The only productive thing you do is more tutorials and courses because you feel your dev skills aren't high enough for a self project
Frustrated.13 -
Being a sysadmin, I never write any code.
But I do want to learn that.
What is the easiest language to learn?
(I am lazy)54 -
This guy has a weird sense of system security if he thinks an SSH MOTD will keep unauthorised people away. Because you know, setting SSH permissions would be too sensible.
15 -
I felt kind of clip-winged as a webdev/wannabe sysadmin without a server.
Merry Christmas, everyone!
🎁
1 -
The joys of being the sole developer and sysadmin of a service with hundreds of thousands of users.
Just spent a couple hours with my family. In that time half the infrastructure died and the service became unstable.
Best of all is that I seem to be the only one getting this so called "java.net.UnknownHostException: System error" exception.
2 -
That feeling when your client connection is more stable than the connection of a fucking game server... Incompetent pieces of shit!!! BEING ABLE TO PUT A COUPLE OF SPRITES DOESN'T MAKE YOU A FUCKING SYSADMIN!!!
Oh and I sent those very incompetent fucks a mail earlier, because my mailers are blocking their servers as per my mailers' security policy. A rant from the old box - their mail servers self-identify a fucking .local!!! Those incompetent shitheads didn't even properly change the values from test into those from prod!! So I sent them an email telling them exactly how they should fix it, as I am running the same MTA on my mailers (Postfix), at some point had to fix my mailers against the exact same issue as well, and clearly noticed in-game that they have deliverability problems (they explicitly mention to unblock their domain). Guess why?! Because their server's shitty configuration triggers fucking security mechanisms that are built against rogue mailers that attempt to spoof themselves as an internal mailer, with that fucking .local! And they STILL DIDN'T CHANGE IT!!!! Your fucking domain has no issues whatsoever, it's your goddamn fucking mail servers that YOU ASOBIMO FUCKERS SHOULD JUST FIX ALREADY!!! MOTHERFUCKERS!!!!!2 -
PM: "Hey, can you send an email to SysAdmin I can't print in colours?"
And vagues off to a meeting.
Me: "what the fuck"16 -
PSA: "sudo apt-get remove nginx" doesn't actually remove nginx. It will still continue to run and block port 80 on every reboot.
Until you run sudo apt-get autoremove, nginx-core and others still remain.
And that's how twenty seconds of scheduled downtime turns into 10 minutes.
Dammit nginx1 -
Back in my sysadmin days we had an IT zoo to look after. And I mean it... Linux side was allright, but unix.... Most unices were no longer supported. Some of their vendors' companies were already long gone.
There was a distant corner in our estate known to like 2 people only, both have left the company long ago. And one server in that corner went down. It took 2 days to find any info about the device. And connecting to it looked like:
1 ssh to a jumpbox #1
2 ssh to a jumpbox #2
3 ssh to a dmz jumpbox
4 ssh to an aix workload
5 fire up a vnc server
6 open up a vnc client on my workstation, connect to than vnc server [forgot to mention, all ssh connections had to forward a vnc port to my pc]
7 in vnc viewer, open up a terminal
8 ssh to hp-uxes' jumpbox
9 ssh to the problematic hp-ux
.....6 -
Earlier I signed up on this forum called NulledBB. Basically some hacker skiddie forum that had a dump of an archive I wanted, unfortunately behind a paywall which I didn't want to bother with.
On signup I noticed that I couldn't use my domain as an email address, as I usually do (the domain is a catch-all which means that mail addresses can be made up for each service I sign up to on the fly, super useful). They did expose the regex that they accepted email as however, which included something along the lines of "@live.*".
So I figured, why not register a subdomain live.nixmagic.com real quick and put that into the mail servers? Didn't take too long and that's what I eventually went with, and registered as somepissedoffsysop@live.nixmagic.com (which I have no trouble putting on a public forum as you'll see in a minute).
Still didn't manage to get that archive I wanted but I figured, fuck it. It's a throwaway account anyway. But eventually that email address started to receive spam. Stupid motherfucker of a forum operator with his Kali skidmachine probably leaked it.
Usually I just blacklist the email address in SpamAssassin by adding an additional spam score of 100 to email sent to such addresses. But in that case it didn't even sit on the main domain, thanks to that stupid regex block from earlier... 😏
*Logs into my domain admin panel*
*Le rm on the live.nixmagic.com record*
Null routed entirely.. nulled, if you will! 🙃3 -
Of course you can call me at 9 o'clock on a saturday morning to fix your f****** login problem!
My private life is just a rumor!1 -
Wk1:, Client wants stack deployed to AWS in a day... Does it in 30 minutes...
Wk2: client complains about cost of service so requests I downsize the stack they asked for... Does so... Just to save some £££ at loss of any DR or HA capability
Wk3: Client wants stack moved to a different AWS account just cos... Advised could cause issues... Client says carry on. Migrates to account as requested on the days they requested.
Wk4: client complains that said migration caused issues and that proper change control wasn't followed..... That was never informed on..
Wk5: issue discovered to actually be network fault linked to clients wanting every £££ saved in AWS... And as the stack specialist I still have to write up a summary and findings?
Wow just get a decent AWS admin will ya....3 -
Our sysadmin just wrote our new work account passwords on our office whiteboard, visible to everyone... Now that's how you create chaos2
-
My school just tried to hinder my revision for finals now. They've denied me access just today of SSHing into my home computer. Vim & a filesystem is soo much better than pen and paper.
So I went up to the sysadmin about this. His response: "We're not allowing it any more". That's it - no reason. Now let's just hope that the sysadmin was dumb enough to only block port 22, not my IP address, so I can just pick another port to expose at home. To be honest, I was surprised that he even knew what SSH was. I mean, sure, they're hired as sysadmins, so they should probably know that stuff, but the sysadmins in my school are fucking brain dead.
For one, they used to block Google, and every other HTTPS site on their WiFi network because of an invalid certificate. Now it's even more difficult to access google as you need to know the proxy settings.
They switched over to forcing me to remote desktop to access my files at home, instead of the old, faster, better shared web folder (Windows server 2012 please help).
But the worst of it includes apparently having no password on their SQL server, STORING FUCKING PASSWORDS IN PLAIN TEXT allowing someone to hijack my session, and just leaving a file unprotected with a shit load of people's names, parents, and home addresses. That's some super sketchy illegal shit.
So if you sysadmins happen to be reading this on devRant, INSTEAD OF WASTING YOUR FUCKING TIME BLOCKING MORE WEBSITES THAN THEIR ARE LIVING HUMANS, HOW ABOUT TRY UPPING YOUR SECURITY, PASSWORDS LIKE "", "", and "gryph0n" ARE SHIT - MAKE IT BETTER SO US STUDENTS CAN ACTUALLY BROWSE MORE FREELY - I THINK I WANT TO PASS, NOT HAVE EVERY OTHER THING BLOCKED.
Thankfully I'm leaving this school in 3 weeks after my last exam. Sure, I could stay on with this "highly reputable" school, but I don't want to be fucking lied to about computer studies, I don't want to have to workaround your shitty methods of blocking. As far as I can tell, half of the reputation is from cheating. The students and sysadmins shouldn't have to have an arms race between circumventing restrictions and blocking those circumventions. Just make your shit work for once.
**On second thought, actually keep it like that. Most of the people I see in the school are c***s anyway - they deserve to have half of everything they try to do censored. I won't be around to care soon.**2 -
I hate when someone throws at me some task all of sudden with a tight deadline.
Wednesday was one of those days.
manager: we want to remove all the offices because of our tight budget this year (multimillionaire company, lol), everyone will use office 365;
me: ahn... ok, but everything was already tested? Some macros, routines, old documents can be a big problem, as far I know (I don't use M$ at home, servers are Linux, so I really don't know about that). I can do some tests, only will need some real documents to make sure everything will do fine;
manager: yeah, yeah, everything will be fine, the high management already decided, don't worry, just remove the offices in the company, ok?;
me: alright...
*me deploys the remotion script in every f*cking machine*
48 hours later...
manager: well... everyone is complaining about the office 365, random complains, can you attend all the calls and reinstall if you can't solve the problem?
WTFFFFFFFFFFFFFFFFFFFFFFFF!!!!!
[RAGES INTERNALLY]
5 -
Cpanel and plesk is just two newly invented words that means the following:
Anger
Insecurity
Hate
Stupidity
Virtual AIDS
"I am pretending to be a sysadmin"6 -
This one's for all the SysAdmins out there.
About 4 years ago I was asked to take over a dental offices systems administration (~20 machines) after their previous guy had allowed their servers RAID 1 to fail and hadn't done any updates or general maintenance. (please take note this office is my parents dental office).
I since have been recovering from his poor configuration and setup by instating an active directory environment and installing up to date software as well as updating machines on the domain to Windows 10 since windows 7 is no longer supported. I have also been properly licensing everything.
My bosses (my parents) are annoyed with this because "it's more expensive" and "it's too complicated we don't know how to manage it" and I don't know how to explain to them that they aren't fucking systems admins. They asked why they could do it before and I tried to explain that now it's secure and things need to be rolled out on the network level. They had every user running full local admin on every workstation plus the server.
Some people don't fucking understand that just because it's simple doesn't make it a good fucking idea. And because it's cheap doesn't mean it will always be (just wait till Microsoft audits you).
Oh and they also don't understand fucking CAL licensing and refuse to pay for gsuite for all their staff who use it. Instead they just have two gsuite accounts and give everyone the fucking password.
I'm going to have an aneurysm6 -
Three days I'm not there, three bloody days and the 2nd sys admin has managed to:
Destroy the auto load balancing that I have scripted and implemented
Fuck up the backups to the cloud, and to a tape drive.
Overload a physical server (in pic, #1 and #2 are hypervisors). This is just one smaller site, I would hate to know what he has done elsewhere.
This is why I'm moving into dev.
5 -
Just got asked by our JIRA admin how to add in JIRA add-ons for our self hosted version.
... if you don't know why are you admin then!?3 -
when you type faster than computer response:
------------------------
Ubuntu 16.04.5 LTS server tty1
server login: sysadmin
adminPassword:
Login incorrect
server login: sysadmin
Password:
sysadmin@server:~$ _
------------------------
"FUCKING SHIT !"
*sees if there are anyone in the back*
*saw no one*
"fiuuh... what a relief"
sysadmin@server:~$ clear3 -
GUYS I DID IT
AND I ONLY HAD TO LOOK AT THE ARCHWIKI FOR GRUB INSTALLATION
AM I COOL NOW
(going to install to my computer a little later, don't worry i'm not done here)
8 -
that moment as a sys admin when everything is in peace and you have nothing to do.
I usually go outside and get me a nice cup of coffee at a local cafe
today its a Latte Machiato with white chocolate
4 -
Sysadmin: Apps on containers and kube is mandatory from now on, scaling is mandatory!
Devs: The systems weren’t designed for containers, we haven’t prepared shirt for scaling!
Sysadmin: Hold my beer!
-
>Installs NodeJS (from default Debian repo)
>Tries to install yarn
>Yarn tries to uninstall nodejs
Weird
>apt-cache depends yarn
yarn
Recommends: nodejs
Conflicts: nodejs
10/10, gave me a good chuckle. Time to add the NodeJS repo.10 -
Who has a DDOS attack story they want to share ? Dyn put up the good fight today... DDOS attacks can be incredibly difficult to deal with ... Internet of Things devices makes this an even more complicated situation. Outside of calling Prolexic, any vets have some good stories ?6
-
To me, it seems like the rise of distributed systems like mesos / kubernetes combined with Docker require you to be master sysadmin, veteran kernel hacker and a part time c developer ALL AT ONCE if you really want to shave off time from debugging/ performance tuning sessions. Anyone wish they paid more attention in class ? Lol.
4 -
What kind of genius pulls the power cable of the main server through the PSU lever of an old useless server?
I had to pull out the PSU and let it hang on the powercord just to be able to pull the old ugly bastard out... fucking genius, really!
Now I have to wait until the evening to shut down the main server and remove the hangman PSU.
I just really fucking hope and pray that restarting the main server will not cause errors. It has been on for 2 years and never been updated since then.5 -
//little Story of a sys admin
Wondered why a Server on my Linux Root couldn't build a network connection, even when it was running.
Checked iptables and saw, that the port of the Server was redirected to a different port.
I never added that rule to the firewall. Checked and a little script I used from someone else generated traffic for a mobile game.
OK beginn the DDoS Penetration. Over 10 Gbit/s on some small servers.
Checked Facebook and some idiot posted on my site:
Stop you little shithead or I will report you to the police!!!
Checked his profile page and he had a small shitty android game with a botnet.
Choose one:
1. let him be
2. Fuck him up for good
Lets Sudo with 2.
I scaled up my bandwith to 25 Gbit/s and found out that guys phone number.
Slowly started to eat away his bandwith for days. 3 days later his server was unreachable.
Then I masked my VoIP adress and called him:
Me: Hi, you know me?
He: No WTF! Why are you calling me.
Me: I love your're game a lot, I really love it.
He: What's wrong with you? Who are you?
Me: I'm teach
He: teach?
Me: Teach me lesson
He: Are you crazy I'm hanging up!
Me: I really love you're game. I even took away all your bandwith. Now you're servers are blocked, you're game banned on the store.
He: WHAT, WHAT? (hearing typing)
Me: Don't fuck with the wrong guys. I teached you a lesson, call me EL PENETRATO
He: FUCK Fuck Fuck you! Who are you???!!! I'm going to report you!
Me: How?
He: I got you're logs!
Me: Check it at Utrace...
He: Holy shit all around the world
Me: Lemme Smash Bitch
*hung up*4 -
hi! I'm your friendly neighborhood sysadmin/operations bastard. I also write mostly okay python, ruby, and c.
This is called devrant because it's where you go to complain about devs, right? /s
anyway, hello!7 -
My old job was almost perfect. I was a systems engineer for a research network. My duties were to configure, build, install, secure, manage and repair Linux hosts used for research on projects so advanced/cutting edge that I could spend days just listening to researchers explaining them and I honestly loved it! I understood less than half of the projects but just seeing how motivated and excited the researchers are made the job my favourite. Unfortunately I had to leave and get a job closer to my house because having a 2 hour (one way) commute for two years was killing me :-/ relocation wasn't an option and still isn't but I'd be lying to myself if I tried to say I wouldn't go back as soon as I could.2
-
You know the worst thing about being a freelancer? You're expected to wear every fucking hat and you don't get normal hours.
Over the past few days I have been working with a client of a client attampting to fix his server. He's running CentOS on VMWare and somehow ended up breaking the system.
Upon inspection there was no way to fix his system remotely. It wouldn't even boot in recovery mode. So we've been attempting to recover his data so that we can reinstall CentOS and not have to start completely from scratch.
So for the past 3 days straight I have been remotely logging in to a Debian Live CD and manually sending folders to a FTP server of his. He has somewhere close to 30 sites on this server, and upwards of 1 million files in total.
Yesterday either the system freaked out or he did something, but the entire fucking system stopped responding which forced me to reboot it, reinsert the live CD, reinstall evertything, and re-mount his broken systems drives.
Here we are 3 days in, we're still not done, and I'm getting slightly pissy because if you don't know Linux well enough to fix this shit yourself, you shouldn't be acting as your own sysadmin for 30+ sites.
Also, backups are a thing right? VMWare also has snapshots. I know the extra storage isn't cheap, but it's a hell of a lot cheaper than paying soemone like me $35/hr to go and fix all of your shitty mistakes.2 -
Aren't the system admin supposed to figure out how to install something on their server when a developer has requested something from them?
They seem to have no idea when I request them to install php gd extension. They also cannot give me a ssh access to their server. So I have to troubleshoot/help by sending one command and ask for the output from them and give another command to run through mail.
I don't even know what to rant or whom to rant at anymore.
// I'm blue.
12 -
Gosh ! I'm a genius !
- Working on a Sysadmin school project
- configuring the firewall
- looking myself out
- beg my teacher for reinstall
Fml right now ...6 -
Recently I've been put in charge of checking if all backups work and the servers aren't getting full.
The first thing I did was write a script that does it for me.
Don't work hard, work smart 😎5 -
"Hello, the drive of your XYZ server is getting full, would it be possible to prune some of the unused and/or old docker images and layers there please? Alternatively, we can offer to replace the drives with a higher capacity models for FOO extra per month"
"Hello, the disk use keeps growing and has reached the 95% mark, please prune some of your images to make space for new. If you wanted to choose the alternate option of disk capacity increase, we would have to do that as soon as possible, otherwise you may run out of space before the RAID array rebuilds"
"Hello, your server XYZ has completely ran out of disk space. Any changes that would require data being saved on disk may and probably will fail. Please free some space as soon as possible"
Ugh, I hate clients that just don't cooperate until shit hits the fan...
And no, we could not prune the space ourselves, its not our data to delete whenever we think it necessary.
We merely manage the machine's operation, keeping it online and its services running.2 -
Unpaid internships are the worst thing. You exploit young people and promise them experience. Seriously business makes tons of money yet they come with ways to exploit a young person in IT. I think it is evil.11
-
A client is like: Help! We got a 500 in our wordpress administration panel and there is no error in the log, it must be your infrastructure at fault!
So I calmly replied to them that wordpress handles its errors on its own, and without the appropriate debug flags enabled, doesn't log it anywhere. Even mentioned that a PHP app can change the error handler no problem, and linked them to both, PHP and Word press docummentation.
Didn't hear from them since.3 -
Yay! Finally got my first job in IT.
They call it just "IT admin/sysadmin", but it's... eh..
What I do is make sure the servers work (sysadmin part) and make desktop/mobile apps. So far, the company seems to be quite nice, there are already 4 devs who are friendly. *knocks on wood*
Will see how it goes, and I'd like to thank you for sharing your stories. Learned quite a lot from them!5 -
>Asks client if the proxy can use self-signed cert
>Client agrees, no problem
>Deploys
>Client complains about "an error they're getting"
>The error: "Error in connection establishment: net::ERR_CERT_AUTHORITY_INVALID"
:|
Am I a joke to you? Or am I just talking to a brick wall over there?7 -
Have a t-shirt saying "Sysadmin - Because even developers need heroes" fellow co-worker said "that's not precise enough" I am looking forward to what he will come up with5
-
what a great idea to do server upgrades on a friday evening...and i can do it alone...and am responsible that everything works as expected...what a great day today...and i hate every single second of this day yet.. :-/
damnit i'm a developer, not a sysadmin, just because i can do it doesnt mean that i'm supposed to do it..what about our admins? what get they paid for? rebooting the coffeemachine? fuckers already left the building1 -
Got released from the miltary after 3 years of CyberSecurity/Sysadmin role.
2 weeks layer I got a job as security sysadmin in a large telco company.
Still wish to be dev instead though -
First rant, technically a sysadmin but getting into the nitty-gritty of programming with some things to improve my job (and hopefully moving into something more technical).
Have been doing a paid internship at my utility company. I do patch management with SCCM and sometimes the updates break. I've been using Powershell to reset the Windows update cache to make the computers work again. Unfortunately, this sometimes involves logging into machines to do some manual work and I have to notify users before I log in if they're already logged in.
Scripts can be run silently but I've spent a few weeks trying to automatically retry Software Center updates with Powershell … before realizing just today that the system center action "Application Deployment Evaluation Cycle" does indeed do the thing I've been attempting to do with Powershell for weeks now.
Wish me luck as I automate that part of the process and completely automate the sole job they gave me to do. Don't tell on me!5 -
Dear past me,
never ever offer a customer to rename their databases during migration.
Renaming databases in mysql is evil and will cost you more time than you think,
sincerely, my present me.
Ugh, stupid triggers and procedures >_>1 -
How a linux sysadmin has sex :
who | grep -i hot && grep -i female | date; cd ~; unzip; touch; strip; finger; mount; gasp; yes; uptime; umount; sleep; -
*Teammate writes some instructions for our users and sends it to the rest of the team to review and let her know before she pushes it out
Me: *reads it and lets her know what is missing
Her: Idgaf what any one thinks. If they can’t figure it out, it’s their problem. And if anyone makes changes to it and the users can’t under it either, I’ll tell them to ask whoever made the changes.
Me: *starts cracking up. Like wtf!?!?! You know what, I’m not touching it so if anyone asks I can tell them to reach out to you!
Like, how are you offended because I’m a junior in your team and you can’t take criticism after you’ve personally asked for it? Smh7 -
GOD DAMN THE STUPID IPTABLES, aaaaah!
Today I learned that
iptables -I INPUT -i !lo -j DROP
and
iptables -I INPUT ! -i lo -j DROP
are two completely different rules, the first of which doesn't work (in ~99.9% of cases)
yet both pass and get added to the firewall. And both rules show exactly the same in the state listing (iptables -L -n -v).
And I was wondering why the hell the firewall wasn't working...8 -
>Be client
>Have an issue with incredibly slow webpage load time
>Blame memcache issues
So... I look into the problem. Yes, the page either loads up fast, or times out. So, into the logs I go. Webserver is fine (except the timeout), PHP though... Error log is fine (just notices), but slow log shows the issue is the database (of course... its always the database... ugh)
So, checking the database, there is one ugly query that seems to be an issue. 5 joins and a huge where condition.
So I run EXPLAIN on the query and... Proceed to bang my head against the wall.
OF COURSE ITS SLOW YOU FU******, NONE OF YOUR TABLES HAVE ANY INDEXES.
What do they expect when the database has to always go down the whole table and do everything in memory, until it runs out and has to dump it all on disk and work with it there.
Ugh... Some clients... -
My colleague and I have been trying to setup a VPN server for devs who wanted to work from home. We asked our sysadmin (who's in another office in another country) to try connecting to the server. She replied after a while that she was unable to connect. We then asked her what port she tried connecting on and she replied, "No, I didn't use any port. I am connecting directly using the hostname".14
-
Being a sysadmin can be the most frustrating thing ever, but it's worth it for those moments when you feel like an absolute ninja.
Switched from single threaded gevent server to an nginx configuration, added ssl, and setup a reverse proxy to flask socketio, all with less than 10 minutes aggregate downtime. On the prod server. \o/3 -
Happy SysAdmin day to all the magnificent SysAdmins all over the world!
Source: http://commitstrip.com/en/2014/...
-
I'm performing a pentest for my client.
So after scanning my client's network I understood they're using IIS 4.5 and windows server 2012 (or 2012 R2)
I know the systems are real old.
And there are known exploits for them.
The tricky part is I have to stay hidden and I only have my own credentials for logging in to the asp page. (Uploading a script is almost crossed cuz it will reveal my identity)
Also I have access to the local network with some of the other employees user/pass.
Any recommendation for exploiting and staying hidden at the same time ?
One more question : will exploits for newer versions work for the older ones necessarily?8 -
So today a Windows sysadmin told me the Registry is a part of Windows he "doesn't know so well" 😂 and the worst part is the faith of about 1000 users lies in his hands 🤦♂️8
-
Our rookie sysadmin frack up our web server today. He wanted to make a single directory and all its content accessible but instead, he used this command...
sudi chmod 644 /.6 -
It’s a bit of a coin toss for me but probably the first sysadmin I worked with Dave, I was a software engineering graduate and tbh he scared the sh*t out of me when I first met him but when he learned I actually enjoyed doing ops stuff, he really took me under his wing taught me so much and I’ll be forever grateful to him for that
-
About slightly more than a year ago I started volunteering at the local general students committee. They desperately searched for someone playing the role of both political head of division as well as the system administrator, for around half a year before I took the job.
When I started the data center was mostly abandoned with most of the computational power and resources just laying around unused. They already ran some kvm-hosts with around 6 virtual machines, including a cloud service, internally used shared storage, a user directory and also 10 workstations and a WiFi-Network. Everything except one virtual machine ran on GNU/Linux-systems and was built on open source technology. The administration was done through shared passwords, bash-scripts and instructions in an extensive MediaWiki instance.
My introduction into this whole eco-system was basically this:
"Ever did something with linux before? Here you have the logins - have fun. Oh, and please don't break stuff. Thank you!"
Since I had only managed a small personal server before and learned stuff about networking, it-sec and administration only from courses in university I quickly shaped a small team eager to build great things which would bring in the knowledge necessary to create something awesome. We had a lot of fun diving into modern technologies, discussing the future of this infrastructure and simply try out and fail hard while implementing those ideas.
Today, a year and a half later, we look at around 40 virtual machines spiced with a lot of magic. We host several internal and external services like cloud, chat, ticket-system, websites, blog, notepad, DNS, DHCP, VPN, firewall, confluence, freifunk (free network mesh), ubuntu mirror etc. Everything is managed through a central puppet-configuration infrastructure. Changes in configuration are deployed in minutes across all servers. We utilize docker for application deployment and gitlab for code management. We provide incremental, distributed backups, a central database and a distributed network across the campus. We created a desktop workstation environment based on Ubuntu Server for deployment on bare-metal machines through the foreman project. Almost everything free and open source.
The whole system now is easily configurable, allows updating, maintenance and deployment of old and new services. We reached our main goal for this year which was the creation of a documented environment which is maintainable by one administrator.
Although we did this in our free-time without any payment it was a great year with a lot of experience which pays off now. -
I already wrote a rant about this yesterday, but since I'm a sysadmin trying to convert to dev.. I dunno, maybe it's not a bad idea to muddy the waters a bit and talk about why not to be a sysadmin.
Personally I think it's that the perceived barrier to entry is just too high, while it isn't. You don't need a huge Ceph cluster and massive servers when you're just starting out. Why overbuild an appliance like that if it's gonna start out at maybe 5 requests a minute?
Let's take an example - DNS servers! So there's been this guy on the bind-users mailing list asking how to set up a DNS server on 2 public servers, along with a website. Nothing special I guess - you can read the thread here: https://0x0.st/ZY-d. Aside from the question being quite confusing, there was advice to read RFC's, get a book, read the BIND ARM, etc etc. And the person to deny this? No one less than Stephane Bortzmeyer, one of the people who works for nic.fr (so he maintains the .fr TLD) and wrote some of those RFC's as part of the DNSOP working group in the IETF. As for valid reasons to set up a DNS server? Could just be to learn how the DNS works, or hell even for fun. As far as professional DNS servers go.. this (https://0x0.st/ZYo9) is the nugget that powers the K root server, one of the 13 root servers that power the root zone of the internet, aka the zone apex. 2 RJ45 connections, and a console connection. The reason why this is possible is the massive recursor networks that ISP's, Google DNS, Cloudflare DNS, Quad9, etc etc provide. Point is, you don't need huge infrastructure to run a server!
Or maybe your business needs email. How many thousands of emails per second are you gonna need to build your mail server against? How many millions will you need to store? If your business has 10 employees and all of those manage about 10k emails total.. well that's easy, 100k emails total. Per second? Hundreds of emails per second per employee? Haha, of course not. Maybe you'll see an email a minute at most. That is not to say that all email services are like this - it is true that ISP's who offer email to their customers, and especially providers like Microsoft and Google do need massive mail servers that can handle thousands of emails per second. But you are not Microsoft or Google. So yeah, focus on the parts of email that are actually hard.. and there is plenty.
Among sysadmins you have this distinction between "professional" sysadmins and homelabbers. I don't mind the distinction itself but I think both augment each other. If you've started out by jumping into a heap of legacy at an established company, you will have plenty of resources, immediately high complexity, and probably a clusterfuck right away. But you will have massive amounts of resources. If you start out with a homelab, you will have not many resources, small workloads, and something completely new for you to build and learn with. And when running a server like that, you'll probably find that the resources required are quite small, to provide you with your new services. My DHCP servers take 12MB memory each. My DNS servers hover around the 40MB mark. The mail server.. to be fair that one consumes around 150. But if you'd hear the people saying that you need huge servers.. omg you need at least a TB of RAM on your server and 72 cores, massive disks and Ceph!1!
No you don't. All that does is scaring people away and creating a toxic environment for everyone. Stop it.1 -
>Be a customer
>Ask to reduce number of servers you rent at our company
>Agree to a date when the one server you no longer needs will be due to be disconnected and taken apart
>Date comes
>I have the honors of sending the final /sbin/poweroff
>All goes neatly... until...
The web that used to be there, now moved to another one of their machines, goes down, wtf???
Oh. a 500. What?
Checks logs...
Cannot connect to Database.
Wtf? Local database works... Oh. OH. OH MY GOD.
>Turn the server back on and tell the customer to fix the app to no longer connect to that machine
Sometimes, being a sysadmin can be a real fun!1 -
>Discovers a new low level profiling tool that could help us at work with stuck process debugging and gets all hyped
>Installs on test machine, tool doesn't work
>Wonders why. Oh. Needs a kernel module to work, compiled and loaded
>"Well, its my test machine... Guess that's no problem..." but... my hype died down a bit. Kernel module installation just for a new tool that aggregates all other commonly used tools? eh... Maybe it will blow me out of my shoes still
>Installs and loads the module
>Tool works. Turns out its just a htop-like tool, with shortcuts to launch specific other profiling tools like strace/ltrace/lsof/netstat/ss etc...
"Oh... That's boring. Maybe it has all those tools built in at least?"
>Tries to run ltrace - tool exits as ltrace is not installed
Lol
>Installs ltrace and launches tool again. Tries to ltrace a process and
>Nothing. Nothing happens. For seconds... Then kicks me off of SSH
WTF?
>Tries to ping machine... silence
Did... our net go down again? (Having issues due to a storm going over our area these few days)
>Pings google and... gets instant reply
More wtf
>Pings the hypervisor the machine was running on
Works like normal
Oh... Oh no. Please tell me it didn't!
>Logs into the hypervisor UI, checks machine state
Running OK
>Opens machine console aaaaand... Yep. Stacktrace as well as a lot of kernel mumbo-jumbo... It took the machine down to kernel panic.
I never went so quick from "We need this tool deployed everywhere" to "Omg I need to get rid of this crap as soon as possible" lol.
And just for those wondering, it was sysdig.1 -
Fuck you sysadmin, what makes you think that running a complete antivirus scan everyday at fucking 11am is a good idea?
What the fuck can I do with the fucking 5% CPU available? Should I play fucking Pong in an emulator? Write the story of my life with notepad?
Go hang yourself on an Ethernet câble in your fucking server room, you fucktard cunt.12 -
Thank you devRant for introducing me to Pi-Hole and F-Droid. My life being my family's "sysadmin" has become so much easier after those two discovery.1
-
This is a rant thats been waiting a long time to be said...
About half a year ago I got a refurbished laptop, and decided to run manjaro on it (primarily because I didn't have the time to setup arch). I spent time configuring it, I tried out different things, and all in all I learnt tons about linux, and just random things about computers in general.
I dont regret this in the slightest!
Despite the many times where something went horribly wrong, like after I moved over to efi (without a hitch, actually!) I forgot to add to fstab my esp and f-ed the whole boot system. Or when, right in the beginning of this adventure, I tried to move over Xorg to my nvidia gpu and left optimus on. Big Mistake! But I learnt, and I came out a better sysadmin, a better dev than when I first went in.
And again, I dont regret it in the slightest!2 -
Sysadmin gives me 32GB RAM for my workstation. Fucking Windows decides to create a 30 GB pagefile just in case the 32GB RAM are not enough. So my systems SSD is getting peppered with rubbish. Thank you Microsoft...4
-
That moment when even your Sysadmin teacher asks you questions about ssh config during a lab ... Ah did I forgot to say that all the other students were also asking me questions ?
Sometimes uni is tiring ...4 -
Just earlier today I was looking at the hosting packages for a local hosting provider in my country (who shall remain unnamed as I want to work there and criticizing them might not be a very good idea right now) and they start at €250/month apparently. I thought - that's fucking ridiculous!
Like for real, I could literally buy a server for.. I dunno, €600 from the likes of bargainhardware.co.uk with some pretty darn good specs, put it in my home, get a business contract with my ISP for say around €100/month (and use it for my own purposes as well instead of my consumer contract, win-win!), and the server would pay for itself in no more than half a year, probably even less! And you're even getting the actual hardware with it!! And that is for the price of that hosting provider's starting option!!!
Now I know what you're thinking, sure there's more to servers than just the server itself, like redundant power, generators, SLA, multiple routers and switches, and all sorts of failover measures. And you are absolutely right. But does that really justify a rental cost of a server of €250/month?
Not only that, even their shared hosting.. shared hosting, the dreaded, shitty shared hosting! solution is starting at around €10/month. I'm paying about €5/month for 3 light-duty servers and a domain for Christ's sake!
So.. is this hosting provider just expensive as fuck or is this really the industry standard, particularly for the dedicated hosting part? And maybe that's why some services like.. say devRant which apparently gets around €600/month from 299 supporters at the time of writing, yet still has @dfox and @trogus pay from their own wallets for it (if at all possible, please let me know if that's still the case).. I wonder if those costs are all really justifiable?
It just strikes me as odd.. you can get *a lot* of server for a couple hundred bucks if you do it well.. no?21 -
Every time I appy for an IT job I can't stop to think about the time I accidentally wrote my root password to a discord group chat instead of the terminal window next to it.9
-
Am I the only one who hates when I enter a simple question like "PHP memory limit" and the first link *isn't* the official PHP documentation? Who gives a flip about some fancy third-party webpage where they write a whole flippin' article about a simple directive?
Ugh... The priority Google...6 -
I thought the “works on my machine” from sysadmins is mostly a joke...
But no. I’m attending the Polish high school and I have a lessons via the Internet. I wasn’t able to hear the voice of the teacher on Linux 4.18 and Chromium browser, so I sent the mail to support to report it. Of course it “works on their machines”...6 -
Got handed a CentOS 7 cluster, previous admin made kernel command line changes in grub.cfg instead of default/grub.
Ah, thank you. -
"Our supplier asks that you double the number of php child processes for this fpm pool"
"Are you aware, that that would lead to about 100% of memory overcommit, taken the current limit of 128MB/child, and that if a lot of them started at once, the system would probably go for OOM-Kill, which would most probably kill your database, that still runs on 100% MyISAM tables that do not support transactions, and you'd have to kiss your data integrity goodbye, right?"
"Uh... Nevermind then"
I get that some people are not IT-versed, but really... Hire someone who knows what they are doing and doesn't live 20 years in the past, god damn it! -
Can a sysadmin start Node web design?
I'm a Linux automation admin, and I always look at my friends developing nodes websites with poor UI and UX. I'd love to fix that but have no idea where to start from.
Any idea or git project / advice on where to start from?
Cheers!
~ exit8 -
Wtf does DevOps engineer even mean. I checked job listings it mostly feels like a sysadmin with a bit of automation.4
-
https://reddit.com/r/sysadmin/...
"How to make $17k in 10 hours for a 5 minutes job"
or
"Live physical server migration to another building"
A nice rant :)
Some folks in my prev workplace tried to move a live SUN machine to a different hall and yet ended up with messed up HDDs (which ofc can only be replaced and rebuilt by SUN, since it's UNIX). Including the system RAID :)
Hats off to Matt!4 -
Holy shit! so after my last news report https://devrant.com/rants/1063342/... , I also shared the news in my site where there is a guy that works in a Washington ISP and also offer mail servers and such for the locals... fun and joke time has ended my friends, any SysAdmins here regarding this want to comment on this topic?
9 -
"Hello,
it turns out we probably caused the downtime ourselves. I didn't know dropping 170 databases and deleting 80 entire projects at once could do that"
Gave me a hearty chuckle. Especially as the client adamantly refused to have SSDs installed for the DB to run on top.
Just imagining the poor read-write heads jerking back and forth in vain attempts to find all the data to delete... So yes, dropping 170 databases at once does in fact take a database server down to its knees, as deleting is all the drives will be doing for a while.
At least it wasn't my or my colleague's mistake this time.6 -
Just noticed that, at my school, the "sysadmin" forgot to put the password on one of the switches. That means I'm gonna have some fun, nothing that will get me in trouble, I'm just gonna mess up with the config a little bit so that he understands he fucked up.
I'm pretty surprised that none of the students discovered this, but I mean, none likes IT at my school. And guess who likes it but didn't take that "course"? Me.
In Italy, we don't have courses, you decide what you gonna do for the next 5 years, and changing isn't very easy. All and all I'm happy with what I chose, I'll have a better resume than them.7 -
>Be a dedicated server owner
>Fuck up and have an issue you do not know how to solve
>Ask us for help
Huh... Okay, fine
>Machine has apt repos from 4 different system releases
>Nope.elf
Some folk should never decide to admin a machine on their own :|6 -
Hello,
Wondering if anyone can give me some advice regarding stress management.
I am a sys admin of a continually amount of growing servers (now at over 130) and I do coding when I am not busy being screamed at by users. The stress is coming from the workload, but also the way that the workplace is running. The manager left, and now I am handling all his shit, and my own shit as well, and all his accounts have been handed over to me (accounts being clients here). The other IT guy who is supposed to help out with the server admin just finds other work to occupy himself, and I am losing my mind. There is literally an insurmountable amount of work that needs to be done, and it just cannot be done in the time that is allocated in the working hours. I am working overtime, unpaid overtime by the way, until 9/10PM at night to try and get through everything (*cannot apply updates and work on the app server while the users are live) and I am just starting to lose grip. I am taking my stress home with me (not taking it out on anyone), but I am not sleeping, not eating properly and even starting to dream about possible ideas to fault resolution when I sleep. I find that I am constantly tired, and it feels like a world is about to cave in on me. There is literally too much work to be done in too little time, and although I am more than capable of doing it (and will get it done, or the director will physically assualt me and accuse me of being useless, again) I feel that the struggle is just a bit too much.
Can anyone give me some advice on how to "wind down" or to "let go" just for a few minutes a day at least, so that I don't feel like I am on the job 24/7.
Thanks.4 -
For those of you who are riding the crypto currency train I have a question. I'm looking to colocate a few antminer s9s and I'm unsure how to verify that the cooling will be adequate.
What is the best way to verify this?5 -
Sysadmin and an ex-employee couldn't fix an issue with an application for many months even with vendor's instructions.
Today the job is passed to me and I follow instructions exactly and resolve the issue.
The other two guys must have thought 'we don't need someone else's documentation, we can fix it ourselves'
This is not the first time something like this has happened. I guess some things just need a fresh perspective. -
Ok so riddle me this. The service for an application were required to run to send clients insurance through (as per government regulations) was working fine all day working super fast. Rare but awesome. I get a call one hour prior to the office closing (I don't work weekdays) and I am told that all of a sudden insurance isn't sending.
My mind goes right to this fu**ing process. Sure enough it's stopped on the server. Well shit ok. I click start..... Nothing. I kill it from task manager.... Nothing. "SERVICE CAN'T START"
I'm like ok that's fine let's check event logs.... Nothing. No problem let's just run it not in a service container and see if there's an error. NOPE IT DOESNT LET ME.
Okok so that's cool let's just try reinstalling the app. NOPE CAN'T DO THAT WITHOUT RESTARTING THE WHOLE FUCKING SERVER WHICH BRINGS THE ENTIRE OFFICES MANAGEMENT SYSTEM OFFLINE BECAUSE THIS FUCKING APP NEEDS TO BE ON THE SAME GODDAMN SERVER.2 -
Just learned about the Linux watch command today. `watch -n 1 ls -l` OMG, where have you been all my life?5
-
TLDR; sometimes I want to murder my friends.
Pratten: Hey Ethan can you image the robotics programming laptops?
Me: Yeah sure no problem. Let me just make a custom windows iso with all the software we need so I don't have to deal with installers after the fact.
Pratten: Ok great!
Me: *makes custom ISO compiles it and puts it on usbs*
Pratten: hey could you also add drivers station?
Me: uggggg... *Recreates iso and preps bootable flash drives*
Me: IS THERE ANYTHING ELSE YOU NEED?
Pratten: nope that should do it ;;;)
Me: ok great. *flashes laptops and runs install. (they're old so it takes a while)
Pratten: ok good job thanks. Did you install *NOT PREVIOUSLY MENTIONED TOOL SUITE 1* or *NOT PREVIOUSLY MENTIONED NEWER TOLL CHAIN THAT ONLY HE KNOWS HOW TO GET* ? If not I'll have you install those later.
Me: *suicides*8 -
That moment when something goes wrong in the server for the first time at 3AM, your last manual backup was 5 hours ago and you stay up until 7AM to rollback and also setup incremental backups every 15 minutes... FML xD15
-
I just interviewed the guy who is interested in my Linux SysAdmin position. He's really cool and I think he'll get the job, but he is a Windows admin actually. (No problem, since he'll work with me for three month's as a mentoring program and it's supposed for him to learn the stuff)
My question to all other Linux Witchers and Witches out there, do you know a mighty spell to seal his windows daemon away, such that he's able to resist the sore temptation by my co-workers (windows enthusiasts)?2 -
The feeling of dread as still a semi-junior sysadmin when an app doesn't work after an update.
I got stressed, triple-checked everything that I changed and that I followed all steps of the documentation of the upgrade process, then, as a last step before going over in half-panic mode to my boss, I try to restart the stupid java app and it starts working.
Wtf. Why. Why didn't it work the first time I started it? D: -
Awesome tool imapsync, now moving a 47gb account without problem, what other tool do you use to migrate emails account without to much hassle?, +1 lml
1 -
IDK man, it took me a while to finally learn iptables and now switch to firewalld? Oh come on. It's not that I'm against learning new things, no. It's just that firewalld looks a bit.. crappy. If I get a server provisioned and run
firewall-cmd --add-port=53/udp --permanent
firewall-cmd --reload
and I get my ssh connection killed that's no good news, no sir! I mean come on, how can I rely on a tool this critical when a single line in its config file can make my machine inaccessible. Even better -- this config file is managed by that tool entirely!!! My commands passed all the tool's checks and they worked, but when I wanted to make those commands permanent and reload state from the config -- the tool starts spitting bile and blood and says "fuck off, it's my server now!"
IDK man.. It's just way too fishy. The good ol' iptables works very well and I'm kicking its retard younger brother out of the server.
shoosh you dirty pig firewalld, shoosh!6 -
#! Linux 4.1 is out
Insanely great but I'm only worried about several massive patch sessions
https://theregister.co.uk/2017/02/... -
Best part of being a dev
When your sysadmin father blocks all social websites and installs keyloggers but you set up linux instead6 -
This happened in the Slack group:
Me: "fu"
Me: "oops sorry i accidentally typed that while I was trying to type 'fdisk -l' on another computer and used the wrong keyboard" -
I remember the day when I mistakenly hit :
# rm -rf /
instead of :
# rm -rf ./
The . changed my day that day. Thank god that the files and configs in the server had a backup in my PC. :P3 -
Do I need ccna for pentest?
I know it's about network administration but I was wondering if it will help me see things the way the sysadmin does.3 -
To the SysAdmins and Linux-Engineers here,
How you got into your job?
I'm a Linux-Fanboy by heart and love tinkering and fiddling with systems, so I want to work in the field?
Please tell me, how you got into your job, and how it is to work in the field.7 -
Since everyone rants about interviews, I think that's the perfect time to ask this.
In a week, I'm going to have my first interview.
Its for an apprenticeship as "Fachinformatiker für Systemintegration".
For the non-german speakers, pretty much sysadmin,server-engineer, sometimes internal tech-support.
The company isn't a tech-company, but a logistics-company.
The interview will be done by the boss of the company's location and the lead of the IT-department.
So, what should I expect, what questions and such ?3 -
Work office, using xencenter for viewing xenserver server, tailing some logs and manjaro (arch linux) for bash stuff, and a coffee for good habits hahaha
4 -
That day when you need to restart your vsphere virtualizer after 2 and 1/2 years of uptime, just to find it won't boot in any possible way, and you rush to install it fresh on an USB drive 20 mins before your sla runs out... that day was today, fml!
-
Right now I enjoyed being SysAdmin and PenTester more than programmng...but I doubt I can get a job without certification from either of those field.6
-
Watch your shell. Someone did it again.
Sysadmin grilled s3 with a typo in his command, shutting down whole subsystems of amazons infrastructure2 -
junior developer raises an issue saying that there's an application deployment error on one of their dev clusters.
sysadmin asks them to go back and look at the error logs and come back with the problem.
they come back saying, "No space left on device"
sysadmin takes a look at server. finds this :
5 -
I initially chose System Administration simply because it was attractive to me to be the HMFIC, and generally above the law as corporate policy is concerned, as said law for the most part applied to people with less comprehensive knowledge about how any given system or technology works.
Since then though, I've learned that there's basically no better way to become a jack of all trades than being a sysadmin. There's no other position in the tech field that more easily and gracefully parlays into other specialties.
I write automation and aggregation software now, but I still consider myself a sysadmin by trade, as automation is just another function of system administration. I write everything in vim, and almost entirely in perl, because I am concerned above most other concerns about performance. I could learn C or Go or Rust or some other low-level compiled language, and I'm sure I could create even more performant software that way, but that would take me farther away from my passion: System Administration. -
Today the connection is so slow. What if I call directly to the server sysadmin to send the request by phone dictating every single bit? I think i could waste less time.
-
Having a shit of a time trying to figure out why Docker containers are not accessing other containers via domain names as they should technically be going through the jwilder nginx proxy container.
Why can't environment setups ever be simple? -
Just came across a job posting on Linkedin, which basically expects the applicant to be a sysadmin, front end and back end developer at the same time.
Almost contacted the job poster just to send a WTF.3 -
Finally got that damn web app to send out mails (2am). Turned out mail server worked, rails was properly configured, delayed jobs were running and were getting proper rights and environment. The issue was wrong configuration in app itself (somebody skipped part of the wizard). But still, fixing somebody's else server with webapp I know just a little about in languages I know even less about (not a web developer) after few guys failed and just within five hours, makes me feel both dumb (should have noticed much sooner) and proud (figured it out in the end).
-
Working in IT fucking sucks.
Why do people willingly do this to themselves is beyond my understanding.2 -
So, I work as a sysadmin junior (6 months and going), and in the past few months, I learned what my boss warned me about - Devs don't understand us admins, and we don't understand the devs.
We have this huge client who is about to migrate to our company (We do mostly server managment/Housing/Renting), and I am so gald I don't have to work on the migration myself!
Just hearing what the company devs say makes me facepalm: No, it won't work. It cannot work on just 3 machines (They use like... 20 in total), no, we won't get rid of our docker swarm, that's essential (Doing the absolute minimum in their infrastructure, just a fancy buzzword to lure people on. Though they've spent like 2 years developing the app that uses it, so they my not want to give it up).
I kid you not, once, they replied to an email that contained the phrase "To be afraid of/worried about" something during the migration, that something could break, not work, be unstable. 7 times.
Might not sound as bad, but it was a rather short mail, and when they're so afraid of everything, its kinda hard to cooperate with them.
My colleague literally spent this entire week mapping out /their/ infrastructure, because they were unable to provide us with the description themselves.
And as a cherry on top, they sent us a "graph" of relationships of all the parts of their infrastructure that was this jumbled mess of rectangles and arrows. Oh, and half of all the machines were not even in the graph at all! Stating that "We also have all this, but I really don't know how to ilustracte the interactions anymore"
Why do companies like that exist? If you build an infrastructure yourself, shouldn't at least someone know exactly how it works?1 -
Probably the weirdest single command I have ever entered so far:
apt-get install postgresql-12 postgresql-11 postgresql-10 postgresql-9.6
In other words - testing an internal tool across all of our supported postgres versions, but... Just found it funny in a way... Dunno, maybe my humor is just weird.5 -
So, I gave my missus an old 512 MB Ram AMD 64 Bit Computer with Old School Windows XP, tried installing Atom and this happened.
Time to install Linux. :D
3 -
Happy system administrator appreciation day! No, really. This is a thing.
http://sysadminday.com/
https://nationaldaycalendar.com/201... -
That feeling when the cpanel is locked from a startup you don't work at anymore, the sysadmin ignores your emails, and you are STILL CC'd on customer inquiry emails from their domain. 🙄1
-
The end of today was extremely fun.
Imagine the surprise. I was importing a simple 8 GB big virtual machine into the Proxmox hypervizor.
First issue: It was in the Open Virtualization Format (.ova) for easy import into... most hypervizors... Not Proxmox, however.
But really, not that bad, there are ways around it. Create a blank virtual machine through the UI, scrap the disk you create, then extract the two disk QCOW2 files from the .ova file, which by itself is just a POSIX TAR archive. Then import them through the commandline.
...So I did just that. The larger of the two was about 8 GBs, the other just like... 50 MBs.
The larger imported fine. The smaller?
Color me surprised, when it created a FUCKING. 1. TB. LOGICAL. VOLUME.
...
That it then proceeded to try and fill full of zeros...
Oh yes, it was one of the fancy dynamic storage files that expand as space is needed.
...
Tomorrow, I'll have to try if I can export just the filesystem data into an individual, shrunken down, normal, plain, old disk. None of this fancy black magic shit.
...Also... I don't get why Proxmox doesn't support that... The filesystem was only a few megs big... Ugh.1 -
Next week I'm beginning a paid intership in an sysadmin/infrastructure manager/bit of devops position. My tutor already told me he would give me things to learn alone so we could work together on stuff, and I can't wait for it to begin.
However, in the meantime I don't have a lot of things to do, so I would like to put this downtime to use and start reading stuff.
I already know I'll be doing a lot of Linux (that, I already master pretty well), and also some Active Directory, Kubernetes, and a bit of DevOps. Those are the main keywords he throwed at me during the interview.
What subject would you advice me to start learning in advance ? Do you have nice resources/books/videos on those matters ?
I would have asked to my tutor but right now he's on holidays and I don't intend to piss him off with job related questions.
On a side note : do you have any good and complete documentation or learning resource about SELinux ? I've had issues with it on my main rig for some months and can't find any good answer so I decided to learn it as best as I can and come up with an answer on my own. Since I intend to work in the field, I should what there's to know about it anyway.6 -
What’s up with HR calling to do technical interview and asking questions she doesn’t even know the answers to? Bruh, all that time I thought I was speaking with the Hiring Manager only to find out she’s HR when I asked her ONE technical question then she goes..”Oh, I won’t be able to answer that. I’m not technical in this role, I’m just the HR but I can schedule an onsite interview with the hiring manager.”
Me: I believe it’ll be beneficial to have a phone conversation or interview with the hiring manager before deciding if it’s worth coming onsite for an in-person interview.
HR: Ok, I’ll see his availability.
I’m not even concerned if she calls back or not. Plus the rate she’s talking about is really disrespectful.2 -
I'm a fool.
Trying to delete local version of domain account:
Supposed to use command:
net user [username] /delete
Tried:
net user "domain\user" /delete
Didn't work, came up with help which said an option was net user [/delete] [/domain]
So I decided to try:
net user "user" /delete /domain
... "The request will be processed at a domain controller for domain domain.local.
The command completed successfully."
Well FUCK
So now the user's account has been deleted on AD, trying to restore it but AD management tools aren't picking up AD's object so I can't find the tombstone.
SHIITTTTTTTTT :((
TL;DR: I've fucked a user's account and can't find what I need to fix it.
Moral: Don't be a fool like me.6 -
So just a normal rant here. .. it was one of those moments you find in yourself in sometimes. You get so caught up in thinking you know everything that you can't implement occams razor into your everyday work routine anymore. You've worked with so many complex workarounds that when you are faced with a simple problem with a simple answer you can't see the blinking neon light shouting at you anymore , and you can't here the bells sound anymore. ..
My rant is about Me vs the infamous mikrotik router. Something I had to set up. Something I had to login to setup. Something I've done so many times before but this time , my inflated ego and overbearing sense of grandeur just could not figure out.
Class how do we login into a router? Well find your gateway and type that sucker into a browser and you will be on your way ... well that's the answer right there. But since I thought that my router was connected to three dummy switches that it would affect anything or the paranoia I had that my isp somehow disabled any connections to the router at all or that I and to open a new port to connect to it or use winbox to connect to it using only the mac address or ssh into it ..would work ...I didn't try using the tried and tested way of doing it.
I wanted it to be an adventure. I wanted it to be a problem to solve so I shoved the ordinary answer out of the way and used other methods to try and connect to it...
All I had to do was used Nmap to scan the gateway for open ports and realise to view it in the Browser on port 8080 instead and finish my journey ...
I was looking for a dragon to slay , a maze to conquer, glory at the end of my mission ... when all I felt was a sheer sense of idiocy.
--Rant Completed-- -
so late hours, and after a 2h MySQL server transference that should take no more than 15min...
looking in to PSR-0/4 PHP code standards
I'm gonna print them all and smack all coworkers with it in the morning! -
An old sysadmin configured the idrac interface of a company VMware server to get the IP address trough DHCP. Of course, without a valid IP, the server doesn't start. For you, where was hosted the VM with the DHCP server?1
-
Hey devs or sysadmin here in devRant I wanna know what hypervisor are you using in production or dev environment??
I will annex the hypervisor that I know and I work on, but are free to add more.
Vote with a "++" in the hypervisors that you use.9 -
Sometimes the smallest things bring joy:
I am currently equipping some old company laptops with SSDs, normally I use a boot CD to do the cloning, booting from CD takes ages. Yesterday I figured I might aswell etch the image of the backup software on one of the SSDs, turns out once booted I can just use that one as target for the process, so now I have a bunch of one time use SSDs, feels pretty cool for no particular reason. -
Software has no pre-built packages. Clones repo and tries to compile from source. Spends 1.5 hrs hunting for the libraries - no list published. Configure of course had trouble finding one I had installed; had to debug the configure file to see how it was search for it, turns out it was applying a subdirectory to whatever path I gave it. FINALLY configures and I run "make all". Everything compiles!!! Try to follow documtation to setup the software, 1st cli command -> Segmentation Fault with no logs....
-
Today was a painful day when I realized that I need to backup my nginx configs like I backup my actual data. 20 minutes of downtime turned into an afternoon when I accidentally deleted the nginx config backups on my server. It's been... let's say fun.4
-
AHHHHHHHHHHGGGH
I HATE VPN SETUP
- Trying OpenSwan
Installing open swan on a Debian machine.. setting up the config.
Restarting openswan. Syntax error. No syntax error to be found.
Different tutorial.. it starts! Try to connect.. I can’t connect. Look at the logs. No errors.
Tcpdump. My traffic is coming through.. all fine.. try to connect again.. it works! (Nothing changed!)
Try to ping somewhere else.. no connectivity.
Try to ping an IP in the same network.. works fine. So I have connectivity, just no internet.
Spend an hour finding out about traffic directions of which no one seems to know what they really mean.
Boss tells me to stop using openswan because it’s deprecated and replaced by strong swan..
- Strongswan
Reinstall Debian machine, install strongswan. Copy openswan config. Oh, they’re incompatible? Look up strong swan config, and the service starts.
Connect to the VPN.. it works! Again, no internet, just connectivity in the same network. Spend 2h debugging the config, disable firewalls everywhere, find an ancient bug in the Debian package related to my issues.. ok, let’s try compiling from source.. you know what, let’s not. I’ll throw this Debian machine away and try something completely different.
- pfSense
Ok, this looks easy enough! Let’s just click through the initial setup, change some firewall rules, create an L2TP VPN with a simple wizard.
Try to connect to VPN. First, it times out. Maybe a firewall issue? Turn off firewall.. ah, something happens now. I get an error message right after trying to connect to the VPN. Hmm, the port doesn’t even get opened when I enable the firewall.. this implementation seems a bit buggy.. let’s try their OpenVPN module.
Configure OpenVPN. Documentation isn’t that clear.. apparently a client isn’t actually a client but a user is a client.. ok, there’s a hidden checkbox somewhere.
Now where do I download my certificate? Oh, I need a plug-in for that.. ok, interesting. Able to download the certificate, import it, connect and.. YES!!! I can ping! But, I have no DNS..
Apparently, ICMP isn’t getting filtered but all outbound ports are.. yet the firewall is completely disabled. Maybe I need outbound NAT? Oh. There’s no clear documentation on where to configure it. Find some ancient doc, set it up, still no outbound connectivity.
AHAHAHAHHHHHHHHHHG
Then I tried VyOS. I had a great L2TP VPN working in less than 15 mins. Thank you VyOS for actually providing proper docs and proper software.3 -
As SysAdmin I did set up a server with WordPress, so that the external webdesigner create the templates and stuff and everything.
And they can't change the language of the backend.
Well. now it seems I am a WordPress Admin? 😑3 -
Fucking AWS Elastic Beanstalk took a week of work to get fucking mounts set up. They invented their own version of docker compose that is missing half the features so I had to work my way from hacking their scripts to include options they don't support, to restarting the whole fucking docker service on every fucking deployment and now the shit finally works. How can most StackOverflow answers just say restart docker, this shit is not ok! I fucking hate sysadmin work. I want to code :(
-
I run two servers, one that runs WordPress multisite and one that runs a vpn. As a self taught sysadmin I learn best through projects. I’m also interested in databases and backends for iOS (swift) apps. Do you have any suggestions for what I should make my servers do next? Thank you.2
-
Our sysadmin is leaving in a few weeks and I'm sure he's going to leave us in the shit.
He's not the type who would format all the drives on the server on the last day, he's more subtle, like resetting our passwords or leave a backdoor into the system so he can keep an eye on us.
Unfortunately our line manager doesn't agree with me1 -
The day when you Plan an Alarm clock which will awake you when one of your server have trouble.
Hm I think that should look like this:
https://youtu.be/umtf9bx8cuo2 -
I have done some experiments on my server in the past. It's a great way to learn new things. However, I am bound to make some mistakes and over time the sever becomes messier and messier.
A week ago I installed UNRAID on my machine and I love it! I can now have my critical infrastructure live and working in docker containers and vms.
Then if I want to do an experiment I spin up a VM in a couple of minutes to do my thing and remove it when I am done. No traces left! -
That lovely moment, when I have to spend an hour on mail delivery issue, only to find out the message was flagged as spam due to a faulty dns blacklist.
Though the way it got flagged is idiotic and funny at the same time, too.
The blacklist domain got parked, dunno why, and of course, all of the dns queries thus got redirected to a different dns server that just returned the A records of the dns park owners.
Guess what that causes when you use that blacklist? Every single email gets flagged, including that one of ours that I had to debug.
Fml, an hour of overtime for a stupid malfunctioning blacklist... -
Any former sysadmins here?
I am one for the past 4 years but I had enough. I really want to switch to the developer world. Currently self-learning in my free time. I already know java, python and javascript.
Any advices?6 -
We have 2 sites. I, dev, set the https system for one in 10 minutes. The sys admin has been working on it for two hours already and he doesn't want help2
-
Gitlab as a product is awesome, the real wtf is the processes (manual, automated or otherwise) and people supporting their cloud offering.3
-
It's those days when you get to spend 10 hours moving your servers from one room to another, that make you think "why didn't I choose to become a podologist"...
-
I had this dell server lying around and finally got to make a virtualisation server out of it.
It is now running xcp-ng as its hypervisor, with a CoreOS VM in it, containing a docker container serving xen-orchestra for managing the server.
Enterprise grade hardware really is a thing of its own. Also sysadmin type of stuff is quite fun. I look forward to be playing with it some more. :)1 -
I'll fucking kill you bitch who the fuck made you the system administrator of this fucking college? The fuckery you do... god damn some students know more than you and you should agree to this fact. The previous sysadmin was knowledgeable and you are just a fucking worker in the finance department. You fucking dare not call yourself a sysadmin when you can't even read the fucking docs provided by the G Suite. At least you didn't make the mail server yourself otherwise god knows what the shit you'd have spread around.
FUCK YOU12 -
When you're getting to get into programming but you can't find a good project fit your skill level3
-
So... I got a simple task of choosing the best fitting NIDS/MIDS, as well as deploying it, configuring to fit a specific use case and monitor its outputs for one client at work today...
I'm a little... Anxious. At a first glance, setting up like... Snort... Doesn't seem all that difficult, but I have no idea where this takes me and if what I come up with will ultimately be useful or not... Until now I did simple service configuration changes like apache, nginx, php... And a bit of database management with things like mariadb, mysql, postgresql, mongo or elastic... I feel so... Out of my usual waters.
Do you guys thing a person without a title in network security (or... Any title for that matter) can even manage this?...1 -
Bribing sysadmin with an ice cold Coke hoping he will deploy my latest tag sort of in a hurry since I fudged up last one :/1
-
When the SysAdmin recommends to rewrite the whole Java (Spring) web app in Clojure. I mean, let's each of us do our assigned jobs.
-
So basically a client's website still works on PHP 5.4. I manage the site's content which works on an old server that takes around 5-6 seconds each time I work on the backend (WordPress).
Asked the client's sysadmin to upgrade but he doesn't want in case some old and non updated plugin would break the site. (Which did about a year ago and I had to fix it).
Feels like working on a minefileld. -
Not my fav because I'm not proficient in anything and don't consider myself a programmer. But, I use many languages day to day as a sysadmin.
I come across Perl enough to know I don't like how their modules and dependencies work. I have the most difficulty when dealing with this. PHP, Python, Ruby, and GoLang never give me as much trouble as Perl.
Also, coming across more Python3 dependencies, dealing with older Python2 environments, as stated by many others as well, is becoming more and more painful.
Maybe all of this can be solved with some unifying virtualenv for all popular languages/environments, supported fully by the underlying OS. -
It is currently 5:30 AM, I've been trying to upgrade a server by one single major OS release for the last 3.5 hours. All major apps were no issue, except one.
Redmine.
That thing runs on Ruby.
Ruby, as it turns out, doesn't really like the way Debian handles package management.
And now, I have Redmine that, even if I completely uninstall / reinstall, won't start
I went down several rabbit holes, trying to operatively find what the issue is. But I never got to the key issue.
Fuck ruby. Fuck Redmine. Fuck nothing-saying error messages. Fuck bundle. Fuck gem. Fuck it all.
I'm redirecting the Redmine domain onto the server backup I've made. Upgrading that thing is a nightmare.
Maybe now I can finally go to bed...5 -
I wanna know who use xenserver and why??
Currently am using xenserver 6.5 in a production environment and today i start to test xenserver 7.
1 -
GOD DAMN THAT OLD BROKEN DATABASE.
Having to work with a legacy old database system running MySQL 4.0 is a pain. Especially when even finding a frikin manual for the thing is hard af.
And a cherry on top is dealing with encoding and collation in a system, that didn't really have a wide support for it yet.
10/10. At least I am only dealing with it so that we could later shut it down for good.1 -
I'm from Tunisia and I'm looking for opportunities in the USA. One point bothers me. What's the average salary of an entry level DevOps/SysAdmin engineer?!1
-
What is the best alternative to cronjobs, guaranteeing high availability and jobs not being duplicated?6
-
Suggest me one of the best sysadmin course online which is really worthy.(paid as well as free)
I'm Linux user from for years but have always been inclined towards scripts.3 -
Question!
Where do you guys get your daily news about Tech, IT, Dev Related things?
I want something with push notifications and just serious sources without bullshit infos.9 -
Open Question?
Can you describe how is your life /work balance as a IT professional?
I will love to hear some insights about it3 -
I am the only Backend developer in my team, so I have to do sysAdmin tasks, deployment and configurations myself....
I HATE it! -
Am I the only one who's hands start shaking when about to send "CHANGE MASTER TO" on a dev server?
Happened to me yesterday, replication got stuck after corrupting a relay log file when the database segfaulted under my hands.
I could check and recheck the positions I was about to reset it to a bilion times and I was still nervous!
Top Tags