The spam denier
_____

An old phone conversation with a client:

Me : Hello

Client : My website and server are suspended? why is that?

Me : Your server sends spam messages.

Client : We do not send spam messages, we are on vacation, there is none in the office.

Me : Yes, but it is not necessarily you, according to our logs, your server sent spam messages in Chinese and Russian, so someone from Russia or China....etc.

Client : I do not believe you, we do not speak russian or chinese, how could we then write spam messages in those languages?

Me : I told you, maybe someone exploited some vulnerability in your website or server firewall. And if you want to activate your services, please check with your webmaster and sysadmin to secure your ....

Client: I tell you my son, because I am old and I have more life experience than you ... I am 60 years old and I tell you, spam does not exist, and YOU suspended my website and server, and created issues to sell me more of your solutions and services.

I won't check my server, I won't hire a webmaster or a sysadmin, AND YOU WILL ACTIVATE MY SERVER NOW !

(I suddenly realized that I am talking to a wall, so I switched to a robotic tone).

Me : Please resolve the issue to activate your services..

Client : YOU WILL ACTIVATE MY S...

Me : Please resolve the issue to activate your services...

Client : WHAT IS THIS SPAM STORY ANYWAY, I DO NOT BELIEVE YOU ...

Me : Please google that word and you will understand what is spam is...

Client : YOU ARE F**ING LIARS, SPAM DOES NOT EXIST... ACTIVATE MY WEBSITE N.... Beeeep !

I hang up.

Well, I thought about configuring an automatic response for this client, or a for-loop.

His voice was really unpleasant, as if he is a heavy smoker.

Story time:

I was once working on a project that dealt with incredibly sensitive financial data.

We needed a client’s database to do a migration.

They wouldn’t send it over the internet because it was too big and they didn’t think it would be secure.

They opt to send it in the post on an encrypted usb drive.

(Fair enough thinks I)

USB drive arrives.
Is indeed encrypted.

MFW there’s a post it note in the envelope with the password on.

MFW this is a billion dollar multinational petrochem company.

MFW this same company’s ‘sysadmin’ and ‘dba’ once complained because a SQL script I sent them didn’t work - they’d pasted it twice and couldn’t work this out from the fucking “table already created” error message management studio was throwing at them.

And here comes the last part of my story so far.
After deploying the domain, configuring PCs, configuring the server, configuring the switch, installing software, checking that the correct settings have been applied, configuring MS Outlook (don't ask) and giving each and every user a d e t a i l e d tutorial on using the PC like a modern human and not as a Homo Erectus, I had to lock my door, put down my phone and disconnect the ship's announcement system's speaker in my room. The reasons?
- No one could use USB storage media, or any storage media. As per security policy I emailed and told them about.
- No one could use the ship's computers to connect to the internet. Again, as per policy.
- No one had any games on their Windows 10 Pro machines. As per policy.
- Everyone had to use a 10-character password, valid for 3 months, with certain restrictions. As per policy.
For reasons mentioned above, I had to (almost) blackmail the CO to draft an order enforcing those policies in writing (I know it's standard procedure for you, but for the military where I am it was a truly alien experience). Also, because I never trusted the users to actually backup their data locally, I had UrBackup clone their entire home folder, and a scheduled task execute a script storing them to the old online drive. Soon it became apparent why: (for every sysadmin this is routine, but this was my first experience)
- People kept deleting their files, whining to me to restore them
- People kept getting locked out because they kept entering their password WRONG for FIVE times IN a ROW because THEY had FORGOTTEN the CAPS lock KEY on. Had to enter three or four times during weekend for that.
- People kept whining about the no-USB policy, despite offering e-mail and shared folders.
The final straw was the updates. The CO insisted that I set the updates to manual because some PCs must not restart on their own. The problem is, some users barely ever checked. One particular user, when I asked him to check and do the updates, claimed he did that yesterday. Meanwhile, on the WSUS console: PC inactive for over 90 days.
I blocked the ship's phone when I got reassigned.
Phiew, finally I got all those off my chest! Thanks, guys. All of the rants so far remind me of one quote from Dave Barry:

Navy story time again. Grab that coffee and fire up Kali, the theme is security.
So, when I got promoted to Lieutenant Jr. I had to attend a 1-year school inside my nostalgic Naval Academy... BUT! I was wiser, I was older... and I was bored. Like, really bored. What could go wrong? Well, all my fellow officers were bored too, so they started downloading/streaming/torrenting like crazy, and I had to wait for hours for the Kali updates to download, so...
mdk3 wlan0mon -d
I had this external wifi atheros card with two antennae and kicked all of them off the wifi. Some slightly smarter ones plugged cables on the net, and kept going, enjoying much faster speeds. I had to go to the bathroom, and once I returned they had unplugged the card. That kind of pissed me off, since they also thought it would be funny to hide it, along with the mouse.
But, oh boy, they had no idea what supreme asshole I can be when I am irked.
So, arpspoof it is. Turns out, there were no subnetworks, and the broadcast domain was ALL of the academy. That means I shut EVERYONE off, except me. Hardware was returned in 1 minute with the requested apologies, but fuck it, I kept the whole academy off the net for 6 hours. The sysadmin ran around like crazy, because nothing was working. Not even the servers.
I finally took pity on the guy (he had gotten the duties of sysadmin when the previous sysad died, so think about that) and he almost assaulted me when I told him. As it turned out, the guy never had any training or knowledge on security, so I had to show him a few things, and point him to where he could study about the rest. But still, some selective arp poison on select douchebags was in order...
Needless to say, people were VERY polite to me after that. And the net speed was up again, so I got bored. Again. So I started scanning the net.
To be continued...

Dear Indian Companies,

Why do you hire for a role and then say: "We dont have that role but then we want you to grow up to be a Generalist"!

6 years as a build, release and SCM guy at Moto and Nokia back then, I shifted to this big Indian IT corp coz Nokia was shutting down...

A week into my orientation (which is a crazy weirdness inducing ritual in and of itself), the new manager I'm supposed to be working with comes up and says- "Here's the code repo, there are 2 open jQuery issues, fix them!"

I'm not really sure what to say at this point because jQuery is nice and all but thats not who I am.. I'm the infra / DevOps guy. And this is circa 2012 when DevOps as a term was just hotting up...

Tell me to setup a multi-stage pipeline and automated test cycles, I'll do it drunk, but oh no! bug fixing on a jQuery script? Noooo!!!!! I just dont have the chops for it.

So long story short, I get reported to HR for insubordination - Yeah, Go Figure!

Cue: HR meeting
HR: You wont work?
Me: I cant work on jQuery. I am a sysadmin / devops guy... Give me a project that involves those skills and I'll work.
HR: But we hired you to work on jQuery.
Me: But you did not mention jQuery / UI / UX in the job description - Pulls up email and shows JD for interview which says Symbian, Build, Release, Configuration Management but NOT jQuery.
HR: ....
Me: :-/
HR: But we want you to be a generalist.
Me: #wtf
HR: We want an engineer to be able to do anything he is tasked with!
Me: Can I know my last working date here?

And thats how my career at a glorious IT corporation just went poof!

When I think back on it, I feel good that I chose to do what I wanted to get better at and what I loved working on...

And this is the problem with IT companies in our country - They play with people's aspirations and passions... To the point that all thats left of a software engineer is the looking forward to pay day so he can start the damn cycle all over again.

Navy story continued.
And continuing from the arp poisoning and boredom, I started scanning the network...
So I found plenty of WinXP computers, even some Win2k servers (I shit you not, the year was 201X) I decided to play around with merasploit a bit. I mean, this had to be a secure net, right?
Like hell it was.
Among the select douchebags I arp poisoned was a senior officer that had a VERY high idea for himself, and also believed he was tech-savvy. Now that, is a combination that is the red cloth for assholes like me. But I had to be more careful, as news of the network outage leaked, and rumours of "that guy" went amok, but because the whole sysadmin thing was on the shoulders of one guy, none could track it to me in explicit way. Not that i cared, actually, when I am pissed I act with all the subtleness of an atom bomb on steroids.
So, after some scanning and arp poisoning (changing the source MAC address this time) I said...
"Let's try this common exploit, it supposedly shouldn't work, there have been notifications about it, I've read them." Oh boy, was I in for a treat. 12 meterpreter sessions. FUCKING 12. The academy's online printer had no authentication, so I took the liberty of printing a few pages of ASCII jolly rogers (cute stuff, I know, but I was still in ITSec puberty) and decided to fuck around with the other PCs. One thing I found out is that some professors' PCs had the extreme password of 1234. Serious security, that was. Had I known earlier, I could have skipped a TON of pointless memorising...
Anyway, I was running amok the entire network, the sysad never had a chance on that, and he seemed preoccupied with EVERYTHING ELSE besides monitoring the net, like fixing (replacing) the keyboard for the commander's secretary, so...
BTW, most PCs had antivirus, but SO out of date that I didn't even need to encode the payload or do any other trick. An LDAP server was open, and the hashed admin password was the name of his wife. Go figure.
I looked at a WinXP laptop with a weird name, and fired my trusty ms08_067 on it. Passowrd: "aaw". I seriously thought that Ophcrack was broken, but I confirmed it. WTF? I started looking into the files... nothing too suspicious... wait a min, this guy is supposed to work, why his browser is showing porn?
Looking at the ""Deleted"" files (hah!) I fount a TON of documents with "SECRET" in them. Curious...
Decided to download everything, like the asshole I am, and restart his PC, AND to leave him with another desktop wallpaper and a text message. Thinking that he took the hint, I told the sysadmin about the vulnerable PCs and went to class...
In the middle of the class (I think it was anti-air warfare or anti-submarine warfare) the sysad burst through the door shouting "Stop it, that's the second-in-command's PC!".
Stunned silence. Even the professor (who was an officer). God, that was awkward. So, to make things MORE awkward (like the asshole I am) I burned every document to a DVD and the next day I took the sysad and went to the second-in-command of the academy.
Surprisingly he took the whole thing in quite the easygoing fashion. I half-expected court martial or at least a good yelling, but no. Anyway, after our conversation I cornered the sysad and barraged him with some tons of security holes, needed upgrades and settings etc. I still don't know if he managed to patch everything (I left him a detailed report) because, as I've written before, budget constraints in the military are the stuff of nightmares. Still, after that, oddly, most people wouldn't even talk to me.
God, that was a nice period of my life, not having to pretend to be interested about sports and TV shows. It would be almost like a story from highschool (if our highschool had such things as a network back then - yes, I am old).
Your stories?

When you're a junior sysadmin but still have to maintain ALL the production server:

How it looks:

$ sudo apt-get update

How it feels:

& sudo [ $[ $RANDOM % 6 ] == 0 ] && rm -rf / || echo *Click*

Story time!

So me, alcoholic I am, went to a local cafe because I didn't have enough booze anymore at home. Turns out that there were quite some people that wanted to get to know me!

And most importantly, there was another sysadmin (that likely saw through my funsies with some people there that I displayed termux' apt update and apt upgrade to as "evil hacking" 😜) that actually wasn't a bluff - he pretty much interviewed me on the spot and was apparently pretty impressed by my skillset. And so am I by his - he asked some pretty techy questions that only a fellow technologist could come up with. In a local cafe of all places!!! Who would've thought?!

I'll probably be going to that cafe some more 😋

My last wk93 story, the time we discovered school faculty was spying on students and we uncovered student's deepest secrets.

I call it, kiddiegate.

So if you've read my past rants you've noticed I did some pretty childish and reckless stuff with my highschool's systems when I was younger, but nothing compares to this thing.

After resetting the sysadmin account pwd on some machines it occurred to me I could write a keylogger to capture teachers Moodle accounts and so on, I decided to try it out on a regular lab computer first.

Imagine my surprise when I found a hidden keylogger already installed! I couldn't believe it but then I thought, what if other PC's have it? So I recruited my mates and teached them the process to check if a PC had been infected...ALL PCs were, over 30 computers we checked had been logging for over 3 months! That damn sysadmin! >:[

We were shocked and angry, but then I thought "hey. . . My work has been done for me, better take advantage"

So we did, we extracted each log and then removed it from the PCs along with the keyloggers. There were hundreds of records and then one day we started snooping into the fb accounts of some students (we shouldn't have) we uncovered so many nasty, shocking secrets...

One of the school's lady's man had a drunk one nighter with one of our gay friends, the most secluded and shy guy was sexting like crazy with 15 chicks at the same time, things like that...we promised to never say a word and deleted the logs.

After that we didn't do much and continued highschool as every teenage minor should, getting drunk and avoiding responsibilities, though we could never see many of our classmates the same way. The sysadmin was fired shortly after I graduated, no reason was stablished.

I want to clear out we were minors and laws in my country weren't clearly stablished at the time plus no harm was ever done. I don't condone hacking or any kind of illegal activity, just thought I'd share.

An sysadmin story

Long rant ahead.. so feel free to refill your cup of coffee and have a seat 🙂

It's completely useless. At least in the school I went to, the teachers were worse than useless. It's a bit of an old story that I've told quite a few times already, but I had a dispute with said teachers at some point after which I wasn't able nor willing to fully do the classes anymore.

So, just to set the stage.. le me, die-hard Linux user, and reasonably initiated in networking and security already, to the point that I really only needed half an ear to follow along with the classes, while most of the time I was just working on my own servers to pass the time instead. I noticed that the Moodle website that the school was using to do a big chunk of the course material with, wasn't TLS-secured. So whenever the class begins and everyone logs in to the Moodle website..? Yeah.. it wouldn't be hard for anyone in that class to steal everyone else's credentials, including the teacher's (as they were using the same network).

So I brought it up a few times in the first year, teacher was like "yeah yeah we'll do it at some point". Shortly before summer break I took the security teacher aside after class and mentioned it another time - please please take the opportunity to do it during summer break.

Coming back in September.. nothing happened. Maybe I needed to bring in more evidence that this is a serious issue, so I asked the security teacher: can I make a proper PoC using my machines in my home network to steal the credentials of my own Moodle account and mail a screencast to you as a private disclosure? She said "yeah sure, that's fine".
Pro tip: make the people involved sign a written contract for this!!! It'll cover your ass when they decide to be dicks.. which spoiler alert, these teachers decided they wanted to be.

So I made the PoC, mailed it to them, yada yada yada... Soon after, next class, and I noticed that my VPN server was blocked. Now I used my personal VPN server at the time mostly to access a file server at home to securely fetch documents I needed in class, without having to carry an external hard drive with me all the time. However it was also used for gateway redirection (i.e. the main purpose of commercial VPN's, le new IP for "le onenumity"). I mean for example, if some douche in that class would've decided to ARP poison the network and steal credentials, my VPN connection would've prevented that.. it was a decent workaround. But now it's for some reason causing Moodle to throw some type of 403.

Asked the teacher for routers and switches I had a class from at the time.. why is my VPN server blocked? He replied with the statement that "yeah we blocked it because you can bypass the firewall with that and watch porn in class".
Alright, fair enough. I can indeed bypass the firewall with that. But watch porn.. in class? I mean I'm a bit of an exhibitionist too, but in a fucking class!? And why right after that PoC, while I've been using that VPN connection for over a year?

Not too long after that, I prematurely left that class out of sheer frustration (I remember browsing devRant with the intent to write about it while the teacher was watching 😂), and left while looking that teacher dead in the eyes.. and never have I been that cold to someone while calling them a fucking idiot.

Shortly after I've also received an email from them in which they stated that they wanted compensation for "the disruption of good service". They actually thought that I had hacked into their servers. Security teachers, ostensibly technical people, if I may add. Never seen anyone more incompetent than those 3 motherfuckers that plotted against me to save their own asses for making such a shitty infrastructure. Regarding that mail, I not so friendly replied to them that they could settle it in court if they wanted to.. but that I already knew who would win that case. Haven't heard of them since.

So yeah. That's why I regard those expensive shitty pieces of paper as such. The only thing they prove is that someone somewhere with some unknown degree of competence confirms that you know something. I think there's far too many unknowns in there.

Nowadays I'm putting my bets on a certification from the Linux Professional Institute - a renowned and well-regarded certification body in sysadmin. Last February at FOSDEM I did half of the LPIC-1 certification exam, next year I'll do the other half. With the amount of reputation the LPI has behind it, I believe that's a far better route to go with than some random school somewhere.

Story time! This happened several years ago, back when I didn't have a computer and I was just using the computers at the university. They had 8 iMacs all in a row, and I would sign into one and do my work.

Now these computers have Deep Freeze on them, which is a fancy hard disk driver that treats the entire drive as copy-on-write, so when anything writes to the drive it makes a copy of the block and writes to that instead. That way all your changes are gone when you reboot. It's a real nifty idea, but it's annoying that you have to reset all your settings the way you like them.

So as part of my setup routine I signed into iCloud. This automatically synced my browser history and my email, and various other things I didn't really care about.

One of those things I didn't care about was Find My Mac. I found this out next time I signed into iCloud and saw the university computer on the list. I had never seen these computers on the list before since normally the computer reboots and forgets everything when you log out. What I think happened is the sysadmin forgot to check the "reboot on logout" option in Deep Freeze. So I was like "I wonder what would happen if I passcode locked the computer?" I clicked the passcode lock option and entered 5555, and it seemed to work.

The next day I come in and the particular computer I locked was gone. I thought "oh God what have I done". So I inquired with the sysadmin (who I really hope is not reading this) and he said "oh, someone got into the Find my Mac thing and locked it down. We were trying different codes, since if we couldn't unlock it we'd have to send it to Apple and provide proof of purchase and that could take weeks. We had tried all the obvious ones like 1234 and that wasn't working so I was about to give up, but then I tried 5555 and it rebooted! So yeah, it'll be back soon, and I decided to try installing OS X 10.11 on it because we'll all need to upgrade sooner or later eventually and it's best to have tested a bit first."

So in the end I somehow made it out with my skin still on, and also with El Capitan on one of the computers, which was the only one I used after that. Not so bad! Oh and if you've manged to read all the way through you deserve a cookie 🍪😄

Gather around folks, I'll paint you a nice picture based on a true story, back from my sysadmin days. Listen up.
It's about HP and their Solaris 5.4/6 support.

- Yet another Prod Solaris dinosaur crashed
- Connected to console, found a dead system disk; for some reason it was not booting on the remaining redundant disk...
- Logged an HP vendor case. Sev1. SLA for response is 30 minutes, SLA for a fix is <24 hours
- It took them 2 days to respond to our Prod server outage due to failed system disks (responses "we are looking into this" do not count)
- it took another day for them to find an engineer who could attend the server in the DC
- The field engineer came to the DC 4 hours before the agreed time, so he had to wait (DC was 4-5 hours of driving away from HP centre)
- Turns out, he came to the wrong datacentre and was not let in even when the time came

- We had to reschedule for two days later. Prod is still down
- The engg came to the DC on time. He confirmed he had the FRU on him. Looks promising
- He entered the Hall
- He replaced the disk on the Solaris server
- It was the wrong disk he replaced. So now the server is beyond rebuild. It has to be built anew... but only after he comes back and replaces the actually faulty disk.
- He replaced that disk on the wrong Solaris server

Did a bunch more cowboy coding today as I call it (coding in vi on production). Gather 'round kiddies, uncle Logan's got a story fer ya…

First things first, disclaimer: I'm no sysadmin. I respect sysadmins and the work they do, but I'm the first to admit my strengths definitely lie more in writing programs rather than running servers.

Anyhow, I recently inherited someone else's codebase (the story of my profession career, but I digress) and let me tell you this thing has amateur hour written all over it. It's written in PHP and JavaScript by a self-taught programmer who apparently discovered procedural programming and decided there was nothing left to learn and stopped there (no disrespect to self-taught programmers).

I could rant for days about the various problems this codebase has, but today I have a very specific story to tell. A story about errors and logs.

And it all started when I noticed the disk space on our server was gradually decreasing.

So today I logged onto our API server (Ubuntu running Apache/PHP) and did a df -h to check the disk space, and was surprised to see that it had noticeably decreased since the last time I'd checked when everything was running smoothly. But seeing as this server does not store any persistent customer data (we have a separate db server) and purely hosts the stateless API, it should NOT be consuming disk space over time at all.

The only thing I could think of was the logs, but the logs were very quiet, just the odd benign message that was fully expected. Just to be sure I did an ls -Sh to check the size of the logs, and while some of them were a little big, nothing over a few megs. Nothing to account for gigabytes of disk space gradually disappearing.

What could it be? I wondered.

cd ../..
du . | sort --sort=numeric

What's this? 2671132 K in some log folder buried in the api source code? I cd into it and it turns out there are separate PHP log files in there, split up by customer, so that each customer of ours (we have 120) has their own respective error log! (Why??)

Armed with this newfound piece of (still rather unbelievable) evidence I perform a mad scramble to search the codebase for where this extra logging is happening and sure enough I find a custom PHP error handler that is capturing (most) errors and redirecting them to these individualized log files.

Conveniently enough, not ALL errors were being absorbed though, so I still knew the main error_log was working (and any time I explicitly error_logged it would go there, so I was none the wiser that this other error-catching was even happening).

Needless to say I removed the code as quickly as I found it, tail -f'd the error_log and to my dismay it was being absolutely flooded with syntax errors, runtime PHP exceptions, warnings galore, and all sorts of other things.

My jaw almost hit the floor. I've been with this company for 6 months and had no idea these errors were even happening!

The sad thing was how easy to fix all the errors ended up being. Most of them were "undefined index" errors that could have been completely avoided with a simple isset() check, but instead ended up throwing an exception, nullifying any code that came after it.

Anyway kids, the moral of the story is don't split up your log files. It makes absolutely no sense and can end up obscuring easily fixable bugs for half a year or more!

Happy coding.

//little Story of a sys admin

Wondered why a Server on my Linux Root couldn't build a network connection, even when it was running.

Checked iptables and saw, that the port of the Server was redirected to a different port.

I never added that rule to the firewall. Checked and a little script I used from someone else generated traffic for a mobile game.

OK beginn the DDoS Penetration. Over 10 Gbit/s on some small servers.

Checked Facebook and some idiot posted on my site:
Stop you little shithead or I will report you to the police!!!

Checked his profile page and he had a small shitty android game with a botnet.

Choose one:
1. let him be
2. Fuck him up for good

Lets Sudo with 2.

I scaled up my bandwith to 25 Gbit/s and found out that guys phone number.

Slowly started to eat away his bandwith for days. 3 days later his server was unreachable.

Then I masked my VoIP adress and called him:

Me: Hi, you know me?
He: No WTF! Why are you calling me.
Me: I love your're game a lot, I really love it.
He: What's wrong with you? Who are you?
Me: I'm teach
He: teach?
Me: Teach me lesson
He: Are you crazy I'm hanging up!
Me: I really love you're game. I even took away all your bandwith. Now you're servers are blocked, you're game banned on the store.
He: WHAT, WHAT? (hearing typing)

Me: Don't fuck with the wrong guys. I teached you a lesson, call me EL PENETRATO

He: FUCK Fuck Fuck you! Who are you???!!! I'm going to report you!
Me: How?
He: I got you're logs!
Me: Check it at Utrace...
He: Holy shit all around the world

Me: Lemme Smash Bitch
*hung up*

funniest thing today: PM asking me to create a Jira subtask for EACH class / data type of the data model that I'm CURRENTLY concipating / designing in this story.

maybe I should write a Jira / Enterprise Architect integrated tool that updates Jira tasks based on my modeling actions, and count minutes until our sysadmin arrives at my desk.

jeez, that guy really has a fetish for kafkaesque bureaucracy.🤦‍♀️

A bit different than wk93, but still connected and a fun story.

Back in high school when it began to digitalize everything, so began our teachers journey with technology. We, as IT class were into these things, but as far as I can say, others in the school including both teachers and students were like cave mans when it came to IT.

Most of them kept the different wifi networks password on the windows desktop, in a file 'wifipassword.txt'. When we were on robotics seminar, we had to use a teacher's laptop. The wifi network was incredibly fast and powerful,, yet so poorly configured that even the configuration page user/pass was the default admin/admin, because the IT admin wasn't the most skilled one.

We got the idea to sell the password of the wifi network to other students. Not much, for about 1 dollar a week. The customer came to us, we took the phone, took note of the MAC address, entered the password, and if the guy were to stop paying every week, we just blacklisted that MAC on the next robotics course.

Went well for months, until a new sysadmin came and immediately found it out, we were almost fired from the school, but my principal realized how awesome this idea was. You may say that we were assholes, and partially that is true, I'd rather say we made use of our knowledge.

I am a mechanical engineer first and my companies go to sysadmin second. So software developing isnt really my main field of expertise buuttt:

WHY IS SLOOPY SOFTWARE WRITING A VIABLE EXCUSE?

Story:
Yesterday i started to migrate some stuff from our old Win 2008 Server to the new 2016. Turns out there are some MS SQL Express Servers running. Quick check for what they are turns out that they are activly used. So far so good. For other reasons we have a new MSSQL 2017 Core Licence. So i thought, hey it would be nice to just move those 2012, 2008 and 2014 Express Servers to a real one that can use the entire machines capabilities.
After some try & error with exporting one of the softwares (where i had to elevate one the user rights to sysadmin for reasons) the entire system stopped working. I didnt deleted anything or changed anything! Well, i elevated user rights. After 2 hours of support call it turns out that the software stopped working cause i gave the database user sysadmin rights. I dont know enough about MSSQL to judge wether that is logical or not, but it sounds super illogical and i suspect sloopy software writing on the manufacturers part. One way or another, the excuse from the telephone support was "yeah, our software is a very fragile child"
Okay.
After i told all that my coworkers two of them were also "yeah, that is just how the [company] software is, you have to be careful with it"

Apparently it broke in the past for other minor stuff.
As an engineer i cannot build bridges that collapse when you use the left and the right lane at the same time. For an architect it isnt okay to build an house where the front door explodes when you open a window. It is not okay for a power tool to go out in a fireball when you accidently drill plastic with it. But for some weird reasons its socially acceptable for programs to be sloopy, buggy and only working under specific conditions. Since when is it okay for a car only to work when you know specific steps to make it run? Like, throwing your spare key in the gas tank, the kick the left wheel exactly three times and finally tapping the steering wheel 5 times left, 4 times right. What? That would be ridiculous? But that is exactly how that software works. You have to follow a specific step guide to make it work, EVERY TIME.

I. JUST. DONT. GET. IT

This week I got a promotion after being a junior for a year. Boss said Im a medior now and my monthly salary raised with 400 euro per month

Feels good but what feels bad is that a coworker of mine which has been contracted recently without any development experience is still making 400 more a month..

The thing is that this "developer" wanted to become a Java developer, he has been given time during work to study Java and in the meanwhile join the team thats working on a saas product (my team, where im lead dev)

During the 3 months ive counted a maximum of 10 commits and i was done with him which conflicted in a very bad vibe at the office.

During a refinement I asked if everybody understood what needs to be done, no questions asked. Next day when i was working at a clients office on another project 9 am i git a Skype message "Can you tell me What to do? I have no idea" where I replied "you should have asked me yesterday, i am not going to help you unless u come up with a question that makes sense.. what have u tried urself?".. Well then he got mad and stopped doing what he was trying to do.

The next morning i talked with him and we agreed to have a 1hour session to talk him through the user story. When we were done, he said that he understood and was going to work on it.

Next day I check, no commits, so during stand up i confronted hmj with this and he admitted hes been lacking and wanted to talk with the boss and me after stand up.

Well he admitted things were going to fast to keep up for him because he is doing some sysadmin stuff aswell.. the plan of becoming a Java dev was now history and he left the team..

Now he is just doing some sysadmin stuff but its been 3 days that hes been saying today ill setup a tomcat on the servers and give you SSH acces to deploy your .war files, today I finally gained access but he couldnt figure out how to move the war to the webapps folder.. And i wasnt allowed to transfer it to there..

Adventures with house IT

I'm currently experimenting with PowerShell but my scripts won't run even though I've got every local permission. The error message indicates it's a GPO problem.
"No problem" le me thinks and calls IT hotline.
After 2 incompetent and unprofessional technicians i've still got no solution. I'm waiting for the second tech to call back because he "needed some time to get to know PowerShell" (he is a trained and certified SysAdmin).
During my call he couldn't decide if it was a GPO problem or not.
And this is just one story of their incompetence...

Boss needs certain stats pulled from database once a year for board meeting. This time I delegate it to a junior dba/sysadmin. He looks at my 3-year-old docs that I hastily jotted down and pasted and included my rambling notes with results from way back then. Mostly they were just to jog my own memory, not to be a really neat, clean instruction guide. He does the queries correctly, but in ticket for boss he pastes also all my notes from the docs. boss gets confused, "what is this other number, I don't get it?!" We have to have a meeting of the 3 of us and waste an hour or so just to figure out what went wrong, finally I realize what junior guy accidentally did. Moral of story: to avoid baffling the nontechs, always simplify, simplify, simplify. Alternate moral of story: before delegating a task that seems old hat to you, always review your notes/docs and make sure they're ready for someone else to use them.

Everyone in this team calls everything a team effort, but once I start offering my help, they be like "no, I can do it. I know more than you".

Hmm. yeah, but you (sysadmin) use jQuery and vanillajs mixed. For example: $('#hello') and document.getElementById('hello').
Also you put console.logs everywhere, I don't mind putting console.logs in development, but not in production.
Oh and he copies the libraries to every folder that needs it, so there are at least 12 jquery libs in this project and the version is not even the same. Lol.... Please slap me to death.

There is another networkadmin that calls himself a (python) developer. He doesn't agree with my simplicity.

His work (just an example, changed names but you get the idea)
"A notebook that is used by x-department"

Model: Notebook
endpoint: department-notebooks

Model: DepartmentConfigs
Endpoint: notebook-department-configs
You won't believe what he put in 'department'configs, it's literally hardware vendor, model, versions.

Like... really? What the hell you doing man?!
Just have these models for example: device, department, vendor, product, category

We do not only have notebooks, but also servers, routers, switches and more.

His argument of having configs in the name is that they do more complex things. Hmm, I don't see it in the code and the data is messed up:
Microsoft, microsoft, micro soft.

He fixed it by hardcoding it in a select box. Mickysoft isn't the only vendor, fuck you!

fuck this team, fuck these people

Another fucking rant, a story was assigned to me. But that stupid fake developer worked on it immediately and message me he fixed it already. I guess he won't let me touch his baby.

Everything is just piling up. This team and people aren't fun at all.

Note: In this rant I will ask for advices, and confess some sins. I will tell my personal story- it will be long.

So basically it has been almost 2 years since I first entered the world of software development. It has been the biggest and most important quest of my life so far, but yet I feel like I missed a lot of my objectives, and lots of stuff did not go the way I wanted them to be, and it makes feel frustrated and it lowered my self esteem greatly. I feel confused and a bit depressed, and don't know what to do.

I'll start: I'm 23 years old. 2 years ago I was still a soldier(where I live there is a forced conscription law) in a sysadmin/security role. I grew tired of the ops world and got drawn more and more into programming. A tremendous passion became to burn in me, as I began to write small programs in Python and shell scripts. I wanted to level up more seriously so I started reading programming books and got myself into a 10 month Java course.

In the meanwhile I got released from army duty and got a job as a security sysadmin at a large local telco company. Job was boring and unchallenging but it payed well. I had worked there for 1 year and at the same time learned more and more stuff from 2 best friends who have been freelance developers for years. I have learned how to build full-stack mobile apps and some webdev, mainly Android and Node.js. However because I was very inexperienced and lacked discipline, all of my side projects failed horribly, and all attempts to work with my experienced friends have failed too- I feel they lost a lot of trust for me(they don't say it, but I feel it, maybe I'm wrong).

I began to realise I had to leave this job and seek a developer job in order to get better, and my wish came true 6 months ago when I finally got accepted into a startup as a fullstack webdev, for a bit lower wage but I felt it was worth it. I was overjoyed.

But now my old problems did not end, they just changed. My new job is a thousand times harder and more intensive than the old one. I feel like it sucks all the energy and motivation that was still left in me, and I have learned almost nothing in my free time, returning home exhausted. My bosses are not impressed from my work despite me being pretty junior level, and I feel like I'm in a vicious cycle that keeps me from advancing my abilities. My developer friends I mentioned earlier have jobs like I do and still manage to develop very impressive side projects and even make a nice sum of money from them, while I can't even concetrate on stupid toy projects and learning.

I don't know why It is like this. I feel pathetic and ashamed of my developer sins and lack of discipline. During that time I also gained some weight that I'm trying t lose now... I know not all of it is my fault but it makes me feel like crap.

Sorry for the long story. I just feel I need to spill it out and hope to get some advices from you guys who may or may not have similar experiences. Thanks in advance for reading this.

I am the responsible for the atlassian Suite at work, as I maintain the systems, set them up, and stuff.

One day, our crowd (the authentication and authorization application) just went crazy. At like lunch time it could not connect to the AD anymore. No reasons. Throwing XSRF errors (cross site scripting), because http would connect to https. "won't do it, fuck you" it told me. Out of the blue. Noone changed anything. And yea, seriously. Noone did.
It just refused to connect (as connecting to AD is connecting yourself with you own api. And refusing yourself talking to yourself). It runs behind a proxy. Therefore http/https. Well, this worked for years. But out of sudden not anymore.

Yea. Fuck you.

It was reported some hours later, at like 3pm, as people could not login to the applications using crowd as authentication and authorization server.

Tried to debug the system, where nothing was did, to make it work. At best time to fail.

First workaround: if you are logged into one of the other applications of atlassian, just refresh the site, so your SSO token gets a refresh and you are signed on again.

Then I searched more and more. And more.
But nothing worked, nothing helped.

So I addressed an emergency maintenance, take down the whole Suite, restart crowd, to apply some changes to it's settings, not knowing what happening then, because all connections of SSO will then be released. Sent out the mail like 30 minutes beforehands.

While waiting for the window, I just typed my credentials... And redid, and redid, so to type and being bored.
Three minutes before the window...

It just worked again.

Well. Wtf. Serioudl
Just came back.

No Intrusion, no changes at all. Just came back, as nothing has happened.

Kind of best part of this story... A headhunter messaged me on my way home to offer me a job as an Atlassian Suite SysAdmin for a company, at kinda the double of my salary.

At first I was thinking to go there, and when someone then asked me sth about Atlassian just start to laugh and then leave still laughing...

But then I very nicely respond that I dont want to cry at work. And wished him best luck.

I am doing some bad upgrades now on our Suite. Very painful.
And I looked into the start scripts. Some Look like the untalented intern tells another one to write scripts. Seriously wtf.

Today I followed the guide to Update a confluence and change database to Postgres. Didnt work, Postgres error.
Try it again, jquery won't load. Next try, tomcat not starting anymore. Did same thing. Every fucking time.

Yea. Maintenance window to get a nice new export soon. Will only take an hour.

To switch database in confluence, you need to set it up very fresh. And then Import your export.
Export takes an hour at our system.
Importing maybe the same time. Hope it will work (hint: Nope).

Oh, can be nice also. Just tell the Bitbucket to migrate databases, there is a fucking setting for it. Enter new database, ready, go, finished.

At least they don't raise costs very much every kinda year.
Oh sorry, yes, they do.

Does anyone from here working by GMX? I am specially looking for a sysadmin.

The story is the following. We can't send emails to GMX addresses in general. I've contacted my provider, and they said, that they've contacted GMX several times but no solution has been made so far. This was almost a month ago and the problem still persists.

If anyone from here willing to help me clear this mess, or just give some explanation, I would be grateful. We are loosing reputation as a company having to answer from a different email address.

If it is a sensitive info please give me a channel where we can speak about the details.

Please note I am not a sysadmin by the hosting company, i am simply a customer of theirs.