So I got the job. Here's a story, never let anyone stop you from accomplishing your dreams!

It all started in 2010. Windows just crashed unrecoverably for the 3rd time in two years. Back then I wasn't good with computers yet so we got our tech guy to look at it and he said: "either pay for a windows license again (we nearly spend 1K on licenses already) or try another operating system which is free: Ubuntu. If you don't like it anyways, we can always switch back to Windows!"

Oh well, fair enough, not much to lose, right! So we went with Ubuntu. Within about 2 hours I could find everything. From the software installer to OpenOffice, browsers, email things and so on. Also I already got the basics of the Linux terminal (bash in this case) like ls, cd, mkdir and a few more.

My parents found it very easy to work with as well so we decided to stick with it.

I already started to experiment with some html/css code because the thought of being able to write my own websites was awesome! Within about a week or so I figured out a simple html site.

Then I started to experiment more and more.

After about a year of trial and error (repeat about 1000+ times) I finally got my first Apache server setup on a VirtualBox running Ubuntu server. Damn, it felt awesome to see my own shit working!

From that moment on I continued to try everything I could with Linux because I found the principle that I basically could do everything I wanted (possible with software solutions) without any limitations (like with Windows/Mac) very fucking awesome. I owned the fucking system.

Then, after some years, I got my first shared hosting plan! It was awesome to see my own (with subdomain) website online, functioning very well!

I started to learn stuff like FTP, SSH and so on.

Went on with trial and error for a while and then the thought occured to me: what if I'd have a little server ONLINE which I could use myself to experiment around?

First rented VPS was there! Couldn't get enough of it and kept experimenting with server thingies, linux in general aaand so on.

Started learning about rsa key based login, firewalls (iptables), brute force prevention (fail2ban), vhosts (apache2 still), SSL (damn this was an interesting one, how the fuck do you do this yourself?!), PHP and many other things.

Then, after a while, the thought came to mind: what if I'd have a dedicated server!?!?!?!

I ordered my first fucking dedicated server. Damn, this was awesome! Already knew some stuff about defending myself from brute force bots and so on so it went pretty well.

Finally made the jump to NginX and CentOS!

Made multiple VPS's for shitloads of purposes and just to learn. Started working with reverse proxies (nginx), proxy servers, SSL for everything (because fuck basic http WITHOUT SSL), vhosts and so on.

Started with simple, one screen linux setup with ubuntu 10.04.

Running a five monitor setup now with many distro's, running about 20 servers with proxies/nginx/apache2/multiple db engines, as much security as I can integrate and this fucking passion just got me my first Linux job!

It's not just an operating system for me, it's a way of life. And with that I don't just mean the operating system, but also the idea behind it :).

Hacking/attack experiences...
I'm, for obvious reasons, only going to talk about the attacks I went through and the *legal* ones I did 😅 😜

Let's first get some things clear/funny facts:
I've been doing offensive security since I was 14-15. Defensive since the age of 16-17. I'm getting close to 23 now, for the record.

First system ever hacked (metasploit exploit): Windows XP.
(To be clear, at home through a pentesting environment, all legal)
Easiest system ever hacked: Windows XP yet again.

Time it took me to crack/hack into today's OS's (remote + local exploits, don't remember which ones I used by the way):
Windows: XP - five seconds (damn, those metasploit exploits are powerful)
Windows Vista: Few minutes.
Windows 7: Few minutes.
Windows 10: Few minutes.
OSX (in general): 1 Hour (finding a good exploit took some time, got to root level easily aftewards. No, I do not remember how/what exactly, it's years and years ago)
Linux (Ubuntu): A month approx. Ended up using a Java applet through Firefox when that was still a thing. Literally had to click it manually xD
Linux: (RHEL based systems): Still not exploited, SELinux is powerful, motherfucker.

Keep in mind that I had a great pentesting setup back then 😊. I don't have nor do that anymore since I love defensive security more nowadays and simply don't have the time anymore.

Dealing with attacks and getting hacked.

Keep in mind that I manage around 20 servers (including vps's and dedi's) so I get the usual amount of ssh brute force attacks (thanks for keeping me safe, CSF!) which is about 40-50K every hour. Those ip's automatically get blocked after three failed attempts within 5 minutes. No root login allowed + rsa key login with freaking strong passwords/passphrases.

linu.xxx/much-security.nl - All kinds of attacks, application attacks, brute force, DDoS sometimes but that is also mostly mitigated at provider level, to name a few. So, except for my own tests and a few ddos's on both those domains, nothing really threatening. (as in, nothing seems to have fucked anything up yet)

How did I discover that two of my servers were hacked through brute forcers while no brute force protection was in place yet? installed a barebones ubuntu server onto both. They only come with system-default applications. Tried installing Nginx next day, port 80 was already in use. I always run 'pidof apache2' to make sure it isn't running and thought I'd run that for fun while I knew I didn't install it and it didn't come with the distro. It was actually running. Checked the auth logs and saw succesful root logins - fuck me - reinstalled the servers and installed Fail2Ban. It bans any ip address which had three failed ssh logins within 5 minutes:
Enabled Fail2Ban -> checked iptables (iptables -L) literally two seconds later: 100+ banned ip addresses - holy fuck, no wonder I got hacked!

One other kind/type of attack I get regularly but if it doesn't get much worse, I'll deal with that :)

Dealing with different kinds of attacks:

Web app attacks: extensively testing everything for security vulns before releasing it into the open.
Network attacks: Nginx rate limiting/CSF rate limiting against SYN DDoS attacks for example.
System attacks: Anti brute force software (Fail2Ban or CSF), anti rootkit software, AppArmor or (which I prefer) SELinux which actually catches quite some web app attacks as well and REGULARLY UPDATING THE SERVERS/SOFTWARE.

So yah, hereby :P

This fucking customer...
I've told that person so many times that they need to FIX THEIR CODE, because it get's pwned all the time.
To make stuff worse - they are still using Debian 5, and we are unable to upgrade because all their shit will break.

I found his fix today - he installed an old version of NGINX because it is "better".

No fuck you.

Fuck the memes.

Fuck the framework battles.

Fuck the language battles.

Fuck the titles.

Anybody who has been in this field long enough knows that it doesn't matter if your linus fucking torvalds, there is no human who has lived or ever will live that simultaneously understands, knows, and remembers how to implement, in multiple languages, the following:

- jest mocks for complex React components (partial mocks, full mocks, no mocks at all!)
- token cancellation for asynchronous Tasks in C#
- fullstack CRUD, REST, and websocket communication (throw in gRPC for bonus points)
- database query optimization, seeding, and design
- nginx routing, https redirection
- build automation with full test coverage and environment consideration
- docker container versioning, restoration, and cleanup
- internationalization on both the front AND backends
- secret storage, security audits
- package management, maintenence, and deprecation reviews
- integrating with dozens of APIs
- fucking how to center a div

and that's a _comically_ incomplete list; barely scratches the surface of the full range of what a dev can encounter in a given day of writing software

have many of us probably done one or even all of these at different times? surely.

but does that mean we are supposed to draw that up at a moment's notice some cookie-cutter solution like a fucking robot and spit out an answer on a fax sheet?

recruiters, if you read this site (perhaps only the good ones do anyway so its wasted oxygen), just know that whoever you hire its literally the luck of the draw of how well they perform during the interview. sure, perhaps some perform better, but you can never know how good someone is until they literally start working at your org, so... have fun with that.

Oh and I almost forgot, again for you recruiters, on top of that list which you probably won't ever understand for the entirety of your lives, you can also add writing documentation, backup scripts, and orchestrating / administrating fucking JIRA or actually any somewhat technical dashboard like a CMS or website, because once again, the devs are the only truly competent ones - and i don't even mean in a technical sense, i mean in a HUMAN sense of GETTING SHIT DONE IN GENERAL.

There's literally 2 types of people in the world: those who sit around drawing flow charts and talking on the phone all day, and those WHO LITERALLY FUCKING BUILD THE WORLD

why don't i just run the whole fucking company at this point? you guys are "celebrating" that you made literally $5 dollars from a single customer and i'm just sitting here coding 12 hours a day like all is fine and well

i'm so ANGRY its always the same no matter where i go, non-technical people have just no clue, even when you implore them how long things take, they just nod and smile and say "we'll do it the MVP way". sure, fine, you can do that like 2 or 3 times, but not for 6 fucking months until you have a stack of "MVPs" that come toppling down like the garbage they are.

How do expect to keep the "momentum" of your customers and sales (I hope you can hear the hatred of each of these market words as I type them) if the entire system is glued together with ducktape because YOU wanted to expedite the feature by doing it the EASY way instead of the RIGHT way. god, just forget it, nobody is going to listen anyway, its like the 5th time a row in my life

we NEED tests!
we NEED to know our code coverage!
we NEED to design our system to handle large amounts of traffic!
we NEED detailed logging!
we NEED to start building an exception database!

BILBO BAGGINS! I'm not trying to hurt you! I'm trying to help you!

Don't really know what this rant was, I'm just raging and all over the place at the universe. I'm going to bed.

NO FIREFOX AND CHROMIUM, I ALREADY SET THE FUCKING CORS HEADERS CORRECTLY ACCORDING TO THE OFFICIAL FUCKING NGINX DOCUMENTATION. WHY THE FUCK ARE YOU STILL DISALLOWING CROSS-ORIGIN REQUESTS?!

I thought this launch (security/privacy blog) would go smooth:
- analytics fell, except for one thing, apart for yet unknown reasons
- MySQL came with a very weird error which took me like half an hour of research before I hacked my way past it.
- the firewall started to fuck around for no reason, works now though.

Nginx worked without issues though, as well as NetData 😅

Yeah, didn't go as planned :P

FUCK YOU WORDPRESS

Omfg never been so fucking pissed in my life.

I just wasted 3 hours because this fucking bullshit rewrites the fucking URL based on the URL on a config fucking file?!!?

It fucking ignores: apache virtual host configs and nginx reverse proxy

omfg...

Not just another Windows rant:

*Disclaimer* : I'm a full time Linux user for dev work having switched from Windows a couple of years ago. Only open Windows for Photoshop (or games) or when I fuck up my Linux install (Arch user) because I get too adventurous (don't we all)

I have hated Windows 10 from day 1 for being a rebel. Automatic updates and generally so many bugs (specially the 100% disk usage on boot for idk how long) really sucked.

It's got ads now and it's generally much slower than probably a Windows 8 install..

The pathetic memory management and the overall slower interface really ticks me off. I'm trying to work and get access to web services and all I get is hangups.

Chrome is my go-to browser for everything and the experience is sub par. We all know it gobbles up RAM but even more on Windows.

My Linux install on the same computer flies with a heavy project open in Android Studio, 25+ tabs in Chrome and a 1080p video playing in the background.

Up until the creators update, UI bugs were a common sight. Things would just stop working if you clicked them multiple times.

But you know what I'm tired of more?

The ignorant pricks who bash it for being Windows. This OS isn't bad. Sure it's not Linux or MacOS but it stands strong.

You are just bashing it because it's not developer friendly and it's not. It never advertises itself like that.

It's a full fledged OS for everyone. It's not dev friendly but you can make it as much as possible but you're lazy.

People do use Windows to code. If you don't know that, you're ignorant. They also make a living by using Windows all day. How bout tha?

But it tries to make you feel comfortable with the recent bash integration and the plethora of tools that Microsoft builds.

IIS may not be Apache or Nginx but it gets the job done.

Azure uses Windows and it's one of best web services out there. It's freaking amazing with dead simple docs to get up and running with a web app in 10 minutes.

I saw many rants against VS but you know it's one of the best IDEs out there and it runs the best on Windows (for me, at least).

I'm pissed at you - you blind hater you.

Research and appreciate the things good qualities in something instead of trying to be the cool but ignorant dev who codes with Linux/Mac but doesn't know shit about the advantages they offer.

Fucking cloud providers always trying to steal your shit and spy on your things, fucking prying eyes. That's why i've decided to go back hosting my own private cloud from home. Running on some very energy efficient shit: dual core intel atom cpu (so slow that it can't fucking run windows normally), 16gb of ram, because why the fuck not? and 1tb 2.5"hdd, along with unlimited data - 100/100 Mbit/s internet connection with a server response time less than 95ms just to backup my shitty Iphone selfies and cat pics, host some very important files and regularly back up my contacts. This shit runs CentOS, Nginx, https, bitch! This platform is more trustworthy than your shitty dropbox or whatever other shit they offer you. I can choose whether i back-up my shit from local network or over internetz, Costing me no more than 25€ annually(just to keep the machine on 24/7/365).

Saw this security blunder a while ago. Went onto some site and it showed me this username/password dialog (probably an apache's htpasswd or nginx one). Went away but returned quickly because I noticed I could see all content. Then I thought 'why the fuck not try?' so I dragged the auth popup thingy to the side of the screen and et voila... I could interact with the page as if nothing was wrong while the authentication popup was hovering above the page on the right!

I sat there giggling dramatically for a while.

Fuck it. Today I broke my production server installing the brotli encryption on nginx. -_-

Debugging a request that got lost in a myriad of containers of a scaled application....

It wouldn't be worth a rant if there wasn't some kinky SM stuff in it, wouldn't it?

Regexes. The fucker who wrote a lot of the NGINX (🤢) configuration decided to use the Perl Regexes with named group matching. A lot.

So now I have to fight wild variables supposedly coming from nowhere (as they stem from the named groups)… fucking single location redirects instead of maps.... A d have to write an explanatory documentation while going down the rabbit hole of trying to find out where the fuck that shitty frigging bastard redirected wrong.

I really wish I could eradicate the person who wrote this shit....

I am so mad, I have no words for how fucking much I hate ever having to work or pass work to other incompetent developers or teams, what a fucking waste of time and resources.

After handing off the frontend - for the client to find some team, that would do it in the short time and budget he needs (multiple developers, more fast, much good), he found a team that seemed to be alright for the job and seemed alright to me too, now maybe a month or two later, the client contacts me, that they fucked something up and if I could talk to them.

The email I then received from them seriously made me speechles, mad and sad, all at same time, I spent multiple upon multiple hours, getting a very good readable documentation up (markdown with TOC, properly rendered headers, bulletpoints, all that shit), with all files, all services used, all credentials, even converted all ssh keys into putty ppk format, in case the developers are using windows and are too dumb to do it themselves, nginx configs, it had seriously everything, even too much to list.

They somehow managed to fuck up the entire server, while attempting to "add ssh keys themselves", EVEN FUCKING THOUGH I have included all the keys they need, all the hosting credentials, everything, yet they decided to fuck with shit themselves and completely annihilate the server in the process (HOW?!), so not even the webserver works anymore.

I am fucking speechless, I made it so fucking easy to gather all info and files they need, all properly put into well named folders, along the documentation in an archive and they somehow managed to nuke the fucking server, while attempting to add ssh keys?!

If you don't know how to config a server, then don't fucking touch it and just use everything, that got served to you on a fucking silver platter.

---

I'll just instantly answer the most annoying comment, that somebody could come up with: "why didn't you do it yourself?"

Because in a perfect world, a fully managed team, can do much more than a single developer can, especially in the same timeframe and from what I heard of said client, atleast they did something in terms of developing the system. (which surprises me, considering it's the same people that nuked a server, while trying to add ssh keys)

Fuck this Apache Server on this elastic beanstalk!!! Nginx is so much better at configuration and ease of use!!!

TLDR;
How much do you earn for your skill set in your country vs your cost of living?
BONUS;
See how much I & others earn.

Recently I became aware of just how massive the gap in developers earnings are between countries. I'd love to calculate a fixed score for income vs cost of living.

I know this stuff is sensitive to some so if you prefer just post your score (avg income p/m after tax / cost of living).

I'm not shy so I'll go first:

MY RATES
Normal Rate (Long term): $23
Consulting / Short term: $30-$74
Pen Test: $1500 once off.
Pen Test Fixes: consulting rate.
Simple work/websites: min $400+
Family & Friends: Dev friends are usually free (when mutually beneficial). Family and others can fuck off, even if they can pay (I pass their info to dev friends with fair warning).

GENERAL INFO
Experience: 9 years
Country: South Africa
Developer rareness in country: Very Rare (+-90 job openings per job seeker).
Middle class wage in country: $1550 p/m (can afford a new car, decent apartment & some luxuries like beer/eating out).
Employment type: Permanent though I can and do freelance occasionally.
Client Locality: Mostly local.
Developer Type: Web Developer (True web dev - I do anything web related from custom HTTP servers to sockets, services, advanced browser api's, apps & more).

STACKS / SKILLSETS

I'M PROFICIENT IN:
python, JavaScript, ASP classic, bash, php, html, css, sql, msql, elastic search, REST, SOAP, DOM, IIS, apache

I DABBLE WITH:
ASP.net, C++, ruby, GO, nginx, tesseract

MY SPECIALTIES:
application architecture, automation, integrations, db's, real time data, advanced browser apps/extensions (webRTC, canvas etc).

SUMMARY
Avg income p/m after tax: $2250
Cost of living (car+rent+food): $1200
Score: 1.85

*Note: For integrity when calculating my cost of living I excluded debt repayments and only kept my necessities which are transport, food & shelter.

I really hope you guy's post your results, it would be great to get an idea of which is really the worst / best country to be a developer in.

I some time's feel stupid when my Ubuntu start acting up on prod, purge this and that ,fuck apache can't start system CTL that ps ax grep in the p#$#@* , damn apt install nginx full good, mysql can't start too ,OK apt install mariadb-client else percona && apt postgresql thanks ,god no client noticed ..

At the beginning of the evening I started creating a snapshot of my webserver Ubuntu 16.04 installation, running 5 websites.

When the snapshot was created I started a release upgrade to Ubuntu 18.04.

Finally after upgrade and reboot... Nothing worked anymore. Nginx was running but none of the websites was working.

I started checking logs & searching for a solution, with no luck.

Wanted to restore my snapshot. Reading the docs of Scaleway: only a manual on how to restore to a *new* server...

Dumb me removing my current server and wanting to create a new server: "All servers tempotary out of stock"

Me: *panicing and clicking the resfresh button every second*

"Low stock"
*HITTING the create server button*
Added my snapshot
*Booting up*

Ssh'ing into server
Server: "nope"

#+#£_&-+{$}¥}•+';!

*Sees 'add snapshot to volume'*
*Sees 'add volume to server'*
*All websites running again after nginx restart*

What the fuck.
*End of evening*

why the fuck those images wouldn't load? they come corrupt from K8S, but they are fine if I run the container locally, like... wtf? is Ingress NGINX doing something to them or did I configure something wrong?!

Fuck I feel fucked up just for completing user account management, authentication, email verification, password reset. Securing all of this with ssl and checking for any security loopholes.

I can't believe this took me more than a couple months.
Well I was lazy and unmotivated.
I fucking hate crafting stupid ass routes in nginx.
I fucking hate making a nice responsive gui.
I have to design even the stupid html for the emails. Fuuuuck.

So much boilerplate on top of that with username and email validation.

I learnt regex 5 times over the past couple months, still not enough.

And now I actually have to build the functional part.

On the plus side I can reuse this stupid boilerplate if I can make it more modular and readable.

There's shit ton of comments to the point where I feel like an idiot for including so much info. It's like I've written it for a toddler to take over.

Gawd. Anyways it's over now. 50% I guess.

I can finish the rest of the server more quickly and then spend another year designing the Android application.

I'm really lazy in places where I have to design UI/UX. Although at this point it's kinda what could put my application at the top. (I'm lazy, I ain't bad.. I just hate implementing my ideas I wish I could just visualize and have it appear on my screen)

I do like parts of gui that involve little math problems that would make motion smooth and efficient.

Looking to sharpen and pursue a SysAdmin/DevOps career, looking at online job offers to get the big picture of required skills and I say FUCK. It would take me a lifetime.
Azure, AWS, Google cloud platform.
CD tools: Ansible, Chef or Puppet
Scripting ninja with Python/Node and Shell/Power shell.
Linux & Windows administration
Mongo, MySQL and their relatives.
Networking, troubleshooting failure in disturbed systems
Familiarity with different stacks. Fuck. (Apache, nginx, etc..)
Monitoring infrastructure ( nagios, datadog .. )
CI tools: jenkins, maven, etc..
DB versioning: liquibase, flyway etc.
FUCK FUCK FUCK.
Are they looking for Voltron? FUCK YOU FROM THE DEEPEST LEVEL OF MY DEEP FUCK.

nginx microservices training through work?? fuck yea!

PISSED.

Fucking Docker, for no fucking reason (no updates, no changes, etc), I tried booting it up following the morning ritual, and nope, ERR_EMPTY_RESPONSE when connecting to my current project (means I managed to connect but for some reason no data is sent). Nginx container doesn't yell about anything.

Everything around works. Accessing the container works. Even pinging my dev domain works. Why the fuck suddenly fucking Docker just **stops**?!

Restarted Hyper-V, updated laradock, recreated containers, disabled AcrylicDNS. NOPE.

"Fuck you Phlisg, I'm not in the mood today" <lunatic Docker is lunatic>

ARG.

*The one where he breaks ssh*

TL;DR: Minikube's dick is too big, and my ass wasnt ready.

So there was a time about 2 weeks ago where i wanted to try and set up a minikube cluster using SOP, and that actually went okay, aside from having to move over to a completely different server after discovering that my processor doesn't support virtualization.

So i set it up on my other server, and everything immediately starts going to shit; i can no longer run commands without processor latency. Also top shows 200% CPU usage. Maybe i should stop... NAHHH... so i continue on, and the biggest fuck up was starting up the nginx pods. I have 6 of them, and the moment i try and stand up my custom container which was the WHOLE POINT of this whole exercise, i lose ssh access and cant get back in. I go over to the server and kill the minikube and virtualbox processes, and everything's back to normal.

Hugo framework:

1 hour to set it up
1 day to read the documentation
1 eternity to actually fucking deploy it

Was having problems on a VPN where my URL was constantly redirecting to https, after https was disabled, spent ages reconfiguring nginx, removing and adding nginx again with no luck. Eventually said fuck it, backed up everything of importance, destroyed the droplet and spun up a new one. Installed nginx and redone the DNS for the domain only for the same thing to happen. It was at that moment I discovered it was chrome caching the HSTS domain. I now have a long night ahead of me configuring the new droplet and restoring the backup data.

Fuck these apache2 and nginx configuration that fucking make my life miserable. Why can\'t I fucking edit the files inside /var/www/html? Fuck that. Why can\'t I access a subdirectory inside it? Fuck! In Node.js, you can fire up your server pretty easily. Then you tell us that JavaScript sucks? Fuck me dead.

Why can no-one, not one single solitary fucker, on StackOverflow get it through their thick skull that when I call PHP's http_response_code() or try to get $_SERVER["REDIRECT_STATUS"], I want the response code from Nginx? No, not Apache. No, I don't want to pass a status code FROM PHP TO NGINX, I want the response code. FROM Nginx. TO PHP.

In what fucking universe does PHP know more about the response code than Nginx? It doesn't. Nginx knows the response code, because that's the fucker that redirected to the error page. I want the error. Passed to the page. From Nginx. To PHP.

NO, http_response_code() DOES NOT MAGICALLY FUCKING WORK, IT RETURNS 200 BY DEFAUL- fuck it.

Today wasted around 5 hours installing nginx, apache stills working and keeping listen on 80 port after uninstalling it reboot finally the thing was i dont change the fucking dns of the server and trying to connect to the domain 5 hours later tried to connect to the ip... Fuck my life