Like most people I needed some extra cash during uni, so I proceeded to learn CSS + Photoshop (yeah, I know). Followed by PHP and WordPress.

It can be a very shitty platform until you realize that you can stop combining plug-ins from all over the place with dubious code quality and roll your own.

Anyhow I kept at it until I was able to join a niche company doing a quite popular caching plug-in for WP (yeah, W3 Total) when I suddenly became *very* interested in anything and everything performance.

This landed me a very cozy consulting gig in the Nordics - they were using WP for an elephant-traffic website and had run into a myriad of perf issues.

Fixing them and breaking the monolith awarded me with skills in nodejs, linux, asynchronous caching among others.

I was soon in charge with managing the dev boxes for the entire team, and when the main operations dude left, I was promoted to owning the entire platform. (!) Tinkering with Linux for most of my life really came in handy here. (remember Debian potato?)

Used saltstack + aws cloudformation to achieve full parity between all environments. Learned myself some python and all various tips and tricks which in the end amounted to 90% reduction in time-to-first-byte and considerable cost savings.

By the end of the 2yr contract I had turned myself into a fullstack systems engineer and never looked back.

Lawyers not getting along resulted in us having to abandon NewRelic, so I got to learn and deploy the ELK stack as a homegrown replacement, which was super-fun.

Now I work in the engineering effectiveness department of a Swedish fintech unicorn where all languages under the Sun are an option (tho we prefer Python), so the tech stack is unlimited. Infinite tools and technologies, but with strong governing principles and with performance always in mind so as to pick the right tool for the job.

It's like that childhood feeling when you've just dumped a ton of Lego on the floor and are about to build something massive.

I guess the morale here is however disappointed you feel by your current stack - don't. Always strive to make things better, faster, more decoupled, easier to test, etc. and always challenge yourself to go outside the comfort zone.

Fuck this Kibana shit and give me back my old grep (or even better: ripgrep). In 2008, I used to find shit in my fucking logfiles. Now I have an ELK stack that smells like liquid shit.

If you ever want to know how NOT to create documentation, try installing Elastic Search for a server.

Interviewer: What is ELK stack?
Me: It's a data structure which follows first in last out.
Interviewer: .......

Follow up sorta...

So I got pulled into a support issue on a day off. Some system was facing timeouts on our servers so had to investigate.

Over the weekend as part of the release, I released the ELK stack I built and today I used that to help.

Pretty much immediately pinpointed which machine was hanging though still had to investigate and confirm so split between KQL and checking the server logs.

One thing I've always griped about is how no one created schema docs for it mongo collections so can't easily figure out what they do or your to get the document needed.

Well guess it's my turn.... Because only I know the schema :)

Hi fellow devRanters, I need some advice on how to detect web traffic coming from bad/malicious bots and block them.

I have ELK (Elastic) stack set up to capture the logs from the sites, I have already blocked the ones that are obviously bad (bad user-agent, IP addresses known for spamming etc). I know you can tell by looking at how fast/frequently they crawl the site but how would I know if I block the one that's causing the malicious and non-human traffic? I am not sure if I should block access from other countries because I think the bots are from local.

I am lost, I don't know what else I can do - I can't use rate limiting on the sites and I can't sign up for a paid service cause management wants everything with the price of peanuts.

Rant:

Someone asked why I can't just read through the logs (from several mid-large scale websites) and pick out the baddies.

*facepalm* Here's the gigabytes log files.