GOD FUCKIN DAMMIT
I WILL FUCKIN KICK YOU ON YOUR FUCKING THROAT.

Programming Languages and Linux groups in facebook are a fuckin pain to watch.

Some people make groups so all can benefit and help each other, talk about mutual interests, BUT NO SOME FUCKERS WILL SPAM SHIT AND MAKE YOU WANNA SMACK THEIR FUCKIN HEAD.

THERE IS A FUCKIN FAQ SECTION THAT ANSWERS ALL THE FUCKIN NEWBIE QUESTIONS. WHY THE FUCKIN HELL YOU SPAM IF YOU HAVE NO FUCKIN CLUE WHAT THE HELL YOU ARE DOING?

You come to a python group and ask if it's possible to get context from a site. I'M NOT MENTIONING THE FUCKIN FACT THAT THIS IS A SIMPLY FUCKIN QUERY TO A SEARCH ENGINE ALSO IT'S MENTIONED IN THE FUCKIN FAQ. Let's move on. We tell you yes, there is BeautifulSoup for that. After 5 fuckin mins YOU COME AND MAKE A NEW POST THAT SHOWS YOU CANT FUCKIN ITERATE A GODDAMN FUCKIN LIST. I'm not pro either, i don't forbid you to learn, BUT FUCKIN LEARN THE BASICS THAT ARE PROVIDED TO YOU FROM GREAT FUCKIN RESOURCES BEFORE TRYING TO ATTEMPT SOMETHING MORE COMPLICATED. AND IF YOU NEED HELP PROVIDE CODE THAT WE CAN USE. NOT A FUCKIN PHOTOGRAPH FROM YOUR MOBILE

Let's go on the Linux groups.
SINCE YOU FUCKIN JOIN A LINUX GROUP YOU ARE SUPPOSED TO KNOW WHAT THE FUCK IS LINUX. IT'S A FUCKIN OPERATING SYSTEM RIGHT?
Then you spam shit like, UBUNTU OR MINT 5 MINUTES AFTER SOMEONE ELSE MADE THE SAME VERY QUESTION 30 MINS AGO. WHICH WAS ANSWERED AGAIN YESTERDAY.

"What are the benefits of Linux". NONE YOU TWAT, IF YOU NEED ME TO TELL YOU THE BENEFITS OF THE SYSTEM THAT YOU USE THEN WHY THE HELL YOU BOTHER.

Next.
You say you have problems setting up XAMPP. We tell you that since you are on linux better use LAMP. You ignore us and spam your fuckin problem with XAMPP. IM GONNA FIND YOU AND IM GONNA MAKE YOU CHEW MY FUCKIN SHOES YOU PIECE OF SHIT.

I'm not even mentioning the kali wannabe hackers.

Conclusion:
DO A FUCKIN SMALL RESEARCH BEFORE SPAMMING THE SHIT OUT OF STUPID FUCKIN QUESTIONS. AND IF YOU CANT EVEN SEARCH, LEARN TO ASK IN ENGLISH THAT IS FUCKIN UNDERSTANDABLE SO SOMEONE CAN GUIDE YOU ABOUT WHAT YOU SHOULD SEARCH

OH FUCKIN GAWD IM GONNA THROW MY LAPTOP OUT OF THE WINDOW

Ok, do you sometimes feel like you get treated as search engine? I'm pretty sure I'm not the only one.

My coworker from my previous workplace is messaging me to suggest about good tutorial about a framework that I'm not familiar about.

"Dude, have you heard about Google?"

Sorry for being late, stuffs came inbetween!

I have done a few privacy rants/posts before but why not another one. @tahnik did one a few days ago so I thought I'd do a new one myself based on his rant.

So, online privacy. Some people say it's entirely dead, that's bullshit. It's up to an individual, though, how far they want to go as for protecting it.

I personally want to retain as much control over my data as possible (this seems to be a weird thing these days for unknown reasons...). That's why I spend quite some time/effort to take precautions, read myself into how to protect my data more and so on.

'Everyone should have the choice of what services they use' - fully agreed, no doubt about that.

I just find one thing problematic. Some services/companies handle data in a way or have certain business models which takes the control which some people want/have over their data away when you communicate with someone using that service.

Some people (like me) don't want anything to do with google but even when I want to email my best fucking friend, I lose the control over that email data since he uses gmail.

So, when someone chooses to use gmail and I *HAVE* to email them, my choice is gone.

TO BE VERY CLEAR: I'm not blaming that on the users, I'm blaming that on the company/service.

Then for example, google analytics. It's a very good/powerful when you're solely looking at its functions.
I just don't want to be part of their data collection as I don't want to get any data into the google engine.

There's a solution for that: installing an addon in order to opt out.

I'm sorry, WHAT?! --> I <-- have to install an addon in order to opt out of something that is happening on my own motherfucking computer?! What the actual fuck, I don't call that a fucking solution. I'll use Privacy Badger + hosts files to block that instead.

Google vs 'privacy' friendly search engines - I don't trust DDG completely because their backend is closed/not available to the public but I'd rather use them then a search engine which is known to be integrated into PRISM/other surveillance engines by default.

I don't mind the existence of certain services, as long as they don't integrated you with data hungry companies/mass surveillance without you even using their services.

Now lets see how fast the comment section explodes!

I bet this guy feels real fuckin helpful, editing not only the question, but literally every answer in the entire thread. His contributions include:

Capitalizing names.
Adding italics to links (Really, he took the time and clicked "edit" just to do that to an answer)
Adding the word "The"

The fact that people spend their time religiously doing this makes my head spin.

Also aren't these edits bad? the title should be in google speak (i.e. short and broken English) so that they can be found from a search engine. I'm sure SO has some rules about the title or something, but I feel like it would be better in google speak.

Friend: 'I have an idea you could totally do: a search engine for travellers that searches hostels, hotels, flights and resteraunts'
Me: 'yeahh I'm currently working on an e-commerse website. Should keep me.busy for a while'
Friend: 'integrate it into your website, it'd be silly not to'

Do people think we just click 'integrate idea' button for a living??!
Seriously the fux

Chrome, Firefox, and yes even you Opera, Falkon, Midori and Luakit. We need to talk, and all readers should grab a seat and prepare for some reality checks when their favorite web browsers are in this list.

I've tried literally all of them, in search for a lightweight (read: not ridiculously bloated) web browser. None of them fit the bill.

Yes Midori, you get a couple of bonus points for being the most lightweight. Luakit however.. as much as I like vim in my terminal, I do not want it in a graphical application. Not to mention that just like all the others you just use webkit2gtk, and therefore are just as bloated as all the others. Lightweight my ass! But programmable with Lua, woo! Not like Selenium, Chrome headless, ... does that for any browser. And that's it for the unique features as far as I'm concerned. One is slow, single-threaded and lightweight-ish (Midori) and another has vim keybindings in an application that shouldn't (Luakit).

Pretty much all of them use webkit2gtk as their engine, and pretty much all of them launch a separate process for each tab. People say this is more secure, but I have serious doubts about that. You're still running all these processes as the same user, and they all have full access to the X server they run under (this is also a criticism against user separation on a single X session in general). The only thing it protects against is a website crashing the browser, where only that tab and its process would go down. Which.. you know.. should a webpage even be able to do that?

But what annoys me the most is the sheer amount of memory that all of these take. With all due respect all of you browsers, I am not quite prepared to give 8 fucking gigabytes - half the memory in this whole box! - just for a dozen or so tabs. I shouldn't have to move my web browser to another lesser used 16GB box, just to prevent this one from going into fucking swap from a dozen tabs. And before someone has a go at the add-ons, there's 4 installed and that's it. None of them are even close to this complete and utter memory clusterfuck. It's the process separation. Each process consumes half a GB of memory, and there's around a dozen of them in a usual browsing session. THAT is the real problem. And I want to get rid of it.

Browsers are at their pinnacle of fucked up in my opinion, literally to the point where I'm seriously considering elinks. Being a sysadmin, I already live my daily life in terminals anyway. As such I also do have resources. But because of that I also associate every process with its cost to run it, in terms of resources required. Web browsers are easily at the top of the list.

I want to put 8GB into perspective. You can store nearly 2 entire DVD movies in that memory. However media players used to play them (such as SMPlayer) obviously don't do that. They use 60-80MB on average to play the whole movie. They also require far less processing power than YouTube in a web browser does, even when you download that exact same video with youtube-dl (either streamed within the media player or externally). That is what an application should be.

Let's talk a bit about these "complicated" websites as well. I hate to break it to you framework web devs, but you're a dime a dozen. The competition is high between web devs for that exact reason. And websites are not complicated. The document itself is plain old HTML, yes even if your framework converts to it in the background. That's the skeleton of your document, where I would draw a parallel with documents in office suites that are more or less written in XML. CSS.. oh yes, markup. Embolden that shit, yes please! And JavaScript.. oh yes, that pile of shit that's been designed in half a day, and has a framework called fucking isEven (which does exactly what it says on the tin, modulo 2 be damned). Fancy some macros in your text editor? Yes, same shit, different pile.

Imagine your text editor being as bloated as a web browser. Imagine it being prone to crashing tabs like a web browser. Imagine it being so ridiculously slow to get anything done in your productivity suite. But it's just the usual with web browsers, isn't it? Maybe Gopher wasn't such a bad idea after all... Oh and give me another update where I have to restart the browser when I commit the heinous act of opening another tab, just because you had to update your fucking CA certs again. Yes please!

I wrote a node + vue web app that consumes bing api and lets you block specific hosts with a click, and I have some thoughts I need to post somewhere.

My main motivation for this it is that the search results I've been getting with the big search engines are lacking a lot of quality. The SEO situation right now is very complex but the bottom line is that there is a lot of white hat SEO abuse.

Commercial companies are fucking up the internet very hard. Search results have become way too profit oriented thus unneutral. Personal blogs are becoming very rare. Information is losing quality and sites are losing identity. The internet is consollidating.

So, I decided to write something to help me give this situation the middle finger.

I wrote this because I consider the ability to block specific sites a basic universal right. If you were ripped off by a website or you just don't like it, then you should be able to block said site from your search results. It's not rocket science.

Google used to have this feature integrated but they removed it in 2013. They also had an extension that did this client side, but they removed it in 2018 too. We're years past the time where Google forgot their "Don't be evil" motto.

AFAIK, the only search engine on earth that lets you block sites is millionshort.com, but if you block too many sites, the performance degrades. And the company that runs it is a for profit too.

There is a third party extension that blocks sites called uBlacklist. The problem is that it only works on google. I wrote my app so as to escape google's tracking clutches, ads and their annoying products showing up in between my results.

But aside uBlacklist does the same thing as my app, including the limitation that this isn't an actual search engine, it's just filtering search results after they are generated.

This is far from ideal because filter results before the results are generated would be much more preferred.

But developing a search engine is prohibitively expensive to both index and rank pages for a single person. Which is sad, but can't do much about it.

I'm also thinking of implementing the ability promote certain sites, the opposite to blocking, so these promoted sites would get more priority within the results.

I guess I would have to move the promoted sites between all pages I fetched to the first page/s, but client side.

But this is suboptimal compared to having actual access to the rank algorithm, where you could promote sites in a smarter way, but again, I can't build a search engine by myself.

I'm using mongo to cache the results, so with a click of a button I can retrieve the results of a previous query without hitting bing. So far a couple of queries don't seem to bring much performance or space issues.

On using bing: bing is basically the only realiable API option I could find that was hobby cost worthy. Most microsoft products are usually my last choice.

Bing is giving me a 7 day free trial of their search API until I register a CC. They offer a free tier, but I'm not sure if that's only for these 7 days. Otherwise, I'm gonna need to pay like 5$.

Paying or not, having to use a CC to use this software I wrote sucks balls.

So far the usage of this app has resulted in me becoming more critical of sites and finding sites of better quality. I think overall it helps me to become a better programmer, all the while having better protection of my privacy.

One not upside is that I'm the only one curating myself, whereas I could benefit from other people that I trust own block/promote lists.

I will git push it somewhere at some point, but it does require some more work:
I would want to add a docker-compose script to make it easy to start, and I didn't write any tests unfortunately (I did use eslint for both apps, though).
The performance is not excellent (the app has not experienced blocks so far, but it does make the coolers spin after a bit) because the algorithms I wrote were very POC.
But it took me some time to write it, and I need to catch some breath.

There are other more open efforts that seem to be more ethical, but they are usually hard to use or just incomplete.

commoncrawl.org is a free index of the web. one problem I found is that it doesn't seem to index everything (for example, it doesn't seem to index the blog of a friend I know that has been writing for years and is indexed by google).

it also requires knowledge on reading warc files, which will surely require some time investment to learn.

it also seems kinda slow for responses,

it is also generated only once a month, and I would still have little idea on how to implement a pagerank algorithm, let alone code it.

!dev

I have a couple of thoughts about social justice controversies from these last years.

I think it's hard to have a good opinion about these events for several reasons.

One reason is that finding good information in 2019 is very hard.

Revenue based sites (thus unneutral) dominate the search results. You search about something and you find thousands of sites basically saying the same thing (because they copy each other).

That's why the existence of a free and open search engine is so important, so it's easier to find neutral hence good information on which to base your opinions, but they are prohibitively big for small groups to build.

Another reason is that controversies generate shock and shock curtails rational thinking. Maybe that's how the primitive brain works?

I'm not much of a scholar to feel confident to say that, but it's so recurrent that it's not too much of a wild guess.

When a controversy happens, a natural reaction is to pick a side. This means that:
a) we assume that there are only 2 sides, and
b) we must pick one of them

So, maybe the human is a bad politician by nature?

Also, because of the shock controversies generate, peaceful dialogue is very rare.

I have yet to see peaceful dialogue online about what patriarchy means to feminists and a lot of other terms they use.

I don't care much about feminists that vandalize or interrupt talks (yelling over someone else is abuse in my opinion).

But for the rest of them, I think discussing their ideas would be good.

I say this because most feminist discourse I see online is not open. Or maybe there are such instances but the web is so big that it's hard to find such instances.

I think some part of the modern feminist doctrine is bullshit, and some part is true.

I for one hate when some men I know in life expect their wives to be their cooks+cleaners (unless they want to do that, willingly). Personally, I'd encourage my wife to get a job (rightfully so, not just to meet some minority quota in some company).

I don't mind either calling a trans person the pronoun she wants.

But other ideas are awful, like the idea that meritocracy is patriarchy, so you need to force minorities to meet a proportionate quota. That's terrible reasoning.

Or the excessive self appreciation culture, like saying to yourself "you are pretty, you are beautiful, you are perfect". I think that grows arrogance and black-or-white thinking.

And some other ideas as well.

I guess the same you can say about any doctrine with different degrees. Some part is bullshit, some part isn't.

Some right wing people hate everyone who isn't white by default, but some want to have more immigration control.

I sure don't like the experiment of separating children from families like the current us govt did, but I wouldn't be happy either to know that by '99 50% of gangs members in the us were hispanic.

With this, I'm not going to say "embrace everyone's ideas" like an idiot. I hate when people do that. It's a stupid and weak reaction to radicalism.

In fact I think the way you fight radicalism and bad doctrines is that you listen to them and maintain good dialogue and counterargue in a respectful but insightful manner.

Making snide remarks, insulting or trolling won't change anyone's mind. That is just throwing fire to the fire.

In fact, when someone gets harassed because of something they believe in, usually it results in even more adherence to their beliefs, because of the usual assumption that success or goodness is full of strife.

So by telling a "sjw" or kkk member that they are idiots over twitter, you are in fact making them stronger believers in their doctrine.

Think of Daryl Davis, a black guy that made 200 members leave the kkk. How? He didn't tell them they were assholes, he somehow made friends with them.

I feel bad now because I've been trolling new devrant users a lot because of how they worsen the quality of the site, but maybe I should tell them that they are ruining the site somehow in a nice way and maybe they'll listen? I dunno...

kinda long but please read (skip to the bullets if you're lazy):

hey dR. I stumpled across a search engine that aims to help the environment. it's called "Ecosia" and it will plant a tree for every ~45 searches you make. just think, one stubborn bug could make you the reason for a new forest! I'm not sure if it's legitimate or not, but apparently it uses 80% of its profits to plant trees, and makes that profit from ads. is it safe to use? I'm not sure.

here is what you should know (some are based on claims by ecosia and aren't proved, but probably true):
- they plant a tree for about every 45 searches you make
- they are able to plant trees by using money from ads
- they "respect privacy"
- they're "fully transparent"
- they're a "social buisness"
- [I hope this isn't a turnoff] the search results are powered by bing
- since 1.9, vivaldi has included ecosia as one of the preset search engines (I'm not sure if it's the default)
- it has opera, firefox, and chrome extensions

thanks!

The Google timeline:
- early 2000: "I use it because it's the best search engine out there"
- mid 2000: "I use it because it's a full ecosystem for mail and office and it's free"
- early 2010: "I use it because it integrates seamlessly with my phone"
- mid 2010: "I use it because it's at least better than the competition"
- now: "I'm willing to try literary anything that's not Google"

C'mon, really?

Okay, I understand that they want to lock down the Chromebooks they send home with us, we don't own them and they have the right to do that. But I'm still annoyed when I find "harmless" stuff is blocked.

They said it themselves that they want us to be able to do basically anything we want web browsing wise on them.

It's not a fun experience to say to your self "hey let's look at the current humble bundles!" just to find that humblebundle.com is blocked for "games". (Which makes sense, but I can't remember any other examples)

Imagine thinking to yourself "I'm going to go to the Os Dev Wiki" and typing that into the Omnibox (tm) and pressing enter, directing you to your favorite search engine duckduckgo, but instead of finding the amazing duckduckgo results page you find the godforsaken securly "THIS PAGE IS BLOCKED" screen.

I can guess why they do that (probably because, to my knowledge, duckduckgo doesn't have any form of "safe-search" feature they can force it to use because they do that) but it's kind of annoying to not be able to use your favorite search engine anymore.

Should I really be getting so annoyed at this? No, because it's not my device, it's theirs and, they have the final say on what goes, but sometimes it really annoys me. I should be, and am, thankful they even let us bring the Chromebooks home, which is pretty cool.

Ugh...

If you want a fun time, just read the reviews on the Securly extension in the chrome web store!

Warning long rambling story cause sleep deprivation

I never really bothered with ssh outside of using putty to remote into my servers and rpi's from my desktop to run updates, install something, or whatever else.

But today I was on a call with my cousin bored cause she was just rambling, so I opened vscode to clean my install of unnecessary extensions I installed and haven't used more than once or twice.

I saw Remote - SSH and as I was bored listening to a teenager complain about high school just like I used to (lol) and responding when she asked me something. I scrolled through the page, then the documentation just casually skimming the text

I setup an ssh key on an rpi I threw manjaro arm following the instructions on their tips and tricks page
I then moved the key to my desktop using winscp (cause lazy)
leading to having a minor hicup of rsa not being an accepted keytype (thanks 'your favorite search engine' for the help)
Finally, I was able to connect using the private key

at this point my cousin went to bed cause she has school tomorrow. But I was still doing stuff with ssh, I created a new ssh connection in VSCode, but had to go to the documentation to figure out how to make it use my fancy new key file, not hard took 30 seconds of looking to get it working.

Now that I was in, I moved to my development folder, created a folder for PiHole, created a compose yml, created a pihole-data folder.
I opened the yml and pasted in a compose from dockerhub.

at this point I thought 'i can't just run this from terminal can I'. and Obviously it worked cause there's literally no reason it wouldn't I'm just stupid to think it might not.

So I created folders and files on a remote system, launched a docker container, checked for package updates after on a linux machine. All from VS-Code on a windows machine.

I know this is simple for some people, i know some people are like 'where's the interesting part'. but ehhh I thought it was cool to get it setup, I now really regret not getting into ssh sooner, and I'm definitely going to uninstall vscode on all my smaller graphical VM's in favor of doing this. and this will definitely help with my headless vm's.

I also will have to thank my cousin, might not have done this if I wasn't stuck at my computer on messenger call with her lol

I'm gonna go to bed now, But I feel accomplished for the first time in a while even if it's for something so simple as setting up anssh key for the first time

1) Learning little to nothing useful in formal post-secondary and wasting tons of time and money just to have pain and suffering.

"Let's talk about hardware disc sectors divisions in the database course, rather than most of you might find useful for industry."

"Lemme grade based on regurgitating my exact definitions of things, later I'll talk about historical failed network protocols, that have little to no relevance/importance because they fucking lost and we don't use them. Practical networking information? Nah."

"Back in the day we used to put a cup of water on top of our desktops, and if it started to shake a lot that's how you'd know your operating system was working real hard and 'thrashing' "

"Is like differentiation but is like cat looking at crystal ball"

"Not all husbands beat their wives, but statistically...." (this one was confusing and awkward to the point that the memory is mostly dropped)

Streams & lambdas in java, were a few slides in a powerpoint & not really tested. Turns out industry loves 'em.

2) Landed my first student job and get shoved on an old legacy project nobody wants to touch. Am isolated and not being taught or helped much, do poorly. Boss gets pissed at me and is unpleasant to work with and get help from. Gets to the point where I start to wonder if he starts to try and create a show of how much of a nuisance I am. He meddle with some logo I'm fixing, getting fussy about individual pixels and shades, and makes a big deal of knowing how to use GIMP and how he's sitting with me micromanaging. Monthly one on one's were uncomfortable and had him metaphorically jerking off about his lifestory career wise.

But I think I learned in code monkey industry, you gotta be capable of learning and making things happen with effectively no help at all. It's hard as fuck though.

3) Everytime I meet an asshole who knows more and accomplish than I do (that's a lot of people) with higher TC than me (also a lot of people). I despair as I realize I might sound like that without realizing it.

4) Everytime I encounter one of my glaring gaps in my knowledge and I'm ashamed of the fact I have plenty of them. Cargo cult programming.

5) I can't do leetcode hards. Sometimes I suck at white board questions I haven't seen anything like before and anything similar to them before.

6) I also suck at some of the trivia questions in interviews. (Gosh I think I'd look that up in a search engine)

7) Mentorship is nigh non-existent. Gosh I'd love to be taught stuff so I'd know how to make technical design/architecture decisions and knowing tradeoffs between tech stack. So I can go beyond being a codemonkey.

8) Gave up and took an ok job outside of America rather than continuing to grind then try to interview into a high tier American company. Doubtful I'd ever manage to break in now, and TC would be sweet but am unsure if the rest would work out.

9) Assholes and trolls on stackoverflow, it's quite hard to ask questions sometimes it feels and now get closed, marked as dupe, or downvoted without explanation.

Imagine a web way ahead of our time where its size goes beyond our imagination...

This is my first rant, and I'll cut to the chase! I don't like how web currently stands. Here's what makes me angry the most altough I know there's a myriad of solutions or workarounds:

- A gazillion credentials/accounts/services in your lifetime.

- Everyone tries to reinvent the wheel.

- There's no single source of truth.

- Why the fuck there's so much design in a vision that started as a network of documents? Why is it that we need to spend time and energy to absorb the page design before we can read what we are after?

- What's up with the JS front end frameworks?! MB's of code I need to download on every page I visit and the worse is the evaluation/parsing of it. Talk about acessibility and the energy bills. I don't freaking need a SPA just give a 20-50ms page load and I'm good to go!

- I understand that there's a whole market based on it but do we really need all that developer tools and services?

- Where's our privacy by the way? Why the fuck do I need ads? Can't I have a clue about what I wan't to buy?

Sticking with this points for now... Got plenty more to discuss though.

What I would like to see:

A unique account where i can subscribe services/forums/whatever. No credentials. Credentials should be on your hardware or OS. Desktop Browser and mobile versions sync everything seemlesly. Something like OpenID.

Each person has his account and a profile associated where I share only what I want with whom I want when I want to.

Sharing stuff individually with someone is easy and secure.

There's no more email system like we know. Email should be just email like it started to be. Why the hell are we allowing companies to send us so much freaking "look at me now, we are awesome", "hey hey buy from me".. Here's an idea, only humans should send emails. Any new email address that sends you an email automatically requests your "permission" to communicate with you. Like a friend request.

Oh by the way did I tell you that static mail is too old for us? What we need is dynamic email. Editing documents on the fly, together, realtime, on the freaking email. Better than mail, slack and google docs combined.

In order for that to work reasonably well, the individual "letter" communication would have to be revamped in a new modern approach.

What about the single source of truth I talked about? Well heres what we should do. Wikipedia (community) and Larry Page (concept) gave us tremendous help. We just need to do better now.

Take the spirit of wikipedia and the discoverability that a good search engine provides us and amp that to a bigger scale. A global encyclopedia about everything known to mankind. Content could be curated from us all just like a true a network.

In this new web, new browser or whatever needed to make this happen I could save whatever I want, notes, files, pictures... and have it as I left it from device to device.

Oh please make web simple again, not easy just simple and bigger.

I'm not old by the way and I don't see a problem with being older btw.

Those are just my stupid rants and ideas. They are worth nothing. What I know for sure is that I'll do something about or fail trying to.

I'm very sad.

I don't pretend to work on the next Facebook, Google search engine or something else.

I would to be part of something useful.

But i work in a shitty company where quality, architecture planning and TDD are underrated.

Only to build very simple webapplications, where things you take for granted like server side input or a simple error page without java stacktrace are missing or not planned properly.

We have functional analysts, but worst specs ever.

I hate all of this...

A very long rant.. but I'm looking to share some experiences, maybe a different perspective.. huge changes at the company.

So my company is starting our microservices journey (we have a 359 retail websites at this moment)

First question was: What to build first?

The first thing we had to do was to decide what we wanted to build as our first microservice. We went looking for a microservice that can be used read only, consumers could easily implement without overhauling production software and is isolated from other processes.

We’ve ended up with building a catalog service as our first microservice. That catalog service provides consumers of the microservice information of our catalog and its most essential information about items in the catalog.

By starting with building the catalog service the team could focus on building the microservice without any time pressure. The initial functionalities of the catalog service were being created to replace existing functionality which were working fine.

Because we choose such an isolated functionality we were able to introduce the new catalog service into production step by step. Instead of replacing the search functionality of the webshops using a big-bang approach, we choose A/B split testing to measure our changes and gradually increase the load of the microservice.

Next step: Choosing a datastore

The search engine that was in production when we started this project was making user of Solr. Due to the use of Lucene it was performing very well as a search engine, but from engineering perspective it lacked some functionalities. It came short if you wanted to run it in a cluster environment, configuring it was hard and not user friendly and last but not least, development of Solr seemed to be grinded to a halt.

Elasticsearch started entering the scene as a competitor for Solr and brought interesting features. Still using Lucene, which we were happy with, it was build with clustering in mind and being provided out of the box. Managing Elasticsearch was easy since there are REST APIs for configuration and as a fallback there are YAML configurations available.

We decided to use Elasticsearch since it provides us the strengths and capabilities of Lucene with the added joy of easy configuration, clustering and a lively community driving the project.

Even bigger challenge? Which programming language will we use

The team responsible for developing this first microservice consists out of a group web developers. So when looking for a programming language for the microservice, we went searching for a language close to their hearts and expertise. At that time a typical web developer at least had knowledge of PHP and Javascript.

What we’ve noticed during researching various languages is that almost all actions done by the catalog service will boil down to the following paradigm:

- Execute a HTTP call to fetch some JSON

- Transform JSON to a desired output

- Respond with the transformed JSON

Actions that easily can be done in a parallel and asynchronous manner and mainly consists out of transforming JSON from the source to a desired output. The programming language used for the catalog service should hold strong qualifications for those kind of actions.

Another thing to notice is that some functionalities that will be built using the catalog service will result into a high level of concurrent requests. For example the type-ahead functionality will trigger several requests to the catalog service per usage of a user.

To us, PHP and .NET at that time weren’t sufficient enough to us for building the catalog service based on the requirements we’ve set. Eventually we’ve decided to use Node.js which is better suited for the things we are looking for as described earlier. Node.js provides a non-blocking I/O model and being event driven helps us developing a high performance microservice.

The leap to start programming Node.js is relatively small since it basically is Javascript. A language that is familiar for the developers around that time. While Node.js is displaying some new concepts it is relatively easy for a developer to start using it.

The beauty of microservices and the isolation it provides, is that you can choose the best tool for that particular microservice. Not all microservices will be developed using Node.js and Elasticsearch. All kinds of combinations might arise and this is what makes the microservices architecture so flexible.

Even when Node.js or Elasticsearch turns out to be a bad choice for the catalog service it is relatively easy to switch that choice for magic ‘X’ or component ‘Z’. By focussing on creating a solid API the components that are driving that API don’t matter that much. It should do what you ask of it and when it is lacking you just replace it.

Many more headaches to come later this year ;)

Is there a search engine indexing pages that work without JavaScript?
Why?
- I use the Lynx text-mode browser
Why else? Maybe I'm naïve:
- At least without JavaScript the advert/tracking methods will not slow load times or break the page.
- This may be a nice way to highlight websites that don't have time to join an SEO/analytics arms race.