I wrote a node + vue web app that consumes bing api and lets you block specific hosts with a click, and I have some thoughts I need to post somewhere.

My main motivation for this it is that the search results I've been getting with the big search engines are lacking a lot of quality. The SEO situation right now is very complex but the bottom line is that there is a lot of white hat SEO abuse.

Commercial companies are fucking up the internet very hard. Search results have become way too profit oriented thus unneutral. Personal blogs are becoming very rare. Information is losing quality and sites are losing identity. The internet is consollidating.

So, I decided to write something to help me give this situation the middle finger.

I wrote this because I consider the ability to block specific sites a basic universal right. If you were ripped off by a website or you just don't like it, then you should be able to block said site from your search results. It's not rocket science.

Google used to have this feature integrated but they removed it in 2013. They also had an extension that did this client side, but they removed it in 2018 too. We're years past the time where Google forgot their "Don't be evil" motto.

AFAIK, the only search engine on earth that lets you block sites is millionshort.com, but if you block too many sites, the performance degrades. And the company that runs it is a for profit too.

There is a third party extension that blocks sites called uBlacklist. The problem is that it only works on google. I wrote my app so as to escape google's tracking clutches, ads and their annoying products showing up in between my results.

But aside uBlacklist does the same thing as my app, including the limitation that this isn't an actual search engine, it's just filtering search results after they are generated.

This is far from ideal because filter results before the results are generated would be much more preferred.

But developing a search engine is prohibitively expensive to both index and rank pages for a single person. Which is sad, but can't do much about it.

I'm also thinking of implementing the ability promote certain sites, the opposite to blocking, so these promoted sites would get more priority within the results.

I guess I would have to move the promoted sites between all pages I fetched to the first page/s, but client side.

But this is suboptimal compared to having actual access to the rank algorithm, where you could promote sites in a smarter way, but again, I can't build a search engine by myself.

I'm using mongo to cache the results, so with a click of a button I can retrieve the results of a previous query without hitting bing. So far a couple of queries don't seem to bring much performance or space issues.

On using bing: bing is basically the only realiable API option I could find that was hobby cost worthy. Most microsoft products are usually my last choice.

Bing is giving me a 7 day free trial of their search API until I register a CC. They offer a free tier, but I'm not sure if that's only for these 7 days. Otherwise, I'm gonna need to pay like 5$.

Paying or not, having to use a CC to use this software I wrote sucks balls.

So far the usage of this app has resulted in me becoming more critical of sites and finding sites of better quality. I think overall it helps me to become a better programmer, all the while having better protection of my privacy.

One not upside is that I'm the only one curating myself, whereas I could benefit from other people that I trust own block/promote lists.

I will git push it somewhere at some point, but it does require some more work:
I would want to add a docker-compose script to make it easy to start, and I didn't write any tests unfortunately (I did use eslint for both apps, though).
The performance is not excellent (the app has not experienced blocks so far, but it does make the coolers spin after a bit) because the algorithms I wrote were very POC.
But it took me some time to write it, and I need to catch some breath.

There are other more open efforts that seem to be more ethical, but they are usually hard to use or just incomplete.

commoncrawl.org is a free index of the web. one problem I found is that it doesn't seem to index everything (for example, it doesn't seem to index the blog of a friend I know that has been writing for years and is indexed by google).

it also requires knowledge on reading warc files, which will surely require some time investment to learn.

it also seems kinda slow for responses,

it is also generated only once a month, and I would still have little idea on how to implement a pagerank algorithm, let alone code it.

Overengineering. Finding the right point between overdesign and no design at all. That's where fancy languages and unusual patterns being hit by real world problems, and you need to deal with all that utter mess you created being architecture astronaut. Isn't that funny how you realize that another fancy tool is fundamentally incompatible with the task you need to solve, and you realize it after a month of writing workarounds and hacks.

But on the other hand, duct tape slacking becomes a mess even quicker.

Not being able to promote projects. You may code the shit out of side project and still get zero response, absolutely no impact. That's why your side projects often becomes abandoned.

Oversleeping. You thought tomorrow was productive day, but you wake up oversleeped, your head aches, your mind is not clear and you be like "fuck that, I'm staying in bed watching memes all day". But there's job that has to be done, and that bothers you.

Writing tests. Oh, words can't describe how much I hate writing tests, any kind of. I tried testing so many times in high school, at university, even at production, but it seems like my mind is just doesn't accept it. I know that testing is fundamentally important, but my mind collapses every time I try to write a single fucking test, resulting in terrible headache. I don't know why it's like that, but it is, and I better repl the shit out of pure function than write fucking tests.

Not writing tests for your code is like leaving it unguided in the hands of future Devs who gonna just change something and break the core logic and there won't be any regression test to run and check the integrity.

Management has reviewed DevOps which is already being shown to customers and wants to introduce a code coverage metric to show customers how good the code is. Code coverage now has to be 80% or higher across the board, it's been decided by management, and there is no discussion on this topic. The front-end presentation layer has minimal business logic and most of the code that exists is generated objects to shift data to the backend (which in turn mostly shifts it to a 3rd party API that does a lot of the business logic). I now have to write thousands of pointless tests while also being 85% billable to the client.
I think code coverage can be used as a tool, but should not be used as a metric to promote the product.
I suggested an automated e2e test might be better placed to prove that things are working, but have been told we're not going in that direction.
I'm close to writing a code coverage test that uses reflection to touch all of the code, catch all exceptions, and return a positive result.

I’m currently working 2 jobs with over 60 hour work weeks in addition to my own SaaS company.

One job is full-time 40 hours, where I am a mid level developer and I just do the waterfall of tickets that is assigned to me. This place is unorganized and has almost no communication within the team.

The second job I am the Senior Dev and project lead. It’s a contract position that I put 20+ hours in on the evenings and weekends. Agile methodology, with a modern tech stack and I promote excellent communication as well as documenting everything.

I’m in a unique position because I’m able to see these differences and compare them side by side. My full-time job doesn’t really know about the second job. I get my work done, and that’s all that matters. This place is a mess. The project lead (CTO) is a helicopter boss that sticks his nose up at any type of formal documentation and practices. No tests are written.. no SIPs or deployment docs.. no stand ups or anything. I must also mention this team has 5 developers and a QA.. my team is only 2 developers and a QA. We get through tickets much faster.. it helps when I go over every single ticket that is created and add requirements and images..

I guess my point is... I’m about to be a full-time contractor because I can’t take this unprofessionalism anymore.

Just because these formalities technical take longer. It does decrease actual time spent developing a project. Spending a couple of hours on tests and requirements can save you days of back and forth in the future. Not to mention... document.. everything.

John Webb, MD Occupational Medicine: Employee Health Screenings, Occupational Health Management, and Work Injury Assessments in St. Louis, MO

At John Webb, MD Occupational Medicine, we understand the importance of maintaining the health and safety of your workforce. Located in Chesterfield, MO, and serving businesses in St. Louis, MO, our practice provides comprehensive employee health screenings, occupational health management, and work injury assessments. Dr. John Webb, a leading occupational health physician, is dedicated to ensuring that your employees stay healthy, safe, and fit for work. Our services are designed to help you manage workplace health risks, improve employee well-being, and maintain a productive work environment.

Employee Health Screenings in St. Louis, MO: A Proactive Approach to Workforce Wellness
One of the most effective ways to promote a healthy workforce is through regular employee health screenings. At John Webb, MD Occupational Medicine, we offer comprehensive employee health screenings in St. Louis, MO, to assess the overall health of your employees and detect potential health issues before they become serious. These screenings are an essential part of any occupational health program, as they help identify risk factors related to chronic conditions, infectious diseases, and workplace hazards.

Our employee health screenings can include evaluations for cardiovascular health, respiratory function, vision and hearing tests, blood pressure monitoring, and more. By conducting these assessments regularly, you can reduce the likelihood of illness and injury, minimize absenteeism, and improve overall productivity. Early detection allows for timely intervention and treatment, helping your employees maintain optimal health and continue to perform their best.

Occupational Health Management in St. Louis, MO: Ensuring Workplace Health and Safety
Occupational health management is a critical component of ensuring the overall health and safety of your employees. At John Webb, MD Occupational Medicine, we offer expert occupational health management services tailored to the unique needs of your business in St. Louis, MO. Whether you are looking to develop a comprehensive health and safety program, ensure compliance with local regulations, or manage the health risks within your workplace, we are here to help.

Our occupational health management services include a range of solutions designed to prevent injuries and illnesses, reduce workplace hazards, and improve the health of your workforce. We offer advice on ergonomic practices, health and safety protocols, employee wellness programs, and more. Our goal is to create a workplace that prioritizes the health and well-being of employees while ensuring that your business operates efficiently and meets industry standards.

With our occupational health management expertise, we help businesses reduce the costs of workplace injuries, minimize downtime, and ensure that your employees are fit to perform their job duties safely. We also provide guidance on implementing OSHA compliance and workers' compensation programs to protect both your employees and your business.

Work Injury Assessments in St. Louis, MO: Expert Evaluation and Care
Workplace injuries can happen, but the right care and attention can minimize the impact on both the employee and the business. At John Webb, MD Occupational Medicine, we specialize in work injury assessments in St. Louis, MO, providing thorough evaluations of injuries that occur on the job. Whether it’s a minor sprain or a more severe injury, we offer the expertise necessary to assess, diagnose, and create a comprehensive treatment plan to ensure employees can return to work safely and efficiently.

Our work injury assessments involve a thorough evaluation of the injury, including physical exams, diagnostic tests, and a review of the work environment to determine the cause. We provide immediate care and offer a tailored rehabilitation plan, ensuring that employees recover fully before returning to their duties. Additionally, we work with employers to develop return-to-work programs that minimize the risk of re-injury and ensure employees’ readiness to resume their roles safely.

By providing work injury assessments, we help prevent further complications, reduce recovery time, and ensure that workplace injuries are managed in accordance with industry best practices.

Why Choose John Webb, MD Occupational Medicine?
Comprehensive Employee Health Screenings: Our employee health screenings are designed to detect early signs of health issues, improve employee well-being, and reduce healthcare costs.