Boss"So, we need to get some data about the users using the APIs from this list of sites."
Me"Alright, sounds feasible enough"

Navigating to first site.

M"Hold on, where's the API?"
B"What do you mean? You're looking at it."
M"This is a website with a search bar, not an API"
B"Same thing. Get to scrapping that data."
M"I-It's written in a JS framework to be reactive in a half-assed way."
B"We need that data"
M"The data is not even consistent!"
B"That's why we need to join it with all these different sources."

The API was a lie. None of the sites had anything remotely similar to an API.

Having to use bloody selenium with chrome driver to scrap all the information because of course, it has to be done programatically every week from now on.

I just hope no captcha of any kind is installed before I finish this project.

Made a mistake in scrapping project, realized it AFTER it spent 3 days running, now I relaunched it (corrected) with 4 times more workers to meet the deadline

When did you scrap your project and start over, and why?

I'm working on a second project at my company.

But as the project got longer and initially unplanned features started to pile up, the software has become a total mess and a pile of conversion layers for all sorts of I/O.

I'm starting to believe that it'd be better to learn from our mistakes and start over with a better plan, if it means we'll be able to achieve performance and stability improvements.

So, what was it like scrapping your project and starting over?

I am working on a pub-sub based protocol (like MQTT) with some added features. I am developing a python based server for my protocol which can be run on distributed architecture with load balancing without any tweaks. I am planning to make this server and the protocol open source.

The whole thing is getting so complex that I think about scrapping this project sometimes. I need your inspiration guys. Really, I need it. I know this protocol will be good enough to help people working on IoT, chat or any pub-sub based application if I can complete it. Cheer me up, please.