4
Froot
6y

So I was working on a web scraper to basically download all listings with detailed info from a e-shop to my database for some analysis.
And I completely forgot throttling which is quite important when writing such things in node.js.
It's funny how in other languages you try to figure out how to make your application faster and in node you're trying to make it slower 😄

Anyhow, I apparently hit the poor site with 5000+ simultaneous requests, all of which hit their database (to gather product info). Suffice to say, the site got visibly slow 🤣

Thankfully I print out where each request is made so I quickly realised my mistake and killed the process.

Now I hope no-one comes knocking on my door lol

The adventures of being a node.js dev

Comments
Add Comment