Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Solved a major scalability issue today.
I'm starting to think I might actually be as good as what I told the recruiters I was.
This hadoop-ecosystem job used to take about 2h40min and cost about USD 1.00/GB.
Now it takes 36 mins. At about 0.85/GB.
Fixed some over shuffling, restructured some bottleneck serial stages, used lots of weird words.
Folks in this company I just entered were struggling with this formerly unwieldy process for a year.
Now it's nimble enough to run every hour.
Maybe that whole "experience" thing people were always yammering about wasn't completely bullshit.
rant
reverse impostor syndrom