14
JsonBoa
166d

Solved a major scalability issue today.
I'm starting to think I might actually be as good as what I told the recruiters I was.

This hadoop-ecosystem job used to take about 2h40min and cost about USD 1.00/GB.
Now it takes 36 mins. At about 0.85/GB.
Fixed some over shuffling, restructured some bottleneck serial stages, used lots of weird words.
Folks in this company I just entered were struggling with this formerly unwieldy process for a year.
Now it's nimble enough to run every hour.

Maybe that whole "experience" thing people were always yammering about wasn't completely bullshit.

Comments
  • 2
    I can't be bothered to bullshit people, urgh

    grats tho

    in my experience if I fix issues some higher dev then causes politics to get rid of me... zzzz
  • 3
    @jestdotty that could be the case, but since changing jobs a couple months ago, I'm now the higher ranking nerd on the department.
    Gotta watch out not do the same low-self-confidence politics revenge to my team, though. There will come a day they will solve something I couldn't, and I better not be petty about it.
  • 2
    @jestdotty "Higher dev" wtf does that even mean.

    "Higher" devs don't even dev anymore. They're supposed to lead and inspire.
  • 2
    High on crack cocaïne, @lungdart, dumbass
Add Comment