Ooh this is good.

At my first job, i was hired as a c++ developer. The task seemed easy enough, it was a research and the previous developer died, leaving behind a lot of documentation and some legacy fortran code. Now you might not know, but fortran can be really easily converted to c, and then refactored to c++.

Fine, time to read the docs. The research was on pollen levels, cant really tell more. Mostly advanced maths. I dug through 500+ pages of algebra just to realize, theres no way this would ever work. Okay dont panic, im a data analyst, i can handle this.

Lets take a look at the fortran code, maybe that makes more sense. Turns out it had nothing to do with the task. It looped through some external data i couldnt find anywhere and thats it. Yay.

So i exported everything we had to a csv file, wrote a java program to apply linefit with linear regression and filter out the bad records. After that i spent 2 days in a hot server room, hoping that the old intel xeon wouldnt break down from sending java outputs directly to haskell, but it held on its own.

Dev nightmares :

- Not finding bug fix on stackoverflow/GitHub .

- Losing code that hasn't been pushed to GitHub.

- Dealing with an unclean and inconsistent database.

- Installing Node Dependencies.

- Resolving CORS and 500s.

- Training a Linear Regression Model with 700 epochs on an entry-level Laptop.

Keep appending to this list.

#devrant #devnightmares

True story🥂

When you’re fundraising, it’s AI
When you’re hiring, it’s ML
When you’re implementing, it’s linear regression
When you’re debugging, it’s printf()

The gap of data science in industry and academia is so large. As a data scientist in a large financial company, I see that people are still using traditional models such as linear regression and SVM, while people in academia keep inventing new concepts and techniques such as deep learning.

I am not saying that we should completely embrace deep learning, or stick to classic methods. But I just feel so surprised that the gap is so large...Sometimes I am even thinking whether I am doing the right "data science"...

Implemented my first linear regression model! Predictions not on point! Whatever.. :p

Linear Regression strikes again...