AboutMath, OSI, Linux, devops nerd!
SkillsTerraform, js, nodejs, python, r, pytorch, aws, react, numpy, dplyr, ubuntu, arch, and more
LocationRichmond Hill, ON, Canada
Joined devRant on 12/15/2016
Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple APILearn More
Wife wanted me to make sure that we can watch the entire season 12 and 13 of degrassi while in the hospital (which has no good internet connection), but there is no good torrent or anything that lets you get it downloaded locally.
So I wrote a super basic script to scrap the season from YouTube and download it. Now just having fun looking at it.5
Getting tired of old business types managing software projects... I really don’t care if you have fifty years of experience in managing loans in banks, please don’t try to manage software or data projects... this is sooo out of your realm of understanding.
PS I actually like this person, they are a wonderful human, but keep nosing in areas they have no knowledge, and derail the conversation from talking about things we actually need to discuss. Like “how do we build this?”5
Some of these things are not like the others. One of these people is a tv scientist not an actual data engineer or data scientist, while another is an activist and while is extremely respected, has no room in a data+ai talk -.-10
Waiting on a baby girl in a few weeks from today! Excited! Scared!
Any other future/current dads with babies with tips on how to handle a work from home with a baby?6
Me: why are we paying for OCR when the API offers both json and pdf format for the data?
Manager: because we need to have the data in a PDF format for reporting to this 3rd party
Me: sure, but can we not just request both json and PDF from the vendor (it’s the same data). send the json for the automated workflow (save time, money and get better accuracy) and send the PDF to the 3rd party?
Manager: we made a commercial decision to use PDF, so we will use PDF as the format.
Me: but ...4
It feels so good actually doing something that saves the company a buttload of money.
Just optimized some ML models. Now it costs a tenth of the money to run them :P2
Just came across this sentence in an oped... some strange times we live in. I still remember when I was getting into tech how everyone was afraid of Microsoft... but to be honest google & apple are doing more EEE than Microsoft so...6
I’m trying to update a job posting so that it’s not complete BS and deters juniors from applying... but honestly this is so tough... no wonder these posting get so much bs in them...
Maybe devRant community can help be tackle this conundrum.
I am looking for a junior ml engineer. Basically somebody I can offload a bunch of easy menial tasks like “helping data scientists debug their docker containers”, “integrating with 3rd party REST APIs some of our models for governance”, “extend/debug our ci”, “write some preprocessing functions for raw data”. I’m not expecting the person to know any of the tech we are using, but they should at least be competent enough to google what “docker is” or how GitHub actions work. I’ll be reviewing their work anyhow. Also the person should be able to speak to data scientists on topics relating to accuracy metrics and mode inputs/outputs (not so much the deep-end of how the models work).
In my opinion i need either a “mathy person who loves to code” (like me) or a “techy person who’s interested in data science”.
What do you think is a reasonable request for credentials/experience?5
Anyone else excited for the purism and pine phones to hit the main market? Wanna hear what the devRant community is thinking of this?4
A really fun story to read. Makes me think of my own work shenanigans. Good to know Uber is just as fallible!
What are people unfiltered thoughts on apples new ARM processor? Especially if you are a Mac user?
(I'm an OS hopper, though my main machine is running ubuntu right now, and work machines are usually mac with windows with wsl)21
This is not a happy rant...
Got a new well paying job. Moving up in life. But my wife is not in the same place... She had quite a few career struggles and just lost her current job... It wasn't even a job she went to school for, just whatever she could find...
Now she's getting quite depressed. Luckily she's not envious at me, but does keep saying how lucky she is to have me.
I really want to help her somehow, but this is really a thing I just don't know how... And it just looks like she's not handling it too well. Joking about suicide and crying about being useless... She also keeps saying that all she can do now is be a housewife. We did seek out help for her. But still... I really want to give her better support. I feel useless here.18
Got the best compliment on an interview :3
Submitted a coding take home assignment. Was told by the tech lead that the code looked like they wrote it themselves! There was nothing they would change in terms of style or approach :38
DevRant needs a save for later option... I see lots of long interesting posts, I start reading them and then something comes up (like a phone call or I need to do something). Then by the time I come back the post is gone, and I’m sad...5
That moment when google ranks gmail with a low speed score... oofff...
This is so true!
Too many companies in the past years have been focusing on moving rendering to the client, and not optimizing their webpages that we lost sight of making efficient and fast websites.8
So today I told my boss “nodejs without typescript is like sex with strangers without a condom. Sure it’s fun and all... but then you get an std”12
A big project in my company. Had some annoying race condition that caused data to get deleted when two processes finished in the wrong order they hit the dB and override each other’s work.
Long story short. Fixed the bug and in the process the codebase shrunk by 60%. I didn’t have to delete the rest of the code, but the bug was due to a function in the legacy section of the code, and found out that it was the only function used in that section.
So I deleted it. Rewrote the function so it upserts. And bam. Smaller, cleaner code :)1
Workstation for last few hours. Just sitting here chilling while waiting for my car to get serviced.10
Everyone I tell this to, thinks it’s cutting edge, but I see it as a stitched together mess. Regardless:
A micro-service based application that stages machine learning tasks, and is meant to be deployed on 4+ machines. Running with two message queues at its heart and several workers, each worker configured to run optimally for either heavy cpu or gpu tasks.
The technology stack includes rabbitmq, Redis, Postgres, tensorflow, torch and the services are written in nodejs, lua and python. All packaged as a Kubernetes application.
Worked on this for 9 months now. I was the only constant on the project, and the architecture design has been basically re-engineered by myself. Since the last guy underestimated the ask.2
How do you go about introducing new technologies to your company/team?
Whether it’s writing the next API in another language or using a new framework/library/approach?6
About 2 months ago, my company wanted to build a micro service that will be used to integrate 3 of our products with external ticketing systems.
So, I was asked to take on this task. Design the service, ensure extendability and universality between our products (all have very different use cases, data models and their own sets of services).
Two weeks of meetings with multiple stakeholders and tech leads. Got the okay by 4-6 people. Built the thing with one other guy in a manner of a week. Stress tested it against one ticketing service that is used in a product my team is developing.
Everyone is happy.
Fast forward to last Thursday night.
“Email from human X”: hey, I extended the shared micro service for ticketing to add support for one of clients ghetto ticketing systems. Review my PR please. P.S. release date is Monday and I am on a personal day on Friday.
I’m thinking. Cool I know this guy. He helped me design this API. He must’ve done good. . . *looks at code* . . . work..... it’s due... Monday? Huh? Personal day? Huh?
So not to shit on the day. He did add much needed support for bear tokens and generalized some of the environment variables. Cleaned up some code. But.... big no no no...
The original code was written with a factory pattern in mind. The solution is supposed to handle communication to multiple 3rd parties, but using the same interfaces.
What did this guy do wrong? Well other than the fact that he basically put me in a spot where if I reject his code, it will look like I’m blocking progress on his code...
His “implementation” is literally copy-paste the entire class. Add 3 be urls to his specific implementation of the API.
Now we have
The latter 3 are his additions... only the last one should have been added in reality... why not just add a type to the payload of the post/put? Is he expecting us to write new endpoints for every damn integration? At this rate we might as well not have this component...
But seriously this cheeses me... especially since Monday is my day off! So not only do I have to reject this code. I also have to have a call now with him on my fucking day off!!!!
Just Realized... I have like 1 of every device... I'm a full on fence sitter...
iPhone X -- work phone
Huawei p20 -- personal phone
MacBook -- work laptop
Lenovo ThinkPad running Arch Linux -- personal laptop
Window Surface -- research/teaching laptop (use it when I teach and need to bring notes)
Gaming PC -- Gaming PC
And all of this took ~5 years to garner... Geez... I like all of these devices too (and sometimes i hate em) Holy...1
Crucial touchpoint with product owners;
- me doing my business on the toliet
- me cleaning up
- me brushing my teeth
** Yes I agree! That feature could****brushbrush****take some time to ***brushbrush*** do**
My laptop was at 500% of course usage and ram... Wtf...
Had to stay till 1am because I was debugging a microservice system with 15 services, 3 databases and a full mail server. Freaking integration tests took all night to build and run. Don't have a remote server to run this on because the guy is sick. Arghhhhhh8
Someone should really close the COVID-19 ticket. Or at least make it ready for technical/PO review.1
C: what technologies are you using in this product?
Me: we are using celery which uses kombu to communicate through redis. Additionally, we have beat tasks that run and check the state of workers
C: was someone hungry when they named these frameworks?
Me: Possibly, but I just got hungry naming them.
So at work, I was trying to convince a senior programmer to adopt Golang for a new project that would need to handle large amount of throughput as oppose to Django with gunicorn and other optimizations (which is what we usually do).
The response was "we'd have a hard maintaining it, as no one really knows Go and it will be hard to hire people with Go knowledge".
So my question: "in your opinion, is that a good reason?" I have some go experience from other jobs, it seems like a superior solution for this problem.7