Deep learning frameworks war.

Hope there is no Data Scientist here...

Neural Network Stones
Credits: Mritunjay

#machinelearning #ml #datascience #tensorflow #pytorch #matrices #ds

!rant

PUBLIC SERVICE ANNOUNCEMENT:

For AI, in particular Deep Learning developers, practitioners, hobbyists and otherwise people interested in the field.

If you go into the Pytorch website, click on resources and scroll down you will see a link to "Deep Learning with Pytorch" by Manning publications. This will give you access to the book, a book that if memory serves me well costs about 40+ in printing and the online book format is about 29 (again, if memory serves well)

The book is currently FREE and it does not ask you for an email address, you can just tell them why you want it for and they will give you the free pdf download.

I don't know how good the book is, but have found Manning to publish really good resources.

Do with this information what you want.

And yes, I am leaving the rant tag, so that more people can see this and take advantage of the opportunity in case of being interested and not having the money to purchase the book after the promotion is done and over with. Fuck you about tags and shit.

6h attempting to correctly install
nvidia driver
cuda
cudnn
pytorch from source
anaconda environment

and this

Credits: Mritunjay

Just installed Keras, theano, PyTorch and Tensorflow on Windows 10 with GPU and CUDA working...
Took me 2 days to do it on my PC, and then another two days of cryptic compiler errors to do it on my laptop. It takes an hour or so on Linux... But now all of my devices are ready to train some Deep Deep Learning models )
I don't think even here many people will understand the pain I had to go through, but I just had to share it somewhere since I am now overcome with peace and joy.

python machine learning tutorials:
- import preprocessed dataset in perfect format specially crafted to match the model instead of reading from file like an actual real life would work
- use images data for recurrent neural network and see no problem
- use Conv1D for 2d input data like images
- use two letter variable names that only tutorial creator knows what they mean.
- do 10 data transformation in 1 line with no explanation of what is going on
- just enter these magic words
- okey guys thanks for watching make sure to hit that subscribe button

ehh, the machine learning ecosystem is burning pile of shit let me give you some examples:

- thanks to years of object oriented programming research and most wonderful abstractions we have "loss.backward()" which have no apparent connection to model but it affects the model, good to know
- cannot install the python packages because python must be >= 3.9 and at the same time < 3.9
- runtime error with bullshit cryptic message
- python having no data types but pytorch forces you to specify float32
- lets throw away the module name of a function with these simple tricks:
"import torch.nn.functional as F"
"import torch_geometric.transforms as T"
- tensor.detach().cpu().numpy() ???
- class NeuralNetwork(torch.nn.Module):
def __init__(self):
super(NeuralNetwork, self).__init__() ????
- lets call a function that switches on the tracking of math operations on tensors "model.train()" instead of something more indicative of the function actual effect like "model.set_mode_to_train()"
- what the fuck is ".iloc" ?
- solving environment -/- brings back memories when you could make a breakfast while the computer was turning on
- hey lets choose the slowest, most sloppy and inconsistent language ever created for high performance computing task called "data sCieNcE". but.. but. you can use numpy! I DONT GIVE A SHIT about numpy why don't you motherfuckers create a language that is inherently performant instead of calling some convoluted c++ library that requires 10s of dependencies? Why don't you create a package management system that works without me having to try random bullshit for 3 hours???
- lets set as industry standard a jupyter notebook which is not git compatible and have either 2 second latency of tab completion, no tab completion, no documentation on hover or useless documentation on hover, no way to easily redo the changes, no autosave, no error highlighting and possibility to use variable defined in a cell below in the cell above it
- lets use inconsistent variable names like "read_csv" and "isfile"
- lets pass a boolean variable as a string "true"
- lets contribute to tech enabled authoritarianism and create a face recognition and object detection models that china uses to destroy uyghur minority
- lets create a license plate computer vision system that will help government surveillance everyone, guys what a great idea

I don't want to deal with this bullshit language, bullshit ecosystem and bullshit unethical tech anymore.

Fucking fuck Nvidia. Shit suckers and ass lickers can't make a fucking thing properly. Everytime I have to compile something involving cuDNN and cuda I wish I could kill myself first. It's a piece of garbage software that we're stuck with. Fuck you mother fuckin Nvidia.

If PyTorch > Tensorflow:
print("True")

Output:
True
True
True
True
......
......
.....
True

Wait a minute? Is this a While True loop?
Doesn't look like an if condition xD

Reinforcement learning is going to be my end. 😩😩😩☠️

(currently stuck at how to put images as well as a bunch of other -motor- values as input... and exactly what am I getting as output again?)

Pulling my own hair out... Ooooooof

Is just me or being a developer has become a complete nightmare?
I mean, I never expected when I got into it to have a simple life in the first place, it's a fucking problems solving job.
But heck, I'm in the field from more than 12 years and something has definitely changed for the worse. Believe me I am just seeking for a general consensus not approval or anything, but it doesn't feel anything like 10 years ago.

I have worked with .NET mostly in all his sauces from aspx, wpf, up to today .NET 10 and C#13 and in the meanwhile it happened that I needed to do tech assistance, code in exoteric shit, use arduinos and raspberries, use perl, java, turn into full stack with databases, devops and shit.

Each year it's worse, the "developer" word gets more and more blurred word to say "the one who must know everything".
I'm asked to know docker, kubernetes, kafka, CI/CD and devop shit, web dev, to get ertifications, to learn how AI works to the level of learning again matht to do matrix interpolations, to get on data science, python, numpy, pandas, pytorch and shit, to know every OS, to know about networking because APIs now have to use rest, a single verb for every action, because if routers and new communication protocols break you have to know and figure out why.

Not to mention that marketing and sales guy shove up the big customers ass every new tecnology to make our work look like bling-a-ling top notch 1% developer stuff that always use latest bleeding edge technology and you're forced to learn new immature frameworks every 2 months or so (latest being various javascript/typescript diagramming libraries).

Every idiot feels entitled to puke out a new framework or supersets of existing languages. I lost count how many supersets css has that I had to peek and learn lol.

Every fucking simple software I did from scratch and designed by myself, web portals for big pharma were much simple than whatever PM i get assigned to are and guess what, I published it and fixed ofc some bugs, but most bugs are related to customer unstable datasets and well, I never had bugs after the first few weeks, except once every few months and nothing serious.

The fucking things they let me do now are hypercomplicated and I spend days fixing other people bugs and we get some hair pulling structural problems becuase they shove in all they can (mediator patterns are a must): kafka, docker, messagebus, whatever javascript clusterfuck they can, patchworks of html and css blurred out in layers of hierarchical scss or sass, slapped into angular (the most immature and crappy shit in js) that has all of his hidden ways to bury and hide DOM (ng-deep: anyone? :host anyone?).

And it's all like this. Whenever I put hands everyone wants to do his little frankeinstein experiment cooking togheter in a cauldron a shit ton of different stuff, overcomplicated patterns.

it's a challenge at shooting flies with bazookas.

I'm really tired of technology at all, not only for my jobs. This fucking trend is a plague spread everywhere and now, since everyone has to deal with it, everything is unstable.

In my daily usage of a smartphone app crashes a lot or have weird troubles, slowness, websites are pretending to be full blown app with this shitty SPA trend and are filled with bugs and incompatibilites.

Basically every tech tool we use is 100% more prone to bugs than 10 years ago.I'm really thinking to find a simple job like baker or shit and get an old phone that just can call and send SMS.
I need to get out of tech for a few years to get back my sanity.

This is not a problem-solving job anymore.
10 years ago I needed to study too but once I got the tools in my hands the job was fun, you got a magic wrench and sky was the limit.

Now you got to fucking learn a ton of bullshit everyday and it's not like you see a end on it, everyday people push out new unstable and bugged shit waiting for devs to be guinea pigs for them. You gotta learn a ton of stuff of which 3/4 will be useless/obsolete/broken and considered inefficient the next month.

jeeeeeez

For my graduate level people(aka Masters degree students or holders)

How normal would you say that: giving dense ass lectures in NN with absolutely NO practical examples and just a fuckload of theory + 1 simulation project in Pytorch in which a robot is to detect collisions is?

is it normal? i mean I knew about Pytorch from a very shallow overview, but these assholes gave that project and expected it completed in a week with a fuckload of dense ass lectures and no practical exmaples.

I know school is supposed to be hard, that is not my gripe, but in yalls experience are teachers more descriptive and fun in other institutions? do I just have shit luck with teachers? I don't feel like wasting my money. If your experience was better then let me know, cuz I want education yes, but i want it better.

One of my minions (erm, I mean, "a valued junior member of my team") asked to be assigned to tasks more "data science related".
Regardless of the very last-decade sounding request, I tried to explain to the Jr that there is more to "data science" than distilling custom llms and downloading pytorch models. There are several entire fields of study. And those are all sciences. In this context, science equals math.
But they said they were not scared of math.
I've seen them using their phones to calculate freaking tips. If you can't do 15% of a lunch bill in your head, hypothesis tests might be a bit more than challenging.
But, ok then. Here we go.

So I had them do some semi-supervisioned clustering. On a database as raw as dirt, but with barely 5Gb, few dimensions and regarding subjects with easily available experts.
Even better, we had hundreds of manually classified training and test cases.
The Jr came back a month later with some convoluted mess of convoluted networks; just the serialized weights of the poor thing were about as large as the database itself.
And when I tried it on some other manually classified test datasets... Freaking 41% error rate, for something that should be a slam dunk. Little better than a coin toss.
One month of their time wasted on an overfitted unusable mess.

I had to re-assign the task to someone else, more experienced, last friday. It was monday when they came up with an iterative KNN approach giving error rates for several values of K... some of them with less than 15% error on the test dataset.

WTF are schools teaching and calling "data science" nowadays?!?!?
I reeeeally need to watch those juniors more closely. Maybe ask for middle-sprint demonstrations. But those are soooo boring and waste so much time from people who know what they are doing...
Does anyone have a better idea to prevent this type of off-track deviation? Without being a total bore, that is.

And... should I start asking people "gotcha" data analysis questions before giving them free reign on this type of tasks? Or is it an asshole boss move? I would hate someone giving me a pop quizz before letting me work... But I got no other ideas.

I tried to sort out a basic Multi layer neural network last night....by hand, just to prove that I was able to do the math by myself and understand that I have the intuition in control rather than just rely on Tensorflow or Pytorch to do shit for me.

I stayed up till 3 in the morning and woke up having nothing but dreams about the endeavor. Shitty part is that i couldn't stop dreaming about partial derivatives and how shit it was that I sucked at them in HS and uni. I get them now, but fuck I just feel that I could have done so much better at uni instead of passing my math classes with 80% to 90% of the grade. I feel as if I was slacking all thanks to being damn near mathematically dyslexic

Flutter.
Tensorflow.
PyTorch.
Back4App,
AppWrite,
Supabase,
Jitsi,
Signal,

I work at a research institute (part of probably the largest research body in whole Europe). And it's driving me nuts. Forget about the lack of interest to improve yourself in terms of software skills or basic digital hygiene so that others don't have to pick up the mop and clean after you. The ancient mindset is what is making me curse everyday. Only a few years ago we switched to GitLab. Before that versioning, if at all a known term, was done explicitly via email messages - code snippets in the message's body, versions in the subject of message attachments...A freaking nightmare. Constantly broken links to files and folders on our NAS since some people have never heard of relative paths or writing even the tiniest bit of support for configuration files in their software so that a tool does not completely brake the moment you transfer it onto another system or - God forbid - the person leaves and there is no information whatsoever what's where. Everyone is complaining about the clutter on our servers but no one is willing to actually clean their own (not someone else's) crap. If you mention to someone something like "Can you please pack your stuff in this GitLab repo with this folder structure, so that I have an easier time integrating it into the main software that we need to ship to our customers in a few days?" all you get as a response is a blank facial expression and the occasional "I have my own processes. Don't bother me with this!". I have been trying for almost 4 years now and its budging a little bit but the lack of support is abysmal. My boss, as enthusiastic as it is, is incapable of putting his foot down. The fact that I have two heads of my team (one not really but acting like it) does not improve the situation at all especially since both are pulling in a completely different direction. We are literally wasting hundreds of thousands of euros of taxpayers' money to buy new hardware that people are either inadequate to use to its fullest potential (think buying the latest GPU to play Minesweeper) or not having even the smallest clue on what they need it for. And we are always complaining about our budget! You don't invest a couple of hours to investigate how PyTorch can work in a distributed manner on multiple CPUs, GPUs and even systems, yet demand you get a new server for 80K with a more powerful GPU and CPU to run your crap models on so that you can publish a half-ass paper that nobody cares for let alone will ever bother reading (beside the AI reviewers).

PyTorch my new journey.

Waiting 3 days for your graphics card to get trough all training data to see if you wasted your life with bad architecture.

HEY ya'll I'm a programmer in python or at least I know the basics. Well I want to get into deep learning and nueral networks and I was wondering what tools should I use because I'm already using pycharm professional as my IDE and I'm playing around with pytorch.

Any fellow devranters in the pytorch udacity scholarship?
Damn,i was so excited for that,but it started 5 days earlier and now here i am , too busy to even start the videos. God i am gonna watch them after waking.

Awww. GPT approved my code <3 Just ignore the text after the comma

Wooooooohoooooooooo!!!!!! I am a udacity pytorch scholar!!!!!!!!!!!💥💥💥💥💥💥💥💥💥
Thankyou udacity !!! Love love love !!!! Damn am so excited!! Time to sharpen my spack skills 💓💓💓

Oh yeah Google why don't you just change the parameter order of functions, remove entire functions between minor versions, and not put a single example on your API docs? And force devs to add 30 lines of boilerplate and start an http server so I can run the debugger? Fuck tensor flow, I'm moving to pytorch.

Setting up an expo react / react native is a far worse feeling than installing GPU drivers + cuda toolkit for pytorch.
I have no idea how react devs are dealing with this shit. This is so horrible. Wtf is babel ? wtf is expo ? Wtf is SDK ? Wtf is eas ????????????????????

Gonna start my AI journey, thinking of tensor flow then PyTorch. Any suggestions, warnings, advice?
I’m just interested in learning more about it and figure out what to use it for later.

I feel really lost in neural network theory.
the mnist sample made sense, but now I'm looking at Gans and CNN's.. and now all of a sudden I'm lost.

True also are the examples I'm finding of something I know I was able to get to work when more at peace once upon a time called wavenet for text to speech.

I used the Onyx model however which was very easy to implement, but I quickly get lost looking at the tensorflow and pytorch code, even though it is very short I feel intimidated.

The ssd mobilenet documentation also is pretty straightforward, but when I look for wavenet information about providing input in what format and interpreting output I'm having some trouble.

Its frustrating.
I'm tense, I'm poorly rested, I'm sick of having to redo crap and I'm surrounded by people who make me hypervigilant, skin crawly and tense.

How to overcome these things when I'm not at peace at all ?

I don't know. Pushing through it isn't compatable with the mindset I've been forced into.

ah yes just another day of fighting the same perceived non functioning of pytorch.. tracing through the tensors per layer to see if anything is changing at all.. listenig to the same dipshit talking about how its getting cold out there but hey it feels good.

sitting in the same place, half sleep deprived in a state of utter destitution waiting for the same dirty fucks to steal all my things because they want to keep me quiet about what they've been doing for as far back as I can now remember wondering when life will continue because lets face it, life should continue, and these people willfully fucking up my country now for nigh a century isn't my goddamn fault !

but hey, they think they got away with murder, and its more like suicide as they get used up when young and then reach older age and they're still walking the same loop and i'm being dragged along with them as they started assigning more zombies to my home area gradually who also brainlessly walk around doing the same things thinking it somehow benefits them.

You know it is not acceptable to not finally answer this goddamn question for myself or with some help !

PyTorch is driving me insane !
I follow the goddamn docs, practically just created a copy of the sample for the training portion, even if my design were to suck, the training steps SHOULD change the model parameters right ?

so if I do a...

model.parameters().clone() before training
and mode.parameters().clone() after training

torch.equal(p1.data,p2.data) should be false right ????

Ros melodic in a strictly python 2.7 environment mixes horribly with a PyTorch based RL module... Time to work around with terminal calls from the latter
*sigh*

If you are required to do a custom Object Detector for Mobile.

Would rather use Tensorflow, Theano, or PyTorch to train the model?

Deep learning

I thought it would be a great course, learn some of the stuff that I always read about but couldn't understand jackshit, and maybe profit form it somehow.

I'm in my last assignment, they want us to pick some SNLI paper and implement, ok, so I find this one with the least amount of params because I thought hey this seems promising.

And boy what a ride it was, I implemented it using PyTorch, the results are way off, I read the paper again and rewrite some parts, still nothing, I get 79%, it's supposed to be 85%, and no matter how I try, nothing.

10 GitHub repos later, 40 hours of complete meltdown,
20 throwaway Google accounts using colab because we don't have GPUs in our uni and using AWS is not feasible.

Same shit, I'm at loss, the world is a lie, and I fell for it...

Fuck.

Overheard from a JavaScript dev: why is there no React to Pytorch transpiler yet?

ohhhhh I am pissseddddddddddd

itss the fucking pytorch.module class it would seem !

I do exactly the same goddamn shit as its supposed to do in a goddamn notebook and run it step by step and the fucking model trains and the output values change !!!! and the loss decreases !!

I do this in the goddamn class derived from model with a call to model.parameters() and the fucker fails !!!

why ???
why ?????
why ??????
is it cloning the goddamn parameters so the references aren't there ????
seems to work goddamn fine when i call a layer and activation function at a goddamn time chaining the calls one after another !!!!!!

UGHHHH IT LOOKS LIKE IF YOU DEFINE THE LOSS AND OPTIMIZER OUTSIDE THE FUCKING CLASS IN A SEPERATE TRAINING FUNCTION IT DOESN'T TRAIN !!!!!!

WHY ??
A REFERENCE IS A GODDAMN REFERENCE !!!!

Wandb sweep runs for an interactive job but gives me a cuda error for illegal memory access for the slurm job. Spent the last 15 hours solving it and still can't enable multi gpu support on it. FML

So, I feel wayyy behind the tech curve right now.

The SSD implementations you see online, they're still just a bunch of seperate sort of chaos machines that contain the standard perceptron-like model of a weight, cost, and bias right ? They just kind of inferred their values by training like any other neural network, in its sep-erate parts and just fed pieces of output data generated by other parts of the neural network to it right ?

I mean it implements with pytorch so its basically a really big array of tuples in a sense that are maniupulated in a specific way.

and then CNN's just feed data back into another trained piece of the model right ?

I'm curious because object classification is about the ONLY thing I've seen work even close to properly lol

there is just so much fraud these days. sigh.
and so many lamentable tech choices and attempts... like node lol

I'm beginning to feel like any kind of specific approximation via neural networks is a myth. That if you can't reduce output to simple categorical values that can be broadly interpreted between two points, that it doesn't work.

I have some questions and they don't seem to be getting answered about the design of the net. How many layers should I use ? How many neurons per layer ? How does this relate to the number of desired quantitive scalar outputs I'm looking to create, even if they are normalized, they can vary GREATLY and will if I'm approximating the out of several mathematical expressions. Based on this and the expected error ranges of these numbers and how many possible major digits could be produced within the domain of the variable inputs being introduced, how many neurons per layer ? What does having more layers do ? In pytorch there don't seem to be a lot of layer types per say, but there are a crap ton of activation functions, and should I just be using these at the tail end or should they actually be inserted between layers so the input of the next layer passes through another series of actiavtion functions ? what does this do to the range of output ?

do I need to be a mathematician to do this ?

remembered successes removed quantifiable scalars entirely from output, meaning that I could interpret successful results from ranges of decimal points.

but i've had no success with actual multi variable regression as of yet, even when those input variables are only 2 and on limited value ranges eg [0,100] and [0, 2pi]

and then there are training epochs to avoid overfitting, and reasonable expectation of batches till quality results will start to form.

Pytorch va Tensorflow for someone getting into machine learning?