Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "pandas"
-
Phone conversation between me and a client:
CLIENT: "I see it weird..."
ME: "Which browser are you using?"
CLIENT: "The one you tech guys don't like"
ME: "Internet Explorer, isn't it?"
CLIENT: "Yeah, I'll switch to Firefox then."7 -
How to properly have fun on a Saturday night:
1. Suddenly become deeply unsatisfied with current linux distro
2. Evaluate alternatives
3. Decide some change is needed but not too much: install fresh version of old distro
4. Once again, experience profound dissatisfaction
5. Opt for radical change
6. Erase all linux partitions, form a super partition and install a new linux distro on it
7. Spend hours familiarising with the new distro
8. Spend more hours googling stuff and typing commands in the terminal
9. Download current devRant avatar, send it to the PC via Telegram and set it as user's avatar for the welcome screen
10. Feel deeply satisfied
11. Accidentally wake girlfriend up while trying to get to bed. Get told off for staying up until 4am and for "being such a nerd"21 -
Two years ago I moved to Dublin with my wife (we met on tour while we were both working in music) as visa laws in the UK didn’t allow me to support the visa of a Russian national on a freelance artists salary.
After we came to Dublin I was playing a lot to pay rent (major rental crisis here), I play(ed) Double Bass which is a physically intensive instrument and through overworking caused a long term injury to my forearm which prevents me playing.
Luckily my wife was able to start working in Community Operations for the big tech companies here (not an amazing job and I want her to be able to stop).
Anyway, I was a bit stuck with what step to take next as my entire career had been driven by the passion to master an art that I was very committed to. It gave me joy and meaning.
I was working as hard as I could with a clear vision but no clear path available to get there, then by chance the opportunity came to study a Higher Diploma qualification in Data Science/Analysis (I have some experience handling music licensing for tech startups and an MA with components in music analysis, which I spun into a narrative). Seemed like a ‘smart’ thing to do to do pick up a ‘respectable’ qualification, if I can’t play any more.
The programme had a strong programming element and I really enjoyed that part. The heavy statistics/algebra element was difficult but as my Python programming improved, I was able to write and utilise codebase to streamline the work, and I started to pull ahead of the class. I put in more and more time to programming and studied personally far beyond the requirements of the programme (scored some of the highest academic grades I’ve ever achieved). I picked up a confident level of Bash, SQL, Cypher (Neo4j), proficiency with libraries like pandas, scikit-learn as well as R things like ggplot. I’m almost at the end of the course now and I’m currently lecturing evening classes at the university as a paid professional, teaching Graph Database theory and implementation of Neo4j using Python. I’m co-writing a thesis on Machine Learning in The Creative Process (with faculty members) to be published by the institute. My confidence in programming grew and grew and with that platform to lift me, I pulled away from the class further and further.
I felt lost for a while, but I’ve found my new passion. I feel the drive to master the craft, the desire to create, to refine and to explore.
I’m going to write a Thesis with a strong focus on programmatic implementation and then try and take a programming related position and build from there. I’m excited to become a professional in this field. It might take time and not be easy, but I’ve already mastered one craft in life to the highest levels of expertise (and tutored it for almost 10 years). I’m 30 now and no expert (yet), but am well beyond beginner. I know how to learn and self study effectively.
The future is exciting and I’ve discovered my new art! (I’m also performing live these days with ‘TidalCycles’! (Haskell pattern syntax for music performance).
Hey all! I’m new on devRant!12 -
Not a dev job but one I quit.
The company had Windows 98 on all their computers and we had to use it for our daily activities. Being relatively young, I'd only used Windows 98 for a couple of years between the age of 5 and 8 and mostly for playing games.
The company was paying me overtime to stay late and familiarise with the OS.
Basically, they were willing to spend money to train employees to use an obsolete OS - what a bunch of idiots and poor decision-makers.
I left after three months because I could no longer cope with their nonsense.1 -
Python and a cup of Earl Gray Tea and I feel like captain Picard commanding the Enterprise. The dark theme of the IDE helps, too.
Welcome new side project.4 -
C: hey mate, what's the best tool to open up this 31.1M rows x 106 cols CSV file?
M: Umh...Pandas DataFrame or R DataTable I guess?
C: all right, Excell will do, thanks!
M: erhm...yeah, anytime?11 -
Late night rant
You've been coding until 2 am. The moment you're about to get into bed (one leg already under the duvet) you have an epiphany about the code. Huge interior dilemma revolving around the "what to do now?" question.3 -
Pretending to have a hard deadline if I don't have one. Gotta trick my brain in putting some effort into it.3
-
>Instructions in the manual -
1. Install Python 3.5
2. After installation is complete, open a new terminal/command prompt window and run 'pip install pandas'
3. Done!
>Client
1. Installs Python 3.7.2
2. Types Python in command prompt, types 'pip install pandas' there
3. Raises a hue and cry over the program not working because the instructions were not clear
Smfh...1 -
Im oficial! After one year of studying python / django I received an acceptance letter today. When I began, I didnt expected it would be so soon.
It's not my first job, but my first one as developer (will be doing stuff with python / django / pandas)
So Im happy...! Which is rare for me8 -
Helping a random junior out with an ML project on an online tutor site.
Her: So what is the syntax for implementing xyz function
Me: *opens Google* *opens pandas docs* *searches the function* *tell her the syntax*
She: Woah thanks a lot!
I collected my tutor fees feeling good about myself.10 -
Programme that simulates the rolling of the dice when playing the Risk board game.
No more dice that fall off the table. No more dice that throw the figures into disarray.12 -
Trying not to think about code when I pee.
Pictures like this hang on the urinals in the office restrooms.3 -
I finally got a job at a tech company (although it's not a tech job) with a very good work/life balance.
Therefore, I plan on getting more serious about properly learning how to program in my spare time, also because, being a tech company, programmers are all over the place and are generally willing to talk about code.
I must say that while job hunting, devRant has been very useful to me since it allowed me to understand what kind of environment I'd like to work in. So far, the first few weeks of work have been great.
Ah, and the view from the office is unbeatable.7 -
I received a LinkedIn message from a recruiter who thinks I'd be a good fit for a role that requires being fluent in German.
My LinkedIn profile clearly states I only have elementary proficiency in German.
I'm gonna reply and see how long it will take for him to notice 😈6 -
That moment when it's Sunday and you cannot wait for the new Weekly Rant theme to be announced on Monday.
-
What is the best project you've done with a Raspberry Pi? Got my model 3 a month ago and was thinking of turning it into Jasper. However I'd like to consider more options before starting it.
Thanks in advance12 -
I hadn't used devRant in a while since I'm preparing for an upcoming job interview and was trying to stay away from my phone.
However, I cracked today and I spent the whole afternoon staring at this beautiful dark-themed interface and reading rants.5 -
Mentors, take note. This is a best practice over here.
I've spent two days digging through obscure documentation trying to accomplish one of those tasks that is simple in word and complex in deed. Namely, I wanted to concatenate (not delete) near-duplicate values in Pandas before rendering the data into a graph. Two days beating my head against the wall.
One of my mentors (I'm an intern) heard about the issue, wrote in the proper line (a very specifically and archaically formatted command), and pushed it to repo without even asking for thanks. Works like a charm and he saved my rear end. What a guy.
Please, mentors, don't leave your interns hanging on problems where the only solution is shrouded in dubious documentation and magic syntax. Especially when there's a deadline involved. Let them struggle on logic flow and writing good code.
Be like this guy. You'll build the importance of teamwork and your intern will think you're a wizard.2 -
Absolutely hate it when HR says: "You'll be told about the outcome of this interview in the next two weeks".
What about putting in place a more stringent timeline for recruitment processes at your company?
What the fuck am I supposed to do for two bloody weeks?8 -
*some* devs with their multiple monitors are like my 3 flatmates with their multiple shower products.5
-
Man....I keep up with this strange love hate relationship I have with Python....
Last night it was python that literally wrote my homework: define all possible equivalent partition tables with cause and effect analysis and boundary value checks for a program. The whole thing wrote itself and all I had to do was verify the inputs. Something that I was able to do using jupyter with pandas and numpy. On one hand, I despise the lack of static typing and use of whitespace as a block delimiter. On the other I cannot but help feeling a high level of gratitude over the language and its high availability and ease of use for this.
Sure, I could have used other tools, but this language has dominated hardcore in this regard enough to the point of not considering it being a crime against humanity.3 -
The day I was only one ++ away from getting the stress ball.
Decided not to claim it in the end, though.7 -
Sorry Google, you got it wrong this time ....
Oh my gosh, look at that function definition ...
Oh my gosh, look at that variable ...
Oh my gosh, look at that zone ...
Oh my gosh, look at that long ...
Oh my gosh, look at that short ...
Oh my gosh, look at that stop ... is more my style.10 -
A parallel universe simulator which will show us what our lives could look like in other universes.5
-
Just thought of this meme template when I got that error. Hope you like it 😅
BTW it was a KeyError in a pandas DataFrame during which, somehow, another KeyError for the same key occurred. I'm not sophisticated enough in Python to explain this though. -
When I first began with Python I really missed the static typed checking from Java, I barely know anything about a returned object from a method and have to read the API extensively for every new library.
After a while I finally understand why Python is so powerful, the combination of dynamic typed language and rich default methods make the language unbeatable for your productivity.
While Java's Object only has toString(), hashCode(), equals() or clone(), Python's basic Class has every fucking method for every scenario I could ever image. No wonder that libraries like numpy or pandas work so well and fluidly.8 -
Pretty much Python automation on steroids.
https://github.com/konradhalas/...
Dacite is an integral part in it, cause it makes most auto generated API wrappers like Cloudflare API "maintenable".
Getting dicts, converting via Dacite to defined data classes...
Then using TOML to define e.g. output parameters (e.g. list of classes / properties one needs)...
... To export them via Pandas to anything what one needs.
It's just so comfortable.
Definining data classes, sprinkle the API calls and dacite in it, some definition via TOML, done.
Yes, lots of dark vodoo / behind the scenes magic... But ... It removes all this annoying fucked up boilerplate writing that takes ages and makes it frustrating.
As long as the data wrapper (e.g. clousflare API) generates Dicts, its really minutes to get an export working.
If you know the pain of having to deal with multiple accounts, different formats (e.g. different companies)… hours of manual copy pasting to aggregate the data etc.
Then you can maybe understand why I love this so much.
Data classes and dacite makes painful confusing workflows so much nicer and self documenting, I get wet in my pants while writing this. :) -
Re-reading old code I wrote some time ago and trying to improve it. It sounds and it definitely is scary to do at first, but it truly helps me approach things from a different perspective and question what I did.
-
The weird moment when you realise you like your devRant avatar way more than your facebook profile pic.
-
When the Python tutorial gets to this point and you try it out while inspired by your previous Star-Trek-themed rant (https://www.devrant.io/rants/439358)2
-
i was learning neural networks, started with keras and was on the first tutorial where they started by importing pandas
so i switched to learning data analysis using pandas in Python where they started by importing matplotlib and i realized data visualization is also important and now I'm reading matplotlib docs...🙄11 -
I'm studying Python at the moment and I'm looking for some easy projects to do in order to gain hands on experience. After having written the code of a dice simulator for Risk!, I'm now thinking of a Twitter bot as my next project. Has anybody done it? Would you reccomend doing it?
Since joining devRant I've felt much more motivated to progress in learning Python and if it felt really rewarding to play Risk! without rolling the dice, it also thanks to you all. Sorry for ths cheesy nuance and for this not being a rant.3 -
Going to the office when the girlfriend is working from home.
But most of the time, I knowingly stay home too3 -
pandas can suck my balls.
N
I
H
I'd rather roll my own.
edit: but also xgboost can suck my balls.
Treating every OBVIOUSLY continuous-valued entry as a 'category'.
All searches for this problem turn up tutorials and documentation on how to CONVERT continuous and numeric values into classes or categories.
Not a single fucking document addresses the problem of when pandas or xgboost refuses to treat numeric inputs as numerics and insists on pushing an error that your data is categorical when every fucking inspection shows the type as numeric.9 -
Stayed up all night to make interactive data visualizations from CFTC data and now I want to show it out.
BUT
No one is interested. Why Earth why?
Whatever, where is keyboard, and turn off the lights.
- Dark music plays. -
Pandas groupBy cumcount() function cracks me up every time I see it. Dask has one too: http://dask.pydata.org/en/latest/...
In general Python libs are so inconsistent with their function names, especially data sci packages.
Abbreviations rule supreme. They could have at least add an underscore for this one if that's the standard. cum_count over and out1 -
What's that? You committed the tmp/dist/cache field for something only YOU run locally and asked me to review it. Just GET OUT.1
-
Argghhhhhh!
Non-unique Multi-Index killing me...
Even when there is no duplicates found in the data frame.
What the hell am I missing.. -
I currently work in data analysis.
Yesterday one of the colleagues working in a different department came to me asking if I could fix his malfunctioning headphone set :/3 -
Just got accepted as a Tutor. I have to teach PhD students in medical field SciKit package for image processing. Been coding in Python using pandas and numpy for years, but I know jack shit on SciKit.
I applied just for fun and got the position. Now I am fucking terrified.
Meanwhile I rejected a Teaching Assistant position because of this one.6 -
My job involves writing a trading bot. Initially I thought it was gonna be cool but God I was wrong. Learning how to write in python (python's oop and indentation is a nightmare), backtesting a strategy, learning how to use libraries like backtester, TaLib , Pandas. All seems to have really steep learning curve and at the same time it is bloody boring.8
-
Data scientists should be charged with animal trafficking and animal abuse because they import Pandas 🐼 and feed them to Python 🐍2
-
When you had planned a special night entirely dedicated to a new side project and it's only 00:30 and you feel too tired to even start typing.
-
Just finished the testing of the first script I wrote at work to automate a vital task that had to be carried out manually on a daily basis.
It feels so damn good to know that the whole process is free from the variable of human error.1 -
It's been a while since I started my first python course on Udemy.
I feel the urge to rant because as I'm leaning more and more and getting increasingly more interested in the subject, the instructor's tone is getting sloppier and sloppier and the quality of the videos is increasingly worse. Really, the videos are so blurred that I cannot read what the guy writes, so I have to turn the captions on hoping that he says out loud everything he types.
Fuck.6 -
I'm not a data scientist but lately I've learned NumPy, Pandas and now I'm learning Matplotlib and Seaborn and after years of Excel the improvement is astounding.
Excel is far easier to approach (I casually use it since I was 6) but once you need to do more advanced stuff it requires a lot of tricks and workarounds which needs to be memorized and are hard to find just by reasoning or are straight impossible without the use of macros which introduces many compatibility issues.
Pandas on the other hand is harder to approach but once you learn the concepts between its basic data structures you can do a lot with little "Google-Fu".3 -
. Resisted the urge of trying Firefox Quantum when everyone was talking about it on here.
. Finally gave in today and tried it.
. Fell in love with it.3 -
I have started to work on pandas and numpy , should I work on scipy and sklearn or should make a strong fundamentals on numpy and pandas?2
-
Fuck yeah ... I have uploaded my major computation file to S3 and create Lambdas from those files(includes numpy and pandas also) and now I have only routes and invoke strategies in my EC3 .. looking for cost reduction....
-
Here I am, 3:18 am, maybe I won't sleep today either, I hope I do... I'm going on with my uni project, a data science project. I've been wasting hours trying to understand why the fUcK 2 dataframes give me substantially different performances when they fucking shouldn't, since they should be the fUcKing sAmE. But apparently pandas is making fun of me... it seems that if you do something like:
df=original_df.loc[:, [some_cols]]
and some columns in [some_cols] don't exist in original_df, pandas won't give a shit and create a NaN column, or 0 based on how many virgin leprechauns ate bananas for Thanksgiving.
Plus I'm fucking freezing, in this apartment the heating system turns off at 23:59, it makes sense if you're in the fucking bed where you'll be fucking warm.
I miss software development... I wanna finish this MSc as soon as possible.
And here I am, listening to post-rock, writing jupyter notebooks, trying to be fucking positive.
It's not like I hate data science (maybe?), but I'm burnout.
Maybe I'll rewatch another time the video of Mr Robot with the song Where Is My Mind.
See ya.2 -
WHY does VS code load up Pandas dataframes so damn slowly? It’s bad enough that it seems to take an extra few seconds to get PyQt5 going, but the dataframes are awful, even with small 50 record Parquet files.
I don’t have the attention span to sit there and wait for this without finding myself playing with my phone or surfing.
I guess for debugging and testing I should just create a column A, column B, column C dataframe on the fly and give it some 1, 2, 3 kind of values.
But, Jesus, man... This shouldn’t take 30 seconds to load a simple form. 🙄2 -
Time zone just sprang into day time savings yesterday
I had a device monitoring data pollution on a roof that goes to a website. The thing didn’t fucking adjust bday the device stayed on standard time
I spent the entire day thinking what I should adjust for something that most countries don’t do any more why do we even bother with saving daylight.
In addition the timezone I wanted didn’t work right with pandas and I had to do the wrong way to get it “right”6 -
The fact that pandas has a function called .isnull() to find nans, but the function to get rid of them is called .dropna(), will never stop bothering me.1
-
can you please help me with this.
I'm creating dataset of [Leet words][1].
This code is for generating [Leet words][1]. it is working fine with less number of strings but I've nearly 3,800+ strings and my pc is not capable to do so. I've Tried to run this on cloud(30gb RAM) not worked for me. but I think possible solution is to convert this code into numpy but I don't know how. if you know any other efficient way to do this it will be helpful.
Thanks!
from itertools import product
import pandas as pd
REPLACE = {'a': '@', 'i': '*', 'o': '*', 'u': '*', 'v': '*',
'l': '1', 'e': '*', 's': '$', 't': '7'}
def Leet2Combos(word):
possibles = []
for l in word.lower():
ll = REPLACE.get(l, l)
possibles.append( (l,) if ll == l else (l, ll) )
return [ ''.join(t) for t in product(*possibles) ]
s="""india
love
USA"""
words = s.split('\n')
print(words)
lst=[]
# ['india', 'love', 'USA']
for word in words:
lst.append(Leet2Combos(word))
k = pd.DataFrame(lst)
k.head()3 -
I love serverless functions but I'm so tired of complex orchestration, juggling event parameters and now scipy+numpy+pandas exceeds size limit of 250MB..
Feel like cramming it all in a monolith like the geezers of yore and be done with it3 -
I starting developing my skills to a pro level from 1 year and half from now. My skillset is focused on Backend Development + Data Science(Specially Deep Learning), some sort of Machine Learning Engineer. I fill my github with personal projects the last 5 months, and im currently working on a very exciting project that involves all of my skills, its about Developing and deploy a Deep Learning Model for Image Deblurring.
I started to look for work two months to now. I applied to dozens of jobs at startups, no response. I changed my strategy a bit, focusing on early stage startups that dont have infinite money for pay all that senior devs, nothing, not even that startups wish to have me in their teams. I even applied to 2 or 3 and claim to do the job for little payment, arguing im not going for money but experience, nothing. I never got a reply back, not an interview, the few that reach back(like 3, from 3 or 4 dozen of startups), was just for say their are not interested on me.
This is frustrating, what i do on my days is just push forward my personal projects without rest. I will be broke in a few months from now if i dont get a job, im still young, i have 21 years, but i dont have economic support from parents anymore(they are already broke). Truly dont know what to do. Currently my brother is helping me with the money, but he will broke in few months as i say.
The worst of all this case is that i feel capable of get things done, i have skills and i trust in myself. This is not about me having doubts about my skills, but about startups that dont care, they are not interested in me, and the other worst thing is that my profile is in high demand, at least on startups, they always seek for backend devs with Machine Learning knowledge. Im nothing for them, i only want to land that first job, but seems to be impossible.
For add to this situation, im from south america, Venezuela, and im only able to get a remote job, because in my country basically has no Tech Industry, just Agencies everywhere underpaying devs, that as extent, dont care about my profile too!!! this is ridiculous, not even that almost dead Agencies that contract devs for very little payment in my country are interested in me! As extra, my economic situation dont allows me to reallocate, i simple cant afford that. planning to do it, but after land some job for a few months. Anyways coronavirus seems to finally set remote work as the default, maybe this is not a huge factor right now.
I try to find job as freelancer, i check the freelancer sites(Freelancer, Guru and so on) every week more or less, but at least from what i see, there is no Backend-Only gigs for Python Devs, They always ask for Fullstack developers, and Machine Learning gigs i dont even mention them.
Maybe im missing something obvious, but feel incredible that someone that has skills is not capable of land even a freelancer job. Maybe im blind, or maybe im asking too much(I feel the latter is not the case). Or maybe im overestimating my self? i think around that time to time, but is not possible, i have knowledge of Rest/GraphQL APIs Development using frameworks like Flask or DJango(But i like Flask more than DJango, i feel awesome with its microframework approach). Familiarized with containerization and Docker. I can mention knowledge about SQL and DBs(PostgreSQL), ORMs(SQLAlchemy), Open Auth, CI/CD, Unit Testing, Git, Soft DevOps Skills, Design Patterns like MVC or MTV, Serverless Environments, Deep Learning Solutions, end to end: Data Gathering, Preprocessing, Data Analysis, Model Architecture Design, Training and Finetunning. Im familiarized with SotA techniques widely used now days, GANs, Transformers, Residual Networks, U-Nets, Sequence Data, Image Data or high Dimensional Data, Data Augmentation, Regularization, Dropout, All kind of loss functions and Non Linear functions. My toolset is based around Python, with Tensorflow as the main framework, supported by other libraries like pandas, numpy and other Data Science oriented utils.
I know lot of stuff, is not that enough for get a Junior Level underpaid job? truly dont get it, what is required for get a job? not even enough for get an interview?
I have some dev friends and everyone seems to be able to land jobs, why im not landing even an interview?
I will keep pushing my Dev career, is that or starve to death. But i will love to read your suggestions! how i can approach this?
i will leave here my relevant social presence:
https://linkedin.com/in/...
https://github.com/ElPapi42
Thanks in advance!9 -
When you change some code which perfectly worked out of boredom. You realise it still works (or at least it looks like it does). You begin to wonder why. You never find out.3
-
Do you recommend hiring junior or mid-level dev for a python role that involves mostly data transformations with pandas, growth and marketing projects like social media bots, consuming APIs for data and some experience with Azure SQL db? I’m worried if we hire too senior then they will leave as the role doesn’t involve any advanced software engineering, like caches, web apps, rest apis, etc. It’s more of a handyman that can automate and hack a solution to a business problem: for example, learning openCV to automatically crop thousands of images extracting only the text3
-
Could we have other dark themes on here? I was thinking of something with different combinations of colours and contrast options. What do you think?2