Finally arrived! :)

C: hey mate, what's the best tool to open up this 31.1M rows x 106 cols CSV file?

M: Umh...Pandas DataFrame or R DataTable I guess?

C: all right, Excell will do, thanks!

M: erhm...yeah, anytime?

I was pulling my hair wondering why the DataFrame was not merging. Took my 1 hour and a half to figure it out.

Just thought of this meme template when I got that error. Hope you like it 😅

BTW it was a KeyError in a pandas DataFrame during which, somehow, another KeyError for the same key occurred. I'm not sophisticated enough in Python to explain this though.

When I began my career, I always thought everyone in the IT-sector loved their Jobs (except a few people who didn't fit in of course) - 10 years later I've met a lot of people who hated their jobs but didn't just quit it (no kids, no wife, no responsibilities at all) - what do you guys think about this?

Personally, I'm looking for a job where I can put my heart into and which has got a meaning for me (haven't found it yet and I don't expect to do in a short time) because everything else makes no sense to me.

Im in IT for nearly 7+ years now. Until now, I worked for companies in germany that give a damn about the projects their doing or somewhat else - the only thing they consider is money (what Im absolutely able to understand, a company needs to pay its employees, etc).

I worked for three companies now and every single one of them was the same alime. I started to get depressive because I always thought there must be a company that takes proud of the craftmanship itself, instead of burning every one of its employees when its needed.

I love to work in IT, but I hate it to work for those kind of companies.

I began to study Computer sciences at a university and now in order to enhance my range of possibilities, and as there is only 1½ year left, Im asking myself what I want to do. I want to work AS a developer or something like this in IT but I also dont want to hate my job - so, what would you recommend to someone like me?

What is this format? Spaces are separated by + sign and columns are separated by & sign and it’s value is followed by = sign. Can I directly convert it to Spark dataframe?

WHY does VS code load up Pandas dataframes so damn slowly? It’s bad enough that it seems to take an extra few seconds to get PyQt5 going, but the dataframes are awful, even with small 50 record Parquet files.

I don’t have the attention span to sit there and wait for this without finding myself playing with my phone or surfing.

I guess for debugging and testing I should just create a column A, column B, column C dataframe on the fly and give it some 1, 2, 3 kind of values.

But, Jesus, man... This shouldn’t take 30 seconds to load a simple form. 🙄

Hi! Is there a complex join operation for “updating” values in Dataframe using before and after values of the Uodate?

Hi guys ,
I need your help
I have to query to impala and store the result in a panda dataframe .
But as the data is very large it not happening can you give any suggestions
Thanks

Why is my Spark Dataframe getting written as empty in the database table when deleting records? There’s no problem when inserting records, and I I have verified that my Spark Dataframe is not empty when trying to rewrite using Overwrite method. Then why is my whole table data getting deleted when I’m just trying to remove a few records using left_anti join?