I never thought that the excel row limit (2 ^ 20) would be a problem/insufficient in my life.

What. The Fuck. Do you do?

@nitwhiz I wanted to store a list that contains the names of my training data examples, so that it's faster loading in and reading the excel file, rather that having to wait for os.listdir() lmao, and I have 1,200,000 training examples.
I actually just ended up saving a .npy file and it works even better. Loads in milliseconds :)

A colleague of mine had the same issue few days ago.

Long story short, he had to import a CSV file into a program and it's supposed to show the result in an excel file. He didn't understand right away the "out of bound" error, but when I opened the initial CSV, I saw around 800 legit rows and the rest was just empty lines with the separator. Yeah, 1.2 millions or so useless lines. No wonder his shit was taking ages to process.

Are you generating some future thedailywtf material by using excel as a database? Please think of people who come after you and don’t

You could use csv files instead

Depending on the task most of the time I use simple text files with tab separated line records and standard unix tools like grep, awk, sed. If I don't need to read in the whole file at once it's the fastest possible solution