Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
nitwhiz11292277dWhat. The Fuck. Do you do?
Mosesrocks729277d@nitwhiz I wanted to store a list that contains the names of my training data examples, so that it's faster loading in and reading the excel file, rather that having to wait for os.listdir() lmao, and I have 1,200,000 training examples.
I actually just ended up saving a .npy file and it works even better. Loads in milliseconds :)
Jilano27831277dA colleague of mine had the same issue few days ago.
Long story short, he had to import a CSV file into a program and it's supposed to show the result in an excel file. He didn't understand right away the "out of bound" error, but when I opened the initial CSV, I saw around 800 legit rows and the rest was just empty lines with the separator. Yeah, 1.2 millions or so useless lines. No wonder his shit was taking ages to process.
Mr-Myrk633277dAre you generating some future thedailywtf material by using excel as a database? Please think of people who come after you and don’t
iamgio3999277dYou could use csv files instead
sirjofri551277dDepending on the task most of the time I use simple text files with tab separated line records and standard unix tools like grep, awk, sed. If I don't need to read in the whole file at once it's the fastest possible solution