Silly question, but why is it that in this age of 64-bit computing and gigabytes of RAM applications still have trouble with text files/SQL dumps over 1MB in size? Surely for something so simple it should be able to store it all in memory without any issues, no?

Maybe because some applications use a shitty data-structure to store them.

Huge text files are always a problem, but some editors handle it worse than others.

For example if you have multiple MBs of text the program may still attempt to format it all for the current window size or do some other stupid preprocessing. And when anything in that processing chain is kinda slow or does a lot of memory allocations, and you suddenly increase the input size beyond the kB range you're in for a long wait.

I'm not sure that's the only reason, but I've seen some benchmarks and the performance difference between certain editors is huge in such scenarios.

Back then when we still had wooden buckets for the bytes, programmers were smart, but computers slow. Today, computers are fast and many programmers stupid. That's why shit is still slow although we now have carbon fibre laminate buckets.

I had a 40mb excel file, once opened in 64-bit excel, it grew more than 2gb. It had lots of pivot tables, that could be the reason but the final size in memory is not same as file size.

Processor caches, a.k.a. where the magic happens are still not that much bigger. If a program is written in a way it loads all data when you open a file it can leat to quite a few cache misses...

@Wack a Ryzen 7 has 16 MB of L3 cache - that's more than computers of the Win95 era had as total RAM!

Sure, but isn't the L3 shared amongst all cores?

@Wack it is, but that's no reason to choke on a measly MB of text. That kind of shit was already fast on CPUs that didn't even have on-chip cache - provided that the devs didn't code O(n^2) or worse left and right.

There shouldn't be any problems in making/reading a 1 MB SQL dump on any standard 64 bit system.

The problem you're looking for is probably elsewhere.