Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "file-loss"
-
Probably the most rage inducing data loss story...
When it comes to my cellphone I'm a data hoarder, I store each relevant meme, conversation, video, contact, nudes, etc. Had to replace my phone? Easy, change the SD.
I did this for about 4 years, had over 11GB of almost everything and anything in a 36GB SD, one afternoon my buddies and I went to a small tech convention and on our way to my car we got mugged by 5 armed men.
They took my brand new phone along with my wallet and all my cash, luckily I had GPS tracking enabled and we were able to pinpoint the exact location of my phone within 30min.
So far so good...
We called the cops and went with them, we found the car with illegal plates and weapons inside (knives, a bat, gun) so I tell the robbers were in there inside a closed cyber cafe and showed him the point on the map confirming this.
Cop: oh we can't do that we don't have an order...
Me: are you kidding me, here's the GPS, there's the car, there's the weapons, doesnt that count as at least probable cause or some shit?
Cop: we don't have that in this country, you can file a report and after 3 business days we can come here to inquire.
Me: (fucking lost it) do you fucking think they'll be here in 3 days?! I'll give you 500 bucks if you go bust their ass now.
Cop: (thinks about it) but what if they are armed? [4 patrols, 8 cops, 4 rifles and at least 6 guns plus vests] Maybe if you had contacts within the bureau we could have an order now...
(┛✧Д✧))┛彡┻━┻
I lost a lot that day, including respect to this fucked up system.
t(ಠ益ಠt) FUCK THE POLICE go eat a dick.10 -
MTP is utter garbage and belongs to the technological hall of shame.
MTP (media transfer protocol, or, more accurately, MOST TERRIBLE PROTOCOL) sometimes spontaneously stops responding, causing Windows Explorer to show its green placebo progress bar inside the file path bar which never reaches the end, and sometimes to whiningly show "(not responding)" with that white layer of mist fading in. Sometimes lists files' dates as 1970-01-01 (which is the Unix epoch), sometimes shows former names of folders prior to being renamed, even after refreshing. I refer to them as "ghost folders". As well known, large directories load extremely slowly in MTP. A directory listing with one thousand files could take well over a minute to load. On mass storage and FTP? Three seconds at most. Sometimes, new files are not even listed until rebooting the smartphone!
Arguably, MTP "has" no bugs. It IS a bug. There is so much more wrong with it that it does not even fit into one post. Therefore it has to be expanded into the comments.
When moving files within an MTP device, MTP does not directly move the selected files, but creates a copy and then deletes the source file, causing both needless wear on the mobile device' flash memory and the loss of files' original date and time attribute. Sometimes, the simple act of renaming a file causes Windows Explorer to stop responding until unplugging the MTP device. It actually once unfreezed after more than half an hour where I did something else in the meantime, but come on, who likes to wait that long? Thankfully, this has not happened to me on Linux file managers such as Nemo yet.
When moving files out using MTP, Windows Explorer does not move and delete each selected file individually, but only deletes the whole selection after finishing the transfer. This means that if the process crashes, no space has been freed on the MTP device (usually a smartphone), and one will have to carefully sort out a mess of duplicates. Linux file managers thankfully delete the source files individually.
Also, for each file transferred from an MTP device onto a mass storage device, Windows has the strange behaviour of briefly creating a file on the target device with the size of the entire selection. It does not actually write that amount of data for each file, since it couldn't do so in this short time, but the current file is listed with that size in Windows Explorer. You can test this by refreshing the target directory shortly after starting a file transfer of multiple selected files originating from an MTP device. For example, when copying or moving out 01.MP4 to 10.MP4, while 01.MP4 is being written, it is listed with the file size of all 01.MP4 to 10.MP4 combined, on the target device, and the file actually exists with that size on the file system for a brief moment. The same happens with each file of the selection. This means that the target device needs almost twice the free space as the selection of files on the source MTP device to be able to accept the incoming files, since the last file, 10.MP4 in this example, temporarily has the total size of 01.MP4 to 10.MP4. This strange behaviour has been on Windows since at least Windows 7, presumably since Microsoft implemented MTP, and has still not been changed. Perhaps the goal is to reserve space on the target device? However, it reserves far too much space.
When transfering from MTP to a UDF file system, sometimes it fails to transfer ZIP files, and only copies the first few bytes. 208 or 74 bytes in my testing.
When transfering several thousand files, Windows Explorer also sometimes decides to quit and restart in midst of the transfer. Also, I sometimes move files out by loading a part of the directory listing in Windows Explorer and then hitting "Esc" because it would take too long to load the entire directory listing. It actually once assigned the wrong file names, which I noticed since file naming conflicts would occur where the source and target files with the same names would have different sizes and time stamps. Both files were intact, but the target file had the name of a different file. You'd think they would figure something like this out after two decades, but no. On Linux, the MTP directory listing is only shown after it is loaded in entirety. However, if the directory has too many files, it fails with an "libmtp: couldn't get object handles" error without listing anything.
Sometimes, a folder appears empty until refreshing one more time. Sometimes, copying a folder out causes a blank folder to be copied to the target. This is why on MTP, only a selection of files and never folders should be moved out, due to the risk of the folder being deleted without everything having been transferred completely.
(continued below)29 -
Today on incompetent profs & classmates...
Dumb student forgets to exit vim and rewrites program 3 times before calling d prof
for help
I ask the prof for approval to use an IDE or a text editor in lab and she has no clue what an IDE is. I installed atom just as she left.
Another kid fooled into thinking web dev has no future.
Apparently I can't use laptop in class to execute programs as "other students may be at loss" i mean wtf that's their fkin problem why do I have to suffer.
Student questions unix prof about the file size limitation in fat32. She had a poker face.
Prof gives "hello world" program to sophomores. Nice.14 -
A few weeks ago, I was kept up until the wee hours of the morning trying to figure out how in the hell the Monty Hall problem works. After finally getting it (I'm slow, okay?), I decided to write a program to run simulations of it.
First incarnation of program took user input. User enters what door they choose (1, 2, or 3), then is told what door Monty opens, then given the decision of staying with the door they originally chose or switching, then informed how that worked out for them.
Second incarnation of program ran on a loop. At the start of each loop, a random door is picked for the user guess. Then the door Monty opens is calculated from the remaining doors (excludes user guess and prize door). Then user switches doors (choosing the door that was not their original door or the door Monty opened). At the end of each loop, if the door they switched to was the prize door, it would increment a win counter, else increment a loss counter. After running the loop 1000000000 times, it printed to console `You always switched doors, resulting in ${wins} wins and ${losses} losses`.
THEN I decided to write a variation to run a while loop on the outside of the loop to increase the number of total doors until the point where the decision to switch doors hurt more often than it helped. At this point, I decided to incorporate file I/O and write to a file rather than a console. And that was neat!
And then I decided it would be cool to go back to the three door variation, printing on each loop the original door, the door Monty opened, the door that was switched too, the result of the switch (win or lose) and what the prize door was.
But for the life of me, I couldn't seem to get the file to write properly. It would, like, always crash my terminal. I tried open + append, I tried append. I tried createWriteStream. Still just failure.
And then I changed it to an appendFileSync and happened to look at one of the files that I was writing to. "Huh, over a gig seems a lot."
"Well, how much are you writing each loop? Did you forget to keep in mind how many bytes that would be?"
TLDR: If you're going to write a program that's going to write data to a file on a loop, you might want to figure out how much it's going to end up writing .... before trying to run it. And running a loop 1000000000 times may be a little excessive.
*face palm*2 -
Living on the edge!
One or two years ago I managed to deploy a DDL change directly on the production server. As I knew there was a backup job which will run every day at noon and at midnight. So I run my script some minutes after noon. So far so good. But somehow I tested it badly in my test environment and the UI of the application throws error after error now in production.
Well, just revert the db to the latest recovery point with the backup, I thought.
It became clear then after a couple of minutes of searching the backup folder for the db backup that there was no such file. The youngest backup file was 3 years old.
Now what happened: The backup script had a switch "simulate=true" and then simulated a successful backup on each run. Therefore the monitoring system got no alerts for not correctly executing those jobs correctly. Then the monitoring job which should do the backupfolder surveillance stuck with green, because there was a valid backup file inside. But it did not check for a specific creation date.
Now this database is the one we need for doing our daily business and is really crucial. Therefore It was easier to emergencyfix the application than doing a rollback of the db 🙄
Well, not really a data loss story, but close to one. -
My .pryrc and a Ruby script it loaded (in another directory) both disappeared seemingly without cause. I lost days of work including a bunch of debugging and performance utilities I wrote over the past year.
But I have no clue how this happened. Neither the .pryrc file nor the script’s folder are tracked by git, so it wouldn’t have been deleted, overwritten, stashed and dropped, etc. None of the other dot files are missing, and the folder is still present, albeit with one fewer files. I wouldn’t delete them, and commands that would delete them do not appear in my zsh history. So I’m at a loss. Figuratively and literally.
They’re just. Gone.
Is there any way to recover missing files on OSX?
I never thought I’d need a backup solution for local scripts.9 -
With the billions of dollars Google has, they can't even build a proper file manager for their Android operating system.
The pre-installed file manager on Android OS, codenamed "DocumentsUI", is functionally crippled and lacks the most basic functionality.
First of all, there is no range selection or A-to-B selection of items. If many items need to be selected, each item has to be tapped individually. Meanwhile, ES File Manager had A-to-B selection since at least 2012, back when Android OS was an operating system of freedom, before Android OS got cucked.
As any low-tier mobile app, the file manager by Google also lacks a draggable scroll bar, so long lists have to be scrolled through manually. Even the file manager of Windows Mobile 6.5 Professional has a draggable scroll bar! And Windows Mobile 6.5 Professional was released in 2009! Samsung "My Files" had a draggable scroll bar in 2013 but it was later unexplainably removed.
Its search feature can only search the entire storage, not an individual folder, and lacks filters such as date and file type.
Obviously, as in any terrible Android file manager, after items are selected for copying and moving, tapping "Copy to..." or "Move to..." navigates back to the initial directory rather than staying in the current directory. The user is forced to navigate all the way to the folder with the selected files if the intention was moving files to a sub folder. Any Android file manager that does this automatically qualifies as a low-tier file manager.
The file manager by Google even lacks a "details" feature which shows information such as the exact file size and name and the total size and file count of a folder. Some file managers such as the one by MediaTek are unable to show the details for multiple selected items, which is somewhat forgivable, but the Google file manager does not have a "details" feature to begin with.
Files are always sorted alphabetically after each start. The Google file manager does not memorize if the user selects sorting "by size" or "by last modified". As one might expect, it indeed lacks reverse sorting.
Of course, there is no "open with" feature where the application can be selected manually, and there is no ability to create new blank files, and it lacks tabbed browsing, and does not show the number of files inside folders in list view. ES File Manager (before it became adware in ~2016) has all of these features.
Last but not least, there has been a bug where cancelling a file move operation deletes the source folder without it having been transferred. Presumably it has been patched by now, however, a bug where tapping "cancel" leads to data loss is inexcuseable. It shows the app has not even been properly tested, let alone properly created.
http://archive.today/2020.10.27-160...
Google could have hired a college student who could have built something better than the scrapyard-worthy "file manager" they have built.
But granted, at least Google's ever-so-terrible file manager does not limit file names to fifty (50) characters like Samsung's TouchWiz file manager, also known as "My Files", did until at least 2016. There is no way to know what went through the head of the programmer who implemented this pointless limitation. Google's file manager also correctly handles file name conflicts by renaming the new files.
Microsoft built a better file manager for their operating system decades earlier than what Google threw together. Microsoft spent more of their money building a proper file manager.6 -
Moving files is emotionally easier than copying and deleting files, and moving eliminates the risk of selecting the wrong files at the deletion part.
I have read that it is safer to manually copy and manually delete files rather than to move it, but copying and deleting has a hidden risk that was not mentioned: selecting the wrong files for deletion.
Moving files feels like moving an obstacle from one room to another. The deletion part of copying and deleting feels like destroying something, which is an added emotional barrier.
Technically, copying and deleting is safer, since there is no risk of source files being deleted without having been transferred as a result of a device disconnecting or the buggy media transfer protocol (MTP) failing to load the entire file list. However, on mass storage devices, this pretty much never happened to me, and on MTP, data loss can be avoided by not moving folders but opening the source folders and selecting all files and moving those out. This prevents a parent folder with incompletely loaded file listing from being deleted.
However, something that is not considered about copying and deleting is that the risk of selecting the wrong files in the deletion step exists. One might end up selecting files that were never copied.
Not only is moving straightforward and time-saving, but it has no emotional barrier and the risk of selecting the wrong files to delete from the source is eliminated, since a proper file manager like Nemo or Windows Explorer (mass storage only, not MTP) only deletes a moved file from the source after it has been properly transferred. The user does not need to pay attention to select the correct files to delete, since the file manager already did it.4 -
It's 2022 and mobile web browsers still lack basic export options.
Without root access, the bookmarks, session, history, and possibly saved pages are locked in. There is no way to create an external backup or search them using external tools such as grep.
Sure, it is possible to manually copy and paste individual bookmarks and tabs into a text file. However, obviously, that takes lots of annoying repetitive effort.
Exporting is a basic feature. One might want to clean up the bookmarks or start a new session, but have a snapshot of the previous state so anything needed in future can be retrieved from there.
Without the ability to export these things, it becomes difficult to find web resources one might need in future. Due to the abundance of new incoming Internet posts and videos, the existing ones tend to drown in the search results and become very difficult to find after some time. Or they might be taken down and one might end up spending time searching for something that does not exist anymore. It's better to find out immediately it is no longer available than a futile search.
----
Some mobile web browsers such as Chrome (to Google's credit) thankfully store saved pages as MHTML files into the common Download folder, where they can be backed up and moved elsewhere using a file manager or an external computer. However, other browsers like Kiwi browser and Samsung Internet incorrectly store saved pages into their respective locked directories inside "/data/". Without root access, those files are locked in there and can only be accessed through that one web browser for the lifespan of that one device.
For tabs, there are some services like Firefox Sync. However, in order to create a text file of the opened tabs, one needs an external computer and needs to create an account on the service. For something that is technically possible in one second directly on the phone. The service can also have outages or be discontinued. This is the danger of vendor lock-in: if something is no longer supported, it can lead to data loss.
For Chrome, there is a "remote debugging" feature on the developer tools of the desktop edition that is supposedly able to get a list of the tabs ( https://android.stackexchange.com/q... ). However, I tried it and it did not work. No connection could be established. And it should not be necessary in first place.7 -
(a slide acoustic guitar plays on the background and the cowboy starts speaking)
It was a dry october day, back in good old 2017. I had this job from a client that I never met and was doing some coding for money.
After days of no sleep, no food and no rest, I finally decided to take a nap so I paused my music.
It was at this moment I found out my machine was making funny noises. Like a dingo makin' a run from it's enemies with a whelping noise.
Clicked on my computer and tried to find an ol' file from the archive drive but the machine won't let me, sayin' the disk ain't ready yet.
I tried disk manager, disk scanner, whatever the tools at my disposal all in vain. Then I said what the hell, I'll just restart my machine and it'll be alright.
The machine rebooted but the disk was gone. It was dead like a deer I ran over. I was upset, but not aware of the calamity headin' my way.
In just a few days my other 2 disks died suddenly. The loss of data, all the effort, none of them mattered. I felt numb and decided it was time for a fresh start.
Plugged in a Windows install disk, started the sequence, a screen came up askin' me which damned and alive disk I wanna install the fresh OS. I had two same make and model SSD disks, chose the one thinkin' it was the Windows drive, hell it wasnt... It was with all "my documents", "downloads", "pictures" folders and now I had two SSD drives with two Windows installations and nothing else.
The folks in town took a dab at me for months, even the bartender of the salloon refused to give me a drink. Sayin' it was a matter of reputation...
Turned out the bastard who fried my disks was the Madde Dog PSU Tannen who had a bad temper so here I am, tellin' my story to milk breathers and cherishing old days of data...3 -
If I could create laws, I would pass a "software usability act" which would eliminate many annoyances we face daily.
For example, the law would mandate range selection in file managers, mandate time-stamped file names in camera and voice recording apps, and require that browsers open a new tab next to the currently open tab instead of at the end, and all user interfaces must have a dark mode to reduce eye strain, and all operating systems must have a blue light filter, text editors must create a temporary copy when saving to avoid corrupting the existing file, camera applications should not corrupt the entire video file when ending unexpectedly (crashing), cancelling file operations must not cause data loss ( https://support.google.com/photos/... ), no mandatory pull-to-refresh ( https://chromestory.com/2019/07/... ), to mention a few examples.
Mobile file managers commonly lack a range selection feature (also known as shift selection or A-to-B selection), where all items between two selected items of a list can be selected immediately. ES File Explorer had this in 2012, yet many fancy new file managers still don't have this. To select many items, each item needs to be tapped individually. This is an unacceptable annoyance.
This is not to be confused with the inferior drag-to-select which requires holding the finger on the screen until all desired items are selected. Drag-to-select is not range selection, only its ugly stepsister.
Ah yes, under the imaginary software usability act, Mozilla would have to say good-bye to its evil add-on signing. "For our protection" my arse.13 -
Not only does every app need to have an export option, but new exports must create new, time-stamped files rather than overwriting an existing export!
A counter-example is "Battery Monitor Widget" by CCC71 or 3C71. That app creates a file in the main user directory, named "bmw_history.txt" (no relation to the car manufacturer).
When a new export is created, the existing bmw_history.txt is overwritten. This could lead to data loss if the user is unaware of this behaviour.
The developer thought of creating an export ability, but messed up at the file naming process.
Mandatory time-stamped user data exports for every app would not be so bad. This makes sure no developer would forget about it. GDPR gave us data portability for social media platforms. Let's do it for apps too. (Sorry, Samsung Internet, you can no longer lock in saved pages. Your users are sick of it.) -
I can work productively and for very long hours with a lot of stuff which many dev considers productivity hurdles:
- single small monitor? No problem (in fact in one occasion in which my roommate accidentally broke my laptop charghing port and I couldn't get a spare I worked on an iPad connected trough SSH to a Linux machine completing one of the hardest tasks I ever did without significant loss of productivity)
- old machine? That's ok as long as I can run a minimal Linux and not struggle with Windows
- noise and chatter around me? A 10€ pair of earbuds are enough for me, no noise cancelling needed
- "legacy" stack/programming language? I'd rather spend my days coding in Swift or Rust but in the end I believe which is the dev and its skill which gets the job done not fancy language features so Java 8 will be fine
- no JetBrains or other fancy IDE? Altough some refactoring and code generation stuff is amazing Neovim or VS Code, maybe with the help of some UNIX CLI tools here and there are more than enough
despite this I found out there is a single thing which is like kryptonite for my productivity bringing it from above average* to dangerously low and it's the lack of a quick feedback loop.
For programming tasks that's not a problem because it doesn't matter the language there's always a compiler/interpreter I can use to quickly check what I did and this helps to get quickly in a good work flow but since I went to work with a customer which wants everything deployed on a lazily put together "private cloud" which needs configurations in non-standard and badly documented file formats, has a lot of stuff which instead of being automated gets done trough slowly processed tickets, sometimes things breaks and may take MONTHS to see them fixed... my productivity took a big hit since while I'm still quick at the dev stuff (if I'm able to put together a decent local environment and I don't depend on the cloud of nightmares, something which isn't always warranted) my productivity plummets when I have to integrate what I did or what someone else did in this "cloud" since lacking decent documentation everything has do be done trough a lot of manual tasks and most importantly slow iterations of trial and error. When I have to do that kind stuff (sadly quite often) my brain feels like stuck on "1st gear": I get slow, quickly tired and often I procrastinate a lot even if I force myself out of non work related internet stuff.
*I don't want this to sound braggy but being a passionate developer which breathes computers since childhood and dedicating part of my freetime on continuously improving my skill I have an edge over who do this without much passion or even reluctantly and I say this without wanting to be an èlitist gatekeeper, everyone has to work and tot everybody as the privilege of being passionate in a skill which nowadays has so much market2 -
The "recycle bin" feature of Samsung "My Files" is amazing for data loss prevention when moving files out of the smartphone.
There used to be two ways to move files out of the smartphone to make space free. One is direct moving, the other is copy-deletion. The first is self-explanatory, the second means first copying the files and then deleting them on the phone.
Thanks to the the recycle bin, which keeps data for a month, files on the phone can be copied out and then put into the recycle bin instead of immediately deleted.
This means that if the copying was incomplete, there is a thirty-day grace period to get the files back from the phone.
The benefit of moving files instead of copy-deleting them is the lack of the deletion step. Moving files out directly does not have the emotional barrier of deleting the files from source like the deletion step of copy-deleting does.
Moving files feels like moving items to a new room, where as the deletion step after copying feels like destroying something.
So why not move files out? Because there is a risk of data loss if the device disconnects while files are moved to an USB OTG device. Due to write buffering, files that are moved out might be deleted on the phone shortly before they are completely written on the USB-OTG.
This is not an issue with MTP (Windows or Linux through USB cable) because the file systems are managed by the computer, so if the phone disconnects while files are moved out of the phone using MTP, the file system is kept intact by Windows or Linux.
Now, thanks to the recycle bin, there is no emotional barrier to deletion because the files on the phone are automatically deleted after 30 days in the absence of the user. The user can press the "delete" button without worries because of knowing "I can get it back until a month from now anyway".