Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Documents folder gone (with three years of work...)

If none of the specific sub-forums seem right for your thread, ask here.
Message
Author
F0 0F C7 C8
Posts: 4
Joined: 2020-09-24 10:07

Documents folder gone (with three years of work...)

#1 Post by F0 0F C7 C8 »

Hello everyone,
My "Documents" folder is missing from my file manager (dolphin) and from my hdd . In system settings, in "Location for Personal Files", the documents folder path is set to my homedir.
When I type "xdg-user-dirs-update" in a terminal I got :
"/home/<username>/Documents was removed, reassigning DOCUMENTS to homedir."
I didn't remove them by mistake, I checked zsh history.
It is likely to be because of updates. I did the standard "apt update", "apt dist-upgrade". The updates were pretty big and was mainly libs and linux-image. Could the updates be the cause ?

I'm running Debian 10 buster, Dolphin 18.08.0, KDE 5.54.0 and Plasma 5.14.5.

andre@home
Posts: 398
Joined: 2011-10-02 08:00

Re: Documents folder gone (with three years of work...)

#2 Post by andre@home »

Put at least your last backup on its place with the contents of that Document folder.

If that's a problem, maybe Testdisk can find back your files.
It is generally advised to make a clone from the problem hard disk where Document folder is on and do the recovery with that other disk.
Stop working now complete on the problem disk as any action may over write the files Testdisk may find.
The best method is to use Testdisk on another computer and use the cloned disk to find your files back.

https://www.cgsecurity.org/wiki/TestDisk
Impression how it works:
https://www.youtube.com/watch?v=-l8Yx3VL6CI

User avatar
sunrat
Administrator
Administrator
Posts: 6412
Joined: 2006-08-29 09:12
Location: Melbourne, Australia
Has thanked: 116 times
Been thanked: 462 times

Re: Documents folder gone (with three years of work...)

#3 Post by sunrat »

It's extremely unlikely upgrades would remove any personal data from your HOME.
If you have 3 years of work you should definitely have it backed up in 2 other places, external drives preferably, and to be extra safe have one copy in another location. Hard drives (and all other storage) should be considered as temporary storage.
It's possible your drive has died or is about to, or maybe another bit of hardware.
As andre@home said, clone your drive to another external disk and work on the copied image with Testdisk/Photorec. Use a live CD/USB Linux to do the clone and then you may be able to use it to see if your files still exist.
“ computer users can be divided into 2 categories:
Those who have lost data
...and those who have not lost data YET ”
Remember to BACKUP!

andre@home
Posts: 398
Joined: 2011-10-02 08:00

Re: Documents folder gone (with three years of work...)

#4 Post by andre@home »

sunrat wrote:It's extremely unlikely upgrades would remove any personal data from your HOME.
I agree... but it can happen... cannot go into too much details because of confidentiality, but someone updated an "office program".. and all kind of important templates were wiped from the Document folder .... It was decided to bring that disk to a recovery company. Cannot tell more... but apparently there were no backups.... unbelievable....

I think to have 5 copies on different media of my private photos and other important data and in principle they are spread over 2 physical locations....


Tip: If you are not sure about your skills... do not do this your self. If its for the company you work for, ask an IT-person who has good/proven recovery skills, to do the jobs.
Or... do as in my example: Ask an recovery company to do the job. The costs depend a lot on the cause of the data loss. If it is "just " a small user failure.. you likely will stay below 1000 US-dollar ?
If it is a hardware failure, it becomes quite unpredictable if the recovery may be successful at all...

sgosnell
Posts: 975
Joined: 2011-03-14 01:49

Re: Documents folder gone (with three years of work...)

#5 Post by sgosnell »

Any files I care about are backed up to external drives, both HDD and SSD, and to both Dropbox and Google Drive. Files with any sensitive information are encrypted before going to the cloud, using gnupg. I also have some encrypted with Minilock in addition to gnupg. I would never trust any single storage device for files that couldn't be easily replaced. The price of storage is so low that it makes no sense at all to me to not use multiple devices, updated daily or more often. Cron makes this easy and reliable.
Take my advice, I'm not using it.

MALsPa
Posts: 678
Joined: 2007-12-07 19:20
Location: albuquerque
Been thanked: 2 times

Re: Documents folder gone (with three years of work...)

#6 Post by MALsPa »

andre@home wrote:but it can happen
Wouldn't the OP be able to examine the info under /var/log/ to determine if this is what happened?

sgage
Posts: 86
Joined: 2013-03-10 21:00
Has thanked: 1 time
Been thanked: 2 times

Re: Documents folder gone (with three years of work...)

#7 Post by sgage »

Three years of work, and no back up? :shock:

andre@home
Posts: 398
Joined: 2011-10-02 08:00

Re: Documents folder gone (with three years of work...)

#8 Post by andre@home »

MALsPa wrote:Wouldn't the OP be able to examine the info under /var/log/ to determine if this is what happened?
Agree, always check the logs. If you keep the original disk untouched it should be easy to find, but I wonder if such a data loss wil be logged?

F0 0F C7 C8
Posts: 4
Joined: 2020-09-24 10:07

Re: Documents folder gone (with three years of work...)

#9 Post by F0 0F C7 C8 »

Thnak you all for your replies.
andre@home wrote:Put at least your last backup on its place with the contents of that Document folder.

If that's a problem, maybe Testdisk can find back your files.
It is generally advised to make a clone from the problem hard disk where Document folder is on and do the recovery with that other disk.
How do I clone a disk ? I have found tools suited for this task but will it really copy every single bit of my disk or just data found by the filesystem ?
Should I use the dd command for example ?
I don't think a mere drag'n'drop would work.

Bulkley
Posts: 6383
Joined: 2006-02-11 18:35
Has thanked: 2 times
Been thanked: 39 times

Re: Documents folder gone (with three years of work...)

#10 Post by Bulkley »

F0 0F C7 C8 wrote:How do I clone a disk ?
I use Clonzilla on a live-CD. Yes, it clones everything including Grub (boot). I clone to a spare drive. I do it before attempting any major changes so that I have an exact replica when I need it. I also save important things elsewhere but that's a different story.

Those who's livelihood depends on computer work need an automatic backup system and there are lots to choose from.

User avatar
sickpig
Posts: 589
Joined: 2019-01-23 10:34

Re: Documents folder gone (with three years of work...)

#11 Post by sickpig »

F0 0F C7 C8 wrote:Should I use the dd command for example ?
dd will work.
Also easier option is gnome disks - allows you to img individual partitions as well.

F0 0F C7 C8
Posts: 4
Joined: 2020-09-24 10:07

Re: Documents folder gone (with three years of work...)

#12 Post by F0 0F C7 C8 »

I ran testdisk and found ... nothing...
Image
The software did find some deleted files and folders but not "Documents".
I ran a deeper search in my disk and the result was :
Image
Lastly, I did a scan of the disk wich resulted in :
Image

It doesn't seem I can recover anything.
Last edited by F0 0F C7 C8 on 2020-09-30 18:42, edited 2 times in total.

F0 0F C7 C8
Posts: 4
Joined: 2020-09-24 10:07

Re: Documents folder gone (with three years of work...)

#13 Post by F0 0F C7 C8 »

It is not that bad anyway. It was mostly homework from previous years. Most projects I worked on are in the git folder. For those not in git, since the projects were in groups, I can still retrive the files from my classmates. Hopefully the pictures and movies folder are not gone :shock:. I will backup everything from now on :D.

User avatar
bester69
Posts: 2072
Joined: 2015-04-02 13:15
Has thanked: 24 times
Been thanked: 14 times

Re: Documents folder gone (with three years of work...)

#14 Post by bester69 »

sgosnell wrote:Any files I care about are backed up to external drives, both HDD and SSD, and to both Dropbox and Google Drive. Files with any sensitive information are encrypted before going to the cloud, using gnupg. I also have some encrypted with Minilock in addition to gnupg. I would never trust any single storage device for files that couldn't be easily replaced. The price of storage is so low that it makes no sense at all to me to not use multiple devices, updated daily or more often. Cron makes this easy and reliable.
I do almost the same, I use Dropbox and Google Drive for my sensible data files with a daily.crontab push daily backups into both clouds by using "rclone sync"; I also use encfs for my private folder and exclude keyfile unlocking from pushing into cloud in order to add a security layer...

For no sensible data but home backups (debs, music, settings files, etc), I use a local deduplication daily borg backup that its pushed each three months to Mega (50Gby) by using rclone and duplicity backend for rclone.. so, borg repository its pushed into cloud by being enveloped in a duplicity upload..; this way you can send a huge amount of data backup to cloud in a very trusted way (Duplicity reliability)..

In resume.:
- My important data documents:
Data + Data encrypted (encfs) => Daily borg repository backup (local backup) => pushed by rclone into Dropbox and GoogleDrive (cloud data and redundancy backup)
- My data Home:
Data Home => Each 3 Months, borg repository backup (local backup2) => pushed by duplicity (huge data upload with rclone's duplicity backend) into Mega (big free Cloud, 50Gb)
bester69 wrote:STOP 2030 globalists demons, keep the fight for humanity freedom against NWO...

tomt
Posts: 65
Joined: 2005-09-09 20:19

Re: Documents folder gone (with three years of work...)

#15 Post by tomt »

You may take some consolation in hearing that this is not the first
time top level dirs in ~/ have disappeared arbitrarily.

Many years ago (2009 & 2013) I had reported instances where multiple
top level dirs had disappeared leaving behind a few as well as all the
dotfiles/dirs. These happened not long after I had migrated to new
versions.

At that time I did not associate the disappearance with updates but
there may have been a link.

Regards, tomt

User avatar
bester69
Posts: 2072
Joined: 2015-04-02 13:15
Has thanked: 24 times
Been thanked: 14 times

Re: Documents folder gone (with three years of work...)

#16 Post by bester69 »

tomt wrote:You may take some consolation in hearing that this is not the first
time top level dirs in ~/ have disappeared arbitrarily.

Many years ago (2009 & 2013) I had reported instances where multiple
top level dirs had disappeared leaving behind a few as well as all the
dotfiles/dirs. These happened not long after I had migrated to new
versions.

At that time I did not associate the disappearance with updates but
there may have been a link.

Regards, tomt
Perhaps, Using chattr +i for important folders... New apps updates might purge its home settings.. its not a good idea to keep data files in those folders without keeping a regular backup.... I dont have those problems since I use btrfs snapshots.. if i delete accidentally a files, I can recover it.
bester69 wrote:STOP 2030 globalists demons, keep the fight for humanity freedom against NWO...

andre@home
Posts: 398
Joined: 2011-10-02 08:00

Re: Documents folder gone (with three years of work...)

#17 Post by andre@home »

Where do you backup these btrfs snapshots?

User avatar
bester69
Posts: 2072
Joined: 2015-04-02 13:15
Has thanked: 24 times
Been thanked: 14 times

Re: Documents folder gone (with three years of work...)

#18 Post by bester69 »

andre@home wrote:Where do you backup these btrfs snapshots?
btrfs snapshots usually goes in a root subvolume path within a hidden folder named .snapshots..
such like this.:
for home subvolume:
/home/.snapshot/1/snapshot
/home/.snapshot/2/snapshot
for root system subvolume:
/.snapshot/1/snapshot
/.snapshot/2/snapshot
for any partition.:
/media/sda2/.snapshots/1/snapshot

Snapshots cant be deleted if not root user, and for deleting , need command "sudo btrfs delete subvolume /path"


I use snapper-gui for automatic managing of snapshots
Image

You wont ever lose the system unless a hardware faillure :wink:
bester69 wrote:STOP 2030 globalists demons, keep the fight for humanity freedom against NWO...

User avatar
conclave
Posts: 23
Joined: 2020-06-02 21:03
Been thanked: 5 times

Re: Documents folder gone (with three years of work...)

#19 Post by conclave »

I'm looking at a book entitled "LInux All-in-One for Dummies" 2014. Last sentence on page 117 reads "A user must have execute permission on a directory before he or she can open and read the contents of the directory".

I'm not quite sure how the search procedures work with the programs you are using to locate this directory yet there may be a possibility that the directory is still on your computer somewhere hiding in the file system or even another operating system if you have others on the same computer.

There are times when upgrading where I have left a directory open as a result of pointing to a directory file to allow rwx permissions through command line chmod, an example: boot/grub/grub.cfg to change the amount of time the boot menu stays open to 300 since 5 seconds is annoyingly too fast. I have then upgraded and when I return to that directory as a result of the upgrade reverting back to the default 5 seconds and my having to change it again the boot directory is locked again. The upgrade automatically creates a new locked boot directory and changes everything back to default. If you run some search programs, which I am not totally sure if they scan through locked files yet maybe the directory is in a place inside a locked directory. This could involve a lot of time browsing through your entire file system where locked directories are looking for that documents directory. Again it could be a waste of time and result in nothing found. Then again maybe the upgrade moved the file somewhere inside a locked directory for some odd reason.

You mention the stuff you have there was homework, which leads me to believe there is a good chance the size in terms of memory used within that directory is relatively low, no large size graphic, video or music files. It is very easy during some other copying of directories and/or files project to accidentally pick up that directory and move it somewhere else and then lock that directory. Again if you have another operating system using the move command it would be possible to accidentally move it to another operating system, that would certainly not show up in any search program ran within that operating system. For a file with less than one hundred mbs of memory that could move fast enough to not even be recognized while working.

User avatar
bester69
Posts: 2072
Joined: 2015-04-02 13:15
Has thanked: 24 times
Been thanked: 14 times

Re: Documents folder gone (with three years of work...)

#20 Post by bester69 »

conclave wrote:I'm looking at a book entitled "LInux All-in-One for Dummies" 2014. Last sentence on page 117 reads "A user must have execute permission on a directory before he or she can open and read the contents of the directory".

I'm not quite sure how the search procedures work with the programs you are using to locate this directory yet there may be a possibility that the directory is still on your computer somewhere hiding in the file system or even another operating system if you have others on the same computer.

There are times when upgrading where I have left a directory open as a result of pointing to a directory file to allow rwx permissions through command line chmod, an example: boot/grub/grub.cfg to change the amount of time the boot menu stays open to 300 since 5 seconds is annoyingly too fast. I have then upgraded and when I return to that directory as a result of the upgrade reverting back to the default 5 seconds and my having to change it again the boot directory is locked again. The upgrade automatically creates a new locked boot directory and changes everything back to default. If you run some search programs, which I am not totally sure if they scan through locked files yet maybe the directory is in a place inside a locked directory. This could involve a lot of time browsing through your entire file system where locked directories are looking for that documents directory. Again it could be a waste of time and result in nothing found. Then again maybe the upgrade moved the file somewhere inside a locked directory for some odd reason.

You mention the stuff you have there was homework, which leads me to believe there is a good chance the size in terms of memory used within that directory is relatively low, no large size graphic, video or music files. It is very easy during some other copying of directories and/or files project to accidentally pick up that directory and move it somewhere else and then lock that directory. Again if you have another operating system using the move command it would be possible to accidentally move it to another operating system, that would certainly not show up in any search program ran within that operating system. For a file with less than one hundred mbs of memory that could move fast enough to not even be recognized while working.
Thats why I hate all filebrowsers but kde's one (dolphin).. In dolphin you cant move accidentally files or directories, cos it doesnt let you drag and drop without asking you previoulsy what type of operation are you comitting (coying, moving or linking)... I also apply chattr +i for sensible folders, that way, if they are moved, unless cant be deleted.. so if you move them accindetally to trash folder, you wont lose them when try to empty trash can... anyway using snapshots you 're more covered.
bester69 wrote:STOP 2030 globalists demons, keep the fight for humanity freedom against NWO...

Post Reply