Fsarchiver and BTRFS (hotbackup)

[bester69]

Hi,

Can you please confirme me these steps allow me to do a properly hotbackup??

1- I created a snapshot from the filesystem
btrfs subvolume snapshot / /.snapshosts/

2- Mounted the snapshot in /mnt:
mount -t btrfs -o subvolid="id" /dev/sda2 /mnt

3- chroot snapshot:
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys
mount --bind /media/fsbackup /mnt/media/fsbackup
chroot /mnt

4- Fsarchiver backup from btrfs snapshot
fsarchiver savefs -Av -s 2400 -z 2 /media/fsbackup/backup.fsa /dev/sda2 \
--exclude=/home \
--exclude=.snapshots \
--exclude=/.snapshots \
-------------------------------------

thanks a lot.

[pylkko]

I don't get it. If you want a backup, why don't you just use a snapshot, that can be done "live", you don't need to take the computer offline.

viewtopic.php?f=16&t=130512

The way I do it now, is that I have a place holder subvolume. Then I make a read only snapshot onto an external disk (this is "hot"). Then I recover by snapshotting the backup to the place holder location. I find this to be the easiest solution for me because I want to be able to make edits to the backup before I bring it online. Unless the stuff that needs to be revered is so minimal, that it makes more sense to just revert the individual files.

Or do you mean that you want to be able to do the entire process without a reboot, like make the backup snapshot and bring it back?

EDIT:

Have you noticed that fsarchiver site says:

Limitations
There are several limitations anyway: it cannot preserve filesystem attributes that are very specific. For instance, if you create a snapshot in a btrfs volume (the new-generation file system for linux), FSArchiver won’t know anything about that, and it will just backup the contents seen when you mount the partition.

[bester69]

I don't get it. If you want a backup, why don't you just use a snapshot, that can be done "live", you don't need to take the computer offline.

viewtopic.php?f=16&t=130512

The way I do it now, is that I have a place holder subvolume. Then I make a read only snapshot onto an external disk (this is "hot"). Then I recover by snapshotting the backup to the place holder location. I find this to be the easiest solution for me because I want to be able to make edits to the backup before I bring it online. Unless the stuff that needs to be revered is so minimal, that it makes more sense to just revert the individual files.

Or do you mean that you want to be able to do the entire process without a reboot, like make the backup snapshot and bring it back?

Thanks for your help

I'd like to put the filesystem backup in another partition, in just 5GB, so im compressing and using fsarchiver for it. If i had enought free space I'd use another btrfs partition. There was a time I had corrupted the btrfs partition and had to start installartion from zero, It takes me much hours to polish my installation and i dont want to repeat it. So im using fsarchiver against a snapshot, and tought i'd be better to use a snapshot to prevent problems, .. My first try failed, the filesystem went up without free space, I dont know what i did wrong, im gonna try again i might be doing some cyclic copy.

[bester69]

Have you noticed that fsarchiver site says:

Limitations
There are several limitations anyway: it cannot preserve filesystem attributes that are very specific. For instance, if you create a snapshot in a btrfs volume (the new-generation file system for linux), FSArchiver won’t know anything about that, and it will just backup the contents seen when you mount the partition.

I know that limitation, It doesnt unserstand btrfs subvolumes, so you cant backup subvolume for separate. Im excluding subvolume home..and snapshots

[pylkko]

But are you using the compression mount flag already or not? Because what point in compressing if it is already compressed?

Why do you use

Code: Select all

btrfs subvolume snapshot / /.snapshosts/


instead of the normal

Code: Select all

btrfs subvolume snapshot / /.snapshosts/snapshot


?

[bester69]

But are you using the compression mount flag already or not? Because what point in compressing if it is already compressed?

Why do you use

Code: Select all

btrfs subvolume snapshot / /.snapshosts/


instead of the normal

Code: Select all

btrfs subvolume snapshot / /.snapshosts/snapshot


?

I wrote it down of memory. sorry

Compression flag???, you're saying btrfs option in fstab??, Im not using that option, but i think that is only for a compression on fly, in order to accelerate IO disks operations , isnt it?,

I just want to use properly fsarchiver to store a btrfs filesystem recoverable backup, for example in two dvds. fsarchiver has compression option available, which safe some Gygas. In past I got with success some recoverable live backups with fsarchiver, altought its suppose there is not garantee to achieve it with a live backup, So that im using btrfs to take advantage of btrfs snapshots for a properly live fsarchiver backup (I dont want to go into a LIvecd session).
I was just asking if my steps were ok and that can be done in that way, thats all.

[bester69]

I finally did it, by booting in a 4.9 kernel, even so, when backup reached 85% it started to rise much cpu, then i couldnt boot in 4.39 (many services failed in boot) and could do it in 4.9, i think olders kernels might give some issues with btrfs. within 4.9 went back to a older snapshot, cleaned some snapshots, and was able to bootback to my default 4.39 kernel. I just hope the fsarchiver backup works if i need eventually, it ended without errors ..

Thanks, again Pytkio.

[pylkko]

If you read the wiki page on compression, you will see that you can mark individual files or entire disks to be compressed. Comperssion does not make disk I/O faster, it makes everything slower since you need "overhead" to calculate things. However, it fits the data into smaller size. There are even userspace programs that can calculate the size of a specific file before and after compression in btrfs:

https://btrfs.wiki.kernel.org/index.php/Compression

[bester69]

I finally did it, by booting in a 4.9 kernel, even so, when backup reached 85% it started to rise much cpu, then i couldnt boot in 4.4.39 (many services failed in boot) and could do it in 4.9, i think olders kernels might give some issues with btrfs. within 4.9 went back to a older snapshot, cleaned some snapshots, and was able to bootback to my default 4.39 kernel. I just hope the fsarchiver backup works if i need eventually, it ended without errors ..

Thanks, again Pytkio.

I tried again yesterday with last kernel version 4.4.x, and It happend the same, When It reaches 65% or so, Its starts to runs out of space and pagging as crazy, I thing its something to do with the free litle space (around just 7GB) , BTRFS very used (a lot of snashots created and deleted since its creation), and perhaps the version of the kernel. Then you cant reboot (many service doesnt rise) at least you delete last btrfs snapshot.

So, my advice is to use fsarchives from a LiveCD session, that way wont be troubles.
My whole file system compressed in two file parts sized around 4,8GB (excluding home), and now I will uploaded to the cloud with duplicty.

[pylkko]

On btrfs you can however always, at any time, make a "dead" snapshot of a live system and while "remaining live", backup that dead snapshot somewhere else (using a million different ways, dd, btrfs send, tar, rsync, cp etc.). I have done it without issues. The snapshot is usable later, but there can some times be minor issues if, for example, you have other partitions that are not rolled back in sync. Many programs for example store configurations and other stuff in user's folders, if you roll back a root subvolume without rolling back a home that goes with it but is on separate subvolume or partition, you can end up with situations where you need to reset user profiles or other such things. But often even this does not happen. You may also need to adjust fstab so that the root partition is mounted correctly, or many services may fail on boot. But even if you forget to do this, you will be able to boot into single user mode and adjust

[bester69]

On btrfs you can however always, at any time, make a "dead" snapshot of a live system and while "remaining live", backup that dead snapshot somewhere else (using a million different ways, dd, btrfs send, tar, rsync, cp etc.). I have done it without issues. The snapshot is usable later, but there can some times be minor issues if, for example, you have other partitions that are not rolled back in sync. Many programs for example store configurations and other stuff in user's folders, if you roll back a root subvolume without rolling back a home that goes with it but is on separate subvolume or partition, you can end up with situations where you need to reset user profiles or other such things. But often even this does not happen. You may also need to adjust fstab so that the root partition is mounted correctly, or many services may fail on boot. But even if you forget to do this, you will be able to boot into single user mode and adjust

yeah, Ive already seen theses issues, they are not big thing, the system can fix them by itself with some little help. im uploading now the 5Gb fsarchiver backup to the cloud (Box), by using webdav and duplicity, It works great the combo duplicity+box (webdav), and box gives a very proffesional service offering 10GB for free. Box, prevent data losing or corruption in the cloud. I cant say the same about dropbox, when i tried to download some chunked big file, it was corrupted, or missed some chunk, and couldnt recover the upload; with box, Ive allways been able to restore my backup with Box, I also use duplicity and box to backup my home setting and main data folder, I use previouslly attic, and then i use duplicity to be able to upload the attic's repository with success. I hightly reccommend it to you for your knowledge.