HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Share your own howto's etc. Not for support questions!

HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Postby bester69 » 2019-06-13 01:20

https://www.fsarchiver.org/QuickStart
Limitations
There are several limitations anyway: it cannot preserve filesystem attributes that are very specific. For instance, if you create a snapshot in a btrfs volume (the new-generation file system for linux), FSArchiver won’t know anything about that, and it will just backup the contents seen when you mount the partition.

>> Fsacrhiver has the problem with btrfs filesystem that we can only backup/restore one subvolume for each partition, the rest of subvolumes are copied as regular directories..




So here comes another interasting way to achieve a filsesytem backup for each subvolume.. We will use pipes and btrfs send/receive functions.

**To prevent this error (ERROR: rename-> failed...) in restoring process, its necessary to use same version of btrf-progs and kernel in backup/restore process.
viewtopic.php?f=5&t=142356&p=700556#p700465

0- Install PV for following progress backup and lzop for a very fast backup compression.
sudo apt install split pv lzop

1- Mount our system root subvolume as a only read snapshot.
sudo mount -t btrfs -o subvol=@root /dev/sda2 /mnt
sudo btrfs subvolume create /mnt /path/to/snapshot
sudo btrfs property set -ts /path/to/snapshot ro false (we might need to set ro flag)


2- Blackup filesytem to an another partition/device by using "btrfs send" + lzop compressor
sudo btrfs send /.snapshots/3/snapshot/ |pv -s 14G| split -b 1000M --filter 'lzop -1 > "$FILE.lzo"'
pv -s 14G : Show progress for a stimated subvolume size of 14Gb, we will query that information previously
split -b 1000M --filter 'lzop -1 > "$FILE.lzo"': Split it in 1Gb size chunks and then compress them with lzop to a fastest rate speed.

3- Restoring filesystem backup.:
3.1 We create an empty subvolume for restoring btrfs filesystem backup
sudo mount -t btrfs -o subvolid=0 /dev/sda2 /mnt
sudo mv /mnt/@root /mnt/@root.delete
sudo btrfs subvolume create /mnt/@root


3.2 We restore lzop chunk files, with "btrfs receive" in the new root subvolume created.
cat xa* | lzop -dN --ignore-warn | sudo btrfs receive -v --max-errors 0 /mnt/@root

4. Reboot and delete old subvolume system.
sudo mount -t btrfs -o subvolid=0 /dev/sda2 /mnt
[b]sudo btrfs subvolume delete /mnt/@root.delete




And that sould make it, try first this method with test data subvolumes to verify everithing works well. :o
Last edited by bester69 on 2019-06-14 19:08, edited 7 times in total.
bester69 wrote:You wont change my mind when I know Im right, Im not an ...
User avatar
bester69
 
Posts: 1493
Joined: 2015-04-02 13:15

Re: Btrfs filesystem backup (Fsarchiver alternative)

Postby bester69 » 2019-06-13 03:06

TESTED WITH SUCCESS!

cat xa* | lzop -dN --ignore-warn|pv -s 14G| sudo btrfs receive -v --max-errors 0 /mnt/@root

The RESTORE/ BACKUP filesystem process was very fast, thanks to btrfs + lzop (around ten minits each one)
It stored a 14Gb root subvolume system backup in a compressed 7Gb splited files.:
Image
Last edited by bester69 on 2019-06-14 06:25, edited 1 time in total.
bester69 wrote:You wont change my mind when I know Im right, Im not an ...
User avatar
bester69
 
Posts: 1493
Joined: 2015-04-02 13:15

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Postby sickpig » 2019-06-14 03:52

does this viewtopic.php?f=5&p=700465#p700465 mean the above is not tested with success?

Edit
just saw it is already linked to the other post
nice, thanks for the howto
User avatar
sickpig
 
Posts: 317
Joined: 2019-01-23 10:34

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Postby bester69 » 2019-06-14 04:57

..
Last edited by bester69 on 2019-06-14 19:11, edited 6 times in total.
bester69 wrote:You wont change my mind when I know Im right, Im not an ...
User avatar
bester69
 
Posts: 1493
Joined: 2015-04-02 13:15

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Postby sickpig » 2019-06-14 05:12

ohh okay, well hope u r able to fix it,
best luck
btrfs is too complex for me. I just use tar as it works without fail every time :)
User avatar
sickpig
 
Posts: 317
Joined: 2019-01-23 10:34

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Postby bester69 » 2019-06-14 19:11

sickpig wrote:ohh okay, well hope u r able to fix it,
best luck
btrfs is too complex for me. I just use tar as it works without fail every time :)


FIXED!

As a workaround that seems to work without breaking system.:
Using (--max-errors 0) allow us to workaround this bug and restoring process without breaking the system:
btrfs receive -v --max-errors 0 /snap/restore

it just create some dozens of empty inode folders in root that you can delete without problem.
------------

**To prevent this error (ERROR: rename-> failed...) in restoring process, its necessary to use same version of btrf-progs and kernel in backup/restore process.
viewtopic.php?f=5&t=142356&p=700556#p700465
bester69 wrote:You wont change my mind when I know Im right, Im not an ...
User avatar
bester69
 
Posts: 1493
Joined: 2015-04-02 13:15

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Postby sickpig » 2019-06-15 02:52

There is nothing to install in linux, from time to time i go to google searching for something fresh to install in linux, but, there is nothing
User avatar
sickpig
 
Posts: 317
Joined: 2019-01-23 10:34

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Postby pylkko » 2019-06-15 12:36

sickpig wrote:ohh okay, well hope u r able to fix it,
best luck
btrfs is too complex for me. I just use tar as it works without fail every time :)


It is actually very simple and there are now ready made higher level tools like snapper and even graphical interfaces. The best thing about using btrfs send to make back-ups is that it can make incremental backups (so backup only any bit or letter that changed in a file, and not all the files entirely)

viewtopic.php?f=16&t=130512
User avatar
pylkko
 
Posts: 1585
Joined: 2014-11-06 19:02

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Postby sickpig » 2019-06-15 13:25

thanks pylkko, but cow implies higher io. tar can do incremental backups quite efficiently with lower resource usage, which of course can be scheduled.

for now tar beats all gui backup utilities like snapper(isn't that for suse)for me. but i appreciate your suggestion nevertheless. Thank you kindly.
User avatar
sickpig
 
Posts: 317
Joined: 2019-01-23 10:34

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Postby bester69 » 2019-06-15 14:23

pylkko wrote:
sickpig wrote:ohh okay, well hope u r able to fix it,
best luck
btrfs is too complex for me. I just use tar as it works without fail every time :)


It is actually very simple and there are now ready made higher level tools like snapper and even graphical interfaces. The best thing about using btrfs send to make back-ups is that it can make incremental backups (so backup only any bit or letter that changed in a file, and not all the files entirely)

viewtopic.php?f=16&t=130512

Yeah, I'hadnt seen until now.. I wanted to be able to upload to cloud incremental backups of system, and I didnt know how do it as I was using fsarchiver for system backup... Now that I understand better btrfs-send/recieve Ive seen it works pretty well and Its very easy to use and apply incremental backups..

Now I will use: "btrfs send -p + rclone", to be able to upload system incremental backups to cloud (MEGA or BOX)..

Here an example of incremental system backup.:
sudo btrfs send -v /.snapshots/1/snapshot/ |pv -s 14G| split -b 1000M --filter 'lzop -1 > "/back/full/$FILE.lzo"'
sudo btrfs send -v -p /.snapshots/1/snapshot/ /.snapshots/2/snapshot/ |pv -s 1G| split -b 1000M --filter 'lzop -1 > "/back/incre1/$FILE.lzo"'

where snapshot/1 is whole system backup and snapshot/2 is incremental backup regarding to one.

To push to cloud, we will use rclone or better, duplicty-rclone (backend https://github.com/GilGalaad/duplicity-rclone). :D


Thanks pylkko :)
bester69 wrote:You wont change my mind when I know Im right, Im not an ...
User avatar
bester69
 
Posts: 1493
Joined: 2015-04-02 13:15

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Postby pylkko » 2019-06-15 19:34

sickpig wrote:thanks pylkko, but cow implies higher io. tar can do incremental backups quite efficiently with lower resource usage, which of course can be scheduled.

for now tar beats all gui backup utilities like snapper(isn't that for suse)for me. but i appreciate your suggestion nevertheless. Thank you kindly.


It is true that tar can do an incremental backup in a way. However, it is not nearly as efficient and nice. Because it works so that tar makes a record file of the state of the backup location upon the first backup. Later it can make an archive with only the changed files, whereas btrfs knows without making any records to send only the diff, not the entire files. This works independent of which path you back up. It will not send the changed files entirely! Similarly, if you change the path of the file, it will send no file in the incremental backup, only the meta data. Also, it is very convenient that the backup is a normal accessible subvolume and not an archive.

With respect to I/O, that mostly comes from other functions offered by btrfs, many of which are so nice (check summing against bit rot, for example, compression) that most people would gladly sacrifice the cost. In addition, it is possible to adjust these. For example, cow can be turned off on a per-file basis. It is hard to think of a real everyday use case where the performance hit would even be noticeable.

That said, I am not a btrfs fanboi. I think that each file system has it's specific advantages. For example, I use xfs extensivly due to performance it offers with larger files. I think using f2fs with flash media might be a sane idea, ext4 is very compatible between systems nowadays and so on.
User avatar
pylkko
 
Posts: 1585
Joined: 2014-11-06 19:02

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Postby sickpig » 2019-06-15 21:23

quite informative, thanks for sharing pylkko. Appreciate experts sharing their hard earned knowledge with everyone.
User avatar
sickpig
 
Posts: 317
Joined: 2019-01-23 10:34


Return to Docs, Howtos, Tips & Tricks

Who is online

Users browsing this forum: No registered users and 3 guests

fashionable