Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

Share your HowTo, Documentation, Tips and Tricks. Not for support questions!.
Post Reply
Message
Author
User avatar
bester69
Posts: 2072
Joined: 2015-04-02 13:15
Has thanked: 24 times
Been thanked: 14 times

HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

#1 Post by bester69 »

https://www.fsarchiver.org/QuickStart
Limitations
There are several limitations anyway: it cannot preserve filesystem attributes that are very specific. For instance, if you create a snapshot in a btrfs volume (the new-generation file system for linux), FSArchiver won’t know anything about that, and it will just backup the contents seen when you mount the partition.
>> Fsacrhiver has the problem with btrfs filesystem that we can only backup/restore one subvolume for each partition, the rest of subvolumes are copied as regular directories..




So here comes another interasting way to achieve a filsesytem backup for each subvolume.. We will use pipes and btrfs send/receive functions.

**To prevent this error (ERROR: rename-> failed...) in restoring process, its necessary to use same version of btrf-progs and kernel in backup/restore process.
http://forums.debian.net/viewtopic.php? ... 56#p700465

0- Install PV for following progress backup and lzop for a very fast backup compression.
sudo apt install split pv lzop

1- Mount our system root subvolume as a only read snapshot.
sudo mount -t btrfs -o subvol=@root /dev/sda2 /mnt
sudo btrfs subvolume create /mnt /path/to/snapshot
sudo btrfs property set -ts /path/to/snapshot ro false (we might need to set ro flag)


2- Blackup filesytem to an another partition/device by using "btrfs send" + lzop compressor
sudo btrfs send /.snapshots/3/snapshot/ |pv -s 14G| split -b 1000M --filter 'lzop -1 > "$FILE.lzo"'
pv -s 14G : Show progress for a stimated subvolume size of 14Gb, we will query that information previously
split -b 1000M --filter 'lzop -1 > "$FILE.lzo"': Split it in 1Gb size chunks and then compress them with lzop to a fastest rate speed.

3- Restoring filesystem backup.:
3.1 We create an empty subvolume for restoring btrfs filesystem backup
sudo mount -t btrfs -o subvolid=0 /dev/sda2 /mnt
sudo mv /mnt/@root /mnt/@root.delete
sudo btrfs subvolume create /mnt/@root


3.2 We restore lzop chunk files, with "btrfs receive" in the new root subvolume created.
cat xa* | lzop -dN --ignore-warn | sudo btrfs receive -v --max-errors 0 /mnt/@root

4. Reboot and delete old subvolume system.
sudo mount -t btrfs -o subvolid=0 /dev/sda2 /mnt
sudo btrfs subvolume delete /mnt/@root.delete



And that sould make it, try first this method with test data subvolumes to verify everithing works well. :o
Last edited by bester69 on 2019-06-14 19:08, edited 7 times in total.
bester69 wrote:STOP 2030 globalists demons, keep the fight for humanity freedom against NWO...

User avatar
bester69
Posts: 2072
Joined: 2015-04-02 13:15
Has thanked: 24 times
Been thanked: 14 times

Re: Btrfs filesystem backup (Fsarchiver alternative)

#2 Post by bester69 »

TESTED WITH SUCCESS!

cat xa* | lzop -dN --ignore-warn|pv -s 14G| sudo btrfs receive -v --max-errors 0 /mnt/@root

The RESTORE/ BACKUP filesystem process was very fast, thanks to btrfs + lzop (around ten minits each one)
It stored a 14Gb root subvolume system backup in a compressed 7Gb splited files.:
Image
Last edited by bester69 on 2019-06-14 06:25, edited 1 time in total.
bester69 wrote:STOP 2030 globalists demons, keep the fight for humanity freedom against NWO...

User avatar
sickpig
Posts: 589
Joined: 2019-01-23 10:34

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

#3 Post by sickpig »

does this http://forums.debian.net/viewtopic.php? ... 65#p700465 mean the above is not tested with success?

Edit
just saw it is already linked to the other post
nice, thanks for the howto

User avatar
bester69
Posts: 2072
Joined: 2015-04-02 13:15
Has thanked: 24 times
Been thanked: 14 times

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

#4 Post by bester69 »

..
Last edited by bester69 on 2019-06-14 19:11, edited 6 times in total.
bester69 wrote:STOP 2030 globalists demons, keep the fight for humanity freedom against NWO...

User avatar
sickpig
Posts: 589
Joined: 2019-01-23 10:34

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

#5 Post by sickpig »

ohh okay, well hope u r able to fix it,
best luck
btrfs is too complex for me. I just use tar as it works without fail every time :)

User avatar
bester69
Posts: 2072
Joined: 2015-04-02 13:15
Has thanked: 24 times
Been thanked: 14 times

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

#6 Post by bester69 »

sickpig wrote:ohh okay, well hope u r able to fix it,
best luck
btrfs is too complex for me. I just use tar as it works without fail every time :)
FIXED!

As a workaround that seems to work without breaking system.:
Using (--max-errors 0) allow us to workaround this bug and restoring process without breaking the system:
btrfs receive -v --max-errors 0 /snap/restore

it just create some dozens of empty inode folders in root that you can delete without problem.
------------

**To prevent this error (ERROR: rename-> failed...) in restoring process, its necessary to use same version of btrf-progs and kernel in backup/restore process.
http://forums.debian.net/viewtopic.php? ... 56#p700465
bester69 wrote:STOP 2030 globalists demons, keep the fight for humanity freedom against NWO...

User avatar
sickpig
Posts: 589
Joined: 2019-01-23 10:34

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

#7 Post by sickpig »

There is nothing to install in linux, from time to time i go to google searching for something fresh to install in linux, but, there is nothing

User avatar
pylkko
Posts: 1802
Joined: 2014-11-06 19:02

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

#8 Post by pylkko »

sickpig wrote:ohh okay, well hope u r able to fix it,
best luck
btrfs is too complex for me. I just use tar as it works without fail every time :)
It is actually very simple and there are now ready made higher level tools like snapper and even graphical interfaces. The best thing about using btrfs send to make back-ups is that it can make incremental backups (so backup only any bit or letter that changed in a file, and not all the files entirely)

http://forums.debian.net/viewtopic.php?f=16&t=130512

User avatar
sickpig
Posts: 589
Joined: 2019-01-23 10:34

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

#9 Post by sickpig »

thanks pylkko, but cow implies higher io. tar can do incremental backups quite efficiently with lower resource usage, which of course can be scheduled.

for now tar beats all gui backup utilities like snapper(isn't that for suse)for me. but i appreciate your suggestion nevertheless. Thank you kindly.

User avatar
bester69
Posts: 2072
Joined: 2015-04-02 13:15
Has thanked: 24 times
Been thanked: 14 times

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

#10 Post by bester69 »

pylkko wrote:
sickpig wrote:ohh okay, well hope u r able to fix it,
best luck
btrfs is too complex for me. I just use tar as it works without fail every time :)
It is actually very simple and there are now ready made higher level tools like snapper and even graphical interfaces. The best thing about using btrfs send to make back-ups is that it can make incremental backups (so backup only any bit or letter that changed in a file, and not all the files entirely)

http://forums.debian.net/viewtopic.php?f=16&t=130512
Yeah, I'hadnt seen until now.. I wanted to be able to upload to cloud incremental backups of system, and I didnt know how do it as I was using fsarchiver for system backup... Now that I understand better btrfs-send/recieve Ive seen it works pretty well and Its very easy to use and apply incremental backups..

Now I will use: "btrfs send -p + rclone", to be able to upload system incremental backups to cloud (MEGA or BOX)..

Here an example of incremental system backup.:
sudo btrfs send -v /.snapshots/1/snapshot/ |pv -s 14G| split -b 1000M --filter 'lzop -1 > "/back/full/$FILE.lzo"'
sudo btrfs send -v -p /.snapshots/1/snapshot/ /.snapshots/2/snapshot/ |pv -s 1G| split -b 1000M --filter 'lzop -1 > "/back/incre1/$FILE.lzo"'

where snapshot/1 is whole system backup and snapshot/2 is incremental backup regarding to one.

To push to cloud, we will use rclone or better, duplicty-rclone (backend https://github.com/GilGalaad/duplicity-rclone). :D


Thanks pylkko :)
bester69 wrote:STOP 2030 globalists demons, keep the fight for humanity freedom against NWO...

User avatar
pylkko
Posts: 1802
Joined: 2014-11-06 19:02

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

#11 Post by pylkko »

sickpig wrote:thanks pylkko, but cow implies higher io. tar can do incremental backups quite efficiently with lower resource usage, which of course can be scheduled.

for now tar beats all gui backup utilities like snapper(isn't that for suse)for me. but i appreciate your suggestion nevertheless. Thank you kindly.
It is true that tar can do an incremental backup in a way. However, it is not nearly as efficient and nice. Because it works so that tar makes a record file of the state of the backup location upon the first backup. Later it can make an archive with only the changed files, whereas btrfs knows without making any records to send only the diff, not the entire files. This works independent of which path you back up. It will not send the changed files entirely! Similarly, if you change the path of the file, it will send no file in the incremental backup, only the meta data. Also, it is very convenient that the backup is a normal accessible subvolume and not an archive.

With respect to I/O, that mostly comes from other functions offered by btrfs, many of which are so nice (check summing against bit rot, for example, compression) that most people would gladly sacrifice the cost. In addition, it is possible to adjust these. For example, cow can be turned off on a per-file basis. It is hard to think of a real everyday use case where the performance hit would even be noticeable.

That said, I am not a btrfs fanboi. I think that each file system has it's specific advantages. For example, I use xfs extensivly due to performance it offers with larger files. I think using f2fs with flash media might be a sane idea, ext4 is very compatible between systems nowadays and so on.

User avatar
sickpig
Posts: 589
Joined: 2019-01-23 10:34

Re: HOWTO: Btrfs filesystem backup (Fsarchiver alternative)

#12 Post by sickpig »

quite informative, thanks for sharing pylkko. Appreciate experts sharing their hard earned knowledge with everyone.

Post Reply