BTRFS on Debian Stable, your experiences?

[nigratruo]

Hi,

I'm using EXT4 everywhere. I use Debian Stretch everywhere and was wondering if BTRFS is already up to snuff, stable and reliable.
I'm asking you this from a Debian Stable perspective, where we wants things to be super stable and don't tolerate buggy beta software. How has BTRFS been working out for you on Debian Stable? Issues? How does it compare to EXT4?

[Head_on_a_Stick]

was wondering if BTRFS is already up to snuff, stable and reliable

https://btrfs.wiki.kernel.org/index.php/Status

How has BTRFS been working out for you on Debian Stable?

I've been using it on the family laptop since jessie was frozen and it's been great, no problems at all.

The laptop has a dead battery and the filesystem has been subjected to about a million[1] hard resets with no data loss at all.

I do have two full backups of the system though, just in case, and those backups are to an xfs filesystem.

I also use btrfs on my own laptop, mainly because the subvolumes allow me to share a single partition between several distributions — at one point I had Arch, Alpine, Debian, Devuan and MX Linux all on the same partition. I don't backup my laptop 'cos I like to live on the edge but I've never lost anything non-deliberately.

The only issue I've encountered has been trying to use an older kernel with a filesystem created under a newer kernel, the system then can't boot because the old kernel can't use some of the new features.

[1] Exaggerated for comic effect.

[p.H]

The only issue I've encountered has been trying to use an older kernel with a filesystem created under a newer kernel, the system then can't boot because the old kernel can't use some of the new features.

The same has happened with ext4 too.

[pylkko]

I hope that the content on this official page gives you some perspective of the "state" of btrfs

https://btrfs.wiki.kernel.org/index.php/Status

I have also been using it for years now.

but the versions of btrfs-tools and the kernel code is in better shape in testing, which is not ready yet.

[nigratruo]

Thank you for your feedback, that is useful!

I'm just messing around with the many layers I need to get LUKS and lvm working, so I got three layers including the fs, which in BTRFS is just one layer, which makes things a lot easier.
Any experience with the encryption layer? Does that work well? Secure?

[milomak]

The only issue I've encountered has been trying to use an older kernel with a filesystem created under a newer kernel, the system then can't boot because the old kernel can't use some of the new features.

The same has happened with ext4 too.

what kind of use case lead to such a situation for you? or what you experienced.

[bester69]

Hi,

I'm using EXT4 everywhere. I use Debian Stretch everywhere and was wondering if BTRFS is already up to snuff, stable and reliable.
I'm asking you this from a Debian Stable perspective, where we wants things to be super stable and don't tolerate buggy beta software. How has BTRFS been working out for you on Debian Stable? Issues? How does it compare to EXT4?

BTRFS is god in my own experience, Ive been using it for four years, and never gave me any issue, and Im creating and removing snapshot everyweeks. btrfs filesyste, was the best thing happend to me with linux, since then Ive never have to reinstall linux again, I always rollback system/home snapshots.

here, I have a small scripts framework I made to do the task easy with the snapshots; perhaps It can help you.:
http://forums.debian.net/viewtopic.php?f=3&t=135738

[barlafuss]

following, very interesting post. Thanks to All

[p.H]

The only issue I've encountered has been trying to use an older kernel with a filesystem created under a newer kernel, the system then can't boot because the old kernel can't use some of the new features.

The same has happened with ext4 too.

what kind of use case lead to such a situation for you? or what you experienced.

I have a multiboot setup with multiple releases of Debian. At that time, my main system was Wheezy and it could not mount an ext4 filesystem created by Stretch with default options. IIRC, Jessie could mount it but its e2fsck could not check it.

[delina]

Hard drive End to End failure This happened on a btrfs drive when i was deleting lots of files and the trash can got full. It has me worried now that this may happen to another drive.

I am wondering if there is a way to set trash bins like a virtual shared drive across every volume on the system so if a drive gets near full it moves trash to another drive.

I use mergerfs to join volumes into larger drives since btrfs does not have a save way to keep directorys on each drive. i wish btrfs could incorporate mergerfs so instead of single non raid it could have a merger nonraid option. then recycle bins could work across all the drive.

[p.H]

btrfs does not have a save way to keep directorys on each drive

What do you mean ?

[Mr. Lumbergh]

I don't suppose there's a way to do an in-place conversion of ext4 to BTRFS is there?

[pylkko]

on the first page of the wiki there is a tutorial called
"Conversion from Ext3/4 and ReiserFS"

but a word of warning, this converter used to be horribly broken. i don't have time to figure out if this is still the case. you can read about it yourself. but in any case, if you use it, use the latest version
also, you may want to change the layout of your disk entirely anyway...

edit: apparently the converter is stable now:
from wikipedia:
As of June 2015 and 4.x versions of the Linux kernel mainline, the in-place ext3/4 conversion was considered untested and rarely used.[65] The feature, however, was rewritten from scratch in 2016 for btrfs-progs 4.6.[42] and is considered stable since then.

[Bloom]

Don't do an in-place conversion without having a backup first. But if you backup the lot first, you can simply format the volume in the new format and restore the lot. I suspect it will be quicker as well.

[Mr. Lumbergh]

on the first page of the wiki there is a tutorial called
"Conversion from Ext3/4 and ReiserFS"

but a word of warning, this converter used to be horribly broken. i don't have time to figure out if this is still the case. you can read about it yourself. but in any case, if you use it, use the latest version
also, you may want to change the layout of your disk entirely anyway...

It looks like ext4 performs better in most cases in benchmarks: https://www.phoronix.com/scan.php?page= ... tems&num=4. I often need to record audio in real time, so don't want the performance hit; I'll just make sure to keep regular backups.
Speaking of, though, those are probably a pretty good candidate for BTRFS because of the snapshot feature.
Hmm...

[LE_746F6D617A7A69]

It looks like ext4 performs better in most cases in benchmarks: https://www.phoronix.com/scan.php?page= ... tems&num=4. I often need to record audio in real time, so don't want the performance hit; I'll just make sure to keep regular backups.
Speaking of, though, those are probably a pretty good candidate for BTRFS because of the snapshot feature.
Hmm...

It's extremely unlikely that BTRFS will affect audio recording, unless You're writing to a cheap SD card

But anyway, BTRFS is indeed rather slow - and those test on Phoronix are not showing what happens when You take few volume snapshots - the CoW is causing write amplification, which is just a performance killer, especially on cheap SSDs (TLC/QLC).

I agree that BTRFS is probably the best choice for servers, using server-grade RAID arrays as BTRFS volumes, but for desktop it makes completely no sense, f.e.:
- BTRFS software RAID is at least 2 times slower than md-raid in case of sequential access, random access is just *deadly* slow.
- Checksums: That would deserve a separate thread, but it's a complete stupidity to use FS checksums for a PC equipped with a non-ECC RAM -> this *feature* only eats the resources, and the *improved safety* is nothing but a placebo.
- scrubbing: just slows down everything even more, and it's far more expensive than in case of md-arrays.
- defragmentation? -> Ext4 does not need it, and SSDs just hate it
- An obvious problem with snapshots: You have to use at least software RAID-1 or make regular backups of the entire drive -> otherwise, the drive failure will kill all the snapshots made so far.

That's obviously just My point of view, and I'm sure that many users are very happy with BTRFS

[pylkko]

Yes, and maybe another opinion would be something like this:

First of all, any performance issue is likely irrelevant, because you can setup the machine so that it does not matter. For example, those distributions that install in btrfs by default (Fedora, OpenSUSE), use btrfs subvolumes with different settings for different parts (logs, system, data whatever) of the disk. OpenSUSE has the home partition on XFS because apparently it has better performance than EXT4 on multimedia / large files.

But anyway, BTRFS is indeed rather slow - and those test on Phoronix are not showing what happens when You take few volume snapshots - the CoW is causing write amplification, which is just a performance killer, especially on cheap SSDs (TLC/QLC).

But exactly where do you think you would notice this in practice? Why would I care if the disk performs X percent slower when it is idle nearly 95% of the time anyway?

I agree that BTRFS is probably the best choice for servers, using server-grade RAID arrays as BTRFS volumes,

if you have hardware raid, why would you use software?

- BTRFS software RAID is at least 2 times slower than md-raid in case of sequential access, random access is just *deadly* slow.

with "deadly" slow you mean technically slow in such a way that you would not ever notice it, if you wanted to use software raid in btrfs, when most desktop users do not use RAID at all.

- Checksums: That would deserve a separate thread, but it's a complete stupidity to use FS checksums for a PC equipped with a non-ECC RAM -> this *feature* only eats the resources, and the *improved safety* is nothing but a placebo.

With complete stupidity you mean that people can notice if their data is corrupted, and in case they are using the RAID even recover it.

- scrubbing: just slows down everything even more, and it's far more expensive than in case of md-arrays.

Just checks your drive is intact while it is idle, and you can always just not do it.

- defragmentation? -> Ext4 does not need it, and SSDs just hate it

although, SSDs for which it would matter have not been made for a decade almost

- An obvious problem with snapshots: You have to use at least software RAID-1 or make regular backups of the entire drive -> otherwise, the drive failure will kill all the snapshots made so far.

Who ever agrees with this would probably also agree with the statement:" an obvious problem of forks is that they are really bad for eating soup". Yes, so what? Nobody has ever claimed that snapshot are replacements for backups and if someone ever does, you should maybe consider taking stuff that person says with a grain of salt.

BTRFS has limitations and differences from other data management ways. It cannot replace backups, RAIDs and other things. But it also does have properties that are unique to it, and it can be used together with other filesystems or raid setups or backup systems. Other filesystems (XFS, F2FS) are now also implementing btrfs-like features. It will probably require that Canonical or other player creates an intuitive integrated GUI system to manage the snapshots like on MacOS before it becomes widely used, however.

[LE_746F6D617A7A69]

Yes, and maybe another opinion would be something like this:

First of all, any performance issue is likely irrelevant, because you can setup the machine so that it does not matter. For example, those distributions that install in btrfs by default (Fedora, OpenSUSE), use btrfs subvolumes with different settings for different parts (logs, system, data whatever) of the disk. OpenSUSE has the home partition on XFS because apparently it has better performance than EXT4 on multimedia / large files.

But anyway, BTRFS is indeed rather slow - and those test on Phoronix are not showing what happens when You take few volume snapshots - the CoW is causing write amplification, which is just a performance killer, especially on cheap SSDs (TLC/QLC).

But exactly where do you think you would notice this in practice? Why would I care if the disk performs X percent slower when it is idle nearly 95% of the time anyway?

Of course it depends on the use case - for Me the show-stopper in BTRFS is a poor performance of databases and build systems -> so I have switched back to md-arrays + ext4.
I agree that for typical desktop use case, it doesn't matter what FS is used and what unique functionalities it offers.

But I think that one aspect needs clarification: BTRFS checksums.
A standard HDD has a theoretical probability of undetected read error of 1 bit in 10^15 bits, what means 1 bit in 938.83TiB - and for non-ECC RAM error rate is undefined. This means, that 2 situations are relatively likely to happen:
1. The checksum can be calculated for damaged data in RAM, just before sending them to a storage device -> the number of copies doesn't matter -> You have damaged data in all the copies, and it's impossible to detect that fact on the FS level.
2. The checksum calculated for correct data loaded from HDD may falsely report inconsistency because of non-ECC RAM error -> consequences are depending on the FS/OS configuration.

The non-ECC RAM is simply far less reliable than a classic HDD in terms of detecting so called "rot bits" -> so it makes completely no sense to use BTRFS checksums on desktop/laptop.

[Mr. Lumbergh]

It looks like ext4 performs better in most cases in benchmarks: https://www.phoronix.com/scan.php?page= ... tems&num=4. I often need to record audio in real time, so don't want the performance hit; I'll just make sure to keep regular backups.
Speaking of, though, those are probably a pretty good candidate for BTRFS because of the snapshot feature.
Hmm...

It's extremely unlikely that BTRFS will affect audio recording, unless You're writing to a cheap SD card

It isn't just the filesystem itself, though, it's the other things that have to slow down to accommodate it. Reading/writing from storage is generally the slowest step in the chain, why knowingly add latency? That's just my thought anyway; a filesystem is just a tool, and as with any tool there are use cases where some are better-suited than others. Having snapshots built in really does make BTRFS attractive for backups though.

[Head_on_a_Stick]

Btrfs is fantastic for the development branches because snapshots. They're not backups but they are a quick and easy way to restore the system from a b0rked update.

See also https://packages.debian.org/buster/snapper & https://packages.debian.org/buster/snapper-gui