BTRFS on Debian Stable, your experiences?

[LE_746F6D617A7A69]

We know that a "next gen" CoW filesystem will always get a performance hit (we pay for the new features in performance), that in itself is not interesting.

That's good - for Me a "feature" which kills the performance is well... not a feature, but rather a misconception in the design process.
And apparently I'm not alone in my judgement - search for "btrfs disable CoW" - You'll find many useful articles on how "fantastic" the BTRFS is in terms of performance with CoW enabled.
example:
https://unix.stackexchange.com/question ... emu-images

Some guy in the Phoronix comments points out how checksumming helped him notice a data corruption in his music collection, which he recovered immediately. He adds that on Ext4 it would have just lain there hidden for years until he would have noticed. That's a real thing

That's bullshit. The chances that he would get a CRC error instead of read error are practically zero - and obviously, it's impossible to investigate and confirm what have happened for "some guy" on "some forum" ...
Numbers do not lie - people do, sometimes unintentionally...

[pylkko]

We know that a "next gen" CoW filesystem will always get a performance hit (we pay for the new features in performance), that in itself is not interesting.

That's good - for Me a "feature" which kills the performance is well... not a feature, but rather a misconception in the design process.

Ok, but then in essence you are saying that btfs is a horrible, unusable technology because it cannot do something that is impossible. It has a "misconception in the design process" because it does something that no file system can ever do (CoW without a performance hit... even though CoW can be disabled if needed). And the evidence you are offering is that you say that you have an undisclosable "rare use case" where there is difference in practice ("it kills the performance"). I'm fine with that argument.

That's bullshit. The chances that he would get a CRC error instead of read error are practically zero - and obviously, it's impossible to investigate and confirm what have happened for "some guy" on "some forum" ...
Numbers do not lie - people do, sometimes unintentionally...

Numbers don't lie... so when you say "practically zero chance" you really expect anyone to believe that? Zero is a number, gee. It just doesn't work like that. Just offer the hard evidence for file system level checksumming not being possible or let it go. You made the claim, you need to prove it, nobody else. Thus far you offer nothing. And you will not, right?

Also, the article you offer is about using CoW on VM images. That is a special case that is common knowledge, even mentioned in the btrfs wiki. Keep them on another partition or switch of CoW since it is not needed. Not an argument against btrfs CoW performance in general, despite you claiming so.

[LE_746F6D617A7A69]

Ok, but then in essence you are saying that btfs is a horrible, unusable technology because it cannot do something that is impossible. It has a "misconception in the design process" because it does something that no file system can ever do. And the evidence you are offering is that you say that you have a "rare use case" where there is difference in practice. I'm fine with that argument.

In essence, CoW has to be disabled for most (if not all) write-intensive tasks, to avoid high performance drop and fragmentation.
This means data bases, virtual machines, temp dirs and logs, just to name few. In the effect, the snapshots also can't be used for all such cases, because this means CoW.
For read-mostly use cases both CoW and snapshots are OK.
I'm not saying that BTRFS is horrible - I'd rather say that it's not as fantastic as some people think

Numbers don't lie... so when you say "practically zero chance" you really expect anyone to believe that? Zero is a number, gee. It just doesn't work like that. Just offer the hard evidence for file system level checksumming not being possible or let it go. You made the claim, you need to prove it, nobody else. Thus far you offer nothing.

The probability of 1 bit in 10^15 bits comes from official datasheets for HDDs, and on the wiki pages You'll find an explanation of why ECC RAM has been invented - I'm not going to copy/paste those information here.

[Deb-fan]

One obvious consideration with Debian stable + btrfs.. Along these lines.

https://www.phoronix.com/scan.php?page= ... provements

A 4.19 kernel isn't ancient by any stretch and btrfs should have decent support with it regardless but has support vs improved support in newer kernel versions, yanno?

???

[Mr. Lumbergh]

It isn't just the filesystem itself, though, it's the other things that have to slow down to accommodate it. Reading/writing from storage is generally the slowest step in the chain, why knowingly add latency?

Sorry, but apparently I don't get what do You mean - what latency?
The writes are buffered, and even very old HDD will easily swallow ~40MB/s on average. Recording 8 audio streams sampled @96kHz/32bit gives 2.93MB/s ...

It isn't just the actual write itself that takes additional time, there are other resources such as CPU cycles and memory that are needed to execute the CoW commands, etc. that can slow things down in other places besides the drive. CPU ticks are the main currency when exporting a large mutlitrack file from a DAW, why add to overhead?

[LE_746F6D617A7A69]

It isn't just the actual write itself that takes additional time, there are other resources such as CPU cycles and memory that are needed to execute the CoW commands, etc. that can slow things down in other places besides the drive.

No. Simply speaking, no matter how many audio streams You have to record in real-time, the thruoutput needed is just a small fraction of what a modern HDD/SSD can handle - unless we are talking about hundreds of audio streams, but I doubt it ...

CPU ticks are the main currency when exporting a large mutlitrack file from a DAW, why add to overhead?

For modern CPUs, the performance limit is the RAM bandwidth - which is around 25...45GiB/s for DDR3/4 - so a few MB/s for audio streams is not even noticeable.

[Mr. Lumbergh]

It isn't just the actual write itself that takes additional time, there are other resources such as CPU cycles and memory that are needed to execute the CoW commands, etc. that can slow things down in other places besides the drive.

No. Simply speaking, no matter how many audio streams You have to record in real-time, the thruoutput needed is just a small fraction of what a modern HDD/SSD can handle - unless we are talking about hundreds of audio streams, but I doubt it ...

CPU ticks are the main currency when exporting a large mutlitrack file from a DAW, why add to overhead?

For modern CPUs, the performance limit is the RAM bandwidth - which is around 25...45GiB/s for DDR3/4 - so a few MB/s for audio streams is not even noticeable.

Ask a drag racer why they don't run AC when they're at the track making a run. It's only 10 or so out of how many hundreds of horsepower, shouldn't be noticeable right? But it's that much less getting to wheels. In the same vein, why knowingly force CPU time to the copy on write operation when I'm more concerned about making sure the track export is going as smoothly as it can? There's a reason why system tuning is a big topic over on Linuxmusicians, every little bit adds up. This or that in and of itself may not make a noticeable difference, but on the aggregate they do.
I'm not going to argue the point further, it's clear that no minds are going to be changed here, so why?

[LE_746F6D617A7A69]

Ask a drag racer why they don't run AC when they're at the track making a run. It's only 10 or so out of how many hundreds of horsepower, shouldn't be noticeable right? But it's that much less getting to wheels. In the same vein, why knowingly force CPU time to the copy on write operation when I'm more concerned about making sure the track export is going as smoothly as it can? There's a reason why system tuning is a big topic over on Linuxmusicians, every little bit adds up. This or that in and of itself may not make a noticeable difference, but on the aggregate they do.
I'm not going to argue the point further, it's clear that no minds are going to be changed here, so why?

Are we talking about a real numbers or about feelings?
I don't think that anyone here is interested in "feelings" - those are meaningless and subjective - and IMO all the "musicians" or "audiophiles" are far from being objective in their judgements - especially, that most of them don't even have a basic idea about how the Hi-Fi amplifier works, not to mention the computers ...

No offense, but Your claim that BTRFS (or any other FS) can cause problems with audio recording simply has nothing to do with the reality (speaking gently ...)

[Mr. Lumbergh]

Ask a drag racer why they don't run AC when they're at the track making a run. It's only 10 or so out of how many hundreds of horsepower, shouldn't be noticeable right? But it's that much less getting to wheels. In the same vein, why knowingly force CPU time to the copy on write operation when I'm more concerned about making sure the track export is going as smoothly as it can? There's a reason why system tuning is a big topic over on Linuxmusicians, every little bit adds up. This or that in and of itself may not make a noticeable difference, but on the aggregate they do.
I'm not going to argue the point further, it's clear that no minds are going to be changed here, so why?

No offense, but Your claim that BTRFS (or any other FS) can cause problems with audio recording simply has nothing to do with the reality (speaking gently ...)

Read what I wrote. I never did make that claim, just I'd rather not add the overhead if I don't have to.

[LE_746F6D617A7A69]

Read what I wrote. I never did make that claim, just I'd rather not add the overhead if I don't have to.

So what is Your claim? - I just can't wait to see the final version ...
Can You define/measure that mysterious "overhead"?
What is the exact problem that You have with the BTRFS? (or any other FS?) , especially with a reference to a Debian stable? Please share Your experience, but please not to forget to include a real-time measurements of the dalays ... if You have any ...

[Mr. Lumbergh]

Read what I wrote. I never did make that claim, just I'd rather not add the overhead if I don't have to.

So what is Your claim? - I just can't wait to see the final version ...
Can You define/measure that mysterious "overhead"?
What is the exact problem that You have with the BTRFS? (or any other FS?) , especially with a reference to a Debian stable? Please share Your experience, but please not to forget to include a real-time measurements of the dalays ... if You have any ...

I already told you, I looked at the differences and decided for myself that making the change wasn't worth the trouble for my particular use case. I'm not sure why you're taking that so personally.
If you're still curious about the slower write times (what I referred to as overhead), I'll refer you back to the benchmark.
And no, I don't have anything against the filesytem. It's just a tool, and one I already noted would facilitate backups quite well.

[pylkko]

Might be of interest to some here:

Btrfs Seeing Some Nice Performance Improvements For Linux 5.9
https://www.phoronix.com/scan.php?page= ... -Linux-5.9

[Deb-fan]

^ Thanks, whatever this thread is, gotta admit its entertaining.

Yeah newer Kernel's clearly improved support in terms of btrfs or whatever else. I wuv Debian gnu/linux of course that includes stable too but on desktop majority of time am not running the stock kernel shipped with. Even on my beloved, crusty 10-11yr old laptop(w Buster n Stretch), thing has 5.4.x kernel set default. Would pretty much have to do the same (use newer kerns-etc)with Debian stable, even in a production environment.

Why kernel v. 5.4.x? For the hell of it of course, wanted to compile the thing and see how it'd run on this ancient hardware, runs fine and also wanted to see if i could get dkms to build the wl.ko module for the systems wireless, also success. Though point im trying to make here is btrfs performance on stock Debian stable vs appropriately modified Debian stable are surely vastly different things. Not that it/btrfs wouldnt be good enough on stock Debian. Blahblah n blah.

[bester69]

Im using a 2008's CPU one simple core, a very old lowspec system with btrfs from years.. and the dropdown performance by having deployed several snapshots is very litle.... I think the filesystem performance the same with just one snapshot than having thousands snapshots... I didnt see any downfall by having more than one snapshot in same subvolume.,,

If Im not wrong Btrfs is core ext4 filesystem..

If Me with a lowspec desktop can use properly everyday a btrfs debian.. (I suppose my 2008 internal hdd disk must very slow in comparison with nowadays disk).. dont see what might be the problem for a very fast nowadays hdd internal disk and btrfs filesystem.

Btrfs in my opinion is best thing ever done for linux... since I installed (since debian Jessie) Ive not had to reinstall system neither got any headache, data loss scares or any bad day...

I alway do the same.. I install/uninstall a lot of things test them.. and when I need to consolide my installation, I rollback to my stables tested snapshot point, update packages and install new tested apps, to create a new snapshot consilidated point.. so this way I keep my system clean and very tested and stable.. just in case I always keep a previous snapshot point in case eventually I realised some thing I installed , made something didint work like before.

As I said, BTRFS, BEST THING EVER DONE for LINUX!!!, god bless whoever develop this filesystem


by the way, I ve made thousands of snapshots/rollbacks reboots.. and never got an issue with filesystem snapshots..it never got corrupted in restoring snapshots process, very easy to use and work with snapshots... Btrfs-Cleaner is quite fast, ultra stable very trustworthy even to work with partitions managers..

[Head_on_a_Stick]

Fedora have just announced they're moving to Btrfs as the default filesystem for v33:

https://fedoramagazine.org/btrfs-coming-to-fedora-33/

[bester69]

Fedora have just announced they're moving to Btrfs as the default filesystem for v33:

https://fedoramagazine.org/btrfs-coming-to-fedora-33/

Great News to know BTRFS goes on fordward... there was a past time there were some doubts about btrfs future consolidation

[LE_746F6D617A7A69]

I'd like to see an unbiased comparison of BTRFS RAID 1/10 vs mdadm RAID 1/10 - I've made the tests for myself, but nobody would believe me - too many fanboys ...

[Mr. Lumbergh]

I'd like to see an unbiased comparison of BTRFS RAID 1/10 vs mdadm RAID 1/10 - I've made the tests for myself, but nobody would believe me - too many fanboys ...

You noticed that too, eh?

[Deb-fan]

How does btrfs do on things like SSD's with compression enabled ? Is there any notable speed gains during transfer, disk i/o operations ? Btrfs is certainly on the things I want to mess with list though. Don't always go with the version chasers in everything tech but yeppers, would be wanting latest-greatest in something like the btrfs filesystem.

[pylkko]

I've made the tests for myself, but nobody would believe me - too many fanboys ...

But if you produced such test results, what is the reason that you cannot publish them? Or have you? Because you cannot really expect people to believe otherwise. Or at least, it would draw much more interest towards the issue.

How does btrfs do on things like SSD's with compression enabled ? Is there any notable speed gains during transfer, disk i/o operations ? Btrfs is certainly on the things I want to mess with list though. Don't always go with the version chasers in everything tech but yeppers, would be wanting latest-greatest in something like the btrfs filesystem.

What do you mean exactly? Are you asking if the performance is better or worse than something else X? What do you want to compare to will decide if there is an already-existing benchmark out there on it. It is probably not possible to give a complete answer as it will depend on the hardware in question. When I install on btrfs I use compression and do it on an SSD. On the fly compression of course consumes compute time on the processor, but it will save space, and as discussed earlier, can increase throughput in cases where the compute operation is quicker than the transfer and what you are moving around compresses well.