BTRFS on Debian Stable, your experiences?

If none of the more specific forums is the right place to ask

Re: BTRFS on Debian Stable, your experiences?

Postby LE_746F6D617A7A69 » 2020-08-03 09:22

pylkko wrote:We know that a "next gen" CoW filesystem will always get a performance hit (we pay for the new features in performance), that in itself is not interesting.
That's good - for Me a "feature" which kills the performance is well... not a feature, but rather a misconception in the design process.
And apparently I'm not alone in my judgement - search for "btrfs disable CoW" - You'll find many useful articles on how "fantastic" the BTRFS is in terms of performance with CoW enabled.
example:
https://unix.stackexchange.com/questions/394973/why-would-i-want-to-disable-copy-on-write-while-creating-qemu-images

pylkko wrote:Some guy in the Phoronix comments points out how checksumming helped him notice a data corruption in his music collection, which he recovered immediately. He adds that on Ext4 it would have just lain there hidden for years until he would have noticed. That's a real thing
That's bullshit. The chances that he would get a CRC error instead of read error are practically zero - and obviously, it's impossible to investigate and confirm what have happened for "some guy" on "some forum" ...
Numbers do not lie - people do, sometimes unintentionally...
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed
LE_746F6D617A7A69
 
Posts: 393
Joined: 2020-05-03 14:16

Re: BTRFS on Debian Stable, your experiences?

Postby pylkko » 2020-08-03 10:00

LE_746F6D617A7A69 wrote:
pylkko wrote:We know that a "next gen" CoW filesystem will always get a performance hit (we pay for the new features in performance), that in itself is not interesting.
That's good - for Me a "feature" which kills the performance is well... not a feature, but rather a misconception in the design process.
Ok, but then in essence you are saying that btfs is a horrible, unusable technology because it cannot do something that is impossible. It has a "misconception in the design process" because it does something that no file system can ever do (CoW without a performance hit... even though CoW can be disabled if needed). And the evidence you are offering is that you say that you have an undisclosable "rare use case" where there is difference in practice ("it kills the performance"). I'm fine with that argument.
pylkko wrote:That's bullshit. The chances that he would get a CRC error instead of read error are practically zero - and obviously, it's impossible to investigate and confirm what have happened for "some guy" on "some forum" ...
Numbers do not lie - people do, sometimes unintentionally...
Numbers don't lie... so when you say "practically zero chance" you really expect anyone to believe that? Zero is a number, gee. :lol: It just doesn't work like that. Just offer the hard evidence for file system level checksumming not being possible or let it go. You made the claim, you need to prove it, nobody else. Thus far you offer nothing. And you will not, right?

Also, the article you offer is about using CoW on VM images. That is a special case that is common knowledge, even mentioned in the btrfs wiki. Keep them on another partition or switch of CoW since it is not needed. Not an argument against btrfs CoW performance in general, despite you claiming so.
User avatar
pylkko
 
Posts: 1791
Joined: 2014-11-06 19:02

Re: BTRFS on Debian Stable, your experiences?

Postby LE_746F6D617A7A69 » 2020-08-03 10:23

pylkko wrote:Ok, but then in essence you are saying that btfs is a horrible, unusable technology because it cannot do something that is impossible. It has a "misconception in the design process" because it does something that no file system can ever do. And the evidence you are offering is that you say that you have a "rare use case" where there is difference in practice. I'm fine with that argument.
In essence, CoW has to be disabled for most (if not all) write-intensive tasks, to avoid high performance drop and fragmentation.
This means data bases, virtual machines, temp dirs and logs, just to name few. In the effect, the snapshots also can't be used for all such cases, because this means CoW.
For read-mostly use cases both CoW and snapshots are OK.
I'm not saying that BTRFS is horrible - I'd rather say that it's not as fantastic as some people think ;)

pylkko wrote:Numbers don't lie... so when you say "practically zero chance" you really expect anyone to believe that? Zero is a number, gee. :lol: It just doesn't work like that. Just offer the hard evidence for file system level checksumming not being possible or let it go. You made the claim, you need to prove it, nobody else. Thus far you offer nothing.

The probability of 1 bit in 10^15 bits comes from official datasheets for HDDs, and on the wiki pages You'll find an explanation of why ECC RAM has been invented - I'm not going to copy/paste those information here.
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed
LE_746F6D617A7A69
 
Posts: 393
Joined: 2020-05-03 14:16

Re: BTRFS on Debian Stable, your experiences?

Postby Deb-fan » 2020-08-03 16:05

One obvious consideration with Debian stable + btrfs.. Along these lines.

https://www.phoronix.com/scan.php?page= ... provements

A 4.19 kernel isn't ancient by any stretch and btrfs should have decent support with it regardless but has support vs improved support in newer kernel versions, yanno? :)

???
Most powerful FREE tech-support tool on the planet * HERE. *
Deb-fan
 
Posts: 951
Joined: 2012-08-14 12:27

Re: BTRFS on Debian Stable, your experiences?

Postby Mr. Lumbergh » 2020-08-03 16:21

LE_746F6D617A7A69 wrote:
Mr. Lumbergh wrote:It isn't just the filesystem itself, though, it's the other things that have to slow down to accommodate it. Reading/writing from storage is generally the slowest step in the chain, why knowingly add latency?
Sorry, but apparently I don't get what do You mean - what latency?
The writes are buffered, and even very old HDD will easily swallow ~40MB/s on average. Recording 8 audio streams sampled @96kHz/32bit gives 2.93MB/s ...

It isn't just the actual write itself that takes additional time, there are other resources such as CPU cycles and memory that are needed to execute the CoW commands, etc. that can slow things down in other places besides the drive. CPU ticks are the main currency when exporting a large mutlitrack file from a DAW, why add to overhead?
Mr. Lumbergh
 
Posts: 97
Joined: 2019-08-02 04:28

Re: BTRFS on Debian Stable, your experiences?

Postby LE_746F6D617A7A69 » 2020-08-03 20:13

Mr. Lumbergh wrote:It isn't just the actual write itself that takes additional time, there are other resources such as CPU cycles and memory that are needed to execute the CoW commands, etc. that can slow things down in other places besides the drive.

No. Simply speaking, no matter how many audio streams You have to record in real-time, the thruoutput needed is just a small fraction of what a modern HDD/SSD can handle - unless we are talking about hundreds of audio streams, but I doubt it ...

Mr. Lumbergh wrote: CPU ticks are the main currency when exporting a large mutlitrack file from a DAW, why add to overhead?

For modern CPUs, the performance limit is the RAM bandwidth - which is around 25...45GiB/s for DDR3/4 - so a few MB/s for audio streams is not even noticeable.
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed
LE_746F6D617A7A69
 
Posts: 393
Joined: 2020-05-03 14:16

Re: BTRFS on Debian Stable, your experiences?

Postby Mr. Lumbergh » 2020-08-03 20:46

LE_746F6D617A7A69 wrote:
Mr. Lumbergh wrote:It isn't just the actual write itself that takes additional time, there are other resources such as CPU cycles and memory that are needed to execute the CoW commands, etc. that can slow things down in other places besides the drive.

No. Simply speaking, no matter how many audio streams You have to record in real-time, the thruoutput needed is just a small fraction of what a modern HDD/SSD can handle - unless we are talking about hundreds of audio streams, but I doubt it ...

Mr. Lumbergh wrote: CPU ticks are the main currency when exporting a large mutlitrack file from a DAW, why add to overhead?

For modern CPUs, the performance limit is the RAM bandwidth - which is around 25...45GiB/s for DDR3/4 - so a few MB/s for audio streams is not even noticeable.

Ask a drag racer why they don't run AC when they're at the track making a run. It's only 10 or so out of how many hundreds of horsepower, shouldn't be noticeable right? But it's that much less getting to wheels. In the same vein, why knowingly force CPU time to the copy on write operation when I'm more concerned about making sure the track export is going as smoothly as it can? There's a reason why system tuning is a big topic over on Linuxmusicians, every little bit adds up. This or that in and of itself may not make a noticeable difference, but on the aggregate they do.
I'm not going to argue the point further, it's clear that no minds are going to be changed here, so why?
Mr. Lumbergh
 
Posts: 97
Joined: 2019-08-02 04:28

Re: BTRFS on Debian Stable, your experiences?

Postby LE_746F6D617A7A69 » 2020-08-03 21:23

Mr. Lumbergh wrote:Ask a drag racer why they don't run AC when they're at the track making a run. It's only 10 or so out of how many hundreds of horsepower, shouldn't be noticeable right? But it's that much less getting to wheels. In the same vein, why knowingly force CPU time to the copy on write operation when I'm more concerned about making sure the track export is going as smoothly as it can? There's a reason why system tuning is a big topic over on Linuxmusicians, every little bit adds up. This or that in and of itself may not make a noticeable difference, but on the aggregate they do.
I'm not going to argue the point further, it's clear that no minds are going to be changed here, so why?

Are we talking about a real numbers or about feelings?
I don't think that anyone here is interested in "feelings" - those are meaningless and subjective - and IMO all the "musicians" or "audiophiles" are far from being objective in their judgements - especially, that most of them don't even have a basic idea about how the Hi-Fi amplifier works, not to mention the computers ...

No offense, but Your claim that BTRFS (or any other FS) can cause problems with audio recording simply has nothing to do with the reality (speaking gently ...)
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed
LE_746F6D617A7A69
 
Posts: 393
Joined: 2020-05-03 14:16

Re: BTRFS on Debian Stable, your experiences?

Postby Mr. Lumbergh » 2020-08-03 21:35

LE_746F6D617A7A69 wrote:
Mr. Lumbergh wrote:Ask a drag racer why they don't run AC when they're at the track making a run. It's only 10 or so out of how many hundreds of horsepower, shouldn't be noticeable right? But it's that much less getting to wheels. In the same vein, why knowingly force CPU time to the copy on write operation when I'm more concerned about making sure the track export is going as smoothly as it can? There's a reason why system tuning is a big topic over on Linuxmusicians, every little bit adds up. This or that in and of itself may not make a noticeable difference, but on the aggregate they do.
I'm not going to argue the point further, it's clear that no minds are going to be changed here, so why?



No offense, but Your claim that BTRFS (or any other FS) can cause problems with audio recording simply has nothing to do with the reality (speaking gently ...)

Read what I wrote. I never did make that claim, just I'd rather not add the overhead if I don't have to.
Mr. Lumbergh
 
Posts: 97
Joined: 2019-08-02 04:28

Re: BTRFS on Debian Stable, your experiences?

Postby LE_746F6D617A7A69 » 2020-08-03 22:12

Mr. Lumbergh wrote:Read what I wrote. I never did make that claim, just I'd rather not add the overhead if I don't have to.

So what is Your claim? - I just can't wait to see the final version ...
Can You define/measure that mysterious "overhead"?
What is the exact problem that You have with the BTRFS? (or any other FS?) , especially with a reference to a Debian stable? Please share Your experience, but please not to forget to include a real-time measurements of the dalays ... if You have any ...
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed
LE_746F6D617A7A69
 
Posts: 393
Joined: 2020-05-03 14:16

Re: BTRFS on Debian Stable, your experiences?

Postby Mr. Lumbergh » 2020-08-03 22:38

LE_746F6D617A7A69 wrote:
Mr. Lumbergh wrote:Read what I wrote. I never did make that claim, just I'd rather not add the overhead if I don't have to.

So what is Your claim? - I just can't wait to see the final version ...
Can You define/measure that mysterious "overhead"?
What is the exact problem that You have with the BTRFS? (or any other FS?) , especially with a reference to a Debian stable? Please share Your experience, but please not to forget to include a real-time measurements of the dalays ... if You have any ...

I already told you, I looked at the differences and decided for myself that making the change wasn't worth the trouble for my particular use case. I'm not sure why you're taking that so personally.
If you're still curious about the slower write times (what I referred to as overhead), I'll refer you back to the benchmark.
And no, I don't have anything against the filesytem. It's just a tool, and one I already noted would facilitate backups quite well.
Mr. Lumbergh
 
Posts: 97
Joined: 2019-08-02 04:28

migh be of interest...

Postby pylkko » 2020-08-04 08:56

Might be of interest to some here:

Btrfs Seeing Some Nice Performance Improvements For Linux 5.9
https://www.phoronix.com/scan.php?page= ... -Linux-5.9
User avatar
pylkko
 
Posts: 1791
Joined: 2014-11-06 19:02

Re: BTRFS on Debian Stable, your experiences?

Postby Deb-fan » 2020-08-04 09:11

^ Thanks, whatever this thread is, gotta admit its entertaining. :)

Yeah newer Kernel's clearly improved support in terms of btrfs or whatever else. I wuv Debian gnu/linux of course that includes stable too but on desktop majority of time am not running the stock kernel shipped with. Even on my beloved, crusty 10-11yr old laptop(w Buster n Stretch), thing has 5.4.x kernel set default. Would pretty much have to do the same (use newer kerns-etc)with Debian stable, even in a production environment.

Why kernel v. 5.4.x? For the hell of it of course, wanted to compile the thing and see how it'd run on this ancient hardware, runs fine and also wanted to see if i could get dkms to build the wl.ko module for the systems wireless, also success. Though point im trying to make here is btrfs performance on stock Debian stable vs appropriately modified Debian stable are surely vastly different things. Not that it/btrfs wouldnt be good enough on stock Debian. Blahblah n blah. :)
Most powerful FREE tech-support tool on the planet * HERE. *
Deb-fan
 
Posts: 951
Joined: 2012-08-14 12:27

Re: BTRFS on Debian Stable, your experiences?

Postby bester69 » 2020-08-11 06:39

Im using a 2008's CPU one simple core, a very old lowspec system with btrfs from years.. and the dropdown performance by having deployed several snapshots is very litle.... I think the filesystem performance the same with just one snapshot than having thousands snapshots... I didnt see any downfall by having more than one snapshot in same subvolume.,,

If Im not wrong Btrfs is core ext4 filesystem..

If Me with a lowspec desktop can use properly everyday a btrfs debian.. (I suppose my 2008 internal hdd disk must very slow in comparison with nowadays disk).. dont see what might be the problem for a very fast nowadays hdd internal disk and btrfs filesystem.

Btrfs in my opinion is best thing ever done for linux... since I installed (since debian Jessie) Ive not had to reinstall system neither got any headache, data loss scares or any bad day...

I alway do the same.. I install/uninstall a lot of things test them.. and when I need to consolide my installation, I rollback to my stables tested snapshot point, update packages and install new tested apps, to create a new snapshot consilidated point.. so this way I keep my system clean and very tested and stable.. just in case I always keep a previous snapshot point in case eventually I realised some thing I installed , made something didint work like before.

As I said, BTRFS, BEST THING EVER DONE for LINUX!!!, god bless whoever develop this filesystem :o


by the way, I ve made thousands of snapshots/rollbacks reboots.. and never got an issue with filesystem snapshots..it never got corrupted in restoring snapshots process, very easy to use and work with snapshots... Btrfs-Cleaner is quite fast, ultra stable very trustworthy even to work with partitions managers..
bester69 wrote:You wont change my mind when I know Im right, Im not an ...
User avatar
bester69
 
Posts: 1746
Joined: 2015-04-02 13:15

Re: BTRFS on Debian Stable, your experiences?

Postby Head_on_a_Stick » 2020-08-24 12:29

Fedora have just announced they're moving to Btrfs as the default filesystem for v33:

https://fedoramagazine.org/btrfs-coming-to-fedora-33/
Black Lives Matter

Debian buster-backports ISO image: for new hardware support
User avatar
Head_on_a_Stick
 
Posts: 12650
Joined: 2014-06-01 17:46
Location: /dev/chair

PreviousNext

Return to General Questions

Who is online

Users browsing this forum: No registered users and 13 guests

fashionable