Btrfs Compression To Enhance Performance?

Kernels & Hardware, configuring network, installing services

Btrfs Compression To Enhance Performance?

Postby bester69 » 2017-12-06 06:27

https://www.phoronix.com/scan.php?page= ... 2635&num=1
https://www.phoronix.com/scan.php?page= ... 2638&num=1

So, Using btrfs with compression will always enhance performance regarding using not that compression?

My system is 1corex2Ghz,
should i enable btrfs compression?;
What about installing kernel > 4.14 and using Zstd to get the maximun speed with btrfs?
User avatar
bester69
 
Posts: 987
Joined: 2015-04-02 13:15

Re: Btrfs Compression To Enhance Performance?

Postby Head_on_a_Stick » 2017-12-06 06:35

bester69 wrote:should i enable btrfs compression?

Run an objective benchmark on your system now and then appy the compression and run the benchmark again.

Asking here is just silly.

What about installing kernel > 4.14 and using Zstd to get the maximun speed with btrfs?

I refer the right honourable gentleman to the answer I gave a moment ago. :mrgreen:
"Sheer [program] size is often an illusion, reflecting only a need for improvement." — Kernighan and Plauger
User avatar
Head_on_a_Stick
 
Posts: 6815
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Btrfs Compression To Enhance Performance?

Postby pylkko » 2017-12-06 07:42

It is important to notice that there are several different kinds of 'performance'. For example, if instead of just reading and writing files, now your CPU is also performing compression algorithms on them, then obviously there will be a 'overhead' associated with that. There will therfore be less CPU cycles for other things.

But for most computers the situation probably is that the CPUs are underused, are idle most of the time and users rarely do multiple tasks simultaneously. Since moving data from disk is often the slowest link in the chain when you are doing that one task on the computer, sacrificing that CPU time might be a good thing. Why it works is in the time your disk would need to write down the extra blocks in the noncompressed version (as opposed to the compressed situation), even a slow CPU can do many many calculations (compression related calculations).

Also, there is the question of throughput vs latency. Often when benchmarking disks or setups for real life use random read/write at 4k size is used, because this is more like what a operating system will be doing in normal use (than writing large single files). Disk latency has a significant impact on this. So it is possible to have a disk that can write large files sequentially at amazing speeds but still be worse at 4k random read write than a slower disk.

Like some one comments in the link you posted, 'depends on which kind of work load is more important for you'

Then there is also this to consider: on a battery powered device, compressing everything all the time might mean more energy use and shorter battery time.

But all these things are non-tested theoretical speculation and the only way to get reliable data would be to do real experiments with real devices.

From what I read, the Zstd is supposed to be really good. Burbot is also a new less tested feature...
User avatar
pylkko
 
Posts: 1213
Joined: 2014-11-06 19:02

Re: Btrfs Compression To Enhance Performance?

Postby bester69 » 2017-12-06 08:47

pylkko wrote:.... Since moving data from disk is often the slowest link in the chain when you are doing that one task on the computer, sacrificing that CPU time might be a good thing. ...
...

Like some one comments in the link you posted, 'depends on which kind of work load is more important for you'

....

From what I read, the Zstd is supposed to be really good. Burbot is also a new less tested feature...

I had already forgotten about IO being the headneck time operations, so now i understand why enhance performance; CPU is waiting in inactivity for disk to respond, so the overhead load its very small in comparison with the inactivity wait; so the compression its suppose to win here to the overheads.
To me I would be matter working with loads type playing hd movies, my system its already very short when playing 1080p.

By the way, Ive just lost my btrfs partition by testing zstd and then uninstalling kernel 4.14 without before defragmenting to lzo or no compression, you can imagine :mrgreen: Now, Ive to install a new debian in other partition with kernel 4.14 in order to be able to recover the lost partition. :cry:

The recovering, would be something like this:
With a kernel 4.14:
Code: Select all
mount -t btrfs -o subvolid=0 /dev/sda2 /mnt
btrfs filesystem defragment -r -v -cno /mnt
Last edited by bester69 on 2017-12-06 08:59, edited 1 time in total.
User avatar
bester69
 
Posts: 987
Joined: 2015-04-02 13:15

Re: Btrfs Compression To Enhance Performance?

Postby bester69 » 2017-12-06 08:50

Head_on_a_Stick wrote:
bester69 wrote:should i enable btrfs compression?

Run an objective benchmark on your system now and then appy the compression and run the benchmark again.

Asking here is just silly.

What about installing kernel > 4.14 and using Zstd to get the maximun speed with btrfs?

I refer the right honourable gentleman to the answer I gave a moment ago. :mrgreen:

This sound some difficult to me, I would have to study benchmark tools. I think there is no need at the moment..

thanks
User avatar
bester69
 
Posts: 987
Joined: 2015-04-02 13:15

Re: Btrfs Compression To Enhance Performance?

Postby pylkko » 2017-12-06 09:03

Unfortunately, for HD video this will probably not help that much because I believe most video storage formats already use compression.

So you created or forced a compression on the latest kernel on a btrfs subvolume and then tried to use it with old kernel tools? I think I even told you that you might have problems if you mix older kernels with newer btrfs user space tools in a previous thread :shock:

Well, did you manage to repair it?
User avatar
pylkko
 
Posts: 1213
Joined: 2014-11-06 19:02

Re: Btrfs Compression To Enhance Performance?

Postby bester69 » 2017-12-06 09:04

pylkko wrote:Also, there is the question of throughput vs latency. Often when benchmarking disks or setups for real life use random read/write at 4k size is used, because this is more like what a operating system will be doing in normal use (than writing large single files). Disk latency has a significant impact on this. So it is possible to have a disk that can write large files sequentially at amazing speeds but still be worse at 4k random read write than a slower disk...

From what I read, the Zstd is supposed to be really good. Burbot is also a new less tested feature...


To me, this sound as if it was a good idea to use compression with a lowlatency kernel to compensate ..
User avatar
bester69
 
Posts: 987
Joined: 2015-04-02 13:15

Re: Btrfs Compression To Enhance Performance?

Postby bester69 » 2017-12-06 09:08

pylkko wrote:Unfortunately, for HD video this will probably not help that much because I believe most video storage formats already use compression.

So you created or forced a compression on the latest kernel on a btrfs subvolume and then tried to use it with old kernel tools? I think I even told you that you might have problems if you mix older kernels with newer btrfs user space tools in a previous thread :shock:

Well, did you manage to repair it?

im on it now, i think its cos kernels under 4.14 cant mount dirty disks with zstd, so you can imagine..Ive to install a new debian and then install kernel 4.14 in order to be able to defrag with no compression so can be accesible again in the grub for the olders kernels.
User avatar
bester69
 
Posts: 987
Joined: 2015-04-02 13:15

Re: Btrfs Compression To Enhance Performance?

Postby bester69 » 2017-12-06 14:56

pylkko wrote:Unfortunately, for HD video this will probably not help that much because I believe most video storage formats already use compression.

So you created or forced a compression on the latest kernel on a btrfs subvolume and then tried to use it with old kernel tools? I think I even told you that you might have problems if you mix older kernels with newer btrfs user space tools in a previous thread :shock:

Well, did you manage to repair it?


I think you warned me, :? . It seems newers kernel alter user space in a way older ones cant work again.. :shock:

I gave into this trouble:
dmesg output
[ 119.698406] BTRFS info (device sdc2): disk space caching is enabled
[ 119.698409] BTRFS: couldn't mount because of unsupported optional features (10).
[ 119.744887] BTRFS: open_ctree failed

https://unix.stackexchange.com/question ... ree-failed
Now, I can only boot the installation with kernel 4.14,..

It seems not possible to go back with a lower kernel if not regenerating the filesystem, so there seems no solution but recreating user-space with a copy of the data.

I guess I could make a btrfs send /receive to replicate installation in a new clean metadata-btrfs partition without dragging issues. Im not sure this method will create a new clean repository.., what do you think about send/receive to regenerate the system?.

In the last case, I could use fsarchiver..
User avatar
bester69
 
Posts: 987
Joined: 2015-04-02 13:15

Re: Btrfs Compression To Enhance Performance?

Postby Head_on_a_Stick » 2017-12-06 18:18

bester69 wrote:This sound some difficult to me, I would have to study benchmark tools.

Code: Select all
empty@Xanadu:~ $ bonnie++                                                                                 
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.97       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
Xanadu       15808M   788  99 300466  11 166272  12  6000  98 687480  27  7348  89
Latency             18933us   22783us     184ms    2656us     221ms     448ms
Version  1.97       ------Sequential Create------ --------Random Create--------
Xanadu              -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency               217us     370us     232us     219us      19us    1884us
1.97,1.97,Xanadu,1,1512580698,15808M,,788,99,300466,11,166272,12,6000,98,687480,27,7348,89,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,18933us,22783us,184ms,2656us,221ms,448ms,217us,370us,232us,219us,19us,1884us
empty@Xanadu:~ $

https://packages.debian.org/stretch/bonnie++

Lather, rinse and repeat ;)

Isn't learning fun? :D
"Sheer [program] size is often an illusion, reflecting only a need for improvement." — Kernighan and Plauger
User avatar
Head_on_a_Stick
 
Posts: 6815
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Btrfs Compression To Enhance Performance?

Postby wizard10000 » 2017-12-06 18:36

bester69 wrote:To me, this sound as if it was a good idea to use compression with a lowlatency kernel to compensate ..


This is a terrible idea, bester. In addition to introducing Even More Instability into your system it has the added benefit of Not Helping You At All :mrgreen:

When troubleshooting performance issues the component that's running at 100% capacity is the bottleneck - it's a rule. In this case the bottleneck is your hard drive, not the processor or the kernel.

A LL kernel will prioritize I/O operations but it won't make your hard drive spin any faster. Think about it :)
we see things not as they are, but as we are.
-- anais nin
User avatar
wizard10000
 
Posts: 1224
Joined: 2011-05-09 20:02
Location: everywhere i go, there i am!

Re: Btrfs Compression To Enhance Performance?

Postby bester69 » 2017-12-07 12:20

I finally removed btrfs compression, Thought some operations were quicker, I was feeling some tinny lag in desktop experience,
I also lost the whole system by using 4.14 with compression, fortunately I have a fsarchiver full system backup 5Gb in cloud and could restore the installation.
User avatar
bester69
 
Posts: 987
Joined: 2015-04-02 13:15


Return to System configuration

Who is online

Users browsing this forum: No registered users and 10 guests

fashionable