btrfs + mergerfs or btrfs single.

[delina]

I have been attempting to make a large single drive using either btrfs in single mode or a mix or btrfs and ext4 with mergerfs.


so far the mix of btrfs and mergerfs seems to fail as the merged drive seems to keep unmounting after a hour or more.

I build a box to test the single with 8 drives but had a power supply failure before i could see how that works.

does anyone have any ideas which works.

Im thinking of scraping this idea and going back to just ext4. I was hoping to use the compresson and dedupe features of the two.

[pylkko]

For myself, I can say that I do not understand at all what you are trying to do, why and what your question even is to be exact.

Do you want to combine several physical drives to a storage pool? I hope you realize that this is a feature in btrfs itself and you don't need mergerfs for that purpose. Or did I misunderstand what you want?

If your mergerfs gets unmounted, what message does it report in the logs?

[delina]

i have lots of drives in a pool with mergerfs now I have created 1 drive as a btrfs and added it to the pool but btrfs seem strange it does not behave like a regular drive. it seems to be there and disappear some times like it goes offline when system is doing things.

--mostly im not trying to create a high performance or a raid or such. I keep a lot of not important files for a period of time and need lots of space. I wanted to use btrfs if i can get it all to work for the compression and the dedupe features.

I made a box with 8 2 tb drives into a 16 tb as single then I removed 1 drive and the whole array goes offline but there is no warning messages when this happens. I suspect this would take some custom coding to get the messages.

I have been trying to find some site were more people have created media pool for home use like this. Raids and such just wear the drives out so fast I rather have them setting idle and not doing much. I am kind of worried about using btrfs to create large single drives after reading that if one dies you will loose all the drives as it does not seem to keep the directory on each drive or metadata as they call it. I think they really need to rethink this and make the single drive array so it can withstand any drive being removed and every drive survive. that is what single means to me.

I think i may just have to make each drive btrfs and just never combine them into an array. Its just to expensive to add enough drives to make it into a raid and drives rarely last 5 years any more i have a pile of dead 2 and 3 tb drives.


I cannot find much info on btrfs and there does not seem to be a real support forum for it. From what i can tell this btrfs is really different than a partition. The os seems to need a gui console for it that gives a visual feedback of what it is doing when it just goes away for long periods of time. Im not very smart at linux but dont care for most linux builds and have sort of cobbled together my own version of debian for a server from a minimal network install.
I have looked at lots of custom linus appliances to build nas boxes but they all seem to want to force you to use raid of some sort if you want to use the utilities that comes with them. So i have been just gathering bits and pieces and creating my own that allows all the drives to be independent. I have run my system like this for 20 years and it works ok I keep it all on a ups power backup and in a low vibration area of the house. all my drives are 3 4 5 6 8 12 tb on the main box. i'm trying to replace all the smaller drives as they are getting older. I'm thinking of creating large single drive arrays from the smaller drives i remove and use them to backup the remaining larger drives now and then. I am creating 5 drive groups of esata drives so i can move them to what ever machine i need. I hoped to use btrfs as single on them but worry about all data being lost if 1 drive fails. there is the possibility of going raid 1 and giving up 1 drive of the 5 for better data also. I am wondering if i create these drives can i migrate them to another machine with no problems also.

I really like the idea behind btrfs and would like to get a gui for it but I do not see one out there. It looks like it should be possible to create.

One of my biggest gripes is the btrfs commands are just a bit confusing. like if i type btrfs status i expect it to reply with something by default. it seems the developers like to make you type 3 4 or more words to get any response from it.

[delina]

What a pain i got it to work finally i created 8 btrfs drives and gave each a label like drive1 drive2.... drive 8

I was trying to figure how to do this all in the gui so i used the disk utility to create the mount for the drives
it only kind of works with btrfs. to make it work i had to trick it.

i had to click change mount options then change to by label drive1
i would copy the path /mount/drive1
then change it to UUID=...... then in this setting i would replace the uuid path with the label path.
on the line that says nosuid,nodev,nofail add a ,compress and change filesystem time from auto to btrfs.

when done it makes a fstab with
UUID=xxxxxxxxxxxxxxxxxxxxxxxxxx /mnt/drive1 btrfs nosuid,nodev,nofail,compress 0 0

make sure to create folders in /mnt for drive1 drive2.. 8

then it will mount all those in those folders.

then you just make the mergerfs mount
/mnt/drive* /mnt/driveshare fuse.mergerfs minfreespace=20g,defaults,allow_other,direct_io,use_ino 0 0


i used the disk utility and set all the drive to mount on start and not be visible.
you shouold end up with a /mnt/driveshare that has the sum of all 8 drives in it.

[delina]

2020 and now im wondering how to make one big drive with btrfs out of 8 drives and then be able to split them up easy later ?

[Bloom]

Try LizardFS (www.lizardfs.org).

[Head_on_a_Stick]

now im wondering how to make one big drive with btrfs out of 8 drives and then be able to split them up easy later

Did you even bother to ask the interweb before posting here?

https://btrfs.wiki.kernel.org/index.php ... le_Devices

[delina]

Btrfs + mergerfs + compression
I have been running this now for a while and only had problem on a old drive. I am running 2 5 port esata multiplexers with 10 drives formated btrfs. i use the mergerfs to join them as It works well to keep the directorys intact on each drive. none of the btrfs join options do that.
with each drive having its only directory you need to make folders on every drive like so

drive1 drive2 drive3 drive4 ...
mediashare mediashare mediashare mediashare
movie tv movie tv tv aname tv aname

this gives you a folder merged like this /mnt/mediashare movie tv aname

with multiple esata controlers you can split the data half on each to spread the heat around in you drive cases this way.

[latingo]

@delina
Your setup is pretty much what I am looking for and I have 2 questions on it.
1. Can you still use the btrfs snapshot functionality on the single disks?
2. Do you use smartd to monitor your disks?

Background:
I want to build a home server for my media data without wasting too much disk space for redundancy.
Redundancy is not the same as backup and as I plan to do daily backups to a second pool of disks, I accept the risk of a disk failure.
I plan to use smart monitoring for predictive action.
Even if a disk eventually fails without premonition I will not lose more than a day of changes on my rather static archive.
I want to use the possibility to pool disks with different sizes and mergerfs is very suitable for this.
In the tutorials which I found, mergerfs is often used together with snapraid but this is very slow.
Btrfs on the other hand has copy on write and a built-in snapshot functionality like NTFS or ZFS.

I know that btrfs can also pool disks but from what I read, in "single" jbod mode the whole pool will be corrupted if one disk fails.
With mergerfs only the data of the failed disk needs to be recovered.
This can make a big difference in recovery time.
Also, mergerfs can spin down inactive disks and thus will possibly extend the lifespan of my disks.

[delina]

I have experimented with this for like 15 years. I used to use ext4 and mhdffs but it got to buggy to use at debian 8 so i changed to mergerfs. I was storing lots of little text files and wanted some compression and btrfs file system looked good for this. After reading a bit at debian 9 I converted from ext4 to btrfs with mergerfs.
During this time I had 3 western digital 3 tb drives show up as failing on smart and copied them off to new drives and accidently wiped 1 - 3 tb drive of not important stuff. I later had 4 drive show up at getting to hot but kept running fine. debian 10 came along and more headaches with os changes and a discovery of a problem using mergerfs to join drives and the ever growing recycle bin.
I one of the drives i deleted so many files that it caused the recycle bin to hit the size of the drive and for some reason caused a hardware error on the drive called "end to end failure" smart recommended replacing it and with that done i emptied all the recycle bins.
Now I have upgraded to debian 11 Bullseye its in alpha or beta and seems to work fairly good. Im considering dropping the mergerfs and going all btrfs with raid 6 but am worried because i use esata multiplexers to make 5 drives on 1 controller. I have 10 drives in my box its more for archival storage and speed is not important. for me heat is the problem and going raid means 5 or 10 drives constantly running and making heat. I like to get some reliability that raid 6 gives but not raise the heat in my room to much and prevent the recycle bin problem.
If you know linux a bit debian is the best linux for setting up a single user nas for home media use as its easiest to install all the tools needed into it. getting around wayland is the biggest hurdle you need learn to use su then sudo and install xfe fileexplorer so you can open admin terminal and file manages. then its easy.