a form of JBOD/SPAN? that is not a RAID

[bobstone]

hello all

I am looking to setup a file server using linux and wanted to ask a few questions ( i use the term jbod/span in the title because even though what I actually want is not a jbod/span,it will look like a jbod/span to the networked computers.)

now what I wanted to do was have multiple drives of different sizes all accessed as individual drives, each one being separate and can be removed and placed in another computer and I can access the data on it with no problems (simple and straight forward so far, next is the part I need help understanding).

however I want my networked computers/media centers to see it as one large drive, and for the server to either divide the data equally across all the drives, or to fill up the largest drive first then move on to the next drive.

does that make any sense?

Thanks for any help
Bob

[bobstone]

so I let it be for a few days,but... did i ask something stupid

also I was looking around and found some info on ZFS, if I am reading this correctly it says it MIGHT do what I am looking to do.

any ideas ?

[smallchange]

The part about "a drive can be removed from one computer and placed in another computer and the data on it accessed" is a little tough, and can only be done partially, as far as I know. If you mean to actually change them after you have started using them in one computer you might be able to do that if you use a RAID that allows discs to fail. If you want to install the discs in separate computers and use them as components of RAID0, but not move them after initializing the RAID that can be done. Let's see if I can make this understandable. There are 2 protocols that would be of use to you, iSCSI and ATAoE. They both allow discs in one computer to appear as discs on another computer. The iSCSI uses SCSI protocol over IP and is quite common but has the overhead of IP packets. ATA over Ethernet does not use IP but operates at the ethernet level. It is much more efficient. Both are supported in linux.

So you can have disc1 in computer1, disc2 in computer2, and disc3 in computer3 and make them all appear to be discs on computer1. You can then create a RAID from them on computer1 which can be formatted as you like and shared via NFS or SMB or anything else you want. If you use RAID0 you will get all the discs looking like one disc to the client computers, but if any of the discs fail, all of your data is gone. If you use RAID5 in this example your available space will be smaller but if one disc fails the others continue to run and your data is safe. So if you want to move a disc, or replace it, you could fail it in the RAID, replace the disc or move it to another system, export it through ATAoE, and add it back in to the array.

There are also things like the Global File System that will allow numerous of these targets to be mounted directly on multiple computers. I don't know ZFS but it might be something like this.

Probably not exactly what you asked for but it is close. Personally I am amazed that this type of thing is possible and reasonably efficient.

[bobstone]

wow that is some crazy stuff, I am with you on the amazed part.

however I probably should of used a different title. there will be NO raid of any kind.

my goal to reexplain it, is 3 fold. each step by itself shouldn't be to complicated, it is all of them at the same time which is difficult.

1. -----------------------------------------

have computer "A" running a form of linux and also have lets say 4, 1.5TB drives in it for data storage(referred to as data drives). and one 60gb ssd for all the os stuff.

computer "A" will have all thoes drives be normal independent drives. if at any time I wish to turn of the computer and remove one of the aforementioned "data drives" I can do it with out affecting anything (except the data stored on the removed drive will not be there cause the drive is not plugged in to computer "A"). and also not having to prep anything (I.E. with out doing anything with the computer, turn it of and remove a "data drive").

I can then take the "data drive" I removed and go the JoeBlow's house and plug it in to his computer and he can see all the files on it.


2. -----------------------------------------

on my network, which has Computer "A" ( from part one ) and Computer "B". I can go to computer "B" and access a "single" share from computer "A" that for all intents and purposes looks like and acts like one 6TB hdd.

then I can copy anything I want to it or copy anything off it to the local computers Hdd.


3. --------------------------------------------

Computer "A" will either automatically fill 1 "data drive" at a time, or it will equally distribute the files between the different "data drives" automatically and behind the scene.

if I remove one of the "data drives"; Computer "B" will only see 4.5TB of space over the network, and still see whatever files are remaining that was not on the removed "data drive"

if instead of removing, I add another 1.5TB "data drive", Computer "B" will now see 7.5TB of space thru the network.


hehe I am not the best at explaining complicated stuff. I hope this makes sense.

Thanks
Bob

[smallchange]

I think I understand what you want and I do not think it is possible. I would not take that as authoritative but I would be very surprised if that can be done. The closest I can think of would be that each disc is mounted inside a directory which is shared to the other computers. Say storage is shared and storage/diskA, storage/diskb ... To some extent it would look like a single file system but what goes into each drive would be only on that drive. If you do come up with anything I would sure like to hear about it.