I am trying to do full volume backups on a device with 1 harddisk and a slow connection to a Samba server that happens to be an old network harddisk.
The Samba server has a write speed over my network link of 3 MB/s but over FTP I do 6MB/s.
Now it is possible to use tar + split + a script for split, that will allow you to push the chunks to whatever server you like.
I didn't mention yet that the filesystem of the device is fat32, but this is quite irrelevant now because having chunks has benefits anyway.
The only problem I have currently is that the ftp client I use (lftp) site, sometimes uses SEEKS on the source file which is a PIPE in my case.
Oops!!
The only way I can deal with the pipe issue is by using a queue composed of 2 chunks, with one being uploaded, and the next being fed in by tar.
However the device is very slow and doing "scratch" writing on the same disk is going to lower the tar performance excessively.
So the only solution I know is to use an external USB device like a stick but the problem of course is that those do not deal well with heavy writing.
I am talking about a 780GB backup currently.
A 780GB backup would mean about 390 writes of 2GB to a 16GB USB stick.
So you say why not use a disk for it? Well I only need 4GB really and I would have to hang a 2.5" external HDD from something that is basically also a network disk.
The other solution is to use an ftp client that can be used to upload files from a pipe.
- Do you have any suggestion for this scratch space queue system?
- Do you know of an ftp client that could be used to reliably receive data from a pipe for uploading?