Splitting Tar Archive and updating it

[larry77]

Dear All,
Probably a linux more than a specific debian question.
Let us say that you have a large folder (including subfolders). You want to create a tar backup split into chunks of a certain size.
You can do this with a one liner such as

tar cvf - /home/user1/dir1 | split -b 1000M - /home/user1/archive.tar

Now, suppose that dir1 changes its content over time. I would like to rerun the same command in such a way that I backup the latest version of dir1 without starting from scratch (do not overwrite existing files, but remove files that do not exist any longer etc...).
I know it is a funny way of making a backup, but there are constraints.
Any idea about how to achieve that?
Cheers

[Head_on_a_Stick]

Looks like the --update option may do what you want, check the man page for details (or ask the interweb).

[reinob]

Dear All,
Probably a linux more than a specific debian question.
Let us say that you have a large folder (including subfolders). You want to create a tar backup split into chunks of a certain size.
You can do this with a one liner such as

tar cvf - /home/user1/dir1 | split -b 1000M - /home/user1/archive.tar

Now, suppose that dir1 changes its content over time. I would like to rerun the same command in such a way that I backup the latest version of dir1 without starting from scratch (do not overwrite existing files, but remove files that do not exist any longer etc...).
I know it is a funny way of making a backup, but there are constraints.
Any idea about how to achieve that?
Cheers

You have some weird requirements. Why tar? why split? why "do not overwrite existing files" (what if a file has been modified, you really want to keep the old one?)

If you drop the tar requirement, you can use rsync, which is perfectly suited for what you want (although rsync will replace existing files if they have been modified, which is normally the sensible thing to do).

If you insist on using tar, then you're on your own but: tar(1) won't delete files from an archive, (2) will not overwrite any files at all, (3) will "update" (as suggested in the previous post) by *appending* the new version to the end of the tar archive -- you have to decide if this is what you want.

Good luck,
(and try to focus on what you really want, not on how to force/tweak an arbitrary tool to do what you want).

[larry77]

I need to archive a large folder (hundreds of GB) and send it online. On the other side, there is someone who will reconstruct the folder with all the chunks.
Yes. I know that rsync makes more sense, but these are (not my own!) requirements.

[reinob]

I need to archive a large folder (hundreds of GB) and send it online. On the other side, there is someone who will reconstruct the folder with all the chunks.
Yes. I know that rsync makes more sense, but these are (not my own!) requirements.

OK :)

One possibility would be to use archivemount to mount the (previous, local) tar archive on a temporary directory. Then rsync as appropriate. Then umount the temporary directory. The end result would be an updated tar archive, which you then split and send over to the destination.

Given that you anyway have to send the whole (splitted) tar archive to the destination (or did I misunderstand that part?) it seems much simpler to recreate the whole tar archive again whenever you want to send it over. It might even be faster than updating the innards of a tar archive (and archivemount might be even slower than unpacking and then re-packing again..)

Good luck.

[larry77]

I need to archive a large folder (hundreds of GB) and send it online. On the other side, there is someone who will reconstruct the folder with all the chunks.
Yes. I know that rsync makes more sense, but these are (not my own!) requirements.

OK

One possibility would be to use archivemount to mount the (previous, local) tar archive on a temporary directory. Then rsync as appropriate. Then umount the temporary directory. The end result would be an updated tar archive, which you then split and send over to the destination.

Given that you anyway have to send the whole (splitted) tar archive to the destination (or did I misunderstand that part?) it seems much simpler to recreate the whole tar archive again whenever you want to send it over. It might even be faster than updating the innards of a tar archive (and archivemount might be even slower than unpacking and then re-packing again..)

Good luck.

Finally, I can only agree with you.
Thanks!