[Software] enterprise backup solutions???

[kent_dorfman766]

I want to create indexed and searchable offline SATA disk archives of all my data accumulated over 20-30 years of work. I have neither the time nor motivation to reinvent the wheel so I'm asking what comprehensive backup packages work well in Debian.

Requirements:
* must maintain a lite database describing media and its contents down to the file level, with possible annotations for backup sets and individual files.
* must allow backup to hot-pluggable SATA disks, as "disk sets" meaning that a backup may span multiple media volumes, and allow arbitrary and logical points where media is switched
* supports globs and meta-instruction files to ignore or schedule directory contents for backup (.ignore file, etc)
* save directories with huge number of files on media in TAR format, but still properly index the files
* datasets of 10-20TB.
* MUST HAVE GOOD DOCUMENTATION AND SUPPORT!!!
* ability to read and extract data from media directly, without going thru the backup system UI (requires media to be standard filesystem formatted, not raw block device)


Nice to have:
* selectable PAR2 recovery data on "safe media" to restore volumes that have degraded
* comprehensive CLI interface with GUI alternatives


Are there any pre-existing (active) packages that support all/most of the above? and no, I'm not interested in off-site or cloud storage.

[Uptorn]

Backup programs have been reinvented a hundred times over. But if you have highly specific requirements, it might make more sense to write your own solution.

If I had to pick one that appears to match your criteria, it might be dar:

Full featured archiver with support for differential backups, slices,
compression, ATTR/ACL support. DAR also supports Pipes for remote
operations, including with ssh.

[kent_dorfman766]

Backup programs have been reinvented a hundred times over.

Exactly...so writing my own is the least preferred path.

[Uptorn]

I wouldn't want to end up in a situation where I had a highly particular backup solution, maintained by only one or a few people which can disappear one day either to disinterest or waning maintainership.

So it made sense (for me at least) to roll my own, using large and established programs that are not likely to fall out of maintenance any time soon.

I hope that you can find something that is established and which also does all the things you want it to do.