[O/S] Cloning a startup disk to a bootable partition on another disk

[hopster]

Hello, and first of all may I say this is my first post here as a new Debian user after many years on and off learning about open source servers on FreeBSD and Ubuntu. This post may not be in the right place and if so then please point me there - I am still learning my way around this site.

I have a 250G SSD drive running Debian 12 as a headless server through ssh. It's mainly for code development and heavy/batch computational work, but also as a file server. I think the SSD may not be in the best shape so to be safe I would like to clone it to a 250G partition on the hard drive in the same box. Can I make a partition of the same size and literally clone it with all permissions, bootable status and everything else in place so that it can take over with no grief if the worst happens, and if so which tools to employ? I am used to rsync but realise there might be something more specific for the task(s).

I should say that last time I tried fdsik and cfdisk to have a look at the disks they both hung, which I wasn't expecting on Debian stable but perhaps that is actually because the SSD is not in great shape? I am not expert in tools like that having relied more on GUI disk partitioners in the past, and being spoilt on my Mac desktop by easy tools.

TIA

Michael

[mrmazda]

You could clone a disk to a partition, but AFAIK it would only be of use as an inaccessible backup, good only for the purpose of cloning back to a disk. IOW, answer is a qualified no.

I wouldn't try, because I don't care much for backups that are inaccessible. A better plan if you want the old SSD's content on the new disk to save as a backup, make sure the new disk size is sufficiently large enough to provide adequate space for the new system and its partitions, and enough space to hold an archive of the entire old disk. You could then put the archive on its own partition and filesystem on the new at its far end, which would then be available to unarchive onto the same disk at some future point in time to the start of the disk. I don't see much point. I'd just save the old SSD as backup, and if I was truly convinced of the old SSD's unreliable state, acquire another to clone to. A 250 isn't very expensive, and extra disks are always good to have in your backup media pool.

The better plan is probably just to clone the old / partition to the new disk, plus each other that the / depends on, or possibly rsync their combined content onto one extra filesystem. In an enforced headless situation though, I can't imagine how you'd be able to switch between two different installed systems.

[CwF]

You can use qemu-img from package qemu-utils.

If you copy the whole disk the boot ability is preserved.

From any temporary boot Debian system install the qemu-utils package, and image the entire disk of the unmounted, not running system to a raw, img, or compressed qcow2 file.
From that file, you can then image to a new drive. You can image from drive to drive where possible.

The new drive will boot all the same.

[hopster]

Thanks for both answers so far. Plenty of ideas there, but in my limited understanding not a solution yet in the way I need it. Perhaps I should describe the position of the server. It's in a cupboard under the stairs and so not exactly inaccessible, but not easy to get at and I certainly don't want to have to drag it out and attach a monitor every time I want to re-clone which is what I assume would be required if I needed to boot off a separate USB for instance.

So in 'pseudo-code' I would like to make a new bootable disk in a 250G partition on the spare disk (perhaps leaving it completely empty?) and then regularly and remotely running as root do something equivalent to:

Code: Select all

rsync -a /* /dev/sdb?/

so that every single file and permission on the current system is cloned onto the new disk partition. One nice thing about this is that after the initial big copy it would be incremental. Perhaps the qemu-utils mentioned could be part of this process, I will investigate. I suspect that cloning a running system may have some drawbacks or wrinkles but if they were simple to fix (perhaps a separate fixup script) I could live with that.

If the SSD ever fails I'm happy to drag it out from the cupboard, attach a keyboard and monitor and do whatever is needed on the new partition to make it boot from there instead.

[hopster]

It seems like some of these might be useful? (along with rsync and dd perhaps)

https://packages.debian.org/bookworm/fsarchiver

https://packages.debian.org/bookworm/timeshift

https://packages.debian.org/bookworm/rsnapshot

https://packages.debian.org/bookworm/backup-manager

https://packages.debian.org/bookworm/borgbackup

https://packages.debian.org/bookworm/duplicity

https://penguins-eggs.net/

It seems the key question might be:

• will any work well when cloning the system I am running on as that is what I wish to do

• which files and directories on the running system should I exclude e.g. with rsync

• should I remount / read only or to a new mount point?

[cds60601]

Hava a look at using dd (disk duplicator)

A very high-level show of what you can do with it is here.

Keep in mind, this is dangerous if used incorrectly - affectionately called Disk Destroyer if not careful.

[mrmazda]

So in 'pseudo-code' I would like to make a new bootable disk in a 250G partition on the spare disk (perhaps leaving it completely empty?) and then regularly and remotely running as root do something equivalent to:

Code: Select all

rsync -a /* /dev/sdb?/

so that every single file and permission on the current system is cloned onto the new disk partition. One nice thing about this is that after the initial big copy it would be incremental. Perhaps the qemu-utils mentioned could be part of this process, I will investigate. I suspect that cloning a running system may have some drawbacks or wrinkles but if they were simple to fix (perhaps a separate fixup script) I could live with that.

If the SSD ever fails I'm happy to drag it out from the cupboard, attach a keyboard and monitor and do whatever is needed on the new partition to make it boot from there instead.

As long as you're willing to drag it out when necessary, and you're going to have a second disk, using a live media boot you could clone the current disk to the spare disk by taking it out of the closet, and assigning new UUIDs to the cloned filesystems. Once that's done, you could put it back in the closet, then periodically update the clone using rsync. If/when the original dies, you could take it out of the closet, remove the bad disk, update the spare's fstab and boot files with the appropriate UUIDs, test boot, add a new spare, clone as before, then put it back in the closet.

Given the speed with which an OS installation can be performed on an SSD, it could be much simpler to just recreate a suitable partition structure on the spare, format a partition to be used for /home/ on it, then put it back in the closet. With this in place, you can use rsync to replicate the current /home/ to the spare disk's /home/ filesystem, and periodically update it. When the time comes to replace the original, you can remove it from the closet, remove the original, install the then current OS version to use the existing /home/ filesystem on the old spare, add a new spare as before, then put it back in the closet. A subdirectory on the spare's /home/ filesystem, or an extra partition-filesystem could be used, for backing up config files in /etc/ to facilitate global reconfiguration when the time would come to do a fresh installation on the original spare after repurposing it as the new main. This in some ways mimics what I do. I have /home/ and other data filesystems on RAID, while the OS on a small single SSD (actually 5 OSes).

[hopster]

In case it is useful for anyone else, this is what I have come up with so far. I have two partitions mounted from the other drive in the box at /big_data and /sys_archive

Code: Select all

# for home directory
rsync -axHAWXS --numeric-ids --info=progress2 --exclude="lost+found" /home/* /big_data/home

# for system settings
sudo rsync -axHAWXS --numeric-ids --info=progress2 /etc/* /big_data/etc

# complete system archive - boot, root, home
sudo fsarchiver savefs /sys_archive/system-backup.fsa /dev/sdb1 /dev/mapper/oito--vg-root /dev/mapper/oito--vg-home -v -j4 -A

It's pretty fast and hopefully stores everything I need. I can remove old .fsa files as I go but the first one is only 1.4G so I have a lot of space to play with (250G partition). Perhaps snapshots (timeshift?) would be more elegant but I can start thinking about that now that I have something working.

I have not yet un-archived one of these compressed system archives but I assume what is required is to build a new disk with approximately the right partition sizes and empty it into those. Does anyone here have any experience doing this?

[manyroads]

You can try Clonezilla.