Clone an SSD drive.

[hack3rcon]

Excuse me, It is a switch not a PC!

same

This is why you do the entire disk and not partitions.

doesn't matter what it is, use qemu-img and image the entire device. This includes boot sectors, magic offsets, propriety hiding places, etc...
Same advice applies to dd on unknowns. The superiority of qemu-img is it does understand file systems and can manipulate accordingly. dd does a stupid blind copy, which at times may be an advantage.

Thank you for your advice.
Can "qemu-img" repair the image that I created by "dd" tool?

[hack3rcon]

Wow that's slow.
So a 120GB SSD with a 8GB swap drive? Not encrypted? With qemu-img you have the compression option for the file stage of an image, so dd doesn't waste time on empty space! ...and I thought my dramless SSD's were slow!

Actually with compression you do need a beefy many thread cpu or it's slower I'd think. That last bullseye I wrote out is a 8.4GB partition size, it slowly gets there with use when residing in tempfs ram. It compressed last step to 1576MB, and the write out to the cheap SSD took about 8 minutes which seems slow to me. Full lvm luks drives write out to device size regardless of free space, so don't bother with compression, but those take about 20-30 minutes.

qemu-img is available from package qemu-utils, it's small. from memory (check)...
first image the drive to a file, if it's not encrypted it will not take full space...

Code: Select all

qemu-img convert -c -p -O qcow2 /path/to/device /path/to/file.qcow2

..from imgae file to device

Code: Select all

qemu-img convert -p -O raw /path/to/file.qcow2 /path/to/device

A few versions ago dd was superior for damaged disk recovery. I think qemu-img has gained some ability lately, so rarely a need for dd anymore. The state of the filesystem should be considered and you should massage the running OS before imaging. IE, Bleachbit, or something to zero unused and deleted sectors, otherwise the undelete data is still available in the image. While a file, a qcow2, you can run virt-sparsify --in-place to re-trim the image if you'd like to compress it more for archival - you'd run the qemu-img command AGAIN from file to file to reclaim the unused sectors. btw, I ignore trim in the running systems as everything goes through this process periodically and the ssd's get cycled. I use this to create the first OS a new system sees, no fresh installs for the last 4 years! Yes, I use a magnifier to read the MAC stickers so the network naming is correct OOB!

Enjoy

Excuse me, these commands do not change the source?

[CwF]

Can "qemu-img" repair the image that I created by "dd" tool?

No.
I assume your dd image is only the data partition and does not include what is considered the boot sector of the disk. In the same way as excluding non-pc efi partitions, hidden propriety sectors, etc, all outside the generic 'data' partition.
In the commands reference the entire disk, ie sdb and not sdb1.
Reference what I typed earlier, first write the image from the original ssd to a file, then file to new disk. If you have the setup you can do a single step device to device, but a file based backup copy is my recommendation.

If there is something odd about the 'switch' OS disk, the way to explore it is to have an image as a file and take that apart - instead of killing your working copy.

Excuse me, these commands do not change the source?

I'm not sure what you mean. They do not 'change' the data, the utility can pack and compress it,but will restore to original. The major error I could imagine is to copy a partition and then write it out as a complete image - which it is not. That is sda1 becomes sda - losing the boot partition?

[hack3rcon]

Can "qemu-img" repair the image that I created by "dd" tool?

No.
I assume your dd image is only the data partition and does not include what is considered the boot sector of the disk. In the same way as excluding non-pc efi partitions, hidden propriety sectors, etc, all outside the generic 'data' partition.
In the commands reference the entire disk, ie sdb and not sdb1.
Reference what I typed earlier, first write the image from the original ssd to a file, then file to new disk. If you have the setup you can do a single step device to device, but a file based backup copy is my recommendation.

If there is something odd about the 'switch' OS disk, the way to explore it is to have an image as a file and take that apart - instead of killing your working copy.

Excuse me, these commands do not change the source?

I'm not sure what you mean. They do not 'change' the data, the utility can pack and compress it,but will restore to original. The major error I could imagine is to copy a partition and then write it out as a complete image - which it is not. That is sda1 becomes sda - losing the boot partition?

Thank you.
But my "dd" image has boot sector:

Code: Select all

                                   Disk: SSD.dd
             Size: 111.8 GiB, 120034123776 bytes, 234441648 sectors
                       Label: dos, identifier: 0x48f7a8ec

    Device        Boot       Start        End   Sectors  Size Id Type
>>  SSD.dd1         *             63    2313359   2313297  1.1G  6 FAT16         
    SSD.dd2                  2313360   16113194  13799835  6.6G  b W95 FAT32
    SSD.dd3                 16113195   45656729  29543535 14.1G  c W95 FAT32 (LBA)
    SSD.dd4                 45656730  234441647 188784918   90G  f W95 Ext'd (LBA)
    ├─SSD.dd5               45656793   47954024   2297232  1.1G  6 FAT16
    ├─SSD.dd6               47954088   61769924  13815837  6.6G  b W95 FAT32
    ├─SSD.dd7               61769988  234436544 172666557 82.3G  b W95 FAT32
    └─Free space         234438656  234441647      2992  1.5M
 ┌────────────────────────────────────────────────────────────────────────────┐
 │  Partition type: FAT16 (6)                                                 │
 │      Attributes: 80                                                        │
 │ Filesystem UUID: 020C-0000                                                 │
 │Filesystem LABEL: /sysro                                                    │
 │      Filesystem: vfat                                                      │
 └────────────────────────────────────────────────────────────────────────────┘


It hasn't?
I used the commands that you said and it created a file about 5.1 GB (I guess removed blank sectors):

Code: Select all

$ sudo qemu-img convert -c -p -O qcow2 /dev/sdc /home/jason/qemu.qcow2
    (100.00/100%)


I tried to restore it on a new SDD drive:

Code: Select all

$ sudo qemu-img convert -p -O raw qemu.qcow2 /dev/sdc
    (100.00/100%)


I must test the drive.
Can I convert this ".qcow2" to another image and use "dd" to restore it on the physical device or give any speed parameter to "qemu-img" ?
Can convert to another file format, corrupt my file?

[CwF]

I must test the drive.

Yes, then maybe we'd be done!

Code: Select all

man qemu-img

Can I convert this ".qcow2" to another image and use "dd" to restore it on the physical device or give any speed parameter to "qemu-img" ?
Can convert to another file format, corrupt my file?

In the commands, the 'qcow2' or 'raw' is declaring the format of the output file. Input is auto detected. So using 'raw' is equivalent to the 'dd' format. Compression option '-c' is only used with qcow2, and as you see it saves much space. You can mount a qcow2 in various ways on a host, even a suitable vm. I have never seen this corrupt anything. I have used it for a known corrupted drive, it preserves the corruption, allows creating ways to investigate and fix the corruption, all on a capable host machine, then write the fixed image back to a disk. You could do a checksum on the image files, you could 'diff' them, you could use a hex editor. That all is done on a raw image (*.img or *.raw).
So yes, you can convert to raw (FULL SIZE) and then use dd

Code: Select all

$ sudo qemu-img convert -p -O raw qemu.qcow2 qemu.img

[hack3rcon]

I must test the drive.

Yes, then maybe we'd be done!

Code: Select all

man qemu-img

Can I convert this ".qcow2" to another image and use "dd" to restore it on the physical device or give any speed parameter to "qemu-img" ?
Can convert to another file format, corrupt my file?

In the commands, the 'qcow2' or 'raw' is declaring the format of the output file. Input is auto detected. So using 'raw' is equivalent to the 'dd' format. Compression option '-c' is only used with qcow2, and as you see it saves much space. You can mount a qcow2 in various ways on a host, even a suitable vm. I have never seen this corrupt anything. I have used it for a known corrupted drive, it preserves the corruption, allows creating ways to investigate and fix the corruption, all on a capable host machine, then write the fixed image back to a disk. You could do a checksum on the image files, you could 'diff' them, you could use a hex editor. That all is done on a raw image (*.img or *.raw).
So yes, you can convert to raw (FULL SIZE) and then use dd

Code: Select all

$ sudo qemu-img convert -p -O raw qemu.qcow2 qemu.img

Thank you so much.
I see that "qemu-img" has "dd" parameter too:

Code: Select all

$ qemu-img dd -f qcow2 -O raw bs=4M if=image.qcow2 of=/dev/sdc


What is your opinion about it?
Can "qemu-img" copy the hidden parts (HPA and DCO) of disk?
I used below "qemu" command to boot the image but nothing booted:

Code: Select all

$ qemu-system-i386 -hda qemu.qcow2 -m 512


It just shows me a black screen with a blinking cursor and nothing booted. What is wrong? I tested other architectures like "ARM" too, but all of them shown me a same result.

[CwF]

Thank you so much.
I see that "qemu-img" has "dd" parameter too:

Yes, I mentioned qemu-img has gained much in the last few versions. I don't know the extents of proprietary disk formats, so this is your job! I've wondered about some hybrid disk and firmware oddities, but don't have purpose to explore it. The last time I chewed on a corrupted disk for somebody was few years ago and I found then qemu-img couldn't finish without dd, and we're back to today and I think it's better now.

Can "qemu-img" copy the hidden parts (HPA and DCO) of disk?

An issue with that is different manufactures are not unified in method, so it may not be possible to transfer if some translation is required. A vendor could use any bogus trick to hook their proprieties.

I use its layering magic and native qcow2 abilities to obsolete many older concepts, like installers, imagers, backups, etc, For creating virtual drives and other magic it's good to also get familiar with libguestfs-tools and other helpers.

I hope I pointed you in the right direction!

[hack3rcon]

Thank you so much.
I see that "qemu-img" has "dd" parameter too:

Yes, I mentioned qemu-img has gained much in the last few versions. I don't know the extents of proprietary disk formats, so this is your job! I've wondered about some hybrid disk and firmware oddities, but don't have purpose to explore it. The last time I chewed on a corrupted disk for somebody was few years ago and I found then qemu-img couldn't finish without dd, and we're back to today and I think it's better now.

Can "qemu-img" copy the hidden parts (HPA and DCO) of disk?

An issue with that is different manufactures are not unified in method, so it may not be possible to transfer if some translation is required. A vendor could use any bogus trick to hook their proprieties.

I use its layering magic and native qcow2 abilities to obsolete many older concepts, like installers, imagers, backups, etc, For creating virtual drives and other magic it's good to also get familiar with libguestfs-tools and other helpers.

I hope I pointed you in the right direction!

Thank you so much for your help.
I restored the image on a new SSD and not worked

[bester69]

Thank you so much.
I see that "qemu-img" has "dd" parameter too:

Yes, I mentioned qemu-img has gained much in the last few versions. I don't know the extents of proprietary disk formats, so this is your job! I've wondered about some hybrid disk and firmware oddities, but don't have purpose to explore it. The last time I chewed on a corrupted disk for somebody was few years ago and I found then qemu-img couldn't finish without dd, and we're back to today and I think it's better now.

Can "qemu-img" copy the hidden parts (HPA and DCO) of disk?

An issue with that is different manufactures are not unified in method, so it may not be possible to transfer if some translation is required. A vendor could use any bogus trick to hook their proprieties.

I use its layering magic and native qcow2 abilities to obsolete many older concepts, like installers, imagers, backups, etc, For creating virtual drives and other magic it's good to also get familiar with libguestfs-tools and other helpers.

I hope I pointed you in the right direction!

Thank you so much for your help.
I restored the image on a new SSD and not worked

qemu seems to me an advanced tool for a begginer user to put yourself in troubles..
use gparted; create a table partiton of whole SSD and copy/paste each partition one by one (make it easy ) .. .

or use clonezilla or a windows clonning disk like GhostNorton

[hack3rcon]

I guess that device running Linux or QNX. Is it matter?

[hack3rcon]

Device running VxWorks operating system.
I compared the original drive partitions hashes with cloned drive and they are not same :/

Code: Select all

Original:

Partition 0:
Size: 1.10 GB

SHA-1: 20f9d5196736771e6bffb0c12052e996d35e8df3
MD5: dd5a5ffb08d35d7758a77c9dacba3874


Partition 1:
Size: 6.58 GB

SHA-1: 8018ce32f313cabfd8ebf9fb7f14a1d79988a1b0
MD5: d5277242c62bde2ec3932049d65229ab



Partition 2:
Size: 14.09 GB

SHA-1: 1a3385d06f9ce95af80669b1af29d6d3a9e436ce
MD5: 7845181ea5c5467a7219ed714d97b09c


Partition 3:
Size: 1.10 GB

SHA-1: 77216106ff98a77479a81b95abd10b4a4a5cc46f
MD5: 032e997221da902b884ce23b26c6d0e9


Partition 4:
Size: 6.59 GB

SHA-1: 507276ebc81fdadd4dcd36b9ac2c024f7d52ddc9
MD5: 7cf0951a2fd6c4e9bbd101688b719ddb


Partition 5:
Size: 82.33 GB

SHA-1: c3d0cf989c17cc8fae518ab8729df3d4f9599eb5
MD5: b779439080a8a19c6f38f99883d86d9a


And:

Code: Select all

Cloned:

Partition 0:
Size: 1.10 GB

SHA-1: 88b4f781a66f8407326bde847c16114e40b7aacf
MD5: 8e67ea69b7d17bf867760175b6697d11


Partition 1:
Size: 6.58 GB

SHA-1: 6f623c54386e4346fd72cb09ea4539b38543575b
MD5: 2f3ad684cac91559646242764e984355



Partition 2:
Size: 14.09 GB

SHA-1: a91ad5962611ca36746283de9bc6c3d28921aeb0
MD5: dfbd62f46af3d3d4c6a2989f0a0c12a3


Partition 3:
Size: 1.10 GB

SHA-1: 6dab8007bed89608bca5bd026c0aee9c45dbd48b
MD5: c5779f6cb8b00106799cbd08d468ca79


Partition 4:
Size: 6.59 GB

SHA-1: 980908929ee34d2ca09e15dab1a7bc474b00e587
MD5: ab7e328be37708cbc1a597a4e115782e


Partition 5:
Size: 82.33 GB

SHA-1: a22fa1ba60062cf4981c23d0b8cbee807679cc82
MD5: 1685d0acb10c0d4e4d2ef8b419c0c588


Why? Is it normal?
MITEL can't boot with cloned drive:

Code: Select all

MITEL SYSTEM ROM R3.1/10 Aug 5 2011 (83 - POWER_ON_RESET)

CCA Number : 76009581 Ra.4
System Model : 00620001 F2500
CPU Model : 8360 R80480021
Reset Config Low : 0804008c
Reset Config High : b4500006
Coherent System Bus Clock (MHz) : 266
CPM Clock (MHz) : 399
Core Clock (MHz) : 533
DDR Clock (MHz) : 133
Local Bus Clock (MHz) : 66
Input Clock (MHz) : 33
Internal Memory Map : f0000000
Main Memory (MB) : 512
Local Memory (MB) : 0
Flash Memory (MB) : 4
MAC Address : 08000f640a40
POST Bypass : 0
Watchdog Bypass : 0
Slot Id : 0

Starting POST...

SDRAM CS1 Data Bit Walk 1................... PASSED
SDRAM CS1 Data Bit Walk 0................... PASSED
SDRAM CS1 Address Walk 1................... PASSED
SDRAM CS1 Address Walk 0................... PASSED
SDRAM CS1 March Test........................ PASSED

Starting vxWorks...

Decompressing FPGA image...615412 bytes decompressed successfully

Programming the FPGA image...OK

FPGA version 0002:6123 loaded

SATA RAID Controller Not Present.











VxWorks System Boot


Copyright 1984-2002 Wind River Systems, Inc.





CPU: Mitel MMC-C PPC83XX F2500
Version: VxWorks5.5.2
BSP version: 3.1/10
Creation date: Aug 5 2011, 18:41:11




Press <SPACE><SPACE><SPACE> to stop auto-boot AFTER countdown starts...
<7><6><5><4><3><2><1><0>
auto-booting...


boot device : ata=0
unit number : 0
processor number : 0
host name : bootHost
file name : /partition1/RTC8260
inet on ethernet (e) : 172.18.6.5:fffffff0
host inet (h) : 192.168.137.10
gateway inet (g) : 172.18.6.14
user (u) : ftp
ftp password (pw) : ftp
flags (f) : 0x0
target name (tn) : MyPhone
other (o) : qefcc


Disk Configuration Started.
Creating block device for controller 0 drive 0... done.
Creating CBIO device for controller 0 drive 0... done.
0KB added to /partition1 partition
free disk space after partition 0 is 0KB
0KB added to /partition2 partition
free disk space after partition 1 is 0KB
0KB added to /partition3 partition
free disk space after partition 2 is 0KB
0KB added to /partition4 partition
free disk space after partition 3 is 0KB
Attaching to partition table for 4 partitions on controller 0 drive 0... done.
Creating Cache Layer for partition 0 (/partition1) on controller 0 drive 0... done.
Creating Cache Layer for partition 1 (/partition2) on controller 0 drive 0... done.
Creating Cache Layer for partition 2 (/partition3) on controller 0 drive 0... done.
Creating Cache Layer for partition 3 (/partition4) on controller 0 drive 0... done.
Creating Dos FS device for partition 0 (/partition1) on controller 0 drive 0... done.
Creating Dos FS device for partition 1 (/partition2) on controller 0 drive 0... done.
Creating Dos FS device for partition 2 (/partition3) on controller 0 drive 0... done.
Creating Dos FS device for partition 3 (/partition4) on controller 0 drive 0... done.
Disk Configuration Complete.
Loading /partition1/RTC8260...66125120
Starting at 0x6857c8...


https://www.udrop.com/Koc/image_3303.png

[p.H]

How did you compute the partition hashes ?
Can you remind us how exactly you cloned the drive ? This thread has been quite long, and I am reluctant to re-read it all.
If the cloned partitions have the exact same sizes and contents as the original ones, they should have the same hashes.

[hack3rcon]

How did you compute the partition hashes ?
Can you remind us how exactly you cloned the drive ? This thread has been quite long, and I am reluctant to re-read it all.
If the cloned partitions have the exact same sizes and contents as the original ones, they should have the same hashes.

I used "dd" and "PassMark OSForensics". The PassMark OSForensics has some options about create image and hash computing.

[CwF]

I used "dd" and "PassMark OSForensics". The PassMark OSForensics has some options about create image and hash computing.

Very good hack3rcon, I admire your diligence.

If the cloned partitions have the exact same sizes and contents as the original ones, they should have the same hashes.

You'd think, unless there is copy protection in place.

So hack3rcon's idea for the hex editor is correct, and where your going to end up.

I claim that qemu-img is forensically accurate - that is a deleted file before the image copy will be retrievable in the same way on the copy... Yes. But, the utilities are focused on data, not structure, so an intentional block of zeros are often discarded, sectors don't line up, yet all the 'data' is there. This is why I keep saying the magic is elsewhere - it's in the firmware! The data likely does have odd offsets designed to be ignored if copied. The firmware knows what sector has what data - that IS a relationship that is NOT relevant to 'preserving data' - that is preserving orientation.

So, spot check data alignment - yes with a hex editor! Divide the image up and check large blocks to find the offsets. You'll likely find the data is the same, what sector is not the same. The secret should be non-standard non-data at boundaries of data blocks that are themselves identical.

In the hexeditor you don't exactly do it 'by sight'. There is a search function ya' know! You can search content or by sector. The exercise here is to verify particular data at a particular sector

Again- get this point = the magic is the firmware of this non-standard device. The data structure on the disk is intentionally non-standard structure and gets 'corrected' by standard methods and the data is not 'changed'.

The simple theoretical compare to understand what is happening is Optical media copy protection. Do these image utilities make a perfect copy of the data, they DO! Do these CD/DVD checksums match? NO. The images don't work do they? Because the copies lost offset info, info deemed irrelevant to data integrity, info only the device firmware knows!

The other possibility is the firmware of the SSD itself. It can be altered from standard. There is no analog to disk geometry though it will emulate that info. That can be paired with the device firmware. Usually the filesystem is responsible for 'where is what' This device does NOT use the filesystem for it's security check - it reads sector 42 (whatever), and that is what is changed....indiscriminately and innocently changed by the copy utility to reside in sector 1.

It's called 'Copy-Protection'. And hack3rcon, you can crack it! And this thread is now off-topic.

[p.H]

dd preserves data and alignment, unless it encountered read errors and conv=sync was not specified.
If MD5 sums do not match, the copy is just wrong.
What "firmware magic" are you talking about ? The drive or the host system ?