Unzipping .iso file, and then zipping it - Failing to boot!

Help with issues regarding installation of Debian

Unzipping .iso file, and then zipping it - Failing to boot!

Postby _nobody_ » 2018-04-15 11:49

I have problem with unzipping debian-live-9.4.0-amd64-gnome.iso to the directory (using 7z x), and then zipping it again (using mkisofs -lJR). I followed several net pages, found on the www, and I understood that there is NOT provided a single program which does the both. 7z a reports me the flag: NOT IMPLEMENTED! :mrgreen:

The original boots happily, but remake does not boot. The installer reports the following error: "FATAL: no bootable madia found! System halted!"

I did the best according what I found, and here is my findings, presented in form of a host execution report!

I have in the iso_files directory the following:
Code: Select all
[vuser@localhost iso_files]$ ls -al
total 5216288
drwxr-xr-x. 3 vuser vboxusers       4096 Apr 15 08:43 .
drwxr-xr-x. 8 vuser vboxusers       4096 Apr  3 10:33 ..
-r--------. 1 vuser vboxusers  305135616 Apr  3 12:47 debian-9.4.0-amd64-netinst.iso
-rw-r--r--. 1 vuser vboxusers 2365587456 Apr 15 09:34 debian-live-9.4.0-amd64-gnome.iso
-rw-r--r--. 1 vuser vboxusers 2365587456 Apr  3 13:17 debian-live-9.4.0-amd64-gnome.iso.genesis
-rw-r--r--. 1 vuser vboxusers  305135616 Mar 24 21:09 gparted-live-0.30.0-1-amd64.iso
-rw-r--r--. 1 vuser vboxusers       2416 Apr 15 08:43 README
drwxr-xr-x. 4 vuser vboxusers       4096 Apr 15 09:30 tmp
[vuser@localhost iso_files]$

Now, when I switch to the directory tmp, I have the following:
Code: Select all
[vuser@localhost tmp]$ ls -al
total 2310168
drwxr-xr-x.  4 vuser vboxusers       4096 Apr 15 09:46 .
drwxr-xr-x.  3 vuser vboxusers       4096 Apr 15 09:45 ..
-rw-r--r--.  1 vuser vboxusers 2365587456 Apr 15 08:48 debian-live-9.4.0-amd64-gnome.iso.genesis
drwxr-xr-x.  9 vuser vboxusers       4096 Apr 12 18:37 extracted_main
drwxr-xr-x. 10 vuser vboxusers       4096 Apr 15 09:29 extracted-modified
-rw-r--r--.  1 vuser vboxusers       2416 Apr 15 09:46 README
[vuser@localhost tmp]$

Then I do:
Code: Select all
[vuser@localhost tmp]$ ls -al extracted-modified/
total 8
drwxr-xr-x. 2 vuser vboxusers 4096 Apr 15 09:48 .
drwxr-xr-x. 4 vuser vboxusers 4096 Apr 15 09:48 ..
[vuser@localhost tmp]$ 7z x -oextracted-modified/ debian-live-9.4.0-amd64-gnome.iso.genesis

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Core(TM) i5-4300U CPU @ 1.90GHz (40651),ASM,AES-NI)

Scanning the drive for archives:
1 file, 2365587456 bytes (2256 MiB)

Extracting archive: debian-live-9.4.0-amd64-gnome.iso.genesis

WARNINGS:
There are data after the end of archive

--
Path = debian-live-9.4.0-amd64-gnome.iso.genesis
Type = Iso
WARNINGS:
There are data after the end of archive
Physical Size = 2364565504
Tail Size = 1021952
Comment =
Volume: d-live 9.4.0 gn amd64
Preparer: XORRISO-1.4.6 2016.09.16.133001, LIBISOBURN-1.4.6, LIBISOFS-1.4.6, LIBBURN-1.4.6
Created = 2018-03-10 13:40:06
Modified = 2018-03-10 13:40:06

Everything is Ok                                                             

Archives with Warnings: 1

Warnings: 1
Folders: 299
Files: 705
Size:       2363461865
Compressed: 2365587456
[vuser@localhost tmp]$

And the following is true:
Code: Select all
[vuser@localhost tmp]$ ls -al extracted-modified/
total 40
drwxr-xr-x. 10 vuser vboxusers 4096 Apr 15 09:48  .
drwxr-xr-x.  4 vuser vboxusers 4096 Apr 15 09:48  ..
drwx------.  3 vuser vboxusers 4096 Mar 10 12:39  boot
drwx------.  2 vuser vboxusers 4096 Apr 15 09:48 '[BOOT]'
drwx------.  3 vuser vboxusers 4096 Mar 10 12:39  d-i
drwx------.  2 vuser vboxusers 4096 Mar 10 12:40  .disk
drwx------.  3 vuser vboxusers 4096 Mar 10 12:40  dists
drwx------.  2 vuser vboxusers 4096 Mar 10 12:40  isolinux
drwx------.  2 vuser vboxusers 4096 Mar 10 12:39  live
drwx------.  3 vuser vboxusers 4096 Mar 10 12:40  pool
[vuser@localhost tmp]$

Then I do squashing, making AGAIN the same .iso to test it!
Code: Select all
[vuser@localhost tmp]$ mkisofs -lJR -o debian-live-9.4.0-amd64-gnome.iso extracted-modified/
I: -input-charset not specified, using utf-8 (detected in locale settings)
  0.43% done, estimate finish Sun Apr 15 09:51:44 2018
  0.87% done, estimate finish Sun Apr 15 09:51:44 2018
  ...[snap]...
 99.55% done, estimate finish Sun Apr 15 09:51:46 2018
 99.98% done, estimate finish Sun Apr 15 09:51:46 2018   
Total translation table size: 0
Total rockridge attributes bytes: 134286
Total directory bytes: 669696
Path table size(bytes): 5204
Max brk space used 14b000
1155222 extents written (2256 MB)
[vuser@localhost tmp]$ ls -al
total 4620616
drwxr-xr-x.  4 vuser vboxusers       4096 Apr 15 09:51 .
drwxr-xr-x.  3 vuser vboxusers       4096 Apr 15 09:45 ..
-rw-r--r--.  1 vuser vboxusers 2365894656 Apr 15 09:51 debian-live-9.4.0-amd64-gnome.iso
-rw-r--r--.  1 vuser vboxusers 2365587456 Apr 15 08:48 debian-live-9.4.0-amd64-gnome.iso.genesis
drwxr-xr-x.  9 vuser vboxusers       4096 Apr 12 18:37 extracted_main
drwxr-xr-x. 10 vuser vboxusers       4096 Apr 15 09:48 extracted-modified
-rw-r--r--.  1 vuser vboxusers       2416 Apr 15 09:46 README
[vuser@localhost tmp]$

Please, do note THE DIFFERENCE in size between original .iso and generated .iso! :shock:

And, when I tried to run the generated .iso, the following message poped-up:
"FATAL: no bootable madia found! System halted!"

What I am doing wrong here??? :roll:

Thank you,
_nobody_
_nobody_
 
Posts: 19
Joined: 2018-04-10 02:07

Re: Unzipping .iso file, and then zipping it - Failing to bo

Postby p.H » 2018-04-15 11:58

1) You don't unzip or zip an ISO file. An ISO file is a disk or filesystem image (like FAT or ext4), not a compressed archive.
2) The bootable part in an ISO image is not in the visible tree (except maybe for EFI boot, if there is an EFI/ directory at the root of the filesystem, not sure).

Are your trying to boot this image from an optical disk or a USB flash drive ?
p.H
 
Posts: 525
Joined: 2017-09-17 07:12

Re: Unzipping .iso file, and then zipping it - Failing to bo

Postby Dai_trying » 2018-04-15 13:48

I have modified a Live-Cd successfully but it is not Debian (it is a derivative though) and use the following procedure:-

  • create separate folder for the task and enter it
  • extract the image to the current location (I use bsdtar)
  • use unsquashfs to unpack the live/filesystem.squashfs file to a new folder
  • mount dev/pts to the unpacked filesystem
  • copy /etc/resolv.conf if you need network access (for installing packages etc)
  • chroot into the new folder with unpacked filesystem
  • mount /proc and /sys to this folder
  • change what is required and/or install/remove packages
  • exit the chroot environment when finished
  • unmount /proc /sys /dev/pts and remove resolv.conf from new folder
  • make a new filesystem.squashfs file (using mksquashfs) and overwrite the old one with it
  • create new iso image using xorriso

This is just an outline of my procedure and I have tested it on a Debian image but back when Jessie was stable as I don't need to mess with the images very often but I have found it to work every time for me (subject to correct parameters for all the commands of course).
Dai_trying
 
Posts: 531
Joined: 2016-01-07 12:25

Re: Unzipping .iso file, and then zipping it - Failing to bo

Postby Dai_trying » 2018-04-16 09:52

I just tested in a VM and the following is a list of commands I used to "remaster" the Debian Live Xfce iso, in this example I have removed libreoffice and quodlibet and installed bash-completion and gparted.

There are two assumptions...

  1. the image is in your $HOME folder
  2. sudo command is available

Code: Select all
sudo apt install bsdtar isolinux squashfs-tools syslinux syslinux-utils xorriso
mkdir my_temp_dir
cd my_temp_dir/
sudo bsdtar xfp ~/debian-live-9.4.0-amd64-xfce.iso
sudo unsquashfs -d some_work_dir live/filesystem.squashfs
sudo mount --bind /dev/pts/ some_work_dir/dev/pts/
sudo cp /etc/resolv.conf some_work_dir/etc/
sudo chroot some_work_dir/

#### Now in chroot environment ####
#### You can also change any other settings/options you prefer for your image ####

mount -t proc /proc /proc
mount -t sysfs /sys /sys
apt update
apt --purge autoremove libreoffice*
apt --purge autoremove quodlibet
apt install bash-completion
apt install gparted
apt upgrade (optional, was not required for my install as it was a recent image)
exit

#### Now into your normal environment ####

sudo umount ~/my_temp_dir/some_work_dir/sys
sudo umount ~/my_temp_dir/some_work_dir/proc
sudo umount ~/my_temp_dir/some_work_dir/dev/pts
sudo rm some_work_dir/etc/resolv.conf
sudo mksquashfs some_work_dir filesystem.squashfs
sudo mv filesystem.squashfs live/filesystem.squashfs
sudo mv some_work_dir/ ../
cd ../
sudo xorriso -as mkisofs -R -r -J -joliet-long -l -isohybrid-mbr /usr/lib/ISOLINUX/isohdpfx.bin -partition_offset 16 -A "Debian Xfce Live (DT)" -publisher "Dai Trying" -V "Debian 9.4.0 Stretch" -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o ~/debian-remastered.iso ~/my_temp_dir


After this you will have a boot-able working live image that can be used in a VM or by writing to a USB stick and booting directly.

You can (and probably should) change the publisher in the xorriso command to relect your own as this has my name, also you can give your image any title you prefer. If you check through the xorriso man age you can also change some of the other options including compression rate for final image.

I hope you (or anyone else) find this of some use.
Dai_trying
 
Posts: 531
Joined: 2016-01-07 12:25

Re: Unzipping .iso file, and then zipping it - Failing to bo

Postby _nobody_ » 2018-04-16 19:16

I am reading you, Die Hard Trying... And what you wrote here is pure GOLD! :D

Really, it is... But I have (even) more die hard problems to solve than what you wrote here. :mrgreen:

This is what I am trying to do (crazy, isn't it)?
viewtopic.php?f=17&t=137198

Let me first try what you wrote here. YES, I'll try this with debian-live-9.4.0-amd64-gnome.iso . And... I did try (before reading your latest post) part with unpacking ISO and packing it back.

And... using very similar command as you are using:
Code: Select all
sudo xorriso -as mkisofs -R -r -J -joliet-long -l -isohybrid-mbr /usr/lib/ISOLINUX/isohdpfx.bin -partition_offset 16 -A "Debian Xfce Live (DT)" -publisher "Dai Trying" -V "Debian 9.4.0 Stretch" -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o ~/debian-remastered.iso ~/my_temp_dir

The remastered .iso does boot. But does NOT do what I aimed for! What I have added to the unpacked .iso. :shock:

Let me first try your script. Since... I did not change anything in /live directory. So, in this lieu, I did not do unsquash (why should I do this if I do not need to change /live directory at all)?

But I did change something else: /boot directory, after unpacking. I actually changed the /boot/grub/grub.cfg, and moved one option: menuentry "Debian Installer" on the top, naively expecting that remastered .iso will reflect this change!

Simplified original /boot/grub/grub.cfg:
Code: Select all
menuentry "Debian GNU/Linux Live (kernel 4.9.0-6-amd64)" {
  linux  /live/vmlinuz-4.9.0-6-amd64 boot=live components "${loopback}"
  initrd /live/initrd.img-4.9.0-6-amd64
}
menuentry "Debian Live with Localisation Support" {
  linux  /live/vmlinuz-4.9.0-6-amd64 boot=live components "${loopback}"
  initrd /live/initrd.img-4.9.0-6-amd64
}
menuentry "Graphical Debian Installer" {
  linux  /d-i/gtk/vmlinuz append video=vesa:ywrap,mtrr vga=788 "${loopback}"
  initrd /d-i/gtk/initrd.gz
}
menuentry "Debian Installer" {
  linux  /d-i/vmlinuz  "${loopback}"
  initrd /d-i/initrd.gz
}
menuentry "Debian Installer with Speech Synthesis" {
  linux  /d-i/gtk/vmlinuz speakup.synth=soft "${loopback}"
  initrd /d-i/gtk/initrd.gz
}

What I re-edit in the /boot/grub/grub.cfg is the following:
Code: Select all
menuentry "Debian Installer" {
  linux  /d-i/vmlinuz  "${loopback}"
  initrd /d-i/initrd.gz
}
menuentry "Debian GNU/Linux Live (kernel 4.9.0-6-amd64)" {
  linux  /live/vmlinuz-4.9.0-6-amd64 boot=live components "${loopback}"
  initrd /live/initrd.img-4.9.0-6-amd64
}
menuentry "Debian Live with Localisation Support" {
  linux  /live/vmlinuz-4.9.0-6-amd64 boot=live components "${loopback}"
  initrd /live/initrd.img-4.9.0-6-amd64
}
menuentry "Graphical Debian Installer" {
  linux  /d-i/gtk/vmlinuz append video=vesa:ywrap,mtrr vga=788 "${loopback}"
  initrd /d-i/gtk/initrd.gz
}
menuentry "Debian Installer with Speech Synthesis" {
  linux  /d-i/gtk/vmlinuz speakup.synth=soft "${loopback}"
  initrd /d-i/gtk/initrd.gz
}

But, still, I see original grub boot sequence after re-packing with xorriso -as mkisofs remastered .iso (I also tried to change the remastered .iso name, and VBox immediately recognized the problem signalling it, so I am after all running the correct remastered .iso)!

My best guess, it has everything to do with the /live directory after all, hasn't it? :mrgreen:

Thank you,
_nobody_
_nobody_
 
Posts: 19
Joined: 2018-04-10 02:07

Re: Unzipping .iso file, and then zipping it - Failing to bo

Postby Dai_trying » 2018-04-16 20:42

I'm definitely no expert with grub, but would you need to update grub (when in the chroot environment) for the changes to take effect? Not too sure with grub though especially with a Live-Cd but might be worth a try.
Dai_trying
 
Posts: 531
Joined: 2016-01-07 12:25

Re: Unzipping .iso file, and then zipping it - Failing to bo

Postby Head_on_a_Stick » 2018-04-17 05:10

_nobody_ wrote:I see original grub boot sequence after re-packing with xorriso -as mkisofs remastered .iso

The Debian ISO images use syslinux to boot in non-UEFI mode so look in the isolinux folder for the .cfg files ;)
User avatar
Head_on_a_Stick
 
Posts: 7906
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Unzipping .iso file, and then zipping it - Failing to bo

Postby _nobody_ » 2018-04-17 19:57

Hello Die Hard Trying, Head on the Stick and Ropes,

I am working on both domains (on Die Hard Trying script), and on YOCTO, somehow I got distracted today with some sticks and ropes here and there, everywhere... :mrgreen:

Die Trying,

Your script is worth Gold, as I mentioned before, I even was not aware that the whole root tree is packed in there as squashed fs... My bad, really! :roll:

I rewrote your script to make it very local... And it does work! The last step: xorriso -as mkisofs I did not try, since in the mean time I got distracted. But everything else does work... Even, I have 4.15.16-300.fc27.x86_64 kernel from my host Fedora 27, but albeit, the Debian root tree, unsquashed, works like a charm, all the apt upgrade and other commands (deleting libreoffice and similar) work seamlessly! :shock:

Head on sticks and ropes,

You seems to be correct. I did not try to modify syslinux directory, but I have spoted in there menu.cfg... GRUB 1.99 config file. It tells a story just looking by/exposing the name... And I do use legacy, not at all UEFI! :idea:

Please, stay tuned for more info/more tests tomorrow!

_nobody_
_nobody_
 
Posts: 19
Joined: 2018-04-10 02:07

Re: Unzipping .iso file, and then zipping it - Failing to bo

Postby _nobody_ » 2018-04-23 10:22

Hello to everyone,

Here are results of my latest investigation, concerning this thread. Not too fast (I need to push myself more to problem solution), since I do other things as well. :idea:

Regarding the last command in Die Hard Trying script:
Code: Select all
sudo xorriso -as mkisofs -R -r -J -joliet-long -l -isohybrid-mbr /usr/lib/ISOLINUX/isohdpfx.bin -partition_offset 16 -A "Debian Xfce Live (DT)" -publisher "Dai Trying" -V "Debian 9.4.0 Stretch" -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o ~/debian-remastered.iso ~/my_temp_dir

It appeared that this line creates debian-remastered.iso, but bootable ONLY as legacy boot (using GRUB 1.99, and the isolinux/menu.cfg).

Die Hard Trying (with the vengeance),

Please, could you find one sudo xorriso -as mkisofs command which actually packs EFI based .iso:?:
_______

I am able to change/re-shuffle order in the execution of GRUB options (and put first in the order which is Debian Installer), and I am able to throw-in the timer (let say, for 5 seconds), after which the leading option - Debian Installer will boot automatically. :twisted:

Here is how I shuffled menu cfg, and how first few lines of it look:
Code: Select all
INCLUDE stdmenu.cfg
MENU title Main Menu

DEFAULT Debian Installer
LABEL Debian Installer
  SAY "Booting Debian Installer..."
  linux /d-i/vmlinuz
  APPEND initrd=/d-i/initrd.gz

LABEL Debian GNU/Linux Live (kernel 4.9.0-6-amd64)
  SAY "Booting Debian GNU/Linux Live (kernel 4.9.0-6-amd64)..."
  linux /live/vmlinuz-4.9.0-6-amd64
  APPEND initrd=/live/initrd.img-4.9.0-6-amd64 boot=live components

Actually, to be precise, GRUB 1.99 boots isolinux.cfg file rather then menu.cfg (menu.cfg is just a part of the isolinux.cfg boot menu)! :mrgreen:

_nobody_
_nobody_
 
Posts: 19
Joined: 2018-04-10 02:07

Re: Unzipping .iso file, and then zipping it - Failing to bo

Postby Dai_trying » 2018-04-23 13:45

_nobody_ wrote:Die Hard Trying (with the vengeance),

Please, could you find one sudo xorriso -as mkisofs command which actually packs EFI based .iso:?:


This would require some research and testing which I am not currently able to do, I will add it to my todo list though and if I find something I will post here. :)
Dai_trying
 
Posts: 531
Joined: 2016-01-07 12:25

Re: Unzipping .iso file, and then zipping it - Failing to bo

Postby Head_on_a_Stick » 2018-04-23 17:18

_nobody_ wrote:could you find one sudo xorriso -as mkisofs command which actually packs EFI based .iso:?

Here's one I made earlier:
Code: Select all
function mkiso {
   xorriso -as mkisofs \
   -iso-level 3 \
   -full-iso9660-filenames \
   -volid "$1" \
   -eltorito-boot isolinux/isolinux.bin \
   -eltorito-catalog isolinux/boot.cat \
   -no-emul-boot -boot-load-size 4 -boot-info-table \
   -isohybrid-mbr isolinux/isohdpfx.bin \
   -eltorito-alt-boot \
   -e boot/grub/efi.img \
   -no-emul-boot -isohybrid-gpt-basdat \
   -output "${1}.img" ./
}

The first argument of the function names the image ;)

The important part for UEFI booting is isolinux/isohdpfx.bin and this can be found in the isolinux package, it is installed to /usr/lib/ISOLINUX/isohdpfx.bin
User avatar
Head_on_a_Stick
 
Posts: 7906
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Unzipping .iso file, and then zipping it - Failing to bo

Postby Dai_trying » 2018-04-23 19:58

That's awesome! Thank you @HOAS that's definitely something I can use :D (and saved me a good bit of time researching it too!)
Dai_trying
 
Posts: 531
Joined: 2016-01-07 12:25

Re: Unzipping .iso file, and then zipping it - Failing to bo

Postby _nobody_ » 2018-04-25 12:55

Head_on_a_Stick wrote:
_nobody_ wrote:could you find one sudo xorriso -as mkisofs command which actually packs EFI based .iso:?

Here's one I made earlier:
Code: Select all
function mkiso {
   xorriso -as mkisofs \
   -iso-level 3 \
   -full-iso9660-filenames \
   -volid "$1" \
   -eltorito-boot isolinux/isolinux.bin \
   -eltorito-catalog isolinux/boot.cat \
   -no-emul-boot -boot-load-size 4 -boot-info-table \
   -isohybrid-mbr isolinux/isohdpfx.bin \
   -eltorito-alt-boot \
   -e boot/grub/efi.img \
   -no-emul-boot -isohybrid-gpt-basdat \
   -output "${1}.img" ./
}

The first argument of the function names the image ;)

The important part for UEFI booting is isolinux/isohdpfx.bin and this can be found in the isolinux package, it is installed to /usr/lib/ISOLINUX/isohdpfx.bin

For me, @HOAS (OAS???) it plays differently! :mrgreen:

Do NOT forget, on my host, I use Fedora 27, so my Debian is just poor unhappy slave VM over VBox! :shock:

Here is my command, if you allow me!
Code: Select all
xorriso -as mkisofs -U -r -v -T -J -joliet-long -V "Debian 9.4.0 Stretch" -volset "Debian 9.4.0 Stretch" -A "Debian 9.4.0 Stretch" -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e boot/grub/efi.img -no-emul-boot -graft-points -isohybrid-mbr /usr/share/syslinux/isohdpfx.bin -o debian-live-9.4.0-amd64-gnome.iso tmp_dir

And so far, it does wok for me on the re-assembled/re-mastered Debian 9.4.0 GNOME image with options introduced by Die Hard Trying (+ I added the following command there, in VBox):
Code: Select all
VboxManage modifyvm "vmname" --firmware efi

Stay tuned, boyzzzzz.... :idea:

_nobody_
_nobody_
 
Posts: 19
Joined: 2018-04-10 02:07


Return to Installation

Who is online

Users browsing this forum: No registered users and 5 guests

fashionable