PXE Install Lenny with files from CD on webserver

[thefiend]

Hello,

I am having serious trouble getting Debian to install over our LAN using PXE/syslinux, a custom preseed.cfg, and the files from the DVD on a webserver. I think the problem is using the files from the DVD on the webserver instead of a mirror, but I believe this should be possible. I do not have enough space on the webserver to house an entire debian mirror.

I am using the latest linux and initrd.gz files from ftp://ftp.us.debian.org/debian/dists/le ... ller/i386/.

I am using the extracted files from http://cdimage.debian.org/debian-cd/5.0 ... -DVD-1.iso on the webserver.

I had to remove the symlinks testing, and unstable from the dists directory before I could get past the choose-mirror stage. After that it seems like the required modules to detect the disk drives are not loaded properly. I get to the disk-detect portion of the install and then I receive the error:

"No disk drive was detected. If you know the name of the driver you can select it from the list"

None of the options in the list allow the install to finish. What is strange is if I install using the physical DVD (same files on the webserver minus the preseed.cfg), the install succeeds just fine, even without being connected to any network.

I don't know if the problem is my preseed file, the fact that I am using the files from the DVD instead of a mirror, or perhaps something else.

I would love to provide the syslog info, but I do not have ftp, scp, or anything else available during this point in the install. I cannot even mount any disks like a flash drive to copy over the syslog...

Here is my preseed file:
----------------------
#### Contents of the preconfiguration file (for lenny)
# Locale sets language and country.
d-i debian-installer/locale string en_US
d-i console-setup/layoutcode string us
d-i console-setup/ask_detect boolean false

# Keyboard selection.
#d-i console-tools/archs select at
d-i console-keymaps-at/keymap select us

### Network configuration
# netcfg will choose an interface that has link if possible. This makes it
# skip displaying a list if there is more than one interface.
d-i netcfg/choose_interface select eth0

# Any hostname and domain names assigned from dhcp take precedence over
# values set here. However, setting the values still prevents the questions
# from being shown, even if values come from dhcp.
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_domain string unassigned-domain

# If non-free firmware is needed for the network or other hardware, you can
# configure the installer to always try to load it, without prompting. Or
# change to false to disable asking.
d-i hw-detect/load_firmware boolean true

### Mirror settings
# If you select ftp, the mirror/country string does not need to be set.
#d-i mirror/protocol string ftp
d-i mirror/country string manual
d-i mirror/http/hostname string 172.16.76.2
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string

# Suite to install.
d-i mirror/suite string lenny
# Suite to use for loading installer components (optional).
d-i mirror/udeb/suite string lenny
### Clock and time zone setup
# Controls whether or not the hardware clock is set to UTC.
d-i clock-setup/utc boolean true

# You may set this to any valid setting for $TZ; see the contents of
# /usr/share/zoneinfo/ for valid values.
d-i time/zone string US/Pacific

# Controls whether to use NTP to set the clock during the install
d-i clock-setup/ntp boolean false

### Partitioning
# Alternatively, you can specify a disk to partition. The device name must
# be given in traditional non-devfs format.
# Note: A disk must be specified, unless the system has only one disk.
# For example, to use the first SCSI/SATA hard disk:
d-i partman-auto/disk string /dev/sda
# In addition, you'll need to specify the method to use.
# The presently available methods are: "regular", "lvm" and "crypto"
d-i partman-auto/method string regular

# If one of the disks that are going to be automatically partitioned
# contains an old LVM configuration, the user will normally receive a
# warning. This can be preseeded away...
d-i partman-lvm/device_remove_lvm boolean true
# The same applies to pre-existing software RAID array:
d-i partman-md/device_remove_md boolean true
# And the same goes for the confirmation to write the lvm partitions.
d-i partman-lvm/confirm boolean true

# If not, you can put an entire recipe into the preconfiguration file in one
# (logical) line. This example creates a small /boot partition, suitable
# swap, and uses the rest of the space for the root partition:
d-i partman-auto/expert_recipe string \
boot-root :: \
100 120 200 ext3 \
$primary{ } $bootable{ } \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext3 } \
mountpoint{ /boot } \
. \
500 10000 1000000000 ext3 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext3 } \
mountpoint{ / } \
. \
64 512 300% linux-swap \
method{ swap } format{ } \
.

# This makes partman automatically partition without confirmation, provided
# that you told it what to do using one of the methods above.
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true

### Base system installation
# Select the initramfs generator used to generate the initrd for 2.6 kernels.
#d-i base-installer/kernel/linux/initramfs-generators string yaird

# The kernel image (meta) package to be installed; "none" can be used if no
# kernel is to be installed.
#d-i base-installer/kernel/image string linux-image-2.6-486

### Account setup
# Skip creation of a root account (normal user account will be able to
# use sudo).
d-i passwd/root-login boolean true
# Alternatively, to skip creation of a normal user account.
d-i passwd/make-user boolean false

# Root password, either in clear text
d-i passwd/root-password password *********
d-i passwd/root-password-again password *********
# or encrypted using an MD5 hash.
#d-i passwd/root-password-crypted password [MD5 hash]

### Apt setup
# You can choose to install non-free and contrib software.
d-i apt-setup/non-free boolean true
#d-i apt-setup/contrib boolean true
# Uncomment this if you don't want to use a network mirror.
#d-i apt-setup/use_mirror boolean false
# Select which update services to use; define the mirrors to be used.
# Values shown below are the normal defaults.
d-i apt-setup/services-select multiselect security, volatile
d-i apt-setup/security_host string security.debian.org
d-i apt-setup/volatile_host string volatile.debian.org

# By default the installer requires that repositories be authenticated
# using a known gpg key. This setting can be used to disable that
# authentication. Warning: Insecure, not recommended.
d-i debian-installer/allow_unauthenticated string true

### Package selection
#tasksel tasksel/first multiselect standard, web-server
# If the desktop task is selected, install the kde and xfce desktops
# instead of the default gnome desktop.
#tasksel tasksel/desktop multiselect kde, xfce
tasksel tasksel/first multiselect standard

# Individual additional packages to install
d-i pkgsel/include string openssh-server
# Whether to upgrade packages after debootstrap.
# Allowed values: none, safe-upgrade, full-upgrade
d-i pkgsel/upgrade select none
d-i pkgsel/update-policy select none

# This is fairly safe to set, it makes grub install automatically to the MBR
# if no other operating system is detected on the machine.
d-i grub-installer/only_debian boolean true

# This one makes grub-installer install to the MBR if it also finds some other
# OS, which is less safe as it might not be able to boot that other OS.
d-i grub-installer/with_other_os boolean true

# Avoid that last message about the install being complete.
d-i finish-install/reboot_in_progress note
----------------------

Here are my PXE kernel options:
----------------------
label Debian 5 i386 UNATTENDED
menu label ^Debian 5 i386 UNATTENDED
kernel Linux/debian/5/i386/linux
append initrd=Linux/debian/5/i386/initrd.gz debian-installer/allow_unauthenticated=true debian-installer/country=US console-keymaps-at/keymap=us debian-installer/language=us debian-installer/locale=en_US console-setup/layoutcode=us netcfg/choose_interface=eth0 netcfg/nameserver=172.16.76.1 netcfg/get_domain=unassigned-domain netcfg/get_hostname=unassigned-hostname text preseed/url=http://172.16.76.1/debian/5/i386/preseed.cfg -- quiet
----------------------

Any troubleshooting advice would be appreciated.

Thank you,
thefiend

[llivv]

What docs are you reading?

[thefiend]

Hello,

Thank you for your reply. That's the thing.I can't find any documentation on how to install with DVD files on a web server and netbooting. I have read plenty docs and used trial and error to get this far. So, mostly:

http://www.debian.org/releases/stable/i386/apb.html.en

and

http://www.debian.org/releases/stable/i ... 05.html.en

The example preseed files I have found online always show ftp.us.debian.org or some other public mirror as the hostname, but I am trying to use a local area network webserver with only the DVD files on it.

Regards,
thefiend

[llivv]

http://www.debian.org/releases/stable/i386/apb.html.en
and
http://www.debian.org/releases/stable/i ... 05.html.en
The example preseed files I have found online always show ftp.us.debian.org or some other public mirror as the hostname, but I am trying to use a local area network webserver with only the DVD files on it.

It may take you a while to find what you need or you may get lucky.... take a look here
http://www.debian.org/releases/stable/i ... 05.html.en

[thefiend]

Hello,

RE: "It may take you a while to find what you need or you may get lucky.... take a look here
http://www.debian.org/releases/stable/i ... 05.html.en"

Can you please be little more descriptive in your reply? I have already read that document thoroughly and it does not exactly apply to my situation. I already have tftp setup with syslinux for remotely installing CentOS, Ubuntu, Fedora and RHEL, which all work flawlessly. All I used that document for was to find out which boot files I needed to add to the tftpboot directory for debian:

"For PXE booting, everything you should need is set up in the netboot/netboot.tar.gz tarball. Simply extract this tarball into the tftpd boot image directory."

I grabbed the linux and initrd.gz files from the netboot.tar.gz and placed them here within the tftpd root directory:

Linux/debian/5/i386/linux
Linux/debian/5/i386/initrd.gz

Are you saying it is possible to install Debian using netboot images and the files from the DVD on the webserver and I am just not reading "How to prepare files for tftpd booting" section correctly? If so, I already have the file prepared for tftp booting and the installer boots fine.

If it is possible to netboot install debian with a preseed file declaring a local webserver with the DVD files on it, then do you see anything wrong with my preseed or kernel arguments?

or do you have any links to documentation that I haven't listed that I already read and are more specific to my case?

Regards,
thefiend

[llivv]

or do you have any links to documentation that I haven't listed that I already read and are more specific to my case?

yes, but
what do you think about his before you get in deeper
from http://www.debian.org/releases/stable/i386/ch04s05.html.en

Next, place the TFTP boot image you need, as found in Section 4.2.1, “Where to Find Installation Images”, in the tftpd boot image directory. You may have to make a link from that file to the file which tftpd will use for booting a particular client. Unfortunately, the file name is determined by the TFTP client, and there are no strong standards.

For PXE booting, everything you should need is set up in the netboot/netboot.tar.gz tarball. Simply extract this tarball into the tftpd boot image directory. Make sure your dhcp server is configured to pass pxelinux.0 to tftpd as the filename to boot.

[7] All in.tftpd alternatives available in Debian should log TFTP requests to the system logs by default. Some of them support a -v argument to increase verbosity. It is recommended to check these log messages in case of boot problems as they are a good starting point for diagnosing the cause of errors.

[llivv]

I don't use PXE for local network installs!

And I don't know nothing about:

CentOS, Ubuntu, Fedora and RHEL, which all work flawlessly

for you.....

But when the installer doesn't connect to the installation image
it typically means one of two or three things...
This page http://wiki.debian.org/Syslinux says the cd uses syslinux isolinux and to netboot you need pxelinux ...

[thefiend]

I think maybe my post is too confusing. The issue is not with booting using PXE or tftp. The install image boots fine. The problem is during the installation the necessary modules to detect the hard drive are not found. So, I PXE boot the client using pxelinux, and the tftpd server sends the install image, the install image loads on the client machine and the installer starts. From there the preseed file is downloaded. Everything goes smoothly after that until the hardware detect stage.

As I mentioned I think this has something to do with the fact that I am not using a full local mirror, but just the files on the first install DVD extracted to a webserver:

### Mirror settings
# If you select ftp, the mirror/country string does not need to be set.
#d-i mirror/protocol string ftp
d-i mirror/country string manual
d-i mirror/http/hostname string 172.16.76.2
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string
###########################

I can view and download all files fine from the webserver (i.e. browsing too http://172.16.76.2/debian).

FYI, I have read through all the documentation for netbooting and preseeding, and searched google extensively for an answer. So, you guys are really my only hope.

Regards,
Darren

[maynim]

Hi thefiend,

I was trying to get this exact setup and I've been tearing my hair out, turns out the solution is quite simple:

The initrd for the netboot image does not contain any ide or sata drivers and they are meant to be retrieved during a regular install, I'm not sure exactly whether they are just not present on the DVD or the installer does not detect or expect them if you are using a mirror of the DVD ( or any install cd ). However there is a solution and that is to make your own initrd.gz with the drivers present.

I've adapted here from
http://wiki.openvz.org/Modifying_initrd_image
http://ubuntuforums.org/archive/index.php/t-423963.html

First I got the initrd.gz for the netboot and off the netboot install cd, but I think the DVD initrd.gz should be fine then:

Code: Select all

mkdir netboot-initrd-dir
mkdir cd-initrd-dir
gunzip netboot-initrd
gunzip cd-initrd
cd netboot-initrd-dir
cpio -i < ../netboot-initrd
cd ../cd-initrd-dir
cpio -i < ../cd-initrd

Now you will have two directories with the contents of both inird.gz files, you can see what driver files are in each one by:

Code: Select all

find ./netboot-initrd-dir -iname \*.ko
find ./cd-initrd-dir -iname \*.ko

you should notice that there is a lot more from the one on the cd, in particular the ide and sata drivers. You can probably be more precise with this is you need to have a smaller image, but I managed to get away with:

Code: Select all

cp -nr cd-initrd-dir/lib/modules/2.6.32-5-486/kernel/* netboot-initrd-dir/lib/modules/2.6.32-5-486/kernel/

now you just have to put it back together with:

Code: Select all

cd netboot-initrd-dir
find . | cpio -H newc -o > ../new-netboot-initrd
cd ../
gzip ./new-netboot-initrd

Now you should be able to take that file and stick it in your tftp directory under initrd.gz or similar and it will now detect your disks!
Hope that helps you out.

[ppcuser]

The post made by Maynim on 3/5/11 is correct, that is the same problem that I found; moreover, I found the same problem in Debian 5's for the PowerPC netboot CD (see viewtopic.php?f=17&t=56172&p=337326&hilit=ppcuser#p331826 ). Maynim did a very good job in tracking down the problem and fixing it --- good job.

Given Debian's reputation of being well tested, stable, etc. it amazes me that the same exact serious defect happend for Intel based PC and the PowerPC version. I could understand the defect happening by itself for PowerPCs as they are old history and it really doesn't matter anymore.

The other thing that has amazed me on this issue and in looking at some, a significant number but not all, of the so called "experts" who are suppose to be helping noobies is that these "experts" really are lacking --- many seem to be doing nothing more than repeating (in an arrogantly condescending tone to the noobies) what the Debian documentation says without really knowing what they are talking about! They pontificate on subjects they have not really any expertise on, have not really studied or checked out.

[llivv]

The post made by Maynim on 3/5/11 is correct, that is the same problem that I found; moreover, I found the same problem in Debian 5's for the PowerPC netboot CD (see viewtopic.php?f=17&t=56172&p=337326&hilit=ppcuser#p331826 ). Maynim did a very good job in tracking down the problem and fixing it --- good job.

Given Debian's reputation of being well tested, stable, etc. it amazes me that the same exact serious defect happend for Intel based PC and the PowerPC version. I could understand the defect happening by itself for PowerPCs as they are old history and it really doesn't matter anymore.

The other thing that has amazed me on this issue and in looking at some, a significant number but not all, of the so called "experts" who are suppose to be helping noobies is that these "experts" really are lacking --- many seem to be doing nothing more than repeating (in an arrogantly condescending tone to the noobies) what the Debian documentation says without really knowing what they are talking about! They pontificate on subjects they have not really any expertise on, have not really studied or checked out.

Calling all "so called "experts"" that pontificate to the noobs.
Stop wasting this forums electrons.. or else...

[gmjs]

Thank you so much maynim. I'd given up on this one as a bad job (just works with the provided netboot files on the Ubuntu alternate install disk) but now Ubuntu sports its own interface by default, I wanted to switch to a Debian installation for teaching purposes. I had just got to realising that I needed to modify the initrd, but I had no idea how to do it.

Thanks again--yours is the first response I've seen that hasn't just explained why I shouldn't be trying to do the install this way!