Kernel compile and install for newbies

[dbbolton]

Has anyone tried to build any of the 3.0 release candidates?

After seeing this post I downloaded and built rc5. Running it right now. No issues - built it the same way as I always do.

Not surprised; it's just an arbitrary version-numbering change:

So what are the big changes?

NOTHING. Absolutely nothing.

https://lkml.org/lkml/2011/5/29/204

[stevepusser]

Another way to build a kernel can be used with the liqourix kernel sources--those on testing or unstable can modify the kernel configuration file in the /debian folder as desired, then build debs with the standard package building commands; i.e. debuild -B -j5 for a four-core machine. Note that the liquorix kernel is using the quilt source format, which means that the Zen and other patches would be applied when extracting with "apt-get source" or manually with "dpkg-source -x", so you would be configuring an already patched kernel.

To get unpatched source, just extract the .orig tarball and the debian tar.gz and merge them. You are then free to disable any patches (even the Zen) or add your own new patches in /debian/patches and debian/patches/series.

Currently, Squeeze users will have to grep or otherwise search the /debian folder for the text string "gcc-4.5", then replace all instances with gcc-4.4, before installing build-depends and building the deb packages.

[milomak]

my suspicion is that the build fails because i am using liquorix as oldconfig when building the release candidates.

[Ibidem]

@stevepusser:

Code: Select all

grep -lr gcc-4.5 ./debian/|xargs sed -e 's/gcc-4.5/gcc-4.4/g' -i

(Quicker method)

My own approach (how I got 3.0.4) to building:

Code: Select all

make localmodconfig
make nconfig #A curses interface, the fastest way to change anything
                      #Remove what you can here and enable new drivers
                      #I'd use an xz image, which must be set in .config
make -j3       #On a dual-core
make -j3 deb-pkg   #And upstream has a builtin way to build Debian packages
cd ..; sudo dpkg -i linux*<version>*.deb

[acimmarusti]

Hey guys,

Ever since kernel.org went down I haven't been able to get the vanilla stable kernels 3.0.x directly.
I've been looking around a bit, but have found nothing. Furthermore, debian package changelog in unstable shows they have incorporated changes of 3.0.5 and 3.0.6 (while the last change listed on kernel.org is 3.0.4)

However, I can't seem to find where these vanilla sources are? Do you guys know?
I know you can get 3.1 from github, but I'm not willing to go so far!

Thanks

Andres

EDIT: sorry, I suck at finding things on the internet. For people interested, here is the git repo for the stable 3.0.x kernels:
http://git.kernel.org/?p=linux/kernel/g ... ;a=summary

[stevepusser]

You can also get vanilla source for a release from the liquorix repo. Note that this is something like 3.0.0-0 source, for example, you will need to apply the point patch to bring it up to something like 3.0.6 vanilla. The Liqourix patch includes these point patches along with lots of other, more exotic , patches.


http://liquorix.net/debian/pool/main/l/linux-liquorix/

[acimmarusti]

You can also get vanilla source for a release from the liquorix repo. Note that this is something like 3.0.0-0 source, for example, you will need to apply the point patch to bring it up to something like 3.0.6 vanilla. The Liqourix patch includes these point patches along with lots of other, more exotic , patches.


http://liquorix.net/debian/pool/main/l/linux-liquorix/

Yes I've used this before as well. It's really good for desktop applications, although it's hard to see a difference between a liquorix kernel and a mainline one (modified for performance) on a very new computer.

Thanks

[acimmarusti]

Do you guys know how to compile a kernel with all the firmware (non-free and free) built-in?

I tried building the kernel activating the option to include the firmware blobs in the kernel, but when I boot this kernel it still cannot bring my wifi card on.

My wireless card only became supported with 3.0.x but I don't want to rely on the firmware shipped by debian.

[Ibidem]

Do you guys know how to compile a kernel with all the firmware (non-free and free) built-in?

I tried building the kernel activating the option to include the firmware blobs in the kernel, but when I boot this kernel it still cannot bring my wifi card on.

My wireless card only became supported with 3.0.x but I don't want to rely on the firmware shipped by debian.

Troubleshooting:

Code: Select all

lsmod
modinfo <drivername>
dmesg|grep drivername

I would not recommend building wireless in--"modprobe -r ; modprobe" often beats having to reboot.

Do you have a RTL819x card?

Anyhow, here are the lines I have that seem relevant to the issue:

Code: Select all

# CONFIG_PREVENT_FIRMWARE_BUILD is not set
CONFIG_FIRMWARE_IN_KERNEL=y
CONFIG_EXTRA_FIRMWARE=""

Debian defaults to CONFIG_PREVENT_FIRMWARE_BUILD=y .
You would need to disable this to build the nonfree firmware.

Run "make deb-pkg" to build the kernel--it will give you about 4 packages (kernel, firmware, headers, and linux-libc-dev). Then I use the resulting firmware package from the compile instead of some unmatching Debianized package that can't be counted on to include any firmware for the driver, let alone have the right version.
I'm not sure if make deb-pkg/other ways of generating debs override some .config options.

[acimmarusti]

Troubleshooting:

Code: Select all

lsmod
modinfo <drivername>
dmesg|grep drivername

I would not recommend building wireless in--"modprobe -r ; modprobe" often beats having to reboot.

Yeah I guess your right about this

Do you have a RTL819x card?

Yes I do. My card is:

Code: Select all

rtl8192se
PCI: 10EC:8172 Realtek Semiconductor Co., Ltd. RTL8191SEvB Wireless LAN Controller

Anyhow, here are the lines I have that seem relevant to the issue:

Code: Select all

# CONFIG_PREVENT_FIRMWARE_BUILD is not set
CONFIG_FIRMWARE_IN_KERNEL=y
CONFIG_EXTRA_FIRMWARE=""

Debian defaults to CONFIG_PREVENT_FIRMWARE_BUILD=y .
You would need to disable this to build the nonfree firmware.

Run "make deb-pkg" to build the kernel--it will give you about 4 packages (kernel, firmware, headers, and linux-libc-dev). Then I use the resulting firmware package from the compile instead of some unmatching Debianized package that can't be counted on to include any firmware for the driver, let alone have the right version.
I'm not sure if make deb-pkg/other ways of generating debs override some .config options.

I see. I normally use 'make-kpkg' to build my customized kernels. Would that also build the extra firmware packages? I still follow the procedure outlined at the beginning of this thread to build the kernels, basically. Is there a newer better way using 'make deb-pkg' ?

Thanks, I will give this a try, though for the moment, the firmware from backports is working well, but I would prefer to try the vanilla one.

[the dsc]

How to apply one of those official debian patches to the kernel sources? I don't get it. It seems that "non-debian" patches would be in some .gz file, and you would apply by symlinking it to the root of the kernel source, then using the command patch, whatever is the syntax.

These patches I'm talking about, however, are installed at /usr/src/kernel-patches/all/3.0.0/.

This one specifically is named just "debian":

http://packages.debian.org/wheezy/linux ... bian-3.0.0

If I just leave it there and keep following some kernel compilation tutorial it will be applied "by itself" at some point, or there's something I need to do?

[stevepusser]

It depends on how the kernel is built. Debian kernels use the standard Debian package build tools, following instructions inside a /debian folder that is added to the original source. The basic set of instructions is in /debian/rules, but from what I've seen for the kernel, it's very complicated. Policy for patches has them inside the /debian/patches folder.

However, for a manual build of the kernel, just apply the patche(s) with the patch command. They don't have to be in the kernel source folder if the full path of the patch is specified.

[acimmarusti]

Run "make deb-pkg" to build the kernel--it will give you about 4 packages (kernel, firmware, headers, and linux-libc-dev). Then I use the resulting firmware package from the compile instead of some unmatching Debianized package that can't be counted on to include any firmware for the driver, let alone have the right version.
I'm not sure if make deb-pkg/other ways of generating debs override some .config options.

I looked at the Debian kernel handbook and I saw that now, the recommended method of building your own kernel packages is using 'make deb-pkg' instead of 'make-kpkg', however using both methods, with the same configuration file, leads to disproportionate package sizes!

With 'make-kpkg' I would usually get a size of kernel installation (as reported by synaptic) of about 116 Mb (for 3.0.7)

But now with 'make deb-pkg' I get 1645 Mb!! which is just CRAZY!

I read here: http://forums.debian.net/viewtopic.php?f=5&t=62969, that this is supposed to happen if I enable debugging symbols in the kernel configuration, but like I said before, I didn't change the config file at all!

As a result of this, the kernel takes noticeably longer to boot! How to you circumvent this problem?

[Ibidem]

I looked at the Debian kernel handbook and I saw that now, the recommended method of building your own kernel packages is using 'make deb-pkg' instead of 'make-kpkg', however using both methods, with the same configuration file, leads to disproportionate package sizes!

With 'make-kpkg' I would usually get a size of kernel installation (as reported by synaptic) of about 116 Mb (for 3.0.7)

But now with 'make deb-pkg' I get 1645 Mb!! which is just CRAZY!

I read here: http://forums.debian.net/viewtopic.php?f=5&t=62969, that this is supposed to happen if I enable debugging symbols in the kernel configuration, but like I said before, I didn't change the config file at all!

As a result of this, the kernel takes noticeably longer to boot! How to you circumvent this problem?

That's the first problem. If you build a custom kernel, why use the default config? That's not meant to provide what you need--it's meant to run on every x86 computer made since around 1995,provided it has enough ram, and support every peripheral known to the kernel developers.
I would strongly recommend that you do "make localmodconfig" and then configure with make nconfig ( or menuconfig/xconfig/gconfig ).
I'm getting install sizes of 9-12 MB, with wireless, KVM, and all. (9 MB when I use XZ compression, 12 MB when I use LZO compression--which is the lowest compression available)
Build times are well under an hour (maybe 30-40 minutes max, without any parallelization). I'm pretty sure a full default kernel would take over 3 hrs to build here.

[acimmarusti]

That's the first problem. If you build a custom kernel, why use the default config? That's not meant to provide what you need--it's meant to run on every x86 computer made since around 1995,provided it has enough ram, and support every peripheral known to the kernel developers.
I would strongly recommend that you do "make localmodconfig" and then configure with make nconfig ( or menuconfig/xconfig/gconfig ).
I'm getting install sizes of 9-12 MB, with wireless, KVM, and all. (9 MB when I use XZ compression, 12 MB when I use LZO compression--which is the lowest compression available)
Build times are well under an hour (maybe 30-40 minutes max, without any parallelization). I'm pretty sure a full default kernel would take over 3 hrs to build here.

Thanks for your response. Well I found out the problem I was having. My kernel images are still about 3 times larger than yours, but they are of the same size as the default debian amd64 kernel image.

The problem is that you have to turn off the kernel option CONFIG_DEBUG_INFO. Which is located under the Kernel hacking menu (if you use make menuconfig / xconfig /gconfig). Curiously, the other method of creating deb packages, strips this debug info out of the kernel image. I guess this eventually becomes the usual debianized linux-*-dbg package if you want it that way.

I'm interested about this localmodconfig option. I realize I can do better at customizing my kernels, but I'm just paranoid about not having a driver for some bizarre peripheral device that I could potentially hook up to my laptop / computer.

Andres

[vbrummond]

I never saw any performance benefit to use make localmodconfig. All it does is detect what modules your system is currently using and disable all (except what you need) of the options (already) marked as a kernel module. So basically it makes your kernel into a monolithic binary. It might save some space or compile time; However if you ever add a new piece of hardware that requires a missing module you will need to recompile a new kernel.

[Ibidem]

I never saw any performance benefit to use make localmodconfig. All it does is detect what modules your system is currently using and disable all (except what you need) of the options (already) marked as a kernel module. So basically it makes your kernel into a monolithic binary. It might save some space or compile time; However if you ever add a new piece of hardware that requires a missing module you will need to recompile a new kernel.

That's all that it does. But the point is to start there, so you won't have to spend 2 hours building the kernel, and have 100+ MB used for the kernel.

Since it only disables modules, you can reconfigure the sources later to re-enable a driver, make modules, and install the modules (I've done this at least a couple times, it's not just speculation).
Then you do the tuning:
check if you need ISA (lspci|grep ISA)--some boards use it for odd purposes;
disable MCA (you almost certainly don't need it);
tune for the lowest CPU you'll use (eg, i586, i686, k{6,7,8}, Core whatever...); select your general options; etc.
If you messed up, you wasted maybe 1-2 hours, not 4-6 hours like you might otherwise.

If you're paranoid about "What if I end up needing to test a DEC network card or find an MGA card for a spare console on my desktop, or get given some Infiniband card?", it's not for you. But otherwise, you can add back the useful stuff faster than you'd remove anything.

[acimmarusti]

make-kpkg is very well documented. It has a very helpful manpage, etc.

Is there any similar documentation regarding 'make deb-pkg' ? I cannot seem to find anything...

Also though I have been using 'make deb-pkg' as of late I have several grievances with it (apart from the apparent lack of documentation). The firmware package gets called "linux-firmware-x.x.x.deb". This package conflicts with the debianized firmware-linux, firmware-linux-free and nonfree. This is annoying, is there a way to change the names? (that's why I'm asking about that documentation).

Recently, nvidia-kernel-dkms from backports failed to compile and install the module on one of my computers running a 3.0.18 kernel customized and compiled by myself with "make deb-pkg". It failed saying that there were no sources for the kernel image (even though kernel headers were installed). Curiously module-assistant had no trouble at all compiling the nvidia module for the kernel... kernels from backports had no trouble with dkms though.

On the other hand make-kpkg doesn't seem to even offer the option of the separate firmware package.

Anyone encountered these problems and found documentation / solutions?

Thanks

[Ibidem]

make-kpkg is very well documented. It has a very helpful manpage, etc.

Is there any similar documentation regarding 'make deb-pkg' ? I cannot seem to find anything...

Also though I have been using 'make deb-pkg' as of late I have several grievances with it (apart from the apparent lack of documentation). The firmware package gets called "linux-firmware-x.x.x.deb". This package conflicts with the debianized firmware-linux, firmware-linux-free and nonfree. This is annoying, is there a way to change the names? (that's why I'm asking about that documentation).

Recently, nvidia-kernel-dkms from backports failed to compile and install the module on one of my computers running a 3.0.18 kernel customized and compiled by myself with "make deb-pkg". It failed saying that there were no sources for the kernel image (even though kernel headers were installed). Curiously module-assistant had no trouble at all compiling the nvidia module for the kernel... kernels from backports had no trouble with dkms though.

On the other hand make-kpkg doesn't seem to even offer the option of the separate firmware package.

Anyone encountered these problems and found documentation / solutions?

Thanks

linux-firmware-x.x.x contains the same files as firmware-lnux,firmware-linux-free, & firmware-linux-nonfree.
Therefore, it must always conflict. You only need one of the options, though.

dkms:
The default links (/lib/modules/`uname -r`/{build,source} point to the original build directory.
This is the one problem with this method...
I wrote a script to handle this.

Code: Select all

#!/bin/sh
#fixkernlink
usage(){
cat <<EOF
kernlinks.sh--clean up /lib/modules/<kernel-release> for building modules.
USAGE:
kernlinks.sh <kernelversion>
cd /lib/modules;for K in * ; do kernlinks.sh $K; done; cd -
EOF
}
if [ ! -d /lib/modules/$1 ]
        then usage; exit 255
fi
#If build is not a link, there are no headers or it's not a kernel version
test -L /lib/modules/$1/build||exit 127 
cd /lib/modules/$1/
export KERNVER=`pwd`; export KERNVER=`basename ${KERNVER}`
#Are headers installed?
export KERNHDR=/usr/src/linux-headers-${KERNVER}
test -d ${KERNHDR} || exit 126
unlink build
unlink source
ln -s ${KERNHDR} build
ln -s ${KERNHDR} source
cd ${KERNHDR}/arch/x86
#Makefile_32.cpu -- for OSS4
test -e Makefile_32.cpu || cp ../../../all/arch/x86/Makefile_32.cpu ./
exit 0

Note that all/arch/x86/Makefile_32.cpu was copied (cp --parents) from the build directory to /usr/src

[javidemon95]

Hello. Wireless don't work with my compiled kernel. What can I do?