Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Kernel compile and install for newbies

Share your HowTo, Documentation, Tips and Tricks. Not for support questions!.
Message
Author
User avatar
Rolling Stone
Posts: 366
Joined: 2009-02-15 18:55
Location: Turku, Finland

Kernel compile and install for newbies

#1 Post by Rolling Stone »

EDIT: This thread was originally written for Debian 5.0 but this method works fine with 6.0. The first 5 pages of discussion deal with 5.0.

I just compiled my first kernel and this is what I did. This guide has since been updated following the recommendations by the good people in this thread. Kernel patching and building additional modules are not covered.

The default kernel that comes with Debian is better suited for servers. It's optimized for high thoughput and low interactivity. When you compile your own, you can among other things optimize it for the processor(s) you have. You can also remove any and all unneeded bloat, there's plenty of it to make the default kernel work on maximum number of system configurations out of the box. Compiling your own is fun, profitable and easy.

Basically it's a 4 step process
* Get the source
* Configure it
* Compile it
* Install it

So, here we go, install needed packages

Code: Select all

sudo apt-get install kernel-package ncurses-dev bzip2 module-init-tools initramfs-tools procps fakeroot
If you plan on using the graphical configuration tools, gconfig or xconfig, you'll need additional packages, respectively libgtk2.0-dev libglib2.0-dev libglade2-dev for gconfig or libqt3-mt-dev for xconfig. There might be other dependencies too and if they're not satisfied you'll be notified once you try to invoke the graphical configurator.

To see what kernel you're now on, input

Code: Select all

uname -r
Acquire a source, e.g.

Code: Select all

sudo apt-get install linux-source-2.6.26
(New Linux versions keep pouring out, this is somewhat dated version by now...)

Make a user-writeable directory to avoid having to compile as root

Code: Select all

mkdir ~/kernel_compiling/
Copy the source to that folder

Code: Select all

cp /usr/src/linux-source-2.6.26.tar.bz2 ~/kernel_compiling/
Goto the directory

Code: Select all

cd ~/kernel_compiling/
Extract the source

Code: Select all

tar xjf linux-source-2.6.26.tar.bz2
Enter the extracted directory

Code: Select all

cd linux-source-2.6.26/
(At this point you would apply any desired patches.)

(You might optionally want to copy a working template for the kernel .config to this directory, you can find some in /boot/.)
(If you're unsure whether you should do it, I recommend you do.)

Code: Select all

cp /boot/config-$(uname -r) ./.config
To see what kind of a processor you have, input

Code: Select all

cat /proc/cpuinfo
Configure the kernel options, there's more than one way to skin a cat, pick one, e.g.

Code: Select all

make config
Image

Code: Select all

make menuconfig
Image

Code: Select all

make xconfig
Image

Code: Select all

make gconfig
Image

I use GNOME so it felt natural for me to choose gconfig. It is pretty clumsy and unintuitive, for example clicking on the actual tick box doesn't do anything, instead you'll have to click on the Y/M/N letter in the very last column... Also, there seem to be undocumented features? Anyway, you can use Ctrl+F to search, Ctrl+G to find next occurrence and Shift+Ctrl+G to find previous occurrence. Quite handy after hitting the expand button. I find xconfig visually uglier but it has a much better search function. (Also activated by Ctrl+F.)

Y means integrated into the kernel and (theoretically) always active
M means built as a module and activated if required
N means not included in any form

One thing you should do is give it a distinctive name, e.g. your initials in general > local version (MUST be lower case), I used the letters 'me' for this example. Start with changing only the obvious things, the ones you're 300% sure of. You will repeat this process many times and will get to know the more esoteric ones... If this is your first time, don't be too bold! Once you're done, save changes and exit.

Clean the slate for kernel compiling

Code: Select all

make-kpkg clean
If you have a multiprocessor machine, you'll want to make all processors work, do e.g.

Code: Select all

export CONCURRENCY_LEVEL=2
(Documentation suggests using "a (small) integer".)

Perform the actual compile, this will take anywhere from 10 minutes to 5 hours depending on your hardware and configuration choices (the more you selected, the longer it will take). You need to include kernel_headers if you want to compile some other modules against your kernel.

Code: Select all

make-kpkg --rootcmd fakeroot --initrd --revision=custom.001 kernel_image kernel_headers
It then will give you all kind of chatter, warnings and notices during, it's quite normal. Only errors are serious, as in the process will halt. If you get an error about missing zlib.h, you need to install the zlib1g-dev package.

Once it's done through with no errors, goto the lower directory where the .deb we just created is

Code: Select all

cd ..
Install the .deb package(s)

Code: Select all

sudo dpkg -i *.deb 
Then it's the moment of truth, the smoke test! Reboot and select from the list the new kernel with your initials or what ever ID you gave it at config time.

If it boots, nice, you made it, congrats!

Checklist
* network
* audio
* usb sticks
* cddvdrwcombo
* any queer peripherals

If not, reboot and select another kernel from the menu. Back to the drawing board...

You'll want to polish the kernel more, go back to the directory you extracted the source into. If everything worked out pretty nicely, make a backup of your .config. If it failed spectacularly, you might want to load a working .config and start working from it again.

Then do some more configuring, and cleanup, recompile... repeat until! :)

Hints and tips:

* Check out the time command to clock your compiles to see how long it takes.
* You might want to nice your compile, so you can use your computer while it is compiling.
* Try patching your kernel with something latest, e.g. http://tuxonice.net/
* Check out the Linux-Libre project for a blob free kernel. EDIT: No need anymore! (in Debian) :D
* I use xxdiff to compare different .config files.
* There's some good info available at http://kernelnewbies.org/FAQ
Last edited by Rolling Stone on 2012-06-27 06:04, edited 35 times in total.

User avatar
patrick
Posts: 35
Joined: 2007-06-10 14:57
Location: Holland Massachusetts U.S.A.
Contact:

#2 Post by patrick »

Thank you very much Rolling Stone. That is a nice howto. I like that gconfig option You are the first person that has even mentioned it. Well atleast that i have seen.
The biggest problem I have had with kernel compile is my Nvidia drivers.
I am far from noob but also far from guru. I have no fear of the cli. I really enjoy Debian Gnu/Linux. The real and only problem I have is making a kernel with my Nvidia drivers installed. I know it has been explained before, but that seemed more for the none free ones. I would like to use the Debian ones if at all possible.
What do I need to make the Nvidia card work after the kernel is built.
My card is:
nVidia Corporation GeForce 6150SE nForce 430 (rev a2)

Thank you very much for this new Howto.
Debian Lenny/Squeeze
Journeyman Plumber Massachusetts U.S.A.
If i can learn Linux any person can. I'm just a Plumber ya know.

infinitycircuit
Posts: 1137
Joined: 2007-07-24 03:31
Location: California

#3 Post by infinitycircuit »

I would recommend:

Code: Select all

make-kpkg --rootcmd fakeroot --initrd kernel_image kernel_headers
It will make the initrd automatically for you this way.

User avatar
patrick
Posts: 35
Joined: 2007-06-10 14:57
Location: Holland Massachusetts U.S.A.
Contact:

#4 Post by patrick »

Hey thank you infinitycircuit
I will give that a try
Debian Lenny/Squeeze
Journeyman Plumber Massachusetts U.S.A.
If i can learn Linux any person can. I'm just a Plumber ya know.

User avatar
Telemachus
Posts: 4574
Joined: 2006-12-25 15:53
Been thanked: 2 times

#5 Post by Telemachus »

My suggestions:
  • Don't do any of this as root (or with sudo), and don't do it in /usr/src
  • Create a directory in your regular user's home directory just for building kernels. Call it whatever you like; I call mine, rather boringly, kernels.
  • Download the vanilla kernel sources, or the Debianized kernel sources to this directory, untar it and cd into the new directory. Do all of this as a regular user with no special privileges.
  • After I cd into the directory for the kernel source, here's what a typical build sequence looks like for me.

    Code: Select all

    cp /boot/config-$(uname -r) ./.config
    make oldconfig
    make menuconfig
    export CONCURRENCY_LEVEL=2
    make-kpkg clean
    fakeroot make-kpkg --append-to-version "-greek-hero-name" --revision "datestamp" --initrd kernel_image kernel_headers
The CONCURRENCY_LEVEL is make-kpkg's preferred way of getting all the juice of out multiple processors. (Don't use -j settings directly, says the man page: it will cause the build to fail.)
Last edited by Telemachus on 2009-03-04 02:22, edited 1 time in total.
"We have not been faced with the need to satisfy someone else's requirements, and for this freedom we are grateful."
Dennis Ritchie and Ken Thompson, The UNIX Time-Sharing System

User avatar
mzilikazi
Forum Account
Forum Account
Posts: 3282
Joined: 2004-09-16 02:14
Location: Colorado Springs, CO

#6 Post by mzilikazi »

Also note that you shouldn't use sudo but instead use fakeroot. Also it is not necessary to build anything in /usr/src. This has been hashed over a good bit on this forum. I realize that by default your kernel source is installed in /usr/src when using apt-get/aptitude so all you need do (as $USER) is something like

Code: Select all

mkdir KERNEL
cd KERNEL
apt-get install linux-source-blah
unp /usr/src/linx-source-blah
This unpacks the kernel source in a directory writable by $USER. This way the only thing you need to be root for is

Code: Select all

 dpkg -i kernel-image-blah
If you'd like a bit more advanced kernel howto perhaps read this.

patrick - also read the thread linked above for building Nvidia the Debian way. It's simple and you can have the driver built before even booting into your new kernel. ;)
Debian Sid Laptops:
AMD Athlon(tm) 64 X2 Dual-Core Processor TK-55 / 1.5G
Intel(R) Pentium(R) Dual CPU T2390 @ 1.86GHz / 3G

User avatar
mzilikazi
Forum Account
Forum Account
Posts: 3282
Joined: 2004-09-16 02:14
Location: Colorado Springs, CO

#7 Post by mzilikazi »

Telemachus wrote: [*] After I cd into the directory for the kernel source, here's what a typical build sequence looks like for me.

Code: Select all

cp /boot/config-$(uname -r) ./.config
make oldconfig

Code: Select all

make oldconfig
is the same as

Code: Select all

cp /boot/config-$(uname -r) ./.config 

Code: Select all

~/KERNEL/linux-source-2.6.28$ make oldconfig
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/basic/docproc
  HOSTCC  scripts/basic/hash
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/kxgettext.o
  SHIPPED scripts/kconfig/zconf.tab.c
  SHIPPED scripts/kconfig/lex.zconf.c
  SHIPPED scripts/kconfig/zconf.hash.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/conf
scripts/kconfig/conf -o arch/x86/Kconfig
#
# using defaults found in /boot/config-2.6.28-7.slh.1-sidux-amd64
#
#
# configuration written to .config
Debian Sid Laptops:
AMD Athlon(tm) 64 X2 Dual-Core Processor TK-55 / 1.5G
Intel(R) Pentium(R) Dual CPU T2390 @ 1.86GHz / 3G

User avatar
Telemachus
Posts: 4574
Joined: 2006-12-25 15:53
Been thanked: 2 times

#8 Post by Telemachus »

mzilikazi wrote:
Telemachus wrote: [*] After I cd into the directory for the kernel source, here's what a typical build sequence looks like for me.

Code: Select all

cp /boot/config-$(uname -r) ./.config
make oldconfig

Code: Select all

make oldconfig
is the same as

Code: Select all

cp /boot/config-$(uname -r) ./.config 
Huh - go figure. I never realized that make oldconfig would automatically copy the current config. Thanks for letting me know. On the other hand, I still wouldn't say it's exactly the same, since make oldconfig will also prompt me to choose options for all new features of the newer source won't it? (Or am I misremembering that?)
"We have not been faced with the need to satisfy someone else's requirements, and for this freedom we are grateful."
Dennis Ritchie and Ken Thompson, The UNIX Time-Sharing System

User avatar
mzilikazi
Forum Account
Forum Account
Posts: 3282
Joined: 2004-09-16 02:14
Location: Colorado Springs, CO

#9 Post by mzilikazi »

Telemachus wrote: make oldconfig will also prompt me to choose options for all new features of the newer source won't it? (Or am I misremembering that?)
If memory serves correctly, what you're referring to is building your image without calling menuconfig. In this case you will be prompted for new options.
Debian Sid Laptops:
AMD Athlon(tm) 64 X2 Dual-Core Processor TK-55 / 1.5G
Intel(R) Pentium(R) Dual CPU T2390 @ 1.86GHz / 3G

User avatar
Rolling Stone
Posts: 366
Joined: 2009-02-15 18:55
Location: Turku, Finland

#10 Post by Rolling Stone »

Great discussion! I wish there was a wiki page we could use for this purpose. That way people who actually know what they're talking about could fix the obvious mistakes easier and the article would stay updated regardless of any one person. Since this is such an interesting subject and really everybody should compile a kernel, I secretly hoped to open the flood gates.

Are you guys familiar with http://wiki.debian.org/ ? Is it easy to get an account there and is it jealously guarded by pricks? I don't think that Wikipedia would either be a completely wrong place, of course, this is a bit specialized as it's on Debian, even though probably half of the worlds GNU/Linux users are on Debian, or Debian based systems.

For now, I'll try to update this post with all the suggestions. But not right now, but soon...
Last edited by Rolling Stone on 2009-03-07 09:29, edited 1 time in total.

User avatar
Telemachus
Posts: 4574
Joined: 2006-12-25 15:53
Been thanked: 2 times

#11 Post by Telemachus »

mzilikazi wrote:
Telemachus wrote: make oldconfig will also prompt me to choose options for all new features of the newer source won't it? (Or am I misremembering that?)
If memory serves correctly, what you're referring to is building your image without calling menuconfig. In this case you will be prompted for new options.
I'm pretty sure that's not what I was thinking of (though what you're describing may happen in that situation).

Here's what I meant: assume my current kernel is for 2.6.26.7, and I download the source for 2.6.28.1. When I run make oldconfig in that situation, I get prompted to say yes or no to any options that exist in the newer source (and which aren't set based on the old .config since they didn't exist then). I'm reasonably certain that happens, but I've been reasonably certain and completely wrong before. I can test it later today or tonight and confirm. Edit: quick follow-up. The make oldconfig does prompt you to say yes or no to new options. I just tested this with the latest rc version of the kernel.
Last edited by Telemachus on 2009-03-05 00:53, edited 1 time in total.
"We have not been faced with the need to satisfy someone else's requirements, and for this freedom we are grateful."
Dennis Ritchie and Ken Thompson, The UNIX Time-Sharing System

User avatar
Rolling Stone
Posts: 366
Joined: 2009-02-15 18:55
Location: Turku, Finland

#12 Post by Rolling Stone »

patrick, I'd love to help you but I'm afraid that's way outside my experience. Maybe somebody else knows how to do it.

infinitycircuit, to make that adjustment to the guide, I'd just have to replace the make-kpkg line with that one and add the fakeroot package to the things to install at the top of it, right? What are those kernel_headers and where are they used exactly? I don't need to use update-grub there do I since the initrd is now made by make-kpkg?

Telemachus, thanks for the input, I have just one processor but I'm sure those who have more will be happy to learn it. Is this multiprocessor only or is this also for multicores?

You can't have 2 kernels with the same name (as in uname -r), can you? (Despite them having separate revision number.) Perhaps I should point this out in the guide, if it indeed is so. EDIT: Actually you can have kernels "with the same name" as long as they have differing revision numbers.

I'll now proceed to edit the guide to heed to this new-to-me knowledge, please do comment if I got it wrong.
Last edited by Rolling Stone on 2010-10-02 09:55, edited 1 time in total.

User avatar
csaba
Posts: 127
Joined: 2007-08-02 05:30

#13 Post by csaba »

nice little howto you got here.

myself just compiled a 2.6.29rc kernel and my memory consumption right after bootup has been halved. really Nice indeed. Altough i still dont know what made such a drastic difference cause i used my old config file as a base for the compile and didnt really messed with the configuration.

User avatar
csaba
Posts: 127
Joined: 2007-08-02 05:30

#14 Post by csaba »

csaba wrote:my memory consumption right after bootup has been halved.
Oh yeah, that would be really cute and awesome. I'm taking this back before someone thinks anything good about my kernel compailing skills. :D
Only fglrx and some vmware modules didnt get loaded. Altough i still gained about 50megs less memory usage at startup.

User avatar
Rolling Stone
Posts: 366
Joined: 2009-02-15 18:55
Location: Turku, Finland

#15 Post by Rolling Stone »

Hehe, thanks for the feedback! ;)

I can't code so I have to come up with other ways of contributing back, besides seeding the torrent like crazy.

Olaf
Posts: 1
Joined: 2009-03-06 19:44

#16 Post by Olaf »

Can someone tell me whats going wrong?

After the compiling I get this lines at the end.

Code: Select all

make[1]: Leaving directory `/home/danne/new/linux-source-2.6.26'
/usr/bin/make    ARCH=i386 \
			     -C Documentation/lguest
make[1]: Entering directory `/home/danne/new/linux-source-2.6.26/Documentation/lguest'
cc -Wall -Wmissing-declarations -Wmissing-prototypes -O3 -I../../include    lguest.c  -lz -o lguest
lguest.c:34:18: error: zlib.h: Filen eller katalogen finns inte
make[1]: *** [lguest] Fel 1
make[1]: Leaving directory `/home/danne/new/linux-source-2.6.26/Documentation/lguest'
make: *** [debian/stamp/build/kernel] Fel 2
Filen eller katalogen finns inte = The file or directory doesnt exist
Fel = Wrong

User avatar
patrick
Posts: 35
Joined: 2007-06-10 14:57
Location: Holland Massachusetts U.S.A.
Contact:

#17 Post by patrick »

I just want to say thank you to all that have responded to my questions.

I did a system update and things went crazy on me. It is fine now. I also got real sick along with the rest of my house. :) I feel a little better now, but need to go and install a water heater for a customer $$ :)

Well anyways I will most likely do the kernel compile and the suggestions everybody has been so kind to offer tonight or tomorrow.

I did not want to be one of those people that asks for help and never gets back to say how things are going.

Thanks all for helping
Debian Lenny/Squeeze
Journeyman Plumber Massachusetts U.S.A.
If i can learn Linux any person can. I'm just a Plumber ya know.

infinitycircuit
Posts: 1137
Joined: 2007-07-24 03:31
Location: California

#18 Post by infinitycircuit »

Rolling Stone wrote:
infinitycircuit, to make that adjustment to the guide, I'd just have to replace the make-kpkg line with that one and add the fakeroot package to the things to install at the top of it, right? What are those kernel_headers and where are they used exactly? I don't need to use update-grub there do I since the initrd is now made by make-kpkg?

You can't have 2 kernels with the same name (as in uname -r), can you? (Despite them having separate revision number.) Perhaps I should point this out in the guide, if it indeed is so.

I'll now proceed to edit the guide to heed to this new-to-me knowledge, please do comment if I got it wrong.
Yeah, that's the way you would change it. kernel_headers include the C headers that are needed if you want to compile modules against the kernel (e.g., if you want to compile nvidia graphics drivers.) update-grub will be called as part of the postinst of dpkg when you install the .deb.

You can have 2 kernels with separate append-to-version's, but not with separate revision's, because make-kpkg's revision refers to the debian revision of the kernel.

User avatar
Rolling Stone
Posts: 366
Joined: 2009-02-15 18:55
Location: Turku, Finland

#19 Post by Rolling Stone »

Somebody really should write a proper graphical tool for configuring the kernel. To put it nicely, our current options are half arsed. :evil:

The search facilities range from quite lacking to non-existent... having such kludgey tools makes the simple excercise a PITA. Every hour spent writing a better tool would save years of collective Linux users' lives. Then again, I believe that (manual, guided or automagical) compiling of a custom kernel should be the last step of the debian-installer process.

/rant
infinitycircuit wrote:compile modules against the kernel (e.g., if you want to compile nvidia graphics drivers.)
I thought the nvidia driver was a binary blob? :x

Stian1979
Posts: 316
Joined: 2007-07-29 14:57

#20 Post by Stian1979 »

Telemachus wrote: [*] Download the vanilla kernel sources, or the Debianized kernel sources to this directory, untar it and cd into the new directory. Do all of this as a regular user with no special privileges.
Where do I find the vanilla kernel?

Post Reply