Kernel confusion clarification thread

[Lavene]

Lately some authors of Debian kernel howtos, my self included, are taking some flack because we use /usr/src for the kernel source. The reason for this is a paragraph in the README that follows the source that says:

Do NOT use the /usr/src/linux area! This area has a (usually incomplete) set of kernel headers that are used by the library header files. They should match the library, and not get messed up by whatever the kernel-du-jour happens to be.

Now, it's easy to miss, but in fact the README does not say you shouldn't use /usr/src. It does say not to use /usr/src/linux, a directory that is not present on Debian systems. Actually, on a default Debian system /usr/src is empty.

When you install a kernel-source via apt-get it is put automatically in /usr/src and when you unpack it it creates /usr/src/linux-source-2.6.x.x. Exactly the same as if you unpacked in, say, ~/kernel. It would then create ~/kernel/linux-source-2.6.x.x

The reason for using /usr/src is mostly that it is a convention, and also that it is the directory where apt actually places it. And although it's true that you can build the kernel wherever you want it does no harm doing it under /usr/src. By doing so you are creating a unique directory for your new kernel source anyway when you unpack it.

So in conclusion; I see no logical reason not to use /usr/src on a Debian system.

If someone can proove me wrong please do. It's really about time this issue gets settled. Also, if the above is wrong the Debian developers needs to change the behaviour of the source in the repositories.

Tina

[thamarok]

Thanks for posting this Tina. Actually, I didn't care what people said about /usr/src/.. I just compiled my kernels there and they work, what else should I watch for?
Btw, I know that someone posted this somewhere, but I can't find it: How can I recompile a kernel from scratch without xconfig or menuconfig to load my current config?

Thanks!

EDIT: Got my own question solved:
make defconfig was what I needed.

[Lux]

I believe that the kernel README is rather old and the advice "Do NOT use the /usr/src/linux area!" is not valid any more.

A long time ago distros used to make /usr/include/asm and /usr/include/linux symlinks that pointed to /usr/src/linux/include/asm and /usr/src/linux/include/linux, respectively. But nowadays the kernel headers that are used for compiling glibc and other libraries are in most modern distros actually located in /usr/include/asm and /usr/include/linux, so messing with the /usr/src/linux symbolic link shouldn't really have any effect even if you decide to recompile some libraries.

/usr/src/linux is usually just a symlink that points to the latest kernel sources. If you want to follow the advice in the kernel README to the letter, then don't create this /usr/src/linux symlink. But if you feel rebellious, you can ignore the warnings and create the symlink. Either way, you shouldn't experience any unpleasant consequences.

BTW, extracting the Linux sources under /usr/src is a convention that follows the FHS recommendation.
http://www.pathname.com/fhs/pub/fhs-2.3.html

[mzilikazi]

Lately some authors of Debian kernel howtos, my self included, are taking some flack

Sorry if you felt that I was giving you 'flack' because that was not at all my intention. I wished only to spread some knowledge. Is that so wrong? If I were to post some incorrect information I would certainly expect to be corrected. In fact, I insist on it because that means that I'm misinforming others because I too am misinformed.

Now, it's easy to miss, but in fact the README does not say you shouldn't use /usr/src. It does say not to use /usr/src/linux, a directory that is not present on Debian systems. Actually, on a default Debian system /usr/src is empty.

It says to unpack the source to someplace that is writable for user and /usr/src is of course not writable for user.

I believe that the kernel README is rather old and the advice "Do NOT use the /usr/src/linux area!" is not valid any more.

Umm no.....the README is from that latest source. Funny how no one reads it.

Here is the snippet causing all of the fuss directly from the 2.6.20-rc1 source:

INSTALLING the kernel:

- If you install the full sources, put the kernel tarball in a
directory where you have permissions (eg. your home directory) and
unpack it:

gzip -cd linux-2.6.XX.tar.gz | tar xvf -

or
bzip2 -dc linux-2.6.XX.tar.bz2 | tar xvf -


Replace "XX" with the version number of the latest kernel.

Do NOT use the /usr/src/linux area! This area has a (usually
incomplete) set of kernel headers that are used by the library header
files. They should match the library, and not get messed up by
whatever the kernel-du-jour happens to be.

It's true, Debian kernel source is installed to /usr/src so there is where it gets built BUT if you download the vanilla kernel source there is absolutely not 1 single reason to use /usr/src.
If there is a reason I'd love to hear it. So IMHO the real question is:

Why would you build in /usr/src?

But don't take my word for it - take Linus'

[Lavene]

Sorry if you felt that I was giving you 'flack' because that was not at all my intention. I wished only to spread some knowledge. Is that so wrong? If I were to post some incorrect information I would certainly expect to be corrected. In fact, I insist on it because that means that I'm misinforming others because I too am misinformed.

Um... I was actually refering to a couple of PMs I recieved (IIRC you actually linked to my howto without mentioning the fact that it too uses /usr/src). And I started this thread so we could discuss the issue...

If there is a reason I'd love to hear it. So IMHO the real question is:
Why would you build in /usr/src?

Mainly for concistency reasons. Apt places it there so it's the logical place to build it. And by placing the vanilla kernels there too you get them all in one place.

Another question is of course if we should start 'lobbying' for a change in the policy so that apt stops using /usr/src in order to avoid this kind of confusion..?

Tina

[mzilikazi]

(IIRC you actually linked to my howto without mentioning the fact that it too uses /usr/src).

Yep it's a good howto and the Debian kernel source (as you mentioned) does get placed in /usr/src - so that's that. Perhaps I should have mentioned the difference between a Debian kernel & vanilla kernel since I use only vanilla source (Debian kernel source is always a behind the latest).

Another question is of course if we should start 'lobbying' for a change in the policy so that apt stops using /usr/src in order to avoid this kind of confusion..?

Tina

Hmm good question.

[Grifter]

Why would you build in /usr/src?

I do, and I symlink linux-xxx to linux

because I always have, and always will

[thamarok]

Why would you build in /usr/src?

Hmm... I build my kernels in /usr/src (not /usr/src/linux) because if I want to build the modules_image, it will look for /usr/src/modules, so why not also builing the vanilla/chocolate kernels there?

[Lux]

Umm no.....the README is from that latest source. Funny how no one reads it.

I know that the README and its warning comes with the latest kernel sources but that warning was actually first written many, many years ago for a much, much older version of the kernel. The post from Linus that you referred to is from the year 2000 -- that's almost six and a half years ago!

His advice was valid back then but it's not valid any more, for the reasons that I pointed out in my earlier post: GNU/Linux distros had different conventions in those days and they used different directories to store the kernel headers that are needed for building the system libraries. In modern distros, including Debian, there is no danger whatsoever that linking your current kernel sources to /usr/src/linux would result in building the system libraries with incompatible kernel headers.

[garrincha]

This post recalled the time I spend in the toilet (:shock:) about a week ago while reading a copy of Debian GNU/Linux 3.1 Bible (published 2005) and in the section Packaging the Kernel, on pg 549, Ch 26, 3rd paragraph:

Quote:
After you have the right packages installed, change the directory to /usr/src. Except for the final step of actually installing the new kernel, you should follow the good security practice of doing all the compilation as a non-privileged user. The /usr/src directory is set up to be writeable by group src, so if the user ID you wish to use is not a member of that group, you should add it. (You may need to log out and log in again for the new group membership to take effect.) The kernel source you downloaded can be found in /usr/src as a tarball compressed with bzip2 called linux-source-2.6.8.tar.bz2. It is installed this way because the Linux kernel is huge; it's about 50MB uncompressed. So to save space, it is not installed in an uncompressed state; you can do that yourself. Go ahead and uncompress it like this:

$ tar xjf linux-source-2.6.8.tar.bz2

If you get an error message, you may have to install the bzip2 package.

When you have the source uncompressed, change to the newly created linux-source-2.6.8 directory. Some people suggest making a symlink from linux-source-2.6.8 to usr/src/linux, but this is not recommended these days as it can cause the compiler and libc to find certain header files in the wrong place.

[End quote].

[Lavene]

This post recalled the time I spend in the toilet (:shock:) about a week ago while reading a copy of Debian GNU/Linux 3.1 Bible (published 2005) and in the section Packaging the Kernel, on pg 549, Ch 26, 3rd paragraph:
<snip>

Thanks! That makes sense I still don't quite get the /usr/src/linux problem though. Oh well... I never make that symlink anyway.

Tina

[meon]

Has this /usr/src thing become a linux myth?
Now it is some time since I compiled my own kernel. The kernels from the repositories works fine with the odd module needed from outside. I did never have a problem with /usr/src. So what is the conclusion?

[sargek]

I have been using /usr/src for as long as I can remember, but I do remember a few years ago reading the very thing Lux mentioned - it was an email from Linus himself, but it only applied many years ago and apparently does not apply now. Back then, I used to build kernels in my home directory, but now I do it in /usr/src - well, in Gentoo I used /usr/src because that was their advice, but I haven't built a custom kernel in Debian yet because the stock 686 kernel is WAY faster than any Gentoo system I have ever run, so no need....I'll get curious one of these days and build one, but I'll use /usr/src.

One side note, I have had no luck installing the Nvidia drivers (from Nvidia) unless I have kernel source installed, unpacked, and symlinked to /usr/src/linux. So, that being said, I always unpack the kernel source and symlink it to /usr/src/linux. Doing the Nvidia drivers the "Debian way" is, IMHO, totally convoluted and doesn't work at all, at least the 47 different "Debian ways" I've tried have never worked.

[Lavene]

Well, I could not get this out of my head. What bugged me was this:

... ...
Quote:
After you have the right packages installed, change the directory to /usr/src. Except for the final step of actually installing the new kernel, you should follow the good security practice of doing all the compilation as a non-privileged user. The /usr/src directory is set up to be writeable by group src, so if the user ID you wish to use is not a member of that group, you should add it.

So I decided to try that approach. I made sure I was in the src group, downloaded a kernel source and proceeded to compile it as user using "make_kpkg kernel_image". And what do you know... after compiling, which seemed to go as usual, it craped out with the following nice message:

This is kernel package version 10.065.
need root privileges
make: *** [binary/linux-image-2.6.18] Error 1

So kernel-package does indeed require root.

Tina

[Telemachus]

Hi all,

I am a (relative) Linux newbie (I installed Ubuntu in June, and just moved to Debian about a week ago), and so I apologize if I only end up fanning the flames here. I was also curious about all this myself. Just before a nice two-week break (I'm a teacher), I bought a lovely new book "Linux Kernel In a Nutshell" by Greg Kroah-Hartman (O'Reilly). The book itself is just out (Dec 06), and K-H is described as a "Linux kernel developer."

In the intro, K-H goes out of his way to say:

The kernel source code should also never be placed in the /usr/src/linux directory, as that is the location of the kernel that the system libraries were built against, not your new custom kernel. Do not do any kernel development under the /usr/src/ directory tree at all, but only in a local user directory where nothing bad can happen to the system (page 4).

He doesn't specify what bad things might happen, but I noted it and I was surprised to see all the on-line Faq and How-To messages saying to stick the thing in /usr/src/.

I don't mean to say that he is right, but if it is an old habit for many people it apparently hasn't gone away. He recommends creating a /linux/ directory in your home directory (with sub dirs for different kernel sources). He also recommends over and over that you do everything as a regular user until you get to actually installing the thing. Part of the problem I think is that he is describing a distro neutral plan, and the "Debian way" has its own logic (as often). The man page for make_kpkg does indeed specify that it requires root or fakeroot or the like:

Typically, you run this command as root, or under fakeroot, or tell make-kpkg how to become root, like so:


make-kpkg --rootcmd fakeroot kernel_image

The Debian package file is created in the parent directory of the kernel source directory where this command is run.

Notice that the man page assumes that you are in the kernel source directory already.

Sorry to go on so long, but this also bugs me. I still don't quite get what makes better sense.

Confused in Debian-ville, Nemo