HowTo Build a Package from Source the Smart Way

[Soul Singin']

Click on the link below to download a printer-friendly copy of this post in PDF format:

Build-Pkg-Smart-Way.pdf.tar.gz

(92.33 KiB) Downloaded 11852 times

==================================================================

changelog

2009 September 11

• append "~bpo50+1" to the DEB's version number, to preserve the upgrade path.
  • (Thanks to julian67 for the debchange suggestion).
• when using dh-make, rename the tarball to: <package>_<version>.orig.tar.gz
  • (Thanks to infinitycircuit for this suggestion).
• added some checkinstall tips.
  • (Thanks to acimmarusti for this suggestion).

2009 May 09

• original post

==================================================================

This HowTo is a response to the unlimited supply of terrible advice that I have seen on GNU/Linux forums, on blogs and in various and sundry corners of the internet.

In particular, I wish people would stop advising others to use the make install routine. It's just as easy -- if not easier -- to build a DEB. Building a DEB gives you all of the advantages of package management, whereas the make install routine does not.

So this HowTo will describe several different ways that you can build a DEB package. Then -- for those of you who insist on using the make install routine -- it will explain how to use make install safely.

==================================================================

Before we begin compiling however, let's review a few compilation basics.

First, you should have a reason for compiling package. There are many valid reasons why you might want to compile a package. The most common reason is to install a more recent version in Debian Stable. If that's your reason, then take a look around to see if someone has already built a DEB package for you (e.g. Backports). You'll save yourself a lot of trouble that way.

Another reason is to compile in support for a particular feature. For example, Debian's most recent version of Gnash uses FFmpeg as its sound handler, but I compiled the same version of Gnash with GStreamer as the sound handler.

In the end, it doesn't matter what your reason is. Just make sure you have one and make sure it's a good one.

Second, if you're using Debian, then you'd be well-advised to use Debian-ized source instead of upstream source because it's very easy to build a DEB package if you're using Debianized-source.

If you build and install a DEB, then APT will be aware of the presence of the package and it will make sure that you do not overwrite the files of a previously installed package. By contrast, the make install command will overwrite anything that gets in its way.

==================================================================

Building a DEB is very easy to do and it's a shame that the members of this forum do not advocate it. If you've never built a DEB before, then try building Backintime. It only takes a minute or two, so it's a great exercise.

In general, building a DEB from Debian-ized source is a simple four-step process.

1. First, you add a "source line" to your /etc/apt/sources.list file and install the build dependencies.
2. Then you download the source (as normal user).
3. Then you run dpkg-buildpackage to build the DEB.
4. Finally, you install the DEB(s) with dpkg -i

To be more specific, add the following line to your /etc/apt/sources.list file:

Code: Select all

deb-src http://ftp.us.debian.org/debian/ sid main

Then install the build dependencies:

Code: Select all

apt-get update
apt-get install fakeroot
apt-get build-dep <packagename>

Those are the last commands you should run as root before installing the DEB. All of the build commands should be run as normal user.

Next, create a directory for the build in your normal user's home directory and download the source into that directory:

Code: Select all

mkdir /home/XXXX/my_build/
cd /home/XXXX/my_build/
apt-get source <packagename>
cd <packagename>-<version>/

If you're running Debian Stable, you may want to change the package's version number, so that the upgrade path to the next version of Debian Stable is preserved. To do that, you edit the debian/changelog file.

For example, I recently backported version 0.8.3a-1 of wxMaxima to Debian Lenny, so at this point, I ran:

Code: Select all

debchange -b -v 0.8.3a-1~bpo50+1

A text editor then opened the debian/changelog file and I added an entry saying that I had backported it to Lenny. After I saved the changes and closed the editor, the following warning message appeared:

Code: Select all

debchange warning: new version (0.8.3a-1~bpo50+1) is less than
the current version number (0.8.3a-1).

That's exactly what I want -- a lower version number, so that the package is upgraded when I upgrade from Lenny to Squeeze.

Next, build the DEB:

Code: Select all

dpkg-buildpackage -rfakeroot -us -uc

That last command may take a minute or an hour or three hours. It all depends on the size of the package and your own hardware.

Once the dpkg-buildpackage command finishes, you'll find the DEB(s) in your /home/XXXX/my_build/ directory and you can install them (as root) with:

Code: Select all

cd /home/XXXX/my_build/
dpkg -i <packagename>_<version>_<architecture>.deb

It doesn't get much simpler than that.

Perhaps more importantly, a Debian Developer would follow a very similar procedure to create a backport from a package that's already in Testing or Unstable.

==================================================================

If you're really eager to compile and there is no Debian-ized source in Unstable or Testing, you can still build a DEB package by using dh-make.

Code: Select all

apt-get install fakeroot dh-make

In general, you begin by installing the build dependencies. If you're building a more recent version of package that's already in Debian, then the build-dependencies might be the same as the previous version. Otherwise, you'll have to figure out what they are (by reading the source package's documentation, for example).

Next, you extract the source into a directory in your home directory as normal user:

Code: Select all

mkdir /home/XXXX/my_build/
cd /home/XXXX/my_build/
tar -zxf <packagename>_<version>.tar.gz

Because dh-make Debian-izes the source, you may need to change a couple of file names. For example, suppose that we were trying to Debian-ize the upstream version of wxMaxima. In that case, we would have to change the name of the directory (to lowercase) and the name of the tarball (to the format: <package>_<version>.orig.tar.gz):

Code: Select all

mv wxMaxima-0.8.3a/ wxmaxima-0.8.3a/
mv wxMaxima-0.8.3a.tar.gz wxmaxima_0.8.3a.orig.tar.gz

Next, enter the <packagename>_<version>/ directory and run dh_make.

Code: Select all

cd <packagename>_<version>/
dh_make

dh_make will then ask you if you want to create a: "single binary, multiple binary, library, kernel module or cdbs" and ask you to complete the package maintainer fields.

You might then have to tweak the debian/rules file (e.g. to set configuration options) or set a shell variable. It all depends on the package in question.

Once you have done that (or accepted all of the defaults), then the next step is to run (as normal user):

Code: Select all

dpkg-buildpackage

and then install the DEB package (as root) with:

Code: Select all

dpkg -i <packagename>_<version>_<architecture>.deb

For a good example of how to use dh-make, see: HowTo Compile PDFedit.

==================================================================

Debian-ized source should be your first choice when building from source. Use dh-make when Debian-ized source is unavailable because it Debian-izes the source for you.

If for some reason Debian-ized source is not available and dh-make doesn't work, then your third choice should be to use checkinstall.

Code: Select all

apt-get install checkinstall

Checkinstall is not as good as the first two options, but at least it produces a DEB, so it will prevent you from unknowingly overwriting system files and it will make APT aware of the package's presence.

In general, using checkinstall is very similar to the make install routine. The only difference is that you don't run the make install command.

First, you pick an installation directory. It's a good idea to pick a directory where no other packages are installed or will be installed. For example, /opt/ or /usr/local/ are two good choices, but because checkinstall will build a DEB, APT will make sure that you don't overwrite any files even if you choose /usr/ (or if you let the defaults choose /usr/ for you).

Suppose you pick /usr/local/. The next step is to pass that choice to the configure script. For example:

Code: Select all

./configure --prefix=/usr/local --<configuration-options>

Then you would run:

Code: Select all

make
su -c checkinstall -D make install

The trouble with checkinstall is that it does not Debian-ize the source or understand package dependencies. It simply keeps track of the files that the DEB contains.

It also annoys me that the files are installed at the same time that the package is built. (That's why you have to become root to use the checkinstall command). I wish it would just build the DEB and then let me decide when I want to install it.

Nonetheless, it does produce a DEB, so the package can be easily removed and won't overwrite other files without first obtaining your permission. Just remember to chown it back to your normal user:

Code: Select all

chown XXXX:XXXX <packagename>_<version>_<architecture>.deb

EDIT: acimmarusti suggests using the command:

Code: Select all

su -c 'checkinstall -D --install=no'

In theory, that command should simply build the DEB without installing it. In practice, it may or may not install the package.

While updating this HowTo, I tried his suggestion (with wxMaxima 0.8.3a) by running:

Code: Select all

./configure --prefix=/opt
make
su -c 'checkinstall -D --install=no'

and unhappily discovered that some of the files were installed without informing APT of the changes.

Specifically, checkinstall installed the /opt/ directory's files, but did not install the /usr/share/doc/wxmaxima/ directory's files.

==================================================================

If an RPM is available, you might want to look into alien.

Code: Select all

apt-get install fakeroot alien

It's just as bad as checkinstall (because it doesn't understand package dependencies), but it's not much worse and -- most importantly -- it produces a DEB.

To use it, you simply download the RPM and run as normal user:

Code: Select all

fakeroot alien -d <packagename>-<version>.<architecture>.rpm

and then become root to install the DEB that alien produces.

Code: Select all

dpkg -i <packagename>_<version>_<architecture>.deb

==================================================================

Finally, there's the make install routine. make install should be your absolute last resort. You should NEVER use it unless all other options have failed and you should ONLY use it IF the package you want to build is of mission critical importance.

The make install routine is the most primitive method of building a package from source and has absolutely no concept of dependencies or package management. There's a reason why GNU/Linux distributions use package managers like APT or RPM. And that reason is to get as far away from make install as possible.

So if you're going to use make install, then at least use it intelligently. Do NOT blindly follow the lemmings running off the cliffs of the internet.

As with checkinstall, the most important thing to do is to pick an installation directory where no other packages are installed or will be installed. Once again, /opt/ and /usr/local/ are two good choices.

As with checkinstall, the next step is to pass that choice to the configure script. Suppose (again) that you pick /usr/local/. You would pass that chouce to the configure script with the --prefix flag:

Code: Select all

./configure --prefix=/usr/local --<configuration-options>

That will cause the files to be installed in your /usr/local/ directory, which Debian doesn't install to by default. This should ensure that you don't overwrite important system files when you run make install.

The next steps are to build the package and install it:

Code: Select all

 make
su -c make install

But that's not the end of the story. If you want to uninstall a package that you installed with the make install routine, then you had better make sure you don't accidentally delete that package that you compiled (even if it grows to 20 GB) and you had better hope that the its make uninstall routine works just as well as its installation routine or you'll be stuck manually deleting all of the files.

That's why you should ALWAYS build a DEB package. Let APT remember where the installed files are. You've got better things to do ... like play with your newly installed package!

==================================================================

In summary:

• It's easier to create DEBs from Debianized source than it is to use the make install routine.
• When Debianized source is unavailable, use a helper application to build a DEB, so that you can take advantage of APT's package management features.
• If you are unable to build a DEB package, then stop and ask yourself if you really need the package you're trying to build.
• You should ONLY use the make install routine as a LAST RESORT for MISSION CRITICAL applications.
• If you do use the make install routine, do it intelligently.

Good Luck and Have Fun!,
Soul Singin'
.

[infinitycircuit]

In general, it's probably better to just rename the tarball to:

package_version.orig.tar.gz

and then run dh_make, than to try to make dh_make figure it out with --create-orig.

Nice guide! The number of atrocious recommendations on installing packages is absurd.

[Hadret]

Very nice HowTo! I've got few questions:

1) In dh_make, there's command dpkg-buildpackage. Doesn't it should look like:

Code: Select all

dpkg-buildpackage -rfakeroot -us -uc

If not necessarily, why?

2) Is checkinstall really better than make install? I doubt that.

3) Is converting RPM using alien is really better than installation from source?

[Telemachus]

Finally, there's the make install routine. make install should be your absolute last resort. You should NEVER use it unless all other options have failed and you should ONLY use it IF the package you want to build is of mission critical importance.

The make install routine is the most primitive method of building a package from source and has absolutely no concept of dependencies or package management. There's a reason why GNU/Linux distributions use package managers like APT or RPM. And that reason is to get as far away from make install as possible.

First, I think that this is (mostly) an outstanding howto: lots of clear, detailed instructions and good information.

However, I have some bones to pick with the bits I quoted. My main complaint is that I think you have gone (way) overboard about how bad make install is. You make it sound as though your computer will implode if you use make - NEVER use it, ONLY mission critical. That seems pretty exaggerated to me. As a way of thinking about all this, here are my top four problems with using make to manage packages.

• First, no dependency tracking and no overall sense of how one item might fit together with or conflict with another.
• Second, if you're not careful, you might forget to add --prefix=/usr/local (or /opt or $HOME or whatever), and you might overwrite a piece of system software that was installed and expected by your operating system.
• Third, uninstallation can be a pain. Not every package includes an uninstall target, and even if it does, people often throw away the source folder.
• Fourth, there's no quick and easy way to track what you installed and when. There's no log (as for example there is with aptitude).

The real question to me is how well does building a .deb handle these problems? Honestly, only mixed results I think. The second problem is nearly a straw man. I can't remember two packages in all the time I've dealt with tarballs that didn't assume /usr/local or /opt. And, frankly, if you can't read enough to check that, then you deserve what you get. The third problem is handled very well. You can now use APT tools to cleanly uninstall software. You can also now search for items, and apt-get or aptitude will see them as installed. Great. As for the log (fourth problem), you can use dpkg's log, but frankly that's not the world's most helpful log. Still, it's something, and that's better than nothing.

But now it gets more intersting. How exactly does building a .deb personally help with the first problem? The real advantage of Debian's package system is not the convenience when you install one package, but that the Debian devs do such an outstanding job of thinking about the whole picture of a system. I have no reason to expect that I (or most people) can or would do nearly as well. If I make a single deb and install it, I haven't necessarily given any thought at all to dependencies, conflicts, etc. If anything, this method gives me a false sense of security, since I think - ah, it's a deb, everything is safe. But for example, I can easily imagine someone thinking "Oh, I want this new Whizbar from Gnome 4.27, but Debian still still has Gnome 3.54 and Whizbar wasn't even invented then. Cool, I'll just make a deb." But it's not cool, since the new Gnome Whizbar requires 457 Python and Cairo libraries to be updated as well. Has our imagined user made new debs for those too? I doubt it. Does he realize that Whizbar actually replaces Whizfoo, and that Gnome 3.54 is built around Whizfoo? I doubt it. Bottom line: building a deb for yourself does not, all by itself, equal dependency tracking and package management. There's a lot more to those than just having a deb.

A second issue is that I think that your advice works best when Debian doesn't have the package you want at all. But what if I need to use and test a package that (1) Debian includes by default and (2) Debian relies on to be a certain way? If you use a deb as you recommend, then I think that you are forced to choose between your version and Debian's. Why? Because a deb always goes into the place Debian would put it: so, for example, a deb of Perl would end up at /usr/bin/perl. But replacing Debian's default Perl interpreter is a terrible, terrible idea. If I make a deb, as I understand it, I have no choice. Once I install the new Perl, the old one must go away. But if I carefully use /usr/local and a sane path (one where /usr/local/bin precedes /usr/bin), then I can have my cake and eat it too: I get my bleeding edge Perl, and Debian can still find its version, just where it expects it. (By the way, if I'm completely wrong, and you can have two debs of one item at the same time, I will gladly eat my words.)

Ok, this is way too long: one quibble and then my closing argument. The quibble: isn't rpm more like dpkg. It installs packages, but it doesn't do dependency checking. Isn't yum the Red Hat APT? Closing argument: I think you way, way overdo the idea that make is dangerous. It isn't dangerous, if you use it with some thought and some careful reading. And frankly, all the debs in the world won't save you if you use them without thought and reading.

[hkoster1]

Well, of course the OP has been worded forcefully and in an entertaining way... overstating
a few things here and there, as Telemachus points out. Its main aim is those noobs that blindly
work outside the Debian packaging system, then come here looking for help when things go
wrong. I'm glad Soul Singin' took the time to make his point!

(BTW, wasn't he called Ed or something in a previous life?)

[Telemachus]

(BTW, wasn't he called Ed or something in a previous life?)

Shhh, I don't think that we're supposed to know that...

[Soul Singin']

(BTW, wasn't he called Ed or something in a previous life?)

Shhh, I don't think that we're supposed to know that...

There's a lot to respond to here. Let's start with the easy ones and then work our way up to Telemachus' excellent response.

1) In dh_make, there's command dpkg-buildpackage. Doesn't it should look like:

Code: Select all

dpkg-buildpackage -rfakeroot -us -uc

If not necessarily, why?

I don't know why you don't need to pass the -rfakeroot argument, but the reason why you don't have to pass the -us -uc arguments is because dh-make prompts you to complete the maintainer fields.

2) Is checkinstall really better than make install? I doubt that.

Could you list your reasons?

checkinstall and make install do the same job. The only difference is that checkinstall gives you a DEB, whereas make install does not.

3) Is converting RPM using alien is really better than installation from source?

If you're going to build a package from source by carefully choosing the build options, etc. etc. etc. then you should install from source.

But if you're going to blindly accept every default and if you're not going to think about the build at all, then you'd be well advised to use alien instead of the make install routine.

Its main aim is those noobs that blindly work outside the Debian packaging system, then come here looking for help when things go wrong.

Thank you. That summarizes my motivation rather nicely.

You make it sound as though your computer will implode if you use make

Yes. That's because when "those noobs ..." use the make install routine, their computers usually do implode.

As a way of thinking about all this, here are my top four problems with using make to manage packages.

• First, no dependency tracking and no overall sense of how one item might fit together with or conflict with another.

[...]

But now it gets more intersting. How exactly does building a .deb personally help with the first problem?

If you build a DEB using one of the first two methods that I described -- using Debian-ized source or dh-make -- then the build process will create a DEB that lists dependencies, conflicts, etc.

To illustrate the point, I grabbed the upstream source for PDFedit and ran dh_make. I didn't do anything else. dh_make automatically produced the following debian/control file:

Code: Select all

Source: pdfedit
Section: unknown
Priority: extra
Maintainer: Xxxx Xxxxx <xxxx@xxxx>
Build-Depends: debhelper (>= 7), autotools-dev
Standards-Version: 3.7.3
Homepage: <insert the upstream URL, if relevant>

Package: pdfedit
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: <insert up to 60 chars description>
 <insert long description, indented with spaces>

In particular, notice the third line from the bottom. The list of package dependencies is automatically generated by the build process.


As I mentioned in my post, DEBs created with checkinstall or alien do not contain a list of dependencies, so -- if you opt for one of the latter options -- you still have to know which packages are needed to make your installed package work.

Nonetheless, installing a DEB produced by checkinstall or alien at least provides APT with a list of installed files, which is more than what the make install routine does.

A second issue is that I think that your advice works best when Debian doesn't have the package you want at all.

It also works rather well when you simply want a newer version of a given package and you don't care at all for the older version. I've got VLC 0.9.8 in Debian Lenny. The Lenny repositories have 0.8.6. What the hell would I want to keep the old one around for?

But what if I need to use and test a package that (1) Debian includes by default and (2) Debian relies on to be a certain way? If you use a deb as you recommend, then I think that you are forced to choose between your version and Debian's. Why? Because a deb always goes into the place Debian would put it: so, for example, a deb of Perl would end up at /usr/bin/perl. But replacing Debian's default Perl interpreter is a terrible, terrible idea.

Your Perl example is valid, but try to keep the bigger picture in mind here. I'm trying to provide general build and installation instructions and to encourage users to think about what they are doing. Hopefully, the thinking part will dissuade them from replacing /usr/bin/perl.

I get my bleeding edge Perl, and Debian can still find its version, just where it expects it. (By the way, if I'm completely wrong, and you can have two debs of one item at the same time, I will gladly eat my words.)

I think that you can have DEBs for both. I've never tried it, but if you were to pass the --prefix=/usr/local argument in the debian/rules file and change the name of the package (e.g. "etruscan-perl") in the debian/control file, then you might be able to have DEBs of both.

The quibble: isn't rpm more like dpkg. It installs packages, but it doesn't do dependency checking. Isn't yum the Red Hat APT?

I have no idea. After watching a former boss suffer through Fedora dependency hell, I made that item number one on my: "Things that I will never do list."

Closing argument: I think you way, way overdo the idea that make is dangerous. It isn't dangerous, if you use it with some thought and some careful reading. And frankly, all the debs in the world won't save you if you use them without thought and reading.

As I mentioned in my response to Hadret ... If you're going to build a package from source by carefully choosing the build options, using the --prefix=/usr/local argument, etc. etc. etc., then you're not going to get yourself in trouble.

But perhaps I should "overdo" it. There are so many bad examples of the make install routine on the internet, that their weight in numbers must be matched by the force of this HowTo.
.

[Telemachus]

I'm impressed by how much of the dependency tracking the Debian tools help you to do automagically. (It's rare that I'm happy to say how completely wrong I was, but this is one of those times.) Building my own Perl is probably too complex a package to choose as my first deb, but maybe Rakudo (ie, Perl6). Etruscan Perl may be closer than we think...

I'm off to take a closer look at the Debian new maintainers guide. Thanks for the tutorial, Soul Singin'.

[mashcaster]

Building a DEB is very easy to do and it's a shame that the members of this forum do not advocate it. If you've never built a DEB before, then try building Backintime. It only takes a minute or two, so it's a great exercise.

In general, building a DEB from Debian-ized source is a simple four-step process.

1. First, you add a "source line" to your /etc/apt/sources.list file and install the build dependencies.
2. Then you download the source (as normal user).
3. Then you run dpkg-buildpackage to build the DEB.
4. Finally, you install the DEB(s) with dpkg -i

To be more specific, add the following line to your /etc/apt/sources.list file:

Code: Select all

deb-src http://ftp.us.debian.org/debian/ sid main

Then run install the build dependencies:

Code: Select all

apt-get update
apt-get install fakeroot
apt-get build-dep <packagename>

Those are the last commands you should run as root before installing the DEB. All of the build commands should be run as normal user.

Next, create a directory for the build in your normal user's home directory, download the source into that directory and build the DEB:

Code: Select all

mkdir /home/XXXX/my_build/
cd /home/XXXX/my_build/
apt-get source <packagename>
cd <packagename>-<version>/
dpkg-buildpackage -rfakeroot -us -uc

That last command may take a minute or an hour or three hours. It all depends on the size of the package and your own hardware.

Once the dpkg-buildpackage command finishes, you'll find the DEB(s) in your /home/XXXX/my_build/ directory and you can install them (as root) with:

Code: Select all

cd /home/XXXX/my_build/
dpkg -i <packagename>_<version>_<architecture>.deb

It doesn't get much simpler than that.

Perhaps more importantly, a Debian Developer would follow a very similar procedure to create a backport from a package that's already in Testing or Unstable.

I am trying to create a distributable deb file which can be installed on a development machine which has no internet connection, but I keep running into problems.

So far I have edited the sources.list file on my internet machine by inserting in a new line

Code: Select all

deb-src http://ftp.us.debian.org/debian/ experimental main

as MySQL 5.1 is only available in experimental at the moment

http://packages.debian.org/experimental/mysql-server

Then after updating

Code: Select all

aptitude  update

I search aptitude

Code: Select all

aptitude search mysql-server

and it just shows mysql-server-5.0. It does not show mysql-server-5.1

Why is that?

[Hadret]

Cause experimental repositories are "disabled" by default and you have to tell it to apt/aptitude, that you really want to install some package from there. There are two ways of doing that:

Code: Select all

# aptitude install package-name/experimental

or (with dependencies also from experimental)

Code: Select all

# aptitude -t experimental package-name

Use at your own risk (:

[drokmed]

Excellent howto, thanks!

I'm off to take a closer look at the Debian new maintainers guide

I started with that document a few hours ago, and then as it suggested, starting studying pbuilder too. Now my head is aching! That document teaches you the hard way to do it, like the devs do, to cover all options and scenarios. I'm shoving cotton balls into my ears, to stop the oozing. I wish I had found this thread before hand!

I want to run a newer version of a deb package (dansguardian) but unfortunately the one in testing/sid is old too! A major version upgrade was released last October, yet the developer is still working on an old version. It's driving me bonkers, to the point I'm willing to make my own. There are several others I want to make too, some of which have no deb version at all.

[Soul Singin']

I'm off to take a closer look at the Debian new maintainers guide

I started with that document a few hours ago, and then as it suggested, starting studying pbuilder too. Now my head is aching

Pop a few more aspirin and take a look at Miriam Ruiz' How to make a Debian package without using a helper.
.

[drokmed]

I actually found that one easier to read. Step by step is a lot clearer from A to Z. Great read, thanks!

[mashcaster]

This guide seems to be missing 1 important section. How do you packages from the debian dvds if your computer is not connected to the internet?

[Soul Singin']

This guide seems to be missing 1 important section. How do you packages from the debian dvds if your computer is not connected to the internet?

Could you rewrite that question? I don't understand what you are trying to ask.
.