Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

HowTo Build a Package from Source the Smart Way

Share your HowTo, Documentation, Tips and Tricks. Not for support questions!.
Message
Author
User avatar
Telemachus
Posts: 4574
Joined: 2006-12-25 15:53
Been thanked: 2 times

Re: HowTo Build a Package from Source the Smart Way

#16 Post by Telemachus »

mashcaster wrote:This guide seems to be missing 1 important section. How do you packages from the debian dvds if your computer is not connected to the internet?
You don't need the internet to install packages from a cd or dvd. Just add the disc to your repos with apt-cdrom add (with the disc in).
"We have not been faced with the need to satisfy someone else's requirements, and for this freedom we are grateful."
Dennis Ritchie and Ken Thompson, The UNIX Time-Sharing System

User avatar
Soul Singin'
Posts: 1605
Joined: 2008-12-21 07:02

Re: HowTo Build a Package from Source the Smart Way

#17 Post by Soul Singin' »

Telemachus wrote:
mashcaster wrote:This guide seems to be missing 1 important section. How do you packages from the debian dvds if your computer is not connected to the internet?
You don't need the internet to install packages from a cd or dvd. Just add the disc to your repos with apt-cdrom add (with the disc in).
I almost wrote a reply with the same response, but -- because this is a thread about building from source -- I thought that that couldn't be what he meant.

Since he hasn't replied, I assume that Telemachus answered his question, but for any future readers who might want to create customized packages or backport packages from Testing to Stable ...

There are DVDs of Debian source code for both Stable and Testing. You can get DVDs of the Stable source from this site and you can get DVDs of the weekly builds of the Testing source from this site.

After burning the DVDs, you would then add them to your /etc/apt/sources.list file (as Telemachus described), grab the source and build your package:

Code: Select all

$ su
Password:
# apt-cdrom add
# apt-get update
# exit
$ mkdir /home/XXXX/my_build/
$ cd /home/XXXX/my_build/
$ apt-get source <packagename>
$ cd <packagename>-<version>/
$ dpkg-buildpackage -rfakeroot -us -uc
.

acimmarusti
Posts: 397
Joined: 2009-02-27 04:59
Location: Portland, OR USA

Re: HowTo Build a Package from Source the Smart Way

#18 Post by acimmarusti »

Soul Singin' wrote: It also annoys me that the files are installed at the same time that the package is built. (That's why you have to become root to use the checkinstall command). I wish it would just build the DEB and then let me decide when I want to install it.
I've been following your howto very closely and I've used it to backport 3 apps from squeeze to lenny. However for the last one I had to use checkinstall. Actually one of the options of checkinstall is that it allows you to choose whether you want to install the package or not:

Code: Select all

sudo checkinstall -D --install=no

User avatar
julian67
Posts: 4633
Joined: 2007-04-06 14:39
Location: Just hanging around
Been thanked: 7 times

Re: HowTo Build a Package from Source the Smart Way

#19 Post by julian67 »

Soul Singin' wrote: It also annoys me that the files are installed at the same time that the package is built. (That's why you have to become root to use the checkinstall command). I wish it would just build the DEB and then let me decide when I want to install it.
Man checkinstall
...................

--install Toggle installation of the created package.
There are also lots of other options including toggling an interactive install of the created deb, excluding files, forcing inclusion of files and so on.

It's not a substitute for Debian's package building tools or paying attention to <src_package>/debian but it's also not as inflexible or limited as often presented/supposed.

edit:seem to have posted same kind of thing as acimmarusti...oh well
Wisdom from my inbox: "do not mock at your pottenocy"


User avatar
budr
Posts: 16
Joined: 2008-01-19 18:28
Location: Oklahoma

Re: HowTo Build a Package from Source the Smart Way

#21 Post by budr »

An excellent howto, Soul Singin'. Couple of things you might add to the last resort, installing from source routine. Most packages do have a make uninstall target that will --usually-- do the right thing. And make has a -n option that just prints the list of commands that it would run. So in most packages, right before you grit your teeth and run make install, try make -n uninstall. Redirect the output to a file and you've got an uninstall script ready made. Failing that, try make -n install and save that. That way at least you've got a pretty good idea where to look for things if you do have to back out of an install. Not nearly as good as making a deb, but better than nothing...
punt() unless $clue;

User avatar
Hadret
Posts: 354
Joined: 2007-07-19 13:04
Location: Berlin

Re: HowTo Build a Package from Source the Smart Way

#22 Post by Hadret »

There's this nifty app called Gwibber and here's Debian package for it: http://mentors.debian.net/cgi-bin/spons ... ge=gwibber. When I was making the package, I did something dirty - I have edited the sources of Gwibber to satisfy lintian. There's only one warning about missing man page (which don't exist, at the moment). I've changed executable flag of two image files and removed from three themes jquery.js file. Than I created two files, postinst and postrm:

postinst file

Code: Select all

#!/bin/sh
set -e

ln -s /usr/share/javascript/jquery/jquery.js /usr/share/gwibber/ui/themes/bluelines/jquery.js
ln -s /usr/share/javascript/jquery/jquery.js /usr/share/gwibber/ui/themes/dark-gwilouche/jquery.js
ln -s /usr/share/javascript/jquery/jquery.js /usr/share/gwibber/ui/themes/gwilouche/jquery.js

#DEBHELPER#
postrm file

Code: Select all

#!/bin/sh
set -e

rm -rf /usr/share/gwibber

#DEBHELPER#
Is this correct way of doing that? I wanted to make use of libjs-jquery package (as lintian suggested) and Gwibber itself is working fine, but is it done good? Any feedback will be very appreciated! (:

pendrachken
Posts: 1394
Joined: 2007-03-04 21:10
Location: U.S.A. - WI.

Re: HowTo Build a Package from Source the Smart Way

#23 Post by pendrachken »

When using checkinstall it is a good idea to use a directory in /tmp, such as /tmp/build. Checkinstall will complain about files on some builds if the source directory is under /home. One example that I have come across lately is pidgin-2.5.7, if built under /home directories most of finch / libfinch won't be installed IF you listen to the checkinstall dialogs. Try building in something like /home/username/build/pidgin-2.5.7, it will give out warnings about "possibly including files from your home directory" or something along those lines and ask if you want to see the file list, then it asks if you want to exclude those files with a default of "yes". Not a good idea to exclude them in this case.

Putting "build checkinstall sources in /tmp/build/" will reduce support headaches a little bit more for the odd program that acts like this in checkinstall.
fortune -o
Your love life will be... interesting.
:twisted: How did it know?

The U.S. uses the metric system too, we have tenths, hundredths and thousandths of inches :-P

cristoper
Posts: 14
Joined: 2008-03-21 01:34
Location: Broomfield, CO

Re: HowTo Build a Package from Source the Smart Way

#24 Post by cristoper »

Finally, there's the make install routine. make install should be your absolute last resort.
There's another option that should be taken rather than 'make install'ing files directly into /usr/local: the stow command (or any of the numerous other programs that do the same thing). With stow you can easily uninstall any software, even if it doesn't have a 'make uninstall' target, quickly change between different versions of software, etc.

I use it instead of building .debs out of non-debian software because I'm lazy, but also because when tracking unstable versions of software I find it easier, for example, to do 'svn up && make install' than rebuilding a package every time I pull in updates.

(The debian package for stow is conveniently named "stow".)

User avatar
julian67
Posts: 4633
Joined: 2007-04-06 14:39
Location: Just hanging around
Been thanked: 7 times

Re: HowTo Build a Package from Source the Smart Way

#25 Post by julian67 »

I think Soul Singin' has made a very good, informative howto. A useful addition to the first post would be on applying patches. Occasionally it's useful to apply an upstream or 3rd party patch while retaining the Debian versioning and other modifications/enhancements and dependency tracking. I do this sometimes using the following procedure:

apply patch to debian source and build package:

apt-get source <package_name>

dpkg-source -x <package_name>.dsc

cd <package_name>/

apply patch: patch -p1 < <some_patch>

change version number with debchange: debchange -b -v <version_number>.patched

dh_make -s -n

build package:

dpkg-buildpackage -rfakeroot -uc -b
Wisdom from my inbox: "do not mock at your pottenocy"

User avatar
s3a
Posts: 831
Joined: 2008-07-17 22:13
Has thanked: 6 times
Been thanked: 2 times

Re: HowTo Build a Package from Source the Smart Way

#26 Post by s3a »

Soul Singin':

I've been attempting this on and off and I am unfortunately still stuck :(. I re-read everything very slowly and I don't know why but this is just one of the things that I can't get through my head. I succesfully installed backintime by "cheating." I just went to the Debian packages page and downloaded the Squeeze backintime debs as well as the debs of the dependencies, dumped them all in a folder and went to the directory using the terminal then did sudo dpkg -i * and it works so I am not here trying to actually get it to work anymore but I am just trying to learn. If I understood correctly, dh-make Debianizes the upstream source so that we can then easily work with the Debianized source? I also noticed the dh-make method lists the required dependencies which I guess is part of the whole "Debianization" definition. I understand if you've given up on me since you explained this many times but if you wouldn't mind explaining the dh-make way, I'd appreciate it.

I am using a Ubuntu 9.04 Live CD in order to preserve order within my hard drive installation to attempt this:

http://pastebin.com/m25a8268a

P.S.
I am following this line and below: If you're really eager to compile and there is no Debian-ized source in Unstable or Testing, you can still build a DEB package by using dh-make.
Use Mnemosyne to Study for School!

User avatar
Soul Singin'
Posts: 1605
Joined: 2008-12-21 07:02

Re: HowTo Build a Package from Source the Smart Way

#27 Post by Soul Singin' »

s3a wrote:I understand if you've given up on me since you explained this many times but ...
Relax. I am here to help. Just bear with me during my occasional grumpy moods.
s3a wrote:I succesfully installed backintime by "cheating." I just went to the Debian packages page and downloaded the Squeeze backintime debs as well as the debs of the dependencies, dumped them all in a folder and went to the directory using the terminal then did sudo dpkg -i * and it works
Not a brilliant method. With most packages, that method will force you to upgrade important system libraries (e.g. libc6).

In this case however, I don't think it's a problem because Back In Time only requires a version of Python that is available in Lenny (though I'm not sure about python-support).
s3a wrote:If I understood correctly, dh-make Debianizes the upstream source so that we can then easily work with the Debianized source? I also noticed the dh-make method lists the required dependencies which I guess is part of the whole "Debianization" definition.
Not exactly.

dh_make helps you Debianize the upstream source. It does not completely Debianize it.

What it produces is a basic set of files in the debian/ directory, so that you can build a DEB package. Usually, those files have to be edited to suit the needs of the particular package that you're trying to compile.

There is no standard way to build a package from source. Each package is different. Some, like Back In Time, are Python-based. Others, like PDFedit, use a Makefile.

The point here is that each package requires a different method of Debianization. The best way to figure out how to Debianize a package is to study the debian/ directories of packages that have already been Debianized.

For example, when I built PDFedit 0.4.3 yesterday, I copied most of the files that were in the debian/ directory of PDFedit 0.4.2 and only made a few small changes.

Specifically, I updated the changelog, changelog and copyright files. Then I deleted the patches because they had been merged upstream. I also deleted the watch and orig-tar.sh files because I didn't understand what they did. Then -- because I had deleted the watch and orig-tar.sh files -- I had to make a few edits to the rules file.

To provide another example, I edited the rules file of Gnash 0.8.5, so that it would use GStreamer (as opposed to FFmpeg like Debian's own package does).
s3a wrote:I am just trying to learn
Relax. You'll get there.

Good Luck!,
- Soul Singin'
.

User avatar
s3a
Posts: 831
Joined: 2008-07-17 22:13
Has thanked: 6 times
Been thanked: 2 times

Re: HowTo Build a Package from Source the Smart Way

#28 Post by s3a »

As shown here: http://pastebin.com/m47bc336d

I am stuck on the dpkg-buildpackage part of your how-to. The folder backintime-0.9.26.orig got generated without a single problem however when you say "You might then have to tweak the debian/rules file (e.g. to set configuration options) or set a shell variable. It all depends on the package in question. Once you have done that (or accepted all of the defaults), then the next step is to run (as normal user):," I have no idea how to "tweak the debian/rules." I did however notice that that folder was added in the original extracted folder. At first I thought you were referring to the debian_specific folders in the subdirectories of the generated orig folder. Also, when I do dpkg-buildpackage, which exact directory am I supposed to be in considering that the folders are on the desktop?
Use Mnemosyne to Study for School!

harishankar
Posts: 75
Joined: 2006-09-17 06:43

Re: HowTo Build a Package from Source the Smart Way

#29 Post by harishankar »

This is a great tutorial and one of the simplest ones I've found, but I request you to include information on how exactly to package small Python programs (not shared modules) for Debian. I am writing one myself and I have read a lot about python-support and python-central, but I have not found a single tutorial that says how to use them for this particular purpose.

Your efforts are much appreciated.

User avatar
Hadret
Posts: 354
Joined: 2007-07-19 13:04
Location: Berlin

Re: HowTo Build a Package from Source the Smart Way

#30 Post by Hadret »

Following Python Applications Packaging Team you should use python-support for python applications. You could use also CDBS, but sooner or later you will have to do it "debian-way", so, I guess, learning python-support is much more reasonable. You're building debian package in same way, as if you would building it for any other application, but in debian/rules you will have to include these lines in proper places:

Code: Select all

python setup.py --root=foo
dh_pysupport
More informations you can find here: http://people.debian.org/~srivasta/manoj-policy/
It's also good idea to download some python application source from Debian and look into debian/ folder to check how it's done (:

Good luck!

harishankar
Posts: 75
Joined: 2006-09-17 06:43

Re: HowTo Build a Package from Source the Smart Way

#31 Post by harishankar »

Thanks Hadret. I've not yet found how exactly to use python-support.

Is there a full tutorial on this subject anywhere online? Every packaging tutorial for Debian uses the standard debian utils. Even the tutorials for python apps don't rely on python-support.

By the way, there is a confusion: they say python-support and python-central should be used only for packaging python libraries/modules and not application/scripts. Is this true?

harishankar
Posts: 75
Joined: 2006-09-17 06:43

Re: HowTo Build a Package from Source the Smart Way

#32 Post by harishankar »

Here's where I am really confused about packaging for Python applications and not a single answer have I found online:

1. How should privately used application modules be handled by distutils? Does distutils provide method for handling private modules? py_modules seem to be for public modules and doesn't seem the right way to package an application.
OR
2. Should I give up distutils and use a standard Makefile instead of using python distutils at all when packaging for Debian?

3. Which is ultimately the easiest and most sane method of python application packaging in Debian? There seem to be an infinite number of policies, rules and instructions which all add up to giving me a headache. :oops:

I've looked at so many documents and help files online, but not a single one is comprehensive enough to give me a clue about the right way of doing things. The official Debian documentation is confusing, very long and doesn't provide step by step instructions on Python packages.

harishankar
Posts: 75
Joined: 2006-09-17 06:43

Re: HowTo Build a Package from Source the Smart Way

#33 Post by harishankar »

I've gone ahead and used the distutils/CDBS method as outlined here: https://wiki.edubuntu.org/PackagingGuide/Python

I'm not sure it's the right "Debian" way, but it seems the easiest method so far.

User avatar
Hadret
Posts: 354
Joined: 2007-07-19 13:04
Location: Berlin

Re: HowTo Build a Package from Source the Smart Way

#34 Post by Hadret »

I'm packaging gwibber application for Debian and, as you may guess, it's python app. As I said before - you can use CDBS, but if you want to see your package in Debian repositories, sooner or later you will have to move it to python-support. If you still want to use CDBS you should look into gwibber package for Ubuntu - it's using CDBS (you could also collate Ubuntu package with Debian package). I think that page to which link I have provided earlier is a bit outdated, but it's still good source for learning how to build python application packages.

Let's look into gwibber source, shall we?

Code: Select all

dget -x http://mentors.debian.net/debian/pool/main/g/gwibber/gwibber_1.2.0+bzr352-1.dsc
Now let's go into debian/ folder and see what do we've got there. Files which are relevant for us are debian/control and debian/rules. Don't hesitate to open them in your favorite editor. Which parts are important? I will bold them:
debian/control

Source: gwibber
Section: gnome
Priority: optional
Build-Depends: debhelper (>=7)
Build-Depends-Indep: python (>=2.5.4), python-distutils-extra, python-support (>=1.0)
Standards-Version: 3.8.1

Package: gwibber
Architecture: all || assuming that your package is arch independent ||
Depends: ${misc:Depends}, ${python:Depends}
debian/rules

#!/usr/bin/make -f

build:

clean:
dh_testdir
dh_testroot

rm -rf build
dh_clean install-stamp

install: install-stamp

install-stamp:
dh_testdir
dh_testroot
dh_prep
dh_installdirs

python setup.py install --root=$(CURDIR)/debian/pkg-name || assuming that you've got setup.py for your python app ||
touch $@

binary-arch:

binary-indep: install
dh_testdir
dh_testroot
dh_installchangelogs
dh_installdocs
dh_installman pkg-name.1 || assuming that you've got man page for your package ||
dh_pysupport
dh_compress
dh_fixperms
dh_installdeb
dh_gencontrol
dh_md5sums
dh_builddeb

binary: binary-indep binary-arch
.PHONY: build clean binary-indep binary-arch binary install
As you see, it's not really harder than CDBS, is it? (:

Remember that this is only example and there are paths that you need to edit, and there might be some options that you will need enter but... I think that after some small changes, this should work (:

It's also a good idea to look into some other debian packages of python application sources.

harishankar
Posts: 75
Joined: 2006-09-17 06:43

Re: HowTo Build a Package from Source the Smart Way

#35 Post by harishankar »

Thanks.

The above is basically the Makefile approach + python distutils right?

I was greatly confused about why we invoke python-support here... This is where my confusion arose from.

It is said that you can use CDBS + python-support or python-central.

So what is the difference between the methods highlighted by you and the one at Ubuntuwiki? Is it only the fact that we are writing our own rules file and not importing the one from CDBS?

I'm sorry if I write incoherently and confused, because I am. I read too many docs recently and it's all swimming in my head.

Post Reply