Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

HowTo Build a Package from Source the Smart Way

Share your HowTo, Documentation, Tips and Tricks. Not for support questions!.
Message
Author
User avatar
budr
Posts: 16
Joined: 2008-01-19 18:28
Location: Oklahoma

Re: HowTo Build a Package from Source the Smart Way

#21 Post by budr »

An excellent howto, Soul Singin'. Couple of things you might add to the last resort, installing from source routine. Most packages do have a make uninstall target that will --usually-- do the right thing. And make has a -n option that just prints the list of commands that it would run. So in most packages, right before you grit your teeth and run make install, try make -n uninstall. Redirect the output to a file and you've got an uninstall script ready made. Failing that, try make -n install and save that. That way at least you've got a pretty good idea where to look for things if you do have to back out of an install. Not nearly as good as making a deb, but better than nothing...
punt() unless $clue;

User avatar
Hadret
Posts: 354
Joined: 2007-07-19 13:04
Location: Berlin

Re: HowTo Build a Package from Source the Smart Way

#22 Post by Hadret »

There's this nifty app called Gwibber and here's Debian package for it: http://mentors.debian.net/cgi-bin/spons ... ge=gwibber. When I was making the package, I did something dirty - I have edited the sources of Gwibber to satisfy lintian. There's only one warning about missing man page (which don't exist, at the moment). I've changed executable flag of two image files and removed from three themes jquery.js file. Than I created two files, postinst and postrm:

postinst file

Code: Select all

#!/bin/sh
set -e

ln -s /usr/share/javascript/jquery/jquery.js /usr/share/gwibber/ui/themes/bluelines/jquery.js
ln -s /usr/share/javascript/jquery/jquery.js /usr/share/gwibber/ui/themes/dark-gwilouche/jquery.js
ln -s /usr/share/javascript/jquery/jquery.js /usr/share/gwibber/ui/themes/gwilouche/jquery.js

#DEBHELPER#
postrm file

Code: Select all

#!/bin/sh
set -e

rm -rf /usr/share/gwibber

#DEBHELPER#
Is this correct way of doing that? I wanted to make use of libjs-jquery package (as lintian suggested) and Gwibber itself is working fine, but is it done good? Any feedback will be very appreciated! (:

pendrachken
Posts: 1394
Joined: 2007-03-04 21:10
Location: U.S.A. - WI.

Re: HowTo Build a Package from Source the Smart Way

#23 Post by pendrachken »

When using checkinstall it is a good idea to use a directory in /tmp, such as /tmp/build. Checkinstall will complain about files on some builds if the source directory is under /home. One example that I have come across lately is pidgin-2.5.7, if built under /home directories most of finch / libfinch won't be installed IF you listen to the checkinstall dialogs. Try building in something like /home/username/build/pidgin-2.5.7, it will give out warnings about "possibly including files from your home directory" or something along those lines and ask if you want to see the file list, then it asks if you want to exclude those files with a default of "yes". Not a good idea to exclude them in this case.

Putting "build checkinstall sources in /tmp/build/" will reduce support headaches a little bit more for the odd program that acts like this in checkinstall.
fortune -o
Your love life will be... interesting.
:twisted: How did it know?

The U.S. uses the metric system too, we have tenths, hundredths and thousandths of inches :-P

cristoper
Posts: 14
Joined: 2008-03-21 01:34
Location: Broomfield, CO

Re: HowTo Build a Package from Source the Smart Way

#24 Post by cristoper »

Finally, there's the make install routine. make install should be your absolute last resort.
There's another option that should be taken rather than 'make install'ing files directly into /usr/local: the stow command (or any of the numerous other programs that do the same thing). With stow you can easily uninstall any software, even if it doesn't have a 'make uninstall' target, quickly change between different versions of software, etc.

I use it instead of building .debs out of non-debian software because I'm lazy, but also because when tracking unstable versions of software I find it easier, for example, to do 'svn up && make install' than rebuilding a package every time I pull in updates.

(The debian package for stow is conveniently named "stow".)

User avatar
julian67
Posts: 4633
Joined: 2007-04-06 14:39
Location: Just hanging around
Been thanked: 7 times

Re: HowTo Build a Package from Source the Smart Way

#25 Post by julian67 »

I think Soul Singin' has made a very good, informative howto. A useful addition to the first post would be on applying patches. Occasionally it's useful to apply an upstream or 3rd party patch while retaining the Debian versioning and other modifications/enhancements and dependency tracking. I do this sometimes using the following procedure:

apply patch to debian source and build package:

apt-get source <package_name>

dpkg-source -x <package_name>.dsc

cd <package_name>/

apply patch: patch -p1 < <some_patch>

change version number with debchange: debchange -b -v <version_number>.patched

dh_make -s -n

build package:

dpkg-buildpackage -rfakeroot -uc -b
Wisdom from my inbox: "do not mock at your pottenocy"

User avatar
s3a
Posts: 831
Joined: 2008-07-17 22:13
Has thanked: 6 times
Been thanked: 2 times

Re: HowTo Build a Package from Source the Smart Way

#26 Post by s3a »

Soul Singin':

I've been attempting this on and off and I am unfortunately still stuck :(. I re-read everything very slowly and I don't know why but this is just one of the things that I can't get through my head. I succesfully installed backintime by "cheating." I just went to the Debian packages page and downloaded the Squeeze backintime debs as well as the debs of the dependencies, dumped them all in a folder and went to the directory using the terminal then did sudo dpkg -i * and it works so I am not here trying to actually get it to work anymore but I am just trying to learn. If I understood correctly, dh-make Debianizes the upstream source so that we can then easily work with the Debianized source? I also noticed the dh-make method lists the required dependencies which I guess is part of the whole "Debianization" definition. I understand if you've given up on me since you explained this many times but if you wouldn't mind explaining the dh-make way, I'd appreciate it.

I am using a Ubuntu 9.04 Live CD in order to preserve order within my hard drive installation to attempt this:

http://pastebin.com/m25a8268a

P.S.
I am following this line and below: If you're really eager to compile and there is no Debian-ized source in Unstable or Testing, you can still build a DEB package by using dh-make.
Use Mnemosyne to Study for School!

User avatar
Soul Singin'
Posts: 1605
Joined: 2008-12-21 07:02

Re: HowTo Build a Package from Source the Smart Way

#27 Post by Soul Singin' »

s3a wrote:I understand if you've given up on me since you explained this many times but ...
Relax. I am here to help. Just bear with me during my occasional grumpy moods.
s3a wrote:I succesfully installed backintime by "cheating." I just went to the Debian packages page and downloaded the Squeeze backintime debs as well as the debs of the dependencies, dumped them all in a folder and went to the directory using the terminal then did sudo dpkg -i * and it works
Not a brilliant method. With most packages, that method will force you to upgrade important system libraries (e.g. libc6).

In this case however, I don't think it's a problem because Back In Time only requires a version of Python that is available in Lenny (though I'm not sure about python-support).
s3a wrote:If I understood correctly, dh-make Debianizes the upstream source so that we can then easily work with the Debianized source? I also noticed the dh-make method lists the required dependencies which I guess is part of the whole "Debianization" definition.
Not exactly.

dh_make helps you Debianize the upstream source. It does not completely Debianize it.

What it produces is a basic set of files in the debian/ directory, so that you can build a DEB package. Usually, those files have to be edited to suit the needs of the particular package that you're trying to compile.

There is no standard way to build a package from source. Each package is different. Some, like Back In Time, are Python-based. Others, like PDFedit, use a Makefile.

The point here is that each package requires a different method of Debianization. The best way to figure out how to Debianize a package is to study the debian/ directories of packages that have already been Debianized.

For example, when I built PDFedit 0.4.3 yesterday, I copied most of the files that were in the debian/ directory of PDFedit 0.4.2 and only made a few small changes.

Specifically, I updated the changelog, changelog and copyright files. Then I deleted the patches because they had been merged upstream. I also deleted the watch and orig-tar.sh files because I didn't understand what they did. Then -- because I had deleted the watch and orig-tar.sh files -- I had to make a few edits to the rules file.

To provide another example, I edited the rules file of Gnash 0.8.5, so that it would use GStreamer (as opposed to FFmpeg like Debian's own package does).
s3a wrote:I am just trying to learn
Relax. You'll get there.

Good Luck!,
- Soul Singin'
.

User avatar
s3a
Posts: 831
Joined: 2008-07-17 22:13
Has thanked: 6 times
Been thanked: 2 times

Re: HowTo Build a Package from Source the Smart Way

#28 Post by s3a »

As shown here: http://pastebin.com/m47bc336d

I am stuck on the dpkg-buildpackage part of your how-to. The folder backintime-0.9.26.orig got generated without a single problem however when you say "You might then have to tweak the debian/rules file (e.g. to set configuration options) or set a shell variable. It all depends on the package in question. Once you have done that (or accepted all of the defaults), then the next step is to run (as normal user):," I have no idea how to "tweak the debian/rules." I did however notice that that folder was added in the original extracted folder. At first I thought you were referring to the debian_specific folders in the subdirectories of the generated orig folder. Also, when I do dpkg-buildpackage, which exact directory am I supposed to be in considering that the folders are on the desktop?
Use Mnemosyne to Study for School!

harishankar
Posts: 75
Joined: 2006-09-17 06:43

Re: HowTo Build a Package from Source the Smart Way

#29 Post by harishankar »

This is a great tutorial and one of the simplest ones I've found, but I request you to include information on how exactly to package small Python programs (not shared modules) for Debian. I am writing one myself and I have read a lot about python-support and python-central, but I have not found a single tutorial that says how to use them for this particular purpose.

Your efforts are much appreciated.

User avatar
Hadret
Posts: 354
Joined: 2007-07-19 13:04
Location: Berlin

Re: HowTo Build a Package from Source the Smart Way

#30 Post by Hadret »

Following Python Applications Packaging Team you should use python-support for python applications. You could use also CDBS, but sooner or later you will have to do it "debian-way", so, I guess, learning python-support is much more reasonable. You're building debian package in same way, as if you would building it for any other application, but in debian/rules you will have to include these lines in proper places:

Code: Select all

python setup.py --root=foo
dh_pysupport
More informations you can find here: http://people.debian.org/~srivasta/manoj-policy/
It's also good idea to download some python application source from Debian and look into debian/ folder to check how it's done (:

Good luck!

harishankar
Posts: 75
Joined: 2006-09-17 06:43

Re: HowTo Build a Package from Source the Smart Way

#31 Post by harishankar »

Thanks Hadret. I've not yet found how exactly to use python-support.

Is there a full tutorial on this subject anywhere online? Every packaging tutorial for Debian uses the standard debian utils. Even the tutorials for python apps don't rely on python-support.

By the way, there is a confusion: they say python-support and python-central should be used only for packaging python libraries/modules and not application/scripts. Is this true?

harishankar
Posts: 75
Joined: 2006-09-17 06:43

Re: HowTo Build a Package from Source the Smart Way

#32 Post by harishankar »

Here's where I am really confused about packaging for Python applications and not a single answer have I found online:

1. How should privately used application modules be handled by distutils? Does distutils provide method for handling private modules? py_modules seem to be for public modules and doesn't seem the right way to package an application.
OR
2. Should I give up distutils and use a standard Makefile instead of using python distutils at all when packaging for Debian?

3. Which is ultimately the easiest and most sane method of python application packaging in Debian? There seem to be an infinite number of policies, rules and instructions which all add up to giving me a headache. :oops:

I've looked at so many documents and help files online, but not a single one is comprehensive enough to give me a clue about the right way of doing things. The official Debian documentation is confusing, very long and doesn't provide step by step instructions on Python packages.

harishankar
Posts: 75
Joined: 2006-09-17 06:43

Re: HowTo Build a Package from Source the Smart Way

#33 Post by harishankar »

I've gone ahead and used the distutils/CDBS method as outlined here: https://wiki.edubuntu.org/PackagingGuide/Python

I'm not sure it's the right "Debian" way, but it seems the easiest method so far.

User avatar
Hadret
Posts: 354
Joined: 2007-07-19 13:04
Location: Berlin

Re: HowTo Build a Package from Source the Smart Way

#34 Post by Hadret »

I'm packaging gwibber application for Debian and, as you may guess, it's python app. As I said before - you can use CDBS, but if you want to see your package in Debian repositories, sooner or later you will have to move it to python-support. If you still want to use CDBS you should look into gwibber package for Ubuntu - it's using CDBS (you could also collate Ubuntu package with Debian package). I think that page to which link I have provided earlier is a bit outdated, but it's still good source for learning how to build python application packages.

Let's look into gwibber source, shall we?

Code: Select all

dget -x http://mentors.debian.net/debian/pool/main/g/gwibber/gwibber_1.2.0+bzr352-1.dsc
Now let's go into debian/ folder and see what do we've got there. Files which are relevant for us are debian/control and debian/rules. Don't hesitate to open them in your favorite editor. Which parts are important? I will bold them:
debian/control

Source: gwibber
Section: gnome
Priority: optional
Build-Depends: debhelper (>=7)
Build-Depends-Indep: python (>=2.5.4), python-distutils-extra, python-support (>=1.0)
Standards-Version: 3.8.1

Package: gwibber
Architecture: all || assuming that your package is arch independent ||
Depends: ${misc:Depends}, ${python:Depends}
debian/rules

#!/usr/bin/make -f

build:

clean:
dh_testdir
dh_testroot

rm -rf build
dh_clean install-stamp

install: install-stamp

install-stamp:
dh_testdir
dh_testroot
dh_prep
dh_installdirs

python setup.py install --root=$(CURDIR)/debian/pkg-name || assuming that you've got setup.py for your python app ||
touch $@

binary-arch:

binary-indep: install
dh_testdir
dh_testroot
dh_installchangelogs
dh_installdocs
dh_installman pkg-name.1 || assuming that you've got man page for your package ||
dh_pysupport
dh_compress
dh_fixperms
dh_installdeb
dh_gencontrol
dh_md5sums
dh_builddeb

binary: binary-indep binary-arch
.PHONY: build clean binary-indep binary-arch binary install
As you see, it's not really harder than CDBS, is it? (:

Remember that this is only example and there are paths that you need to edit, and there might be some options that you will need enter but... I think that after some small changes, this should work (:

It's also a good idea to look into some other debian packages of python application sources.

harishankar
Posts: 75
Joined: 2006-09-17 06:43

Re: HowTo Build a Package from Source the Smart Way

#35 Post by harishankar »

Thanks.

The above is basically the Makefile approach + python distutils right?

I was greatly confused about why we invoke python-support here... This is where my confusion arose from.

It is said that you can use CDBS + python-support or python-central.

So what is the difference between the methods highlighted by you and the one at Ubuntuwiki? Is it only the fact that we are writing our own rules file and not importing the one from CDBS?

I'm sorry if I write incoherently and confused, because I am. I read too many docs recently and it's all swimming in my head.

User avatar
Hadret
Posts: 354
Joined: 2007-07-19 13:04
Location: Berlin

Re: HowTo Build a Package from Source the Smart Way

#36 Post by Hadret »

I'm sorry, it's my bad! I wasn't thinking about CDBS but python-central - use python-support NOT python-central. Using CDBS isn't something bad, but not every sponsor/maintainer likes it. Check this out for more informations: http://wiki.debian.org/Games/ToolsDiscuss/CDBS.

Here you can find some examples for python applications on Debian: http://wiki.debian.org/DebianPython/NewPolicy and http://wiki.debian.org/DebianPython.

One more time - I'm sorry for writing wrong informations about CDBS and making you more confused! :oops:

User avatar
stoffepojken
Posts: 705
Joined: 2007-01-25 01:21
Location: Stockholm, Sweden

Re: HowTo Build a Package from Source the Smart Way

#37 Post by stoffepojken »

Hadret wrote:I'm packaging gwibber application for Debian and, as you may guess, it's python app. As I said before - you can use CDBS, but if you want to see your package in Debian repositories, sooner or later you will have to move it to python-support. If you still want to use CDBS you should look into gwibber package for Ubuntu - it's using CDBS (you could also collate Ubuntu package with Debian package). I think that page to which link I have provided earlier is a bit outdated, but it's still good source for learning how to build python application packages.

Let's look into gwibber source, shall we?

Code: Select all

dget -x http://mentors.debian.net/debian/pool/main/g/gwibber/gwibber_1.2.0+bzr352-1.dsc
Now let's go into debian/ folder and see what do we've got there. Files which are relevant for us are debian/control and debian/rules. Don't hesitate to open them in your favorite editor. Which parts are important? I will bold them:
debian/control

Source: gwibber
Section: gnome
Priority: optional
Build-Depends: debhelper (>=7)
Build-Depends-Indep: python (>=2.5.4), python-distutils-extra, python-support (>=1.0)
Standards-Version: 3.8.1

Package: gwibber
Architecture: all || assuming that your package is arch independent ||
Depends: ${misc:Depends}, ${python:Depends}
debian/rules

#!/usr/bin/make -f

build:

clean:
dh_testdir
dh_testroot

rm -rf build
dh_clean install-stamp

install: install-stamp

install-stamp:
dh_testdir
dh_testroot
dh_prep
dh_installdirs

python setup.py install --root=$(CURDIR)/debian/pkg-name || assuming that you've got setup.py for your python app ||
touch $@

binary-arch:

binary-indep: install
dh_testdir
dh_testroot
dh_installchangelogs
dh_installdocs
dh_installman pkg-name.1 || assuming that you've got man page for your package ||
dh_pysupport
dh_compress
dh_fixperms
dh_installdeb
dh_gencontrol
dh_md5sums
dh_builddeb

binary: binary-indep binary-arch
.PHONY: build clean binary-indep binary-arch binary install
As you see, it's not really harder than CDBS, is it? (:

Remember that this is only example and there are paths that you need to edit, and there might be some options that you will need enter but... I think that after some small changes, this should work (:

It's also a good idea to look into some other debian packages of python application sources.
Hi Hadret

Will we see gwibber in Sid anytime soon?

harishankar
Posts: 75
Joined: 2006-09-17 06:43

Re: HowTo Build a Package from Source the Smart Way

#38 Post by harishankar »

Thanks. Now I understand. Yes, I think python-support seems to be the better bet at the moment.

Also I'll try to avoid using CDBS and doing it the manual way next time. I agree with a lot of the sentiments on the Debian wiki. CDBS *is* a bit of black magic and hides too much behind the scenes. However, it does provide a very simple method of creating a package when you just need a very simple and straightforward mechanism.

I think I'll look at some official Python software packaged in Debian and see how they do it.

User avatar
Hadret
Posts: 354
Joined: 2007-07-19 13:04
Location: Berlin

Re: HowTo Build a Package from Source the Smart Way

#39 Post by Hadret »

stoffepojken wrote:Will we see gwibber in Sid anytime soon?
I certainly hope so (: http://ftp-master.debian.org/new/gwibbe ... 350-2.html
harishankar wrote:I think I'll look at some official Python software packaged in Debian and see how they do it.
This is one of the best ways of learning IMO. You can also look here, find some package that you'd like to adopt into Debian and ask for help on mentors.d.n. When you will find mentor, he will explain to you all the black magic, not only the CDBS one (:

User avatar
kanonmat
Posts: 59
Joined: 2008-10-28 21:33
Location: Linköping, Sweden

Re: HowTo Build a Package from Source the Smart Way

#40 Post by kanonmat »

(Originally I was going to compile htpdate, as my ISP blocks ntp. What's up with that? o.O )
Following this guide I'm trying with the backintime example and get the following errors:
1. Some key error with gpgv. It doesn't help to add ED75F599 with seahorse. (apt-get source backintime as root gives an error too.)
2. dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2

Running "apt-get build-dep backintime" shows I have the needed dependencies.

Code: Select all

[~/bin/paket/backintime]$ apt-get source backintime
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Need to get 877kB of source archives.
Get:1 http://ftp.sunet.se sid/main backintime 0.9.26-1 (dsc) [1243B]
Get:2 http://ftp.sunet.se sid/main backintime 0.9.26-1 (tar) [872kB]
Get:3 http://ftp.sunet.se sid/main backintime 0.9.26-1 (diff) [3445B]
Fetched 877kB in 1s (783kB/s)      
gpgv: keyblock resource `/home/duke/.gnupg/trustedkeys.gpg': general error
gpgv: Signature made Tue 19 May 2009 10:17:24 PM CEST using DSA key ID ED75F599
gpgv: Can't check signature: public key not found
dpkg-source: warning: failed to verify signature on ./backintime_0.9.26-1.dsc
dpkg-source: info: extracting backintime in backintime-0.9.26
dpkg-source: info: unpacking backintime_0.9.26.orig.tar.gz
dpkg-source: info: applying backintime_0.9.26-1.diff.gz


[~/bin/paket/backintime]$ cd backintime-0.9.26/
[~/bin/paket/backintime/backintime-0.9.26]$ dpkg-buildpackage -rfakeroot -us -uc
dpkg-buildpackage: set CFLAGS to default value: -g -O2
dpkg-buildpackage: set CPPFLAGS to default value: 
dpkg-buildpackage: set LDFLAGS to default value: 
dpkg-buildpackage: set FFLAGS to default value: -g -O2
dpkg-buildpackage: set CXXFLAGS to default value: -g -O2
dpkg-buildpackage: source package backintime
dpkg-buildpackage: source version 0.9.26-1
dpkg-buildpackage: source changed by Jonathan Wiltshire <debian@jwiltshire.org.uk>
dpkg-buildpackage: host architecture amd64
 fakeroot debian/rules clean
dh clean 
   dh_testdir
   dh_auto_clean
   dh_clean
rm -rf locale common/po/*.mo
find /home/duke/bin/paket/backintime/backintime-0.9.26 -name "*\.py[co]" -delete
rm -f common/Makefile gnome/Makefile kde4/Makefile
 dpkg-source -b backintime-0.9.26
dpkg-source: info: using source format `1.0'
dpkg-source: info: building backintime using existing backintime_0.9.26.orig.tar.gz
dpkg-source: info: building backintime in backintime_0.9.26-1.diff.gz
dpkg-source: info: building backintime in backintime_0.9.26-1.dsc
 debian/rules build
dh build
   dh_testdir
   debian/rules override_dh_auto_configure
make[1]: Entering directory `/home/duke/bin/paket/backintime/backintime-0.9.26'
cd common && ./configure
All OK. Now run:
    make
    sudo make install
cd gnome && ./configure --no-check
All OK. Now run:
    make
    sudo make install
cd kde4 && ./configure --no-check 
All OK. Now run:
    make
    sudo make install
make[1]: Leaving directory `/home/duke/bin/paket/backintime/backintime-0.9.26'
   dh_auto_build
   dh_auto_test
 fakeroot debian/rules binary
dh binary
   dh_testroot
   dh_prep
   dh_installdirs
   debian/rules override_dh_auto_install
make[1]: Entering directory `/home/duke/bin/paket/backintime/backintime-0.9.26'
cd common && DESTDIR=../debian/backintime-common /usr/bin/make install
make[2]: Entering directory `/home/duke/bin/paket/backintime/backintime-0.9.26/common'
Makefile:3: *** missing separator.  Stop.
make[2]: Leaving directory `/home/duke/bin/paket/backintime/backintime-0.9.26/common'
make[1]: *** [override_dh_auto_install] Error 2
make[1]: Leaving directory `/home/duke/bin/paket/backintime/backintime-0.9.26'
make: *** [binary] Error 2
dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2


[~/bin/paket/backintime/backintime-0.9.26]$ ls
AUTHORS  common/  gnome/  LICENSE      maketar.sh  README  TRANSLATIONS      VERSION
CHANGES  debian/  kde4/   makedeb.sh*  plugins/    TODO    updateversion.sh
I have a look with tree. As I dont have gnome or kde, I try to run:

Code: Select all

[~/bin/paket/backintime/backintime-0.9.26]$ ./common/backintime &
python: can't open file '/usr/share/backintime/common/backintime.py': [Errno 2] No such file or directory
[1] 11890
There is a makedeb.sh and there are other executables, are those of any use?
phenomII965, gigabyte ma790xt-ud4p, 8600gt on desktop
hp 6910p laptop. athlon 3200+, asus an8e, 6600gt on spare desktop
sid user

Post Reply