Howto: Set up and Maintain a Mixed Testing/Unstable System

Share your own howto's etc. Not for support questions!

Howto: Set up and Maintain a Mixed Testing/Unstable System

Postby rickh » 2007-06-02 00:54

This recent Howto started out as a simple explanation of the process of getting a program from an outside repository installed in Etch. It quickly became an in-depth discussion of the vagaries associated with running a mixed system. In the interest of making it more available to new users, I have undertaken the effort to consolidate the pertinent information from that thread, here. For those of you who find yourselves quoted here without attribution, sorry about that. All the cut and pasted portions of this thread came from the one linked above.

First, a few brief statements of introduction.

(1.) It is my opinion that Stable (Etch) should not be run as part of a mixed system under any circumstance. The developers have gone to great lengths to design and release a trouble free environment for people who want it to "just work," and introducing elements from outside repositories is just undermining that design. If you need a newer version of a program than is available from the Stable repositories, check to see if it is available from backports.org. (Instructions.) If it is not, you should compile and build it yourself within your own installation. (Assuming you want a package from Lenny in Etch, and your sources.list file contains a line like this...
deb-src ftp://ftp.us.debian.org/debian/ testing main non-free contrib
...run "# apt-get source <programname>" and see if the package builds against Etch first. Most of the time, it will.)

People who find themselves needing this sort of upgrade with any regularity should upgrade to Testing.

(2.) Not everyone should be running a mixed system. A commonly quoted piece of Debian Unstable advice is, "If it breaks, you get to keep both pieces." By the nature of your decision to run a mixed system, you have considered as many ramifications as possible, and judged yourself capable of managing it. The "normal" scenario would be to run a system that is primarily Testing with just a few applications drawn from Unstable. Mixed systems are particularly suitable for home desktops and hobby boxes which do not include critical data for which you have no backup, or on systems for which occasional downtime is acceptable. This essay is intended to show how it is done, in principle, not how well it will work for you.

(3.) Any examples I include here will assume that your mixed system is Testing/Unstable. Testing/Unstable mixes are very common and generally OK, since the way testing is updated is very similar to mixing Testing and Unstable. As the Stable release matures, Stable/Testing or Stable/Unstable mixes get more and more risky.

I prefer Aptitude as the program installer, and examples of Apt commands will reflect that. If you are capable of running a mixed system, you are capable of translating that to apt-get, or even Synaptic or Adept. If you are determined to run a mixed-Stable system, you should be able to make the necessary translations for that, as well.

Getting down to business...

The first source of information is the official Apt Howto. You probably have it installed on your system someplace and it should be bookmarked (along with the Aptitude User's Manual).

There are two regularly used methods of accomplishing this "mixing." First, the easy to understand, but "incorrect" method. We'll assume you are starting out with a sources.list file that looks something like this:
deb http://ftp.us.debian.org/debian/ testing main contrib non-free
deb-src http://ftp.us.debian.org/debian/ testing main

deb http://www.debian-multimedia.org testing main

deb http://security.debian.org/ testing/updates main contrib
deb-src http://security.debian.org/ testing/updates main contrib


#deb http://ftp.us.debian.org/debian/ unstable main contrib non-free
#deb-src http://ftp.us.debian.org/debian/ unstable main

That is a standard Testing (Lenny) installation with the Unstable (Sid) repository readily available, but commented (#) out, in case you decide to "mix" something.

When you discover that your preferred Desktop UI (Ratpoison) has a newer version in Unstable, and you just can't wait, you simply change the commenting, thusly:
#deb http://ftp.us.debian.org/debian/ testing main contrib non-free
#deb-src http://ftp.us.debian.org/debian/ testing main

deb http://www.debian-multimedia.org testing main

deb http://security.debian.org/ testing/updates main contrib
deb-src http://security.debian.org/ testing/updates main contrib

deb http://ftp.us.debian.org/debian/ unstable main contrib non-free
deb-src http://ftp.us.debian.org/debian/ unstable main

# aptitude update
# aptitude install ratpoison

... change the commenting back to its original form, "# aptitude update" ... again, and there you are; safely back in Testing with the new Desktop UI from Unstable.

Unfortunately, there are other considerations. Binary applications don't come from external repositories all by themselves. Usually, there are associated dependencies accompanying them which are upgrading libraries, etc. on your system, and which may or may not affect other installed programs. Depending on the difference between your installed system and the repository from which you are upgrading, the residual effect can resemble a veritable tsunami, upgrading half of your system.

The first key to resolving that problem is to pay attention to what dependencies are going to accompany your upgrade. If you decide to reverse an upgrade, you will need to know exactly what was replaced.

There is another issue, as well. Suppose that the day after you have happily updated your Desktop UI from Unstable a horrible security problem is discovered in it. A new version will be uploaded to Sid, but you will know nothing about it. So ... we've got to be a little more sophisticated.

Doing it right:

Let's go back to my sample sources.list file:
deb http://ftp.us.debian.org/debian/ testing main contrib non-free
deb-src http://ftp.us.debian.org/debian/ testing main

deb http://www.debian-multimedia.org testing main

deb http://security.debian.org/ testing/updates main contrib
deb-src http://security.debian.org/ testing/updates main contrib

deb http://ftp.us.debian.org/debian/ unstable main contrib non-free
deb-src http://ftp.us.debian.org/debian/ unstable main

Notice that the comments have now been removed from both Testing and Unstable. An "# aptitude update" (or "$ aptitude search") now would recognize all the available files in both Testing and Unstable. However, an "# aptitude upgrade" would upgrade everything to Unstable since that is where it would find the newest applications. That is not what we want.

Here is the key. We need to let Apt know which is our preferred update repository. That information comes from the /etc/apt/apt.conf file. If it doesn't exist, create it. It needs to include this line:

APT::Default-Release "testing";

With that in place, aptitude will first look at testing, and if it sees the program you have identified, it will look no further. In our scenario, since we know we want our application from Unstable, our command will be:

# aptitude install -t unstable ratpoison

All this is a beautiful thing, but it's not foolproof. At the beginning of the essay, I said, "...you have ... judged yourself capable of managing it." That "managing" is what we will look at next.

Other Considerations

Suppose that Testing is the default release, but the application in which we're interested exists only in Unstable. In that case, Aptitude will happily install it (and its dependencies) without any particular warning. This is further complicated by the fact that those dependencies, along with any programs you chose by name to install, are "marked" by Apt as OK to upgrade from Unstable. From now on (until your Unstable package moves to Testing), those programs will be updated from Sid as the updates become available.

The critical result of that behavior is that it will behoove you to be aware of what programs have been installed from your secondary repositories, so that you can be alert for subsequent upgrades from that repository which can easily happen without your realization. (Note: The best way I have discovered to check exactly what packages are currently installed from unstable is: $ apt-show-versions | grep unstable )

The next step in preventing such unforeseen updates is "pinning," but that is just beyond the scope of this Howto.

(Insertion: There is quite a bit of discussion in the comments below about the need for an /etc/apt/preferences file, and the "pinning" instructions it contains. I have been watching my own system very closely to see how a package upgrade is handled once you have chosen to upgrade it from Unstable. My considered opinion, at this point, is that /etc/apt/preferences is not necessary, unless you feel that you need absolute control over the source of future upgrades to the package in question.

My observation is that, without pinning, a package installed from Unstable will continue to get upgrades from unstable until it is migrated from Unstable to Testing. Once that happens, aptitude will change it's source for that package back to the default release (Testing). IMO, that's the way you should allow it to work. /Insertion.)

Many people also recommend the installation of apt-listbugs. That program will identify any bugs filed against a program you have requested before aptitude actually installs it. If you feel that a reported bug might affect your system adversely, it gives you the opportunity to back out at the very last moment. I don't use it myself because I am confident that aptitude's remove process will do a good job of reversing any installation disaster that I encounter.

One of the ways you can make yourself very useful running a mixed Testing/Unstable system is by reporting bugs. It's certainly not in any way obligatory, but if you have time on your hands, it's nice. Certainly, the developers and maintainers provide some testing before the applications are moved to Unstable, but their hardware and usage patterns are not just like yours, and bugs do emerge in specific environments. You can even simply request a new feature by sending a bug report with the "wishlist" severity.

A nice Howto for using the Debian Reportbug tool can be found here. You might want to be aware that the editing tool incorporated into reportbug seems to be vim, so if you're not comfortable with that, "vimtutor" might be a better first step. At the very least, you would be well advised to get accustomed to using the Debian Bug Tracking System.

All this is not intended to discourage people from setting up a mixed system, but rather to encourage them to go into it with eyes fully open. My "safe" system is a Testing/Unstable mix, but I'm very careful, indeed, about what is going on there during my weekly upgrades.
Last edited by rickh on 2008-08-07 21:54, edited 17 times in total.
Debian-Lenny/Sid 32/64
Desktop: Generic Core 2 Duo, EVGA 680i, Nvidia
Laptop: Generic Intel SIS/AC97
User avatar
rickh
 
Posts: 3475
Joined: 2006-06-29 02:13
Location: Albuquerque, NM USA

Postby Pobega » 2007-06-02 01:11

Following your walkthrough didn't work, until I messed with /etc/apt/preferences, which you forgot to mention.

http://debian-book-bg.openfmi.net/queue ... nning.html

Using the setup for "following testing" on that page, everything worked fine. Just thought you should know.

Nice writeup, I've been meaning to do this for a few days now (Since I have a couple of packages from unstable)
Jabber: pobega@gmail.com
Pronunciation: Poh - Bay - Guh
User avatar
Pobega
 
Posts: 870
Joined: 2007-01-04 04:30
Location: New York

Postby Lou » 2007-06-02 01:15

Thanks rickh, nice howto!

I reinstalled Etch and then compiled Ratpoison 1.4.1 :lol:
Wheezy - Ratpoison - vimperator - no DM
KISS - Keep It Simple, Stupid
Lou
 
Posts: 1767
Joined: 2006-05-08 02:15
Location: Panama

Postby rickh » 2007-06-02 01:19

Glad you approve, Lou.

Probega, I will check out your link, and may add some details. I don't want to get into specific details about pinning, tho. That's "the next step." I did try to include warnings about the possibility of getting farther out from Testing than was intended.
Debian-Lenny/Sid 32/64
Desktop: Generic Core 2 Duo, EVGA 680i, Nvidia
Laptop: Generic Intel SIS/AC97
User avatar
rickh
 
Posts: 3475
Joined: 2006-06-29 02:13
Location: Albuquerque, NM USA

Postby Pobega » 2007-06-02 01:28

Well, from what I can tell, without pinning you're just grabbing right from unstable, even with /etc/apt/apt.conf containing your default release.
Jabber: pobega@gmail.com
Pronunciation: Poh - Bay - Guh
User avatar
Pobega
 
Posts: 870
Joined: 2007-01-04 04:30
Location: New York

Postby rickh » 2007-06-02 01:46

without pinning you're just grabbing right from unstable, even with /etc/apt/apt.conf containing your default release.

Only for programs that you have chosen to install from Unstable. For instance, "# aptitude upgrade" will upgrade all your installed applications from Testing (default) except those you have previously upgraded from Unstable.
Debian-Lenny/Sid 32/64
Desktop: Generic Core 2 Duo, EVGA 680i, Nvidia
Laptop: Generic Intel SIS/AC97
User avatar
rickh
 
Posts: 3475
Joined: 2006-06-29 02:13
Location: Albuquerque, NM USA

Postby sinical » 2007-06-02 01:48

You do need to edit hte /etc/apt/preferences and do pinning, otherwise as Pobega said you just pull the latest package (which will be sid/lenny)
Every cloud has a silver lining, except for the mushroom shaped ones, which have a lining of Strontium 90.
---------------------------------------------
umop apisdn
User avatar
sinical
 
Posts: 1022
Joined: 2007-03-25 11:52

Postby rickh » 2007-06-02 01:59

You do need to edit hte /etc/apt/preferences and do pinning,
Again, not true, unless you want to prevent subsequent updates from Unstable for the particular application you chose to get from there once. The real danger is not so much a package that you choose to upgrade from Unstable, but the dependencies it brings along which you may not understand.
Debian-Lenny/Sid 32/64
Desktop: Generic Core 2 Duo, EVGA 680i, Nvidia
Laptop: Generic Intel SIS/AC97
User avatar
rickh
 
Posts: 3475
Joined: 2006-06-29 02:13
Location: Albuquerque, NM USA

Postby Pobega » 2007-06-02 02:20

It works fine now without /etc/apt/preferences, but before I put it there originally it was upgrading me to unstable. I'd at least put a sample /etc/apt/preferences in your HowTo, just for clarity purposes.
Jabber: pobega@gmail.com
Pronunciation: Poh - Bay - Guh
User avatar
Pobega
 
Posts: 870
Joined: 2007-01-04 04:30
Location: New York

Postby Lux » 2007-06-02 02:38

"apt-cache policy" is a useful command for checking the priorities of different package archives that you've enabled in your sources.list.
"Hit the philistines three times over the head with the Elisp reference manual."
-- Michael A. Petonic --
User avatar
Lux
 
Posts: 474
Joined: 2006-01-25 13:00
Location: Finland

Postby rickh » 2007-06-02 03:15

I'd at least put a sample /etc/apt/preferences in your HowTo, just for clarity purposes.


OK. I already had a link to the "pinning" section of the Apt Howto, but I strengthed that refererence by replacing:
The critical result of that behavior is that it will behoove you to be aware of what programs have been installed from your secondary repositories, so that you can be alert for subsequent upgrades from that repository which can easily happen without your realization. The next step in preventing such unforeseen updates is "pinning," but that is just beyond the scope of this Howto.

with:
The critical result of that behavior is that it will behoove you to be aware of what programs have been installed from your secondary repositories, so that you can be alert for subsequent upgrades from that repository which can easily happen without your realization.

The next step in preventing such unforeseen updates is "pinning," but that is just beyond the scope of this Howto. Nevertheless, I do suggest that you read quickly through that link, so you will have at least a general idea of what is involved in "the next step" for protecting the integrity of your installation.
Debian-Lenny/Sid 32/64
Desktop: Generic Core 2 Duo, EVGA 680i, Nvidia
Laptop: Generic Intel SIS/AC97
User avatar
rickh
 
Posts: 3475
Joined: 2006-06-29 02:13
Location: Albuquerque, NM USA

Postby diego1116 » 2007-06-03 16:59

Nice howto, Rickh!

I'll just add the links of two nice articles about mixed systems (the latter refers to Sarge, but it' still useful):

How I mix Debian testing, unstable and experimental
http://beranger.org/index.php?fullarticle=1062&page=3k

Using Sarge with backports and other sources (even sid)
http://www.beranger.org/index.php?article=931&page=3k
User avatar
diego1116
 
Posts: 361
Joined: 2007-03-28 17:49
Location: Santa Maria, RS, Brazil

Re: Howto: Set up and Maintain a Mixed Testing/Unstable Syst

Postby Telemachus » 2007-06-03 19:19

rickh wrote:(Assuming you want a package from Lenny in Etch, and your sources.list file contains a line like this...
deb-src ftp://ftp.us.debian.org/debian/ testing main non-free contrib
...run "# aptitude source <programname>" and see if the package builds against Etch first. Most of the time, it will.)

One quick thing: unless I'm wrong, "aptitude source" doesn't work. To get and build from source, you need to do "apt-get source packagename". It's a small point and maybe I'm wrong, but I figured I would mention it. When Chris suggested it on Lou's post, he used apt-get, and when I just tried it using aptitude, I got the "No Super Cow Powers" message. (I also can't find source building in aptitude's docs.)
User avatar
Telemachus
 
Posts: 4676
Joined: 2006-12-25 15:53

Postby rickh » 2007-06-05 02:39

...unless I'm wrong, "aptitude source" doesn't work. To get and build from source, you need to do "apt-get source packagename".

You are, of course, completely correct, and I have changed the original to reflect it.
Debian-Lenny/Sid 32/64
Desktop: Generic Core 2 Duo, EVGA 680i, Nvidia
Laptop: Generic Intel SIS/AC97
User avatar
rickh
 
Posts: 3475
Joined: 2006-06-29 02:13
Location: Albuquerque, NM USA

Postby Pobega » 2007-06-06 11:18

Why is it that whenever I upgrade Apt begins downloading unstable packages?

/etc/apt/sources.list wrote:deb http://ftp.debian.org/debian/ unstable main
deb-src http://ftp.debian.org/debian/ unstable main

deb http://ftp.debian.org/debian/ testing main
deb-src http://ftp.debian.org/debian/ testing main

deb http://security.debian.org/ testing/updates main
deb-src http://security.debian.org/ testing/updates main


/etc/apt/preferences wrote:cat /etc/apt/preferences
Package: *
Pin: release o=Debian,a=testing
Pin-Priority: 900

Package: *
Pin: release o=Debian,a=unstable
Pin-Priority: 300


/etc/apt/apt.conf wrote:APT::Default-Release "testing";


From aptitude upgrade

34 packages upgraded, 0 newly installed, 0 to remove and 28 not upgraded.
Need to get 28.2MB/28.5MB of archives. After unpacking 5645kB will be used.
Do you want to continue? [Y/n/?] y
Get:1 http://ftp.debian.org unstable/main dpkg 1.14.4 [2045kB]
5% [1 dpkg 1590345/2045kB 77%]


Everytime I do an upgrade I have to comment unstable out of my sources.list and it works fine.

34 packages upgraded, 0 newly installed, 0 to remove and 28 not upgraded.
Need to get 28.2MB/28.5MB of archives. After unpacking 5645kB will be used.
Do you want to continue? [Y/n/?] y
Get:1 http://ftp.debian.org testing/main dpkg 1.14.4 [2045kB]
6% [1 dpkg 1920105/2045kB 93%]
Jabber: pobega@gmail.com
Pronunciation: Poh - Bay - Guh
User avatar
Pobega
 
Posts: 870
Joined: 2007-01-04 04:30
Location: New York

Next

Return to Docs, Howtos, Tips & Tricks

Who is online

Users browsing this forum: No registered users and 12 guests

fashionable