Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

NetInstall package set changing too much

If none of the specific sub-forums seem right for your thread, ask here.
Post Reply
Message
Author
ajhlinuxuser
Posts: 11
Joined: 2014-12-17 14:03

NetInstall package set changing too much

#1 Post by ajhlinuxuser »

Hi,

Fifteen days ago I installed using debian-9.0.0-amd64-netinst.iso onto a VM. Today I installed another VM using the same ISO file. The package group selection options for both installs were only "SSH Server" and "Standard system utilities."

I am frustrated that the set of packages differs so much between the two VMs. I first noticed when I found that the 'mail' command was missing from the second VM.

Here is the output of a diff on the lists of packages for both VMs.

Code: Select all

50,53d49
< exim4
< exim4-base
< exim4-config
< exim4-daemon-light
68d63
< guile-2.0-libs:amd64
134d128
< libfribidi0:amd64
136d129
< libgc1c2:amd64
146d138
< libgsasl7
167d158
< libkyotocabinet16v5:amd64
174d164
< libltdl7:amd64
178d167
< liblzo2-2:amd64
181,182d169
< libmailutils5:amd64
< libmariadbclient18:amd64
196d182
< libntlm0:amd64
213d198
< libpython2.7:amd64
270,271d254
< mailutils
< mailutils-common
278d260
< mysql-common
299d280
< psmisc
Also, while I can understand that in the past fifteen days there was a need to add a few packages to the default selection, I really don't understand why there are now less packages than before.

Can anybody explain this? Is it supposed to be this way?

User avatar
acewiza
Posts: 357
Joined: 2013-05-28 12:38
Location: Out West

Re: NetInstall package set changing too much

#2 Post by acewiza »

If I understand what you are claiming, my observation would be that you must have overlooked something, because that is just not logically possible. Using the same install media with the same install options cannot produce anything but identical systems.

An old Einstein adage comes to mind: "repeating the same thing and expecting different results is one definition of insanity." Repeating the same thing and actually getting different results would be uncharted territory for my feeble mind. :lol:

Are you sure you didn't manually add or remove something from one of the VM's?
Nobody would ever ask questions If everyone possessed encyclopedic knowledge of the man pages.

User avatar
dasein
Posts: 7680
Joined: 2011-03-04 01:06
Location: Terra Incantationum

Re: NetInstall package set changing too much

#3 Post by dasein »

Can anybody explain this?
Multiple explanations are available and viable. The first few just off the top of my head include:
  • - PEBKAC (I know you think you did the exact same thing both times, but humans have been known to be wrong)

    - One or more errors reading the install medium

    - One or more remote hosts being unavailable during the second install

    - One or more packages missing due to server sync
You'll notice that none of these explanations involves the autonomous, spontaneous self-modification that you posit.

tl;dr: Your post makes a lot of assumptions that lack supporting evidence. Unverified assumption is not a particularly robust problem-solving tool.

ajhlinuxuser
Posts: 11
Joined: 2014-12-17 14:03

Re: NetInstall package set changing too much

#4 Post by ajhlinuxuser »

Hi,

No I am not insane.
If I understand what you are claiming, my observation would be that you must have overlooked something, because that is just not logically possible. Using the same install media with the same install options cannot produce anything but identical systems.

An old Einstein adage comes to mind: "repeating the same thing and expecting different results is one definition of insanity." Repeating the same thing and actually getting different results would be uncharted territory for my feeble mind. :lol:

Are you sure you didn't manually add or remove something from one of the VM's?

I am not doing the exact same operation and getting different results. There is a variable and it is time. I am using the net install ISO and unlike the CD/DVD install media, it relies heavily on the on-line repositories. Which I am guessing have changed with time. (What I really want to know is why remove important packages from the standard set?)
I know you think you did the exact same thing both times, but humans have been known to be wrong
I can assure you that I input the same selections both times. I did not manually add packages to the VM installed on July 12th. To make extra certain, I have run the same exact procedure another 3 times. (Used the TUI in all cases, never the graphical install)
  • Language: English
  • Country: Canada
  • Keyboard: American English
  • Hostname: debian#, where # has been a number from 1-5, Domain: local
  • root password, username and password identical between installs
  • Timezone: eastern
  • Partitioning: Guided-use entire disk, all files in one partition
  • Mirror: ftp.ca.debian.org, blank field for proxy, popcon=no
  • Software Selection: only "SSH Server" and "Std System Utilities"
  • GRUB installed to MBR
Yet somehow, between July 12 2017 and July 27/28 2017, the install produced systems with different package sets. Specifically, the systems installed on the 27th & 28th did not install 19 packages that the system on the 12th did install.
- One or more errors reading the install medium

- One or more remote hosts being unavailable during the second install

- One or more packages missing due to server sync
Errors reading the ISO are highly unlikely because it is being read directly from the hard drive by VirtualBox as a file. Were a read error to occur, it would produce an error message. Hosts becoming unavailable (like ftp.ca.debian.org) or packages missing from the repository would also produce an error message. I received no error messages during any of the installs done. Further, the packages are just plain absent, not partially installed nor corrupted.

So the only explanation my human brain can think of is that the package lists were changed (by some human) in that 15 day span, and somebody removed the previously mentioned packages, causing programs, including 'mail' to be missing from installs done on the 27th.

Further, to my knowledge, there is no way for a user using the Debian installer to specifically stop it from installing packages - some of which relate to mail - using the normal interface.

EDIT: I notice that in that span, a new version of the Netinstall ISO became available. This could be related. Hmm?

User avatar
dasein
Posts: 7680
Joined: 2011-03-04 01:06
Location: Terra Incantationum

Re: NetInstall package set changing too much

#5 Post by dasein »

ajhlinuxuser wrote:There is a variable and it is time.
There are lots of variables at play that you've not yet eliminated. By your own admission, you're guessing without supporting data that there is exactly one pertinent variable. But guesswork is not data.

If you're saying that you can reproduce the problem reliably, then there's at least one more step necessary before filing a bug report: choose a different install site (two different ones, actually, just to convince hard-nosed skeptics like me).

Lastly, IMO, you'd do well to lose the defensive attitude. You're making an extraordinary claim, which obliges you to offer up extraordinary evidence. Surely it's clear to you that what you've presented thus far is nowhere near that threshold.

edit: I see you posted an edit while I was typing. I can't speak to what the changes were in the netinstall .iso, but you just torpedoed your own argument. First, you claimed that the different results came from the exact same .iso, but now you're saying that you used two different .isos. Sorry, but you can't make both claims simultaneously. Plus, you've now introduced yet another uncontrolled variable into the mix, which further weakens your claims regarding causality.

Now perhaps you can understand why your claims have been met with such skepticism.

User avatar
GarryRicketson
Posts: 5644
Joined: 2015-01-20 22:16
Location: Durango, Mexico

Re: NetInstall package set changing too much

#6 Post by GarryRicketson »

Post by ajhlinuxuser »Further, to my knowledge, there is no way for a user using the Debian installer to specifically stop it from installing packages - some of which relate to mail - using the normal interface.
Actually there is, by not checking the selections, some packages are not installed,
only the packages selected are installed.

A possible explanation:
If the OP forgot to check the "standard utilities" as shown here:
Cick-to-view-Image
https://debian-handbook.info/browse/sta ... -1.7.12.18

Sometimes we all think we did something, or didn't, the OP may think they did select the
the "standard utilities" but didn't. Why not just try it again, and be more careful, be sure
to select what you want, ???
I think one can just run 'taskel' and add the missing selection ,..
You also can connect to the mirror you used, and look at the list of the packages
available, if there are actually some missing , then use a different mirror.
Also you could notify the person maintaining that mirror about it, if there really
are packages missing. There is nothing we, "debian users" can do about that.
Since the specific mirror used is not mentioned, nobody can verify if the packages
are there or not.

ajhlinuxuser
Posts: 11
Joined: 2014-12-17 14:03

Re: NetInstall package set changing too much

#7 Post by ajhlinuxuser »

dasein wrote:now you're saying that you used two different .isos. ...
I did not use the new ISO in the above report. I merely stated that it was released by debian.org in the same time span where the package set changed, which could be related to the overall problem. I used the same ISO.
GarryRicketson wrote:If the OP forgot to check the "standard utilities" ...

Sometimes we all think we did something, or didn't, the OP may think they did select the
the "standard utilities" but didn't. Why not just try it again, and be more careful ...

Since the specific mirror used is not mentioned, nobody can verify if the packages
are there or not.
1. Are you saying it is more likely that I de-selected the same item in 4 different instances without being aware of it, rather than just leaving it selected? (it is selected by default) Further, I did specifically report that I installed these systems with "Standard System Utilities" set being selected.

2. I did mention the mirror that I used in all the installs up to this point: ftp.ca.debian.org.


I don't know why there is such a tendency to assume that I am incompetent or insane, or why there is such resistance to the idea that there might actually be a changed package set. The friendly approach would be to take me for my word that at least something isn't working as I expected. Is this forum's policy that suspicion of users is more important than friendliness?


Anyway, I am still interested in resolving this. I have just completed 3 new installs using the new ISO I just downloaded (and verified). I repeat, I just downloaded the new ISO - debian-9.1.0-amd64-netinst.iso - which was not used in the previous installs.

As requested, I made use of two different mirrors when installing, for the next three VMs that I created.

The first install attempt VM #6 used the same mirror as prior installs: ftp.ca.debian.org.
The second install attempt VM #7 used mirror.csclub.uwaterloo.ca
The third install attempt VM #8 used debian.mirror.netelligent.ca

Apart from mirror selctions the installations were conducted using the TUI as before, the selections were identical to the prior installs, and I did not unconsciously de-select the "standard system utilities" 3 times in a row.

All three installs have a total of 339 packages, and the exact same 19 packages are absent compared to the July 12th install. (see the diff in the original post)
Last edited by ajhlinuxuser on 2017-07-28 23:54, edited 1 time in total.

User avatar
acewiza
Posts: 357
Joined: 2013-05-28 12:38
Location: Out West

Re: NetInstall package set changing too much

#8 Post by acewiza »

GarryRicketson wrote:There is nothing we, "debian users" can do about that.
Since the specific mirror used is not mentioned, nobody can verify if the packages
are there or not.
Wait - isn't that why they call it a "mirror?" So barring the rare anomalies mentioned above, we can have a high confidence level in repo integrity? Like that would be around the last thing to look at trying to solve this conundrum? :lol:
Nobody would ever ask questions If everyone possessed encyclopedic knowledge of the man pages.

User avatar
dasein
Posts: 7680
Joined: 2011-03-04 01:06
Location: Terra Incantationum

Re: NetInstall package set changing too much

#9 Post by dasein »

ajhlinuxuser wrote:I don't know why there is...such resistance to the idea that there might actually be a changed package set.
Then you don't understand Debian's release model. Arbitrary changes to the Stable release are basically nonexistent.
ajhlinuxuser wrote:The friendly approach would be to take me for my word that at least something isn't working as I expected. Is this forum's policy that suspicion of users is more important than friendliness?
You need to stop confusing skepticism with personal attack. I'll say it again: you're making an extraordinary claim, which obliges you to offer extraordinary evidence. Guesswork doesn't meet that threshold (no, not even yours). And the longer you persist in whining about that fact, the less is going to be accomplished, and the more potential helpers you will alienate.

If you want grouphugs and life-affirmations, join an encounter group. This is a technical forum, and effective problem-solving requires a commitment to Occam's Razor.
All three installs have a total of 339 packages, and the exact same 19 packages are absent compared to the July 12th install. (see the diff in the original post)
Excellent. Although this result refutes your speculation that changes to the .iso are somehow related to your issue, this is progress. Reproducible problems are fixable.

The list of packages being left out strikes me as odd. AFIAK, Debian has insisted on installing some sort of mail server on every install since Wheezy (and maybe even Squeeze). I recall some mention of a transition from MySQL to MariaDB, but nothing that would suggest that *SQL was being removed from a minimal install.

Your best next step is to check for an existing bug report against the installer (just to be sure you don't file a duplicate bug report). If you don't find one, then a bug report is not only in order, it's a service to the entire community.

ajhlinuxuser
Posts: 11
Joined: 2014-12-17 14:03

Re: NetInstall package set changing too much

#10 Post by ajhlinuxuser »

Hi,

I have filed a bug report. I was unable to find a similar bug report in the list that 'reportbug' command presented to me.

https://bugs.debian.org/cgi-bin/bugrepo ... bug=870030

Just to add one clarification, I have since been able to manually install the missing packages on all the VMs where they were omitted by the NetInstaller. I never claimed to believe that the packages had been removed from the mirrors, I claimed that the installer was just not downloading or installing them, when based on the July 12th install, I believe strongly that it should.

ajhlinuxuser
Posts: 11
Joined: 2014-12-17 14:03

Re: NetInstall package set changing too much

#11 Post by ajhlinuxuser »

GarryRicketson wrote: ...by not checking the selections, some packages are not installed,
only the packages selected are installed.

A possible explanation:
If the OP forgot to check the "standard utilities" ...

Sometimes we all think we did something, or didn't, the OP may think they did select the
the "standard utilities" but didn't. Why not just try it again, and be more careful, be sure
to select what you want, ???
...
To prove that I did not de-select the 'standard system utilities' set, I have installed a 9th VM and did just that. I entered the same selections as before, chose the mirror ftp.ca.debian.org, and ONLY the 'SSH Server' package. The result is a system with a total of 257 packages installed, which compares to 339 installed in the 2nd, 3rd, 4th, 5th, 6th, 7th and 8th installs that I previously did.

Code: Select all

7d6
< apt-listchanges
13,14d11
< bash-completion
< bind9-host
20,21d16
< bzip2
< ca-certificates
33d27
< debian-faq
35d28
< dh-python
40d32
< distro-info-data
43,44d34
< dnsutils
< doc-debian
50d39
< file
53d41
< geoip-database
59d46
< groff-base
65,66d51
< hdparm
< host
82d66
< iso-codes
91d74
< less
100d82
< libbind9-140:amd64
109d90
< libclass-isa-perl
112d92
< libcurl3-gnutls:amd64
120d99
< libdns162:amd64
133d111
< libgeoip1:amd64
148,150d125
< libisc160:amd64
< libisccc140:amd64
< libisccfg140:amd64
159,160d133
< libldap-2.4-2:amd64
< libldap-common
162d134
< liblockfile-bin
165d136
< liblwres141:amd64
168,169d138
< libmagic-mgc
< libmagic1:amd64
172d140
< libmpdec2:amd64
179d146
< libnghttp2-14:amd64
192d158
< libperl5.24:amd64
198,203d163
< libpython-stdlib:amd64
< libpython2.7-minimal:amd64
< libpython2.7-stdlib:amd64
< libpython3-stdlib:amd64
< libpython3.5-minimal:amd64
< libpython3.5-stdlib:amd64
205,208d164
< librtmp1:amd64
< libsasl2-2:amd64
< libsasl2-modules:amd64
< libsasl2-modules-db:amd64
218d173
< libssh2-1:amd64
222d176
< libswitch-perl
253,256d206
< lsb-release
< lsof
< man-db
< manpages
258d207
< mime-support
266d214
< netcat-traditional
270d217
< openssl
274d220
< perl
276d221
< perl-modules-5.24
278d222
< powermgmt-base
281,301d224
< python
< python-apt-common
< python-minimal
< python2.7
< python2.7-minimal
< python3
< python3-apt
< python3-chardet
< python3-debian
< python3-debianbts
< python3-httplib2
< python3-minimal
< python3-pkg-resources
< python3-pycurl
< python3-pysimplesoap
< python3-reportbug
< python3-requests
< python3-six
< python3-urllib3
< python3.5
< python3.5-minimal
303,304d225
< rename
< reportbug
320,321d240
< telnet
< traceroute
341d259
< xz-utils
That means that de-selecting the 'standard system utilities' would have resulted in 82 packages omitted in addition to the ones that I have reported NetInstaller for not installing. I'm only talking about 19 packages, so that shows that I did, in fact, leave that package set selected.


ajhlinuxuser
Posts: 11
Joined: 2014-12-17 14:03

Re: NetInstall package set changing too much

#13 Post by ajhlinuxuser »

dasein wrote:Image
I'm sorry?



Anyway, so more information:

I was trying to install the 'standard system utilities' on VM #9, which I had intentionally omitted, and upon discovering that they aren't available through the tasksel or tasksel --new-install commands, I went searching for the way to do it.

I found on https://wiki.debian.org/tasksel#A.22standard.22_task that the standard set can be discovered running aptitude search ~pstandard ~prequired ~pimportant -F%p which, I understand, are categories designed around the priority of the packages.

The result of that command for me was:

Code: Select all

adduser
apt
apt-listchanges
apt-utils
base-files
base-passwd
bash
bash-completion
bind9-host
bsdmainutils
bsdutils
bzip2
coreutils
cpio
cron
dash
dbus
debconf
debconf-i18n
debian-archive-keyring
debian-faq
debianutils
diffutils
dmidecode
dnsutils
doc-debian
dpkg
e2fslibs
e2fsprogs
file
findutils
gcc-6-base
gettext-base
gnupg
gpgv
grep
groff-base
gzip
hdparm
host
hostname
ifupdown
init
init-system-helpers
iproute2
iptables
iputils-ping
isc-dhcp-client
isc-dhcp-common
kmod
krb5-locales
less
libacl1
libapt-inst2.0
libapt-pkg5.0
libasprintf0v5
libattr1
libbind9-140
libblkid1
libbz2-1.0
libc-bin
libc-l10n
libc6
libcap2
libclass-isa-perl
libcomerr2
libdb5.3
libdns162
libedit2
libestr0
libevent-2.0-5
libfdisk1
libgc1c2
libgcc1
libgcrypt20
libgdbm3
libgnutls-openssl27
libgnutls30
libgpg-error0
libgpm2
libgssapi-krb5-2
libgssglue1
libgssrpc4
libidn11
libirs141
libisc160
libk5crypto3
libkadm5clnt-mit11
libkadm5srv-mit11
libkdb5-8
libkeyutils1
libkmod2
libkrad0
libkrb5-3
libkrb5support0
libldap-2.4-2
libldap-common
liblockfile-bin
liblockfile1
liblogging-stdlog0
liblwres141
liblzma5
libmagic-mgc
libmagic1
libmount1
libncursesw5
libnewt0.52
libnfsidmap2
libp11-kit0
libpam-modules
libpam-modules-bin
libpam-runtime
libpam-systemd
libpam0g
libpci3
libpcre3
libpipeline1
libpopt0
libprocps6
libreadline7
librpcsecgss3
libsasl2-2
libsasl2-modules-db
libselinux1
libsepol1
libslang2
libsmartcols1
libsqlite3-0
libss2
libssl1.0.2
libssl1.1
libstdc++6
libswitch-perl
libtasn1-6
libtinfo5
libtirpc1
libtokyocabinet9
libudev1
libuuid1
libwgdb0
libwrap0
libxapian30
libxml2
locales
login
logrotate
lsb-base
lsof
man-db
manpages
mawk
mime-support
mount
multiarch-support
nano
ncurses-base
ncurses-bin
ncurses-term
netbase
netcat-traditional
openssh-client
passwd
pciutils
perl
perl-base
perl-modules-5.24
procps
python
python-minimal
python2.7
python3-reportbug
readline-common
reportbug
rsyslog
sed
sensible-utils
systemd
systemd-sysv
sysvinit-utils
tar
tasksel
tasksel-data
telnet
traceroute
tzdata
ucf
udev
util-linux
vim-common
vim-tiny
wamerican
wget
whiptail
xz-utils
zlib1g
I ran a check for the missing packages, and as of July 29 2017, only one (libgc1c2) of them are on this list. Suspecting that the other 18 packages had their priorities changed or removed since July 12, I checked using apt-cache show. And indeed, they are set to Priority: optional.

Code: Select all

Package: exim4
Version: 4.89-2+deb9u1
Priority: optional

Package: exim4-base
Source: exim4
Version: 4.89-2+deb9u1
Priority: optional

Package: exim4-config
Source: exim4
Version: 4.89-2+deb9u1
Priority: optional

Package: exim4-daemon-light
Source: exim4
Version: 4.89-2+deb9u1
Priority: optional

Package: guile-2.0-libs
Source: guile-2.0
Version: 2.0.13+1-4
Priority: optional

Package: libfribidi0
Source: fribidi (0.19.7-1)
Version: 0.19.7-1+b1
Priority: optional

Package: libgc1c2
Source: libgc
Version: 1:7.4.2-8
Priority: standard

Package: libgsasl7
Source: gsasl (1.8.0-8)
Version: 1.8.0-8+b2
Priority: optional

Package: libkyotocabinet16v5
Source: kyotocabinet (1.2.76-4.2)
Version: 1.2.76-4.2+b1
Priority: optional

Package: libltdl7
Source: libtool
Version: 2.4.6-2
Priority: optional

Package: liblzo2-2
Source: lzo2 (2.08-1.2)
Version: 2.08-1.2+b2
Priority: optional

Package: libmailutils5
Source: mailutils
Version: 1:3.1.1-1
Priority: optional

Package: libmariadbclient18
Source: mariadb-10.1
Version: 10.1.23-9+deb9u1
Priority: optional

Package: libntlm0
Source: libntlm
Version: 1.4-8
Priority: optional

Package: libpython2.7
Source: python2.7
Version: 2.7.13-2
Priority: optional

Package: mailutils
Version: 1:3.1.1-1
Priority: optional

Package: mailutils-common
Source: mailutils
Version: 1:3.1.1-1
Priority: optional

Package: mysql-common
Source: mysql-defaults (1.0.2)
Version: 5.8+1.0.2
Priority: optional

Package: psmisc
Source: psmisc (22.21-2.1)
Version: 22.21-2.1+b2
Priority: optional
It looks like this change occured very recently. I picked exim4 and searched for its past priority using Google, and I found references as recent as 22 days ago that claimed it's priority was standard. The following sites have made such claims: Well, guess my bug report won't do much good. Unless this priority change was an accident, it looks like the people at debian.org chose to omit packages related to mail from the 'standard system utilities' of a Debian install. I don't like the change, but whatever. Not much I can do about it.

ajhlinuxuser
Posts: 11
Joined: 2014-12-17 14:03

Re: NetInstall package set changing too much

#14 Post by ajhlinuxuser »

A year later, there is still a bad taste in my mouth from the attitude of people who confronted me in this thread. I find it interesting that they went completely silent when the evidence was on the table. All this talk of "technical discussion" went out the window when there was nothing left to tease me about.

It is amazing; just one forum thread and I have concluded that the entire Debian community pride themselves with being inconsiderate and arrogant. If there's any nice people here, i'd like to hear from you. Otherwise I'll have to assume there aren't any.

I'm not expecting a reply; you're all probably too busy insulting and belittling other users with legitimate questions.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: NetInstall package set changing too much

#15 Post by Head_on_a_Stick »

This issue has been discussed before:

http://forums.debian.net/viewtopic.php?t=45804

I'm going to blame your experiences on VirtualBox 'cos that's a piece of crap — can you reproduce this under a credible virtualisation system? QEMU/KVM is very good :)

EDIT: oh, and the insulting and belittling remarks are features of this forum: why not just enjoy them as entertainment, like the rest of us do? :mrgreen:
deadbang

CwF
Global Moderator
Global Moderator
Posts: 2638
Joined: 2018-06-20 15:16
Location: Colorado
Has thanked: 41 times
Been thanked: 192 times

Re: NetInstall package set changing too much

#16 Post by CwF »

I'd second the native qemu/kvm notion. And above, the term 'image' may refer to 8 episodes of wasted time, ie; instal, then image, then play. You could try the package deselect to iron out your minimal install, and then carry forward that package selection to your next install. I'd imagine the results do vary over time as packages and dependencies change slightly. There are also those times of year when things change rapidly, then those period of seeming stagnation. From the same netinst, try the same thing once a week starting Jan1. I think all 8 might be a little different. In Aug/Sept the results might be similar. I forgot the question...

Post Reply