Debian testing. KERNEL PANIC on boot after libc updates.

If none of the more specific forums is the right place to ask
Post Reply
Message
Author
help!
Posts: 3
Joined: 2013-02-18 18:49

Debian testing. KERNEL PANIC on boot after libc updates.

#1 Post by help! »

Hello.

I've just installed latest updates via synaptic on Debian testing x64 (with multiarch enabled) machine and got broken system. Here is the update log( I believe that libc and it's multiarch version were messed during the update process):

Code: Select all

Log started: 2013-02-18  22:44:14
(Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 99921 files and directories currently installed.)
Preparing to replace libc6-amd64 2.13-37 (using .../libc6-amd64_2.13-38_i386.deb) ...
Unpacking replacement libc6-amd64 ...
Replaced by files in installed package libc6:amd64 ...
Preparing to replace libc-bin 2.13-37 (using .../libc-bin_2.13-38_amd64.deb) ...
Unpacking replacement libc-bin ...
Processing triggers for man-db ...
Setting up libc-bin (2.13-38) ...
(Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 99921 files and directories currently installed.)
Preparing to replace libc6-i386 2.13-37 (using .../libc6-i386_2.13-38_amd64.deb) ...
Unpacking replacement libc6-i386 ...
Replaced by files in installed package libc6:i386 ...
Preparing to replace libc-dev-bin 2.13-37 (using .../libc-dev-bin_2.13-38_amd64.deb) ...
Unpacking replacement libc-dev-bin ...
Preparing to replace libc6-dev:amd64 2.13-37 (using .../libc6-dev_2.13-38_amd64.deb) ...
Unpacking replacement libc6-dev:amd64 ...
Preparing to replace libc6:amd64 2.13-37 (using .../libc6_2.13-38_amd64.deb) ...
De-configuring libc6:i386 ...
Unpacking replacement libc6:amd64 ...
dpkg-deb: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory
dpkg: error processing /var/cache/apt/archives/libc6_2.13-38_amd64.deb (--unpack):
 subprocess dpkg-deb --fsys-tarfile returned error exit status 127
dpkg-deb: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory
dpkg: error processing /var/cache/apt/archives/libc6_2.13-38_i386.deb (--unpack):
 subprocess dpkg-deb --control returned error exit status 127
Processing triggers for man-db ...
debconf: unable to initialize frontend: Gnome
debconf: (Can't load '/usr/lib/perl5/auto/Glib/Glib.so' for module Glib: libgobject-2.0.so.0: cannot open shared object file: No such file or directory at /usr/lib/perl/5.14/DynaLoader.pm line 184.)
debconf: falling back to frontend: Dialog
/usr/bin/mandb: error while loading shared libraries: libgdbm.so.3: cannot open shared object file: No such file or directory
Errors were encountered while processing:
 /var/cache/apt/archives/libc6_2.13-38_amd64.deb
 /var/cache/apt/archives/libc6_2.13-38_i386.deb
Log ended: 2013-02-18  22:44:28

Log started: 2013-02-18  22:45:42
/usr/bin/dpkg: error while loading shared libraries: libselinux.so.1: cannot open shared object file: No such file or directory
Log ended: 2013-02-18  22:45:42

Log started: 2013-02-18  22:46:33
/usr/bin/dpkg: error while loading shared libraries: libselinux.so.1: cannot open shared object file: No such file or directory
Log ended: 2013-02-18  22:46:33

Log started: 2013-02-18  22:46:51
/usr/bin/dpkg: error while loading shared libraries: libselinux.so.1: cannot open shared object file: No such file or directory
Log ended: 2013-02-18  22:46:51
When i try to boot the system, i get kernel panic and message like the last lines in the log about libselinux (no kern.log is written). Is there any way to fix such error?

User avatar
sunrat
Global Moderator
Global Moderator
Posts: 3679
Joined: 2006-08-29 09:12
Location: Melbourne, Australia
Has thanked: 3 times
Been thanked: 5 times

Re: Debian testing. KERNEL PANIC on boot after libc updates.

#2 Post by sunrat »

How did you enable multiarch?
“ computer users can be divided into 2 categories:
Those who have lost data
...and those who have not lost data YET ”
Remember to BACKUP!

lightspd
Posts: 13
Joined: 2013-02-19 02:17

Re: Debian testing. KERNEL PANIC on boot after libc updates.

#3 Post by lightspd »

Interesting enough, just had the same issue myself. Now I'm stuck at a point where I can't update and even if I try apt-get -f install I run into the issue you did with the error: dpkg-deb: error while loading shared libraries: libz.so.1.

Anyway, boot up debian install cd and go into recovery mode and run /sbin/ldconfig, that will more than likely fix your issue and allow you to boot, but you will be stuck at the same place I am. Hope that helps.


kmathern
Posts: 603
Joined: 2011-02-05 19:20

Re: Debian testing. KERNEL PANIC on boot after libc updates.

#5 Post by kmathern »

Did you guys do the libc6 upgrade/dist-upgrade while running from the desktop (gnome3, kde4, etc.) that had been started with a display manager like kdm or gdm3?

Sometimes during libc6 related upgrades a question/warning will popup that it needs to stop and restarting some services that are running (including kdm in my case). I think that can screw up the upgrade process. So when I see that libc6 related are included in the packages to be upgraded, I logout of the desktop session (for me that's KDE4), goto cli mode, kill the display manager (`service kdm stop`), and then do the upgrade/dist-upgrade. I did that yesterday on my 32bit install and it's working okay.

mirix
Posts: 41
Joined: 2010-01-08 11:18

Re: Debian testing. KERNEL PANIC on boot after libc updates.

#6 Post by mirix »

Did you guys do the libc6 upgrade/dist-upgrade while running from the desktop (gnome3, kde4, etc.) that had been started with a display manager like kdm or gdm3?
Yes, this was the case (with LightDM). But, in my case, there was not warning.

I just hope that this is not related to the ext4 bug with certain kernels... I have 3 HDs configured with LVM.

UPDATE: I am reinstalling the system (I had not done it for years).

lightspd
Posts: 13
Joined: 2013-02-19 02:17

Re: Debian testing. KERNEL PANIC on boot after libc updates.

#7 Post by lightspd »

mirix: Why are you reinstalling your system? It's not a difficult fix to get your system running again.

kmathern: Thanks, I'll try stopping gdm and then running the update again and see if it works. If that doesn't work, I'll just have to wait for a new libc package.

help!
Posts: 3
Joined: 2013-02-18 18:49

Re: Debian testing. KERNEL PANIC on boot after libc updates.

#8 Post by help! »

Well, after some investigation i have found, that ld.so.cache is missing. The reason i believe was that four libc packages were installed. Two for old biarch and two for new multiarch. Essentially only one copy for x84 and one for amd64 is required.

Partial solution is to manually make lib32/lib64 in / and in /usr simlinks to DIR/lib/ARCHLIBS and run ldconfig from livecd with appropriate -r option (all lib paths should be also listed in /etc/ld.so.conf). This broke proprietary nvidia drivers and package system state is also broken (so using apt with libc package cause similar problems again).

Possible reasons: bug somewhere in migration process to multiarach, libc* package control scrips bugs, or improper multiarch support from package manages (e.g. synaptic).

PLEASE, someone who has multiarch and has successfully installed all testing updates, post installed libc* package list, and /lib32 /lib64 /usr/lib32 /usr/lib64 file list. So other could know what is messed in theres systems.

mirix
Posts: 41
Joined: 2010-01-08 11:18

Re: Debian testing. KERNEL PANIC on boot after libc updates.

#9 Post by mirix »

Yes, I have reinstalled the system and it is already up and running. I have already resumed the two aborted calculations. All in all, I have only lost half an hour of my time and a few hours of computing time (the calculations need to check what has already been done before actually resuming).

I am kind of happy to reinstall because I had installed the system when Squeeze was still testing and therefore the base system was kind of messy.

The good thing is that I keep my data in separated partitions (including the compiled software and other software that is not installable from the repos via apt-get). This means that restoring the system is quite simple.

lightspd
Posts: 13
Joined: 2013-02-19 02:17

Re: Debian testing. KERNEL PANIC on boot after libc updates.

#10 Post by lightspd »

Ok, so tried installing with gdm stopped, but same issue.

help!: I'm not sure why you had to create symlinks, simply runing /sbin/ldconfig as root should have rebuilt the cache fine.

UPDATE:
If I had to guess the issue. This was changed in the preinst script. It was 2.13-5 before.

if dpkg --compare-versions "$preversion" lt 2.13-38; then
# upgrading from a pre-multiarch libc to a multiarch libc; we have
# to blow away /etc/ld.so.cache, otherwise the old unpacked libc
# is still first in the cache and segfaults when combined with
# our newly-unpacked ld.so. Do this last to avoid slowing down the
# rest of the upgrade. Version number bumped to 2.13-38 to also
# cover cache format upgrades for ARM.
rm -f /etc/ld.so.cache
fi

I've never created my own deb package, but I might just have to edit that and give it a try, since the change was for ARM.

UPDATE 2: Well, I was able to get it installed and working with building my own package, but it was really hacky and ugly. It's not something I would recommend doing if you don't have to.

help!
Posts: 3
Joined: 2013-02-18 18:49

Re: Debian testing. KERNEL PANIC on boot after libc updates.

#11 Post by help! »

I'm not sure why you had to create symlinks, simply runing /sbin/ldconfig as root should have rebuilt the cache fine.
Just to provide clean minimal data to boot the sytem. Debian scripts for whatever reason couldn't handle previous lib file layout.

I've removed so called "biarch" version of libc (several reboots with manual file moving), manually copy lib i386/amd64 files, boot to text mode, reinstall nvidia dkms and all i386 libs. Now every thing is working, including reinstallation of libc* packages from repo.

merkosh
Posts: 1
Joined: 2013-03-16 14:06

Re: Debian testing. KERNEL PANIC on boot after libc updates.

#12 Post by merkosh »

I had exactly the same problem and managed to fix this, doing the following:
- boot from rescue cd into installer environment
- mount root partition (/target)
- to recreate /etc/ld.so.cache, execute: /target/sbin/ldconfig -r /target
- chroot to /target
- provide apt with library path to that it can continue:
$ LD_LIBRARY_PATH=/lib/x86_64-linux-gnu/ apt-get -f install

install process completed, ld.so.cache was recreated and everything is up running again.
Hope this helps!

Post Reply