Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Debian 11 fresh install - systemctl (ex: daemon reload) hangs / extremely slow

If none of the specific sub-forums seem right for your thread, ask here.
Post Reply
Message
Author
vom
Posts: 2
Joined: 2022-01-28 14:23

Debian 11 fresh install - systemctl (ex: daemon reload) hangs / extremely slow

#1 Post by vom »

Hello,

I've been pulling my hair out on this one. Also disclaimer - I'm not a systemd expert by any stretch but I'm trying to build that knowledge up.

I have a fresh install of Debian 11. This is a server, so no X/GUI. I have noticed sometimes (not consistent) when I install packages - it seems like the postinst portion hangs for several minutes. It eventually completes just fine. No errors visible. Also during this time, I cannot ssh in anew. My existing sessions are responsive, but a new session will not connect while in this hung state. (I'm guessing this is systemd's "login" mechanics being stuck as well during this time).

So seeing what's going on during this hang, I see that a lot of package installs eventually lead to a 'systemctl daemon-reload' behind the scenes. So I've been running that at random times, and frequently it will hang for minutes at a time.

This morning I did a daemon-reload, and managed to capture some data while this was running. The daemon-reload ran for 90 seconds

I increased systemd logging with:

Code: Select all

systemd-analyze set-log-level debug
I also noticed that during the hang (at the beginning specifically), This was running for some time:

Code: Select all

root      180486  0.0  0.0 167376  5956 ?        S    09:26   0:00 (sd-executor)
root      180496  0.0  0.0  13120  5552 ?        D    09:26   0:00  \_ /usr/lib/systemd/system-generators/systemd-fstab-generator /run/systemd/generator /run/systemd/generator.early /run/systemd/generator.late
Another thing that caught my eye, was that looking at logs, I get HUNDREDS of these:

Code: Select all

Jan 28 09:27:38 ice systemd[1]: Sent message type=signal sender=org.freedesktop.systemd1 destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=UnitNew cookie=634 reply_cookie=0 signature=so error
-name=n/a error-message=n/a
Jan 28 09:27:38 ice systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=UnitNew cookie=32733 reply_cookie=0 signature=so error-name=n/a error-mes
sage=n/a
Jan 28 09:27:38 ice systemd[1]: Sent message type=signal sender=org.freedesktop.systemd1 destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=UnitNew cookie=635 reply_cookie=0 signature=so error
-name=n/a error-message=n/a
Jan 28 09:27:38 ice systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=UnitNew cookie=32734 reply_cookie=0 signature=so error-name=n/a error-mes
sage=n/a
Jan 28 09:27:38 ice systemd[1]: Sent message type=signal sender=org.freedesktop.systemd1 destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=UnitNew cookie=636 reply_cookie=0 signature=so error
-name=n/a error-message=n/a
Jan 28 09:27:38 ice systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=UnitNew cookie=32735 reply_cookie=0 signature=so error-name=n/a error-mes
sage=n/a
In my various trips through Google troubleshooting this, I saw some folks talking about possible dbus issues (dbus-daemon etc). I don't think I have this issue, here is some info that:

Code: Select all

root@ice:/home/vom# busctl
NAME                         PID PROCESS        USER CONNECTION    UNIT                   SESSION DESCRIPTION
:1.0                           1 systemd        root :1.0          init.scope             -       -
:1.1                         585 systemd-logind root :1.1          systemd-logind.service -       -
:1.144                    180675 busctl         root :1.144        session-1.scope        1       -
:1.4                        2314 systemd        vom  :1.4          user@1000.service      -       -
org.freedesktop.DBus           1 systemd        root -             init.scope             -       -
org.freedesktop.hostname1      - -              -    (activatable) -                      -       -
org.freedesktop.locale1        - -              -    (activatable) -                      -       -
org.freedesktop.login1       585 systemd-logind root :1.1          systemd-logind.service -       -
org.freedesktop.network1       - -              -    (activatable) -                      -       -
org.freedesktop.resolve1       - -              -    (activatable) -                      -       -
org.freedesktop.systemd1       1 systemd        root :1.0          init.scope             -       -
org.freedesktop.timedate1      - -              -    (activatable) -                      -       -

root@ice:/home/vom# ps auxw | grep dbus | grep -v grep
message+     577  0.0  0.0   8220  3824 ?        Ss   Jan24   0:01 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
Has anyone ever ran into this ? If not - what else can I do in systemd land to troubleshoot ?

PS: One more theory. Possibly related to the fstab generator thing. I do have a few NFS mounts on this box. They are in fstab, they automatically mount at boot, no errors etc. They are very responsive and quick. I temporarily unmounted them just to see if that changes any behavior here.


Post Reply