Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Clocksource issues on Bullseye with recent 5.10.103-1 kernel

Linux Kernel, Network, and Services configuration.
Post Reply
Message
Author
Talkless
Posts: 41
Joined: 2013-05-13 10:18
Been thanked: 1 time

Clocksource issues on Bullseye with recent 5.10.103-1 kernel

#1 Post by Talkless »

Anyone else having syslog "spammed" with these kind of messages?

Code: Select all

/var/log/kern.log:Mar 27 00:11:27 hostname kernel: [785690.322271] clocksource: timekeeping watchdog on CPU1: hpet wd-wd read-back delay of 72425ns
/var/log/kern.log:Mar 27 00:11:27 hostname kernel: [785690.322282] clocksource: wd-tsc-wd read-back delay of 147714ns, clock-skew test skipped!
/var/log/kern.log:Mar 27 05:36:41 hostname kernel: [801603.852804] clocksource: timekeeping watchdog on CPU4: hpet wd-wd read-back delay of 51612ns
/var/log/kern.log:Mar 27 05:36:41 hostname kernel: [801603.852815] clocksource: wd-tsc-wd read-back delay of 114330ns, clock-skew test skipped!
/var/log/kern.log:Mar 27 09:58:48 hostname kernel: [817331.400178] clocksource: timekeeping watchdog on CPU3: hpet wd-wd read-back delay of 65371ns
/var/log/kern.log:Mar 27 09:58:48 hostname kernel: [817331.400197] clocksource: wd-tsc-wd read-back delay of 115866ns, clock-skew test skipped!
/var/log/kern.log:Mar 27 10:23:10 hostname kernel: [818792.906413] clocksource: timekeeping watchdog on CPU14: hpet wd-wd read-back delay of 55523ns
/var/log/kern.log:Mar 27 10:23:10 hostname kernel: [818792.906416] clocksource: wd-tsc-wd read-back delay of 100990ns, clock-skew test skipped!
/var/log/kern.log:Mar 28 07:05:35 hostname kernel: [893338.501186] clocksource: timekeeping watchdog on CPU1: hpet wd-wd read-back delay of 56012ns
/var/log/kern.log:Mar 28 07:05:35 hostname kernel: [893338.501189] clocksource: wd-tsc-wd read-back delay of 100222ns, clock-skew test skipped!
/var/log/kern.log:Mar 28 07:25:26 hostname kernel: [894529.027103] clocksource: timekeeping watchdog on CPU14: hpet wd-wd read-back delay of 60761ns
/var/log/kern.log:Mar 28 07:25:26 hostname kernel: [894529.027105] clocksource: wd-tsc-wd read-back delay of 103015ns, clock-skew test skipped!
/var/log/kern.log:Mar 28 07:25:33 hostname kernel: [894536.519111] clocksource: timekeeping watchdog on CPU13: hpet wd-wd read-back delay of 55244ns
/var/log/kern.log:Mar 28 07:25:33 hostname kernel: [894536.519115] clocksource: wd-tsc-wd read-back delay of 102596ns, clock-skew test skipped!
/var/log/kern.log:Mar 28 09:56:29 hostname kernel: [903592.016263] clocksource: timekeeping watchdog on CPU12: hpet wd-wd read-back delay of 60971ns
/var/log/kern.log:Mar 28 09:56:29 hostname kernel: [903592.016277] clocksource: wd-tsc-wd read-back delay of 100082ns, clock-skew test skipped!
/var/log/kern.log:Mar 28 11:27:06 hostname kernel: [909029.528086] clocksource: timekeeping watchdog on CPU7: hpet wd-wd read-back delay of 122920ns
/var/log/kern.log:Mar 28 11:27:06 hostname kernel: [909029.528108] clocksource: wd-tsc-wd read-back delay of 112514ns, clock-skew test skipped!
/var/log/kern.log:Mar 28 11:28:06 hostname kernel: [909089.528053] clocksource: timekeeping watchdog on CPU15: hpet wd-wd read-back delay of 54825ns
It started after 5.10.103-1 update. I have reported bug: https://bugs.debian.org/cgi-bin/bugrepo ... ug=1008585

I believe these changes in kernel might be responsible:
- clocksource: Reduce clocksource-skew threshold
- clocksource: Avoid accidental unstable marking of clocksources
I could just update my logcheck filters to simply ignore these logs, but maybe it's actually an issue and I should fix it, somehow..?

LE_746F6D617A7A69
Posts: 932
Joined: 2020-05-03 14:16
Has thanked: 7 times
Been thanked: 65 times

Re: Clocksource issues on Bullseye with recent 5.10.103-1 kernel

#2 Post by LE_746F6D617A7A69 »

I have various machines with Intel, AMD and ARM CPUs, but I haven't seen this message for years - most likely this regression is specific for Your particular CPU model only.

Can You tell what CPU model do You have in that HP ProLiant DL380 G6?
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed

Talkless
Posts: 41
Joined: 2013-05-13 10:18
Been thanked: 1 time

Re: Clocksource issues on Bullseye with recent 5.10.103-1 kernel

#3 Post by Talkless »

LE_746F6D617A7A69 wrote: 2022-04-12 20:16 Can You tell what CPU model do You have in that HP ProLiant DL380 G6?

Code: Select all

Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
Maybe this thread is talking about same issue:

https://lore.kernel.org/all/20211111000 ... P17-Gen-1/

Hard to tell, as it's hard to read kernel-mailing-list'ese :) .

LE_746F6D617A7A69
Posts: 932
Joined: 2020-05-03 14:16
Has thanked: 7 times
Been thanked: 65 times

Re: Clocksource issues on Bullseye with recent 5.10.103-1 kernel

#4 Post by LE_746F6D617A7A69 »

The TSC problems can be a result of CPU microcode upgrade/update (a regression) - while in this case the problem seem to be non-critical, You can always try to degrade the microcode to an earlier version (using debian-snapshots). That would allow to confirm whether it is a regression in the kernel or in the microcode ...
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed

Talkless
Posts: 41
Joined: 2013-05-13 10:18
Been thanked: 1 time

Re: Clocksource issues on Bullseye with recent 5.10.103-1 kernel

#5 Post by Talkless »

LE_746F6D617A7A69 wrote: 2022-04-13 20:32 The TSC problems can be a result of CPU microcode upgrade/update (a regression) - while in this case the problem seem to be non-critical, You can always try to degrade the microcode to an earlier version (using debian-snapshots). That would allow to confirm whether it is a regression in the kernel or in the microcode ...
I've downgraded intel-microcode:

Code: Select all

$ apt policy intel-microcode
intel-microcode:
  Installed: 3.20210608.2
  Candidate: 3.20220207.1~deb11u1
  Version table:
     3.20220207.1~deb11u1 500
        500 http://httpredir.debian.org/debian bullseye/non-free amd64 Packages
 *** 3.20210608.2 100
        100 /var/lib/dpkg/status
Tonight machines was rebooted, but still got that message:

Code: Select all

May 18 06:55:05 dl380 kernel: [25392.675842] clocksource: timekeeping watchdog on CPU0: hpet wd-wd read-back delay of 61180ns
May 18 06:55:05 dl380 kernel: [25392.675845] clocksource: wd-tsc-wd read-back delay of 122222ns, clock-skew test skipped!

Post Reply