Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

[Solved] High CPU utilization after update to Stretch - 100%

Linux Kernel, Network, and Services configuration.
Post Reply
Message
Author
Niko Z.
Posts: 15
Joined: 2015-07-08 05:25

[Solved] High CPU utilization after update to Stretch - 100%

#1 Post by Niko Z. »

After update to Stretch I have occasionally a very high CPU utilization (usually 100%).
I noticed that it happens mostly when I plug in a lan network cable for internet access.
Process cannot be killed by sudo kill PID, once it starts it stays at 100% and only a reboot fixes it (temporarily).

What I tried:

I have modified /etc/network/interface and /etc/NetworkManager/NetworkManager.conf to not make mention of eth0 and that seemed
to 'solve' the problem for a while, but the problem now occasionally happens even when I don't use a cable and rely on wireless instead.
At this point I don't know whether it is fundamentally a network issue.

I have also tried reinstalling the kernel and network-manager, doesn't seem to make a difference.
Could anyone point me in the right direction to help resolve this?

This is the content of /etc/network/interfaces:

Code: Select all

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface inet loopback

# The primary network interface
#allow-hotplug eth0
#iface eth0 inet dhcp
#commented out last two lines to prevent the system from automatically connecting 
This is the content of /etc/NetworkManager/NetworkManager.conf

Code: Select all

[main]
plugins=ifupdown,keyfile

[ifupdown]
managed=false
This is the content of /etc/apt/sources.list. The only external source is for infinality fonts, and that doesn't pull in additional dependencies.

Code: Select all

deb http://httpredir.debian.org/debian/ stretch main contrib non-free 
deb-src http://httpredir.debian.org/debian/ stretch main contrib non-free 

deb http://security.debian.org/ stretch/updates main contrib non-free
deb-src http://security.debian.org/ stretch/updates main contrib non-free

# stretch-updates, previously known as 'volatile'
deb http://httpredir.debian.org/debian/ stretch-updates main contrib non-free
deb-src http://httpredir.debian.org/debian/ stretch-updates main contrib non-free

#Infinality Fonts
deb http://ppa.launchpad.net/no1wantdthisname/ppa/ubuntu xenial main
deb-src http://ppa.launchpad.net/no1wantdthisname/ppa/ubuntu xenial main

#jessie backports
deb http://ftp.debian.org/debian stretch-backports main contrib
MSi laptop, Nvidia GEFORCE, mostly inactive (Bumblebee)

Code: Select all

 
                    Kernel: x86_64 Linux 4.9.0-3-amd64
                    CPU: Intel Core i7-4700MQ CPU @ 3.4GHz
                     GPU: Mesa DRI Intel(R) Haswell Mobile 
                    RAM: 2308MiB / 15972MiB
           
Last edited by Niko Z. on 2017-07-01 13:22, edited 1 time in total.

User avatar
habu
Posts: 63
Joined: 2016-03-13 09:07

Re: High CPU utilization after update to Stretch - kworker 1

#2 Post by habu »

haw you tried to search for rootkit " rkhunter" "chkrootkit"
debian-8.8
Lenovo Thinkpad: Intel(R) Core(TM) i7-3610QM
CPU 2.30GHz amd64

Niko Z.
Posts: 15
Joined: 2015-07-08 05:25

Re: High CPU utilization after update to Stretch - kworker 1

#3 Post by Niko Z. »

rkhunter check came up with a warning:

Warning: The command '/usr/bin/lwp-request' has been replaced by a script: /usr/bin/lwp-request: Perl script text executable
Same as above for /usr/bin/GET

Running

Code: Select all

dpkg -S /usr/bin/lwp-request
pointed to libwww-perl package. I have reinstalled it and rerun the rkhunter, but still same warnings come out.

Chkrootkit has flagged my Insync installation (google drive sync) but it is legit. In addition, it threw up a following warning.
It seems to be related to intel hardware acceleration.

Code: Select all

Checking `chkutmp'...                                        
The tty of the following user process(es) were not found  in /var/run/utmp !
! RUID          PID TTY    CMD
! ;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann    3553 ;3,8,3553;3,9,3553;3,10,3553;3,11,;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann 6,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann
! ;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann    3553 ;3,8,3553;3,9,3553;3,10,3553;3,11,;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann 6,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann
! ;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann    3553 ;3,8,3553;3,9,3553;3,10,3553;3,11,;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann 6,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann
! ;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann    3553 ;3,8,3553;3,9,3553;3,10,3553;3,11,;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann 6,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann
! ;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann    3553 ;3,8,3553;3,9,3553;3,10,3553;3,11,;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann 6,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann
! ;3,11,3553;3,12,3553;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --s    3553 53;3,6,3553;3,7,3553;3,8,3553;3,9,;3,11,3553;3,12,3553;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --s 4,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --s
! ;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann    3553 ;3,8,3553;3,9,3553;3,10,3553;3,11,;3,13,3553;3,14,3553;3,15,3553;3,16,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann 6,3553;4,0,3553;4,1,3553;4,2,3553;4,3,3553;4,4,3553;4,5,3553;4,6,3553;4,7,3553;4,8,3553;4,9,3553;4,10,3553;4,11,3553;4,12,3553;4,13,3553;4,14,3553;4,15,3553;4,16,3553 --disable-accelerated-video-decode --disable-webrtc-hw-vp8-encoding --service-request-chann
chkutmp: nothing deleted

User avatar
habu
Posts: 63
Joined: 2016-03-13 09:07

Re: High CPU utilization after update to Stretch - kworker 1

#4 Post by habu »

debian-8.8
Lenovo Thinkpad: Intel(R) Core(TM) i7-3610QM
CPU 2.30GHz amd64

Niko Z.
Posts: 15
Joined: 2015-07-08 05:25

Re: High CPU utilization after update to Stretch - kworker 1

#5 Post by Niko Z. »

The article suggests installing sniffers on unaffected machines and analyzing the traffic. This is not possible for me in present circumstance.

Would downgrading a kernel have any effect?

Niko Z.
Posts: 15
Joined: 2015-07-08 05:25

Re: High CPU utilization after update to Stretch - kworker 1

#6 Post by Niko Z. »

Code: Select all

ps axjf
Output of this command that the offending kworker process belongs to kthreadd.

Code: Select all

    0     2     0     0 ?           -1 S        0   0:00 [kthreadd]

 2  2718     0     0 ?           -1 R        0   4:32  \_ [kworker/7:0]


sgian
Posts: 24
Joined: 2017-04-10 16:37

Re: High CPU utilization after update to Stretch - kworker 1

#7 Post by sgian »

Check your system log for what was going on at the time of the problem. For example what I did when my cpu maxed out on stretch and froze up was enter in a terminal as root

Code: Select all

less /var/log/syslog
and page down to what was going on at the time of my problem. (Someone better than me at these commands can probably give a more efficient command to enter.) What caused this in my case was that IPv6 was enabled by default but my area has no IPv6 service. So occasionally the cpu would max out and freeze the computer while it tried to resolve that issue. So I set IPv6 to ignore in the gui for configuring my network, and it hasn't happened again so far.

Niko Z.
Posts: 15
Joined: 2015-07-08 05:25

Re: High CPU utilization after update to Stretch - kworker 1

#8 Post by Niko Z. »

It does seem to be network related, as messages refer to eth0, and I did notice problems usually appears when I
plug in lan network cable. I can also confirm that editing /etc/NetworkManager/NetworkManager.conf
to change ifupdown to manage = true and then restarting network service reliably reproduces the 100% kworker process.

I had the managed = true line in Debian Jessie without issues though, the problem appeared after update.

Code: Select all

[ifupdown]
managed=true

Code: Select all

Jun 22 00:05:19 fluxbox systemd[1]: Reloaded Anonymizing overlay network for TCP.
Jun 22 00:05:19 fluxbox kernel: [ 2289.948492] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2289.956684] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2289.964876] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2289.973069] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2289.981262] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2289.989454] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2289.997645] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2290.005837] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2290.014030] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2290.022222] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2290.030415] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2290.038607] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2290.046799] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2290.054992] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2290.063184] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting
Jun 22 00:05:19 fluxbox kernel: [ 2290.071377] alx 0000:04:00.0 eth0: fatal interrupt 0x4019607, resetting

sgian
Posts: 24
Joined: 2017-04-10 16:37

Re: High CPU utilization after update to Stretch - kworker 1

#9 Post by sgian »

I googled the action in the log and came up with this github report regarding tor and systemd. https://github.com/systemd/systemd/issues/567 Is this relevant to your issue? I've tried the tor browser but haven't installed tor itself separately so this might be beyond my experience.

Niko Z.
Posts: 15
Joined: 2015-07-08 05:25

Re: High CPU utilization after update to Stretch - kworker 1

#10 Post by Niko Z. »

I have since purged tor, but 100% kworker process still popped out every time I tried connecting my lan cable, and would sometimes appear randomly when using wireless as well.

I have tried torghost a couple months ago. Perhaps the anonymizing process that shows up in a log is a remainder of some sort, I was not running tor at a time that log was made.

As you pointed to systemd as being potentially part of the problem, I have tried reinstalling it, and that seems to have done the trick for the most part. I don't have weird 100% kworker process eating up the CPU anymore. However, the processor is still not quite as quiet as it used to be, kworker process pops out every now and then to take up a 1%-2% of CPU, but it doesn't escalate.

Niko Z.
Posts: 15
Joined: 2015-07-08 05:25

Re: High CPU utilization after update to Stretch - kworker 1

#11 Post by Niko Z. »

Well, the laptop was working all right on a lan network for a few hours, but now the 100% CPU started again.
Is it possible to downgrade a kernel in Stretch? I seem to see only one version is available.

Niko Z.
Posts: 15
Joined: 2015-07-08 05:25

Re: High CPU utilization after update to Stretch - kworker 1

#12 Post by Niko Z. »

The problem was due to Debian 9 renaming network interfaces. Eth0 became something else. I am not exactly sure what I did there during upgrade process from Debian 8, and whether it was highlighted sufficiently, but it didn't catch my attention at the time.
I found out about the issue by reading Distrowatch review of Stretch.

The solution was to:

Code: Select all

# mv /etc/udev/rules.d/70-persistent.net.rules /etc/udev/rules.d/70-persistent.net.rules.old
I also edited /etc/network/interfaces. I only have this now:

Code: Select all

auto
iface inet dhcp
I have a stable lan connection now and the CPU is quiet again.

More details found here:

http://forums.debian.net/viewtopic.php?t=122795

Post Reply