Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

[SOLVED] Diagnosing high cpu load, no xorg

If none of the specific sub-forums seem right for your thread, ask here.
Post Reply
Message
Author
n_hologram
Posts: 459
Joined: 2013-06-16 00:10

[SOLVED] Diagnosing high cpu load, no xorg

#1 Post by n_hologram »

Solution in final post. System was reproducing, over and over again, an unusual dhclient error message, which decided to run constantly because of a lingering sendmail file. The clue was in htop's "red bars" (kernel threads), which signaled that the issue was probably deeper than a single process. Once that file was removed, the system worked as expected.

My netbook, which functions as a small, unimpressive project "server," is reporting an abnormally high cpu load on the core and virtual core: between 50-60%, and not subsiding. On two-to-five-minute intervals, I hear the little fan whirr for a few seconds and subside. All of that would be fine, if I knew which program was consuming all the resources, but htop reports that only four programs are using cpu, and the values are ~4%, 2%, 1%, and 1%. Rebooting changed nothing. I'm looking for suggestions for narrowing the source of this issue.
Last edited by n_hologram on 2017-05-19 00:41, edited 4 times in total.
bester69 wrote:There is nothing to install in linux, from time to time i go to google searching for something fresh to install in linux, but, there is nothing
the crunkbong project: scripts, operating system, the list goes on...

User avatar
phenest
Posts: 1702
Joined: 2010-03-09 09:38
Location: The Matrix

Re: Diagnosing high cpu load, no xorg

#2 Post by phenest »

What program is showing the high cpu load?
ASRock H77 Pro4-M i7 3770K - 32GB RAM - Pioneer BDR-209D

n_hologram
Posts: 459
Joined: 2013-06-16 00:10

Re: Diagnosing high cpu load, no xorg

#3 Post by n_hologram »

phenest wrote:What program is showing the high cpu load?
dhclient is the 4%-er
bester69 wrote:There is nothing to install in linux, from time to time i go to google searching for something fresh to install in linux, but, there is nothing
the crunkbong project: scripts, operating system, the list goes on...

User avatar
phenest
Posts: 1702
Joined: 2010-03-09 09:38
Location: The Matrix

Re: Diagnosing high cpu load, no xorg

#4 Post by phenest »

I'll rephrase: What program did you use to measure the high cpu load?
ASRock H77 Pro4-M i7 3770K - 32GB RAM - Pioneer BDR-209D

n_hologram
Posts: 459
Joined: 2013-06-16 00:10

Re: Diagnosing high cpu load, no xorg

#5 Post by n_hologram »

phenest wrote:I'll rephrase: What program did you use to measure the high cpu load?
htop

EDIT: I also checked with top, same results: http://stackoverflow.com/questions/9229 ... ux#9229692

Code: Select all

top -bn1 | grep "Cpu(s)" | \
           sed "s/.*, *\([0-9.]*\)%* id.*/\1/" | \
           awk '{print 100 - $1"%"}
bester69 wrote:There is nothing to install in linux, from time to time i go to google searching for something fresh to install in linux, but, there is nothing
the crunkbong project: scripts, operating system, the list goes on...

User avatar
orythem27
Posts: 252
Joined: 2017-05-11 07:59
Location: P.R. China

Re: Diagnosing high cpu load, no xorg

#6 Post by orythem27 »

n_hologram wrote:between 50-60%, ...... only four programs are using cpu, and the values are ~4%, 2%, 1%, and 1%.
Use `htop` to verify that if it's indeed some kernel threads that is actually acting up. Look for the red portion in the CPU usage bar. See this page for detail.

For further investigation you can use the following tools to unveil more details.

User avatar
phenest
Posts: 1702
Joined: 2010-03-09 09:38
Location: The Matrix

Re: Diagnosing high cpu load, no xorg

#7 Post by phenest »

n_hologram wrote:My netbook, ... is reporting an abnormally high cpu load on the core and virtual core: between 50-60%, and not subsiding.
Reported by what?
n_hologram wrote:but htop reports that only four programs are using cpu, and the values are ~4%, 2%, 1%, and 1%.
I don't understand. If htop is showing low CPU usage < 10% combined, how do you know there is a 50-60% load?
ASRock H77 Pro4-M i7 3770K - 32GB RAM - Pioneer BDR-209D

User avatar
phenest
Posts: 1702
Joined: 2010-03-09 09:38
Location: The Matrix

Re: Diagnosing high cpu load, no xorg

#8 Post by phenest »

@ OP:
Please reply to this thread rather than PM'ing me. Your replies are helpful to others, not just me.

So you're saying that htop is reporting a total CPU load of 50-60%, but only 4 programs show any usage and they are < 10% combined. Yes?
ASRock H77 Pro4-M i7 3770K - 32GB RAM - Pioneer BDR-209D

n_hologram
Posts: 459
Joined: 2013-06-16 00:10

Re: Diagnosing high cpu load, no xorg

#9 Post by n_hologram »

phenest wrote:@ OP:
Please reply to this thread rather than PM'ing me. Your replies are helpful to others, not just me.
Sure thing:
I had already said I used htop, along with a "top" command, and they both yielded the same results. I thought "reported" would be an appropriate verb.

I also do not understand why it's doing this, but I don't know much about kernel threads because I've never had a reason to investigate them before. I intend to update the post once I get a chance to look into that.

It's also a pain because, and I'm not sure if this is related or not, but any sort of local or external access to that system (ssh, mpd, etc.) will just stop working until I reboot (after which it works as expected for awhile, and then doesn't), so I have to be physically present at that computer to check anything until that gets resolved. Which is annoying.
So you're saying that htop is reporting a total CPU load of 50-60%, but only 4 programs show any usage and they are < 10% combined. Yes?
Yes.

EDIT: Screenshot for your viewing pleasure.
bester69 wrote:There is nothing to install in linux, from time to time i go to google searching for something fresh to install in linux, but, there is nothing
the crunkbong project: scripts, operating system, the list goes on...

n_hologram
Posts: 459
Joined: 2013-06-16 00:10

Re: Diagnosing high cpu load, no xorg

#10 Post by n_hologram »

Alright, you guys are gonna love this. I have kismet installed and set-up in server mode, using the wireless card in monitor mode. I'm using sysv, and /etc/rc.local is configured as so:

Code: Select all

# Start the wifi card in monitor mode for Kismet reasons
airmon-ng start wlan0
/usr/bin/sudo -u pi "/usr/bin/kismet_server" -n --daemonize -p /home/pi/kismet_server
At boot, I watched the startup messages (I don't know the proper term for those), and noticed that, when it runs this command, it complains that dhclient and dhclient script might interfere -- which is strange, since when I run the airmon command as root (sudo su) within a console, it doesn't produce that error. Also, "~# killall -9 dhclient" solved the original issue, albeit superficially. So, I think, I'll just comment-out those two lines from rc.local. Sure enough, after a reboot, my cpu usage runs exactly as expected.

Now, fully intrigued, I uncomment those lines, reboot, and double-check htop. That screenshot I sent in my previous post, it shows a dhclient command constantly running, around 4%, but without those lines in rc.local it doesn't show any dhclient at all, and neither do any of my other Debian systems. I suddenly have an epiphany: why is it running the entire time? So, I try a simple "~# dhclient eth0", and in a normal system, it would show a simple, one time "RTNETLINK answers: File Exists". However, when I run the same command in my server, it gives me two messages, ad infinitum:

Code: Select all

...
/sbin/dhclient-script: 28: .: Can't open /usr/share/sendmail/dynamic
RTNETLINK answers: File exists
/sbin/dhclient-script: 28: .: Can't open /usr/share/sendmail/dynamic
RTNETLINK answers: File exists
...
until I CTRL+C. So, it appears the issue was that I had daemonized an error message, and the little atom processor wasn't happy with that.

This was, however, a nice tidbit of information. I ran across a thread which articulated solutions to the same issue. Sendmail was already removed through apt-get, but I guess a lingering file interfered with dhclient's normal behavior, so I went ahead and removed it:

Code: Select all

rm /etc/dhcp/dhclient-exit-hooks.d/sendmail
And now everything is up to normal. Thanks to everyone for helping out, however, I'm still curious as to why htop didn't show anything abnormal in the process list; and if that's normal to not show, I still wonder if there would have been a better way to narrow that without crapshooting through programs and bootscreens. Ah, well...
bester69 wrote:There is nothing to install in linux, from time to time i go to google searching for something fresh to install in linux, but, there is nothing
the crunkbong project: scripts, operating system, the list goes on...

User avatar
phenest
Posts: 1702
Joined: 2010-03-09 09:38
Location: The Matrix

Re: Diagnosing high cpu load, no xorg

#11 Post by phenest »

n_hologram wrote:however, I'm still curious as to why htop didn't show anything abnormal in the process list; and if that's normal to not show
Maybe it's because htop only refreshes about once a second and it misses it.
ASRock H77 Pro4-M i7 3770K - 32GB RAM - Pioneer BDR-209D

n_hologram
Posts: 459
Joined: 2013-06-16 00:10

Re: Diagnosing high cpu load, no xorg

#12 Post by n_hologram »

phenest wrote:
n_hologram wrote:however, I'm still curious as to why htop didn't show anything abnormal in the process list; and if that's normal to not show
Maybe it's because htop only refreshes about once a second and it misses it.
It would update with the same frequency as "while [ true ]; do echo 'hello'; done", and that loop shows up in htop/top as a bash process, 98% cpu use. I'm trying to understand why the error message I found wouldn't even show up at all, or if the culprit dhclient command was it, why it would only show up as using 4% of the cpu.
bester69 wrote:There is nothing to install in linux, from time to time i go to google searching for something fresh to install in linux, but, there is nothing
the crunkbong project: scripts, operating system, the list goes on...

Post Reply