Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

[Solved] esoteric taskset issue

Linux Kernel, Network, and Services configuration.
Post Reply
Message
Author
User avatar
kent_dorfman766
Posts: 535
Joined: 2022-12-16 06:34
Location: socialist states of america
Has thanked: 57 times
Been thanked: 70 times

[Solved] esoteric taskset issue

#1 Post by kent_dorfman766 »

I have been trying to balance tasks across my CPUs so that the CPU0 memory channel doesnt run so hot. To that end I added

ExecStart=taskset 0xf0 /usr/bin/lightdm to the unit file so that the greeter and any of its children should be tied to the cores of CPU1.

I then view the results in htop where htop displays processes, but also threads within each process. It looks mostly correct but the tree display shows a couple of lightdm Xorg threads with the wrong core bindings as 0 and 2 instead of 4-7, like they should be.
I added the screen dump attachment.


Any thoughts on why a couple of the threads are bound to the wrong cores?
Attachments
screen.jpg
Last edited by kent_dorfman766 on 2023-05-24 15:35, edited 1 time in total.

CwF
Global Moderator
Global Moderator
Posts: 2638
Joined: 2018-06-20 15:16
Location: Colorado
Has thanked: 41 times
Been thanked: 192 times

Re: [Software] esoteric taskset issue

#2 Post by CwF »

If we are talking a single numa group, frankly I don't think manual user space clock and affinity tools will have any useful effect. If you want to tone it down, shut off the threads. If I'm reading this right 4-7 are threads.

when there is more physical space between things, then numactl comes into play and can help control power/thermal factors. Within one 'die' the cpu itself will make better choices. It can and does move execution around due to thermal/power limits. Anymore we don't know better! This has been the case for most cpu's in the last decade.

User avatar
kent_dorfman766
Posts: 535
Joined: 2022-12-16 06:34
Location: socialist states of america
Has thanked: 57 times
Been thanked: 70 times

Re: [Software] esoteric taskset issue

#3 Post by kent_dorfman766 »

CwF wrote: 2023-05-23 22:45 If we are talking a single numa group, frankly I don't think manual user space clock and affinity tools will have any useful effect. If you want to tone it down, shut off the threads. If I'm reading this right 4-7 are threads.

when there is more physical space between things, then numactl comes into play and can help control power/thermal factors. Within one 'die' the cpu itself will make better choices. It can and does move execution around due to thermal/power limits. Anymore we don't know better! This has been the case for most cpu's in the last decade.
It is a dual CPU SMP machine.

My understanding is that taskset should "tighly bind" a process, all it's children, and any concurrent threads of execution within that process by a defined affinity mask.

What I'm seeing is that a "couple" threads are being reported as renegades and running on other cores. 0-3 is cpu0 and 4-7 is cpu1. My taskset config should make EVERYTHING under lightdm run on cpu1, but I'd got a couple renegades threads that I cannot account for.

CwF
Global Moderator
Global Moderator
Posts: 2638
Joined: 2018-06-20 15:16
Location: Colorado
Has thanked: 41 times
Been thanked: 192 times

Re: [Software] esoteric taskset issue

#4 Post by CwF »

Well sure, you have right idea but miss my point maybe that per socket control is more appropriate than per core control. Let the socket do what it likes with the cores. So, numactl. If pci-e and memory controllers are per socket then that forces what cores do what, so a gpu in a slot has a socket to work on no matter what we say. Your gpu may be in numa zone 1, cpu1, that's common. If it's older and those externals are chipset controlled then you are back to core control as the only avenue.

Then again, I don't use lightdm but don't remember it being much a load? Worrying about affinity or numa zone is for big things.

User avatar
kent_dorfman766
Posts: 535
Joined: 2022-12-16 06:34
Location: socialist states of america
Has thanked: 57 times
Been thanked: 70 times

Re: [Software] esoteric taskset issue

#5 Post by kent_dorfman766 »

re below comments...rereading your post I understand what you are getting at but my understanding is that cpu affinity is at the core or hyperthread level, not the physical cpu slot level. I may be mistaken, but am not willing to concede that just yet. :wink:


This is more about a "bug" or something that is not working per documented spec. than it is about justifying the observed behaviour, unless there is a real documented engineering reason that it is not honouring the affinity mask. There are 4 xorg threads that are showing up on cpu0 (core0, core2), even though I specified to only use cpu1 (cores 4-7). To me that signifies either a bug in the scheduler or in the reporting of system metrics. has nothing to do with GPU, and only minimally relates to the core number within the CPU. The key issue is that it is reporting running on a CPU that is outside of the mask supplied in taskset.

taskset 0xf0 program on my E5440 SMP server means can run on any core in second CPU.

So, the $1,000 question is why 4 threads out of about 30 were incorrectly scheduled on the wrong CPU.

User avatar
kent_dorfman766
Posts: 535
Joined: 2022-12-16 06:34
Location: socialist states of america
Has thanked: 57 times
Been thanked: 70 times

Re: [Software] esoteric taskset issue

#6 Post by kent_dorfman766 »

@CwF

Thanks for your patience. The issue was that I was addressing the cores incorrectly. I forgot that they are enumerated on an interlaaved basis between the physical CPUs, so cpu0 is 0,2,4,6 and cpu1 is 1,3,5,7. That is what accounted for my odd thread bindings.

CwF
Global Moderator
Global Moderator
Posts: 2638
Joined: 2018-06-20 15:16
Location: Colorado
Has thanked: 41 times
Been thanked: 192 times

Re: [Software] esoteric taskset issue

#7 Post by CwF »

I'm not so familiar with the 5400 series I've built a few 55/5600's, maybe similar. I think the cpu's are indeed 0-3 and 4-7, not interleaved. I can't remember thread order I don't use them but thought it went 0-3, 4-7 T's. then cpu1 is 8-11 and its threads 12-15. ? Anyway, the pci-e lanes are not cpu controlled, but the memory is. So only in later cpu's would slots make a difference.

Try https://packages.debian.org/bullseye/numactl

By virtue of restricting to memory numa zone 1, you inherently resrict to cpu1. The goal is the same, open end. closed end wrench, your choice!

I assume there are certain task that ignore user level request.

I recycled on of my first Debian boxes for XP32! Dual X5687's! I used the bios to tame it and it rocks. With that simplistic smp scheduler most everything runs on cpu1

Random_Troll
Posts: 444
Joined: 2023-02-07 13:35
Been thanked: 105 times

Re: [Software] esoteric taskset issue

#8 Post by Random_Troll »

I like lstopo(1). It doesn't show anything not available from /proc/cpuinfo but the picture is nice.
Jeder nach seinen Fähigkeiten, jedem nach seinen Bedürfnissen.

User avatar
kent_dorfman766
Posts: 535
Joined: 2022-12-16 06:34
Location: socialist states of america
Has thanked: 57 times
Been thanked: 70 times

Re: [Solved] esoteric taskset issue

#9 Post by kent_dorfman766 »

As I said, it's solved. The core numbers are interleaved between the two sockets and I didn't realize that was the case. taskset is working properly now. However, I'm also wondering if using the 0x mask format was not being parse properly. When I fixed it I went to the --cpu-list format instead.

Post Reply