Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

[HowTo] Disabling SMT

Share your HowTo, Documentation, Tips and Tricks. Not for support questions!.
Post Reply
Message
Author
User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

[HowTo] Disabling SMT

#1 Post by Head_on_a_Stick »

Note: kernel 4.18 or newer now has a nosmt kernel command line parameter.

Some of you may be aware of the recently announced Portsmash vulnerability:

https://www.openwall.com/lists/oss-secu ... 18/11/01/4

The problem is caused by the design of the cpu, specifically the so-called hyperthreads (symmetric multi-threading technology, or SMT for short) are not subject to the same sort of security checks that are carried out in the physical cores.

OpenBSD 6.4 has disabled SMT by default because of this (the devs predicted the vulnerability) and the advice for Linux is now to disable SMT via the firmware ("BIOS") settings, if possible.

Unfortunately, my machine has no such option so I have to use systemd unit file instead :)

To write the unit we first need to determine which cpu(s) to turn off, so run this command:

Code: Select all

empty@buster:~ $ lscpu --extended
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ    MINMHZ
0   0    0      0    0:0:0:0       yes    2400.0000 1199.0000
1   0    0      0    0:0:0:0       yes    2400.0000 1199.0000
2   0    0      1    1:1:1:0       yes    2400.0000 1199.0000
3   0    0      1    1:1:1:0       yes    2400.0000 1199.0000
empty@buster:~ $
^ The CORE column shows which physical cpu is hosting which virtual cpu and in my case cpu1 & cpu3 are hyperthreads and need to be disabled.

To disable them, use this script (saved to /usr/local/bin/nosmt):

Code: Select all

#!/bin/sh
for n in 1 3
   do echo 0 > /sys/devices/system/cpu/cpu${n}/online
done
^ Change the `for n in 1 3` line according to the hardware in use.

And a matching onsmt script (to re-enable SMT when the .service is stopped):

Code: Select all

#!/bin/sh
for n in 1 3
   do echo 1 > /sys/devices/system/cpu/cpu${n}/online
done
Save both of those files and make them executable:

Code: Select all

chmod +x /usr/local/bin/{no,on}smt
And this is the systemd custom unit file:

Code: Select all

# /etc/systemd/system/nosmt.service
[Unit]
Description=Disable SMT

[Service]
RemainAfterExit=yes
ExecStart=/usr/local/bin/nosmt
ExecStop=/usr/local/bin/onsmt

[Install]
WantedBy=multi-user.target
Once the unit file is saved, enable and start the .service with:

Code: Select all

systemctl enable --now nosmt
Check that the hyperthreads have been disabled with `lscpu --extended` and check the .service with `systemctl status nosmt` (it should be reported as "active").

SMT can be enabled again by stopping the .service:

Code: Select all

systemctl stop nosmt
If there are any problems, check the journal:

Code: Select all

journalctl -u nosmt
Alternative methods for different init systems are listed here: http://forums.debian.net/viewtopic.php?p=684521#p684521
Last edited by Head_on_a_Stick on 2018-11-12 18:39, edited 8 times in total.
deadbang

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: [HowTo] Disabling SMT

#2 Post by Head_on_a_Stick »

Bumping this thread because the maxcpus kernel parameter just disables an entire physical core on my machine but keeps SMT running :?

I've written some udev rules instead, they seem to work.
deadbang

pcalvert
Posts: 1939
Joined: 2006-04-21 11:19
Location: Sol Sector
Has thanked: 1 time
Been thanked: 2 times

Re: [HowTo] Disabling SMT

#3 Post by pcalvert »

What kind of effect does disabling SMT have on performance?

Phil
Freespoke is a new search engine that respects user privacy and does not engage in censorship.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: [HowTo] Disabling SMT

#4 Post by Head_on_a_Stick »

^ That depends on the applications being run — I've been rendering scenes with Blender/Cycles today and onlining the cpus during the render doesn't alter the finish time at all, AFAICT.

I know that under OpenBSD disabling SMT can actually speed things up but that's because most of their kernel is still giant locked so I would expect some performance hit under Linux, especially in respect of scaling.

The Portsmash vulnerability is described as "local" but Ted Uangst has pointed out that any javascript executed through a browser is "local" so the risk is very real (IMO).
deadbang

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: [HowTo] Disabling SMT

#5 Post by Head_on_a_Stick »

The good folks over at the ArchLabs forums have reported that my udev rules don't work very well.

I've switched over to a custom unit file for systemd instead, here it is:

Code: Select all

# /etc/systemd/system/nosmt.service
[Unit]
Description=Disable SMT

[Service]
ExecStart=/usr/local/bin/nosmt

[Install]
WantedBy=multi-user.target
And this is the /usr/local/bin/nosmt script:

Code: Select all

#!/bin/sh
for n in 1 3
   do echo 0 > /sys/devices/system/cpu/cpu${n}/online
done
^ That’s for my hardware, edit it to match the machine.

Make the script executable with `chmod +x /usr/local/bin/nosmt` and then enable the .service:

Code: Select all

systemctl enable --now nosmt.service
This method logs to the journal, which is nice:

Code: Select all

empty@buster:~ $ journalctl -u nosmt --no-p
-- Logs begin at Mon 2018-11-05 21:10:02 GMT, end at Mon 2018-11-05 21:24:58 GMT. --
Nov 05 21:10:02 buster systemd[1]: Started Disable SMT.
empty@buster:~ $
deadbang

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: [HowTo] Disabling SMT

#6 Post by Head_on_a_Stick »

Updated version of nosmt.service:

Code: Select all

[Unit]
Description=Disable SMT

[Service]
RemainAfterExit=yes
ExecStart=/usr/local/bin/nosmt
ExecStop=/usr/local/bin/onsmt

[Install]
WantedBy=multi-user.target
With the new /usr/local/bin/onsmt script:

Code: Select all

#!/bin/sh
for n in 1 3
   do echo 1 > /sys/devices/system/cpu/cpu${n}/online
done
With these changes nosmt.service will now report itself as “active” when it has been run and if it is stopped then the ExecStop line will run the /usr/local/bin/onsmt script and re-enable the hyperthreads to give a boost when needed:

Code: Select all

empty@buster:~ $ lscpu --extended                                                  
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ    MINMHZ
0   0    0      0    0:0:0:0       yes    2400.0000 1199.0000
1   -    -      -    :::           no     2400.0000 1199.0000
2   0    0      1    1:1:1:0       yes    2400.0000 1199.0000
3   -    -      -    :::           no     2400.0000 1199.0000
empty@buster:~ $ sudo systemctl stop nosmt
[sudo] password for empty: 
empty@buster:~ $ lscpu --extended                                                  
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ    MINMHZ
0   0    0      0    0:0:0:0       yes    2400.0000 1199.0000
1   0    0      0    0:0:0:0       yes    2400.0000 1199.0000
2   0    0      1    1:1:1:0       yes    2400.0000 1199.0000
3   0    0      1    1:1:1:0       yes    2400.0000 1199.0000
empty@buster:~ $
deadbang

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: [HowTo] Disabling SMT

#7 Post by Head_on_a_Stick »

I've edited the OP, @cynwulf over at debianuserforums.org thinks you're all a bit thick and suggested that I add more detail so thanks go to that user.
deadbang

anticapitalista
Posts: 428
Joined: 2007-12-14 23:16
Has thanked: 12 times
Been thanked: 13 times

Re: [HowTo] Disabling SMT

#8 Post by anticapitalista »

For those not using systemd?
antiX with runit - lean and mean.
https://antixlinux.com

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: [HowTo] Disabling SMT

#9 Post by Head_on_a_Stick »

^ That's a good question.

I'm unfamiliar with sysvinit but for OpenRC I would use something like this:

Code: Select all

#!/sbin/openrc-run

description="Disables SMT"

start() {
   ebegin "Disabling SMT"
   for n in 1 3
      do echo 0 > /sys/devices/system/cpu/cpu${n}/online
   done
   eend "$?"
}

stop() {
   ebegin "Re-enabling SMT"
   for n in 1 3
      do echo 1 > /sys/devices/system/cpu/cpu${n}/online
   done
   eend "$?"
}
^ Save that to /etc/init.d/nosmt and then enable the script with

Code: Select all

# rc-update add nosmt
I think the sysvinit equivalent would be something like:

Code: Select all

#!/bin/sh

case "$1" in
   start)
      echo "Disabling SMT"
      for n in 1 3
         do echo 0 > /sys/devices/system/cpu/cpu${n}/online
      done
      ;;
   stop)
      echo "Re-enabling SMT"
      for n in 1 3
         do echo 1 > /sys/devices/system/cpu/cpu${n}/online
      done
      ;;
esac

exit 0
Save the file & `chmod 755` it then run

Code: Select all

# update-rc.d nosmt defaults
I can't test these methods though and I've never written a sysvinit script before so any feedback or improvements would be most appreciated, thanks!
deadbang

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: [HowTo] Disabling SMT

#10 Post by Head_on_a_Stick »

Kernel 4.18 in testing/unstable now has a nosmt kernel command line parameter that will disable hyperthreading.

Add the parameter to GRUB_CMDLINE_LINUX in /etc/default/grub and run `update-grub` (as root) to apply it.

There is also a nosmt=force parameter that will disable the ability to on-line the cores via sysfs.

https://github.com/torvalds/linux/blob/ ... .txt#L2818
deadbang

User avatar
stevepusser
Posts: 12930
Joined: 2009-10-06 05:53
Has thanked: 41 times
Been thanked: 71 times

Re: [HowTo] Disabling SMT

#11 Post by stevepusser »

4.18.19, 4.19.2 have a commit to enable STIBP as default to address the issue, though Phoronix is reporting that it does have a noticeable performance hit.

Usually, these make their way to the 4.9 Debian LTS kernel pretty quick, too. I'm building the latest Liquorix kernel based on 4.18.19 to see if I lose six threads...
commit 02fb68711ba6d540087999d7cd4a67c16b69f2c0
Author: Jiri Kosina <jkosina@suse.cz>
Date: Tue Sep 25 14:38:55 2018 +0200

x86/speculation: Enable cross-hyperthread spectre v2 STIBP mitigation

commit 53c613fe6349994f023245519265999eed75957f upstream.

STIBP is a feature provided by certain Intel ucodes / CPUs. This feature
(once enabled) prevents cross-hyperthread control of decisions made by
indirect branch predictors.

Enable this feature if

- the CPU is vulnerable to spectre v2
- the CPU supports SMT and has SMT siblings online
- spectre_v2 mitigation autoselection is enabled (default)

After some previous discussion, this leaves STIBP on all the time, as wrmsr
on crossing kernel boundary is a no-no. This could perhaps later be a bit
more optimized (like disabling it in NOHZ, experiment with disabling it in
idle, etc) if needed.

Note that the synchronization of the mask manipulation via newly added
spec_ctrl_mutex is currently not strictly needed, as the only updater is
already being serialized by cpu_add_remove_lock, but let's make this a
little bit more future-proof.
MX Linux packager and developer

User avatar
stevepusser
Posts: 12930
Joined: 2009-10-06 05:53
Has thanked: 41 times
Been thanked: 71 times

Re: [HowTo] Disabling SMT

#12 Post by stevepusser »

It now seems that the kernel devs realize that having STIBP by default was a mistake and affected performance far more then they expected, so expect a revert in the next release: http://lkml.iu.edu/hypermail/linux/kern ... 01328.html

So don't expect this in the 4.9 kernel after all.
MX Linux packager and developer

wurstkraft
Posts: 2
Joined: 2018-01-05 20:53

Re: [HowTo] Disabling SMT

#13 Post by wurstkraft »

There we go: https://cdn.kernel.org/pub/linux/kernel ... og-4.9.140

4.9.135 is in proposed updates now.

Post Reply