[HowTo] Disabling SMT

Share your own howto's etc. Not for support questions!

[HowTo] Disabling SMT

Postby Head_on_a_Stick » 2018-11-02 21:24

Note: kernel 4.18 or newer now has a nosmt kernel command line parameter.

Some of you may be aware of the recently announced Portsmash vulnerability:

https://www.openwall.com/lists/oss-secu ... 18/11/01/4

The problem is caused by the design of the cpu, specifically the so-called hyperthreads (symmetric multi-threading technology, or SMT for short) are not subject to the same sort of security checks that are carried out in the physical cores.

OpenBSD 6.4 has disabled SMT by default because of this (the devs predicted the vulnerability) and the advice for Linux is now to disable SMT via the firmware ("BIOS") settings, if possible.

Unfortunately, my machine has no such option so I have to use systemd unit file instead :)

To write the unit we first need to determine which cpu(s) to turn off, so run this command:
Code: Select all
empty@buster:~ $ lscpu --extended
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ    MINMHZ
0   0    0      0    0:0:0:0       yes    2400.0000 1199.0000
1   0    0      0    0:0:0:0       yes    2400.0000 1199.0000
2   0    0      1    1:1:1:0       yes    2400.0000 1199.0000
3   0    0      1    1:1:1:0       yes    2400.0000 1199.0000
empty@buster:~ $

^ The CORE column shows which physical cpu is hosting which virtual cpu and in my case cpu1 & cpu3 are hyperthreads and need to be disabled.

To disable them, use this script (saved to /usr/local/bin/nosmt):
Code: Select all
#!/bin/sh
for n in 1 3
   do echo 0 > /sys/devices/system/cpu/cpu${n}/online
done

^ Change the `for n in 1 3` line according to the hardware in use.

And a matching onsmt script (to re-enable SMT when the .service is stopped):
Code: Select all
#!/bin/sh
for n in 1 3
   do echo 1 > /sys/devices/system/cpu/cpu${n}/online
done

Save both of those files and make them executable:
Code: Select all
chmod +x /usr/local/bin/{no,on}smt

And this is the systemd custom unit file:
Code: Select all
# /etc/systemd/system/nosmt.service
[Unit]
Description=Disable SMT

[Service]
RemainAfterExit=yes
ExecStart=/usr/local/bin/nosmt
ExecStop=/usr/local/bin/onsmt

[Install]
WantedBy=multi-user.target

Once the unit file is saved, enable and start the .service with:
Code: Select all
systemctl enable --now nosmt

Check that the hyperthreads have been disabled with `lscpu --extended` and check the .service with `systemctl status nosmt` (it should be reported as "active").

SMT can be enabled again by stopping the .service:
Code: Select all
systemctl stop nosmt

If there are any problems, check the journal:
Code: Select all
journalctl -u nosmt

Alternative methods for different init systems are listed here: viewtopic.php?p=684521#p684521
Last edited by Head_on_a_Stick on 2018-11-12 18:39, edited 8 times in total.
dbruce wrote:Ubuntu forums try to be like a coffee shop in Seattle. Debian forums strive for the charm and ambience of a skinhead bar in Bacau. We intend to keep it that way.
User avatar
Head_on_a_Stick
 
Posts: 8315
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: [HowTo] Disabling SMT

Postby Head_on_a_Stick » 2018-11-03 09:56

Bumping this thread because the maxcpus kernel parameter just disables an entire physical core on my machine but keeps SMT running :?

I've written some udev rules instead, they seem to work.
dbruce wrote:Ubuntu forums try to be like a coffee shop in Seattle. Debian forums strive for the charm and ambience of a skinhead bar in Bacau. We intend to keep it that way.
User avatar
Head_on_a_Stick
 
Posts: 8315
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: [HowTo] Disabling SMT

Postby pcalvert » 2018-11-04 19:30

What kind of effect does disabling SMT have on performance?

Phil
“Property is the fruit of labor; property is desirable; it is a positive good
in the world. That some should be rich shows that others may become
rich, and hence is just encouragement to industry and enterprise.”
— Abraham Lincoln
pcalvert
 
Posts: 1806
Joined: 2006-04-21 11:19
Location: Sol Sector

Re: [HowTo] Disabling SMT

Postby Head_on_a_Stick » 2018-11-04 19:50

^ That depends on the applications being run — I've been rendering scenes with Blender/Cycles today and onlining the cpus during the render doesn't alter the finish time at all, AFAICT.

I know that under OpenBSD disabling SMT can actually speed things up but that's because most of their kernel is still giant locked so I would expect some performance hit under Linux, especially in respect of scaling.

The Portsmash vulnerability is described as "local" but Ted Uangst has pointed out that any javascript executed through a browser is "local" so the risk is very real (IMO).
dbruce wrote:Ubuntu forums try to be like a coffee shop in Seattle. Debian forums strive for the charm and ambience of a skinhead bar in Bacau. We intend to keep it that way.
User avatar
Head_on_a_Stick
 
Posts: 8315
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: [HowTo] Disabling SMT

Postby Head_on_a_Stick » 2018-11-05 20:37

The good folks over at the ArchLabs forums have reported that my udev rules don't work very well.

I've switched over to a custom unit file for systemd instead, here it is:
Code: Select all
# /etc/systemd/system/nosmt.service
[Unit]
Description=Disable SMT

[Service]
ExecStart=/usr/local/bin/nosmt

[Install]
WantedBy=multi-user.target

And this is the /usr/local/bin/nosmt script:
Code: Select all
#!/bin/sh
for n in 1 3
   do echo 0 > /sys/devices/system/cpu/cpu${n}/online
done

^ That’s for my hardware, edit it to match the machine.

Make the script executable with `chmod +x /usr/local/bin/nosmt` and then enable the .service:
Code: Select all
systemctl enable --now nosmt.service

This method logs to the journal, which is nice:
Code: Select all
empty@buster:~ $ journalctl -u nosmt --no-p
-- Logs begin at Mon 2018-11-05 21:10:02 GMT, end at Mon 2018-11-05 21:24:58 GMT. --
Nov 05 21:10:02 buster systemd[1]: Started Disable SMT.
empty@buster:~ $
dbruce wrote:Ubuntu forums try to be like a coffee shop in Seattle. Debian forums strive for the charm and ambience of a skinhead bar in Bacau. We intend to keep it that way.
User avatar
Head_on_a_Stick
 
Posts: 8315
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: [HowTo] Disabling SMT

Postby Head_on_a_Stick » 2018-11-06 16:33

Updated version of nosmt.service:
Code: Select all
[Unit]
Description=Disable SMT

[Service]
RemainAfterExit=yes
ExecStart=/usr/local/bin/nosmt
ExecStop=/usr/local/bin/onsmt

[Install]
WantedBy=multi-user.target

With the new /usr/local/bin/onsmt script:
Code: Select all
#!/bin/sh
for n in 1 3
   do echo 1 > /sys/devices/system/cpu/cpu${n}/online
done

With these changes nosmt.service will now report itself as “active” when it has been run and if it is stopped then the ExecStop line will run the /usr/local/bin/onsmt script and re-enable the hyperthreads to give a boost when needed:
Code: Select all
empty@buster:~ $ lscpu --extended                                                 
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ    MINMHZ
0   0    0      0    0:0:0:0       yes    2400.0000 1199.0000
1   -    -      -    :::           no     2400.0000 1199.0000
2   0    0      1    1:1:1:0       yes    2400.0000 1199.0000
3   -    -      -    :::           no     2400.0000 1199.0000
empty@buster:~ $ sudo systemctl stop nosmt
[sudo] password for empty:
empty@buster:~ $ lscpu --extended                                                 
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ    MINMHZ
0   0    0      0    0:0:0:0       yes    2400.0000 1199.0000
1   0    0      0    0:0:0:0       yes    2400.0000 1199.0000
2   0    0      1    1:1:1:0       yes    2400.0000 1199.0000
3   0    0      1    1:1:1:0       yes    2400.0000 1199.0000
empty@buster:~ $
dbruce wrote:Ubuntu forums try to be like a coffee shop in Seattle. Debian forums strive for the charm and ambience of a skinhead bar in Bacau. We intend to keep it that way.
User avatar
Head_on_a_Stick
 
Posts: 8315
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: [HowTo] Disabling SMT

Postby Head_on_a_Stick » 2018-11-07 17:51

I've edited the OP, @cynwulf over at debianuserforums.org thinks you're all a bit thick and suggested that I add more detail so thanks go to that user.
dbruce wrote:Ubuntu forums try to be like a coffee shop in Seattle. Debian forums strive for the charm and ambience of a skinhead bar in Bacau. We intend to keep it that way.
User avatar
Head_on_a_Stick
 
Posts: 8315
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: [HowTo] Disabling SMT

Postby anticapitalista » 2018-11-08 09:37

For those not using systemd?
antiX "Heather Heyer" - lean and mean.
http://antix.mepis.org
anticapitalista
 
Posts: 332
Joined: 2007-12-14 23:16

Re: [HowTo] Disabling SMT

Postby Head_on_a_Stick » 2018-11-08 17:38

^ That's a good question.

I'm unfamiliar with sysvinit but for OpenRC I would use something like this:
Code: Select all
#!/sbin/openrc-run

description="Disables SMT"

start() {
   ebegin "Disabling SMT"
   for n in 1 3
      do echo 0 > /sys/devices/system/cpu/cpu${n}/online
   done
   eend "$?"
}

stop() {
   ebegin "Re-enabling SMT"
   for n in 1 3
      do echo 1 > /sys/devices/system/cpu/cpu${n}/online
   done
   eend "$?"
}

^ Save that to /etc/init.d/nosmt and then enable the script with
Code: Select all
# rc-update add nosmt

I think the sysvinit equivalent would be something like:
Code: Select all
#!/bin/sh

case "$1" in
   start)
      echo "Disabling SMT"
      for n in 1 3
         do echo 0 > /sys/devices/system/cpu/cpu${n}/online
      done
      ;;
   stop)
      echo "Re-enabling SMT"
      for n in 1 3
         do echo 1 > /sys/devices/system/cpu/cpu${n}/online
      done
      ;;
esac

exit 0

Save the file & `chmod 755` it then run
Code: Select all
# update-rc.d nosmt defaults

I can't test these methods though and I've never written a sysvinit script before so any feedback or improvements would be most appreciated, thanks!
dbruce wrote:Ubuntu forums try to be like a coffee shop in Seattle. Debian forums strive for the charm and ambience of a skinhead bar in Bacau. We intend to keep it that way.
User avatar
Head_on_a_Stick
 
Posts: 8315
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: [HowTo] Disabling SMT

Postby Head_on_a_Stick » 2018-11-12 18:37

Kernel 4.18 in testing/unstable now has a nosmt kernel command line parameter that will disable hyperthreading.

Add the parameter to GRUB_CMDLINE_LINUX in /etc/default/grub and run `update-grub` (as root) to apply it.

There is also a nosmt=force parameter that will disable the ability to on-line the cores via sysfs.

https://github.com/torvalds/linux/blob/ ... .txt#L2818
dbruce wrote:Ubuntu forums try to be like a coffee shop in Seattle. Debian forums strive for the charm and ambience of a skinhead bar in Bacau. We intend to keep it that way.
User avatar
Head_on_a_Stick
 
Posts: 8315
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: [HowTo] Disabling SMT

Postby stevepusser » 2018-11-19 21:10

4.18.19, 4.19.2 have a commit to enable STIBP as default to address the issue, though Phoronix is reporting that it does have a noticeable performance hit.

Usually, these make their way to the 4.9 Debian LTS kernel pretty quick, too. I'm building the latest Liquorix kernel based on 4.18.19 to see if I lose six threads...

commit 02fb68711ba6d540087999d7cd4a67c16b69f2c0
Author: Jiri Kosina <jkosina@suse.cz>
Date: Tue Sep 25 14:38:55 2018 +0200

x86/speculation: Enable cross-hyperthread spectre v2 STIBP mitigation

commit 53c613fe6349994f023245519265999eed75957f upstream.

STIBP is a feature provided by certain Intel ucodes / CPUs. This feature
(once enabled) prevents cross-hyperthread control of decisions made by
indirect branch predictors.

Enable this feature if

- the CPU is vulnerable to spectre v2
- the CPU supports SMT and has SMT siblings online
- spectre_v2 mitigation autoselection is enabled (default)

After some previous discussion, this leaves STIBP on all the time, as wrmsr
on crossing kernel boundary is a no-no. This could perhaps later be a bit
more optimized (like disabling it in NOHZ, experiment with disabling it in
idle, etc) if needed.

Note that the synchronization of the mask manipulation via newly added
spec_ctrl_mutex is currently not strictly needed, as the only updater is
already being serialized by cpu_add_remove_lock, but let's make this a
little bit more future-proof.
The MX Linux repositories: Backports galore! If we don't have something, just ask and we'll try--we like challenges. New packages: Flightgear 2018.2.2, tlp-ui, wine-staging 3.21, Pale Moon 28.2.2, Mesa 18.2.6, Midori 7.0
User avatar
stevepusser
 
Posts: 10245
Joined: 2009-10-06 05:53

Re: [HowTo] Disabling SMT

Postby stevepusser » 2018-11-19 22:51

It now seems that the kernel devs realize that having STIBP by default was a mistake and affected performance far more then they expected, so expect a revert in the next release: http://lkml.iu.edu/hypermail/linux/kern ... 01328.html

So don't expect this in the 4.9 kernel after all.
The MX Linux repositories: Backports galore! If we don't have something, just ask and we'll try--we like challenges. New packages: Flightgear 2018.2.2, tlp-ui, wine-staging 3.21, Pale Moon 28.2.2, Mesa 18.2.6, Midori 7.0
User avatar
stevepusser
 
Posts: 10245
Joined: 2009-10-06 05:53

Re: [HowTo] Disabling SMT

Postby wurstkraft » 2018-11-24 18:22

There we go: https://cdn.kernel.org/pub/linux/kernel ... og-4.9.140

4.9.135 is in proposed updates now.
wurstkraft
 
Posts: 1
Joined: 2018-01-05 20:53


Return to Docs, Howtos, Tips & Tricks

Who is online

Users browsing this forum: No registered users and 7 guests

fashionable