Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

11 Bullseye kernel update disables SATA hotplug

Linux Kernel, Network, and Services configuration.
Post Reply
Message
Author
stunted
Posts: 6
Joined: 2007-11-29 02:39

11 Bullseye kernel update disables SATA hotplug

#1 Post by stunted »

On my AMD x570 chipset based system the current kernel linux-image-5.10.0-10-amd64 broke SATA hotplug, if I revert to linux-image-5.10.0-9-amd64 hotplug works, or on my very old AMD 780G chipest based system linux-image-5.10.0-10-amd64 has working SATA hotplug.

06:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0])
Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode]
Flags: bus master, fast devsel, latency 0, IRQ 59, IOMMU group 17
Memory at fc900000 (32-bit, non-prefetchable) [si ze=2K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Capabilities: [64] Express Endpoint, MSI 00
Capabilities: [a0] MSI: Enable+ Count=16/16 Maskable- 64bit+
Capabilities: [d0] SATA HBA v1.0
Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [270] Secondary PCI Express
Capabilities: [400] Data Link Feature <?>
Capabilities: [410] Physical Layer 16.0 GT/s <?>
Capabilities: [440] Lane Margining at the Receiver <?>
Kernel driver in use: ahci
Kernel modules: ahci

07:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0])
Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode]
Flags: bus master, fast devsel, latency 0, IRQ 75, IOMMU group 18
Memory at fc800000 (32-bit, non-prefetchable) [si ze=2K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Capabilities: [64] Express Endpoint, MSI 00
Capabilities: [a0] MSI: Enable+ Count=16/16 Maskable- 64bit+
Capabilities: [d0] SATA HBA v1.0
Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [270] Secondary PCI Express
Capabilities: [400] Data Link Feature <?>
Capabilities: [410] Physical Layer 16.0 GT/s <?>
Capabilities: [440] Lane Margining at the Receiver <?>
Kernel driver in use: ahci

What steps can I take to help debug?

Thanks for any help.

<edit 1>
I get the identical output form lspci -v under both kernels..
Nothing appears in dmesg when you insert a SATA drive under 5.10.0-10-amd64. is it a power saving thing?

<edit2>
this machine also has a 4 port PCI-E SATA card on which hotplug works flawlessly. I have checked BIOS & hotplug is enabled on the motherboard ports...

03:00.0 SATA controller: ASMedia Technology Inc. Device 1064 (rev 02) (prog-if 01 [AHCI 1.0])
Subsystem: ZyDAS Technology Corp. Device 2116
Flags: bus master, fast devsel, latency 0, IRQ 40, IOMMU group 19
Memory at fcb82000 (32-bit, non-prefetchable) [si ze=8K]
Memory at fcb80000 (32-bit, non-prefetchable)
Expansion ROM at fcb00000 [disabled] [siz e=512K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [80] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [130] Secondary PCI Express
Kernel driver in use: ahci
Kernel modules: ahci

CwF
Global Moderator
Global Moderator
Posts: 2625
Joined: 2018-06-20 15:16
Location: Colorado
Has thanked: 41 times
Been thanked: 190 times

Re: 11 Bullseye kernel update disables SATA hotplug

#2 Post by CwF »

stunted wrote: 2022-01-15 16:00 Nothing appears in dmesg when you insert a SATA drive under 5.10.0-10-amd64.
This is a correct start, I would check for messages when powering off a drive that was booted connected, it has a switch and not just insert and go I hope. Cycle power looking for a message.

There has been a bug in libata..something, that would spam with SSControl... once 32GB worth. It's spastic, a drive power cycle would stop it after a few tries. Not specific, but hopefully something to try. The error I've seen has also occurred where no drive was connected, perhaps something has been addressed, wrongly.

stunted
Posts: 6
Joined: 2007-11-29 02:39

Re: 11 Bullseye kernel update disables SATA hotplug

#3 Post by stunted »

No switch & when I pulled an unmounted drive that had been present during boot there was no immediate message. when I clicked on the drive in Thunar it looked like it was trying to mount the no longer present drive, after some time I got the following in dmsg

[ 166.529380] ata33.00: exception Emask 0x0 SAct 0x80000 SErr 0xd0000 action 0x6 frozen
[ 166.529385] ata33: SError: { PHYRdyChg CommWake 10B8B }
[ 166.529388] ata33.00: failed command: READ FPDMA QUEUED
[ 166.529391] ata33.00: cmd 60/20:98:00:20:12/00:00:00:00:00/40 tag 19 ncq dma 16384 in
res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[ 166.529393] ata33.00: status: { DRDY }
[ 166.529396] ata33: hard resetting link
[ 166.843748] ata33: SATA link down (SStatus 0 SControl 300)
[ 171.872107] ata33: hard resetting link
[ 172.186350] ata33: SATA link down (SStatus 0 SControl 300)
[ 177.250637] ata33: hard resetting link
[ 177.565943] ata33: SATA link down (SStatus 0 SControl 300)
[ 177.565953] ata33.00: disabled
[ 177.565981] sd 32:0:0:0: [sdd] tag#19 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=41s
[ 177.565983] sd 32:0:0:0: [sdd] tag#19 Sense Key : Not Ready [current]
[ 177.565984] sd 32:0:0:0: [sdd] tag#19 Add. Sense: Logical unit not ready, hard reset required
[ 177.565986] sd 32:0:0:0: [sdd] tag#19 CDB: Read(10) 28 00 00 12 20 00 00 00 20 00
[ 177.565989] blk_update_request: I/O error, dev sdd, sector 1187840 op 0x0:(READ) flags 0x80700 phys_seg 4 prio class 0
[ 177.566005] ata33: EH complete
[ 177.566013] ata33.00: detaching (SCSI 32:0:0:0)
[ 177.566048] blk_update_request: I/O error, dev sdd, sector 1187840 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 177.566052] Buffer I/O error on dev sdd2, logical block 0, async page read
[ 177.566192] blk_update_request: I/O error, dev sdd, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
[ 177.568866] sd 32:0:0:0: [sdd] Stopping disk
[ 177.568882] sd 32:0:0:0: [sdd] Start/Stop Unit failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK

everything seems to be behaving as if the onboard SATA ports were not hotswapable, I might try turning hotswap OFF in BIOS, I'll let you know.

Thank you for your help, hope you figure out your odd sounding bug..

<edit> Yeah that makes no difference...

CwF
Global Moderator
Global Moderator
Posts: 2625
Joined: 2018-06-20 15:16
Location: Colorado
Has thanked: 41 times
Been thanked: 190 times

Re: 11 Bullseye kernel update disables SATA hotplug

#4 Post by CwF »

stunted wrote: 2022-01-17 18:02 [ 166.529393] ata33.00: status: { DRDY }
[ 166.529396] ata33: hard resetting link
[ 166.843748] ata33: SATA link down (SStatus 0 SControl 300)
[ 171.872107] ata33: hard resetting link
[ 172.186350] ata33: SATA link down (SStatus 0 SControl 300)
[ 177.250637] ata33: hard resetting link
[ 177.565943] ata33: SATA link down (SStatus 0 SControl 300)
[ 177.565953] ata33.00: disabled
actually that is the bug, your example tried 3 times and gave up. The bug is never giving up and not getting to the disabled point! The bug starts with particular controllers and maybe disk where a timing error of some kind prevents a conclusion, a run away loop. Since your machine gave up, something has changed from my issue. I have switches and simply cycle power to clear it. When an empty slot causes the issue, insert random disk, power up power off, fixed.

find your specific examples of /sys and check

Code: Select all

:/sys/devices/pci0000:00/0000:00:1f.2/ata2/host1/target1:0:0/1:0:0:0$
$ cat state
running
# echo 1 > rescan
The echo of 1 into rescan (by root) should both release a powered off drive, and find a freshly powered up drive.

Since you use Thunar you could do all this graphically with CA's to the certain objections of doing so graphically, including the root parts! If a rescan works, make buttons and scripts to sidestep the issue of why it is not being handled.

stunted
Posts: 6
Joined: 2007-11-29 02:39

Re: 11 Bullseye kernel update disables SATA hotplug

#5 Post by stunted »

still happening with
uname -a
Linux PC_X570 5.10.0-11-amd64 #1 SMP Debian 5.10.92-1 (2022-01-18) x86_64 GNU/Linux

I'll have to figure out what info I need to scrape together for a bug report when work allows.

Thanks everyone.

stunted
Posts: 6
Joined: 2007-11-29 02:39

Re: 11 Bullseye kernel update disables SATA hotplug

#6 Post by stunted »

Still happening with 5.10.0-14-amd64 #1 SMP Debian 5.10.113-1 (2022-04-29) x86_64 GNU/Linux
& on another install I just installed kernel image from backports currently linux-image-5.16.0-0.bpo.4-amd64 (5.16.12-1~bpo11+1) & it's still there, including working with the ASMedia PCI-E SATA card just not with the motherboard ports.

Sorry to keep posting here, just trying to keep all the info in one place for when I get some time & can file a bug report..

<edit 1>
I just found this & wonder if it's a manifestation of the same bug, tho that's a much earlier kernel it's for a LTS release so a lot of subsequent bug fixes get ported back
https://www.mail-archive.com/kernel-pac ... 77882.html

LE_746F6D617A7A69
Posts: 932
Joined: 2020-05-03 14:16
Has thanked: 7 times
Been thanked: 65 times

Re: 11 Bullseye kernel update disables SATA hotplug

#7 Post by LE_746F6D617A7A69 »

stunted wrote: 2022-05-16 11:33 Still happening with 5.10.0-14-amd64 #1 SMP Debian 5.10.113-1 (2022-04-29) x86_64 GNU/Linux
& on another install I just installed kernel image from backports currently linux-image-5.16.0-0.bpo.4-amd64 (5.16.12-1~bpo11+1) & it's still there, including working with the ASMedia PCI-E SATA card just not with the motherboard ports.
For all the stable kernels SATA hotplug is working for Me.

From You 2nd post:
stunted wrote: 2022-01-17 18:02 [ 172.186350] ata33: SATA link down (SStatus 0 SControl 300)
[ 177.250637] ata33: hard resetting link
[ 177.565943] ata33: SATA link down (SStatus 0 SControl 300)
[ 177.565953] ata33.00: disabled
[ 177.565981] sd 32:0:0:0: [sdd] tag#19 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=41s
[ 177.565983] sd 32:0:0:0: [sdd] tag#19 Sense Key : Not Ready [current]
[ 177.565984] sd 32:0:0:0: [sdd] tag#19 Add. Sense: Logical unit not ready, hard reset required
[ 177.565986] sd 32:0:0:0: [sdd] tag#19 CDB: Read(10) 28 00 00 12 20 00 00 00 20 00
[ 177.565989] blk_update_request: I/O error, dev sdd, sector 1187840 op 0x0:(READ) flags 0x80700 phys_seg 4 prio class 0
[ 177.566005] ata33: EH complete
[ 177.566013] ata33.00: detaching (SCSI 32:0:0:0)
[ 177.566048] blk_update_request: I/O error, dev sdd, sector 1187840 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 177.566052] Buffer I/O error on dev sdd2, logical block 0, async page read
[ 177.566192] blk_update_request: I/O error, dev sdd, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
[ 177.568866] sd 32:0:0:0: [sdd] Stopping disk
[ 177.568882] sd 32:0:0:0: [sdd] Start/Stop Unit failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
This means that the SATA cable is damaged or the drive is faulty.
In case of damaged SATA cable the drive's SMART report usually will show increasing number of UDMA_CRC_Error_Count.
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed

piexil
Posts: 1
Joined: 2022-09-27 16:33

Re: 11 Bullseye kernel update disables SATA hotplug

#8 Post by piexil »

This is a kernel bug: https://bugs.launchpad.net/ubuntu/+sour ... ug/1971576
kernel commandline option ahci.mobile_lpm_policy=1 should fix it

stunted
Posts: 6
Joined: 2007-11-29 02:39

Re: 11 Bullseye kernel update disables SATA hotplug

#9 Post by stunted »

I can Confirm it does, thank you...

Post Reply