[SOLVED] Random Freeze NVMe + Firefox [BLUETOOTH]

[Danielsan]

Hi Guys,

I am having a very annoying issue with Debian over a NVMe disk, randomly but quite soon whatever thing you are doing with Firefox the system freezes up and not even "REISUB" can reboot the system.

A couple of time the X session crashed hence I saw a TTY console showing an infinite list of INODE issue with the NVME / partition, silly I didn't take any photos, and I could not find any logs about it as well. Later I entered with a live session to run a FSCK over the partition but it didn't report any issue, the same when I checked out the log with the nvm-cli tools; however the computer continues to freeze unrecoverably.

Now this is the Debian partition I setup for my son and his virtual school, since it is not working he is using the Windows partition, and I have to deal with my wife that is saying Debian is not worth, you see that is unbearable. I have to dig to fix the cause of this instability a soon as possible but I don't know where to begin because apparently nothing is broken, without considering that journalctl is the most useless tool I have ever seen on Debian.

I remember that NVMe disks suffer about instability but as usual the Debian wiki is not the best place to find a good suggestion.

This laptop uses Debian Testing.

Thanks in advance!

[Head_on_a_Stick]

not even "REISUB" can't reboot the system

I presume you mean "not even REISUB can reboot the system" — it needs to be enabled in Debian: https://unix.stackexchange.com/question ... -on-debian

journalctl is the most useless tool I have ever seen on Debian

Please post the journal contents for one of the "freezes", that will list those "INODE" issues without needing a camera

You will have to enable persistence beforehand though, ask the interweb how (or search the boards, I've just answered a query about that).

This laptop uses Debian Testing

Why? I think the stable release would offer a more, er, stable working environment.

FWIW I have an NVMe disk and have never suffered any "freezes" at all under Debian buster.

[Danielsan]

Thank you very much!
I have another laptop with a NVM and never had an issue as well. I hope to have time to check up the PC this night I really want get rid of this issue as well as Windows...

[Danielsan]

REISUB works out of the box...

the logs

Code: Select all

Aug 04 18:55:12 motile systemd-user-runtime-dir[33695]: Failed to acquire number of inodes for runtime directory: Unknown interface org.freedesktop.login1.Manager or property RuntimeDirectoryInodesMax.
Aug 21 00:23:05 motile systemd-fsck[307]: home: Clearing orphaned inode 9052716 (uid=1002, gid=1002, mode=0100644, size=3275)
Aug 24 17:33:38 motile systemd-fsck[310]: home: Clearing orphaned inode 9175839 (uid=1002, gid=1002, mode=0100644, size=0)

Code: Select all

nvme get-log /dev/nvme0n1p1 --log-id=2 --log-len=512
[Device:nvme0n1p1 log-id:2 namespace-id:0xffffffff
       0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
0000: 00 2c 01 64 0a 00 00 00 00 00 00 00 00 00 00 00 ".,.d............"
0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0020: 2d 27 01 00 00 00 00 00 00 00 00 00 00 00 00 00 "-'.............."
0030: 50 b8 01 00 00 00 00 00 00 00 00 00 00 00 00 00 "P..............."
0040: e2 fb 0c 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0050: eb ec 13 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0060: 06 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0070: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0080: 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0090: 0e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
00a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
00b0: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
00c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
00d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
00e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
00f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
0190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
01a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
01b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
01c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
01d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
01e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
01f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"

[Danielsan]

Today the session hasn't blocked so far (crossing fingers) however I noticed that Zoom (flatpak) through the MESA/AMDGPU driver is ugly while in my laptop with a Nvidia GPU the looks and feel it is just like Windows...

[Head_on_a_Stick]

the logs

I think we need to see more than that, use the --since= option to filter the entries by time stamp.

[LE_746F6D617A7A69]

I remember that NVMe disks suffer about instability but as usual the Debian wiki is not the best place to find a good suggestion.

This laptop uses Debian Testing.

Why do You think that the problem is related to NVME?

Do You have any system logs/symptoms similar to this:
http://forums.debian.net/viewtopic.php?f=7&t=146830
http://forums.debian.net/viewtopic.php?f=7&t=146747
???

[Danielsan]

I remember that NVMe disks suffer about instability but as usual the Debian wiki is not the best place to find a good suggestion.

This laptop uses Debian Testing.

Why do You think that the problem is related to NVME?

Do You have any system logs/symptoms similar to this:
http://forums.debian.net/viewtopic.php?f=7&t=146830
http://forums.debian.net/viewtopic.php?f=7&t=146747
???

I just implied it because the disk is a NVME and because I saw an inode issue like one that you linked:
http://forums.debian.net/viewtopic.php? ... 47#p723770

However I also realized that I've never optimized this laptop for a better SSD set up.

[Danielsan]

the logs

I think we need to see more than that, use the --since= option to filter the entries by time stamp.

I don't know why but those output messages I saw were not saved anywhere...

Code: Select all

journalctl --since "2020-07-01" --no-pager | grep inode
Aug 04 18:55:12 motile systemd-user-runtime-dir[33695]: Failed to acquire number of inodes for runtime directory: Unknown interface org.freedesktop.login1.Manager or property RuntimeDirectoryInodesMax.
Aug 21 00:23:05 motile systemd-fsck[307]: home: Clearing orphaned inode 9052716 (uid=1002, gid=1002, mode=0100644, size=3275)
Aug 24 17:33:38 motile systemd-fsck[310]: home: Clearing orphaned inode 9175839 (uid=1002, gid=1002, mode=0100644, size=0)

Code: Select all

journalctl --since "2020-07-01" -k --no-pager | grep nvme 
Aug 25 22:31:46 motile kernel: nvme nvme0: pci function 0000:01:00.0
Aug 25 22:31:46 motile kernel: nvme nvme0: allocated 32 MiB host memory buffer.
Aug 25 22:31:46 motile kernel: nvme nvme0: 16/0/0 default/read/poll queues
Aug 25 22:31:46 motile kernel:  nvme0n1: p1 p2
Aug 25 22:31:46 motile kernel: EXT4-fs (nvme0n1p1): mounted filesystem with ordered data mode. Opts: (null)
Aug 25 22:31:46 motile kernel: EXT4-fs (nvme0n1p1): re-mounted. Opts: errors=remount-ro
Aug 25 22:31:46 motile kernel: EXT4-fs (nvme0n1p2): mounted filesystem with ordered data mode. Opts: (null)

Code: Select all

 nvme get-feature /dev/nvme0 -f 0x0c -H
get-feature:0xc (Autonomous Power State Transition), Current value:0x000001
	Autonomous Power State Transition Enable (APSTE): Enabled
	Auto PST Entries	.................
	Entry[ 0]   
	.................
	Idle Time Prior to Transition (ITPT): 745 ms
	Idle Transition Power State   (ITPS): 3
	.................
	Entry[ 1]   
	.................
	Idle Time Prior to Transition (ITPT): 745 ms
	Idle Transition Power State   (ITPS): 3
	.................
	Entry[ 2]   
	.................
	Idle Time Prior to Transition (ITPT): 745 ms
	Idle Transition Power State   (ITPS): 3
	.................
	Entry[ 3]   
	.................
	Idle Time Prior to Transition (ITPT): 2200 ms
	Idle Transition Power State   (ITPS): 4
	.................
	Entry[ 4]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[ 5]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[ 6]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[ 7]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[ 8]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[ 9]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[10]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[11]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[12]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[13]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[14]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[15]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[16]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[17]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[18]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[19]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[20]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[21]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[22]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[23]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[24]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[25]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[26]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[27]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[28]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[29]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[30]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................
	Entry[31]   
	.................
	Idle Time Prior to Transition (ITPT): 0 ms
	Idle Transition Power State   (ITPS): 0
	.................

Code: Select all

smartctl -a /dev/nvme0
smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.7.0-2-amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       WDC WDS250G2B0C-00PXH0
Serial Number:                      20045F803536
Firmware Version:                   211070WD
PCI Vendor/Subsystem ID:            0x15b7
IEEE OUI Identifier:                0x001b44
Total NVM Capacity:                 250,059,350,016 [250 GB]
Unallocated NVM Capacity:           0
Controller ID:                      1
Number of Namespaces:               1
Namespace 1 Size/Capacity:          250,059,350,016 [250 GB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            001b44 8b4657443e
Local Time is:                      Tue Aug 25 23:06:14 2020 EDT
Firmware Updates (0x14):            2 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x005f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp
Maximum Data Transfer Size:         128 Pages
Warning  Comp. Temp. Threshold:     80 Celsius
Critical Comp. Temp. Threshold:     85 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     3.50W    1.80W       -    0  0  0  0        0       0
 1 +     2.40W    1.60W       -    0  0  0  0        0       0
 2 +     1.90W    1.50W       -    0  0  0  0        0       0
 3 -   0.0200W       -        -    3  3  3  3     3900   11000
 4 -   0.0050W       -        -    4  4  4  4     5000   39000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         2
 1 -    4096       0         1

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        32 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    80,231 [41.0 GB]
Data Units Written:                 120,707 [61.8 GB]
Host Read Commands:                 903,658
Host Write Commands:                1,511,385
Controller Busy Time:               8
Power Cycles:                       281
Power On Hours:                     11
Unsafe Shutdowns:                   14
Media and Data Integrity Errors:    0
Error Information Log Entries:      1
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0

Error Information (NVMe Log 0x01, max 256 entries)
No Errors Logged

It seems everything fine and today worked without issues... No clues...

[Danielsan]

Just another freeze but I couldn't find anything it get stuck and completely freezed...

[Head_on_a_Stick]

I was thinking that the entire journal contents (following a specific time stamp) would be more illuminating because the problem might actually lie elsewhere, I'd be happy to glance over them if you upload the log to https://paste.debian.net (or so).

I presume you have installed all the required non-free firmware and CPU µcode?

[Danielsan]

I was thinking that the entire journal contents (following a specific time stamp) would be more illuminating because the problem might actually lie elsewhere, I'd be happy to glance over them if you upload the log to https://paste.debian.net (or so).

I presume you have installed all the required non-free firmware and CPU µcode?

Hey, thank you very much as first thing!

You were correct, free and non-free firmware were installed, I also fixed the groups, I forgot that when you create a new user it is not automatically added to specific groups like: video, audio, lpadmin, cdrom etc... This fixed a lot of weird issues but it didn't prevent the freezings...

I'll try to upload the log as soon as I can get the hands over the laptop which is a Motile AMD Ryzen 3 with Radeon Vega 3 Graphics...

p.s. My wife just called me upset because Debian froze out...

[Danielsan]

Paste Debian has some limitation, I have to figure out something else...
However I tried also a stress test but nothing happened...

Code: Select all

sudo stressapptest -f /media/file1 -f /media/file2 -s 600 -M 2048 -m 2 -C 2 -W
Log: Commandline - stressapptest -f /media/file1 -f /media/file2 -s 600 -M 2048 -m 2 -C 2 -W
Stats: SAT revision 1.0.6_autoconf, 64 bit binary
Log: buildd @ x86-ubc-01 on Fri Mar 17 03:54:25 UTC 2017 from open source release
Log: 1 nodes, 4 cpus.
Log: Prefer plain malloc memory allocation.
Log: Using memaligned allocation at 0x7fe7ebfff000.
Stats: Starting SAT, 2048M, 600 seconds
Log: Region mask: 0x1
Log: Seconds remaining: 590

[...]

Log: Seconds remaining: 10
Stats: Found 0 hardware incidents
Stats: Completed: 8181498.00M in 600.47s 13625.04MB/s, with 0 hardware incidents, 0 errors
Stats: Memory Copy: 7973050.00M at 13288.21MB/s
Stats: File Copy: 208448.00M at 347.40MB/s
Stats: Net Copy: 0.00M at 0.00MB/s
Stats: Data Check: 0.00M at 0.00MB/s
Stats: Invert Data: 0.00M at 0.00MB/s
Stats: Disk: 0.00M at 0.00MB/s

Status: PASS - please verify no corrected errors

A dropbox link with the log from 24 to 27 August...
https://www.dropbox.com/s/pde1yyndho75u ... g.log?dl=0

[Head_on_a_Stick]

A dropbox link with the log from 24 to 27 August...

D00d please, that's too big

Wait for a freeze then reboot the machine and run

Code: Select all

journalctl --since="5 minutes ago" --no-pager | curl -F 'f:1=<-' ix.io

Then post the returned URL here.

[Danielsan]

Oh boy... I misunderstood your request...

Yes of course I'll do it but it hasn't crashed yet...

[LE_746F6D617A7A69]

Hmm, this is a strange situation - if the problem is related to NVME, You would have continuous problems with unstable filesystem - but it seems that this is not the case here.
Your logs are showing that You have a "AMD Ryzen 3 3200U with Radeon Vega Mobile Gfx (family: 0x17, model: 0x18, stepping: 0x1)"
This an CPU-integrated GFX card, so crashes in the GFX driver can mess-up the system RAM, causing crazy errors in any other system components.
Taking into account that only Firefox is crashing Your system, and assuming that You have a clean Debian installed (e.g. no Liqourix kernel and assuming that none of the flatpacks/ppa's have damaged Your OS), I would say that it can be an issue with the testing snapshot of Debian which You are using - so in such case You should fill a bug report.

[Danielsan]

Hmm, this is a strange situation - if the problem is related to NVME, You would have continuous problems with unstable filesystem - but it seems that this is not the case here.
Your logs are showing that You have a "AMD Ryzen 3 3200U with Radeon Vega Mobile Gfx (family: 0x17, model: 0x18, stepping: 0x1)"
This an CPU-integrated GFX card, so crashes in the GFX driver can mess-up the system RAM, causing crazy errors in any other system components.
Taking into account that only Firefox is crashing Your system, and assuming that You have a clean Debian installed (e.g. no Liqourix kernel and assuming that none of the flatpacks/ppa's have damaged Your OS), I would say that it can be an issue with the testing snapshot of Debian which You are using - so in such case You should fill a bug report.

The issue has not appeared so far, I tried to stress this laptop in any condition, but everything now seems fine. About the GPU: on Xorg log I didn't see any issue or warning.

This Debian is setup to testing maybe some fixes arrived with some kernel updates, however I would like to report the bug if I would be able to find and to replicate it...

[Danielsan]

Eventually I was able to replicate the issue...

It wasn't the NVMe but an override to the systemd bluetooth.service I did a month ago to fix the bluetooth that was probably fixed later and it goes in conflict with my override.

Weird...