Unable to Recover From Suspend

Kernels & Hardware, configuring network, installing services

Unable to Recover From Suspend

Postby bootsie123` » 2019-01-05 14:53

Hi. I'm trying to set up my server to automatically go into suspend mode when it's not in use and then using WoL wake the server up from suspend mode when it needs to be accessed. The problem I'm running into is when the server is recovering from suspend mode everything powers back on but then freezes. No disk activity, no video output, no network connection via ethernet, nothing.

==================================================================================================================================================

The Problem in Short

The system freezes after everything gets powered up again when recovering from suspend mode


==================================================================================================================================================

The System

Dell Precision T7500:

OS: Debian Stretch (running Proxmox)
CPUs: 2 x Xeon X5660
RAM: 12GB per CPU (24GB Total)
SWAP: 16GB
STORAGE: 2 x 1TB in raid1 (zfs file system)
GRAPHICS: Nvidia Quadro K5000

==================================================================================================================================================

Things I've Tried

Tried adding
Code: Select all
resume=/dev/zvol/rpool/swap
to the grub boot config - Grub isn't able to find the swap file due to the zfs file system (I think this may be the biggest cause, let me you know if you have any alternative ways to do this or create a swap file with zfs accessible with grub)

Tried using
Code: Select all
systemctl suspend
and
Code: Select all
pm-suspend
- same results

Looked into the journalctl logs - nothing of value (can still post that if needed)
Also looked into the syslogs, kernel logs, and pm logs - nothing of value either (can also post if needed) | Shows that suspend is indeed successful, although it does not show anything about the recovery

Also tried using uswsusp and using
Code: Select all
s2ram
- I keep running into an error saying "KMS disabled by user" even though I am not blacklisting the nouveau drivers or using anything in grub which would stop KMS from being enabled (I can supply my grub settings if needed)

Tried removing the graphics card to see if it's an issue loading the drivers or something like that - same results | could also be that maybe the drivers still were loaded, not sure though

Looked into the bios to make sure that nothing is disabling suspend mode - everything should be set there

Tried limiting the CPU C-State to C2 - same issue

==================================================================================================================================================

Other Comments

Thanks for helping me out here. If you need any logs then let me know. I will gladly provide them. Thanks!
Last edited by bootsie123` on 2019-01-05 19:47, edited 3 times in total.
bootsie123`
 
Posts: 10
Joined: 2019-01-02 23:22

Re: Unable to Recover From Suspend

Postby llivv » 2019-01-05 19:15

bootsie123` wrote:I keep running into an error saying "KMS disabled by user" even though I am not blacklisting the nouveau drivers or using anything in grub which would stop KMS from being enabled
Thanks!
Code: Select all
$ cat /proc/cmdline
Code: Select all
$ uname -r
Code: Select all
$ cat /etc/debian_version
Code: Select all
# fdisk -l
In memory of Ian Ashley Murdock (1973 - 2015) founder of the Debian project.
User avatar
llivv
 
Posts: 5867
Joined: 2007-02-14 18:10
Location: cold storage

Re: Unable to Recover From Suspend

Postby bootsie123` » 2019-01-05 19:45

Output from
Code: Select all
cat /proc/cmdline


Code: Select all
BOOT_IMAGE=/ROOT/pve-1@/boot/vmlinuz-4.15.18-9-pve root=ZFS=rpool/ROOT/pve-1 ro root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on video=efifb:off


*NOTE 1: intel_iommu=on and video=efifb:off have been off when I've been trying to use s2ram, it's currently there so I can have GPU passthrough in the meantime

Note 2: For suspend I think the resume parameter is needed with the location of the swap, due to the zfs file system grub can't find it. I may be wrong about this

Output from
Code: Select all
uname -r


Code: Select all
4.15.18-9-pve


Output from
Code: Select all
cat /etc/debian_version


Code: Select all
9.6


Output from
Code: Select all
fdisk -l


Code: Select all
Disk /dev/sda: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
Units: sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytesDisklabel type: gpt
Disk identifier: C6D351ED-215E-40DD-9051-B531455BFCC7
Device       Start        End    Sectors  Size Type/dev/sda1       34       2047       2014 1007K BIOS boot
/dev/sda2     2048    1050623    1048576  512M EFI System
/dev/sda3  1050624 1953525134 1952474511  931G Solaris /usr & Apple ZFS

Disk /dev/sdb: 931.5 GiB, 1000204886016 bytes, 1953525168 sectorsUnits: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gptDisk identifier: FBA07C1E-CFE4-4D28-8E10-3247D2F104B9

Device       Start        End    Sectors  Size Type
/dev/sdb1       34       2047       2014 1007K BIOS boot
/dev/sdb2     2048    1050623    1048576  512M EFI System/dev/sdb3  1050624 1953525134 1952474511  931G Solaris /usr & Apple ZFS

Disk /dev/zd0: 16 GiB, 17179869184 bytes, 33554432 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/zd16: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes
Disklabel type: gpt
Disk identifier: 6EBFEF6B-CF70-47B0-9C54-BD6893058E84

Device        Start      End  Sectors  Size Type
/dev/zd16p1    2048  1023999  1021952  499M Windows recovery environment
/dev/zd16p2 1024000  1226751   202752   99M EFI System
/dev/zd16p3 1226752  1259519    32768   16M Microsoft reserved
/dev/zd16p4 1259520 67106815 65847296 31.4G Microsoft basic data


Disk /dev/zd32: 52 GiB, 55834574848 bytes, 109051904 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes
Disklabel type: gpt
Disk identifier: 6EBFEF6B-CF70-47B0-9C54-BD6893058E84

Device        Start       End   Sectors  Size Type
/dev/zd32p1    2048   1023999   1021952  499M Windows recovery environment
/dev/zd32p2 1024000   1226751    202752   99M EFI System
/dev/zd32p3 1226752   1259519     32768   16M Microsoft reserved
/dev/zd32p4 1259520 109047807 107788288 51.4G Microsoft basic data


Disk /dev/zd48: 128 KiB, 131072 bytes, 256 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes


Disk /dev/zd64: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes
Disklabel type: gpt
Disk identifier: BBDF5C11-0640-4DC2-89EC-FA5888AF515E

Device       Start      End  Sectors  Size Type
/dev/zd64p1     40   409639   409600  200M EFI System
/dev/zd64p2 409640 67108823 66699184 31.8G unknown

Partition 1 does not start on physical sector boundary.
Partition 2 does not start on physical sector boundary.


Disk /dev/zd80: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes
Disklabel type: dos
Disk identifier: 0x08aebbb8

Device      Boot  Start      End  Sectors  Size Id Type
/dev/zd80p1 *      2048   499711   497664  243M 83 Linux
/dev/zd80p2      501758 67106815 66605058 31.8G  5 Extended
/dev/zd80p5      501760 67106815 66605056 31.8G 8e Linux LVM

Partition 2 does not start on physical sector boundary.


Disk /dev/zd96: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes
Disklabel type: gpt
Disk identifier: 6EBFEF6B-CF70-47B0-9C54-BD6893058E84

Device        Start      End  Sectors  Size Type
/dev/zd96p1    2048  1023999  1021952  499M Windows recovery environment
/dev/zd96p2 1024000  1226751   202752   99M EFI System
/dev/zd96p3 1226752  1259519    32768   16M Microsoft reserved
/dev/zd96p4 1259520 67106815 65847296 31.4G Microsoft basic data


Disk /dev/zd112: 128 KiB, 131072 bytes, 256 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes


Disk /dev/zd128: 128 KiB, 131072 bytes, 256 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes


Disk /dev/zd144: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes
Disklabel type: dos
Disk identifier: 0x08aebbb8

Device       Boot  Start      End  Sectors  Size Id Type
/dev/zd144p1 *      2048   499711   497664  243M 83 Linux
/dev/zd144p2      501758 67106815 66605058 31.8G  5 Extended
/dev/zd144p5      501760 67106815 66605056 31.8G 8e Linux LVM

Partition 2 does not start on physical sector boundary.


Disk /dev/zd160: 128 KiB, 131072 bytes, 256 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes


Disk /dev/zd176: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes
Disklabel type: dos
Disk identifier: 0x5e099284

Device       Boot Start      End  Sectors Size Id Type
/dev/zd176p1 *     2048 67108863 67106816  32G 83 Linux


Note: zd0 is swap
bootsie123`
 
Posts: 10
Joined: 2019-01-02 23:22

Re: Unable to Recover From Suspend

Postby llivv » 2019-01-05 20:03

bootsie123` wrote:
Code: Select all
BOOT_IMAGE=/ROOT/pve-1@/boot/vmlinuz-4.15.18-9-pve root=ZFS=rpool/ROOT/pve-1 ro root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on video=efifb:off


*NOTE 1: intel_iommu=on and video=efifb:off have been off when I've been trying to use s2ram, it's currently there so I can have GPU passthrough in the meantime
Well if you're still wondering why you get the KMS message? now you know.

here's the original dirt for swap on zfs
https://community.oracle.com/thread/1910129

stable uses zfs on fuse - unless I'm getting my debian versions confused
or yotts zfsers are getting zfs outside debianland
In memory of Ian Ashley Murdock (1973 - 2015) founder of the Debian project.
User avatar
llivv
 
Posts: 5867
Joined: 2007-02-14 18:10
Location: cold storage

Re: Unable to Recover From Suspend

Postby bootsie123` » 2019-01-05 20:15

I have tried s2ram without that and I get the same message.

Code: Select all
s2ram -k

Machine is unknown.
This machine can be identified by:
    sys_vendor   = "Dell Inc."
    sys_product  = "Precision WorkStation T7500  "
    sys_version  = ""
    bios_version = "A18"

When I run s2ram by itself I get

Code: Select all
KMS graphics driver is in use, skipping quirks.

and then the system goes into suspend mode and I still get the same suspend recovery issue. Thanks for trying though.
bootsie123`
 
Posts: 10
Joined: 2019-01-02 23:22

Re: Unable to Recover From Suspend

Postby Head_on_a_Stick » 2019-01-05 22:43

bootsie123` wrote:when the server is recovering from suspend mode everything powers back on but then freezes. No disk activity, no video output, no network connection via ethernet, nothing

Can you ssh into the "dead" machine?

FWIW, I run stretch on zfs and have no problems resuming from suspend, even with no swap at all (IIUC swap is only needed for hibernation).

EDIT: I used this method to install:

https://github.com/zfsonlinux/zfs/wiki/ ... oot-on-ZFS
"French riots get results! U lot are instagram zzzombies" — graffiti over Euston underpass
User avatar
Head_on_a_Stick
 
Posts: 8610
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Unable to Recover From Suspend

Postby bootsie123` » 2019-01-05 23:59

Unfortunately, I can't ssh into the "dead" machine. Seeing as you said that suspend works fine for you using zfs and Debian Stretch I mean reinstall Debian and install Proxmox from scratch. I used the Proxmox iso which installed the operating system with Proxmox pre-installed.
bootsie123`
 
Posts: 10
Joined: 2019-01-02 23:22

Re: Unable to Recover From Suspend

Postby llivv » 2019-01-06 00:30

bootsie123` wrote: I used the Proxmox iso which installed the operating system with Proxmox pre-installed.

That explains uname -r

I did give it the old college try - didn't I
In memory of Ian Ashley Murdock (1973 - 2015) founder of the Debian project.
User avatar
llivv
 
Posts: 5867
Joined: 2007-02-14 18:10
Location: cold storage

Re: Unable to Recover From Suspend

Postby bootsie123` » 2019-01-06 00:39

llivv wrote:
bootsie123` wrote: I used the Proxmox iso which installed the operating system with Proxmox pre-installed.

That explains uname -r

I did give it the old college try - didn't I


At least you tried. That's all I can ask for :wink:
bootsie123`
 
Posts: 10
Joined: 2019-01-02 23:22

Re: Unable to Recover From Suspend

Postby Head_on_a_Stick » 2019-01-06 10:27

bootsie123` wrote:I used the Proxmox iso which installed the operating system with Proxmox pre-installed.

Then why are you asking here?

https://www.proxmox.com/en/proxmox-ve/support
"French riots get results! U lot are instagram zzzombies" — graffiti over Euston underpass
User avatar
Head_on_a_Stick
 
Posts: 8610
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Unable to Recover From Suspend

Postby bootsie123` » 2019-01-06 11:42

Already went there. They said it wasn't a Proxmox issue. I have a spare drive so I think I'll install Debian on there and see how it goes.

EDIT: Installed a fresh version of Debian on a spare drive and I got hibernate to work. Although I keep running into the same issues with suspend.
bootsie123`
 
Posts: 10
Joined: 2019-01-02 23:22


Return to System configuration

Who is online

Users browsing this forum: No registered users and 4 guests

fashionable