cannot get sshd to start

[Jethro_UK]

Have nuked and reinstalled openssh-server, so this is the out-of-the-box variety with no config changes made at all.

I also know WHY it won't start. For some reason, the /run/sshd file is either (a) not being created or (b) somehow getting deleted after creation and before /usr/sbin/sshd gets invoked.

All files are in place, including /usr/lib/systemd/tmpfiles.d/sshd.conf which

Code: Select all

systemd-tmpfiles --create

should run to create the directory.

I have a workaround. I can access the system via Webmin or NoMachine, run

Code: Select all

systemd-tmpfiles --create /usr/lib/tmpfiles.d/sshd.conf

and then

Code: Select all

systemctl start ssh

and all is well.

If I just try

Code: Select all

systemctl start ssh

on it's own, I get the following message via journalctl -xe

Code: Select all

May 16 17:00:26 DebServer64 sshd[969]: Missing privilege separation directory: /run/sshd
May 16 17:00:26 DebServer64 systemd[1]: ssh.service: Control process exited, code=exited status=255
May 16 17:00:26 DebServer64 systemd[1]: Failed to start OpenBSD Secure Shell server.

FWIW, Googling the error suggests loads of people have had this problem, but none of the posts I've read seem to apply to my situation. Which suggests there is something else on this system which is conflicting ?

[pylkko]

Have you seen this bug report:

https://askubuntu.com/questions/1110828 ... r-run-sshd

[reinob]

Have nuked and reinstalled openssh-server, so this is the out-of-the-box variety with no config changes made at all.

Could you post the output of (as root):

# systemctl status systemd-tmpfiles-setup
and
# systemctl cat systemd-tmpfiles-setup

You may have masked or disabled the service. This explains (or would explain) why it works when you manually run "systemd-tmpfiles --create".

[Jethro_UK]

Have nuked and reinstalled openssh-server, so this is the out-of-the-box variety with no config changes made at all.

Could you post the output of (as root):

# systemctl status systemd-tmpfiles-setup
and
# systemctl cat systemd-tmpfiles-setup

You may have masked or disabled the service. This explains (or would explain) why it works when you manually run "systemd-tmpfiles --create".

Many thanks for taking the time to reply ...

# systemctl status systemd-tmpfiles-setup

Code: Select all

● systemd-tmpfiles-setup.service - Create Volatile Files and Directories
   Loaded: loaded (/lib/systemd/system/systemd-tmpfiles-setup.service; static; vendor preset: enabled)
   Active: active (exited) since Fri 2019-05-17 16:41:21 BST; 23h ago
     Docs: man:tmpfiles.d(5)
           man:systemd-tmpfiles(8)
 Main PID: 330 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 4915)
   CGroup: /system.slice/systemd-tmpfiles-setup.service

May 17 16:41:19 DebServer64 systemd[1]: Starting Create Volatile Files and Directories...
May 17 16:41:21 DebServer64 systemd[1]: Started Create Volatile Files and Directories.

and
# systemctl cat systemd-tmpfiles-setup

Code: Select all

# /lib/systemd/system/systemd-tmpfiles-setup.service
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=Create Volatile Files and Directories
Documentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8)
DefaultDependencies=no
Conflicts=shutdown.target
After=local-fs.target systemd-sysusers.service
Before=sysinit.target shutdown.target
RefuseManualStop=yes

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/systemd-tmpfiles --create --remove --boot --exclude-prefix=/dev

If there's no obvious fix, I'll write a script to run the systemd-tempfiles utility for sshd and then do a service start, and run it a minute after bootup.

[reinob]

@Jethro_UK,

The systemd unit looks OK to me. Do you have a separate partition for /var? or some sort of "non-standard" partitioning? (modern linux distributions tend to dump everything in one partition, except perhaps /boot and /home, and tend to disregard old traditions such as a separate /var and/or /usr).

(I've had a fight or two with systemd-tmpfiles just for having e.g. /var/cache as a symlink to another place.)

You may want to post your /etc/fstab, in case there's something suspicious there.

[Jethro_UK]

@Jethro_UK,

The systemd unit looks OK to me. Do you have a separate partition for /var? or some sort of "non-standard" partitioning? (modern linux distributions tend to dump everything in one partition, except perhaps /boot and /home, and tend to disregard old traditions such as a separate /var and/or /usr).

(I've had a fight or two with systemd-tmpfiles just for having e.g. /var/cache as a symlink to another place.)

You may want to post your /etc/fstab, in case there's something suspicious there.

Once again, thanks for taking the trouble to reply.

as far as I know, the system is all on the same partition - I didn't do anything odd when installing. Here's /etc/fstab. The only thing is I have 3 RAID arrays hanging off the machine. But they *should* be data only. Nothing to do with the system.

Code: Select all

## /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sdb1 during installation
UUID=88413846-f1d8-4ec6-b28e-a36c61e61fea /               ext4    errors=remount-ro 0       1
# swap was on /dev/sdb5 during installation
UUID=9e95d173-4af8-4629-9db3-d7a641fb0188 none            swap    sw              0       0
# /dev/sr0        /media/cdrom0   udf,iso9660 user,noauto     0       0
/dev/cdrom        /media/cdrom0   udf,iso9660 user,noauto     0       0
/dev/md0	/mnt/MediaRAID	ext4	errors=continue,_netdev	0	0
/dev/md1	/mnt/DataRAID	ext4	_netdev,errors=continue	0	0
/mnt/MediaRAID/Media_Files/Audio/Podcasts	/export/Podcasts	bind	bind	0	
LABEL=sda_spare	/mnt/sda_spare	ext4	defaults	0	0
/mnt/md2/BackupPC/backuppc	/var/lib/backuppc	bind	bind	0	0
/dev/md2	/mnt/md2	ext4	defaults	0	0
#/dev/md2	/var/lib/backuppc	ext4	relatime,data=ordered	0	0

[Jethro_UK]

In the absence of a fix, the only way to get SSH to start is to run a script:

systemd-tmpfiles --create /usr/lib/tmpfiles.d/sshd.conf
systemctl start ssh

and all is well.

Word of warning: if you're using NFS shares from the server, you'll need to (re)start the nfs-kernel-server daemon.

[Jethro_UK]

Whatever had happened, the upgrade to Buster seems to have put it straight.