cannot get sshd to start

Kernels & Hardware, configuring network, installing services

cannot get sshd to start

Postby Jethro_UK » 2019-05-17 10:03

Have nuked and reinstalled openssh-server, so this is the out-of-the-box variety with no config changes made at all.

I also know WHY it won't start. For some reason, the /run/sshd file is either (a) not being created or (b) somehow getting deleted after creation and before /usr/sbin/sshd gets invoked.

All files are in place, including /usr/lib/systemd/tmpfiles.d/sshd.conf which
Code: Select all
systemd-tmpfiles --create
should run to create the directory.

I have a workaround. I can access the system via Webmin or NoMachine, run
Code: Select all
systemd-tmpfiles --create /usr/lib/tmpfiles.d/sshd.conf
and then
Code: Select all
systemctl start ssh
and all is well.

If I just try
Code: Select all
systemctl start ssh
on it's own, I get the following message via journalctl -xe

Code: Select all
May 16 17:00:26 DebServer64 sshd[969]: Missing privilege separation directory: /run/sshd
May 16 17:00:26 DebServer64 systemd[1]: ssh.service: Control process exited, code=exited status=255
May 16 17:00:26 DebServer64 systemd[1]: Failed to start OpenBSD Secure Shell server.


FWIW, Googling the error suggests loads of people have had this problem, but none of the posts I've read seem to apply to my situation. Which suggests there is something else on this system which is conflicting ?
Jethro_UK
 
Posts: 24
Joined: 2014-08-12 09:31

Re: cannot get sshd to start

Postby pylkko » 2019-05-18 06:34

User avatar
pylkko
 
Posts: 1526
Joined: 2014-11-06 19:02

Re: cannot get sshd to start

Postby reinob » 2019-05-18 12:42

Jethro_UK wrote:Have nuked and reinstalled openssh-server, so this is the out-of-the-box variety with no config changes made at all.


Could you post the output of (as root):

# systemctl status systemd-tmpfiles-setup
and
# systemctl cat systemd-tmpfiles-setup

You may have masked or disabled the service. This explains (or would explain) why it works when you manually run "systemd-tmpfiles --create".
reinob
 
Posts: 699
Joined: 2014-06-30 11:42

Re: cannot get sshd to start

Postby Jethro_UK » 2019-05-18 14:31

reinob wrote:
Jethro_UK wrote:Have nuked and reinstalled openssh-server, so this is the out-of-the-box variety with no config changes made at all.


Could you post the output of (as root):

# systemctl status systemd-tmpfiles-setup
and
# systemctl cat systemd-tmpfiles-setup

You may have masked or disabled the service. This explains (or would explain) why it works when you manually run "systemd-tmpfiles --create".


Many thanks for taking the time to reply ...

# systemctl status systemd-tmpfiles-setup
Code: Select all
● systemd-tmpfiles-setup.service - Create Volatile Files and Directories
   Loaded: loaded (/lib/systemd/system/systemd-tmpfiles-setup.service; static; vendor preset: enabled)
   Active: active (exited) since Fri 2019-05-17 16:41:21 BST; 23h ago
     Docs: man:tmpfiles.d(5)
           man:systemd-tmpfiles(8)
 Main PID: 330 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 4915)
   CGroup: /system.slice/systemd-tmpfiles-setup.service

May 17 16:41:19 DebServer64 systemd[1]: Starting Create Volatile Files and Directories...
May 17 16:41:21 DebServer64 systemd[1]: Started Create Volatile Files and Directories.


and
# systemctl cat systemd-tmpfiles-setup
Code: Select all
# /lib/systemd/system/systemd-tmpfiles-setup.service
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=Create Volatile Files and Directories
Documentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8)
DefaultDependencies=no
Conflicts=shutdown.target
After=local-fs.target systemd-sysusers.service
Before=sysinit.target shutdown.target
RefuseManualStop=yes

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/systemd-tmpfiles --create --remove --boot --exclude-prefix=/dev


If there's no obvious fix, I'll write a script to run the systemd-tempfiles utility for sshd and then do a service start, and run it a minute after bootup.
Jethro_UK
 
Posts: 24
Joined: 2014-08-12 09:31

Re: cannot get sshd to start

Postby reinob » 2019-05-18 16:43

@Jethro_UK,

The systemd unit looks OK to me. Do you have a separate partition for /var? or some sort of "non-standard" partitioning? (modern linux distributions tend to dump everything in one partition, except perhaps /boot and /home, and tend to disregard old traditions such as a separate /var and/or /usr).

(I've had a fight or two with systemd-tmpfiles just for having e.g. /var/cache as a symlink to another place.)

You may want to post your /etc/fstab, in case there's something suspicious there.
reinob
 
Posts: 699
Joined: 2014-06-30 11:42

Re: cannot get sshd to start

Postby Jethro_UK » 2019-05-19 09:18

reinob wrote:@Jethro_UK,

The systemd unit looks OK to me. Do you have a separate partition for /var? or some sort of "non-standard" partitioning? (modern linux distributions tend to dump everything in one partition, except perhaps /boot and /home, and tend to disregard old traditions such as a separate /var and/or /usr).

(I've had a fight or two with systemd-tmpfiles just for having e.g. /var/cache as a symlink to another place.)

You may want to post your /etc/fstab, in case there's something suspicious there.


Once again, thanks for taking the trouble to reply.

as far as I know, the system is all on the same partition - I didn't do anything odd when installing. Here's /etc/fstab. The only thing is I have 3 RAID arrays hanging off the machine. But they *should* be data only. Nothing to do with the system.

Code: Select all
## /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sdb1 during installation
UUID=88413846-f1d8-4ec6-b28e-a36c61e61fea /               ext4    errors=remount-ro 0       1
# swap was on /dev/sdb5 during installation
UUID=9e95d173-4af8-4629-9db3-d7a641fb0188 none            swap    sw              0       0
# /dev/sr0        /media/cdrom0   udf,iso9660 user,noauto     0       0
/dev/cdrom        /media/cdrom0   udf,iso9660 user,noauto     0       0
/dev/md0   /mnt/MediaRAID   ext4   errors=continue,_netdev   0   0
/dev/md1   /mnt/DataRAID   ext4   _netdev,errors=continue   0   0
/mnt/MediaRAID/Media_Files/Audio/Podcasts   /export/Podcasts   bind   bind   0   
LABEL=sda_spare   /mnt/sda_spare   ext4   defaults   0   0
/mnt/md2/BackupPC/backuppc   /var/lib/backuppc   bind   bind   0   0
/dev/md2   /mnt/md2   ext4   defaults   0   0
#/dev/md2   /var/lib/backuppc   ext4   relatime,data=ordered   0   0
Jethro_UK
 
Posts: 24
Joined: 2014-08-12 09:31

Re: cannot get sshd to start

Postby Jethro_UK » 2019-06-18 12:52

In the absence of a fix, the only way to get SSH to start is to run a script:

systemd-tmpfiles --create /usr/lib/tmpfiles.d/sshd.conf
systemctl start ssh

and all is well.

Word of warning: if you're using NFS shares from the server, you'll need to (re)start the nfs-kernel-server daemon.
Jethro_UK
 
Posts: 24
Joined: 2014-08-12 09:31

Re: cannot get sshd to start

Postby Jethro_UK » 2019-08-14 15:33

Whatever had happened, the upgrade to Buster seems to have put it straight.
Jethro_UK
 
Posts: 24
Joined: 2014-08-12 09:31


Return to System configuration

Who is online

Users browsing this forum: No registered users and 13 guests

fashionable