> on the Devuan machine in /etc/ssh/sshd_config.d/local.conf (you shouldn't
> change /etc/ssh/sshd_config -> eases next upgrade for you). Restart ssh.
Hmm, "eases" is putting it mildly. Last month, the Debian VM hosting a
GitLab instance I maintain at the office refused to start and didn't let
me ssh in after an unattended-upgrades induced reboot.
The VM had been configured to run sshd on a custom port some five years
ago by changes the `Port` in /etc/ssh/sshd_config. I don't think the .d
directory was supported at the time. This setup allowed me to do system
maintance over ssh on the custom port while the GitLab Docker container
could use the default port 22 for its sshd.
That worked like a charm until last month's upgrade of ssh. It silently
clobbered /etc/ssh/sshd_config, nuking the port customization. When the
VM rebooted ssh came up first *on the default port* and the GitLab
container refused to start because it couldn't use that port anymore.
I noticed our GitLab not responding, tried to ssh in like I had been
doing the last five years only to find out I couldn't connect ...
Fortunately, I have logcheck running on the VM so the bulk of the reboot
log messages ended up in my mailbox. Pouring over those, I noticed that
ssh had started and GitLab couldn't get port 22. It was only after a
bit of thinking that I realised sshd was using port 22 so I should be
able to ssh in via that port instead of the custom port.
I also use etckeeper, so have a blow-by-blow record of the changes below
/etc. That is, I can see what that sshd_config file looked like at any
given time since it first appeared. The file was obviously clobbered
and reading up I noticed that sshd_config.d support had been added some
time back. Now my custom port settings lives in a file there.
Restarting the ssh service and the GitLab container put things back in
working order .
While everything worked out fine, eventually, finding out that your
carefully maintained office-wide service is down first thing Monday
morning is not exactly a fun start of the week.
BTW, when things like this happen, I usually go for a coffee break so I
calm down and think things through. Panicking and trying "fixes" has a
tendency to pull you deeper into trouble.
This message was posted to the following mailing lists: