Build, install, and run the latest OpenSSH Server as a systemd service.

• openssh latest
  • About
  • Prepare /opt
  • apt install
  • Download
    • Verify the download using gpg
      • Import the public signing key
      • Verify
  • Build
    • Change the banner (optional)
  • Create Symlink for -latest
  • Set aside built-in OpenSSH
  • Create systemd service -latest
    • Copy the default environment file
    • Enable and restart
  • Verify new ssh service and old ssh service
• openssh updates
  • Copy the Previous -latest to the New -latest
    • Use default key files (optional)
  • Update Symlinks for -latest
  • Restart Service
  • Test

Running the latest OpenSSH Server is easy and security-wise. This can run side-by-side with the package-manager installed version of OpenSSH (this is recommended).

This is for a Debian-derived Linux System using systemd. This document will use version 9.0p1.

OpenSSH will be built on the system that will also run the ssh service.

Do this once.

mkdir -vp /opt/

Do this once.

List the dpkg requirements for OpenSSH server

apt show openssh-server

Most likely, the packages listed under Depends: will be needed to build OpenSSH.

My typical Debian system addtionally needed these

apt install \
    libssl-dev \
    gcc g++ gdb cpp \
    make cmake \
    libtool \
    libc6 \
    autoconf automake pkg-config \
    build-essential \
    gettext \

(I am not sure if all of these packages are needed, but it did the trick)

I suspect you also need these

apt install \
    libzstd1 zlib1g \
    libssh-4 libssh-dev libssl3 \
    libc6-dev libc6 \
    libcrypt-dev

These guesses have not been thoroughly tested.

Other helpful tools for the build and install process:

apt install netcat lsof wget diffutils

Do this each update.

1. Pick a mirror

2. Download the latest archive. This will example will use https://cdn.openbsd.org/pub/OpenBSD/OpenSSH/openssh-9.0p1.tar.gz.

   wget https://cdn.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-9.0p1.tar.gz
   

Do this once.

wget https://cdn.openbsd.org/pub/OpenBSD/OpenSSH/RELEASE_KEY.asc
gpg --import RELEASE_KEY.asc

Do this each update.

wget https://cdn.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-9.0p1.tar.gz.asc
gpg --verbose --verify openssh-9.0p1.tar.gz.asc

Do this each update.

Preview the next subsection before running these commands.

VER=9.0p1
tar -xvf openssh-${VER}.tar.gz
cd openssh-${VER}
./configure --prefix=/opt/openssh-${VER}
make
make install

If ./configure is missing then the non-portable version was downloaded.

This section has not yet successfully tested 😔. Come back later.

Change string that all connecting SSH clients receive (this occurs before authentication). This requires a change before running make from the previous Build section.

The statement that writes the "banner" string to the connected SSH channel is in file ./ssh_api.c.

if ((r = sshbuf_putf(banner, "SSH-2.0-%.100s\r\n", SSH_VERSION)) != 0)

Before running make, change the file ./ssh_api.c

sed -i -Ee 's|(sshbuf_putf\(banner, )("SSH-.*", SSH_VERSION)(\))|\1"please_go_away"\3|' -- ./ssh_api.c

This should change the statement to

if ((r = sshbuf_putf(banner, "please_go_away")) != 0)

Do this once.

ln -fvs /opt/openssh-${VER} /opt/openssh-latest

Do this once.

Very Important! Keep the functioning built-in sshd service in-place or running at a different port.

1. Change the system built-in SSH Service /etc/ssh/sshd_config, add a non-typical port to the Ports declaration, for example 2222.
2. Restart the system built-in SSH Service.
3. Login to the SSH service listening using the non-typical port 2222. ssh -p 2222 user@server
4. Remove from the Ports declaration the typical port 22.
5. Restart the system built-in SSH Service.

OR

1. Have the latest SSH service use a non-standard port(s), like 2222.

Now there is a reliable fallback SSH Service.

Do this once.

Copy the systemd template service files for the ssh service.

This path may vary among /lib/systemd/system, or /usr/lib/systemd/system, or something else. Be sure not to use the generated systemd files typically found at /etc/systemd.

Find all *ssh* files

find / -xdev -name '*ssh*' | sort

Copy the systemd service files. In my case on Debian 11

cd /usr/lib/systemd/system/
cp -av ssh.service ssh-latest.service
cp -av [email protected] [email protected]
cp -av ssh.socket ssh-latest.socket
cp -av rescue-ssh.target rescue-ssh-latest.target

In the newly copied ssh-latest* systemd files, manually update references from ssh to ssh-latest.

In my case on Debian 11, the changes looked like:

$ cd /usr/lib/systemd/system

$ diff -y --suppress-common-lines [email protected] [email protected]
EnvironmentFile=-/opt/openssh-latest/default/ssh              | EnvironmentFile=-/etc/default/ssh
ExecStart=/opt/openssh-latest/sbin/sshd -i $SSHD_OPTS         | ExecStart=-/usr/sbin/sshd -i $SSHD_OPTS
RuntimeDirectory=sshd-latest                                  | RuntimeDirectory=sshd

$ diff -y --suppress-common-lines ssh-latest.service ssh.service
ConditionPathExists=!/opt/openssh-latest/etc/sshd_not_to_be_r | ConditionPathExists=!/etc/ssh/sshd_not_to_be_run
EnvironmentFile=-/opt/openssh-latest/default/ssh              | EnvironmentFile=-/etc/default/ssh
ExecStartPre=/opt/openssh-latest/sbin/sshd -t                 | ExecStartPre=/usr/sbin/sshd -t
ExecStart=/opt/openssh-latest/sbin/sshd -D $SSHD_OPTS         | ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
ExecReload=/opt/openssh-latest/sbin/sshd -t                   | ExecReload=/usr/sbin/sshd -t
Type=exec                                                     | Type=notify
RuntimeDirectory=sshd-latest                                  | RuntimeDirectory=sshd
Alias=sshd-latest.service                                     | Alias=sshd.service

$ diff -y --suppress-common-lines ssh-latest.socket ssh.socket
Before=ssh-latest.service                                     | Before=ssh.service
Conflicts=ssh-latest.service                                  | Conflicts=ssh.service
ConditionPathExists=!/opt/openssh-latest/etc/sshd_not_to_be_r | ConditionPathExists=!/etc/ssh/sshd_not_to_be_run
ListenStream=2222                                             | ListenStream=22

$ diff -y --suppress-common-lines rescue-ssh.target rescue-ssh-latest.target
Requires=network-online.target ssh.service                    | Requires=network-online.target ssh-latest.service
After=network-online.target ssh.service                       | After=network-online.target ssh-latest.service

Notice the change of Type from notify to exec.

mkdir -vp /opt/openssh-latest/default/
cp -av /etc/default/ssh /opt/openssh-latest/default/

The /opt/openssh-latest/default/ssh file sets the $SSHD_OPTS which is used by the service file.

Tell the systemd daemon to enable and start the new services

systemctl enable ssh-latest.service
systemctl enable ssh-latest.socket
systemctl daemon-reload
systemctl start ssh-latest

The command systemctl daemon-reload will generate new files under /etc/systemd.

Watch the service logs

journalctl -f -x -u ssh-latest

Check services status

systemctl status ssh-latest

Check the service replies with the expected latest openssh version. For each port with a listening ssh service

echo | nc localhost 22
echo | nc localhost 2222

This should look like:

$ echo | nc localhost 2222
SSH-2.0-OpenSSH_9.0
Invalid SSH identification string.

Check there are two different sshd -D daemon processes

ps -ef --forest | grep "sshd -D"

Check the ports in-use

lsof -PVn -iTCP | grep sshd | grep LISTEN

Do this each update.

After a new version of OpenSSH Server is released, it's relatively easy to update things.

Repeat section Download, and Build.

cd /opt/openssh-${VER}/etc/
# set aside default config files
mkdir -vp _original
mv -v ssh_host_* sshd_config ssh_config _original/
cp -av /opt/openssh-latest/etc/ssh_host_* /opt/openssh-latest/etc/{sshd_config,ssh_config} .

Optionally, the /opt/openssh-latest/etc/sshd_config can refer to the key files at /etc/ssh. Then there is no need to copy the /opt/openssh-latest/etc/ssh_host_* files.

The file /opt/openssh-latest/etc/sshd_config would have lines

HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key

systemctl stop ssh-latest
cd /opt
rm -v openssh-latest && ln -fvs openssh-${VER} openssh-latest

Watch the service logs

journalctl -f -x -u ssh-latest

Restart

systemctl restart ssh-latest

For each port with a listening ssh service

echo | nc localhost 22
echo | nc localhost 2222