Table of Contents
Sometimes when using (newer) OpenSSH 7.x or higher clients you may have multiple issues trying to connect and use public key authentication (PKI) with an OpenVMS server. The error messages will often complain about “no matching key exchange method found.” You may also get error messages related to the use of ssh-dss such as “no matching host key type found. Their offer: ssh-dss”. Finally, on some older versions of VMS, you might also see errors related to ciphers (such as aes-128-CBC) being deprecated. All of these issues are pretty easy to fix, but require some specific flags to your ssh client.
The other problem is that OpenSSH and the commercial Secure Shell server in VMS do not use the same key file format. So, we'll need to convert the key before it's ready for use.
Why It Happened
It's crypto. Sometimes researchers find that the ways in which they were doing things no longer works because of new discoveries which put old methods at risk. Because of this, we've seen ciphers, key exchange algorithms, host key algorithms, and other areas become deprecated in newer OpenSSH clients and servers. Most of the problems with VMS seem to occur after OpenSSH version 7.4p1.
Here is a truth table of what you can work around and what you can't
|OpenSSH v6.x Server
|OpenSSH v7.x Server
|OpenVMS Secure Shell 8.4 or older Server
|OpenVMS 8.4 MultiNet
|2 Workarounds needed
|OpenVMS 8.4 TCP/IP Services
|2 Workaround needed
|3 workarounds needed
|Red Hat 8 or newer
|Broken. No workaround.
Workaround For Key Exchange Problem
You can force newer OpenSSH clients (such as for Linux) to support backwards compatibility for VMS using a special option “-oKexAlgorithms=+diffie-hellman-group1-sha1”. This works for ssh, scp, and sftp command line programs in the same way.
Workaround for ssh-dss HostKey Problem
You can likewise also fix the ssh-dss issue by adding the option “-oHostKeyAlgorithms=+ssh-dss” to your command line. This also works for ssh, scp, and sftp. Remember, these options only work when your Linux/Unix box is the client and VMS is the server. These options absolutely do no work for VMS clients, but then again, you don't need them in that context, either.
Workaround for Missing Ciphers
If you get an error from your ssh client about a missing cipher on the server side, it's probably because the client as deprecated all the algos that the server supports. However, you can usually force the client into backwards compatibility. The way to do this is with a “-c mycipher” flag to your command. The cipher name should be one that is listed in the error under “their offer”. You can see what ciphers you can support by typing ssh -Q cipher. You'll have to pick one they both agree on. I usually pick aes128-cbc for VMS servers.
Using the workarounds
You can use all the workarounds together and in many cases you will need to. Here is an example where all three workarounds are in play.
ssh -o KexAlgorithms=+diffie-hellman-group1-sha1 \ -o HostKeyAlgorithms=+ssh-dss \ -c aes256-cbc \ myuser@remotehost
If you don't want to have to type all that out every time, you may consider setting up a host entry in your local ssh client configuration. This would be your $HOME/.ssh/config file. This file might not exist if you haven't used it before (it's not created by default). Here is an example entry in the configuration file (remember this is on your Linux/Unix box, not VMS).
Host MyVMSbox KexAlgorithms +diffie-hellman-group1-sha1 HostKeyAlgorithms +ssh-dss Ciphers aes128-cbc
That will free you from having to type out all the options every time. Additionally, you can remove the “Host” line and just put those options at the top of your config file and thus that would change your default configuration.
Using OpenSSH Public Keys on OpenVMS Systems
You can use OpenSSH keys on VMS systems, but there are some gotchas. First off, you need to be using RSA keys. DSA keys don't always work depending on the version of VMS you happen to be dealing with. The procedure goes something like this:
- On the Linux/Unix side run ssh-keygen -t rsa then answer the prompts. You'll notice the default place to save the keys is $HOME/.ssh
- Now convert the public key to RFC4716 format for OpenVMS to use. ssh-keygen -e -f id_rsa.pub | tee mykey.pub
- Copy the file to your VMS system and put it in the .SSH sub-directory of your home-directory on VMS.
- On the VMS server edit or create a file called AUTHORIZATION under you .SSH sub-directory in your home-directory. Add a line to the file that specifies your key file like this “KEY mykey.pub” (or whatever the file name actually is for your key).
- make sure the permissions on your key file and AUTHORIZATION file are strict enough, otherwise the server will reject your changes. Something like this usually works: SET FILE /PROTECTION=(S:WRED,O:WRED,G:RE,W:R) [.SSH2]mykey.pub
That should do it. Now you should be able to ssh from Linux/Unix into VMS using only your key.