Using ssh key for authentication has become standard practice in many organizations. It is considered much secure than using a password. While that is true, the flaw in using the ssh key is before we can actually use it for authentication. And this seldom gets highlighted, leading to many IT professional don’t even realize that this is a problem.

Have ever bumped into this when you try to ssh into a new host first time?

$ ssh kamal@51.251.110.111
The authenticity of host '[51.251.110.111] ([51.251.110.111]:22)' can't be established.
ECDSA key fingerprint is SHA256:xouheGRcbdddfdfdfaaaaaaaaaaaaacxxxxxxcadd*
*Are you sure you want to continue connecting (yes/no/[fingerprint])? yes*

In a lot of tutorials around the net, usually, you’ll just be told to type yes, and that’s where all the problems begin.

Unlike TLS/SSL which certificate that requires third party (CA) to prove its validity, SSH doesn’t use any third party to prove the authenticity of the key. Instead, they employ a mechanism called Trust on First Use or ToFU. And that’s what the prompt you see above.

For an ssh connection to establish, it must first make sure 2 things are correct:-

1. The key that the user uses to access the server is correct and allowed - this is basically by adding the public key into the user’s authorized_key file on the server.

2. The client can verify the identity of the server is correct. That’s what the prompt above is. It asks you to verify that you trust that indeed the server that you want to log in.

To verify that, you have to take the server key fingerprint (shown in the prompt) and then compared it with the key that you know belongs to the server. Only if you have verified that it matched, then you should type “yes”.

In the old days, that was not a problem. Usually, you will set up a server in the data center by being present physically in the data center. After setting up the server, you can run a command to see what is its key fingerprint, write it down and go back to the office. Then you will try to ssh first time to the server from your office’s computer. You got the same prompt as above, so you compare the fingerprint presented to the fingerprint that you wrote down on paper while in the data center. It matched and now you have a secure connection from your office to the server in the data center.

But today in all cloud environments, we are no longer present physically in the data center to copy the server’s key fingerprint. Some cloud providers will tell you the fingerprint when you provision the instance in their management console. This is the most ideal situation, assuming we already access their management console securely, with HTTPS for example.

Another mechanism is by providing virtual console access (similar to you plugging the keyboard and monitor into the server) so can you check the fingerprint yourself. But do not mistake this for the web terminal that some cloud provides, most of them just ssh clients, which means you still need to authenticate via ssh, which brings us back to the original problem.

AWS however doesn’t provide such an ideal solution. The only way we can discover the fingerprint is by inspecting the launch log, or the boot log, I’m not sure. But it is such a hassle that people simply type yes when prompted to verify the server fingerprint.

Now we have talked about the problem, I’ll just link to this article that proposes the use of a certificate that will be verified by CA instead to replace the old key authentication.