Using SSH Keys to Log Into Remote Servers

8 min readJan 28, 2018

Let’s quickly set the stage.
In order to administer, or just work on remote servers, we must use SSH to make a connection from our terminal.

This article will show you exactly how to generate a public/private SSH key-pair, and it will demonstrate how to install your newly generated public SSH key into remote servers. This will help streamline your experience connecting from your local machine’s terminal, and it will allow for the possibility of the remote server to be put into a state where it only accepts connections from users with SSH keys. This approach can help to harden the security of the remote server, especially if the remote is not behind a dedicated firewall.

this is an example of a firewall.. aka a Cisco Adaptive Security Appliance..

After completion of the steps outlined in this article, you will be able to connect to one or more remote servers using your SSH keys, rather than entering your Linux password for authentication to the remote server.

The pre-requisite for this article is that you already have access credentials to a remote server. There are free or almost free services from AWS, Digital Ocean and the like that would allow you to create a remote server instance, if you would like to go that route for experimentation.

I thought everyone already did this, but I’m discovering that using SSH keys to authenticate to your remote servers is less common than I had previously imagined.

At the time of writing, my personal machine is running Arch-based Manjaro Linux with the i3 improved, tiling window manager, with Kernel: Linux 4.9.77–1-MANJARO. The commands I’ll list in this article work on this setup, but they should also be translatable to your Unix environment.

You have the option to generate a RSA or DSA cryptographic key-pair. Without digressing into a discussion of the difference between RSA and DSA, let’s use RSA for this example:

Generate Your SSH Key-Pair

$ ssh-keygen -t rsa

After entering the above command, you will get to confirm where the resultant key-pair is stored on your machine, and you will be prompted to enter a passphrase to protect your private key locally. It’s fine to just hit enter and write your public and private key pair to the default location of ~/.ssh/id_rsa. Leaving the passphrase blank is also acceptable and practical in many cases, especially if your hard drive is encrypted and you physically protect your machine. If you’re in the habit of leaving your machine on, unencrypted and lying around in public, then you might want an additional passphrase here. You will then see you public key’s fingerprint and randomart image.

In order to check out the resultant key-pair, you might use:

$ cd ~/.ssh && ls

Install your Public Key to the Remote Server, by executing a command on your local machine

Next, we should install our public key on our remote server. The remote servers that I currently administer are all running RHEL. For this example, I’ll list some commands that work on my systems. I’m also going to assume that you use the default naming, so that your public key is “id_rsa.pub”

On your local machine, from the directory where your public key resides, in this case “~/.ssh” or “/home/username/.ssh”, use this command to install your public key onto the remote server we’re accessing.
Remember, this should be run from your local machine:

$ ssh-copy-id <username>@<IP-address-of-remote>

So, let’s say that your username on the remote machine is granular, and the IP address of the remote host is 192.168.1.1. In this case, we would use:

$ ssh-copy-id granular@192.168.1.1

After seeing success, similar to the image above, we can attempt to connect to the remote serer using simply:

$ ssh <username>@<IP-address-of-remote>
$ ssh granular@192.168.1.1

It’s also possible that you got a “WARNING: UNPROTECTED PRIVATE KEY FILE!” error. You might see something like: “Permissions 0644 for ‘id_rsa.pub’ are too open. This private key will be ignored.” If you see this, it just means that you’ve specified the public key instead of the private key in your .ssh/config file.

If you entered a passphrase when you were creating your SSH key-pair, you’ll have to enter it after executing “” from your local machine.
If your goal is ultimate convenience and your local machine’s hard drive is strongly encrypted, you might think about not entering a password when creating your SSH key-pair.
Personally, my hard drive is strongly encrypted, and I use a passphrase to protect my local SSH keys.

In my world, currently there are 10 remote servers that I need to log into on a regular basis. Installing my public SSH key on all of them allows me to conveniently log into each of them while also having unique passwords on each of them. When I execute:

$ ssh <username>@<IP-address-of-remote>

I can simply enter the passphrase used to secure my local private key, so SSH can manage the asymmetric cryptographic transaction that authenticates me to the remote machines.
This allows me to have ridiculously-long, random, unique passwords on all of the remote machines, but all I have to do to connect is enter the above command and the passphrase protecting my local private SSH key.
With this setup, authentication to remote servers is BOTH more convenient and potentially more secure because the remote machines can be set to only allow connections from client machines with SSH keys installed.

Edit-20180524: It might be helpful to add or amend to the ~/.ssh/config

$ vim ~/.ssh/config# Here's an example:Host sandbox_server
 HostName 112.13.20.7
 User granular
 IdentityFile ~/.ssh/id_rsa

So, the “Host” value in this case, sandbox_server, is the name that you want to resolve the “HostName” to for “User” granular and pointing to our user’s private ssh key. Make sure you are specifying the private key and not the public key here.
This setup allows for a very convenient:

$ ssh sandbox_server
-- notice ^^^ how the IP address and username are not required because we already specified those in our ~/.ssh/config file.

If we’re set up properly, that alone will log you in to your remote machine. Likely it’s easier to remember a name like sandbox_server rather than the IP address. So just define these in a file you call “config” in your ~/.ssh directory and ssh from the command line becomes more convenient.
This is what I’m using at the time of writing.

Best,
Tim Beach

2019–04–09 added section for Windows users:

— install Git for Windows from https://git-scm.com/download/win
— Use all the defaults (this will also get you vim by default) until you get to the “Adjusting your PATH environment,” at which point, I recommend you use the top radio button option: “Use Git from Git Bash only,” unless you know what you’re doing
— Keep following all the defaults unless you know what you’re doing
— Launch Git Bash which will get you a terminal that will be useful for the next steps:
— your default directory will be the Windows equivalent of “~/” or your home folder “/c/Users/your-user. From here, you can create your ssh key pair with

$ ssh-keygen -t rsa

(see section above Generate Your SSH Key-Pair, if you’re following along, you’ll notice it’s the same. Quick summary below):
— hit enter to place keys in default location
— hit enter to leave the passphrase protection off for local keys
— you’ll then see the key’s randomart and your keys are created
— check your newly created .ssh directory:

$ cd .ssh && ls -lsah

create your config file

$ vim config

— type “i” to get to insert mode, then paste in the details for the server you are trying to connect to. For example:

Host server-name
 HostName 123.45.67.8
 User serverUserName
 IdentityFile ~/.ssh/id_rsa

— type Escape to exit insert mode
— type “:” then “wq” to write and quit
— now you’re ready to connect to the remote machine from the terminal
— set permissions on the files id_rsa (your private key) and your config, while you’re still in the same directory:

$ chmod 600 id_rsa && chmod 600 config

You might need to first connect to a VPN if the server you’re connecting to is protected in that way. If you need a Windows VPN client, you can get a nice one here: https://github.com/openconnect/openconnect-gui/releases

— now ssh to the server

$ ssh server-name

You’ll get an authenticity warning with the ECDSA key fingerprint. If you know what you’re doing, type “yes” then Enter.

This assumes that your public key was installed to your Linux user on the remote machine you are connecting to.

pps ~

I made a video describing some of this stuff and going into detail how to manually install your colleague’s public ssh key on your remote server: