Editor’s note: This post was originally published on Medium. We are republishing it with Abir’s permission.
Photo Credit: New Yorker
I remember the first time I used SSH. We were told on campus that we couldn’t
telnet into servers anymore (PS I’m old). I had no idea why at the time
(spoiler: it was hackers). Hard to believe this was over 20 years ago, and I’m
still learning new SSH tricks (sidenote: I feel like I’ve been learning
20 years too). It was only a few years ago I learned that I could save myself
from entering a password by using public key authentication for SSH.
At first it sounded magical, but I worried the process would be really complicated. This deterred me from learning how to do it for the longest time.
Me avoiding learning SSH. Photo Credit: giphy
It turns out that it’s not complicated at all! However, a lot can go wrong and confusing error messages related to SSH drove me mad!
I’ve seen a lot of articles written about how to setup public key authentication for SSH, but I hadn’t seen articles about what to do if things go wrong. So I decided to write one.
Let’s say I’ve got 3 machines:
- professorx.local † - my personal Mac laptop
- wolverine.vm † - centos 7 vm
- deadpool.vm † - ubuntu 16.04 vm
I want to be able to easily access the centos and ubuntu machines from my Mac.
In this example, I’ll want to be able to log in as the
abir user on those
The first thing I need to do create a SSH key on my laptop,
ssh-keygen. To make my life easier, I'm gonna
leave the passphrase blank. If you're concerned about security (and you should
be) you should not leave the passphrase blank. If you want to set a passphrase
I recommend reading this article. For
more information about public key authentication for ssh, check out the
[[email protected]]$ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/Users/abir/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /Users/abir/.ssh/id_rsa. Your public key has been saved in /Users/abir/.ssh/id_rsa.pub. The key fingerprint is: b9:67:64:cc:27:19:3a:d9:e0:65:41:bd:f9:cd:18:13 [email protected] The key's randomart image is: +--[ RSA 2048]----+ | .o. | | .. E | | . + o . | | . @ oo o | | S O .. * | | = o o o| | . o | | o | | | +-----------------+ [[email protected]]$ cat /Users/abir/.ssh/id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDsR/hrRCRCgJBFwOPhkcsZYFgvTL4GeeCPAdlCPbDbi3N9v8/mco27VF6Daxf8+nwUBW5gC5uuj1LbrrubdSHh6UniP1uzgLElI47jedB14sfs+wdtTvWgr+mWJIr/efNRqMY1CMqE1A7KXc9B2oMsVSo1UOKEfuRbdNW8DHkDof7L1dJ3ZmmP87Vx+r6wUMWHK5sVGb9k+69aikqEVJzifIaZDN6PbnPdKIDqKL3WThee1QXtSypb/3V2IH/nZKHwhEEZQH4FxA7Yz7K8G9a/wmGjvboqeLCfnw6vogDlQeHx7InGNuqaoaFP+8ccXi837RhgOhHXLHkjFCRjB/+n [email protected]
Once I’ve got the public key generated I need to append the contents of it to
.ssh/authorized_keys file on the remote machines. I have 3 ways to get
that public key on those machines:
a. Copy and Paste
I can copy the contents of
into my clipboard.
Pro Tip: If you're on a Mac, you can easily copy the contents of a file into the
clipboard from the command line using the
[[email protected]]$ pbcopy < ~/.ssh/id_rsa.pub
Now I need to log into wolverine, and take the output from the clipboard and
paste it into
[[email protected]]$ echo ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDsR/hrRCRCgJBFwOPhkcsZYFgvTL4GeeCPAdlCPbDbi3N9v8/mco27VF6Daxf8+nwUBW5gC5uuj1LbrrubdSHh6UniP1uzgLElI47jedB14sfs+wdtTvWgr+mWJIr/efNRqMY1CMqE1A7KXc9B2oMsVSo1UOKEfuRbdNW8DHkDof7L1dJ3ZmmP87Vx+r6wUMWHK5sVGb9k+69aikqEVJzifIaZDN6PbnPdKIDqKL3WThee1QXtSypb/3V2IH/nZKHwhEEZQH4FxA7Yz7K8G9a/wmGjvboqeLCfnw6vogDlQeHx7InGNuqaoaFP+8ccXi837RhgOhHXLHkjFCRjE/+n [email protected] >> /home/abir/.ssh/authorized_keys
Caution: Copy and pasting the contents can be tricky and error prone.
b. Use scp
To simplify things, I could have also scp’d
id_rsa.pub to wolverine, and then
appended the contents of that file to
[email protected]]$ scp id_rsa.pub [email protected]:./id_rsa_professorx.pub password: **** [[email protected]]$ sudo cat ./id_rsa_professorx.pub >> /home/abir/.ssh/authorized_keys
I could have also used the
[[email protected]]$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected] password: ****
I should be able to ssh into wolverine.vm now.
[[email protected]]$ ssh [email protected] The authenticity of host 'wolverine.vm (10.32.174.159)' can't be established. ECDSA key fingerprint is 51:31:cb:67:ce:28:c8:62:87:3a:a6:b7:87:0a:c5:d7. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'wolverine.vm,10.32.174.159' (ECDSA) to the list of known hosts. Last login: Wed Nov 1 16:26:26 2017 from 10.0.9.38 [[email protected]]$
Woo hoo! Sounds easy, right? Here are common problems that I’ve run into in the past…
Problem #1: Unknown host
If you ping the machine you want to log into and you’re seeing an “Unknown host” error then the hostnames are not resolvable by the client.
[[email protected]]$ ping wolverine.vm -c 2 ping: cannot resolve wolverine.vm: Unknown host
I can make wolverine.vm resolvable by appending an entry to my
[[email protected]]$ sudo -- sh -c "echo '10.32.174.160 wolverine.vm wolverine' >> /etc/hosts"
Let’s try again.
[[email protected]]$ ping wolverine.vm -c 2 PING wolverine.vm (10.32.174.159): 56 data bytes 64 bytes from 10.32.174.159: icmp_seq=0 ttl=60 time=1.350 ms 64 bytes from 10.32.174.159: icmp_seq=1 ttl=60 time=2.356 ms --- wolverine.vm ping statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 1.350/1.853/2.356/0.503 ms
Problem #2: Connection refused
If you try to ssh into the machine using a username and password but you’re getting “Connection refused” then:
- the SSH daemon (
sshd) is not running on the machine and/or
- the port 22 (the default SSH port) isn’t open.
You can use
ssh -v to see if the port is open. If the port wasn't open, the
results would look something like this:
[[email protected]]$ ssh -v deadpool.vm OpenSSH_7.4p1, LibreSSL 2.5.0 debug1: Reading configuration data /Users/abir/.ssh/config debug1: /Users/abir/.ssh/config line 1: Applying options for * debug1: Reading configuration data /etc/ssh/ssh_config debug1: Connecting to deadpool.vm [10.32.173.180] port 22. debug1: connect to address 10.32.173.180 port 22: Connection refused ssh: connect to host deadpool.vm port 22: Connection refused
To resolve this issue you’ll need to install and configure
openssh on the
machine, and make sure the machine’s firewall has port 22 open. There are
different ways to set firewall rules and install
sshd on different operating
systems so I’m not going into them here.
Problem #3: Permission Denied
If you try to ssh into the machine you could also get “Permission Denied” error.
[[email protected]]$ ssh [email protected] Last login: Wed Nov 1 16:26:26 2017 from 10.0.9.38 [[email protected]]$ [[email protected]]$ ssh [email protected] Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
This most likely means that you don’t have your SSH creds set up correctly on the remote machine. To resolve this you’ll need to run through “The Basics” from above.
General Troubleshooting Tips
- Pass the
-vflag to the
sshcommand to get more information about what might be happening.
- Try re-creating the public/private key. I had problems when I tried passing
ssh-keygen. When I tried executing simply
ssh-keygenwith no additional parameters and passed in parameters via stdin it worked fine.
- Sometimes the
.ssh/known_hostsfile on the client machine can cause issues. Try removing the remote machine entries from the file, and then try
ssh’ing into those machines again.
- Make sure you generate the public key on the client (professorx.local in my
case) and put it on the remote servers (wolverine.vm and deadpool.vm). A
common mistake is to do the reverse: generate the key on the remote servers and
place them on the client.
Nerdy analogy: Professor X needs permission from Wolverine and Deadpool to read their minds. Think of the
authorized_keysfile as a list of people Deadpool and Wolverine let control their minds.
- Make sure you’ve got the right permissions set on the files in
[[email protected]]$ chmod 700 ~/.ssh && chmod 600 ~/.ssh/* [[email protected]]$ chmod 600 ~/.ssh/authorized_keys && chmod 700 ~/.ssh/
Once you’ve got public key authentication for SSH set up you can use a tool like Puppet Bolt to easily execute commands on those remote machines. For example, if I want to see the free disk space on wolverine.vm and deadpool.vm I could run:
[[email protected]]$ bolt command run 'df -h' --nodes wolverine.vm,deadpool.vm -u abir wolverine.vm: Filesystem Size Used Avail Use% Mounted on /dev/vda1 16G 2.6G 14G 17% / devtmpfs 902M 0 902M 0% /dev tmpfs 920M 0 920M 0% /dev/shm tmpfs 920M 81M 840M 9% /run tmpfs 920M 0 920M 0% /sys/fs/cgroup tmpfs 184M 0 184M 0% /run/user/1000 deadpool.vm: Filesystem Size Used Avail Use% Mounted on /dev/vda1 16G 12G 4.1G 75% / devtmpfs 901M 0 901M 0% /dev tmpfs 920M 0 920M 0% /dev/shm tmpfs 920M 97M 824M 11% /run tmpfs 920M 0 920M 0% /sys/fs/cgroup tmpfs 184M 0 184M 0% /run/user/1000 Ran on 2 nodes in 0.40 seconds
† All Marvel characters and the distinctive likeness(es) thereof are Trademarks & Copyright © 1941–2017 Marvel Characters, Inc. All rights reserved. Please don’t sue me.
Abir Majumdar is a sales engineer at Puppet.