IP Monitoring & Diagnostics With Command Line Tools: Part 4 - SSH Public Keys

Installing public SSH keys created on your workstation in a server will authenticate you without needing a password. This streamlines the SSH interaction and avoids the need to use stored and visible passwords in your scripts.


More articles in this series:


By creating and installing SSH public keys in your remote systems, remote commands can then be called to action as easily as running them on the local machine.

Older techniques that hid passwords in secret files or magically typed them on a virtual keyboard are no longer necessary. Everything is now much more secure.

Why this is a good thing

Initiating remote activity on a server or calling a remote agent to action with SSH is convenient because you can use the resulting output on a client machine dedicated to monitoring. Automated monitoring is impractical if a password must be entered manually to retrieve a result from the agent. Avoid this by creating an SSH key and copying it to the target server. The key authenticates the client machine when it requests a connection to login or execute a command.

Creating an SSH shared key

Use the ssh-keygen utility to manufacture the secure key package on your client machine. It will ask you where to store the keys it generates and what to use as a pass-phrase. Press the [RETURN] key in both cases to use the default. The files are stored in your home directory and the pass-phase will be empty. This chooses RSA encryption by default.

ssh-keygen

Generating public/private rsa key pair.

Enter file in which to save the key (/Home/user/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /Home/user/.ssh/id_rsa.

Your public key has been saved in /Home/user/.ssh/id_rsa.pub.

The key fingerprint is:

SHA256:tGXh2ZwadGcV84BkmQ1PIVseES3H5n8aYQ9FzDpTBfo [email protected]

The key's randomart image is:
+---[RSA 3072]----+
|          o.**%#*|
|         o B+%oo%|
|        . * * oO.|
|       . + o .B .|
|        S .  .E*.|
|              . +|
|               o.|
|              .  |
|                 |
+----[SHA256]-----+


Optional command line options will configure the keys in other ways. The default should work just fine for now.

You now have a public key file that authenticates your account on the client computer.

Deploy the key

The public key file can be deployed to the remote login account with the ssh-copy-id utility. If the remote system is correctly set up, this is all you need to do.

Use this command:

ssh-copy-id {user_account}@{hostname_or_IP_address}

You should see something like this:

ssh-copy-id [email protected]

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed:
"/Home/user/.ssh/id_rsa.pub"

/usr/bin/ssh-copy-id: INFO: attempting to log in with the
new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed
-- if you are prompted now it is to install the new keys

user @192.168.1.253's password:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

The public key has been installed in the list of authorised keys on the server and remote commands should now work correctly.

When you access the remote server with SSH, your computer will pass the key details to it. The target login account will check the list of authorised keys for a match. If your key is present in the list, the remote command is accepted and called to action. It is a very simple and elegant solution.

What might go wrong

If this did not work, it must be fixed on the remote server. Any of these broken configuration items could prevent the key installation from working. Correcting them requires elevated administrator privileges:

• The .ssh directory may not exist in the target home folder. This needs to be present in the home directory for the target login account. Use the mkdir command to create it.
• The .ssh directory may have incorrect permissions. They should be set to 700 with the chmod command.
• The authorized_keys file may not exist. Create an empty file if necessary.
• The authorized_keys file may have incorrect permissions. Set the permissions to 644 with the chmod command.
• The SSH configuration may not have the correct options enabled to allow public key authentication. You will need to edit it to turn SSH key based authorisation on.
• The SSH config file may not have the authorized_keys file name description enabled. Check that this is also active and correctly describing the location and name of the file.
• Try the ssh-copy-id command from the client again after fixing the rest of these issues.
• Alternatively, manually upload your key file to the .ssh directory and append it to the authorized_keys file by hand. Be careful to append and not overwrite so that previously installed keys are undamaged.
• Check that your key is only appended once.
• Once these configurations are all correct, the SSH service needs to be restarted to load the configuration changes.

Now it all works!

After correcting these issues, this remote disk space command (df) works as expected without needing a password:

ssh [email protected] df

Filesystem         1K-blocks       Used  Available Use% Mounted on
/dev/root            2451064    1020972    1327692  44% /
none                  512652          0     512652   0% /dev
/tmp                  516844        816     516028   1% /tmp
/run                  516844       4664     512180   1% /run
/dev/shm              516844          4     516840   1% /dev/shm
/dev/vg1/volume_1 7681053612 5856395392 1824555820  77% /volume1

List the contents of a remote directory

Use the ls -la command on the remote machine to inspect the remote .ssh directory in the target account:

ssh [email protected] ls -la ./.ssh


drwx------  2 admin users 4096 Feb  2 10:11 .
drwxr-xr-x  4 admin users 4096 Feb  3 10:40 ..
-rw-r--r--  1 admin users  571 Feb  2 10:59 authorized_keys
-rwxrwxrwx+ 1 admin users  571 Feb  2 09:54 id_rsa.pub

This is where the SSH authorized_keys file lives.

Copying files securely

Use the scp command to remotely copy files the machines. This example puts an image file on a remote machine. The first parameter is the source file and the second is the target destination where it is copied to.

scp diagram.png {user_account}@{hostname}:./remote_diagram.png
This is the complementary command to pull that image file back again:

scp {user_account}@{hostname}:./remote_diagram.png recovered_diagram.png
Now remove the temporary remote file to clean up after testing:

ssh {user_account}@{hostname} rm remote_diagram.png

Use the same technique to acquire a web server error log. The remote account may need additional permissions to read this file:

scp {user_account}@{hostname}:/var/log/httpd/apache-error_log error.log

Check the contents of the error log to see if there are any security intrusion attempts on the remote server or errors to fix in your web site code.

Acquire and filter remote data

In this example, the cat, grep and tail commands are used to acquire the status of a Synology NAS. The whole dmesg file is copied to the client machine and filtered locally:

ssh {user_account}@{hostname} "cat /var/log/dmesg" |
grep ' operational ' |
tail -4

[Jan 1 00:05:31] md/raid:md2: device sda5 operational as raid disk 0
[Jan 1 00:05:31] md/raid:md2: device sdd5 operational as raid disk 3
[Jan 1 00:05:31] md/raid:md2: device sdc5 operational as raid disk 2
[Jan 1 00:05:31] md/raid:md2: device sdb5 operational as raid disk 1

Move the second double quote to the end of the line. Now the dmesg file is filtered remotely and only the result lines are delivered to the client machine.

ssh {user_account}@{hostname} "cat /var/log/dmesg |
grep ' operational ' |
tail -4"

The grep and tail commands now happen in the remote machine and the ssh command transfers the filtered result. Much less data is being hauled across the network. If network capacity is at a premium you might prefer the second method.

Check a remote network config

The ifconfig command lists the available network devices. Examine the network settings in a remote machine with this command:

ssh [email protected] ifconfig

What actually happens during the transaction?

A conversation happens between the client and server when the ssh command is being used. Add the -v flag to the ssh command to observe the exchange. Change this to the -vv or -vvv flags to see progressively more verbose output.

Conclusion

Even the smallest of configuration errors can stop something working. Following the remedial process carefully step by step will solve the problem. Take care and do not panic if the first thing you try does not work. Concentrate on one problem at a time. I needed to fix all of the issues described above in my test system. Any single one of them was a potential show stopper.

Placing quote symbols in different locations in the command line without changing anything else will control whether to call commands to action in the local machine or the remote server. Sometimes the effects of introducing a single character into the command line can have profound and subtle effects.

When you discover something that is broken, decide whether it is a candidate for creating a diagnostic test that could be run automatically to report a systemic problem that might occur again.

You might also like...

Designing IP Broadcast Systems

Designing IP Broadcast Systems is another massive body of research driven work - with over 27,000 words in 18 articles, in a free 84 page eBook. It provides extensive insight into the technology and engineering methodology required to create practical IP based broadcast…

If It Ain’t Broke Still Fix It: Part 2 - Security

The old broadcasting adage: ‘if it ain’t broke don’t fix it’ is no longer relevant and potentially highly dangerous, especially when we consider the security implications of not updating software and operating systems.

Standards: Part 21 - The MPEG, AES & Other Containers

Here we discuss how raw essence data needs to be serialized so it can be stored in media container files. We also describe the various media container file formats and their evolution.

NDI For Broadcast: Part 3 – Bridging The Gap

This third and for now, final part of our mini-series exploring NDI and its place in broadcast infrastructure moves on to a trio of tools released with NDI 5.0 which are all aimed at facilitating remote and collaborative workflows; NDI Audio,…

Microphones: Part 2 - Design Principles

Successful microphones have been built working on a number of different principles. Those ideas will be looked at here.