IP Monitoring & Diagnostics With Command Line Tools: Part 4 - SSH Public Keys
Installing public SSH keys created on your workstation in a server will authenticate you without needing a password. This streamlines the SSH interaction and avoids the need to use stored and visible passwords in your scripts.
More articles in this series:
By creating and installing SSH public keys in your remote systems, remote commands can then be called to action as easily as running them on the local machine.
Older techniques that hid passwords in secret files or magically typed them on a virtual keyboard are no longer necessary. Everything is now much more secure.
Why this is a good thing
Initiating remote activity on a server or calling a remote agent to action with SSH is convenient because you can use the resulting output on a client machine dedicated to monitoring. Automated monitoring is impractical if a password must be entered manually to retrieve a result from the agent. Avoid this by creating an SSH key and copying it to the target server. The key authenticates the client machine when it requests a connection to login or execute a command.
Creating an SSH shared key
Use the ssh-keygen utility to manufacture the secure key package on your client machine. It will ask you where to store the keys it generates and what to use as a pass-phrase. Press the [RETURN] key in both cases to use the default. The files are stored in your home directory and the pass-phase will be empty. This chooses RSA encryption by default.
ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/Home/user/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Home/user/.ssh/id_rsa.
Your public key has been saved in /Home/user/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:tGXh2ZwadGcV84BkmQ1PIVseES3H5n8aYQ9FzDpTBfo [email protected]
The key's randomart image is:
+---[RSA 3072]----+
| o.**%#*|
| o B+%oo%|
| . * * oO.|
| . + o .B .|
| S . .E*.|
| . +|
| o.|
| . |
| |
+----[SHA256]-----+
Optional command line options will configure the keys in other ways. The default should work just fine for now.
You now have a public key file that authenticates your account on the client computer.
Deploy the key
The public key file can be deployed to the remote login account with the ssh-copy-id utility. If the remote system is correctly set up, this is all you need to do.
Use this command:
ssh-copy-id {user_account}@{hostname_or_IP_address}
You should see something like this:
ssh-copy-id [email protected]
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed:
"/Home/user/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the
new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed
-- if you are prompted now it is to install the new keys
user @192.168.1.253's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.
The public key has been installed in the list of authorised keys on the server and remote commands should now work correctly.
When you access the remote server with SSH, your computer will pass the key details to it. The target login account will check the list of authorised keys for a match. If your key is present in the list, the remote command is accepted and called to action. It is a very simple and elegant solution.
What might go wrong
If this did not work, it must be fixed on the remote server. Any of these broken configuration items could prevent the key installation from working. Correcting them requires elevated administrator privileges:
• The .ssh directory may not exist in the target home folder. This needs to be present in the home directory for the target login account. Use the mkdir command to create it.
• The .ssh directory may have incorrect permissions. They should be set to 700 with the chmod command.
• The authorized_keys file may not exist. Create an empty file if necessary.
• The authorized_keys file may have incorrect permissions. Set the permissions to 644 with the chmod command.
• The SSH configuration may not have the correct options enabled to allow public key authentication. You will need to edit it to turn SSH key based authorisation on.
• The SSH config file may not have the authorized_keys file name description enabled. Check that this is also active and correctly describing the location and name of the file.
• Try the ssh-copy-id command from the client again after fixing the rest of these issues.
• Alternatively, manually upload your key file to the .ssh directory and append it to the authorized_keys file by hand. Be careful to append and not overwrite so that previously installed keys are undamaged.
• Check that your key is only appended once.
• Once these configurations are all correct, the SSH service needs to be restarted to load the configuration changes.
Now it all works!
After correcting these issues, this remote disk space command (df) works as expected without needing a password:
ssh [email protected] df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 2451064 1020972 1327692 44% /
none 512652 0 512652 0% /dev
/tmp 516844 816 516028 1% /tmp
/run 516844 4664 512180 1% /run
/dev/shm 516844 4 516840 1% /dev/shm
/dev/vg1/volume_1 7681053612 5856395392 1824555820 77% /volume1
List the contents of a remote directory
Use the ls -la command on the remote machine to inspect the remote .ssh directory in the target account:
ssh [email protected] ls -la ./.ssh
drwx------ 2 admin users 4096 Feb 2 10:11 .
drwxr-xr-x 4 admin users 4096 Feb 3 10:40 ..
-rw-r--r-- 1 admin users 571 Feb 2 10:59 authorized_keys
-rwxrwxrwx+ 1 admin users 571 Feb 2 09:54 id_rsa.pub
This is where the SSH authorized_keys file lives.
Copying files securely
Use the scp command to remotely copy files the machines. This example puts an image file on a remote machine. The first parameter is the source file and the second is the target destination where it is copied to.
scp diagram.png {user_account}@{hostname}:./remote_diagram.png
This is the complementary command to pull that image file back again:
scp {user_account}@{hostname}:./remote_diagram.png recovered_diagram.png
Now remove the temporary remote file to clean up after testing:
ssh {user_account}@{hostname} rm remote_diagram.png
Use the same technique to acquire a web server error log. The remote account may need additional permissions to read this file:
scp {user_account}@{hostname}:/var/log/httpd/apache-error_log error.log
Check the contents of the error log to see if there are any security intrusion attempts on the remote server or errors to fix in your web site code.
Acquire and filter remote data
In this example, the cat, grep and tail commands are used to acquire the status of a Synology NAS. The whole dmesg file is copied to the client machine and filtered locally:
ssh {user_account}@{hostname} "cat /var/log/dmesg" |
grep ' operational ' |
tail -4
[Jan 1 00:05:31] md/raid:md2: device sda5 operational as raid disk 0
[Jan 1 00:05:31] md/raid:md2: device sdd5 operational as raid disk 3
[Jan 1 00:05:31] md/raid:md2: device sdc5 operational as raid disk 2
[Jan 1 00:05:31] md/raid:md2: device sdb5 operational as raid disk 1
Move the second double quote to the end of the line. Now the dmesg file is filtered remotely and only the result lines are delivered to the client machine.
ssh {user_account}@{hostname} "cat /var/log/dmesg |
grep ' operational ' |
tail -4"
The grep and tail commands now happen in the remote machine and the ssh command transfers the filtered result. Much less data is being hauled across the network. If network capacity is at a premium you might prefer the second method.
Check a remote network config
The ifconfig command lists the available network devices. Examine the network settings in a remote machine with this command:
ssh [email protected] ifconfig
What actually happens during the transaction?
A conversation happens between the client and server when the ssh command is being used. Add the -v flag to the ssh command to observe the exchange. Change this to the -vv or -vvv flags to see progressively more verbose output.
Conclusion
Even the smallest of configuration errors can stop something working. Following the remedial process carefully step by step will solve the problem. Take care and do not panic if the first thing you try does not work. Concentrate on one problem at a time. I needed to fix all of the issues described above in my test system. Any single one of them was a potential show stopper.
Placing quote symbols in different locations in the command line without changing anything else will control whether to call commands to action in the local machine or the remote server. Sometimes the effects of introducing a single character into the command line can have profound and subtle effects.
When you discover something that is broken, decide whether it is a candidate for creating a diagnostic test that could be run automatically to report a systemic problem that might occur again.
You might also like...
HDR & WCG For Broadcast: Part 3 - Achieving Simultaneous HDR-SDR Workflows
Welcome to Part 3 of ‘HDR & WCG For Broadcast’ - a major 10 article exploration of the science and practical applications of all aspects of High Dynamic Range and Wide Color Gamut for broadcast production. Part 3 discusses the creative challenges of HDR…
IP Security For Broadcasters: Part 4 - MACsec Explained
IPsec and VPN provide much improved security over untrusted networks such as the internet. However, security may need to improve within a local area network, and to achieve this we have MACsec in our arsenal of security solutions.
Standards: Part 23 - Media Types Vs MIME Types
Media Types describe the container and content format when delivering media over a network. Historically they were described as MIME Types.
Building Software Defined Infrastructure: Part 1 - System Topologies
Welcome to Part 1 of Building Software Defined Infrastructure - a new multi-part content collection from Tony Orme. This series is for broadcast engineering & IT teams seeking to deepen their technical understanding of the microservices based IT technologies that are…
IP Security For Broadcasters: Part 3 - IPsec Explained
One of the great advantages of the internet is that it relies on open standards that promote routing of IP packets between multiple networks. But this provides many challenges when considering security. The good news is that we have solutions…