Smallstep SSH Host Quickstart

Before you begin

  • You'll need a account on the smallstep platform. Need one? Register here
  • You'll need the enrollment token you received upon signup.
  • We support the following host platforms:
    • CentOS 8
    • CentOS 7
    • Amazon Linux 2
    • Ubuntu 20.04 LTS (Focal)
    • Ubuntu 18.04 LTS (Bionic)
    • Ubuntu 16.04 LTS (Xenial)
    • Debian 10 (Buster)
    • Debian 9 (Stretch)
    • Debian 8 (Jessie)
  • Running this quickstart will modify config files related to systemd, PAM, NSS, and SSHD:
    • We add step-ssh-renew and step-ssh-metadata services to systemd:
      • step-ssh-renew rotates the SSH host certificate every eight hours.
      • step-ssh-metadata syncs user ACLs with your CA every 3 seconds.
    • And we modify the following files:
      • /etc/pam.d/sshd
      • /etc/pam.d/sudo
      • /etc/pam.d/su
      • /etc/ssh/sshd_config
      • /etc/nsswitch.conf
    • We suggest backing these up before you begin setup on a particular base machine type, so that you can quickly revert changes if needed.

Step-by-Step Instructions

Step 0. Understanding Host Tags

This step only applies to multi-user environments.

Host Tags (key-value pairs) are the pillar our access control model. Rather than mapping people or groups directly to hosts, you'll map tag combinations to your hosts and to your user groups. First you'll put your hosts into logical groups using tags, eg. role:web or env:staging. Then, you'll grant user groups access to all hosts with a specific tag combination. Finally, you'll choose which user group tag combinations will allow sudo privileges on any matching hosts.

Let's look at an example:

  • The developers group will have access to myserver #1 only.
  • The data group will have access to myserver #2 and myserver #3.
  • The ops group will have sudo access to myserver #2 and myserver #3.

Of course, hosts and groups can have as many tag combinations as you like. Take a minute to think about how you'd like to use Host Tags in your environment.

Step 1. Run all steps as root

Now let's set up your host. You'll run the entire Host Configuration setup as the root user:

sudo su

Step 2. Install step CLI tool

$ curl -L -o step $ install -m 0755 -t /usr/bin step

Step 3. Install the step-ssh utilities

This step will install modules and services.

  • Install on Ubuntu & Debian (DEB package)

    $ curl -LO $ dpkg -i step-ssh_0.19.0-1_amd64.deb

    Comment out the broken in common-session, which causes a delay at login (this is a known Ubuntu/Debian bug):

    sed -e '/^/#/g' -i /etc/pam.d/common-session
  • Install on CentOS or Amazon Linux 2 (RPM package)

    $ curl -LO "" $ yum -y install step-ssh-0.19.0-1.x86_64.rpm

Step 4. Configure step to connect to your CA

step ca bootstrap --team="[your smallstep team ID]"

Step 5. Get an SSH host certificate

Remember the enrollment token you got when you signed up? You'll need it now. If you downloaded it, the file is called enrollment_token.

_👇 This leading space will (usually) keep the token out of your shell's history.

$ export enrollment_token="[your enrollment token]" $ export hostname="[your hostname]"

The hostname is your host's canonical hostname or IP. This will be the name clients use to SSH to this host.

For hosts with a single hostname

Run the following to issue a certificate for your host:

step ssh certificate $hostname /etc/ssh/ \ --host --sign --provisioner "Service Account" --token $enrollment_token
If a host has multiple hostnames...

Note: When a host has multiple hostnames, your users will only be able to ssh to the canonical $hostname, as shown by the step ssh hosts command.

If you need multiple hostnames in your host certificate (e.g., public and private hostnames, or a hostname and an IP address), you can pass each of them to step ssh certificate via the --principal flag:

step ssh certificate $hostname /etc/ssh/ \ --host --sign --provisioner "Service Account" --token $enrollment_token \ --principal $hostname --principal

When multiple hostnames are needed, the canonical $hostname must be passed twice: Once to establish the certificate's Key ID, and again explicitly as its first Principal.

Step 6. Configure SSHD to use certificate authentication

step ssh config --host --set \ --set Key=ssh_host_ecdsa_key

This command will add a few lines of configuration to the end of your /etc/ssh/sshd_config to enable certificate authentication. These lines are annotated with a comment that says # autogenerated by step @ <timestamp> so you can identify them later if you need to modify or revert these changes.

Step 7. Activate PAM/NSS Modules & HUP SSHD

step-ssh activate "$hostname"

The step-ssh activate command will leverage a short-lived identity certificate to authenticate itself to the host inventory.

Step 8. Register the host and add tags(s)

This command will leverage the host identity certificate to authenticate itself to the host inventory.

step-ssh-ctl register --hostname "$hostname"
Registering a host with host tags

For access control in multi-user environments, host tags can be assigned via the --tag flag.

step-ssh-ctl register --tag <key=value> --tag <role=web> --hostname "$hostname"

It is possible to rerun step-ssh-ctl register multiple times, to rename the host, replace its tags, or change the bastion settings. Note: This command replaces all existing tags and bastion settings for a host.

Registering a bastion host (jump boxes)

If the host you're registering is a bastion, add the --is-bastion flag:

step-ssh-ctl register --hostname "$hostname" --is-bastion

Note: Your bastion host will need the nc command installed. Our bastion host support uses nc (along with the ProxyCommand directive) because it's widely compatible with older SSHD servers.

Registering a host behind a bastion

If the host you're registering is behind a bastion, add the --bastion flag:

step-ssh-ctl register --hostname "$hostname" --bastion "[bastion hostname]"

Step 9. Test your installation

Before you sign out of your sudo session, test your installation by logging in and running sudo in a separate session.

This step is especially important if you have made any non-standard changes to your PAM or NSS stacks.

Now sign in at[Team ID]

You should see your host listed under the "Hosts" tab.

Need to remove a host?

On the host, run:

step-ssh-ctl unregister

This will remove the host from the host inventory on the CA, using the host identity certificate.

Optional Hardening

  • Now that you're using certificates, you may wish to explicitly disallow the use of authorized_keys files on the host.

    You can do this by setting AuthorizedKeysFile none in sshd_config. You may want to allow authorized_keys for an emergency access account, however. A configuration for this might look like:

Match User *,!ubuntu
    AuthorizedKeysFile none

Troubleshooting Tips

  • Having trouble? You should be able to revert any changes by running step-ssh deactivate.

  • Suspect host or user certificates are not working? In other words, your ssh client fails to log in or shows "trust on first use" warning? Try this:

    • Be sure you can connect to our test server, with
    • On your client run step ssh list and find your user certificate marked (ECDSA-CERT), e.g. 256 SHA256:Bb2TcimUYj8Nc5w4FhpZ/gmeNIIvLIzphTx35NzaRoA (ECDSA-CERT), which you can inspect with step ssh list --raw | step ssh inspect. Be sure the current time is neither before or after the period specified in Valid: .
    • On the destination host run step ssh inspect /etc/ssh/ Make sure the current time is within the period of Valid:.
    • The destination host's sshd_config should show included the following lines for sshd (make sure the service has reloaded its config) to leverage certificate-based authentication:
    $ tail -n 7 /etc/ssh/sshd_config
    #       ForceCommand cvs server
    # autogenerated by step @ 2020-04-02T21:16:05Z
    TrustedUserCAKeys /etc/ssh/
    HostCertificate /etc/ssh/
    HostKey /etc/ssh/ssh_host_ecdsa_key
    # end
    • You can run sshd -t (as root) to test you SSHD configuration. No output means the file is OK; you'll get an error if any of the referenced files (eg. host keys) are not accessible to sshd.
  • Suspect your ssh client might have a problem? Use verbose logging ssh -v <hostname>. Look out for following key lines/events below in your log. If you don't see errors the absence of these events might indicate your config is invalid / not being applied.

    • Your team's config is applied for this specific host:

      debug1: Executing command: 'step ssh check-host'
      debug1: Reading configuration data /etc/ssh/ssh_config
      debug1: /etc/ssh/ssh_config line 48: Applying options for *
      debug1: /etc/ssh/ssh_config line 52: Applying options for *
      debug1: Executing proxy command: exec step ssh proxycommand --provisioner "okta" alice 22
    • The host certificate passed authentication on the client:

      debug1: Server host certificate: SHA256:ON/csaCQ4yDKfxblQnOLTiZIOoOtsHPAjQn6DrW+k9I, serial 2468062085372141930 ID "ec2-52-200-74-193.compute-1.ama
      debug1: Server host certificate: SHA256:46gC0CEzXWN4acTHGQldL6H+QlbhB4+KPZjkoRToI/w, serial 8551898981883739717 ID "" CA ecdsa-sha2-nistp256 SHA256:sqfZG6AOPUvcheFUIZDX+DEesnyfNZQ5JwqpcxUzY+0 valid from 2020-04-14T04:45:10 to 2020-05-14T04:46:10
      debug1: Host '' is known and matches the ECDSA-CERT host certificate.
      debug1: Found CA key in /Users/alice/.step/ssh/known_hosts:1
    • The username being used for authentication:

      debug1: Authenticating to as 'alice'
    • The ssh client offers the user's certificate to the destination host:

      debug1: Offering public key: ECDSA-CERT SHA256:VIa1uWhBTjjtpW3IBkUG/aFGfqlUhjkXNQVk6Hc1lXc agent
      debug1: Server accepts key: ECDSA-CERT SHA256:VIa1uWhBTjjtpW3IBkUG/aFGfqlUhjkXNQVk6Hc1lXc agent
      debug1: sign_and_send_pubkey: no separate private key for certificate ""

Unsubscribe anytime. See our privacy policy.