Smallstep Certificate Manager How It Works

Certificates power modern software. Kubernetes, service meshes, containers, microservices, distributed databases & queues, configuration management & orchestration systems, BeyondCorp proxies, and secure software supply chains all use certificates. Organizations running modern infrastructure make extensive use of certificates, whether they're aware of it or not.

For many organizations, this is a blind spot, with long-lived certificates issued manually or from poorly secured deploy pipelines. Secure, reliable certificate management requires automation. Smallstep Certificate Manager is a flexible, extensible toolchain with all of the server and client-side components you'll need to solve this problem, carefully designed to be operationally simple, easy to use, and hard to misuse.

Certificate Manager builds on two open source projects, maintained by smallstep:

  • step-ca: a private online certificate authority for secure automated certificate management.
  • step: a general-purpose cryptography toolkit and the client-side counterpart to step-ca.

Certificate Lifecycle Management

Here's how the elements described in Core Concepts work are used to enable the different stages of a certificate lifecycle management workflow:

Bootstrapping Trust

Before issuing a certificate, your software and systems must be configured to trust your authorities. More precisely, you need to configure everything to trust your root certificate (i.e., the certificate that belongs to your root CA). You can include your root certificate in base images, distribute it using configuration management, or use step ca bootstrap or step ca root to securely download your root certificate from an issuing authority. You can also download the root certificate for your authorities from the Certificate Manager dashboard.

$ step ca root root.crt
The root certificate has been saved in root.crt.

$ step certificate install root.crt
Password:
Certificate r.crt has been installed.
X.509v3 Root CA Certificate (ECDSA P-256) [Serial: 1829...9147]
  Subject:     Root CA
  Issuer:      Root CA
  Valid from:  2020-09-18T20:55:03Z
          to:  2030-09-16T20:55:03Z

Or configure software to trust it (see Hello mTLS for more examples):

$ cat nginx.conf
server {
    listen                 443 ssl;
    server_name            myserver.internal.net;
    ssl_client_certificate /etc/nginx/client_certs/root.crt;
    ssl_verify_client      optional;
    location / {
      if ($ssl_client_verify != SUCCESS) {
        return 403;
      }
    }
}

Issuing Certificates

Issuing a certificate using step is simple. In it's most basic form, the step ca certificate command provides simple interactive workflows for people to get a certificate from an issuing authority. Flags and options make scripting and other non-interactive workflows easy. We'll use several variations of this command in the examples below.

$ step ca certificate example.com example.crt example.key
Use the arrow keys to navigate: ↓ ↑ → ←
What provisioner key do you want to use?
  ▸ mike@example.com (JWK) [kid: -6v0dwnSlz2D5opq-B5prQmyD9DhzdJSHMidiu1TjYg]
    sshpop (SSHPOP)
    x5c (X5C)
    Google (OIDC) [client: 650445034027-jsjdrkiskeq9ke99ud2rqkst82ft8uch.apps.googleusercontent.com]
✔ Provisioner: mike@smallstep.com (JWK) [kid: -6v0dwnSlz2D5opq-B5prQmyD9DhzdJSHMidiu1TjYg]
✔ Please enter the password to decrypt the provisioner key:
✔ CA: https://ca.example.com
✔ Certificate: example.crt
✔ Private Key: example.key

Renewal

Certificates expire. Certificate Manager makes renewing a certificate ahead of expiration easy. Renewals are authenticated using your existing certificate, and produce an identical certificate with a new serial number and extended lifetime.

In its most primitive form, renewal is a simple single-command operation:

step ca renew svc.crt svc.key
step ca renew --daemon --exec "nginx -s reload" svc.crt svc.key

The --daemon flag starts a long-running process that will continuously renew svc.crt after 2/3 of its lifetime has elapsed. The --exec "nginx -s reload" option tells Nginx to reload svc.crt after each renewal. We even take care of retries if renewal fail (with exponential backoff and jitter to mitigate denial of service issues and thundering herds).

step ca renew is designed to integrate nicely with cron or systemd, which is what we recommend for production deploys. For more details see our documentation on production-ready automated renewal.

It's also common to trigger renewal from configuration management or from a container sidecar. Regardless, step makes renewal simple and secure.

Disabling Renewal

Finally, sometimes you don't want certificates to be renewed. For example, if you're issuing certificates using single sign-on (the OIDC provisioner), you probably want your users to re-authenticate with your identity provider periodically to get a new certificate instead of renewing. Renewal can be enabled and disabled per-provisioner to accommodate this (it's disabled by default for the OIDC provisioner).

Renewal After Expiry

  • This is an Advanced Authority feature

Some use cases demand an option to renew expired certificates. For example, intermittently-connected devices may not be able to reach the CA in time for a renewal. The ability to renew expired certificates can be enabled on a per-provisioner basis. Enable it in the Smallstep dashboard when you create a new provisioner, or on the command line for an existing provisioner. While sometimes necessary, this feature comes with risks and is disabled by default.

Revocation

Certificates should be revoked if they're compromised or no longer needed. Smallstep Certificate Manager supports both passive and active revocation.

  • Passive revocation disables renewal for a particular certificate. The certificate will be trusted until it expires.
  • Active revocation explicitly invalidates a certificate ahead of expiry. CRL and OCSP are used to distribute revocation status to relying parties. Active revocation is often necessary for incident response. However, caching, availability, and configuration challenges can make active revocation unreliable. For systems that support it, we've found that revoking authorization can be easier to implement and more reliable than active certificate revocation. We've covered this topic in more detail on our blog.

Using Passive Revocation

Passive revocation is on by default and works well in conjunction with short-lived certificates. To revoke a certificate, run step ca revoke and pass the certificate's serial number.

$ step ca revoke 31952909121245729293840904958100006722

Once revoked, the certificate is no longer renewable.

Using Active Revocation

Active revocation (CRL and OCSP) is available if you're using an Advanced Authority. It can be enabled when you create the authority. To revoke a certificate, you will need either the certificate and its private key (for mutual TLS), or a JWK provisioner (for JWT authentication). You cannot revoke a certificate using the default administrative OIDC provisioner.

To revoke a certificate using mutual TLS, run:

$ step ca revoke --cert example.crt --key example.key

With a JWK provisioner, to revoke a certificate using JWT authentication, run:

$ SERIAL=31952909121245729293840904958100006722
$ TOKEN=$(step ca token --provisioner jwk-provisioner --revoke $SERIAL)
$ step ca revoke --token $TOKEN $SERIAL

Observability & Alerting

Certificate Manager is designed to support large, rapidly evolving infrastructures. In these environments, it's easy for a small, unnoticed failure to cascade into a large outage.

Certificate Manager can alert you ahead of certificate expiry via email or by calling a webhook. Alerts can be fed into your SIEM or sent to slack so you can remediate ahead of an outage. Issued certificates are catalogued for easy auditability. The Certificate Manager interface allows you to sort and filter, and quickly list and export certificate subjects, issue & expiry dates, issuing authority, provisioner, and other certificate details.

Smallstep API

The Smallstep API can be used to manage your authorities programmatically. The API is currently an alpha release with limited availability. Request access to this API for your account by creating a support ticket for our Customer Engineering team. Be sure to include your team name in your request.

Certificate Manager in Action

Let's look at how these features come together in a few common workflows.

To configure step to use our authority, run:

step ca bootstrap \
  --ca-url https://prod.yourco.ca.smallstep.com
  --fingerprint 6d04ff00618125114b97870f835c4a28fce2fec91e4b86946147f5c6a5555ff8

Simple Password-based Issuance

To get a certificate, run step ca certificate. You'll be prompted to select a provisioner. In this example, we'll use the JWK provisioner in a password-based flow:

 $ step ca certificate admin@yourco.com --san=any-name any.crt any.key
✔ Provisioner: pass (JWK) [kid: x6v0dwnSlz2D5opq-B5prQmyD9DhzdJSHMidiu1TjYg]
✔ Please enter the password to decrypt the provisioner key:
✔ CA: https://prod.yourco.ca.smallstep.com
✔ Certificate: any.crt
✔ Private Key: any.key

Once authenticated, the authority will obtain inventory metadata and apply any templates associated with the selected provisioner. That's it. The issued certificate is ready to use.

$ step certificate inspect any.crt -short
X.509v3 TLS Certificate (ECDSA P-256) [Serial: 3329...8201]
  Subject:     any-name
               admin@yourco.com
  Issuer:      Prod Intermediate CA
  Provisioner: pass [ID: TvGU...0zXw]
  Valid from:  2021-07-07T22:59:05Z
          to:  2021-07-08T23:00:05Z

Renewing this certificate is equally straightforward:

$ step ca renew any.crt any.key --force
Your certificate has been saved in any.crt.

If you inspect the newly issued certificate, you'll find it has an extended lifetime.

One-time Tokens

The JWK provisioner is very flexible. Under the hood, it's based on short-lived one-time tokens that are signed by a private key. The issuing authority escrows an encrypted copy of the private key. In the password flow above, step obtained the encrypted private key from the issuing authority and prompted for the password to decrypt it. It generated a one-time token, then immediately used it to get a certificate.

Instead of broadly distributing the JWK provisioner password, we can generate a token in one place (e.g., CI/CD, configuration management, or orchestration):

$ step ca token alice@yourco.com

Then pass the token along for a host, container, or service to obtain a certificate:

$ step ca certificate alice@yourco.com alice.crt alice.key --token $TOKEN

Single Sign-On

Certificates provide a secure, flexible, scalable mechanism for authenticating people, too (e.g., for SSH access, API access, or to connect to a BeyondCorp identity-aware proxy). Most organizations already have an identity provider (IdP) for authenticating people. The OIDC provisioner lets you leverage authentication services from Google Workspace, Okta, Microsoft Entra ID, and any other IdP that supports OAuth OIDC to authenticate a certificate request.

Developer single sign on for TLS certificate

$ step certificate inspect ee.crt -short
X.509v3 TLS Certificate (ECDSA P-256) [Serial: 2676...4589]
  Subject:     developer@yourco.com
               05dc4b5d-769a-4659-92ee-9cf5d3b720b3
               https://auth.okta.com#05dc4b5d-769a-4659-92ee-9cf5d3b720b3
  Issuer:      Prod Intermediate CA
  Provisioner: okta [ID: 5310...fc2a]
  Valid from:  2021-06-30T22:52:09Z
          to:  2021-07-01T14:53:09Z

ACME

The ACME provisioner implements the ACME standard, created and used by Let's Encrypt to secure over 260 million public websites. ACME lets you automatically get a certificate for a domain name or IP address. ACME's rich client ecosystem makes it especially easy to integrate.

ACME is a simple challenge-response protocol. A client orders a certificate and the authority responds with a set of challenges for the client to complete to prove control over the identifier(s) in the order. For example, if a client orders a certificate for a DNS identifier, the authority may challenge the client to serve a random number over HTTP from a random path to prove control over the domain name.

You can use any standards-compliant ACME client to get a certificate from your Certificate Manager authorities. Here's an example workflow using Certbot, an open source ACME client maintained by the EFF:

sudo REQUESTS_CA_BUNDLE=$(step path)/certs/root_ca.crt \
  certbot certonly -n --standalone -d foo.internal \
  --server https://ca.internal/acme/acme/directory

ACME works almost anywhere and is especially useful for issuing certificates to web servers for use with TLS. To issue certificates for internal names that aren't in public DNS the ACME server does need access to local DNS. It's common to use a linked CA or a network-local RA. Reference the Certificate Manager docs for more details.

Next steps

To learn more:

  1. Get hands-on - Mint your free hosted Authority in less than five minutes.
  2. Get advice- Talk to Smallstep about your project.