Run a private ACME server and issue X.509 certificates using an ACME challenge

ACME support in step-ca allows software to leverage existing ACME clients and libraries to get X.509 certificates from your own certificate authority (CA) using an ACME challenge. ACME is a standardized protocol. It describes a mechanism for automatic validation and issuance of X.509 certificates from a certificate authority to clients.

About this tutorial

  • Learn how to use an ACME challenge to issue X.509 certificates to endpoints automatically.
  • Examples include a description of the different ACME challenges and a client configuration example.
  • When complete, you will have an understanding of the different ACME challenge types and client configuration.
  • Estimated effort: Reading time ~3 mins, Lab time ~10 to 15 mins.

Requirements

Overview

Why ACME?

There are many reasons you might want to run your own ACME CA.

For example, you can:

  • Use ACME in production to issue X.509 certificates to internal workloads, proxies, queues, databases, etc. so you can use mutual TLS for authentication and encryption.
  • Simulate Let’s Encrypt’s certificate authority in development and pre-production scenarios where connecting to Let’s Encrypt’s staging server is problematic.

Rundown

At a high level, ACME is pretty simple. An ACME client creates an account with an ACME server and submits a certificate order. The server responds with a set of challenges for the client to complete in order to prove it has control over domain names, or identifiers, in the certificate. Once the client successfully completes the ACME challenge, it submits a certificate signing request (CSR) and the server issues a certificate.

There is not a single standard way to prove control over an identifier, so the core ACME specification makes this an extension point. There are three challenge types broadly used in practice, all of which are supported by step-ca.

ACME Challenges

The HTTP Challenge (http-01)

The ACME server challenges the client to host a random number at a random URL on the domain in question and verifies client control by issuing an HTTP GET request to that URL.

Use Case

A service that commonly uses this approach is Google Analytics.

The DNS Challenge (dns-01)

The ACME server challenges the client to provision a random DNS TXT record for the domain in question and verifies client control by querying DNS for that TXT record.

Use Case

You might see a challenge like this when you are trying to use third party tools, like plugins or some Amazon cloud services.

The TLS alpn challenge (tls-alpn-01)

The ACME server uses application layer protocol negotiation to request an ACME challenge validation. The client then presents a self-signed tls certificate containing the identifier in question as a special x509 extension.

Use Case

The TLS ALPN challenge is commonly used in cases where an ingress controller fronts clients attempting to receive a certificate, or in cases where, for any number of reasons, TLS is the only allowed external protocol.

ACME with step-ca

Now, let's demonstrate provisioning a X.509 certificate with ACME using step-ca.

Add an ACME provisioner to your step-ca configuration to enable ACME:

step ca provisioner add acme --type ACME

Restart step-ca to pick up the configuration changes.

Configuring Clients

To configure an ACME client to connect to step-ca you need to do two things:

  1. point the client to the right ACME directory URL, and
  2. tell the client to trust your CA’s root certificate.

Pointing clients to the right ACME Directory URL

Most ACME clients connect to Let’s Encrypt’s CA by default. To connect to step-ca, you need to point the client to the right ACME Directory URL.

A single instance of step-ca can have multiple ACME provisioners, each with their own ACME Directory URL. The ACME directory URL has the form:

https://{ca-host}/acme/{provisioner-name}/directory

We just added an ACME provisioner named acme in step 2. Its ACME directory URL is:

https://ca.internal/acme/acme/directory

Telling Clients to Trust Your CA’s Root Certificate

Clients will validate the server’s HTTPS certificate using the public root certificates in your system’s default trust store. When you’re connecting to Let’s Encrypt, it’s a public certificate authority and its root certificate is already in your system’s default trust store. Your internal root certificate isn’t, so HTTPS connections from ACME clients to step-ca will fail.

There are two ways to address this challenge. Either:

  • explicitly configure your ACME client to trust step-ca’s root certificate, or
  • add the step-ca root certificate to your system’s default trust store.

step-cli provides a helper command to do the latter:

step certificate install

If you are using your certificate authority for TLS in production, explicitly configuring your ACME client to only trust your root certificate is a better option. You will see how this method works with an example below. You can find several other examples here.

If you are simulating Let’s Encrypt in pre-production, installing your root certificate is a more realistic simulation of production. Once your root certificate is installed, no additional client configuration is necessary.

Examples

step-ca should work with any ACMEv2 (RFC8555) compliant client that supports the http-01 or dns-01 challenge. If you run into any issues please let us know on GitHub Discussions or in an issue.

Certbot

certbot is the most commonly used ACME client. Built and supported by the EFF, it’s the standard-bearer for production-grade command-line ACME.

To get a certificate from step-ca using certbot you need to:

  • tell certbot to trust your root certificate using the REQUESTS_CA_BUNDLE environment variable
  • point certbot at your ACME directory URL using the --server flag

For example:

sudo REQUESTS_CA_BUNDLE=$(step path)/certs/root_ca.crt \ certbot certonly -n --standalone -d foo.internal \ --server https://ca.internal/acme/acme/directory

sudo is required in certbot’s standalone mode so it can listen on port 80 to complete the http-01 challenge. If you already have a webserver running, you can use webroot mode instead.

With the appropriate plugin, certbot also supports the dns-01 challenge for most popular DNS providers. Deeper integrations with nginx and Apache can even configure your server to use HTTPS automatically and you can do this with step-ca.

You can renew all of the certificates you have installed using cerbot by running:

$ sudo REQUESTS_CA_BUNDLE=$(step path)/certs/root_ca.crt certbot renew

You can automate renewal with a simple cron entry:

*/15 * * * * root REQUESTS_CA_BUNDLE=$(step path)/certs/root_ca.crt certbot -q renew

The certbot packages for some Linux distributions will create a cron entry or systemd timer like this for you. This entry won’t work with step-ca because it doesn’t set the REQUESTS_CA_BUNDLE environment variable. You need to manually update it.

By default, step-ca issues certificates with 24 hour lifetimes. Certbot’s default renewal job is tuned for Let’s Encrypt’s 90 day certificate lifetimes. It is run every 12 hours with actual renewals occurring for certificates within 30 days of expiration.

The cron entry above accounts for this by running certbot renew every 15 minutes. You will also want to configure your domain to only renew certificates when they are within a few hours of expiration by adding a line like the following to the top of your renewal configuration file, like in /etc/letsencrypt/renewal/foo.internal.conf.

renew_before_expiry = 8 hours