Before you can teach your client to speak TLS, you will need a certificate issued by a trusted certificate authority (CA). If your organization already runs its own CA and you have a private key and certificate for your Kafka Command Line Tools client, along with your CA's root certificate, you can skip to the next step.
If your organization does not yet run its own internal CA, you can read more about creating and running a CA using the open source smallstep software here.
To request a certificate from your CA using the
step CLI, run the following command.
$ step ca certificate "myuser" client.crt client.key
Your certificate and private key will be saved in
Request a copy of your CA root certificate, which will be used to make sure each application can trust certificates presented by other applications.
$ step ca root ca.crt
Your certificate will be saved in
Now, we need only to configure our Kafka Command Line Tools client to make authenticated requests using our certificate and private key. The CA root certificate will be used to verify that the client can trust the certificate presented by the server.
openssl to package up the client private key and certificate into PKCS12 format. You'll be prompted to create a password here. Hold on to this, as you'll need it in the next step and in configuration later.
$ openssl pkcs12 -export -in client.crt -inkey client.key -name myuser > client.p12
keytool to create a Java KeyStore (JKS) with the certificate and key. You'll be prompted to create a new password for the resulting file as well as enter the password for the PKCS12 file from the previous step. Hang onto the new JKS password for use in configuration below.
$ keytool -importkeystore -srckeystore client.p12 -destkeystore kafka.client.keystore.jks -srcstoretype pkcs12 -alias myuser
Note: It's safe to ignore the following warning from
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore client.p12 -destkeystore kafka.client.keystore.jks -srcstoretype pkcs12".
You'll also need a trust store in JKS format containing the root certificate from your CA. The command line tools will use this trust store to make sure the certificate presented by the broker was signed by your CA. Create the password and agree to trust your CA certificate (type "yes"). Hold onto thie password for this one as well.
$ keytool -keystore kafka.client.truststore.jks -alias CARoot -import -file ca.crt
Copy your Java KeyStore files into place.
$ sudo mkdir -p /var/private/ssl $ sudo cp kafka.client.*.jks /var/private/ssl/ $ sudo chown kafka:kafka /var/private/ssl/kafka.client.*.jks
Create a new properties file named
client.properties that you'll reference when you use the command line tools to open a connection. Configure it to use your key store and trust store JKS files.
security.protocol=SSL ssl.keystore.location=/var/private/ssl/kafka.client.keystore.jks ssl.keystore.password=<keystore password> ssl.key.password=<pkcs12 password> ssl.truststore.location=/var/private/ssl/kafka.client.truststore.jks ssl.truststore.password=<truststore password>
Since this file contains passwords for your key stores, we'll want to tighten up filesystem permissions.
$ chmod 0600 client.properties
Finally, use your
client.properties file when making connections to a broker from the Kafka command line tools.
$ kafka-console-consumer --bootstrap-server myserver.internal.net:443 -topic mytopic --consumer.config client.properties $ kafka-console-producer --broker-list myserver.internal.net:443 -topic mytopic --producer.config client.properties
step-ca issues certificates with a 24 hour expiration. Short-lived certificates have many benefits but also require that you renew your certificates each day before they expire. How you renew certificates is often dependent on how you deploy your application. See the
step-ca certificate lifecycle management docs for more information.
Smallstep is building Certificate Manager, a hosted certificate authority offering with integrations that make it easy to manage mututal TLS in technologies like Kafka Command Line Tools.
Certificate Manager is currently in early access (by request only). Please request an invitation and let us know you'd be interested in an integration with Kafka Command Line Tools.
All documentation content from the Hello mTLS project is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0).