Kenna increased deployment speed 60x with Smallstep
Kenna replaced SSH Keys with Certificates and increased security with less effort resulting in 60x faster Ansible deployments. Kenna Security is the enterprise leader in risk-based vulnerability management. Kenna Engineering Operations is responsible for maintaining the systems that run code across ten different environments running on GCP and AWS. The traditional method of delivering the right SSH access to developers by deploying static key files is a significant operational challenge that, Kenna found, demanded increasing time and attention as they grew.
Smallstep SSH is exactly what we needed. Significantly reducing the work required to manage SSH keys.
— Joe Doss, Director of Engineering Operations
Managing User-Generated Keys At Scale
Moving quickly and keeping developers happy is not always easy. “We had a meeting about the things that sucked within our department, and more often than not, people were saying it's the tooling, things like SSH access.” Said Joe Doss, Director of Engineering Operations at Kenna Security. Upon further examination, the team discovered that SSH access management had grown into an operational expense. “We had all the classic pain points with managing SSH keys.”
Like most organizations, new user onboarding was a challenging experience for all involved parties. “When a new user came on board, they would follow a Confluence template. It described how to generate SSH keys, and in bold letters, it stated, please use a strong password. Not only was that putting the burden onto that individual signing up, it didn’t ensure strong security.
Everyone's concept of a strong SSH key password is different. To some, it’s the password to their luggage. Right? It could be. You have no idea. Joe Doss
When the user finished creating the key, they would create a JIRA ticket with their new public key (occasionally, an errant private key would be submitted -- a clear indication of a flawed and confusing process). The JIRA ticket would become the responsibility of the Fleet Operations Team. This team is responsible for maintaining order across all of the deployments using Ansible to automate configurations. “It started as a relatively straightforward task to update SSH keys across our environment,” said Tommy Santoyo, System Engineer at Kenna Security. “But now that it is ten different environments, it's much more painful. It takes hours to set up and run these playbooks.”
The challenge of managing SSH keys extended beyond new users. “We also have to consider deleting users who leave the company, rekey operations, and managing hostname reuse.” Tommy continued. “And let’s not forget the pain of when we have people blow away old laptops for a new one.” Like many, the toil of managing SSH keys added up quickly for the Kenna Security team.
SSH Certificates Are The Answer
Certificate Authorities (CAs) are not a new concept. They are the foundation for security on the internet. Historically these CAs have been challenging to operate, and mistakes can be disastrous. “When it came up that we should be doing SSH certificates, my immediate response was: do you want to run the CA? Because I don't,” said Joe Doss. “Even after I learned that smallstep manages the CA, I still had a bias against running a CA.”
