kcf-tech.svg

KCF Technologies

[KCF Technologies](https://kcftech.com/) replaced passwords with certificates and improved collaboration while better securing OpenVPN endpoints. “One of the things that I saw that we could be doing better was the security around our embedded devices and how they authenticate to the cloud,” said Myron Semack, Chief Infrastructure Architect at KCF Technologies. “Today each base station has a unique secret, effectively a password, and we wanted to upgrade to a more robust certificate-based method.” KCF Technologies is a machine health company with a stated goal is to transform American manufacturing. Data is collected using vibration sensors on machines and passed to a central base station at each site. The base stations securely connect to the KCF Technologies cloud analytics platform. “We work with our customers to analyze the data, to understand what's happening with their machines, and identify issues before they happen.”

“A big project of mine was figuring out a way to improve OpenVPN authentication to base stations in the field,” said J. Hunter Hawke, Cybersecurity Analyst at KCF Technologies. “I started searching, and I ran into large PKI vendors that were very pricey. I kept looking into options and found smallstep on a Reddit comment thread. I went to the website and what I saw looked like something we could quickly make fit our use case. More importantly, it looked like an option we could expand into the long-term plan for KCF technologies as opposed to just something that can solve this one problem.”

<Quote content="Smallstep is a very powerful yet simple to use toolchain." author="J. Hunter Hawke, Cybersecurity Analyst at KCF Technologies" />

## Building on open-source step-ca

“I came across [a smallstep blog post](https://smallstep.com/blog/x509-certificate-flexibility/) explaining the certificate chaining and OpenVPN configurations,” continues Hunter. “When I was doing my proof of concept, I copied and pasted files from the blog post over to an EC2 instance, ran it, and up it came. Compared to our other applications, it was a similar amount of effort building the architecture around smallstep.”

[High availability](https://smallstep.com/docs/step-ca/certificate-authority-server-production) was a challenge the KCF team needed to solve. “We created a Fargate cluster running on AWS ECS to host the certificate authority instances. We set up an application load balancer with three instances behind it at the moment. As needed, we can scale them up and down.” Getting to this state required some creative configuration, including a shared EFS file system and a set of lambdas to keep things in sync. “We were running into the problem of how to handle the configuration file, making sure that it always stays up to date on every instance,” said Hunter. “So I wrote a lambda that pulls parameters from a secret store from a second account as one boundary of security.” “We also separated the MySQL database, hosted in RDS, and set up protections around the whole system to make sure that it's highly available.

The KCF team used [the smallstep provisioners](https://smallstep.com/docs/step-ca/configuration#provisioners) to help bootstrap certificate issuance across many scenarios. “I wanted to use OIDC provisioner because I thought that was so cool,” said Hunter. “We are using that to issue any developer certificates. There are other in-house applications where OIDC provisioner was a great play that we will look to expand. We are using a JWT provisioner for OpenVPN server certificates. Finally, we are also using the X5C provisioner for authenticating the base stations to the cloud.”

The smallstep solution is now completely automated. “We describe our infrastructure as code using Terraform," Hunter continued.  "I have a module defined to stand up a certificate authority and pull the necessary files from the original parent configuration. It then sets up a lambda, a file system, a database, and event notifications. Finally, the module creates an intermediate certificate authority and populates all the necessary configurations. If we need another certificate authority, I can instantiate the model module, hit enter, and spin everything right up."

“Ultimately, it wasn't too much work. It was more or less just fiddling with the smallstep pieces at each step of the way, ensuring that the configuration was consistent and reliable, said Hunter. “I would highly recommend reading [all the smallstep documentation](https://smallstep.com/docs/), read it end to end, and every word. That would have saved me lots of time.”

##  More secure, happier customers, and big plans

“Previously, we'd always have to fiddle around with the best method of authenticating,” Hunter continued. “For example, one of my colleagues needed to set up a system that distributes files to base stations. He was initially just going to do it by setting up different passwords. Instead, with smallstep, he used a signed certificate from the certificate authority that all of our base stations automatically already trust. This reduced a bunch of complexity.”

“Smallstep has also improved our collaboration with the Hardware team,” said Hunter. Now that they can use the certificate authority to pull certificates and issue them to devices, they were quite impressed with the product and very happy with all the new fun things they can do to make our hardware more secure. It's easy to take the smallstep toolchain and customize it to what we need.”

Myron also weighed in on some of the benefits of the smallstep toolchain. “Being able to host a certificate authority under the certificates.kcftech.com domain makes it much easier for our customer's I.T. departments to understand and apply firewall rules. It's the KCF Technology domain, as opposed to some random website elsewhere on the Internet, which some larger PKI providers require.”

“The fact that we are going to certificate-based authentication is a Gold Star on our product when compared to many other options out in the industry,” said Myron. “The biggest thing is getting certificates rolled into our products. It's going live with our next generation of hardware, and there is a lot of interest from the Hardware Team in retrofitting this into the existing base stations.”

The KCF team plans to extend the use of the smallstep toolchain. “Now that we have improved the security of pushing data into our cloud, I'll say the next step for this solution is securing remote management of base stations. We can leverage the same certificates to provide a more secure, more traceable, shorter-lived mechanism for our engineers and support staff to check base stations’ status,” said Myron.

The KCF Technologies team is happy with the results. “Smallstep works exactly as intended,” said Hunter. “I haven't run into any problems. I admit I was initially a little worried about handling state among all of them. But I found that there have been no problems whatsoever.”  Myron piled on, “I will say you guys have one of the more polished open source solutions. Many open-source things are a random tool by itself that's not useful without a whole bunch of other stuff around it. You guys have built a complete toolchain that makes implementing certificates much easier.“


thumbnail_kcf-technologies.png

kcf-technologies.png

KCF Technologies improved OpenVPN security for embedded devices

verint.svg

Conversocial, Inc.

[Conversocial](https://www.conversocial.com/) helps brands develop meaningful relationships with their customers at scale. Tapping into the unique nature of messaging and combining human agents with adaptive automation, Conversocial enables brands to deliver conversational customer experiences that delight consumers and transforms customer service, marketing, and sales. Conversational managed SSH keys using a hand-rolled solution using configuration management. “The automation part of our platform came on from a company called Assist, which we acquired a couple of years ago,” said James Legg, Team Leader of the Platforms and Infrastructure Engineering Team at Conversocial. “It’s a VPN plus an SSH key solution managed by saltstack and used to push the keys out to machines and the agent workspace,” said James. “We were building out new environments, and I didn't want to manage SSH Keys if I could help it. I was looking for something that could be integrated with Google Workspace single-sign-on and found Smallstep.” Historically SSH key management fell on James’ team and was often a manual effort. “We have internal tools that generate big, long, horrible SSH strings which you copy and paste in the terminal,” he continued. “The aim is not to have too many things to decommission when someone leaves the workplace. We've been building role-based access control for internal engineering system access managed using Google Group membership based on that aim for a while now. And what smallstep created fit all those boxes with minimal effort.”

<Quote content="We were building out new environments, and I didn't want to manage SSH Keys if I could help it." author="James Legg, Platform & Infrastructure Team Lead" />

## Say goodbye to manual SSH processes.

“I didn’t find any other vendors that were doing the same thing. The trial button was quite nice and easy, so we didn't have to go through too much to try this out and play with it. So I decided, OK, we'll give it a go. The ease of doing it, without having to think too hard about getting it right, made a big difference”, James continued. “SSH, and especially certificate based ssh is one of those areas where you can think you've got it right and you've actually screwed it up entirely. And the consequences when you discover that it’s incorrect are either a massive security hole, or you’ve accidentally locked yourself out of the system. Neither of which are things you want to think about happening.

“I think compared to what we were doing before, the user experience has been pretty nice. Before Smallstep, we're forever telling people, oh yeah, you're running that ssh helper script in the wrong python virtual environment. You've got your dependencies broken, or we've upgraded it, and it's telling you that. But you didn't need it because it fell over somewhere else in your script. All of that is no longer a problem.” Adding smallstep to the standard shell scripts for MacBook users removed users’ need to even think about SSH access. “One of the scripts installs Smallstep and sets up the SSH configuration, so people don't think about it. It's just kind of magic when the user gets a popup single-sign-on window for SSH.”

“It's nice for the Engineering team to have the ability to go and look at a database in development or administer something on a machine without having to think very hard about it.” 


## A managed offering is paying early dividends.

James benefited from the value of using a SaaS offering. “Oh, look, someone else is prepared to do the work for us for a fairly nominal fee. And hopefully, I don't have to debug why someone's lost their SSH key or why it has space where it shouldn't be. Or maybe they truncated it. I’m avoiding all this kind of stuff, which is never very much fun to do. I'm not missing SSH Keys, that's for sure.”

Smallstep is just beginning to provide value for the Conversocial team. “we've still got lots more to do in terms of migrating applications to reap the value of Smallstep for lots of people.”  James has one last message for folks considering Smallstep SSH. “If you want to not think about managing SSH keys and to be able to have access revoked at the same time you revoke the access to the rest of the systems, it's really good.”


thumbnail_convertsocial-featured.png

convertsocial-featured.png

Conversocial simplified role-based access control for internal engineering systems

cisco-kenna.svg

Kenna Security

Kenna replaced SSH Keys with Certificates and increased security with less effort resulting in 60x faster Ansible deployments. Kenna Security is the enterprise leader in risk-based vulnerability management. Kenna Engineering Operations is responsible for maintaining the systems that run code across ten different environments running on GCP and AWS. The traditional method of delivering the right SSH access to developers by deploying static key files is a significant operational challenge that, Kenna found, demanded increasing time and attention as they grew.

<Quote content="Smallstep SSH is exactly what we needed. Significantly reducing the work required to manage SSH keys." author="Joe Doss, Director of Engineering Operations" />

## Managing User-Generated Keys At Scale

Moving quickly and keeping developers happy is not always easy. “We had a meeting about the things that sucked within our department, and more often than not, people were saying it's the tooling, things like SSH access.” Said Joe Doss, Director of Engineering Operations at Kenna Security. Upon further examination, the team discovered that SSH access management had grown into an operational expense. “We had all the classic pain points with managing SSH keys.”

Like most organizations, new user onboarding was a challenging experience for all involved parties. “When a new user came on board, they would follow a Confluence template. It described how to generate SSH keys, and in bold letters, it stated, please use a strong password. Not only was that putting the burden onto that individual signing up, it didn’t ensure strong security.

> Everyone's concept of a strong SSH key password is different. To some, it’s the password to their luggage. Right? It could be. You have no idea.
Joe Doss

When the user finished creating the key, they would create a JIRA ticket with their new public key (occasionally, an errant private key would be submitted -- a clear indication of a flawed and confusing process). The JIRA ticket would become the responsibility of the Fleet Operations Team. This team is responsible for maintaining order across all of the deployments using Ansible to automate configurations. “It started as a relatively straightforward task to update SSH keys across our environment,” said Tommy Santoyo, System Engineer at Kenna Security. “But now that it is ten different environments, it's much more painful. It takes hours to set up and run these playbooks.”

The challenge of managing SSH keys extended beyond new users. “We also have to consider deleting users who leave the company, rekey operations, and managing hostname reuse.” Tommy continued. “And let’s not forget the pain of when we have people blow away old laptops for a new one.” Like many, the toil of managing SSH keys added up quickly for the Kenna Security team.

## SSH Certificates Are The Answer

Certificate Authorities (CAs) are not a new concept. They are the foundation for security on the internet. Historically these CAs have been challenging to operate, and mistakes can be disastrous. “When it came up that we should be doing SSH certificates, my immediate response was: do you want to run the CA? Because I don't,” said Joe Doss. “Even after I learned that smallstep manages the CA, I still had a bias against running a CA.”


thumbnail_kenna-featured.png

kenna-featured.png

Kenna increased deployment speed 60x with Smallstep

kameleoon.svg

Kameleoon

[Kameleoon](https://www.kameleoon.com/en) is a powerful and easy-to-use A/I driven, A/B testing personalization platform helping 500+ brands across e-commerce, financial, and healthcare verticals, deliver exceptional digital experiences and products to their customers.

Jimmy Passemard, Kameleoon's _Chief Information Security Officer_, has embarked on a journey in the past 18 months to build a security program that will help the organization become SOC 2 certified after obtaining the ISO 27001 certification. Kameleoon's customers are in industries that require security and compliance within their business and across their partnerships. A SOC 2 certification will further assure customers that Kameleoon is safe, credible, and their data is being protected. 

## Kameleoon's Journey to SOC 2 Compliance

Prior to deploying Smallstep, Jimmy Passemard's greatest challenges were:
- Time spent managing user and server access
- Not having an efficient system to record keep
- Bringing awareness to audit risks in order to stay in line with ISO 27001 and SOC 2 compliant standards

Scattered and unmonitored SSH keys resulted in a lack of trust within the org's infrastructure; weakening Kameleoon's trust posture. This block impacted the audit process and initially hindered Kameleoon's goal in staying compliant. 

It took Jimmy Passemard about 2-3 hours to manually audit each developer's session, meaning that Jimmy's team was constantly in reactive mode and not able to invest time into strategically improving Kameleoon's trust posture. 

## Smallstep's Role in Kameleoon's Infrastructure Transformation

Kameleoon sought out [enterprise SSH access](https://smallstep.com/solutions/sso/) for its DevOps team, a solution that would enable the organization to eliminate the need for multiple credentials that were once being utilized by developers and personnel for root access to key systems. The first step to improving security is to eliminate the need for SSH keys.

With several hundred hosts machines that are accessible by systems and users via SSH, Smallstep's [SSH Professional Principal](https://smallstep.com/sso-ssh/) feature, automated role based access for groups and users on machines. This implementation comes without compromising access to certain servers with the same access/privileges. In the event that an employee departs from the organization, Kameleoon now has a modernized approach to immediately rescind access, prevent operational overhead, and maintain a well-defined audit trail. 

"Now it's a lot easier for me to at least set a policy," said Jimmy Passemard. "This population will have access to these resources or this server with or without root access. It's so easy for me to put people in the appropriate groups. Something we can do automatically. Something that is quite easy to keep track of and audit. I know at which point who has access to what server which I really liked, and my auditors liked," Jimmy Passemard continued. 

<Quote content="I know who has access to any server, which I liked, and my auditors loved." author="Jimmy Passemard, CISO" />

## Smallstep's Impact on Kameleoon's Security and Audit Process

With Smallstep, Kameleoon has unlocked granular control and better visibility into its infrastructure. Jimmy Passemard, _CISO_, describes Smallstep as a "lightweight solution in terms of usage," that is closing potential security gaps to redefine a more safe and secure ecosystem. Since Kameleoon made the switch to Smallstep, the organization has stopped deploying SSH keys and adopted a seamless SSH certificate management process for its team of developers. 

Smallstep SSH integrated easily with Kameleoon's IDE, which uses an FTP tunnel to access server resources. Smallstep SSH uses the standard OpenSSH software that developers are used to. It adds no extra web interfaces for users to learn, so no additional training or onboarding is required.

"[Smallstep] helped us build a lot of trust during the audit process. We were able to see so much progress and pay close attention to something that is at the core aspect of infrastructure security," said Jimmy Passemard. Jimmy Passemard recognized the Smallstep Platform as falling-in-line and encouraging the practice of the Zero Trust philosophy. However, this depth of trust is ultimately at the discretion of the organization. Our offering doesn't stop delivering at small, medium, or enterprise scale implementations and uses.

Refining processes and staying on top of best security practices without affecting the health of its infrastructure is currently top of mind at Kameleoon. Smallstep has and will continue to be "more sufficient and accomplish all our needs," Jimmy Passemard concluded. 

Serving teams that leverage modern infra as a competitive advantage