Kubernetes SSL Certificate Automation using Certmanager - Part 1

Published on 24 November 2021 03:04 PM

Kubernetes is an open-source system for automating deployment, scaling and management of containerized applications. It's being widely adopted in the industry at a much faster rate. When we deploy any web app on top of K8s (Kubernetes), we have to create an ingress object so that it can be accessible from the internet. But the problem is, to allow that ingress, url be served on https, browser needs an SSL certificate signed by a trusted authority and nowadays most of the browsers block the clients from viewing the webpage if you use a self-signed certificate. There are some trusted authorities which offer paid ssl certificates and on the other side we have services like openssl and LetsEncrypt authority from which we can generate ssl certificates for free of cost. We are going to use letsencrypt for our use-case.

About LetsEncrypt

Let's Encrypt is a free, automated, and open certificate authority brought to you by the nonprofit Internet Security Research Group (ISRG). It uses the Automatic Certificate Management Environment (ACME) Protocol to verify that you are the rightful owner of a domain for which you are trying to generate the certificate for and then issues a certificate for the same. ACME can also be used to automate the process of verification and certificate issuance as well as certificate revocation. But don't worry if you didn't understand any of this as we won't be doing it on our own and the whole process will be automated using cert-manager. Let's Encrypt is completely free of cost to use but the only issue is that it issues ssl certificates for a period of 3 months only and we have to renew the certificates manually after every 3 months but good news is certmanager handles the automatic renewal part pretty swiftly.

What is Cert Manager

Cert-Manager adds certificates and certificate issuers as resource types in a Kubernetes clusters, and simplifies the process of obtaining, renewing and using those certificates. It can issue certificates from a variety of supported sources, including Let’s Encrypt, HashiCorp Vault, and Venafi as well as private PKI. You can also see the list of supported issuers here It will ensure certificates are valid and up to date, and attempt to renew certificates at a configured time before expiry. The default time to renew certificates is 30 days before the expiry date.

How to use Certmanager with Kubernetes

To use certmanager in a Kubernetes cluster, we need to install it first and for that we will use it's helm chart. Helm is a package manager for Kubernetes. Helm is the best way to find, share, and use software built for Kubernetes. Now, to deploy using Helm, we have 2 options - either go with Helm CLI or use a GUI dashboard and we are going to do it both ways.

Setup Certmanager using Helm

Make sure you have your Kubernetes cluster ready and it is accessible using kubectl. After that follow the given steps to setup cert-manager on Kubernetes cluster

Download and Install Helm in your system/bastion.
Add jetstack helm repository

helm repo add jetstack https://charts.jetstack.io

Install cert-manager helm chart from the above repository in namespace cert-manager

helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --set installCRDs=true

Certmanager setup is completed now, you can confirm the pods running in cert-manager namespace

kubectl get po -n cert-manager

Create Clusterissuer

For the timing we'll create an ingress based clusterissuer which will issue certificates for subdomains specific to your host that you mention in the ingress resource. We'll update the same clusterissuer in the next part where we'll generate a wildcard certificate and it globally for all the ingress resources. So, let's begin

Run the following command to create a clusterissuer

cat <<EOF | kubectl create -f -
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
 name: letsencrypt-staging
spec:
 acme:
   # You must replace this email address with your own.
   # Let's Encrypt will use this to contact you about expiring
   # certificates, and issues related to your account.
   email: user@example.com
   server: https://acme-staging-v02.api.letsencrypt.org/directory
   privateKeySecretRef:
     # Secret resource that will be used to store the account's private key.
     name: example-issuer-account-key
   # Add a single challenge solver, HTTP01 using nginx
   solvers:
   - http01:
       ingress:
         class: nginx
EOF

What's there:

letsencrypt-staging - Name of clusterissuer
http01 Solver - It will create a job which will verify that the domain is live and you are the owner using ACME
ingress class - Class name that you have setup in your ingress controller. I have given the default class nginx in the yaml, you can change it accordingly if you have setup some other class in the controller For more http01 options, refer to the doc

Creating an Ingress

Make sure that the host for which you are trying to generate the certificate is already mapped to the ingress controller loadbalancer in your DNS before creating the ingress because as soon as we create the ingress resource, it will start trying to verify that the host is working and it's live.

To create an ingress with command line interface using kubectl, run the given command:

cat <<EOF | kubectl create -f -
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 annotations:
   cert-manager.io/cluster-issuer: letsencrypt-staging
   nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
   nginx.ingress.kubernetes.io/ssl-redirect: "true"
 name: demo-ingress
 namespace: devtron-demo
spec:
 ingressClassName: nginx
 rules:
   - host: demo.example.com
     http:
       paths:
         - backend:
             service:
               name: demo-service
               port:
                 number: 80
           path: /
           pathType: ImplementationSpecific
 tls:
   - hosts:
       - demo.example.com
     secretName: demo-service-tls
EOF

What's going on:

letsencrypt-staging - Name of clusterissuer which will issue certificate
host - Subdomain for which you are trying to issue the certificate. This method cannot provide certificate for root domain like example.com or wildcard certificate like *.example.com
ingressClassName - Class name that you have setup in your clusterissuer as well as ingress controller.
secretName - After the certificate is issued, it will store the certificate and key in that secret inside kubernetes cluster which will be used whenever someone try to visit the host url to server https requests.

Verify the SSL Certificate

As soon as the ingress with cert-manager annotation and tls secretName is created, letsencrypt ACME resolver will start verifying that the domain is pointing to right loadbalancer and it's live. It will create a resource named CertificateRequest which will request for certificate issuance which in turn creates an Order resource. Order resources are used by the ACME issuer to manage the lifecycle of an ACME ‘order’ for a signed TLS certificate. Order creates a Challenge resource which are used by the ACME issuer to manage the lifecycle of an ACME ‘challenge’ that must be completed in order to complete an ‘authorization’ for a single DNS name/identifier. More details on ACME orders and domain validation can be found on the Let’s Encrypt website here.

As a user, you won't have to worry about anything as everything is managed by cert-manager itself and as soon as everything is completed, you can verify the status of certificate. To verify using cli, run the command

kubectl describe certificate demo-service-tls -n devtron-demo | egrep "Message|Status|Type"

Don't forget to change the namespace with the namespace where you have deployed the ingress because certificates are stored and accessible in that particular namespace only. The command should return the following response

Status:
   Message:               Certificate is up to date and has not expired
   Status:                True
   Type:                  Ready

To verify the status on Devtron - Just check if the resource Certificate in the app under Custom Resource is showing healthy or not. If healthy, that means Certificate is successfully issued and stored in the secret. You can also hover on the Certificate and click on Manifest to check the status given above at the end of Certificate Manifest.

After that your host will be successfully accessible on https without any warnings using an ssl certificate from an authorized issuer and Certmanager with automatically keep a check on when to renew the certificate. It will automatically renew the certificates issued by it before 1 month of expiry so it's kind of lifetime free ssl certificate for you and you don't have to worry about anything.

Sharing the Certificate across namespaces

To share the certificate between different namespaces or multiple clusters, just export the secret from this namespace using

kubectl get secret demo-service-tls -n devtron-demo -o yaml > demo-tls-secret.yaml

and apply the same in other namespace or cluster

kubectl apply -f demo-tls-secret.yaml -n new-namespace

this new namespace can be in the same cluster or different also but it will work with the ingress of same host as you'll just have to give the reference to ingress resource in the tls section as shown in above example but don't use cert-manager annotation.

I hope you got some idea about how SSL certificate generation works in Kubernetes and how things work with cert-manager, ingress and how easily you can perform all these by using Devtron's intuitive dashboard. In the next part, we'll cover DNS based SSL certificate generation and that too a wildcard one which can be used directly in any ingress or can be directly integrated with ingess controller. You won't even need to use a LoadBalancer as it can also be used on NodePort as it will be a wildcard certificate. Till then, do share the article with your friends and colleagues so that they can also enjoy a free lifetime SSL certificate without having to worry about it's renewal.