How to generate certificates for Kubeadm with Vault
If you are using Kubeadm to setup a Kubernetes cluster you see that it generates a default kubectl configuration file. It contains a base64 encoded certificate and private key to access the Kube-Api. This certificate has full Kubernetes access because it has the Organisation value system:masters. As you can see in the RBAC Documentation the system:masters ClusterRoleBinding has full access as super-user.
One disatvantage is, that this certificate has a long livetime of 1 year and the second one is, that it is only one certificate but you won’t share the same certificate with other users. If one of the users leave your company you have to invalidate the certificate. Another gloomy scenario is, what happens if someone gets access to the certificate. If the livetime is very long you have problems in invalidating it, you can try it via a blacklisting, but how to tell the Kube-Api Server to backlist this certificate?
So there are many scenarios that maybe make it relevant to have certificates with a short livetime and maybe with different ClusterRoleBindings then system:masters.
Here I’ll show you how you can handle this by using Hashicorp Vault. Vault has a build in Secret Engine for PKI’s and we will use it to generate a PKI that signs certificates with different access levels to the Kubernetes API.
Setup
First we have to generate a new mountpoint (kubernetes-pki) from the PKI Secret Engine:
vault secrets enable -path=kubernetes-pki -description="Kubernetes certificate chain" pki
After this we can download the root certificate and key from the master server. There the certificates are located in the /etc/kubernetes/pki folder. Maybe you have changed this folder via kubeadm init –cert-dir /path then you have to look at this folder. There you will find many crt and key files that are used by the different components like apiserver, kubelet etc. Furthermore you can find there two files called ca.crt and ca.key these are the root certificates used to generate the other certificates placed in this folder.
Generate a bundle file that contains both files, the ca.key and ca.crt file in the written order:
-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
Now we can use this bundle as root certificate for our mountpoint kubernetes-pki:
vault write kubernetes-pki/config/ca \
pem_bundle=@bundle.pem
After this we can start writing different roles and assign them to policies and then to different users with access to our Vault Server.
Generate custom roles with short TTL
Here is an example how we can write a role for system:masters access but in this case with only a short livetime:
vault write kubernetes-pki/roles/admin-role \
allowed_domains="kubernetes-admin" \
organization="system:masters" \
enforce_hostnames=false \
allow_bare_domains=true \
server_flag=false \
client_flag=true \
ttl=1h \
max_ttl=2h
This will generate a role and you can now issue certificates with this role.
vault write kubernetes-pki/issue/admin-role common_name=kubernetes-admin
The interresting part here is, that the certificate issued with this role has by default only a livetime of 1 hour and a max livetime of 2 hours. You can also write differnt roles for admin access, then you can change the allowed_domains value for example to usernames. If you have enabled the Auditing feature you will see that the access information are stored with the username specified in the allowed_domains value.
By changing the organization you can also specify another ClusterRoleBinding to be used. Therefore you can also define your own RBAC policies and use the organization certificate value to connect them.
Connect roles with policies
Now you can connect your generated roles with policies to give different users the ability to generate certificates for the different roles. Here is a short hcl file for giving a user access to issue new certificates:
path "kubernetes-pki/issue/admin-role" {
capabilities = ["create", "update"]
}
Then you can apply the role and assign it for example to a user:
vault write sys/policy/kubernetes-admin policy=@admin-role.hcl
vault write auth/userpass/users/<username>/policies policies=kubernetes-admin
Automatically generate Kubeconfig file
With the above desribed information we now can write a short script that automatically issues a certificate and generates the corresponding kubeconfig.yml file:
response=$(vault write -format=json kubernetes-pki/issue/admin-role common_name=kubernetes-admin)
certificate=$(echo ${response} | jq .data.certificate -r | base64)
privateKey=$(echo ${response} | jq .data.private_key -r | base64)
ca=$(echo ${response} | jq .data.issuing_ca -r | base64)
cat <<EOF >KUBECONFIG
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: ${ca}
server: https://<master-ip>:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: ${certificate}
client-key-data: ${privateKey}
EOF
echo "export KUBECONFIG=$(PWD)/KUBECONFIG"
With the export command in the last line you are now able to use kubectl with short living certificates.
