Sign in Subscribe
Containers

Understanding secrets in Kubernetes

Highlights some essential Kubernetes knowledges that will make us comfortable when dealing with secret objects for our applications workloads.

Gmk Ziz 👓

• 6 min read
Understanding secrets in Kubernetes

Table of contents

Overview

  • Kubernetes secret object can be used to store sensitive informations
  • The sensitive informations could then be securely transferred to pods or directly used by other Kubernetes resources
  • Sensitive informations inside secret object are by default base64 encoded
  • For other infos and cautions see Kubernetes Secrets

Secret types

Asking for help on secret object creation shows 3 types of secrets :

$ kubectl create secret -h
Create a secret using specified subcommand.

Available Commands:
  docker-registry   Create a secret for use with a Docker registry
  generic           Create a secret from a local file, directory, or literal
value
  tls               Create a TLS secret
  • We use the docker-registry secret type to create secrets for authenticating with container registries from different providers, not only Docker. A typical use case of this type of secret is pulling/pushing private container images from/to container registries. To create a secret of that type, you need :
    • the address of the container registry
    • the name of the authenticating user
    • the associated password for the authenticating user
  • The generic secret type will be used to store key:value pair data. The data could then be retrieved inside pod(s) as files or environment variables
  • The tls secret type will be used to store TLS certificates and keys. The certificate and associated private key data must be .PEM encoded. Could be used for instance with ingress controllers to make our apps accessible in HTTPS

Creating secrets

Docker-registry

Here is the creation command :

$ kubectl create secret docker-registry my-docker-registry-secret --docker-server=my-registry.example.com --docker-username=myuser --docker-password=mypassword

Let's have a look at the created secret :

$ kubectl get secret my-docker-registry-secret
NAME                        TYPE                             DATA   AGE
my-docker-registry-secret   kubernetes.io/dockerconfigjson   1      12s

$ kubectl get secret my-docker-registry-secret -o yaml
apiVersion: v1
data:
  .dockerconfigjson: eyJhdXRocyI6eyJteS1yZWdpc3RyeS5leGFtcGxlLmNvbSI6eyJ1c2VybmFtZSI6Im15dXNlciIsInBhc3N3b3JkIjoibXlwYXNzd29yZCIsImF1dGgiOiJiWGwxYzJWeU9tMTVjR0Z6YzNkdmNtUT0ifX19
kind: Secret
metadata:
  creationTimestamp: "****-**-**T**:**:**Z"
  name: my-docker-registry-secret
  namespace: juiceshop
  resourceVersion: "546844"
  uid: fd824b18-aacb-4341-a056-c3beb24b1162
type: kubernetes.io/dockerconfigjson

Let's see the base64 decoded value of the .dockerconfigjson data key :

$ echo "eyJhdXRocyI6eyJteS1yZWdpc3RyeS5leGFtcGxlLmNvbSI6eyJ1c2VybmFtZSI6Im15dXNlciIsInBhc3N3b3JkIjoibXlwYXNzd29yZCIsImF1dGgiOiJiWGwxYzJWeU9tMTVjR0Z6YzNkdmNtUT0ifX19" | base64 -d | jq
{
  "auths": {
    "my-registry.example.com": {
      "username": "myuser",
      "password": "mypassword",
      "auth": "bXl1c2VyOm15cGFzc3dvcmQ="
    }
  }
}
The Google Container Registry case

For the Google Container Registry, we can't generate a couple username/password for authentication. Instead, we have to create a Google service account and give that service account the permission to read data from GCS (Google Cloud Storage).

We can then use that service account token to create the docker-registry secret. Assuming that the ~/gcr-image-pull-token.json file contains the service account token, we will use the following values for the registry username and password within the docker-registry secret creation command :

  • --docker-username=_json_key
  • --docker-password="$(cat ~/gcr-image-pull-token.json)"

TLS

Here is the creation command :

kubectl create secret tls my-tls-secret --key=tls-key.pem --cert=tls-cert.pem

Here is a truncated output of the content of the TLS certificate and private key files we used :

  • tls-cert.pem
-----BEGIN CERTIFICATE-----
MIIDzDCCArSgAwIBAgIUOiPOFvdtyD/wOCrgaoGNsV6CuIEwDQYJKoZIhvcNAQEL
BQAwYTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM
(...)
jJBYxC4Hi6ZNnb2/UUliumVmmQZvLLcyNK+rcbGGPtP0wB+QoXhDUrKAsrM/McTK
f1IAy/XYLdk9UUcLxpN2Iw==
-----END CERTIFICATE-----
  • tls-key.pem
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA2OmXD24qOvorrGqB3GOdeNdHF1v6QIb3CSA4tNfqH3fqQ22F
iEsXHt8s2dfHlSqwZ4J5bwwzjp9eXqwwNRPDwWIaFzesWh1qsWxJIvh9Z2b/9TUD
(...)
5C9jvJzMZRAqQ2dWd0yO00a6kf/Uyi/dzVpx96iwf0nwj0R0dEndudHkH0awBAwi
JZvsLB7gYCceCXs+S/OROtFh0322pBTHwmBTSyUARRF2VOfC3bQj
-----END RSA PRIVATE KEY-----

Now let's have a look at the created secret :

$ kubectl get secret my-tls-secret
NAME            TYPE                DATA   AGE
my-tls-secret   kubernetes.io/tls   2      12m

$ kubectl get secret my-tls-secret -o yaml
apiVersion: v1
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR6RENDQXJTZ0F3SUJBZ0lVT2lQT0Z2ZHR5RC93T0
  (...truncated)
hoRFVyS0Fzck0vTWNUSwpmMUlBeS9YWUxkazlVVWNMeHBOMkl3PT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBMk9tWEQyNHFPdm9yck
  (...truncated)
Z1lDY2VDWHMrUy9PUk90RmgwMzIycEJUSHdtQlRTeVVBUlJGMlZPZkMzYlFqCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==       
kind: Secret
metadata:
  creationTimestamp: "****-**-**T**:**:**Z"
  name: my-tls-secret
  namespace: juiceshop
  resourceVersion: "651996"
  uid: b5913af1-5fab-45b2-be2d-252b7a829ea7
type: kubernetes.io/tls

Generic

The kubectl create secret generic -h command shows examples for creating generic secrets. Let's try some of them and see the result.

  • Create a new secret named my-secret with key1=supersecret and key2=topsecret
$ kubectl create secret generic my-secret --from-literal=key1=supersecret --from-literal=key2=topsecret
secret/my-secret created

$ kubectl get secret my-secret -o yaml
apiVersion: v1
data:
  key1: c3VwZXJzZWNyZXQ=
  key2: dG9wc2VjcmV0
kind: Secret
metadata:
  creationTimestamp: "****-**-**T**:**:**Z"
  name: my-secret
  namespace: juiceshop
  resourceVersion: "655183"
  uid: 4bec3a1f-f7b9-4d5f-bd5e-f218e94a218f
type: Opaque
  • Create a new secret named my-secret using a combination of a file and a literal
$ cat bar/db_password
superpass

$ kubectl create secret generic my-secret --from-file=bar/db_password --from-literal=passphrase=topsecret
secret/my-secret created

$ kubectl get secret my-secret -o yaml
apiVersion: v1
data:
  db_password: c3VwZXJwYXNzCg==
  passphrase: dG9wc2VjcmV0
kind: Secret
metadata:
  creationTimestamp: "****-**-**T**:**:**Z"
  name: my-secret
  namespace: juiceshop
  resourceVersion: "655590"
  uid: 5fe8546b-489c-4986-a711-69a4155e7f11
type: Opaque
  • Create a new secret named my-secret with specified key name. From the previous example, we will set the key name to database_password instead of the default db_password taken from the filename
$ cat bar/db_password
superpass

$ kubectl create secret generic my-secret --from file=database_password=bar/db_password
secret/my-secret created

$ kubectl get secret my-secret -o yaml
apiVersion: v1
data:
  database_password: c3VwZXJwYXNzCg==
kind: Secret
metadata:
  creationTimestamp: "****-**-**T**:**:**Z"
  name: my-secret
  namespace: juiceshop
  resourceVersion: "655654"
  uid: 362c9267-7e35-4101-a659-ae7e56c0e420
type: Opaque
  • Create a new secret named my-secret from a file containing a bunch of lines with key=value pairs
$ cat app.env 
db_user=user
db_password=superpass

$ kubectl create secret generic my-secret --from-env-file=app.env
secret/my-secret created

$ kubectl get secret my-secret -o yaml
apiVersion: v1
data:
  db_password: c3VwZXJwYXNz
  db_user: dXNlcg==
kind: Secret
metadata:
  creationTimestamp: "****-**-**T**:**:**Z"
  name: my-secret
  namespace: juiceshop
  resourceVersion: "653254"
  uid: 0297bafd-fffa-4af2-adbf-97cd3147de1f
type: Opaque
  • Create a new secret named my-secret with keys taken from each filename in folder bar and values from the associated files contents
$ ls bar/
db_password  db_user

$ cat bar/db_user 
user

$ cat bar/db_password 
superpass

$ kubectl create secret generic my-secret --from-file=bar/
secret/my-secret created

$ kubectl get secret my-secret -o yaml
apiVersion: v1
data:
  db_password: c3VwZXJwYXNzCg==
  db_user: dXNlcgo=
kind: Secret
metadata:
  creationTimestamp: "****-**-**T**:**:**Z"
  name: my-secret
  namespace: juiceshop
  resourceVersion: "654889"
  uid: ea7d7829-240c-4cbe-a3ec-9bb6750377ea
type: Opaque

Using secrets

From pods as file

Here is how we make existing Kubernetes secrets available inside pods as file(s) :

  • In our Kubernetes manifests used to create pods, we need to declare a volume inside pods spec section, that will be supplied with one or many of our existing secrets
  • According to Kubernetes documentation, volumes containing secrets are backed by tmpfs (a RAM-backed filesystem) so they are never written to non-volatile storage
  • We then have to declare a volumeMounts inside pods spec.containers section, to mount the secret(s) as file(s) to specific path(s) inside the pod(s) container(s). Here is an example :
apiVersion: apps/v1
kind: Deployment
(...)
spec:
  (...)
  template:
   (...)
    spec:
      containers:
      - name: db
        image: mariadb
        env:
          (...)
          - name: MYSQL_PASSWORD_FILE
            value: "/run/secrets/mysql/user_password"
          - name: MYSQL_ROOT_PASSWORD_FILE
            value: "/run/secrets/mysql/root_password"
        volumeMounts:
        (...)
        - name: secret-volume
          mountPath: /run/secrets/mysql
        (...)
      volumes:
      (...)
      - name: secret-volume
        secret:
          secretName: mysql-secret
  • The data from the existing Kubernetes secret named mysql-secret will be made available to pods into the /run/secrets/mysql directory
  • All data keys from the mysql-secret secret will be name of files inside the /run/secrets/mysql directory and the associated key values the files contents

Here is an example. The following key/value pairs data from the mysql-secret secret :

user_password: O3hndfn!dSZu
root_password: kHj483iYh?d2

will be available inside the pods containers as the following files :

  • /run/secrets/mysql/user_password containning O3hndfn!dSZu
  • /run/secrets/mysql/root_password containning kHj483iYh?d2

We could also specify a specific file path from where we want a specific secret data to be available. That path is relative to the mountPath of the secret volume (/run/secrets/mysql in this case).

Here is an example. If we wanted to make the user_password secret available from /run/secrets/mysql/other_dir/other_user_password_filename instead of /run/secrets/mysql/user_password, we would have used :

        (...)
        volumeMounts:
        - name: secret-volume
          mountPath: /run/secrets/mysql
        (...)
      volumes:
      - name: secret-volume
        secret:
          secretName: mysql-secret
          items:
          - key: user_password
            path: other_dir/other_user_password_filename

From pods as environment variables

  • The Kubernetes generic secret named mysecret exists and has been created with the --from-env-file option of the kubectl create secret generic command, from the following file content :
username=myusername
password=mypassword
  • The following Kubernetes Deployment manifest will make username and password data from mysecret available to pod container(s) as environment variables :
apiVersion: apps/v1
kind: Deployment
(...)
spec:
  (...)
  template:
   (...)
    spec:
      containers:
      - name: my-container-name
        image: my-container-image
        env:
          - name: USERNAME
            valueFrom:
              secretKeyRef:
                name: mysecret
                key: username
          - name: PASSWORD
            valueFrom:
              secretKeyRef:
                name: mysecret
                key: password
  • The created environment variables inside the pod container(s) are :
USERNAME=myusername
PASSWORD=mypassword

To pull private container images

Simply use the spec.imagePullSecrets to specify the name of the docker-registry secret to use for authentication when pulling private container images defined inside Kubernetes manifests files. Here is an example :

apiVersion: apps/v1
kind: Deployment
(...)
spec:
  (...)
  template:
   (...)
    spec:
      imagePullSecrets:
        - name: my-docker-registry-secret
      containers:
      - name: my-container-name
        image: myregistry.example.com/my-private-container-image
        (...)

Sign up for more like this.

Enter your email
Subscribe