How can I store a binary file in a Kubernetes ConfigMap?

Question

Can one store a binary file in a Kubernetes ConfigMap and then later read the same content from a volume that mounts this ConfigMap? For example, if directory /etc/mycompany/myapp/config contains binary file keystore.jks, will

kubectl create configmap myapp-config --from-file=/etc/mycompany/myapp/config

include file keystore.jks in ConfigMap myapp-config that can later be mapped to a volume, mounted into a container, and read as a binary file?

For example, given the following pod spec, should keystore.jks be available to myapp at /etc/mycompany/myapp/config/keystore.jks?

apiVersion: v1
kind: Pod
metadata:
  name: myapp
spec:
  containers:
  - name: myapp
    image: mycompany/myapp
    volumeMounts:
    - name: myapp-config
      mountPath: /etc/mycompany/myapp/config

  volumes:
  - name: myapp-config
    configMap:
      name: myapp-config

Kubernetes version details:

derek@derek-HP-EliteOne-800-G1-AiO:~/Documents/platinum/fix/brvm$ kubectl version
Client Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.6", GitCommit:"ae4550cc9c89a593bcda6678df201db1b208133b", GitTreeState:"clean", BuildDate:"2016-08-26T18:13:23Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.6+coreos.0", GitCommit:"f6f0055b8e503cbe5fb7b6f1a2ee37d0f160c1cd", GitTreeState:"clean", BuildDate:"2016-08-29T17:01:01Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}

Solution 1

Binary ConfigMaps are now supported since Kubernetes version 1.10.0. From the readme notes:

ConfigMap objects now support binary data via a new binaryData field. When using kubectl create configmap --from-file, files containing non-UTF8 data will be placed in this new field in order to preserve the non-UTF8 data. Note that kubectl's --append-hash feature doesn't take binaryData into account. Use of this feature requires 1.10+ apiserver and kubelets. (#57938, @dims)

See the changelog for more details: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.10.md#apps

Solution 2

What I would do is encode this file in base64 and then the container that uses decoded to be able to use it

Solution 3

Based on other answers, Base64 works for me (just once)

Steps:

on my workstation

base64 -w 0 cacerts   > cacerts.base64
sha256sum.exe cacerts.base64
keytool.exe  -list -v -keystore cacerts

OpenShift

I connect to openshift and create the config map

oc create configmap cacerts.base64 --from-file=cacerts.base64

deployment configuration

  ...
  template:
    metadata:
      name: mydeployment...
    spec:
      volumes:
        - name: cacerts-volume
          configMap:
            name: cacerts.base64
      containers:
        - name: crg-driver
          command:
            - base64
      args:
            - '--decode'
            - '-w 0'
            - '/opt/axatech/openpaas/certificates/cacerts.base64 >   /opt/axatech/openpaas/certificates/cacerts' #this does not work yet
          env:
            - name: SWARM_JVM_ARGS
              value: >-
                -Djavax.net.ssl.trustStore=/opt/certificates/cacerts.base64
                -Djavax.net.ssl.trustStorePassword=changeit

         volumeMounts:
           - name: cacerts-volume
             mountPath: /opt/certificates

The easiest way to edit/update an existing cacerts is to encode the new cacerts to base64 (with option -w 0) , open it with file editor (ie Notepad), copy the content and paste it via the OpenShift Console UI

https://osconsole.mycloud.something.example/console/project/project-dev/browse/config-maps/cacerts.base64

or in command line

oc edit configmap cacerts.base64

Solution 4

According to Jorgan Liggitt in Kubernetes issue "Enable ConfigMaps to store binary files as well as character files.", Kubernetes 1.3.6 cannot store a binary file in a ConfigMap.

GitHub comment 1:

config maps store data as string, not []byte... not sure I'd expect to be able to put arbitrary binary content in them"

GitHub comment 2:

@liggitt Do ConfigMaps not encode binary content as strings?

they do not, they store strings. base64-encoding could be layered on top with application logic if desired

I subsequently demonstrated that ConfigMaps do not support binary files.

Solution 5

In case you are using Secret to store value, you can leverage the property of stringData field. Encode the value once and put it in stringData field instead of data field. Kubernetes encodes the value present stringData. This way we reduce manual base64 encoding from two to one.

apiVersion: v1
kind: Secret
metadata:
  name: my-secret
type: Opaque
stringData:
  some_key: YWJjZA==

For more information on why double encoding is needed for binary data, you can refer to this link

View more solutions

Comments

[Derek Mahar]

Good idea! I might do this until Kubernetes implements github.com/kubernetes/kubernetes/issues/32432.

[Jules Ivanic]

Does it work ? I encoded my keystore in base64, added this base64 value in my Secret, mounted my Secret as a Volume as explained here: kubernetes.io/docs/concepts/configuration/secret/…, and finally deployed to my cluster. All my pods including this volume now fail to launch :/

[Jules Ivanic]

Actually it works but you have to "base64 encode" the binary file twice. If you do it only once, the value saved in the store is in its binary form (as a string). This string contains a lot of strange characters that the Go code is not able to understand. Finally, to create the .jks file in the server you could use a initContainer: kubernetes.io/docs/tasks/configure-pod-container/… which will "base64 decode" the value in the Secret and put it in a file.

[Akito]

Link is broken. Whats the status of this feature in 2020?

[Ronak Patel]

@Thamaraiselvam How can I add binary data in kubectl secret?

[Thamaraiselvam]

@RonakPatel kubectl create configmap <name> - - from-file <binary file >

[B.Z.]

@Akito kubernetes.io/docs/concepts/configuration/configmap/… Also an example from openshift site: docs.openshift.com/container-platform/4.4/builds/…

[Amala]

How can you override this autodetection of text/binary?