Kubernetes, Docker and vm.max_map_count

Question

While attempting to run ElasticSearch on K8 I ran into an error that would kill the container:

max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

Fortunately this is pretty well documented and I was able to figure out a configuration setting for the container that got it stable. This question isn't about that.

The question that arises for me is why does this happen on K8 versus Docker? I'm using the same image (The official ES image) in both cases so what are some of the reasons why the value would be different? Is there a way I could configure K8 or the machines I'm running it on so that this situation wouldn't arise?

Solution

The setting for vm.max_map_count can be changed on the host level. Your can read the current value like this: sysctl vm.max_map_count. To change it run: sudo sysctl -w vm.max_map_count=262144. This will be reverted by the next boot. To set it permanently add vm.max_map_count=262144 to /etc/sysctl.conf.

Most Elasticsearch setups for Kubernetes use an Init Container to make sure this value is set like required. The drawback here is that it needs to in privileged mode: Elasticsearch Helm Chart

It is also possible to set sysctls in the securityContext of a Pod. But for now virtual memory settings are considered unsafe and need some more tweaking on your Kubernetes setup.

Comments

[webwurst]

Yes, I would guess so. Can you maybe read out the values with sysctl vm.max_map_count?

[Spencer Ruport]

I don't have access to do this on the K8 hosts so I'm trying to make sure I'm using the correct language when I ask the people who do.

[Will Nilges]

Apologies for dragging this out, but is there a way to set this value for one container only? Perhaps this can be discussed in chat.

[webwurst]

@WillNilges No, this is a Kernel setting and the Kernel is shared by all containers. Specific settings for containers are possible via control groups.