Allow only one pod of a type on a node in Kubernetes

Question

How to allow only one pod of a type on a node in Kubernetes. Daemon-sets doesn't fit into this use-case.

For e.g. - Restricting scheduling of only one Elasticsearch pod on a node, to prevent data loss in case the node goes down.

It can be achieved by carefully planning CPU/memory resource of pod and machine type of cluster.

Is there any other way to do so?

Solution 1

Kubernetes 1.4 introduced Inter-pod affinity and anti-affinity. From the documentation: Inter-pod affinity and anti-affinity allow you to constrain which nodes your pod is eligible to schedule on based on labels on pods that are already running on the node.

That won't prevent a pod to be scheduled on a node, but at least the pod will be scheduled on the node if and only if the scheduler has no choice.

Solution 2

If you assign a constraint to your pod that can only be met at most once per node, then the scheduler will only be able to place one pod per node. A good example of such a constraint is a host port (the scheduler won't try to put two pods that both require the same host port onto the same node because the second one will never be able to run).

Also see How to require one pod per minion/kublet when configuring a replication controller?

Solution 3

I'm running MC jobs on kubernetes/GCE cluster, and for me M:N scheduling is important in a sense that out of M jobs, I want one job/pod per node for N nodes running (M >> N).

For me solution was to have explicit CPU limit set in pod JSON file

"resources": {
    "limits": {
        "cpu": "700m"
    }
}

ANd I have no replication controller, just pure batch-style cluster. Numbe of nodes N is typically 100-200-300, M is about 10K-20K