Kubernetes AutoScaler Not Scaling, HPA shows target <unknown>
Solution 1
As Heapster is deprecated in later version(v 1.13) of kubernetes, You can expose your metrics using metrics-server
also, Please check following answer for step by step instruction to setup HPA:
Solution 2
To answer the question directly, if you have set up an HPA resource and using the metrics server, and ran into this error.
This error means that in your pods, which may have more than one container, either one or both of the containers have not defined resource requests:
Resources:
requests:
cpu: <this is missing! Add it>
Check that all containers have resource requests defined.
To add edit and add resources directly, use
kubectl edit <resource>
Kubernetes may not allow you to update if your using helm even with the force flag.
A note, this may cause downtime if you have not set PodDisruptionBudgets
so set those before you ran your edit
Matt The Ninja
Updated on June 21, 2022Comments
-
Matt The Ninja almost 2 years
I'm pretty new to kubernetes, not so much with docker.
I've been working through the example but I am stuck with the autoscaler, (which doesn't seem to scale).
I am working through the example here https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#step-one-run--expose-php-apache-server
You'll find the build at the bottom
kubectl create -f https://k8s.io/docs/tasks/run-application/hpa-php-apache.yaml
Which looks like this
apiVersion: autoscaling/v1 kind: HorizontalPodAutoscaler metadata: name: php-apache namespace: default spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: php-apache minReplicas: 1 maxReplicas: 10 targetCPUUtilizationPercentage: 50
kubectl get hba
showsNAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE php-apache Deployment/php-apache <unknown>/50% 1 10 0 7s
Clue in the
<unknown>
section.Then
kubectl describe hba
showsName: php-apache Namespace: default Labels: <none> Annotations: <none> CreationTimestamp: Sat, 14 Apr 2018 23:05:05 +0100 Reference: Deployment/php-apache Metrics: ( current / target ) resource cpu on pods (as a percentage of request): <unknown> / 50% Min replicas: 1 Max replicas: 10 Conditions: Type Status Reason Message ---- ------ ------ ------- AbleToScale False FailedGetScale the HPA controller was unable to get the target's current scale: deployments/scale.extensions "php-apache" not found Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedGetScale 12s (x14 over 6m) horizontal-pod-autoscaler deployments/scale.extensions "php-apache" not found
So then I manually add it in with...
kubectl run php-apache --image=k8s.gcr.io/hpa-example --requests=cpu=200m --expose --port=80
Then if i run
kubectl describe hpa
again I get this error..Name: php-apache Namespace: default Labels: <none> Annotations: <none> CreationTimestamp: Sat, 14 Apr 2018 23:05:05 +0100 Reference: Deployment/php-apache Metrics: ( current / target ) resource cpu on pods (as a percentage of request): <unknown> / 50% Min replicas: 1 Max replicas: 10 Conditions: Type Status Reason Message ---- ------ ------ ------- AbleToScale True SucceededGetScale the HPA controller was able to get the target's current scale ScalingActive False FailedGetResourceMetric the HPA was unable to compute the replica count: unable to get metrics for resource cpu: unable to fetch metrics from API: the server could not find the requested resource (get pods.metrics.k8s.io) Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedGetScale 1m (x21 over 11m) horizontal-pod-autoscaler deployments/scale.extensions "php-apache" not found Warning FailedGetResourceMetric 8s (x2 over 38s) horizontal-pod-autoscaler unable to get metrics for resource cpu: unable to fetch metrics from API: the server could not find the requested resource (get pods.metrics.k8s.io) Warning FailedComputeMetricsReplicas 8s (x2 over 38s) horizontal-pod-autoscaler failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from API: the server could not find the requested resource (get pods.metrics.k8s.io)
-
Satish Kumar Nadarajan over 2 yearsShouldn't this be Resource limits and not requests ?