Hystrix command fails with "timed-out and no fallback available"

Question

I've noticed that some of the commands in my application fail with

Caused by: ! com.netflix.hystrix.exception.HystrixRuntimeException: GetAPICommand timed-out and no fallback available.
out: ! at com.netflix.hystrix.HystrixCommand.getFallbackOrThrowException(HystrixCommand.java:1631)
out: ! at com.netflix.hystrix.HystrixCommand.access$2000(HystrixCommand.java:97)
out: ! at com.netflix.hystrix.HystrixCommand$TimeoutObservable$1$1.tick(HystrixCommand.java:1025)
out: ! at com.netflix.hystrix.HystrixCommand$1.performBlockingGetWithTimeout(HystrixCommand.java:621)
out: ! at com.netflix.hystrix.HystrixCommand$1.get(HystrixCommand.java:516)
out: ! at com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:425)
out: Caused by: ! java.util.concurrent.TimeoutException: null
out: !... 11 common frames omitted

This is my Hystrix configuration override:

hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds=210000
hystrix.threadpool.default.coreSize=50
hystrix.threadpool.default.maxQueueSize=100
hystrix.threadpool.default.queueSizeRejectionThreshold=50

What kind of timeout is this? Is it a read/connection timeout to the external application? How do I go about debugging this?

Solution 1

This is a Hystrix Command Timeout, this timeout is enabled by default per each command, you define the value using the property:

execution.isolation.thread.timeoutInMilliseconds: This property sets the time in milliseconds after which the caller will observe a timeout and walk away from the command execution. Hystrix marks > the HystrixCommand as a TIMEOUT, and performs fallback logic.

So you can increase your timeout value or disable the default time out (if apply in your case) for your command using the property:

@HystrixProperty(name = "hystrix.command.default.execution.timeout.enabled", value = "false")

You can find more information here: https://github.com/Netflix/Hystrix/wiki/Configuration#CommandExecution

Solution 2

It might be you are in debug or your connection too slow, default thread execution timeout is only 1 second, so you could get this message easily if you put a break-point in your command let's say

Although this is not your case but might help somebody else

Solution 3

Looking at the stacktrace this is an exception thrown by Hystrix after the 210 seconds you defined above.

As TimeoutException is a checked exception that needs to be declared on each method that could throw this exception. You would see this declared in the run() method of your code.

You can debug this like any other program, but be aware that the run() method runs in a thread separate from the caller. After 210 seconds the caller will just continue despite your debugging session.

Comments

[dkulkarni]

My run() method is just a GET HTTP call that looks like this. This is a jersey client. ClientResponse response = client.resource(buildUri()) .get(ClientResponse.class);

[dkulkarni]

This is my code. Like I said it is a jersey client<code>@Override protected FetchProvisionedIdsResponse run() throws Exception { ClientResponse response = client.resource(buildUri()) .get(ClientResponse.class); return response.getEntity(MyResponse.class);</code> }

[ahus1]

You have declared Exception in the method, a good style is to narrow it down. From the API description I see that get() will throw a UniformInterfaceException, but no TimeoutException. Therefore it's Hystrix throwing this exception.

[Robin Jonsson]

What happens if you catch HystrixTimeoutException and return your own kind of exception? Will the circuit breaker still work? Or does the exception need to propagate somewhere?

[Simon Forsberg]

@RobinJonsson The exception is called HystrixRuntimeException and it's the same exception for timeout as for if the command threw another exception.

[abhi]

This is what i experienced!

[reza ramezani matin]

Where is it to config @HystrixProperty? Are there zuul?