Java Thread Pools/Executor Service and wait()s - what happens to the threads & task queue?

Question

I've looked around but haven't found an answer so I wanted to confirm this for certain.

Say I have a fixed size thread pool - ExecutorService pool = Executors.newFixedThreadPool(5);

And I have some code:

pool.execute(new Runnable(){
    try{
        Object waitForMe = doSomethingAndGetObjectToWaitFor();
        waitForMe.wait();
        doSomethingElse();
    }catch(Exception e){ throw new RunTimeException(e) } 

});

Lets assume that the above code is called a few 100 times. There are only 5 threads in the pool (so only 5 of the above statements should be live at one point). Also assume that the wait() is on an object doing some I/O calls to a thrid party and waiting for a callback when the operation is complete so it will naturally take a while to complete.

Now my question is what is the behavior when one of these tasks reaches a wait(), does the task go to sleep and then the thread from the thread pool takes another task off queue and starts running it?

If the task that is waiting goes to sleep what happens when it gets a notify() and wakes up? Does the thread go back into the queue (at the front or back) for the thread pool and wait until one of the 5 threads can continue to execute it (i.e. call doSomethingelse())? Or does the thread that was executing it also go to sleep i.e. one of the 5 executor threads sits waiting with the task (this is what I'm assuming)? Or does the executor thread pick up another task and simply get interrupted when the first task returns from the wait()?

Solution 1

wait() is a blocking operation:

Causes the current thread to wait until another thread invokes the notify() method or the notifyAll()

This means that the thread in the pool will wait, but from outside it just looks like the current task takes so much time to complete. This also means that if 5 tasks are executed and they all wait(), the Executor cannot handle remaining tasks that, ekhem, wait in the queue.

True, the executor thread itself goes to sleep allowing other threads to switch and consume CPU (so you can have hundreds of threads waiting at the same time and your system is still responsive) but still the thread is "unusable" and blocked.

Another interesting feature is interrupting - if the thread waits for something or sleeps you can interrupt it. Note that both wait() and Thread.sleep() declare InterruptedException. With ExecutorService you can take advantage of this by simply calling: future.cancel() (future is the object you got in return when submit task to ExecutorService).

Finally I think you should redesign your solution. Instead of actively waiting for an external system to finish, provide an API with callbacks:

pool.execute(new Runnable(){
    try{
        doSomethingAndCallMeBackWhenItsDone(new Callback() {
            public void done() {
                doSomethingElse();
            }
        });
    }catch(Exception e){ throw new RunTimeException(e) } 

});

This way the external system's API will simply notify you when the results are ready and you won't have to wait and block ExecutorService. Finally, if doSomethingElse() takes a lot of time, you might even decide to schedule it as well rather than using external third-party I/O thread:

pool.execute(new Runnable(){
    try{
        doSomethingAndCallMeBackWhenItIsDone(new Callback() {
            public void done() {
                pool.submit(new Callbale<Void>() {
                    public Void call() {
                        doSomethingElse();
                    }
                }
            }
        });
    }catch(Exception e){ throw new RunTimeException(e) } 

});

UPDATE: you are asking what to do about timeouts? Here is my idea:

pool.execute(new Runnable(){
    try{
        doSomethingAndCallMeBackWhenItsDone(new Callback() {
            public void done() {
                doSomethingElse();
            }
            public void timeout() {
                //opps!
            }
        });
    }catch(Exception e){ throw new RunTimeException(e) } 

});

I guess you can implement timeout on the third-party side and if timeout occurs there, simply call timeout() method.

Solution 2

The wait() cannot know anything about the tread pool. And the thread pool cannot know anything about the wait(). So they cannot interact anyhow.

They work as usual - the wait() just is a long running blocking operation, the thread pool is just a queue of runnables to run on limited pool of threads.

Comments

[NightWolf]

Thanks for the great answer! With the callback option, what would be the best way to go about setting a timeout (say want to wait x seconds for a callback if nothing throw an error). The only way I can think is to take a note of the current system millisecs and keep that in a list and have another thread monitor the list for calls which have exceeded the current time and trigger an error. I need to find a good book which deals with concurrency, callbacks etc. Thanks again!

[Tomasz Nurkiewicz]

@NightWolf: wrt to timeouts see my updated answer. When it comes to a good book, Java Concurrency in Practice is a must.

[NightWolf]

Thanks heaps for the update and the book link, good idea. Sadly I dont have any control over the third party side.