Java set a callback from ExecutorService

Question

I have a fixedThreadPool that I am using to run a bunch of worker threads to achieve parallel execution of a task with many components.

When all threads have finished, I retrieve their results (which are quite large) using a method (getResult) and write them to a file.

Ultimately, to save memory and be able to see intermediate results, I'd like each thread to write its result to the file as soon as it finishes execution and then free its memory.

Ordinarily, I'd add code to that effect to the end of the run() method. However, certain other objects in this class also calls these threads, but DO NOT want them to write their results to file - instead they use their results to perform other calculations, which are eventually written to file.

So, I was wondering if it's possible to attach a callback function to the event of a thread finishing using the ExecutorService. That way, I can immediately retrieve its result and free the memory in that scenario, but not break the code when those threads are used in other scenarios.

Is such a thing possible?

Solution 1

If using Google Guava is an option, you could utilize the ListenableFuture interface in the following manner:

1. Convert an ExecutorService to a ListeningExecutorService via MoreExecutors.listeningDecorator(existingExecutorService)
2. The submit(Callable<V>) method of ListeningExecutorService has been narrowed to return a ListenableFuture, which is a subinterface of Future.
3. ListenableFuture has an addListener() method so you can register a callback to be run when the future is completed.

Solution 2

You can add a callback for when a thread returns in Java 8+ using CompletableFuture as in the following, where t is the result of your long-running computation,

CompletableFuture.supplyAsync(() -> {
    T t = new T();
    // do something
    return t;
}).thenApply(t -> {
    // process t
});

If you want to use callbacks in just Java 7, you could do something like,

int x = 10;
ExecutorService fixedThreadPool = Executors.newFixedThreadPool(x);
Future<T> result = fixedThreadPool.submit(() -> {
    // do calculation
    return T;
});
fixedThreadPool.submit(() -> {
    long minutesToWait = 5;
    T t = null;
    try {
        t = result.get(minutesToWait, TimeUnit.MINUTES);
    } catch (InterruptedException | ExecutionException | TimeoutException e) {
        LOGGER.error(e);
    }
    if (t != null) {
        // process t
    }
});

Solution 3

ExecutorService#submit return FutureTask<T> which helps you to retrieve result and the ExecutorService#get method will block execution until the computation is not completed. Example -

ExecutorService executor = Executors.newFixedThreadPool(10);
Future<Long> future = executor.submit(new Callable<Long>(){
       @Override
       public Long call() throws Exception {
           long sum = 0;
           for (long i = 0; i <= 10000000l; i++) {
               sum += i;
           }
           return sum;
       }
});
Long result = future.get();
System.out.println(result);

Solution 4

So, I was wondering if it's possible to attach a callback function to the event of a thread finishing using the ExecutorService.

Not directly, no, but there are a couple of ways you could accomplish this. The easiest way that comes to mind is to wrap your Runnable in another Runnable that does the reaping of the results.

So you'd do something like:

threadPool.submit(new ResultPrinter(myRunnable));
...

private static class ResultPrinter implements Runnable {
    private final MyRunnable myRunnable;
    public ResultPrinter(MyRunnable myRunnable) {
        this.myRunnable = myRunnable;
    }
    public void run() {
        myRunnable.run();
        Results results = myRunnable.getResults();
        // print results;
    }
}

View more solutions

Comments

[JasonMing]

The myRunnable.getReulst still will block the executing thread, it is not the pure asynchronous mode with "callback".

[Gray]

I assumed that the getResult() call didn't block and that the run() method stored all of the results in your object. Having a get... method block isn't the best pattern @JasonMing.

[Gray]

He specifically did not want to change the run() method as stated in his question.

[Max]

The reason you'd submit the task to a separate thread is so you can avoid blocking the current thread for a long computation, and you can use the current thread for other things. Since future.get() blocks, it should be called in a task which is also submitted to the ExecutorService as well.

[xyman]

Think you've forgot to provide fixedThreadPool to supplyAsync()method in first example.