LoadingCache with async loading

Question

In guava, when using LoadingCache CacheLoader is called synchronously. However, my load() operation may take too long (~1 sec), I want to take a default action in case it takes too long (>200 ms) and load the value asynchronously.

Is there a way to achieve this? Or are there any other approaches you can recommend?

Solution 1

The Caffeine library is a Java 8 rewrite of Guava's cache that allows asynchronous automatic loading of entries into a cache, returning CompletableFutures. It is written by people who were directly involved in creating the Guava cache, and uses a Guava-inspired API (including an adapter to Guava's interfaces).

According to posts on the Guava mailing list, it is based on the original proposal for the Guava caching library, and includes changes that were originally intended for Guava itself, but were not included for various reasons (including Guava's need to be compatible with older versions of Java).

In fact, some projects now consider the Guava cache deprecated and use Caffeine instead, e.g. Spring has switched to Caffeine and the author states that "Caffeine is the Java 8 successor to ConcurrentLinkedHashMap and Guava's cache. Projects should prefer Caffeine and migrate when requiring JDK8 or higher."

Solution 2

You could just do this the normal way: submit a task to get the cache value to an ExecutorService, call get(200, MILLISECONDS) on the Future and do whatever else if that times out.

Example:

final LoadingCache<Key, Result> cache = ...
final Key key = ...
ExecutorService executor = ...

Future<Result> future = executor.submit(new Callable<Result>() {
  @Override public Result call() throws Exception {
    return cache.get(key);
  }
});

try {
  Result result = future.get(200, TimeUnit.MILLISECONDS);
  // got the result; do stuff
} catch (TimeoutException timeout) {
  // timed out; take default action
}

Solution 3

You can check if the value is null with the getIfPresent command. if the value is null then you can submit a task which loads values asynchronously and continue with your flow.

example :

Map<String, String> map = cache.getIfPresent(key);
if(map == null){
 executorService.submit(new Callable<Map<String, String>>() {
    @Override
    public Map<String, String> call() throws Exception {
        return cache.get(key);
    }   
 }); 
}
else{
    continue with the flow...
}

you should also use the refreshAfterWrite feature and implement the reload method if you want to refresh the values in the cache while you are still reading the old value. In this ways the cache will be always updated and the main thread that read the values won't be affected.

example :

cache = CacheBuilder.newBuilder()
    .refreshAfterWrite(30, TimeUnit.SECONDS)
    .build( 
        new CacheLoader<String, Map<String,String>>() {

            public Map<String, String> load(String key) throws Exception {
                map = hardWork(key)//get map from from DB -- expensive time commend
                return map;
            }

            @Override
                public ListenableFuture<Map<String, String>> reload(final String key, Map<String, String> oldValue) throws Exception {
                    // we need to load new values asynchronously, so that calls to read values from the cache don't block
                    ListenableFuture<Map<String, String>> listenableFuture = executorService.submit(new Callable<Map<String, String>>() {

                        @Override
                        public Map<String, String> call() throws Exception {
                            //Async reload event
                            return load(key);
                        }
                    }); 

                    return listenableFuture;
                }
    });

Comments

[aamir]

Can you provide little code snippet to explain the idea. I didn't quite understand it fully

[ColinD]

@AamirKhan: Added a simplified example.