Event loop in java

Question

I'm actually trying to write an event loop for Nashorn (java 8) so that callbacks from asynchronous operations (threads i launch to, for example, connect to remote services or do long-running calculations) will be put in a queue and executed in sequence (not in parallel). I'm doing it by placing the callback functions on a ConcurrentLinkedQueue and using a ScheduledExecutorService as the loop that checks the queue for a callback to execute.

Works fine, but my questions are:

1) how short an interval can I use without dragging my CPU? I will have multiple of these running and they must be independent of one another. Thus there may be 50 threads all running their own "event loop". This executor attempts to run my runnable every 10ms, for example....

Executors.newSingleThreadScheduledExecutor().scheduleAtFixedRate(<cbRunnable>, 0, 10, TimeUnit.MILLISECONDS);

2) Does this approach have an advantage over:

while (true) {
   // check queue and execute any callback...
}

3) Is there a better way?

Solution

The answer totally depends on what you're doing inside this block:

while (true) {
    // check queue and execute any callback...
}

If the queue check blocks until an element is available, then this is the "most efficient" way to poll, in terms of CPU usage. If the check does not block, then the calling thread will spin and you'll be running one hardware thread at full capacity for the duration of the loop - however, this eliminates synchronization costs and will give you the absolute best response time. Here are some examples:

Blocking Queue (least taxing on the CPU, but comes at the cost of synchronization)

Queue<T> q = new LinkedBlockingQueue<>();
...
while(true){
    T t = q.take();
    //t will never be null, do something with it here.
}

Non-blocking Queue (most taxing on the CPU, but no synchronization costs)

Queue<T> q = new LinkedList<>();
...
while(true){
    T t;
    while((t = q.poll()) == null); //busy polling!
    //t will never be null, do something with it here
}

ScheduledExecutorService

Finally ... if you end up using the scheduled executor service, you're forcing some non-zero wait time between polls. This will be almost as efficient CPU-wise when compared to a full-on BlockingQueue implementation, but you're also forced to take a hit on response time, up to the scheduling interval. What is "best" for your application will depend on whether or not you can afford to wait the minimum sleep time between polls (I think you can schedule down the microsecond ...?), or if you need something faster like a busy polling scheme.

Comments

[mister blinky]

thanks. Accepted your answer. I wonder if you (or anyone) has any info about what the minimal delay would be for the ScheduledExecutorService option. I would assume at some point the work of the wake-up/wait switching would affect the efficiency on the CPU that would be gained using the ScheduledExecutorService. I wonder what that delay threshold is -- 100ms, 10ms, 1ms?

[Tamir Daniely]

There are optimal implementations of the fifo blocking queue that use a clever combination of spin waits and OS wait handles (no polling) and have practically zero overhead. I optimistically assume that SingleThreadExecutor is actually using something similar. An optimal SingleThreadScheduledExecutor would use a similar approach, starting with a spin wait for intervals that approach the context switch duration, but then instead of waiting for an item to enter the queue it would wait for an OS interrupt event from the high performance timer api.