How to benchmark functions in Clojure?

clojure benchmarking

12,222

Solution 1

You might want to look into Hugo Duncan's benchmarking library for Clojure -- Criterium.

From the README:

Criterium measures the computation time of an expression. It is designed to address some of the pitfalls of benchmarking, and benchmarking on the JVM in particular.

This includes:

statistical processing of multiple evaluations

inclusion of a warm-up period, designed to allow the JIT compiler to optimise its code

purging of gc before testing, to isolate timings from GC state prior to testing

a final forced GC after testing to estimate impact of cleanup on the timing results

Solution 2

If it's just a matter of wanting to capture the string programmatically, you can bind *out* to something else before using time.

user=> (def x (with-out-str (time (+ 2 2))))
#'user/x
user=> x
"\"Elapsed time: 0.119 msecs\"\n"

If you want more control over the format, then you can create your own version of time using Java's System time methods, which is what the time macro uses under the hood anyway:

user => (macroexpand '(time (+ 2 2)))
(let* [start__4197__auto__ (. java.lang.System (clojure.core/nanoTime))  
       ret__4198__auto__ (+ 2 2)] 
     (clojure.core/prn (clojure.core/str "Elapsed time: " (clojure.core//     
          (clojure.core/double 
               (clojure.core/- (. java.lang.System (clojure.core/nanoTime)) 
                                start__4197__auto__)) 1000000.0) " msecs"))
 ret__4198__auto__)

Take that basic structure and replace the call to prn with whatever reporting mechanism you would prefer.

12,222

Author by

Peter Tillemans

I am a father of 3 kids with too many interests and not enough time. I have been a pretty decent Perl guy in the past, but now my interests lie mostly on the JVM. I've been an IT guy in a semiconductor company since the early 90's.

Updated on June 06, 2022

Comments

Peter Tillemans almost 2 years

I know I can get the time take to evaluate a function can be printed out on the screen/stdout using the time function/macro.

The time macro returns the value of the evaluated function, which makes it great to use it inline. However I want to automatically measure the runtime under specific circumstances.

Is there a function which returns the elapsed time in some library to help with this benchmarking?
Peter Tillemans almost 14 years

Thanks, I was looking at this to start from too. The criterium library fits really the bill. But unless you know the Haskell library you're not going to stumble over it by accident using google.
Tim Clemons over 10 years

@PeterTillemans Well, now we are, given that it's mentioned in StackOverflow. :-)