how to calculate a net's FLOPs in CNN

Question

I want to design a convolutional neural network which occupy GPU resource no more than Alexnet.I want to use FLOPs to measure it but I don't know how to calculate it.Is there any tools to do it,please?

Solution 1

For online tool see http://dgschwend.github.io/netscope/#/editor . For alexnet see http://dgschwend.github.io/netscope/#/preset/alexnet . This supports most wide known layers. For custom layers you will have to calculate yourself.

Solution 2

For future visitors, if you use Keras and TensorFlow as Backend then you can try the following example. It calculates the FLOPs for the MobileNet.

import tensorflow as tf
import keras.backend as K
from keras.applications.mobilenet import MobileNet

run_meta = tf.RunMetadata()
with tf.Session(graph=tf.Graph()) as sess:
    K.set_session(sess)
    net = MobileNet(alpha=.75, input_tensor=tf.placeholder('float32', shape=(1,32,32,3)))

    opts = tf.profiler.ProfileOptionBuilder.float_operation()    
    flops = tf.profiler.profile(sess.graph, run_meta=run_meta, cmd='op', options=opts)

    opts = tf.profiler.ProfileOptionBuilder.trainable_variables_parameter()    
    params = tf.profiler.profile(sess.graph, run_meta=run_meta, cmd='op', options=opts)

    print("{:,} --- {:,}".format(flops.total_float_ops, params.total_parameters))

Solution 3

If you're using Keras, you could just use the patch in this pull request: https://github.com/fchollet/keras/pull/6203

Then just call print_summary() and you'll see both the flops per layer and the total.

Even if not using Keras, it may be worth it to recreate your nets in Keras just so you can get the flops counts.