Difference between tf.layers.conv2d and tf.contrib.slim.conv2d

Question

I'm trying to convert a network that I'm using from using tf-slim's conv2d to using tf.layers.conv2d, since it looks like tf.layers is the more supported and future-proof option. The function signatures are fairly similar, but is there something algorithmically different between the two? I'm getting different output tensor dimensions than expected.

x = tf.layers.conv2d(inputs=x,
                     filters=256,
                     kernel_size=[3,3],
                     trainable=True)

As opposed to this:

x = slim.conv2d(x, 256, 3)

Solution 1

I'm getting different output tensor dimensions than expected.

This is due to the fact that, by default, slim.conv2d uses same padding whereas tf.layers.conv2d uses valid padding.

If you want to reproduce the exact same behavior, here is the correct implementation:

x = tf.layers.conv2d(x, 256, 3, padding='same')

Solution 2

The description of the tf.slim package sheds a little more light onto the differences: Specifically, if you look under "Layers", the following can be found:

Layers

While the set of TensorFlow operations is quite extensive, developers of neural networks typically think of models in terms of higher level concepts like "layers", "losses", "metrics", and "networks". A layer, such as a Convolutional Layer, a Fully Connected Layer or a BatchNorm Layer are more abstract than a single TensorFlow operation and typically involve several operations. Furthermore, a layer usually (but not always) has variables (tunable parameters) associated with it, unlike more primitive operations. For example, a Convolutional Layer in a neural network is composed of several low level operations:

• Creating the weight and bias variables
• Convolving the weights with the input from the previous layer
• Adding the biases to the result of the convolution.
• Applying an activation function.

Using only plain TensorFlow code, this can be rather laborious:

input = ...
with tf.name_scope('conv1_1') as scope:
  kernel = tf.Variable(tf.truncated_normal([3, 3, 64, 128], dtype=tf.float32,
                                           stddev=1e-1), name='weights')
  conv = tf.nn.conv2d(input, kernel, [1, 1, 1, 1], padding='SAME')
  biases = tf.Variable(tf.constant(0.0, shape=[128], dtype=tf.float32),
                       trainable=True, name='biases')
  bias = tf.nn.bias_add(conv, biases)
  conv1 = tf.nn.relu(bias, name=scope)

To alleviate the need to duplicate this code repeatedly, TF-Slim provides a number of convenient operations defined at the more abstract level of neural network layers. For example, compare the code above to an invocation of the corresponding TF-Slim code:

input = ...
net = slim.conv2d(input, 128, [3, 3], scope='conv1_1')

In short, the slim operators do some neat abstractions for you so that you do not have to worry about all the nitty-gritty details of TensorFlow - a nice addition if you ask me. It seems, though, that this is still under active development, so I would read up a little more on it before actively using it in (future-proof) development.

Comments

[Drew M]

Thanks!! Fixed the issue