TensorFlow: Max of a tensor along an axis

Question

My question is in two connected parts:

1. How do I calculate the max along a certain axis of a tensor? For example, if I have

   x = tf.constant([[1,220,55],[4,3,-1]])
   

   I want something like

   x_max = tf.max(x, axis=1)
   print sess.run(x_max)
   
   output: [220,4]
   

   I know there is a tf.argmax and a tf.maximum, but neither give the maximum value along an axis of a single tensor. For now I have a workaround:

   x_max = tf.slice(x, begin=[0,0], size=[-1,1])
   for a in range(1,2):
       x_max = tf.maximum(x_max , tf.slice(x, begin=[0,a], size=[-1,1]))
   

   But it looks less than optimal. Is there a better way to do this?

2. Given the indices of an argmax of a tensor, how do I index into another tensor using those indices? Using the example of x above, how do I do something like the following:

   ind_max = tf.argmax(x, dimension=1)    #output is [1,0]
   y = tf.constant([[1,2,3], [6,5,4])
   y_ = y[:, ind_max]                     #y_ should be [2,6]
   

   I know slicing, like the last line, does not exist in TensorFlow yet (#206).

   My question is: what is the best workaround for my specific case (maybe using other methods like gather, select, etc.)?

   Additional information: I know x and y are going to be two dimensional tensors only!

Solution 1

The tf.reduce_max() operator provides exactly this functionality. By default it computes the global maximum of the given tensor, but you can specify a list of reduction_indices, which has the same meaning as axis in NumPy. To complete your example:

x = tf.constant([[1, 220, 55], [4, 3, -1]])
x_max = tf.reduce_max(x, reduction_indices=[1])
print sess.run(x_max)  # ==> "array([220,   4], dtype=int32)"

If you compute the argmax using tf.argmax(), you could obtain the the values from a different tensor y by flattening y using tf.reshape(), converting the argmax indices into vector indices as follows, and using tf.gather() to extract the appropriate values:

ind_max = tf.argmax(x, dimension=1)
y = tf.constant([[1, 2, 3], [6, 5, 4]])

flat_y = tf.reshape(y, [-1])  # Reshape to a vector.

# N.B. Handles 2-D case only.
flat_ind_max = ind_max + tf.cast(tf.range(tf.shape(y)[0]) * tf.shape(y)[1], tf.int64)

y_ = tf.gather(flat_y, flat_ind_max)

print sess.run(y_) # ==> "array([2, 6], dtype=int32)"

Solution 2

As of TensorFlow 1.10.0-dev20180626, tf.reduce_max accepts axis and keepdims keyword arguments offering the similar functionality of numpy.max.

In [55]: x = tf.constant([[1,220,55],[4,3,-1]])

In [56]: tf.reduce_max(x, axis=1).eval() 
Out[56]: array([220,   4], dtype=int32)

To have a resultant tensor of the same dimension as the input tensor, use keepdims=True

In [57]: tf.reduce_max(x, axis=1, keepdims=True).eval()Out[57]: 
array([[220],
       [  4]], dtype=int32)

---

If the axis argument is not explicitly specified then the tensor level maximum element is returned (i.e. all axes are reduced).

In [58]: tf.reduce_max(x).eval()
Out[58]: 220