Tensorflow: Replacement for tf.nn.rnn_cell._linear(input, size, 0, scope)
Solution 1
I met this error while using SkFlow's TensorFlowDNNRegressor. The first time I saw the answer of ruoho ruots, I am a bit confused. But the next day I realized what he meant.
Here is what I do:
from tensorflow.python.ops import rnn_cell_impl
replace tf.nn.rnn_cell._linear
with rnn_cell_impl._linear
Solution 2
With version 1.0, stuff has moved all around. I've had similar hunts updating tf.nn.rnn_cell.LSTMCell
to tf.contrib.rnn.BasicLSTMCell
.
For your case tf.nn.rnn_cell._linear
now lives in tf.contrib.rnn.python.ops.core_rnn_cell_impl
as well as the definition of the BasicRNNCell
. Checking the BasicRNNCell docs and source code, we see at L113-L118 the use of _linear.
def __call__(self, inputs, state, scope=None):
"""Most basic RNN: output = new_state = act(W * input + U * state + B)."""
with _checked_scope(self, scope or "basic_rnn_cell", reuse=self._reuse):
output = self._activation(
_linear([inputs, state], self._num_units, True))
return output, output
the _linear method is defined at line 854 as a:
Linear map: sum_i(args[i] * W[i]), where W[i] is a variable.
Good luck!
Solution 3
The answer of ruoho ruotsi is almost correct:
Yet, the definition of linear
is not located in tf.contrib.rnn.basicRNNCell
, but in tf.contrib.rnn.python.ops.rnn_cell
, or tf.contrib.rnn.python.ops.core_rnn_cell_impl
, respectively.
You can find their source code here and here.
Solution 4
To solving this problem,we can define a linear() function.
def linear(input_, output_size, scope=None):
'''
Linear map: output[k] = sum_i(Matrix[k, i] * args[i] ) + Bias[k]
Args:
args: a tensor or a list of 2D, batch x n, Tensors.
output_size: int, second dimension of W[i].
scope: VariableScope for the created subgraph; defaults to "Linear".
Returns:
A 2D Tensor with shape [batch x output_size] equal to
sum_i(args[i] * W[i]), where W[i]s are newly created matrices.
Raises:
ValueError: if some of the arguments has unspecified or wrong shape.
'''
shape = input_.get_shape().as_list()
if len(shape) != 2:
raise ValueError("Linear is expecting 2D arguments: %s" % str(shape))
if not shape[1]:
raise ValueError("Linear expects shape[1] of arguments: %s" % str(shape))
input_size = shape[1]
# Now the computation.
with tf.variable_scope(scope or "SimpleLinear"):
matrix = tf.get_variable("Matrix", [output_size, input_size], dtype=input_.dtype)
bias_term = tf.get_variable("Bias", [output_size], dtype=input_.dtype)
return tf.matmul(input_, tf.transpose(matrix)) + bias_term
def highway(input_, size, num_layers=1, bias=-2.0, f=tf.nn.relu, scope='Highway'):
"""Highway Network (cf. http://arxiv.org/abs/1505.00387).
t = sigmoid(Wy + b)
z = t * g(Wy + b) + (1 - t) * y
where g is nonlinearity, t is transform gate, and (1 - t) is carry gate.
"""
with tf.variable_scope(scope):
for idx in range(num_layers):
g = f(linear(input_, size, scope='highway_lin_%d' % idx))
t = tf.sigmoid(linear(input_, size, scope='highway_gate_%d' % idx) + bias)
output = t * g + (1. - t) * input_
input_ = output
return output
Solution 5
tensorflow.python.ops.rnn_cell_impl._linear now is at tensorflow.contrib.rnn.python.ops.core_rnn_cell._linear. And I prefer to use tf.layers.Dense to replace. for example, change
from tensorflow.contrib.rnn.python.ops import core_rnn_cell
core_rnn_cell._linear(states, length, bias=True)
to
tf.layers.Dense(units=length)(states)
I'm using tensorflow 1.6.
Chris Stenkamp
Updated on July 05, 2022Comments
-
Chris Stenkamp almost 2 years
I am trying to get the SequenceGAN (https://github.com/LantaoYu/SeqGAN) from https://arxiv.org/pdf/1609.05473.pdf to run.
After fixing the obvious errors, like replacingpack
withstack
, it still doesn't run, since the highway-network part requires thetf.nn.rnn_cell._linear
function:# highway layer that borrowed from https://github.com/carpedm20/lstm-char-cnn-tensorflow def highway(input_, size, layer_size=1, bias=-2, f=tf.nn.relu): """Highway Network (cf. http://arxiv.org/abs/1505.00387). t = sigmoid(Wy + b) z = t * g(Wy + b) + (1 - t) * y where g is nonlinearity, t is transform gate, and (1 - t) is carry gate. """ output = input_ for idx in range(layer_size): output = f(tf.nn.rnn_cell._linear(output, size, 0, scope='output_lin_%d' % idx)) #tf.contrib.layers.linear instad doesn't work either. transform_gate = tf.sigmoid(tf.nn.rnn_cell._linear(input_, size, 0, scope='transform_lin_%d' % idx) + bias) carry_gate = 1. - transform_gate output = transform_gate * output + carry_gate * input_ return output
the
tf.nn.rnn_cell._linear
function doesn't appear to be there anymore in Tensorflow 1.0 or 0.12, and I have no clue what to replace it with. I can't find any new implementations of this, or any information on tensorflow's github or (unfortunately very sparse) documentation.Does anybody know the new pendant of the function? Thanks a lot in advance!
-
Majid Alfifi about 7 yearsI tried this solution on v 1.0 but got
AttributeError: type object 'BasicRNNCell' has no attribute '_linear'
-
dennlinger about 7 yearsIt is actually defined in another file. Please see my answer for the correct code location.
-
ruoho ruotsi about 7 yearsI referenced the
tf.contrib.rnn.python.ops.core_rnn_cell_impl
but you're right, I wasn't clear that theBasicRNNCell
andlinear
both live in this file. github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/… Good catch. I'll update my Answer. -
Escachator over 6 yearsNot in version 1.2
-
dennlinger over 6 yearsThis was valid for versions 1.0 and 1.1 only. They (at that time) said they would move the functions to other places in a later release, which is probably what happened in 1.2
-
Thava almost 4 yearsFor tensorflow version 1.15 this is what worked for me!