Thursday, November 15, 2018

[RNN] What are the difference of input and output's tensor shape in dynamic_rnn and static_rnn using TensorFlow

When studying RNN, my first issue encountered in my program is about the shape of input and output tensors. Shape is a very important information to connect between layers. Here I just directly point out what are differences in input/output shape of static RNN and dynamic RNN.
P.S: If you use Keras to write your RNN model, you won't need to deal with these details.



The short example of Static RNN

Please pay a tension about the output shape in the following picture.
batch_size = 32
time_step = 5
input_size = 4
rnn_cell = 20
X = tf.placeholder(tf.float32, shape=[batch_size, time_step, input_size])
x=tf.unstack(X,axis=1)
lstm_cell = rnn.BasicLSTMCell(rnn_cell)
outputs, states = rnn.static_rnn(lstm_cell, x, dtype=tf.float32)
output=outputs[-1]









The short example of Dynamic RNN

Please pay a tension about the output shape in the following picture.

batch_size = 32
time_step = 5
input_size = 4
rnn_cell = 20
X = tf.placeholder(tf.float32, shape=[batch_size, time_step, input_size])
lstm_cell = rnn.BasicLSTMCell(rnn_cell)
outputs, states = rnn.static_rnn(lstm_cell, x, dtype=tf.float32)
outputs=tf.transpose(outputs, [1, 0, 2])
output=outputs[-1]




RNN API:
https://www.tensorflow.org/api_docs/python/tf/nn/static_rnn
https://www.tensorflow.org/api_docs/python/tf/nn/dynamic_rnn

No comments: