Unroll RNN manually in tensorflow 1.0 -
i trying implement: https://arxiv.org/pdf/1511.01432.pdf (semi-supervised sequence learning dai, le)
in paper use rnn 2 parts: encoder rnn , decoder rnn. use unsupervised process weights of rnn initialize weights of supervised task later.
it easy final state , final output of encoder rnn:
def rnn_deep(x,dropout,averaging,shsh): n_input=dimension # permuting batch_size , n_steps x = tf.transpose(x, [1, 0, 2]) # reshaping (n_steps*batch_size, n_input) x = tf.reshape(x, [-1, n_input]) # split list of 'n_steps' tensors of shape (batch_size, n_input) x = tf.split(x, n_steps, 0) x =tf.split(1, n_steps, x) num_layers = 1 lstm_cell =tf.contrib.rnn.grucell(n_hidden) tf.variable_scope("myrnn2") scope: outputs, states = tf.contrib.rnn.static_rnn(lstm_cell, x, dtype=tf.float32) ## outputs list of matrices return outputs[-1] , states[-1],lstm_cell as next step trying last output , last state feed first cell of decoder rnn. not quite sure how this. have tried:
def rnn_deep_decoder(x,last_output,init_state,mycell): state=init_state input_t=last_output lenlen in range(seqlen): output,state = mycell(input_t, state) input_t=output outputs.append(output) is right way proceed? above example gives error : linear expecting 2d arguments: [tensorshape([dimension(none), dimension(25)]), tensorshape([dimension(25)])]
is there better way achieve desired result? make autoencoder train until reaches desired loss.
Comments
Post a Comment