python - After the training is finished, data remains in the shuffle_batch queue -


when run training code, ends warning

w tensorflow/core/framework/op_kernel.cc:1152] out of range: fifoqueue '_0_tower_0/input_producer' closed , has insufficient elements (requested 1, current size 0)          [[node: tower_0/readerreadv2 = readerreadv2[_device="/job:localhost/replica:0/task:0/cpu:0"](tower_0/tfrecordreaderv2, tower_0/input_producer)]]`

also, shuffle batch log: shows not totally dequeued until end of training.

does knows why program quits before consuming remaining data in shuffle_batch queue?

below code simplified version of training model.

i'm storing images , labels(1000~2000 samples) in single tfrecord file.

so, set flags.datafile = ['some.tfrecord'], flags.max_epoch=1000, , flags.batch_size=350

def read_and_crop(fqueue):     reader = tf.tfrecordreader()     key, data = reader.read(fqueue)     features = tf.parse_single_example(             data,             features={                 'name': tf.fixedlenfeature([], tf.string),                 'image': tf.fixedlenfeature([], tf.string),                 })     decoded_image = tf.image.decode_jpeg(features['image'], channels=3)     return decoded_image, features['name']  def batch_input(flags, min_after_dequeue=3000, num_threads=3):     fqueue = tf.train.string_input_producer(             flags.datafile,             num_epochs=flags.max_epoch,             )     image, name = read(fqueue)     capacity = min_after_dequeue+int(num_threads*1.5)*flags.batch_size     images, names = tf.train.shuffle_batch(             [image, name],             batch_size=flags.batch_size,              capacity=capacity,             min_after_dequeue=min_after_dequeue,             num_threads=num_threads,             allow_smaller_final_batch=true)     return images, names     def train():     tf.graph().as_default():         global_step = tf.train.create_global_step()         images, names = batch_input(flags)         logits = my_model.inference(images)         loss = my_model.loss(logits, names)         train_op = my_model.train(loss)          init = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())         tf.session() sess:             sess.run(init)             coord = tf.train.coordinator()             threads = tf.train.start_queue_runners(sess=sess, coord=coord)             try:                 while not coord.should_stop():                     sess.run(train_op)             except tf.errors.outofrangeerror:                 pass             finally:                 coord.request_stop()             coord.join(threads)


Comments

Post a Comment

Popular posts from this blog

c# - Update a combobox from a presenter (MVP) -

i using mvp in project, new in mvp. have 2 comboboxes. when select option in combobox, combobox should filled new data. this action in presenter. view 'view1' in presenter, , introduced combobox1 , combobox2 properties in 'view1', because need 'datasource', 'displaymember', 'valuemember' , 'refresh()' in method below. but, when using pattern, enough send property like public string combobox2 { { return combobox1.selectedvalue.tosstring(); } } into presenter not whole combobox. how can solve problem? public void onselectedindexchangedcombobox1() { if (view1.combobox1.selectedindex == -1) { return; } datatable dt = tools.getdatatable("a path"); var query = (from o in dt.asenumerable() o.field<string>("afield") == farmerview.combobox1.selectedvalue.tostring() orderby o.field<string>("anotherfield") select...
Read more

How to understand 2 main() functions after using uftrace to profile the C++ program? -

i trying uftrace profile following simple c++ program: #include <iostream> class { public: a() {std::cout << "a created" << std::endl;} ~a() {std::cout << "a destroyed" << std::endl;} }; int main() { a; return 0; } the profile result this: # uftrace a.out created destroyed # duration tid function 2.026 [ 4828] | __cxa_atexit(); [ 4828] | main() { [ 4828] | __static_initialization_and_destruction_0() { 89.397 [ 4828] | std::ios_base::init::init(); 0.768 [ 4828] | __cxa_atexit(); 93.029 [ 4828] | } /* __static_initialization_and_destruction_0 */ 94.425 [ 4828] | } /* main */ [ 4828] | main() { [ 4828] | a::a() { 11.104 [ 4828] | std::operator <<(); 10.825 [ 4828] | std::basic_ostream::operator <<(); 24.514 [ 4828] | } /* a::a */ [ 4828] | a::~a() { 0.978 [ 4828] | ...
Read more

How to put a lock and transaction on table using spring 4 or above using jdbcTemplate and annotations like @Transactional? -

i trying put lock on table while writing on table , if happened in between roll-back . trying convert below code lock table test_g1 read; lock table test_g write; -- begin; start transaction; insert test_g1 values(143); insert test_g values(145); select * test_g1; select * test_g; rollback; select * test_g; unlock tables; how convert above code @transactional spring jdbctemplate code? @transactional(rollbackfor={dataaccessexception.class}) public void test(){ jdbctemplate.execute("insert test1 (id, nam) values (4, 'a')"); throw new dataaccessexception("error") { }; } here trying throw error, insert statement should rollback it's not happening . thanks edit-1 attaching code , doing in jdbcdaoimpl.java , mentioned problem comment above test() . app.java package com.cgiri.javabrains.spring4; import org.springframework.beans.factory.annotation.autowired; import org.springframework.context.applic...
Read more