Detect last record in standard input (stdin) in python -


i reading input through stdin (hadoop streaming in reducer).

i need detect when last record comes in. running loop on stdin data.

i tried read stdin first calculate total records , again read proceed business processing, read record stdin calculate total_cnt records goes out stream , later when try read stdin processing there no record in stdin.

total_cnt = 0   line in stdin:       total cnt += 1  line in stdin:      ##some processing## 

i don't want store stdin somewhere , read data location twice (1. total record count , 2. data processing).

is there way can detect when last record comes in stdin?

i using python version 2.7.11 , need implement in approach in hadoop reducer.

process previous line each time take in new one. when loop exits, line have last, unprocessed line please.

example:

old_line = none line in range(10):     if old_line none:         old_line = line         continue  # skip processing on first loop: we'll make after     print "do stuff with: %i" % old_line     old_line = line print "double last line: %i" % (line*2) 

which gives:

do stuff with: 0 stuff with: 1 stuff with: 2 stuff with: 3 stuff with: 4 stuff with: 5 stuff with: 6 stuff with: 7 stuff with: 8 double last line: 18 

Comments

Popular posts from this blog

javascript - Confirm a form & display message if form is valid with JQuery -

Retrieving ETA (estimated time of arrival) with Google Distance Matrix API and public transit as transport mode -

ionic framework - Meteor - Error: Failed to execute 'insertBefore' on 'Node' -