java - Hadoop Text Object toString() Split Problems -


my hadoop mapper emitting csv line line text object follows:

public void map(object key, text value, context context) throws ioexception, interruptedexception {      string datarows[] = value.tostring().trim().split("\\r?\\n");      (int = 0; < datarows.length; i++) {         random r = new random();         int partition = r.nextint(3);         hc.set(integer.tostring(partition));         data.set(datarows[i]);         context.write(hc, data);     } }

within reducer, need split csv , use strings further action. here reducer code:

public static class intsumreducer extends reducer<text, text, text, text> {     private text data = new text();      public void reduce(text key, iterable<text> values, context context) throws ioexception, interruptedexception {         list<coord> coords = new arraylist<coord>();         iterator<text> iter = values.iterator();         while (iter.hasnext()) {             string[] elems = iter.next().tostring().split(",");             double[] x = new double[elems.length];             try {                 (int = 0; < x.length; i++) {                     x[i] = integer.parseint(elems[i]);                 }             } catch (exception e) {                 continue;             }             coords.add(new coord(x));         }         try {             cluster cluster = runclusterer(coords);             data.set(cluster.tonewick()+";");             context.write(key, data);         } catch (ioexception e) {             e.printstacktrace();         }     } }

here strange thing, elems string array has length of 1 , contains left elements of every line of csv.

for e.g - suppose csv contains 2 rows. first row - {1,2} , second row {3,4}

the elems array getting populated {1,3}.

any appreciated.


Comments

Post a Comment

Popular posts from this blog

How to understand 2 main() functions after using uftrace to profile the C++ program? -

i trying uftrace profile following simple c++ program: #include <iostream> class { public: a() {std::cout << "a created" << std::endl;} ~a() {std::cout << "a destroyed" << std::endl;} }; int main() { a; return 0; } the profile result this: # uftrace a.out created destroyed # duration tid function 2.026 [ 4828] | __cxa_atexit(); [ 4828] | main() { [ 4828] | __static_initialization_and_destruction_0() { 89.397 [ 4828] | std::ios_base::init::init(); 0.768 [ 4828] | __cxa_atexit(); 93.029 [ 4828] | } /* __static_initialization_and_destruction_0 */ 94.425 [ 4828] | } /* main */ [ 4828] | main() { [ 4828] | a::a() { 11.104 [ 4828] | std::operator <<(); 10.825 [ 4828] | std::basic_ostream::operator <<(); 24.514 [ 4828] | } /* a::a */ [ 4828] | a::~a() { 0.978 [ 4828] | ...
Read more

c# - Update a combobox from a presenter (MVP) -

i using mvp in project, new in mvp. have 2 comboboxes. when select option in combobox, combobox should filled new data. this action in presenter. view 'view1' in presenter, , introduced combobox1 , combobox2 properties in 'view1', because need 'datasource', 'displaymember', 'valuemember' , 'refresh()' in method below. but, when using pattern, enough send property like public string combobox2 { { return combobox1.selectedvalue.tosstring(); } } into presenter not whole combobox. how can solve problem? public void onselectedindexchangedcombobox1() { if (view1.combobox1.selectedindex == -1) { return; } datatable dt = tools.getdatatable("a path"); var query = (from o in dt.asenumerable() o.field<string>("afield") == farmerview.combobox1.selectedvalue.tostring() orderby o.field<string>("anotherfield") select...
Read more

How to put a lock and transaction on table using spring 4 or above using jdbcTemplate and annotations like @Transactional? -

i trying put lock on table while writing on table , if happened in between roll-back . trying convert below code lock table test_g1 read; lock table test_g write; -- begin; start transaction; insert test_g1 values(143); insert test_g values(145); select * test_g1; select * test_g; rollback; select * test_g; unlock tables; how convert above code @transactional spring jdbctemplate code? @transactional(rollbackfor={dataaccessexception.class}) public void test(){ jdbctemplate.execute("insert test1 (id, nam) values (4, 'a')"); throw new dataaccessexception("error") { }; } here trying throw error, insert statement should rollback it's not happening . thanks edit-1 attaching code , doing in jdbcdaoimpl.java , mentioned problem comment above test() . app.java package com.cgiri.javabrains.spring4; import org.springframework.beans.factory.annotation.autowired; import org.springframework.context.applic...
Read more