python - Dynamically adding elements to a NumPy array of unknown final length -


i want create large vector of numbers, length of vector not known. however, give maximum length (which around 100k), although list around 10k. essentially, have loop, within keep adding numbers vector, until criterion met.

my first attempt @ doing python lists, used following:

x = [] in range(k):     y = get_list_of_numbers()     x += y

however, want convert list numpy array, further processing. if using a = np.array(x), takes long time create array.

so, second solution create empty numpy array beginning, , add elements go along:

x = np.empty([]) in range(k):     y = get_list_of_numbers()     np.append(x, y)

however, here np.append(x, y) takes long time process.

so, both solutions slow. there quicker solution out there?

the remaining solution can think of create huge numpy array @ maximum length, , insert each element appropriate slot in array. however, memory inefficient, not have estimate of maximum vector length...

thanks!

if define:

def get_list_of_numbers():     n = np.random.randint(0,10)     return list(range(n))  def foo(k):     x=[]     in range(k):         y = get_list_of_numbers()         x.extend(y)     return x

simply calling get_list_of_numbers takes time. turning result array doesn't take time:

in [69]: timeit foo(1000) 100 loops, best of 3: 5.9 ms per loop in [70]: timeit np.array(foo(1000)) 100 loops, best of 3: 6.38 ms per loop in [73]: timeit -n1000 get_list_of_numbers() 1000 loops, best of 3: 6.04 µs per loop

lets try preallocation approach:

def foo1(k):     x = np.zeros(k*10,int)     cnt = 0     in range(k):         y = get_list_of_numbers()         n = len(y)         x[cnt:cnt+n] = y         cnt += n     x = x[:cnt]     return x  in [80]: timeit foo1(1000) 100 loops, best of 3: 10.1 ms per loop

the array concatenation approach

in [48]: def foo1(k):     ...:     x = np.zeros(0,int)     ...:     in range(k):     ...:         y = get_list_of_numbers()     ...:         x = np.concatenate((x, y), axis=0)     ...:     return x in [51]: timeit foo1(1000).shape 100 loops, best of 3: 15.9 ms per loop

Comments

Post a Comment

Popular posts from this blog

c# - Update a combobox from a presenter (MVP) -

i using mvp in project, new in mvp. have 2 comboboxes. when select option in combobox, combobox should filled new data. this action in presenter. view 'view1' in presenter, , introduced combobox1 , combobox2 properties in 'view1', because need 'datasource', 'displaymember', 'valuemember' , 'refresh()' in method below. but, when using pattern, enough send property like public string combobox2 { { return combobox1.selectedvalue.tosstring(); } } into presenter not whole combobox. how can solve problem? public void onselectedindexchangedcombobox1() { if (view1.combobox1.selectedindex == -1) { return; } datatable dt = tools.getdatatable("a path"); var query = (from o in dt.asenumerable() o.field<string>("afield") == farmerview.combobox1.selectedvalue.tostring() orderby o.field<string>("anotherfield") select...
Read more

How to understand 2 main() functions after using uftrace to profile the C++ program? -

i trying uftrace profile following simple c++ program: #include <iostream> class { public: a() {std::cout << "a created" << std::endl;} ~a() {std::cout << "a destroyed" << std::endl;} }; int main() { a; return 0; } the profile result this: # uftrace a.out created destroyed # duration tid function 2.026 [ 4828] | __cxa_atexit(); [ 4828] | main() { [ 4828] | __static_initialization_and_destruction_0() { 89.397 [ 4828] | std::ios_base::init::init(); 0.768 [ 4828] | __cxa_atexit(); 93.029 [ 4828] | } /* __static_initialization_and_destruction_0 */ 94.425 [ 4828] | } /* main */ [ 4828] | main() { [ 4828] | a::a() { 11.104 [ 4828] | std::operator <<(); 10.825 [ 4828] | std::basic_ostream::operator <<(); 24.514 [ 4828] | } /* a::a */ [ 4828] | a::~a() { 0.978 [ 4828] | ...
Read more

How to put a lock and transaction on table using spring 4 or above using jdbcTemplate and annotations like @Transactional? -

i trying put lock on table while writing on table , if happened in between roll-back . trying convert below code lock table test_g1 read; lock table test_g write; -- begin; start transaction; insert test_g1 values(143); insert test_g values(145); select * test_g1; select * test_g; rollback; select * test_g; unlock tables; how convert above code @transactional spring jdbctemplate code? @transactional(rollbackfor={dataaccessexception.class}) public void test(){ jdbctemplate.execute("insert test1 (id, nam) values (4, 'a')"); throw new dataaccessexception("error") { }; } here trying throw error, insert statement should rollback it's not happening . thanks edit-1 attaching code , doing in jdbcdaoimpl.java , mentioned problem comment above test() . app.java package com.cgiri.javabrains.spring4; import org.springframework.beans.factory.annotation.autowired; import org.springframework.context.applic...
Read more