python - Django: Extremely Large Queryset Slow to Execute (non-linear retrieval time) -


i having django (oracle backend, cx_oracle 5.2.1) execute query simple

   select x.y, x.z x x.a = 'foo' , x.b = 'bar'

both x.a , x.b indexed (db_index=true)

the page use calculates summary statistics on various records using pandas, data sizes quite large.

i have 1 page loading subset of ~15000 records. takes ~2 seconds load

the exact same page, when using different timeframe, using same type of query loads ~50000 records. takes ~30 seconds load. i've benchmarked performance, , pandas not bottleneck.

the simple

my_queryset.all().values(('y', 'z'))

takes overwhelming majority of time (~90%)

is there reason why is, or way speed up?

of note, when executing these queries in database console, both fast. seems django orm has issues.


Comments

Post a Comment

Popular posts from this blog

'hasOwnProperty' in javascript -

i use obj.hasownproperty judge whether object has property, when replaced obj[prop] !== undefined , not normal implementation, ask, why behind method can not use it? object.hasownproperty(prop); object[prop] !== undefined; obj[prop] !== undefined wrong 2 reasons: you can explicitly set property undefined , obj[prop] = undefined; . obj.hasownproperty(prop) return true in case. obj[prop] follow prototype chain, return property that's inherited. obj.hasownproperty(prop) returns true if property exists directly in object, returns false inherited properties.
Read more

How to put a lock and transaction on table using spring 4 or above using jdbcTemplate and annotations like @Transactional? -

i trying put lock on table while writing on table , if happened in between roll-back . trying convert below code lock table test_g1 read; lock table test_g write; -- begin; start transaction; insert test_g1 values(143); insert test_g values(145); select * test_g1; select * test_g; rollback; select * test_g; unlock tables; how convert above code @transactional spring jdbctemplate code? @transactional(rollbackfor={dataaccessexception.class}) public void test(){ jdbctemplate.execute("insert test1 (id, nam) values (4, 'a')"); throw new dataaccessexception("error") { }; } here trying throw error, insert statement should rollback it's not happening . thanks edit-1 attaching code , doing in jdbcdaoimpl.java , mentioned problem comment above test() . app.java package com.cgiri.javabrains.spring4; import org.springframework.beans.factory.annotation.autowired; import org.springframework.context.applic...
Read more

How to understand 2 main() functions after using uftrace to profile the C++ program? -

i trying uftrace profile following simple c++ program: #include <iostream> class { public: a() {std::cout << "a created" << std::endl;} ~a() {std::cout << "a destroyed" << std::endl;} }; int main() { a; return 0; } the profile result this: # uftrace a.out created destroyed # duration tid function 2.026 [ 4828] | __cxa_atexit(); [ 4828] | main() { [ 4828] | __static_initialization_and_destruction_0() { 89.397 [ 4828] | std::ios_base::init::init(); 0.768 [ 4828] | __cxa_atexit(); 93.029 [ 4828] | } /* __static_initialization_and_destruction_0 */ 94.425 [ 4828] | } /* main */ [ 4828] | main() { [ 4828] | a::a() { 11.104 [ 4828] | std::operator <<(); 10.825 [ 4828] | std::basic_ostream::operator <<(); 24.514 [ 4828] | } /* a::a */ [ 4828] | a::~a() { 0.978 [ 4828] | ...
Read more