node.js - Why are MongoDB upserts so much slower than inserts (with a unique index)? -
i've been testing limits of mongodb see whether work upcoming project , i've noticed upserts quite slow compared inserts.
of course, i'd expect them slower, not (almost) order of magnitude slower (7400 vs 55000 ops/sec). here's (nodejs native driver) bench-marking code used:
(async function() { let db = await require('mongodb').mongoclient.connect('mongodb://localhost:27017/mongo-benchmark-8764824692947'); db.collection('text').createindex({text:1},{unique:true}) let batch = db.collection('text').initializeorderedbulkop(); let totalopcount = 0; let batchopcount = 0; let start = date.now(); while(1) { totalopcount++; batchopcount++; if(batchopcount === 1000) { // batch 1000 ops @ time await batch.execute(); batch = db.collection('text').initializeorderedbulkop(); batchopcount = 0; let secondselapsed = (date.now() - start)/1000; console.log(`(${math.round(totalopcount/secondselapsed)} ops per sec) (${totalopcount} total ops)`) } ///////// insert test ///////// (~55000 ops/sec) // batch.insert({text:totalopcount}); ///////// upsert test ///////// (~7400 ops/sec) let text = math.floor(math.random()*1000000); batch.find({text}).upsert().updateone({$setoninsert:{text}}); if(totalopcount > 500000) { console.log("<< finished >>"); await db.dropcollection('text'); db.close(); break; } } })();
you can run pasting index.js
, running npm init -y
, npm install --save mongodb
, node .
when upsert
document, mongo engine has check whether there's existing document matches it. might have explained of slowdown, doesn't insert
command on unique index require same collision checking? thanks!
edit: turns out $setoninsert
is needed else duplicate key errors.
i made each batch bigger changing if(batchopcount === 1000)
if(batchopcount === 50000)
, , got ~90000 ops/sec insert
, ~35000 ops/sec upsert
. i'm not sure why batch size makes relative difference. makes sense smaller batches result in fewer ops/sec (communication overhead), i'm not sure why upserts
suffer more inserts
, though topic separate question.
this (roughly) 3-fold speed difference closer performance difference between 2 operations i'd expect, still seems upserts
little slower should be, given inserts must check collisions (due unique index). half-answer, i'm going leave question open in hope mongo pro come along , provide more complete answer.
Comments
Post a Comment