java - Two Kafka consumers causing odd behavior with one another -


i have 2 consumers different client id's , group id's. aside retention hour , max partitions, kafka installation contains default configuration. i've looked around see if else has had same issue can't pull results.

so scenario goes this:

consumer a: connects kafka, consumes 3 million messages need consumed, , sits idle waiting more messages.

consumer b: different client / group id, connects same kafka topic, , causes consumer repeat of 3 million messages while consumer b consumes them well.

the 2 consumers 2 different java applications different client , group id's running on same computer. kafka server on computer.

is normal behavior in kafka? @ complete loss.

here consumer config:

bootstrap.servers=192.168.110.109:9092 acks=all max.block.ms=2000 retries=0 batch.size=16384 auto.commit.interval.ms=1000 linger.ms=0 key.deserializer=org.apache.kafka.common.serialization.stringdeserializer value.deserializer=org.apache.kafka.common.serialization.stringdeserializer block.on.buffer.full=true enable.auto.commit=false auto.offset.reset=none session.timeout.ms=30000 zookeeper.session.timeout=100000 rebalance.backoff.ms=8000  group.id=consumergroupa zookeeper.connect=192.168.110.109:2181 poll.interval=100

and obvious difference in consumer b group.id=consumergroupb

this correct behavior. because based on configs, consumers don't commit offset of records have read!

when consumer read record, must commit reading it, can ensure consumers commit offsets automatically setting enable.auto.commit=true or commit each record manually. in case think auto commit fine you.


Comments

Post a Comment

Popular posts from this blog

Command prompt result in label. Python 2.7 -

i want put result(output) of shell command label. command works, label shows "0", however, in command prompt, result shown correctly, need showed in label. doing ubuntu. myg1 = button(root, text="rodyti informacija", command=lambda: gauti()) myg1.pack(side=bottom) def gauti(): imti = tekstas.get("1.0", "end-1c") info = subprocess.call("id '{imti}' ".format(imti=imti), shell=true) w = label(root, text= info) w.pack(side = bottom) subprocess.call() returns exit code of process created (in case 0 success). if want text output of process, should instead call subprocess.check_output() .
Read more

javascript - How do I use URL parameters to change link href on page? -

i change href value of link using url parameters. example: default <a id="dlink" href="link-1">link</a> but, when user goes https://url.com/?dlink=link-2 it swaps out link new link <a id="dlink" href="link-2">link</a> first need value url. if you're not using javascript framework can create function using jquery. please check get url parameter jquery or how query string values in js it. after that, element "a" id , set value there. $("#dlink").attr("href", hreffromparam);
Read more

amazon web services - AWS Route53 Trying To Get Site To Resolve To www -

so have domain nameservers pointed aws route 53. site hosted on ec2 instance , reachable resolves site.com, if input www.site.com. i'm curious how site resolve www.site.com. have under route53 domains hosted zone 2 additional record sets outside of ns , soa: a site.com ec2 elastic ip cname www.site.com ec2 public dns i can reach site using both site.com , www.site.com, resolve site.com. there way make if enter either, resolves www.site.com? at first, both should record set. a site.com ec2 elastic ip www.site.com ec2 elastic ip and need redirect setting in web server (nginx, apache, ..) following. (source: how redirect www non-www nginx on centos 7 ) redirect non-www www if want redirect users plain, non-www domain www domain, add server block: server { server_name example.com; return 301 $scheme://www.example.com$request_uri; } save , exit. configures nginx redirect requests "example.com" "www.example.com". not...
Read more