timeout - Error receiving heartbeat for connection - client connection -
i'm using hazelcast 3.8 4-node cluster distributed computations. starting problem size start getting errors like
apr 07, 2017 4:58:42 pm com.hazelcast.client.connection.clientconnectionmanager warning: hz.client_0 [kpts-cluster] [3.8] error receiving heartbeat connection: clientconnection{alive=true, connectionid=4, socketchannel=defaultsocketchannelwrapper{socketchannel=java.nio.channels.socketchannel[connected local=/172.20.65.8:56570 remote=/172.20.65.5:5701]}, remoteendpoint=[172.20.65.5]:5701, lastreadtime=2017-04-07 16:57:35.294, lastwritetime=2017-04-07 16:58:41.841, closedtime=never, lastheartbeatrequested=2017-04-07 16:58:41.841, lastheartbeatreceived=2017-04-07 16:57:21.841, connected server version=3.8} java.util.concurrent.executionexception: com.hazelcast.spi.exception.targetdisconnectedexception: heartbeat timed out clientconnection{alive=true, connectionid=4, socketchannel=defaultsocketchannelwrapper{socketchannel=java.nio.channels.socketchannel[connected local=/172.20.65.8:56570 remote=/172.20.65.5:5701]}, remoteendpoint=[172.20.65.5]:5701, lastreadtime=2017-04-07 16:57:35.294, lastwritetime=2017-04-07 16:58:41.841, closedtime=never, lastheartbeatrequested=2017-04-07 16:58:41.841, lastheartbeatreceived=2017-04-07 16:57:21.841, connected server version=3.8} @ com.hazelcast.client.spi.impl.clientinvocationfuture.resolve(clientinvocationfuture.java:66) @ com.hazelcast.spi.impl.abstractinvocationfuture$1.run(abstractinvocationfuture.java:243) @ java.util.concurrent.executors$runnableadapter.call(executors.java:511) @ java.util.concurrent.futuretask.run(futuretask.java:266) @ java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.access$201(scheduledthreadpoolexecutor.java:180) @ java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.run(scheduledthreadpoolexecutor.java:293) @ com.hazelcast.util.executor.loggingscheduledexecutor$loggingdelegatingfuture.run(loggingscheduledexecutor.java:128) @ java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1142) @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:617) @ java.lang.thread.run(thread.java:745) @ com.hazelcast.util.executor.hazelcastmanagedthread.executerun(hazelcastmanagedthread.java:76) @ com.hazelcast.util.executor.hazelcastmanagedthread.run(hazelcastmanagedthread.java:92) caused by: com.hazelcast.spi.exception.targetdisconnectedexception: heartbeat timed out clientconnection{alive=true, connectionid=4, socketchannel=defaultsocketchannelwrapper{socketchannel=java.nio.channels.socketchannel[connected local=/172.20.65.8:56570 remote=/172.20.65.5:5701]}, remoteendpoint=[172.20.65.5]:5701, lastreadtime=2017-04-07 16:57:35.294, lastwritetime=2017-04-07 16:58:41.841, closedtime=never, lastheartbeatrequested=2017-04-07 16:58:41.841, lastheartbeatreceived=2017-04-07 16:57:21.841, connected server version=3.8} @ com.hazelcast.client.spi.impl.clientinvocationservicesupport$cleanresourcestask.notifyexception(clientinvocationservicesupport.java:241) @ com.hazelcast.client.spi.impl.clientinvocationservicesupport$cleanresourcestask.run(clientinvocationservicesupport.java:224) @ java.util.concurrent.executors$runnableadapter.call(executors.java:511) @ java.util.concurrent.futuretask.runandreset(futuretask.java:308) @ java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.access$301(scheduledthreadpoolexecutor.java:180) @ java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.run(scheduledthreadpoolexecutor.java:294) ... 6 more
what might issue? doesn't happen smaller problems. tried increase heartbeat timeouts in client properties didn't help:
<properties> <property name="hazelcast.heartbeat.interval.seconds">60</property> <property name="hazelcast.client.heartbeat.interval">3000000</property> <property name="hazelcast.client.heartbeat.timeout">50000000</property> <property name="hazelcast.client.max.no.heartbeat.seconds">10000</property> </properties>
even if did guess shouldn't happen. way nodes 100% loaded during computations. sure there no connectivity issues. i'm running hazelcast members directly hazelcast binaries adding program jar classpath... client standalone application using hazelcast client.
Comments
Post a Comment