spymemcached :某个mc节点操作连续超时超过998次就 Auto-Reconnect 的特性
spymemcached 中,一个 mc client 与某一个 mc node 之间已建立的 connection 上,
如果执行操作屡屡超时(net.spy.memcached.OperationTimeoutException),
那么,有一个计数器专门记录超时次数(connected或reconnecting就清零)。
当某一个 mc node 的连续超时次数达到一定阈值(如 998 次),
就会……
- 关闭原来的 socket channel,并将该 mc node 放入重连队列 reconnectQueue :lostConnection(mn),
- 遍历 reconnectQueue ,逐一尝试重新建立 socket 连接:attemptReconnects(),
- 然后不管连没连上,rehash 队列中的操作:redistributeOperations(retryOps)。(rehash 指重新计算hash值,看是否要连接其他的节点)。
这个超时次数阈值,既可以在 spring 里如下这样设置,也可以采用默认值 998 次:
<beanid="memcachedClient"class="net.spy.memcached.spring.MemcachedClientFactoryBean">
|
相关代码:
每一次 handle io over the connections 时,都会检测一下这个 connection 对应的节点上登记的“连续超时次数”是否已经达到阈值:
MemcachedConnection.java
|
private final int timeoutExceptionThreshold;
……
timeoutExceptionThreshold = f.getTimeoutExceptionThreshold();
……
|
/**
* MemcachedClient calls this method to handle IO over the connections.
*/
public void handleIO() throws IOException {
……
// see if any connections blew up with large number of timeouts
for (SelectionKey sk : selector.keys()) {
MemcachedNode mn = (MemcachedNode) sk.attachment();
if (mn.getContinuousTimeout() > timeoutExceptionThreshold) {//检查一下
getLogger().warn("%s exceeded continuous timeout threshold", sk);
lostConnection(mn);//断开连接,放入重连队列
}
}
if (!shutDown && !reconnectQueue.isEmpty()) {
attemptReconnects();//重新连接
}
// rehash any operations that are in retry state
redistributeOperations(retryOps);//
retryOps.clear();
|
如果配置文件也没配 timeoutExceptionThreshold ,初始化时也没传这个参数, 那么 DefaultConnectionFactory 就取默认值 DEFAULT_MAX_TIMEOUTEXCEPTION_THRESHOLD,即998次(anyway,也许是1000次)! |
DefaultConnectionFactory.java |
/**
* Maximum number + 2 of timeout exception for shutdown connection.
*/
public static final int DEFAULT_MAX_TIMEOUTEXCEPTION_THRESHOLD = 998;
|
DefaultConnectionFactory.java |
public int getTimeoutExceptionThreshold() {
return DEFAULT_MAX_TIMEOUTEXCEPTION_THRESHOLD;
}
|
针对每一个 memcache node 设置一个“连续超时次数”计数器:
TCPMemcachedNodeImpl.java
|
// operation Future.get timeout counter
private final AtomicInteger continuousTimeout = new AtomicInteger(0);
……
public void setContinuousTimeout(boolean timedOut) {
if (timedOut && isActive()) {
continuousTimeout.incrementAndGet();
} else {
continuousTimeout.set(0);
}
}
|
MemcachedConnection.java |
/**
* helper method: increase timeout count on node attached to this op.
*
* @param op
*/
public static void opTimedOut(Operation op) {
MemcachedConnection.setTimeout(op, true);
}
|
参考资源:
1)智深,2012,spymemcached源码深入分析;
2)2011,Issue 173: Spymemcached fail reconnect to local memcahed under high load ..;
3)最佳实践系列:前端代码标准和最佳实践 (2012-12-19 23:33)
2)电商课题V:分布式锁 (2012-11-17 22:16)
3)电商课题:cookie防篡改 (2012-11-17 22:24)
4)电商课题VI:分布式Session (2012-11-17 22:30)
5)电商课题:RBAC权限控制 (2012-11-17 22:47)
6)电商课题:幂等性 (2012-11-22 23:52)
7)电商课题:客户端的IP地址伪造、CDN、反向代理、获取的那些事儿 (2012-09-19 01:17)
8)电商课题:对付秒杀器等恶意访问行为的简单梳理 (2012-09-18 03:51)
9)电商课题VII:支付交易一般性准则 (2012-12-14 01:38)赠图一枚