Nginx加权轮询负载均衡

Nginx 1.22.1

默认负载均衡策略

Nginx默认采用加权轮询策略。

src/http/ngx_http_upstream.c中ngx_http_upstream_init_main_conf函数

省略
for (i = 0; i < umcf->upstreams.nelts; i++) {
init = uscfp[i]->peer.init_upstream ? uscfp[i]->peer.init_upstream:
ngx_http_upstream_init_round_robin;
    if (init(cf, uscfp[i]) != NGX_OK) {
    	return NGX_CONF_ERROR;
    }
}
省略

平滑加权轮询方案

For edge case weights like { 5, 1, 1 } we now produce { a, a, b, a, c, a, a }
sequence instead of { c, b, a, a, a, a, a } produced previously.

Algorithm is as follows: on each peer selection we increase current_weight
of each eligible peer by its weight, select peer with greatest current_weight
and reduce its current_weight by total number of weight points distributed
among peers.

In case of { 5, 1, 1 } weights this gives the following sequence of
current_weight's:

     a  b  c
     0  0  0  (initial state)

     5  1  1  (a selected)
    -2  1  1

     3  2  2  (a selected)
    -4  2  2

     1  3  3  (b selected)
     1 -4  3

     6 -3  4  (a selected)
    -1 -3  4

     4 -2  5  (c selected)
     4 -2 -2

     9 -1 -1  (a selected)
     2 -1 -1

     7  0  0  (a selected)
     0  0  0

To preserve weight reduction in case of failures the effective_weight
variable was introduced, which usually matches peer's weight, but is
reduced temporarily on peer failures.

代码具体实现

初始化weight、currentWeight和effectiveWeight
src/http/ngx_http_upstream_round_robin.c中ngx_http_upstream_init_round_robin函数

省略
for (i = 0; i < us->servers->nelts; i++) {
    if (server[i].backup) {
        continue;
    }

    for (j = 0; j < server[i].naddrs; j++) {
        省略
        peer[n].weight = server[i].weight;
        peer[n].effective_weight = server[i].weight;
        peer[n].current_weight = 0;
        省略
    }
}
省略

weight	始终是初始权重
effectiveWeight	开始是初始权重
currentWeight	开始是0

src/http/ngx_http_upstream_round_robin.c中ngx_http_upstream_get_peer函数

省略
for (peer = rrp->peers->peer, i = 0;
     peer;
     peer = peer->next, i++)
{
    省略
    peer->current_weight += peer->effective_weight;
    total += peer->effective_weight;

    省略

    if (best == NULL || peer->current_weight > best->current_weight) {
        best = peer;
        p = i;
    }
}
省略
best->current_weight -= total;
省略

1. 所有节点的effectiveWeight之和作为totalWeight
2. 遍历所有节点，currentWeight += effectiveWeight，currentWeight值最大的节点作为选中节点
3. 选中节点currentWeight -= totalWeight

检测server不可用机制

先默认根据加权负载均衡算法得到server，再检测是否可以正常把请求转发到server，转发失败后fails++。

fail_timeout默认值是10s，max_fails默认值是1次。
fails>=max_fails后，下一个fail_timeout时间窗内server不可用。

src/http/ngx_http_upstream_round_robin.c中ngx_http_upstream_free_round_robin_peer函数

省略
if (state & NGX_PEER_FAILED) {
    now = ngx_time();

    peer->fails++;
    // peer->accessed唯一更新的地方
    peer->accessed = now;
    peer->checked = now;

    if (peer->max_fails) {
        省略

        if (peer->fails >= peer->max_fails) {
            ngx_log_error(NGX_LOG_WARN, pc->log, 0,
                          "upstream server temporarily disabled");
        }
    }

    省略

} else {

    /* mark peer live if check passed */

    if (peer->accessed < peer->checked) {
        peer->fails = 0;
    }
}
省略

server被选中后更新checked时间，等检测到可以正常访问该server后再把fails改成0。

src/http/ngx_http_upstream_round_robin.c中ngx_http_upstream_get_peer函数

省略
for (peer = rrp->peers->peer, i = 0;
     peer;
     peer = peer->next, i++)
{
省略
    if (peer->max_fails
        && peer->fails >= peer->max_fails
        && now - peer->checked <= peer->fail_timeout)
    {
        continue;
    }
省略
}
省略
if (now - best->checked > best->fail_timeout) {
    best->checked = now;
}

return best;

 参考资料

《深入剖析Nginx》