详解proxy_pass、upstream与resolver

详解proxy_pass、upstream与resolver

96 
boldcautious 
2018.06.04 10:48 字数 1204 阅读 1434评论 0

应用场景

这里列举几个应用场景，下文会针对这几个场景并结合代码进行分析。

  1. proxy_pass + upstream
    upstream foo.example.com {
        server 127.0.0.1:8001;
    }
 
    server {
        listen       80;
        server_name  localhost;
 
        location /foo {
            proxy_pass http://foo.example.com;
        }
    }

访问http://localhost/foo，proxy模块会将请求转发到127.0.0.1的8001端口上。

  1. 只有proxy_pass，没有upstream与resolver
    server {
        listen       80;
        server_name  localhost;
 
        location /foo {
            proxy_pass http://foo.example.com;
        }
    }

实际上是隐式创建了upstream，upstream名字就是foo.example.com。upstream模块利用本机设置的DNS服务器（或/etc/hosts），将foo.example.com解析成IP，访问http://localhost/foo，proxy模块会将请求转发到解析后的IP上。

如果本机未设置DNS服务器，或者DNS服务器无法解析域名，则nginx启动时会报类似如下错误：

nginx: [emerg] host not found in upstream "foo.example.com" in /path/nginx/conf/nginx.conf:110

  1. proxy_pass + resolver（变量设置域名）
    server {
        listen       80;
        server_name  localhost;
 
        resolver 114.114.114.114;
        location /foo {
            set $foo foo.example.com;
            proxy_pass http://$foo;
        }
    }

访问http://localhost/foo，nginx会动态利用resolver设置的DNS服务器（本机设置的DNS服务器或/etc/hosts无效），将域名解析成IP，proxy模块会将请求转发到解析后的IP上。

  1. proxy_pass + upstream（显式） + resolver（变量设置域名）
    upstream foo.example.com {
        server 127.0.0.1:8001;
    }
 
    server {
        listen       80;
        server_name  localhost;
 
        resolver 114.114.114.114;
        location /foo {
            set $foo foo.example.com;
            proxy_pass http://$foo;
        }
    }

访问http://localhost/foo时，upstream模块会优先查找是否有定义upstream后端服务器，如果有定义则直接利用，不再走DNS解析。所以proxy模块会将请求转发到127.0.0.1的8001端口上。

  1. proxy_pass + upstream（隐式） + resolver（变量设置域名）
    server {
        listen       80;
        server_name  localhost;
 
        resolver 114.114.114.114;
        location /foo {
            set $foo foo.example.com;
            proxy_pass http://$foo;
        }
 
        location /foo2 {
            proxy_pass http://foo.example.com;
        }
    }

location /foo2实际上是隐式定义了upstream foo.example.com，并由本地DNS服务器进行了域名解析，访问http://localhost/foo时，upstream模块会优先查找upstream，即隐式定义的foo.example.com，proxy模块会将请求转发到解析后的IP上。

  1. proxy_pass + resolver（不用变量设置域名）
    server {
        listen       80;
        server_name  localhost;
 
        resolver 114.114.114.114;
        location /foo {
            proxy_pass http://foo.example.com;
        }
    }

不使用变量设置域名，则resolver的设置不起作用，此时相当于场景2，只有proxy_pass的场景。

  1. proxy_pass + upstream + resolver（不用变量设置域名）
    upstream foo.example.com {
        server 127.0.0.1:8001;
    }
 
    server {
        listen       80;
        server_name  localhost;
 
        resolver 114.114.114.114;
        location /foo {
            proxy_pass http://foo.example.com;
        }
    }

不使用变量设置域名，则resolver的设置不起作用，此时相当于场景1 proxy_pass + upstream。

  1. proxy_pass 直接指定IP加端口号
    server {
        listen       80;
        server_name  localhost;
 
        location /foo {
            proxy_pass http://127.0.0.1:8001/;
        }
    }

实际上是隐式创建了upstream，proxy_pass会将请求转发到127.0.0.1的8001端口上。

主要代码

解析proxy_pass指令的代码：

static char *
ngx_http_proxy_pass(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
{
    ngx_http_proxy_loc_conf_t *plcf = conf;
 
    size_t                      add;
    u_short                     port;
    ngx_str_t                  *value, *url;
    ngx_url_t                   u;
    ngx_uint_t                  n;
    ngx_http_core_loc_conf_t   *clcf;
    ngx_http_script_compile_t   sc;
 
    if (plcf->upstream.upstream || plcf->proxy_lengths) {
        return "is duplicate";
    }
 
    clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
 
    clcf->handler = ngx_http_proxy_handler;
 
    if (clcf->name.data[clcf->name.len - 1] == '/') {
        clcf->auto_redirect = 1;
    }
 
    value = cf->args->elts;
 
    url = &value[1];
 
    /* 查找指令中$符号的位置，判断是否使用了变量 */
    n = ngx_http_script_variables_count(url);
 
    if (n) {
        /* 使用变量设置域名 */
        ngx_memzero(&sc, sizeof(ngx_http_script_compile_t));
 
        sc.cf = cf;
        sc.source = url;
        sc.lengths = &plcf->proxy_lengths;
        sc.values = &plcf->proxy_values;
        sc.variables = n;
        sc.complete_lengths = 1;
        sc.complete_values = 1;
 
        if (ngx_http_script_compile(&sc) != NGX_OK) {
            return NGX_CONF_ERROR;
        }
 
#if (NGX_HTTP_SSL)
        plcf->ssl = 1;
#endif
 
        return NGX_CONF_OK;
    }
 
    if (ngx_strncasecmp(url->data, (u_char *) "http://", 7) == 0) {
        add = 7;
        port = 80;
 
    } else if (ngx_strncasecmp(url->data, (u_char *) "https://", 8) == 0) {
 
#if (NGX_HTTP_SSL)
        plcf->ssl = 1;
 
        add = 8;
        port = 443;
#else
        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                           "https protocol requires SSL support");
        return NGX_CONF_ERROR;
#endif
 
    } else {
        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid URL prefix");
        return NGX_CONF_ERROR;
    }
 
    ngx_memzero(&u, sizeof(ngx_url_t));
 
    u.url.len = url->len - add;
    u.url.data = url->data + add;
    u.default_port = port;
    u.uri_part = 1;
    u.no_resolve = 1;
 
    plcf->upstream.upstream = ngx_http_upstream_add(cf, &u, 0);
    if (plcf->upstream.upstream == NULL) {
        return NGX_CONF_ERROR;
    }
 
    plcf->vars.schema.len = add;
    plcf->vars.schema.data = url->data;
    plcf->vars.key_start = plcf->vars.schema;
 
    ngx_http_proxy_set_vars(&u, &plcf->vars);
 
    plcf->location = clcf->name;
 
    if (clcf->named
#if (NGX_PCRE)
        || clcf->regex
#endif
        || clcf->noname)
    {
        if (plcf->vars.uri.len) {
            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                               "\"proxy_pass\" cannot have URI part in "
                               "location given by regular expression, "
                               "or inside named location, "
                               "or inside \"if\" statement, "
                               "or inside \"limit_except\" block");
            return NGX_CONF_ERROR;
        }
 
        plcf->location.len = 0;
    }
 
    plcf->url = *url;
 
    return NGX_CONF_OK;
}

upstream定义的后端服务器的处理逻辑，包括显式定义的和隐式定义的。隐式定义，即proxy_pass指定的后端服务器的地址没有显式用upstream定义，nginx内部会定义一个。

ngx_int_t
ngx_http_upstream_init_round_robin(ngx_conf_t *cf,
    ngx_http_upstream_srv_conf_t *us)
{
    ngx_url_t                      u;
    ngx_uint_t                     i, j, n, w;
    ngx_http_upstream_server_t    *server;
    ngx_http_upstream_rr_peer_t   *peer, **peerp;
    ngx_http_upstream_rr_peers_t  *peers, *backup;
 
    us->peer.init = ngx_http_upstream_init_round_robin_peer;
 
    /* 
     * 使用upstream指令显式定义upstream 
     * 或者proxy_pass直接指定IP的场景
     */
    if (us->servers) {
        server = us->servers->elts;
 
        n = 0;
        w = 0;
 
        for (i = 0; i < us->servers->nelts; i++) {
            if (server[i].backup) {
                continue;
            }
 
            n += server[i].naddrs;
            w += server[i].naddrs * server[i].weight;
        }
 
        if (n == 0) {
            ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                          "no servers in upstream \"%V\" in %s:%ui",
                          &us->host, us->file_name, us->line);
            return NGX_ERROR;
        }
 
        peers = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peers_t));
        if (peers == NULL) {
            return NGX_ERROR;
        }
 
        peer = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peer_t) * n);
        if (peer == NULL) {
            return NGX_ERROR;
        }
 
        peers->single = (n == 1);
        peers->number = n;
        peers->weighted = (w != n);
        peers->total_weight = w;
        peers->name = &us->host;
 
        n = 0;
        peerp = &peers->peer;
 
        for (i = 0; i < us->servers->nelts; i++) {
            /* 设置sockaddr、weight、max_fails、fail_timeout等属性 */
        }
 
        us->peer.data = peers;
 
        /* 处理backup servers相关逻辑 */
 
        return NGX_OK;
    }
 
    /* 未使用upstream指令，使用proxy_pass隐式定义upstream */
    /* an upstream implicitly defined by proxy_pass, etc. */
 
    if (us->port == 0) {
        ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                      "no port in upstream \"%V\" in %s:%ui",
                      &us->host, us->file_name, us->line);
        return NGX_ERROR;
    }
 
    ngx_memzero(&u, sizeof(ngx_url_t));
 
    u.host = us->host;
    u.port = us->port;
 
    if (ngx_inet_resolve_host(cf->pool, &u) != NGX_OK) {
        if (u.err) {
            ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                          "%s in upstream \"%V\" in %s:%ui",
                          u.err, &us->host, us->file_name, us->line);
        }
 
        return NGX_ERROR;
    }
 
    n = u.naddrs;
 
    peers = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peers_t));
    if (peers == NULL) {
        return NGX_ERROR;
    }
 
    peer = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peer_t) * n);
    if (peer == NULL) {
        return NGX_ERROR;
    }
 
    peers->single = (n == 1);
    peers->number = n;
    peers->weighted = 0;
    peers->total_weight = n;
    peers->name = &us->host;
 
    peerp = &peers->peer;
 
    for (i = 0; i < u.naddrs; i++) {
        /* 设置sockaddr、weight、max_fails、fail_timeout等属性 */
    }
 
    us->peer.data = peers;
 
    /* implicitly defined upstream has no backup servers */
 
    return NGX_OK;
}

upstream模块初始化请求时的逻辑：

static void
ngx_http_upstream_init_request(ngx_http_request_t *r)
{
    ngx_str_t                      *host;
    ngx_uint_t                      i;
    ngx_resolver_ctx_t             *ctx, temp;
    ngx_http_cleanup_t             *cln;
    ngx_http_upstream_t            *u;
    ngx_http_core_loc_conf_t       *clcf;
    ngx_http_upstream_srv_conf_t   *uscf, **uscfp;
    ngx_http_upstream_main_conf_t  *umcf;
 
    if (r->aio) {
        return;
    }
 
    u = r->upstream;
 
    /* NGX_HTTP_CACHE 等其他处理 */
 
    cln->handler = ngx_http_upstream_cleanup;
    cln->data = r;
    u->cleanup = &cln->handler;
 
    if (u->resolved == NULL) {
        /* 如果没有使用resolver设置DNS，直接取upstream的设置 */
        uscf = u->conf->upstream;
 
    } else {
 
#if (NGX_HTTP_SSL)
        u->ssl_name = u->resolved->host;
#endif
 
        host = &u->resolved->host;
 
        if (u->resolved->sockaddr) {
 
            if (u->resolved->port == 0
                && u->resolved->sockaddr->sa_family != AF_UNIX)
            {
                ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                              "no port in upstream \"%V\"", host);
                ngx_http_upstream_finalize_request(r, u,
                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
                return;
            }
 
            if (ngx_http_upstream_create_round_robin_peer(r, u->resolved)
                != NGX_OK)
            {
                ngx_http_upstream_finalize_request(r, u,
                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
                return;
            }
 
            ngx_http_upstream_connect(r, u);
 
            return;
        }
 
        umcf = ngx_http_get_module_main_conf(r, ngx_http_upstream_module);
 
        uscfp = umcf->upstreams.elts;
 
        /* 在显式/隐式定义的upstream中查找 */
        for (i = 0; i < umcf->upstreams.nelts; i++) {
 
            uscf = uscfp[i];
 
            if (uscf->host.len == host->len
                && ((uscf->port == 0 && u->resolved->no_port)
                     || uscf->port == u->resolved->port)
                && ngx_strncasecmp(uscf->host.data, host->data, host->len) == 0)
            {
                goto found;
            }
        }
 
        if (u->resolved->port == 0) {
            ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                          "no port in upstream \"%V\"", host);
            ngx_http_upstream_finalize_request(r, u,
                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
            return;
        }
 
        temp.name = *host;
 
        ctx = ngx_resolve_start(clcf->resolver, &temp);
        if (ctx == NULL) {
            ngx_http_upstream_finalize_request(r, u,
                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
            return;
        }
 
        if (ctx == NGX_NO_RESOLVER) {
            ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                          "no resolver defined to resolve %V", host);
 
            ngx_http_upstream_finalize_request(r, u, NGX_HTTP_BAD_GATEWAY);
            return;
        }
 
        ctx->name = *host;
        ctx->handler = ngx_http_upstream_resolve_handler;
        ctx->data = r;
        ctx->timeout = clcf->resolver_timeout;
 
        u->resolved->ctx = ctx;
 
        if (ngx_resolve_name(ctx) != NGX_OK) {
            u->resolved->ctx = NULL;
            ngx_http_upstream_finalize_request(r, u,
                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
            return;
        }
 
        return;
    }
 
found:
 
    if (uscf == NULL) {
        ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,
                      "no upstream configuration");
        ngx_http_upstream_finalize_request(r, u,
                                           NGX_HTTP_INTERNAL_SERVER_ERROR);
        return;
    }
 
#if (NGX_HTTP_SSL)
    u->ssl_name = uscf->host;
#endif
 
    if (uscf->peer.init(r, uscf) != NGX_OK) {
        ngx_http_upstream_finalize_request(r, u,
                                           NGX_HTTP_INTERNAL_SERVER_ERROR);
        return;
    }
 
    u->peer.start_time = ngx_current_msec;
 
    if (u->conf->next_upstream_tries
        && u->peer.tries > u->conf->next_upstream_tries)
    {
        u->peer.tries = u->conf->next_upstream_tries;
    }
 
    ngx_http_upstream_connect(r, u);
}

详细分析

场景1

解析proxy_pass的函数ngx_http_proxy_pass中，没有找到$符号（即，变量设置域名），走ngx_http_proxy_pass后半部分的处理逻辑。ngx_http_upstream_init_round_robin初始化upstream时，走显式定义upstream的逻辑。proxy_pass转发请求初始化时，ngx_http_upstream_init_request中直接使用upstream中的后端server建立连接。

场景2

ngx_http_upstream_init_round_robin初始化upstream时，走隐式定义upstream的逻辑，会调用ngx_inet_resolve_host对proxy_pass中的域名进行解析，设置upstream。proxy_pass转发请求初始化时，ngx_http_upstream_init_request中直接使用upstream中的设置，也就是利用本地设置的DNS服务器解析出的IP，建立连接。

场景3

解析proxy_pass指令时，找到了$符号，设置ngx_http_script_compile_t，并利用ngx_http_script_compile进行编译，不走后半部分逻辑。配置文件没有显式/隐式定义upstream，所以不会调用ngx_http_upstream_init_round_robin方法。proxy_pass转发请求初始化时，ngx_http_upstream_init_request中发现没有显式也没有隐式定义的upstream，随后调用ngx_resolve_start，对域名进行解析，之后将请求转发过去。

场景4

解析proxy_pass指令时，找到了$符号，设置ngx_http_script_compile_t，并利用ngx_http_script_compile进行编译，不走后半部分逻辑。显式调用了upstream，所以调用ngx_http_upstream_init_round_robin方法中的显式upstream的处理逻辑。proxy_pass转发请求初始化时，ngx_http_upstream_init_request中优先查找upstream，如果找到了，直接将请求转发到upstream中的后端server上。如果upstream中没有找到，则对域名进行解析，然后将请求转发到解析后的IP上。

场景5

基本与场景4相同，不同之处在于调用ngx_http_upstream_init_round_robin方法时，走隐式upstream部分的处理逻辑。

场景6

与场景2相同。

场景7

与场景1相同。

场景8

实际上是隐式创建了upstream，但是因为proxy_pass中指定了IP和端口号，所以ngx_http_upstream_init_round_robin初始化upstream时，us->servers不为空，所以走该函数的上半部分逻辑。与场景1有些类似。

https://www.jianshu.com/p/5caa48664da5

posted @   silentmuh  阅读(1501)  评论(0)  编辑  收藏  举报
编辑推荐：
· Linux系列：如何用heaptrack跟踪.NET程序的非托管内存泄露
· 开发者必知的日志记录最佳实践
· SQL Server 2025 AI相关能力初探
· Linux系列：如何用 C#调用 C方法造成内存泄露
· AI与.NET技术实操系列（二）：开始使用ML.NET
阅读排行：
· 被坑几百块钱后，我竟然真的恢复了删除的微信聊天记录！
· 没有Manus邀请码？试试免邀请码的MGX或者开源的OpenManus吧
· 【自荐】一款简洁、开源的在线白板工具 Drawnix
· 园子的第一款AI主题卫衣上架——"HELLO! HOW CAN I ASSIST YOU TODAY
· Docker 太简单，K8s 太复杂？w7panel 让容器管理更轻松！
Live2D
欢迎阅读『详解proxy_pass、upstream与resolver』
  1. 1 Walk Thru Fire Vicetone
  2. 2 爱你 王心凌
  3. 3 Inspire Capo Productions - Serenity
  4. 4 Welcome Home Radical Face
  5. 5 粉红色的回忆 李玲玉
Welcome Home - Radical Face
00:00 / 00:00
An audio error has occurred, player will skip forward in 2 seconds.

作词 : Ben P. Cooper

作曲 : Cooper

Sleep don't visit, so I choke on sun

And the days blur into one

And the backs of my eyes hum with things I've never done

Sheets are swaying from an old clothesline

Was never much but we've made the most

Welcome home

Ships are launching from my chest

Some have names but most do not

If you find one,please let me know what piece I've lost

Heal the scars from off my back

I don't need them anymore

You can throw them out or keep them in your mason jars

I've come home

All my nightmares escaped my head

Bar the door, please don't let them in

You were never supposed to leave

Now my head's splitting at the seams

And I don't know if I can

Here, beneath my lungs

I feel your thumbs press into my skin again

点击右上角即可分享
微信分享提示