harbor两层nginx代理导致push不成功401

环境: harbor本身是http的, 外边又套一层https的nginx反向代理

症状: docker login可以成功, docker push的时候提示unauthorized: authentication required, 如下

$ docker login harbor.example.com
Username: chenmin
Password:
Login Succeeded
$ docker push harbor.example.com/project/image:bbda1375
The push refers to repository [harbor.example.com/project/image]
77cae8ab23bf: Layer already exists
unauthorized: authentication required

registry日志如下, 前面日志正常, 最后一下PATCH的时候401

Aug  6 13:57:22 172.22.0.1 registry[905]: time="2020-08-06T05:57:22.497932736Z" level=warning msg="error authorizing context: authorization token required" go.version=go1.7.3 http.request.host=harbor.example.com http.request.id=89e8a520-fbf4-4fb2-9249-75e697567caf http.request.method=GET http.request.remoteaddr=10.1.0.129 http.request.uri="/v2/" http.request.useragent="docker/18.06.1-ce go/go1.10.3 git-commit/e68fc7a kernel/4.9.93-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.1-ce \\(darwin\\))" instance.id=3b45545d-5a53-4a1e-a3f9-19dfa946240d service=registry version=v2.6.2
Aug  6 13:57:22 172.22.0.1 registry[905]: 172.22.0.6 - - [06/Aug/2020:05:57:22 +0000] "GET /v2/ HTTP/1.1" 401 87 "" "docker/18.06.1-ce go/go1.10.3 git-commit/e68fc7a kernel/4.9.93-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.1-ce \\(darwin\\))"
Aug  6 13:57:22 172.22.0.1 registry[905]: time="2020-08-06T05:57:22.592417248Z" level=info msg="response completed" go.version=go1.7.3 http.request.host=harbor.example.com http.request.id=e22170d0-112e-40a3-a613-823713d41e90 http.request.method=HEAD http.request.remoteaddr=10.1.0.129 http.request.uri="/v2/project/image/blobs/sha256:89d9c30c1d48bac627e5c6cb0d1ed1eec28e7dbdfbcc04712e4c79c0f83faf17" http.request.useragent="docker/18.06.1-ce go/go1.10.3 git-commit/e68fc7a kernel/4.9.93-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.1-ce \\(darwin\\))" http.response.contenttype="application/octet-stream" http.response.duration=2.9709ms http.response.status=200 http.response.written=0 instance.id=3b45545d-5a53-4a1e-a3f9-19dfa946240d service=registry version=v2.6.2
Aug  6 13:57:22 172.22.0.1 registry[905]: 172.22.0.6 - - [06/Aug/2020:05:57:22 +0000] "HEAD /v2/project/image/blobs/sha256:89d9c30c1d48bac627e5c6cb0d1ed1eec28e7dbdfbcc04712e4c79c0f83faf17 HTTP/1.1" 200 0 "" "docker/18.06.1-ce go/go1.10.3 git-commit/e68fc7a kernel/4.9.93-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.1-ce \\(darwin\\))"
Aug  6 13:57:22 172.22.0.1 registry[905]: time="2020-08-06T05:57:22.633988818Z" level=error msg="response completed with error" auth.user.name=chenmin err.code="blob unknown" err.detail=sha256:965ea09ff2ebd2b9eeec88cd822ce156f6674c7e99be082c7efac3c62f3ff652 err.message="blob unknown to registry" go.version=go1.7.3 http.request.host=harbor.example.com http.request.id=45cea475-185e-45c8-a37d-ecb777c67cc2 http.request.method=HEAD http.request.remoteaddr=10.1.0.129 http.request.uri="/v2/project/image/blobs/sha256:965ea09ff2ebd2b9eeec88cd822ce156f6674c7e99be082c7efac3c62f3ff652" http.request.useragent="docker/18.06.1-ce go/go1.10.3 git-commit/e68fc7a kernel/4.9.93-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.1-ce \\(darwin\\))" http.response.contenttype="application/json; charset=utf-8" http.response.duration=2.453939ms http.response.status=404 http.response.written=157 instance.id=3b45545d-5a53-4a1e-a3f9-19dfa946240d service=registry vars.digest="sha256:965ea09ff2ebd2b9eeec88cd822ce156f6674c7e99be082c7efac3c62f3ff652" vars.name="project/image" version=v2.6.2
Aug  6 13:57:22 172.22.0.1 registry[905]: 172.22.0.6 - - [06/Aug/2020:05:57:22 +0000] "HEAD /v2/project/image/blobs/sha256:965ea09ff2ebd2b9eeec88cd822ce156f6674c7e99be082c7efac3c62f3ff652 HTTP/1.1" 404 157 "" "docker/18.06.1-ce go/go1.10.3 git-commit/e68fc7a kernel/4.9.93-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.1-ce \\(darwin\\))"
Aug  6 13:57:22 172.22.0.1 registry[905]: time="2020-08-06T05:57:22.681416107Z" level=info msg="response completed" go.version=go1.7.3 http.request.host=harbor.example.com http.request.id=5470ded0-8d8c-4a57-825d-5801e50171ee http.request.method=POST http.request.remoteaddr=10.1.0.129 http.request.uri="/v2/project/image/blobs/uploads/" http.request.useragent="docker/18.06.1-ce go/go1.10.3 git-commit/e68fc7a kernel/4.9.93-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.1-ce \\(darwin\\))" http.response.duration=6.76585ms http.response.status=202 http.response.written=0 instance.id=3b45545d-5a53-4a1e-a3f9-19dfa946240d service=registry version=v2.6.2
Aug  6 13:57:22 172.22.0.1 registry[905]: 172.22.0.6 - - [06/Aug/2020:05:57:22 +0000] "POST /v2/project/image/blobs/uploads/ HTTP/1.1" 202 0 "" "docker/18.06.1-ce go/go1.10.3 git-commit/e68fc7a kernel/4.9.93-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.1-ce \\(darwin\\))"
Aug  6 13:57:22 172.22.0.1 registry[905]: time="2020-08-06T05:57:22.704009832Z" level=warning msg="error authorizing context: authorization token required" go.version=go1.7.3 http.request.host="harbor.example.com:80" http.request.id=56abcaa5-71a5-4ba9-bacf-a9d28f5aa740 http.request.method=PATCH http.request.remoteaddr=10.1.0.129 http.request.uri="/v2/project/image/blobs/uploads/02e17f97-0ff2-4e47-9e4b-0f09184a304a?_state=7QIgWeJwgEAv0M8IU_ikhqsdr3LqBob5ccYqu4MxiJ17Ik5hbWUiOiJzaHVuc2h1bi9zaGFuZ3h1ZXl1YW4tYmFja2VuZCIsIlVVSUQiOiIwMmUxN2Y5Ny0wZmYyLTRlNDctOWU0Yi0wZjA5MTg0YTMwNGEiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjAtMDgtMDZUMDU6NTc6MjIuNjc4Mzg0MDQ5WiJ9" http.request.useragent="docker/18.06.1-ce go/go1.10.3 git-commit/e68fc7a kernel/4.9.93-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.1-ce \\(darwin\\))" instance.id=3b45545d-5a53-4a1e-a3f9-19dfa946240d service=registry vars.name="project/image" vars.uuid=02e17f97-0ff2-4e47-9e4b-0f09184a304a version=v2.6.2
Aug  6 13:57:22 172.22.0.1 registry[905]: 172.22.0.6 - - [06/Aug/2020:05:57:22 +0000] "PATCH /v2/project/image/blobs/uploads/02e17f97-0ff2-4e47-9e4b-0f09184a304a?_state=7QIgWeJwgEAv0M8IU_ikhqsdr3LqBob5ccYqu4MxiJ17Ik5hbWUiOiJzaHVuc2h1bi9zaGFuZ3h1ZXl1YW4tYmFja2VuZCIsIlVVSUQiOiIwMmUxN2Y5Ny0wZmYyLTRlNDctOWU0Yi0wZjA5MTg0YTMwNGEiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjAtMDgtMDZUMDU6NTc6MjIuNjc4Mzg0MDQ5WiJ9 HTTP/1.1" 401 260 "" "docker/18.06.1-ce go/go1.10.3 git-commit/e68fc7a kernel/4.9.93-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.1-ce \\(darwin\\))"

解决方法一:

引用自: https://github.com/goharbor/harbor/issues/3114#issuecomment-424992795

在common/config/registry/config.yml中修改realm为https

解决方法二:

引用自: https://github.com/goharbor/harbor/issues/3114#issuecomment-394139225

删除/注释掉common/config/nginx/nginx.conf中的proxy_set_header X-Forwarded-Proto $scheme;

操作如下: 

sed -i '/X-Forwarded-Proto/d' common/config/nginx/nginx.conf
docker restart nginx

 

推荐方法一