gpushare-scheduler-extender 升级依赖引发关于 golang dep 工具的思考

gpushare-scheduler-extender 依赖的 go package 升级带来的问题

gpushare-scheduler-extender 的依赖软件包默认是 kubernetes 1.11，版本较低，需要升级。该项目使用 dep 来管理软件包，我们之前使用过 dep，但是没有非常仔细的使用过。

dep 主要有 4 个命令：

  • dep init
  • dep status
  • dep ensure
  • dep check

当你有一个全新的项目时，请使用 dep init 来构建你的依赖，执行后，会在项目中生成 Gopkg.lock、Gopkg.toml，同时所有的依赖包都在 vendor 下面。

但是通常，中国特色的开发基本上基于开源项目，而开源项目因为种种原因依赖的 go package 会非常的旧，导致出现一系列问题，我们就需要使用 dep ensure -v 来升级这些旧的依赖包。

# gpushare-scheduler-extender 的 Gopkg.toml 节选
[[constraint]]
  name = "k8s.io/kubernetes"
  version = "v1.11.2"

[[constraint]]
  name = "k8s.io/apimachinery"
  branch = "release-1.11"

[[constraint]]
  name = "k8s.io/client-go"
  version = "~v8.0.0"

[[override]]
  name = "k8s.io/api"
  version = "kubernetes-1.11.2"

我们需要将其升级到 1.15.0

升级详解

总的来说，非常的简单：

  1. 修改 Gopkg.toml
  2. 执行 dep ensure -v 升级依赖
  3. 执行 dep status 查看依赖

原始 Gopkg.toml

[root@localhost gpushare-scheduler-extender]# cat Gopkg.toml
# Dependency rules: constraints and overrides allow the user to specify which versions of dependencies are acceptable, and where they should be retrieved from.
# 你依赖哪个版本的 go package？从哪里能获得到？
# Dependency rules: [[constraint]] and [[override]]
[[constraint]]
  # name 就是 go package 的名称（go get 能够下载的地址）
  name = "k8s.io/kubernetes"
  # 下面 3 选 1
  # version = "1.0.0"   github 项目中的 tag
  # branch = "master"   github 中的 branch
  # revision = "abc123" github 中的 commit
  version = "v1.11.2"
  # 下载的额外来源，以这个为准，一般不用
  # source = "https://github.com/myfork/package.git"

  # 没用过，不知道
  #[metadata]
  #key1 = "XXX"
  #system1-data = "value that is used by a system"
  #system2-data = "value that is used by another system"


[[constraint]]
  name = "k8s.io/apimachinery"
  branch = "release-1.11"

[[constraint]]
  name = "k8s.io/client-go"
  # version 的规则
  * `=`: equal
  * `!=`: not equal
  * `>`: greater than
  * `<`: less than
  * `>=`: greater than or equal to
  * `<=`: less than or equal to
  * `-`: literal range. E.g., 1.2 - 1.4.5 is equivalent to >= 1.2, <= 1.4.5
  * `~`: minor range. E.g., ~1.2.3 is equivalent to >= 1.2.3, < 1.3.0
  * `^`: major range. E.g., ^1.2.3 is equivalent to >= 1.2.3, < 2.0.0
  * `[xX*]`: wildcard. E.g., 1.2.x is equivalent to >= 1.2.0, < 1.3.0
  version = "~v8.0.0"

# constraint 和 override 的区别
# 个人的理解是，constraint 填写的项目中，会依赖其他项目，这些所有的相关性依赖都必须遵守
# 如果有两个 constraint 中依赖同一个项目，但是版本不同，则需要使用 override 明确这个依赖的版本是什么
[[override]]
  name = "k8s.io/api"
  version = "kubernetes-1.11.2"

[[override]]
  name = "github.com/gregjones/httpcache"
  revision = "787624de3eb7bd915c329cba748687a3b22666a6"

[[override]]
  name = "golang.org/x/time"
  revision = "f51c12702a4d776e4c1fa9b0fabab841babae631"

[[override]]
  name = "github.com/docker/docker"
  revision = "4f3616fb1c112e206b88cb7a9922bf49067a7756"

[[override]]
  name = "github.com/docker/distribution"
  revision = "edc3ab29cdff8694dd6feb85cfeb4b5f1b38ed9c"

# 排除掉一些 packages，全局性的
# https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md#prune
[prune]
  go-tests = true
  unused-packages = true

# required and ignored
# https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md#package-graph-rules-required-and-ignored
# 我个人的理解是，我们依赖一个项目（project），但是往往只依赖这个项目中的几个 packages，而不是所有，因此，可以指定我要依赖这个项目的哪个package，或者哪个 packages 我是肯定不需要的

执行原始 dep ensure -v

[root@localhost gpushare-scheduler-extender]# dep ensure -v
# Gopkg.lock is out of sync with Gopkg.toml and project imports:
k8s.io/apimachinery/pkg/labels: imported or required, but missing from Gopkg.lock's input-imports

Root project is "github.com/AliyunContainerService/gpushare-scheduler-extender"
 7 transitively valid internal packages
 19 external packages imported from 6 projects
(0)   ✓ select (root)
(1)	? attempt github.com/comail/colog with 1 pkgs; at least 1 versions to try
(1)	    try github.com/comail/colog@master
(1)	✓ select github.com/comail/colog@master w/1 pkgs
(2)	? attempt github.com/julienschmidt/httprouter with 1 pkgs; at least 1 versions to try
(2)	    try github.com/julienschmidt/httprouter@v1.2.0
(2)	✓ select github.com/julienschmidt/httprouter@v1.2.0 w/1 pkgs
(3)	? attempt k8s.io/api with 1 pkgs; at least 1 versions to try
(3)	    try k8s.io/api@kubernetes-1.11.2
(3)	✓ select k8s.io/api@kubernetes-1.11.2 w/1 pkgs
(4)	? attempt github.com/gogo/protobuf with 2 pkgs; at least 1 versions to try
(4)	    try github.com/gogo/protobuf@v1.1.1
(4)	✓ select github.com/gogo/protobuf@v1.1.1 w/2 pkgs
(5)	? attempt k8s.io/apimachinery with 6 pkgs; at least 1 versions to try
(5)	    try k8s.io/apimachinery@release-1.11
(5)	✓ select k8s.io/apimachinery@release-1.11 w/22 pkgs
(6)	? attempt github.com/google/gofuzz with 1 pkgs; at least 1 versions to try
(6)	    try github.com/google/gofuzz@master
(6)	✓ select github.com/google/gofuzz@master w/1 pkgs
(7)	? attempt golang.org/x/net with 1 pkgs; at least 1 versions to try
(7)	    try golang.org/x/net@master
(7)	✓ select golang.org/x/net@master w/4 pkgs
(8)	? revisit k8s.io/apimachinery to add 4 pkgs
(8)	  ✓ include 11 more pkgs from k8s.io/apimachinery@release-1.11
(8)	? attempt k8s.io/client-go with 9 pkgs; at least 1 versions to try
(9)	    try k8s.io/client-go@v8.0.0
(9)	✓ select k8s.io/client-go@v8.0.0 w/124 pkgs
(9)	? revisit k8s.io/api to add 28 pkgs
(10)    ✓ include 29 more pkgs from k8s.io/api@kubernetes-1.11.2
(9)	? attempt github.com/googleapis/gnostic with 1 pkgs; at least 1 versions to try
(11)      try github.com/googleapis/gnostic@v0.2.0
(11)  ✓ select github.com/googleapis/gnostic@v0.2.0 w/3 pkgs
(10)  ? revisit k8s.io/apimachinery to add 18 pkgs
(12)    ✓ include 40 more pkgs from k8s.io/apimachinery@release-1.11
(10)  ? attempt github.com/davecgh/go-spew with 1 pkgs; at least 1 versions to try
(13)      try github.com/davecgh/go-spew@v1.1.1
(13)  ✓ select github.com/davecgh/go-spew@v1.1.1 w/1 pkgs
(11)  ? attempt github.com/ghodss/yaml with 1 pkgs; at least 1 versions to try
(14)      try github.com/ghodss/yaml@v1.0.0
(14)  ✓ select github.com/ghodss/yaml@v1.0.0 w/1 pkgs
(12)  ? attempt github.com/golang/groupcache with 1 pkgs; at least 1 versions to try
(15)      try github.com/golang/groupcache@master
(15)  ✓ select github.com/golang/groupcache@master w/1 pkgs
(13)  ? attempt github.com/peterbourgon/diskv with 1 pkgs; at least 1 versions to try
(16)      try github.com/peterbourgon/diskv@v2.0.1
(16)  ✓ select github.com/peterbourgon/diskv@v2.0.1 w/1 pkgs
(14)  ? attempt github.com/google/btree with 1 pkgs; at least 1 versions to try
(17)      try github.com/google/btree@master
(17)  ✓ select github.com/google/btree@master w/1 pkgs
(15)  ? attempt k8s.io/kubernetes with 1 pkgs; at least 1 versions to try
(18)      try k8s.io/kubernetes@v1.12.1
(18)  ✓ select k8s.io/kubernetes@v1.12.1 w/1 pkgs
(16)  ? revisit k8s.io/client-go to add 1 pkgs
(19)    ✓ include 14 more pkgs from k8s.io/client-go@v8.0.0
(16)  ? attempt github.com/json-iterator/go with 1 pkgs; at least 1 versions to try
(20)      try github.com/json-iterator/go@v1.1.5
(20)  ✓ select github.com/json-iterator/go@v1.1.5 w/1 pkgs
(17)  ? attempt github.com/golang/protobuf with 2 pkgs; at least 1 versions to try
(21)      try github.com/golang/protobuf@v1.2.0
(21)  ✓ select github.com/golang/protobuf@v1.2.0 w/5 pkgs
(18)  ? attempt github.com/spf13/pflag with 1 pkgs; at least 1 versions to try
(22)      try github.com/spf13/pflag@v1.0.3
(22)  ✓ select github.com/spf13/pflag@v1.0.3 w/1 pkgs
(19)  ? attempt github.com/modern-go/concurrent with 1 pkgs; at least 1 versions to try
(23)      try github.com/modern-go/concurrent@1.0.3
(23)  ✓ select github.com/modern-go/concurrent@1.0.3 w/1 pkgs
(20)  ? attempt github.com/gregjones/httpcache with 2 pkgs; at least 1 versions to try
(24)      try github.com/gregjones/httpcache@787624de3eb7bd915c329cba748687a3b22666a6
(24)  ✓ select github.com/gregjones/httpcache@787624de3eb7bd915c329cba748687a3b22666a6 w/2 pkgs
(21)  ? attempt github.com/petar/GoLLRB with 1 pkgs; at least 1 versions to try
(25)      try github.com/petar/GoLLRB@master
(25)  ✓ select github.com/petar/GoLLRB@master w/1 pkgs
(22)  ? revisit github.com/golang/protobuf to add 1 pkgs
(26)    ✓ include 1 more pkgs from github.com/golang/protobuf@v1.2.0
(22)  ? attempt golang.org/x/text with 3 pkgs; at least 1 versions to try
(27)      try golang.org/x/text@v0.3.0
(27)  ✓ select golang.org/x/text@v0.3.0 w/14 pkgs
(23)  ? attempt github.com/modern-go/reflect2 with 1 pkgs; at least 1 versions to try
(28)      try github.com/modern-go/reflect2@1.0.1
(28)  ✓ select github.com/modern-go/reflect2@1.0.1 w/1 pkgs
(24)  ? attempt golang.org/x/crypto with 1 pkgs; at least 1 versions to try
(29)      try golang.org/x/crypto@release-branch.go1.11
(29)  ✓ select golang.org/x/crypto@release-branch.go1.11 w/1 pkgs
(25)  ? attempt github.com/golang/glog with 1 pkgs; at least 1 versions to try
(30)      try github.com/golang/glog@master
(30)  ✓ select github.com/golang/glog@master w/1 pkgs
(26)  ? attempt github.com/hashicorp/golang-lru with 1 pkgs; at least 1 versions to try
(31)      try github.com/hashicorp/golang-lru@v0.5.0
(31)  ✓ select github.com/hashicorp/golang-lru@v0.5.0 w/2 pkgs
(27)  ? attempt github.com/imdario/mergo with 1 pkgs; at least 1 versions to try
(32)      try github.com/imdario/mergo@v0.3.6
(32)  ✓ select github.com/imdario/mergo@v0.3.6 w/1 pkgs
(28)  ? attempt golang.org/x/sys with 2 pkgs; at least 1 versions to try
(33)      try golang.org/x/sys@release-branch.go1.11
(33)  ✓ select golang.org/x/sys@release-branch.go1.11 w/2 pkgs
(29)  ? attempt golang.org/x/time with 1 pkgs; at least 1 versions to try
(34)      try golang.org/x/time@f51c12702a4d776e4c1fa9b0fabab841babae631
(34)  ✓ select golang.org/x/time@f51c12702a4d776e4c1fa9b0fabab841babae631 w/1 pkgs
(30)  ? revisit golang.org/x/net to add 1 pkgs
(35)    ✓ include 1 more pkgs from golang.org/x/net@master
(30)  ? attempt gopkg.in/inf.v0 with 1 pkgs; at least 1 versions to try
(36)      try gopkg.in/inf.v0@v0.9.1
(36)  ✓ select gopkg.in/inf.v0@v0.9.1 w/1 pkgs
(31)  ? attempt gopkg.in/yaml.v2 with 1 pkgs; at least 1 versions to try
(37)      try gopkg.in/yaml.v2@v2.2.1
(37)  ✓ select gopkg.in/yaml.v2@v2.2.1 w/1 pkgs
(32)  ? attempt k8s.io/kube-openapi with 1 pkgs; at least 1 versions to try
(38)      try k8s.io/kube-openapi@feature-serverside-apply
(38)  ✓ select k8s.io/kube-openapi@feature-serverside-apply w/1 pkgs
  ✓ found solution with 250 packages from 32 projects

Solver wall times by segment:
         b-list-pkgs: 2.689677652s
              b-gmal:   2.0545678s
  b-deduce-proj-root: 1.040181941s
     b-source-exists: 218.179794ms
             satisfy: 136.320187ms
         select-atom: 133.297563ms
    b-rev-present-in:   6.873667ms
            new-atom:   3.712946ms
         select-root:    982.554µs
               other:    144.423µs
            add-atom:     86.194µs

  TOTAL: 6.284024721s

(1/32) Wrote gopkg.in/inf.v0@v0.9.1
(2/32) Wrote github.com/golang/glog@master
(3/32) Wrote gopkg.in/yaml.v2@v2.2.1
(4/32) Wrote github.com/google/gofuzz@master
(5/32) Wrote golang.org/x/time@f51c12702a4d776e4c1fa9b0fabab841babae631
(6/32) Wrote github.com/peterbourgon/diskv@v2.0.1
(7/32) Wrote github.com/modern-go/concurrent@1.0.3
(8/32) Wrote github.com/julienschmidt/httprouter@v1.2.0
(9/32) Wrote github.com/gregjones/httpcache@787624de3eb7bd915c329cba748687a3b22666a6
(10/32) Wrote github.com/modern-go/reflect2@1.0.1
(11/32) Wrote github.com/davecgh/go-spew@v1.1.1
(12/32) Wrote github.com/hashicorp/golang-lru@v0.5.0
(13/32) Wrote github.com/ghodss/yaml@v1.0.0
(14/32) Wrote github.com/imdario/mergo@v0.3.6
(15/32) Wrote github.com/comail/colog@master
(16/32) Wrote github.com/google/btree@master
(17/32) Wrote github.com/golang/groupcache@master
(18/32) Wrote k8s.io/kube-openapi@feature-serverside-apply
(19/32) Wrote github.com/spf13/pflag@v1.0.3
(20/32) Wrote github.com/petar/GoLLRB@master
(21/32) Wrote github.com/json-iterator/go@v1.1.5
(22/32) Wrote github.com/golang/protobuf@v1.2.0
(23/32) Wrote golang.org/x/sys@release-branch.go1.11
(24/32) Wrote k8s.io/api@kubernetes-1.11.2
(25/32) Wrote k8s.io/apimachinery@release-1.11
(26/32) Wrote golang.org/x/crypto@release-branch.go1.11
(27/32) Wrote github.com/googleapis/gnostic@v0.2.0
(28/32) Wrote golang.org/x/net@master
(29/32) Wrote k8s.io/client-go@v8.0.0
(30/32) Wrote golang.org/x/text@v0.3.0
(31/32) Wrote github.com/gogo/protobuf@v1.1.1
(32/32) Wrote k8s.io/kubernetes@v1.12.1

执行原始 dep status

[root@localhost gpushare-scheduler-extender]# dep status
PROJECT                              CONSTRAINT                       VERSION                          REVISION  LATEST   PKGS USED
github.com/comail/colog              branch master                    branch master                    fba8e7b   fba8e7b  1
github.com/davecgh/go-spew           v1.1.1                           v1.1.1                           8991bc2   v1.1.1   1
github.com/ghodss/yaml               v1.0.0                           v1.0.0                           0ca9ea5   v1.0.0   1
github.com/gogo/protobuf             v1.1.1                           v1.1.1                           636bf03   v1.1.1   2
github.com/golang/glog               branch master                    branch master                    23def4e   23def4e  1
github.com/golang/groupcache         branch master                    branch master                    c65c006   611e8ac  1
github.com/golang/protobuf           v1.2.0                           v1.2.0                           aa810b6   v1.2.0   5
github.com/google/btree              branch master                    branch master                    4030bb1   479b5e8  1
github.com/google/gofuzz             branch master                    branch master                    24818f7   db92cf7  1
github.com/googleapis/gnostic        v0.2.0                           v0.2.0                           7c66326   v0.2.0   3
github.com/gregjones/httpcache       787624d (override)                                                787624d            2
github.com/hashicorp/golang-lru      v0.5.0                           v0.5.0                           20f1fb7   v0.5.0   2
github.com/imdario/mergo             v0.3.6                           v0.3.6                           9f23e2d   v0.3.6   1
github.com/json-iterator/go          v1.1.5                           v1.1.5                           1624edc   v1.1.5   1
github.com/julienschmidt/httprouter  v1.2.0                           v1.2.0                           348b672   v1.2.0   1
github.com/modern-go/concurrent      1.0.3                            1.0.3                            bacd9c7   1.0.3    1
github.com/modern-go/reflect2        1.0.1                            1.0.1                            4b7aa43   1.0.1    1
github.com/petar/GoLLRB              branch master                    branch master                    53be0d3   33fb24c  1
github.com/peterbourgon/diskv        v2.0.1                           v2.0.1                           5f041e8   v2.0.1   1
github.com/spf13/pflag               v1.0.3                           v1.0.3                           298182f   v1.0.3   1
golang.org/x/crypto                  branch release-branch.go1.11     branch release-branch.go1.11     56440b8   56440b8  1
golang.org/x/net                     branch master                    branch master                    9b4f9f5   daa7c04  5
golang.org/x/sys                     branch release-branch.go1.11     branch release-branch.go1.11     98c5dad   98c5dad  2
golang.org/x/text                    v0.3.0                           v0.3.0                           f21a4df   v0.3.0   14
golang.org/x/time                    f51c127 (override)                                                f51c127            1
gopkg.in/inf.v0                      v0.9.1                           v0.9.1                           d2d2541   v0.9.1   1
gopkg.in/yaml.v2                     v2.2.1                           v2.2.1                           5420a8b   v2.2.1   1
k8s.io/api                           kubernetes-1.11.2 (override)     kubernetes-1.11.2                2d6f90a            29
k8s.io/apimachinery                  branch release-1.11              branch release-1.11              def12e6   c182ff3  41
k8s.io/client-go                     ~8.0.0                           v8.0.0                           7d04d0e   v8.0.0   124
k8s.io/kube-openapi                  branch feature-serverside-apply  branch feature-serverside-apply  f442ecb   f442ecb  1
k8s.io/kubernetes                    ^1.11.2                          v1.12.1                          4ed3216   v1.16.2  1

修改 Gopkg.toml

我们可以看到，k8s.io/XXX 使用的都是 1.11 的内容。通过上文对 Gopkg.toml 的分析，我们应该如何修改呢？

[[constraint]]
  # https://github.com/kubernetes/client-go
  name = "k8s.io/client-go"
  version = "=kubernetes-1.15.0"
  source = "https://github.com/kubernetes/client-go.git"

[[constraint]]
  # https://github.com/kubernetes/api
  name = "k8s.io/api"
  version = "=kubernetes-1.15.0"
  source = "https://github.com/kubernetes/api.git"

[[constraint]]
  # https://github.com/kubernetes/apimachinery
  name = "k8s.io/apimachinery"
  version = "=kubernetes-1.15.0"
  source = "https://github.com/kubernetes/apimachinery.git"

[[constraint]]
  # https://github.com/kubernetes/kubernetes
  name = "k8s.io/kubernetes"
  version = "=v1.15.0"
  source = "https://github.com/kubernetes/kubernetes.git"

[[override]]
  name = "github.com/gregjones/httpcache"
  revision = "787624de3eb7bd915c329cba748687a3b22666a6"

[[override]]
  name = "golang.org/x/time"
  revision = "f51c12702a4d776e4c1fa9b0fabab841babae631"

[[override]]
  name = "github.com/docker/docker"
  revision = "4f3616fb1c112e206b88cb7a9922bf49067a7756"

[[override]]
  name = "github.com/docker/distribution"
  revision = "edc3ab29cdff8694dd6feb85cfeb4b5f1b38ed9c"

[prune]
  go-tests = true
  unused-packages = true

执行修改后的 Gopkg.toml

[root@localhost gpushare-scheduler-extender]# dep ensure -v

...
kubernetes-1.14.9-beta.0: Could not introduce k8s.io/api (from https://github.com/kubernetes/api.git)@kubernetes-1.14.9-beta.0, as it is not allowed by constraint =kubernetes-1.15.0 from project github.com/AliyunContainerService/gpushare-scheduler-extender.
        kubernetes-1.15.0: Could not introduce k8s.io/api (from https://github.com/kubernetes/api.git)@kubernetes-1.15.0, as it is not allowed by constraint =kubernetes-1.15.0 from project github.com/AliyunContainerService/gpushare-scheduler-extender.
        kubernetes-1.15.0-alpha.0: Could not introduce k8s.io/api (from https://github.com/kubernetes/api.git)@kubernetes-1.15.0-alpha.0, as it is not allowed by constraint =kubernetes-1.15.0 from project github.com/AliyunContainerService/gpushare-scheduler-extender.
...

明明 k8s.io/api (from https://github.com/kubernetes/api.git)@kubernetes-1.15.0 能够找到，为什么就不能匹配到 constraint =kubernetes-1.15.0? 这困扰了我很久很久，我仔细阅读了官方文档 发现举的例子中，版本都是 2.0.1、1.3.8 等，都是以数字开头的，这就引起了我的警惕，是不是不支持以字符开头的版本？

You might, for example, include a rule that specifies version = "=2.0.0" to pin a dependency to version 2.0.0, or constrain to minor releases with: version = "~2.1.0". Refer to the semver library documentation for more info.
From https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md

查看 semver 项目的说明，在 README.md 中有提示：Optionally work with a v prefix，大胆的推测，version 一定不能支持形如 kubernetes-1.15.0 的版本。证明如下：

# 下载 semver
(nni) root@ubuntu:/home/lihao/codes/testgo# go get https://github.com/Masterminds/semver.git

# 编写测试代码
(nni) root@ubuntu:/home/lihao/codes/testgo# cat test-semver.go
package main

import (
	"fmt"
	"sort"

	"github.com/Masterminds/semver"
)

func main() {
    // 使用 kubernetes-1.14.0 这种 version
	raw := []string{"kubernetes-1.15.0", "kubernetes-1.14.0", "kubernetes-1.16.4"}
	vs := make([]*semver.Version, len(raw))
	for i, r := range raw {
		v, err := semver.NewVersion(r)
		if err != nil {
			fmt.Printf("Error parsing version: %s", err)
		}

		vs[i] = v
	}

	sort.Sort(semver.Collection(vs))
	fmt.Println(vs)
}

# 编译
(nni) root@ubuntu:/home/lihao/codes/testgo# go build /home/lihao/codes/testgo/test-semver.go

# 执行，果然报错！
(nni) root@ubuntu:/home/lihao/codes/testgo# ./test-semver
Error parsing version: Invalid Semantic VersionError parsing version: Invalid Semantic VersionError parsing version: Invalid Semantic Versionpanic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4c1bda]

goroutine 1 [running]:
github.com/Masterminds/semver.(*Version).Compare(0x0, 0x0, 0x5c7170)
	/opt/ml/bin/work/src/github.com/Masterminds/semver/version.go:387 +0x3a
github.com/Masterminds/semver.(*Version).LessThan(...)
	/opt/ml/bin/work/src/github.com/Masterminds/semver/version.go:362
github.com/Masterminds/semver.Collection.Less(...)
	/opt/ml/bin/work/src/github.com/Masterminds/semver/collection.go:17
sort.insertionSort(0x51fc00, 0xc00006e2a0, 0x0, 0x3)
	/usr/local/go/src/sort/sort.go:27 +0xc4
sort.quickSort(0x51fc00, 0xc00006e2a0, 0x0, 0x3, 0x4)
	/usr/local/go/src/sort/sort.go:209 +0x201
sort.Sort(0x51fc00, 0xc00006e2a0)
	/usr/local/go/src/sort/sort.go:218 +0x79
main.main()
	/home/lihao/codes/testgo/test-semver.go:22 +0x135

如果将 raw := []string{"kubernetes-1.15.0", "kubernetes-1.14.0", "kubernetes-1.16.4"} -> raw := []string{"v1.15.0", "v1.14.0", "v1.16.4"}，确实可以正常运行：

(nni) root@ubuntu:/home/lihao/codes/testgo# ./test-semver
[1.14.0 1.15.0 1.16.4]

折腾我一周的问题，终于得到了解决，原来是这样....换成了下面的内容，再执行 dep ensure -v 将无报错：

[[constraint]]
  # https://github.com/kubernetes/client-go
  name = "k8s.io/client-go"
  version = "kubernetes-1.15.0"
  # 因为 kubernetes-1.15.0 = v12.0.0 因此：
  # version = "=v12.0.0" 也不会出错！
  source = "https://github.com/kubernetes/client-go.git"

[[constraint]]
  # https://github.com/kubernetes/api
  name = "k8s.io/api"
  version = "kubernetes-1.15.0"
  source = "https://github.com/kubernetes/api.git"

[[constraint]]
  # https://github.com/kubernetes/apimachinery
  name = "k8s.io/apimachinery"
  version = "kubernetes-1.15.0"
  source = "https://github.com/kubernetes/apimachinery.git"

[[constraint]]
  # https://github.com/kubernetes/kubernetes
  name = "k8s.io/kubernetes"
  # 或者 version = "=v1.15.0" 也是可以的
  version = "v1.15.0"
  source = "https://github.com/kubernetes/kubernetes.git"

[[override]]
  name = "github.com/gregjones/httpcache"
  revision = "787624de3eb7bd915c329cba748687a3b22666a6"

[[override]]
  name = "golang.org/x/time"
  revision = "f51c12702a4d776e4c1fa9b0fabab841babae631"

[[override]]
  name = "github.com/docker/docker"
  revision = "4f3616fb1c112e206b88cb7a9922bf49067a7756"

[[override]]
  name = "github.com/docker/distribution"
  revision = "edc3ab29cdff8694dd6feb85cfeb4b5f1b38ed9c"

[prune]
  go-tests = true
  unused-packages = true

尝试过删去 k8s.io/api、k8s.io/apimachinery 会报错

[[constraint]]
  # https://github.com/kubernetes/client-go
  name = "k8s.io/client-go"
  version = "=v12.0.0"
  #version = "kubernetes-1.15.0"
  source = "https://github.com/kubernetes/client-go.git"

[[constraint]]
  # https://github.com/kubernetes/kubernetes
  name = "k8s.io/kubernetes"
  version = "=v1.15.0"
  source = "https://github.com/kubernetes/kubernetes.git"

[[override]]
  name = "github.com/gregjones/httpcache"
  revision = "787624de3eb7bd915c329cba748687a3b22666a6"

[[override]]
  name = "golang.org/x/time"
  revision = "f51c12702a4d776e4c1fa9b0fabab841babae631"

[[override]]
  name = "github.com/docker/docker"
  revision = "4f3616fb1c112e206b88cb7a9922bf49067a7756"

[[override]]
  name = "github.com/docker/distribution"
  revision = "edc3ab29cdff8694dd6feb85cfeb4b5f1b38ed9c"

[prune]
  go-tests = true
  unused-packages = true

报错内容节选

# 对于 k8s.io/api，dep 工具默认给你选了个 k8s.io/api@release-1.10
? attempt k8s.io/api with 1 pkgs; 283 versions to try
(2)         try k8s.io/api@master
(2)     ✗   unable to update checked out version: fatal: reference is not a tree: 0171b7c15da1e52c17b83d3e2524d0100c080d5d
(2)       : command failed: [git checkout 0171b7c15da1e52c17b83d3e2524d0100c080d5d]: exit status 128
(2)         try k8s.io/api@release-1.10
(2)     ✓ select k8s.io/api@release-1.10 w/1 pkgs
(3)     ? attempt github.com/julienschmidt/httprouter with 1 pkgs; 16 versions to try
(3)         try github.com/julienschmidt/httprouter@v1.3.0
(3)     ✓ select github.com/julienschmidt/httprouter@v1.3.0 w/1 pkgs
(4)     ? attempt k8s.io/apimachinery with 4 pkgs; 308 versions to try
(4)         try k8s.io/apimachinery@master
(4)     ✗   unable to update checked out version: fatal: reference is not a tree: fee41ff082ed4e5e059da79fa0a8fb0713c67710
(4)       : command failed: [git checkout fee41ff082ed4e5e059da79fa0a8fb0713c67710]: exit status 128
(4)         try k8s.io/apimachinery@f2f3a405f61d
(4)     ✗   unable to update checked out version: fatal: reference is not a tree: c18f71bf29474a53323469937882e0ebb873cd8c
(4)       : command failed: [git checkout c18f71bf29474a53323469937882e0ebb873cd8c]: exit status 128
(4)         try k8s.io/apimachinery@release-1.10
(4)     ✓ select k8s.io/apimachinery@release-1.10 w/10 pkgs

# 选择 client-go v12.0.0，依赖 k8s.io/api@release-1.10 是不可以的，因此需要改变 client-go 版本，但是因为我们定死了，必须用 v12.0.0，肯定找不到合适的版本，因此出错。

# 这告诉我们一个道理，项目的重要依赖，一定要写全，否则它会给你乱选，导致问题
try k8s.io/client-go (from https://github.com/kubernetes/client-go.git)@v12.0.0
(11)  ✗   k8s.io/client-go (from https://github.com/kubernetes/client-go.git)@v12.0.0 depping on k8s.io/api at release-1.10 has problem subpkg(s):      k8s.io/api/auditregistration/v1alpha1 is missing        k8s.io/api/autoscaling/v2beta2 is missing       k8s.io/api/coordination/v1 is missing   k8s.io/api/coordination/v1beta1 is missing      k8s.io/api/networking/v1beta1 is missing        k8s.io/api/node/v1alpha1 is missing     k8s.io/api/node/v1beta1 is missing      k8s.io/api/scheduling/v1 is missing     k8s.io/api/scheduling/v1beta1 is missing
(10)      try k8s.io/client-go (from https://github.com/kubernetes/client-go.git)@v11.0.0
(11)  ✗   k8s.io/client-go (from https://github.com/kubernetes/client-go.git)@v11.0.0 not allowed by constraint 12.0.0:
(11)      12.0.0 from (root)
(10)      try k8s.io/client-go (from https://github.com/kubernetes/client-go.git)@v10.0.0
(11)  ✗   k8s.io/client-go (from https://github.com/kubernetes/client-go.git)@v10.0.0 not allowed by constraint 12.0.0:

执行 dep status

[root@localhost gpushare-scheduler-extender]# dep status
PROJECT                              CONSTRAINT                       VERSION                          REVISION  LATEST   PKGS USED
github.com/comail/colog              branch master                    branch master                    fba8e7b   fba8e7b  1
github.com/davecgh/go-spew           v1.1.1                           v1.1.1                           8991bc2   v1.1.1   1
github.com/gogo/protobuf             v1.3.1                           v1.3.1                           5628607   v1.3.1   2
github.com/golang/groupcache         branch master                    branch master                    611e8ac   611e8ac  1
github.com/golang/protobuf           v1.3.2                           v1.3.2                           6c65a55   v1.3.2   5
github.com/google/go-cmp             v0.3.1                           v0.3.1                           2d0692c   v0.3.1   5
github.com/google/gofuzz             v1.0.0                           v1.0.0                           f140a64   v1.0.0   1
github.com/googleapis/gnostic        v0.3.1                           v0.3.1                           ab0dd09   v0.3.1   3
github.com/hashicorp/golang-lru      v0.5.3                           v0.5.3                           7f827b3   v0.5.3   2
github.com/imdario/mergo             v0.3.8                           v0.3.8                           1afb360   v0.3.8   1
github.com/json-iterator/go          v1.1.8                           v1.1.8                           03217c3   v1.1.8   1
github.com/julienschmidt/httprouter  v1.3.0                           v1.3.0                           4eec211   v1.3.0   1
github.com/modern-go/concurrent      1.0.3                            1.0.3                            bacd9c7   1.0.3    1
github.com/modern-go/reflect2        1.0.1                            1.0.1                            4b7aa43   1.0.1    1
github.com/spf13/pflag               v1.0.5                           v1.0.5                           2e9d26c   v1.0.5   1
golang.org/x/crypto                  branch release-branch.go1.11     branch release-branch.go1.11     56440b8   56440b8  1
golang.org/x/net                     branch release-branch.go1.10     branch release-branch.go1.10     0ed95ab   0ed95ab  6
golang.org/x/oauth2                  branch master                    branch master                    0f29369   0f29369  2
golang.org/x/sys                     branch release-branch.go1.11     branch release-branch.go1.11     98c5dad   98c5dad  2
golang.org/x/text                    v0.3.2                           v0.3.2                           342b2e1   v0.3.2   16
golang.org/x/time                    f51c127 (override)                                                f51c127            1
google.golang.org/appengine          v1.6.5                           v1.6.5                           971852b   v1.6.5   7
gopkg.in/inf.v0                      v0.9.1                           v0.9.1                           d2d2541   v0.9.1   1
gopkg.in/yaml.v2                     v2.2.5                           v2.2.5                           f90ceb4   v2.2.5   1
k8s.io/api                           kubernetes-1.15.0                kubernetes-1.15.0                7cf5895            36
k8s.io/apimachinery                  kubernetes-1.15.0                kubernetes-1.15.0                1799e75            42
k8s.io/client-go                     12.0.0                           v12.0.0                          78d2af7   v12.0.0  148
k8s.io/klog                          v1.0.0                           v1.0.0                           2ca9ad3   v1.0.0   1
k8s.io/kube-openapi                  branch feature-serverside-apply  branch feature-serverside-apply  f442ecb   f442ecb  1
k8s.io/kubernetes                    1.15.0                           v1.15.0                          e8462b5   v1.15.0  2
k8s.io/utils                         branch master                    branch master                    2b95a09   2b95a09  3
sigs.k8s.io/yaml                     v1.1.0                           v1.1.0                           fd68e98   v1.1.0   1

总结

  1. go 项目使用的版本，最好是 1.x.y 或者 v1.x.y，而不要是什么 kubernetes-1.x.y 这样会导致很大的问题
  2. 如果你依赖的项目中，真的使用了形如 kubernetes-1.x.y 的 tag，你有如下两个办法：
    • 使用 branch
    • 依旧使用 version，但是不要加任何 =、>= 符号，应该写成 version = "kubernetes-1.x.y"，这表明 version 必须是 kubernetes-1.x.y
  3. 把你知道的尽可能多的依赖写入 Gopkg.toml 文件中，否则 dep ensure 会帮你瞎选（尤其是该项目的 version 命名不规范时）
posted on 2019-11-11 15:14  silenceli  阅读(731)  评论(0)  编辑  收藏  举报

Powered by: 博客园 Copyright © 2024 silenceli
Powered by .NET 9.0 on Kubernetes