sigs.k8s.io controller-runtime——GetConfigOrDie()方法详解

1、概述

在使用controller-runtime框架进行kubernetes二次开发的时候，程序中通常都会使用GetConfigOrDie()方法获取集群配置以便与kubernetes集群进行连接，示例如下：

opts := ctrl.Options{
    Scheme: scheme,
    MetricsBindAddress: metricsAddr,
    LeaderElection: enableLeaderElection,
    Port: 9443,
}
 
mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), opts)

本文主要通过剖析GetConfigOrDie()方法来进行讲解controller-runtime框架的获取kubernetes集群认证配置文件的加载顺序。

2、 GetConfigOrDie()源码剖析

源码文件位置：sigs.k8s.io/controller-runtime/pkg/client/config/config.go

注意：本文中不特殊说明源码文件位置的代码片段都是sigs.k8s.io/controller-runtime/pkg/client/config/config.go文件,本文sigs.k8s.io/controller-runtime框架版本为v0.9.6。

// GetConfigOrDie creates a *rest.Config for talking to a Kubernetes apiserver.
// If --kubeconfig is set, will use the kubeconfig file at that location.  Otherwise will assume running
// in cluster and use the cluster provided kubeconfig.
//
// Will log an error and exit if there is an error creating the rest.Config.
func GetConfigOrDie() *rest.Config {
    config, err := GetConfig()
    if err != nil {
        log.Error(err, "unable to get kubeconfig")
        os.Exit(1)
    }
    return config
}

可以看到GetConfigOrDie()方法如同它的名字，假设获取*rest.Config失败便会以异常状态退出程序。

func GetConfig() (*rest.Config, error) {
    return GetConfigWithContext("")
}
 
func GetConfigWithContext(context string) (*rest.Config, error) {
    cfg, err := loadConfig(context)
    if err != nil {
        return nil, err
    }
 
    if cfg.QPS == 0.0 {
        cfg.QPS = 20.0
        cfg.Burst = 30.0
    }
 
    return cfg, nil
}

以上方法可以配置*rest.Config.QPS，访问kubernetes集群时的认证配置文件的加载顺序核心逻辑方法为loadConfig方法，逻辑如下：

var (
    kubeconfig string
    log        = logf.RuntimeLog.WithName("client").WithName("config")
)
 
func init() {
    // 启动程序时客户端传参kubeconfig
    flag.StringVar(&kubeconfig, "kubeconfig", "",
        "Paths to a kubeconfig. Only required if out-of-cluster.")
}
 
// loadConfig loads a REST Config as per the rules specified in GetConfig.
func loadConfig(context string) (*rest.Config, error) {
    // If a flag is specified with the config location, use that
    // 1、如果flag初始化了kubeconfig，则从kubeconfig中读取集群配置
    if len(kubeconfig) > 0 {
        return loadConfigWithContext("", &clientcmd.ClientConfigLoadingRules{ExplicitPath: kubeconfig}, context)
    }
 
    // If the recommended kubeconfig env variable is not specified,
    // try the in-cluster config.
    // 2、 否则从环境变量KUBECONFIG读取，若没有则从集群内部读取，这种场景适用于部署到kubernetes中的场景
    kubeconfigPath := os.Getenv(clientcmd.RecommendedConfigPathEnvVar)
    if len(kubeconfigPath) == 0 {
        if c, err := loadInClusterConfig(); err == nil {
            return c, nil
        }
    }
 
    // If the recommended kubeconfig env variable is set, or there
    // is no in-cluster config, try the default recommended locations.
    //
    // NOTE: For default config file locations, upstream only checks
    // $HOME for the user's home directory, but we can also try
    // os/user.HomeDir when $HOME is unset.
    //
    // TODO(jlanford): could this be done upstream?
    loadingRules := clientcmd.NewDefaultClientConfigLoadingRules()
    // 是否存在环境变量HOME
    if _, ok := os.LookupEnv("HOME"); !ok {
        u, err := user.Current()
        if err != nil {
            return nil, fmt.Errorf("could not get current user: %v", err)
        }
        // 3、如果通过KUBECONFIG环境变量和集群内部这两个方式都没有找到（包括从集群内部读取集群配置报错），则会读取默认配置$HOME/.kube/config，如果不存在环境变量HOME，则会读取配置文件user.HomeDir/.kube/config
        loadingRules.Precedence = append(loadingRules.Precedence, path.Join(u.HomeDir, clientcmd.RecommendedHomeDir, clientcmd.RecommendedFileName))
    }
 
    return loadConfigWithContext("", loadingRules, context)
}
 
func loadConfigWithContext(apiServerURL string, loader clientcmd.ClientConfigLoader, context string) (*rest.Config, error) {
    return clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
        loader,
        &clientcmd.ConfigOverrides{
            ClusterInfo: clientcmdapi.Cluster{
                Server: apiServerURL,
            },
            CurrentContext: context,
        }).ClientConfig()
}

接下来再看下从集群内部和通过环境变量（包括读取默认配置）获取集群配置的代码逻辑。

1）集群内部获取集群配置：

InClusterConfig方法,逻辑很简单：在Pod容器内，通过serviceaccout提供的token值和容器内部集群的环境变量生成*rest.Config对象。

InClusterConfig方法源码文件位置：k8s.io/client-go/rest/config.go

// InClusterConfig returns a config object which uses the service account
// kubernetes gives to pods. It's intended for clients that expect to be
// running inside a pod running on kubernetes. It will return ErrNotInCluster
// if called from a process not running in a kubernetes environment.
func InClusterConfig() (*Config, error) {
    const (
        tokenFile  = "/var/run/secrets/kubernetes.io/serviceaccount/token"
        rootCAFile = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
    )
    host, port := os.Getenv("KUBERNETES_SERVICE_HOST"), os.Getenv("KUBERNETES_SERVICE_PORT")
    if len(host) == 0 || len(port) == 0 {
        return nil, ErrNotInCluster
    }
 
    token, err := ioutil.ReadFile(tokenFile)
    if err != nil {
        return nil, err
    }
 
    tlsClientConfig := TLSClientConfig{}
 
    if _, err := certutil.NewPool(rootCAFile); err != nil {
        klog.Errorf("Expected to load root CA config from %s, but got err: %v", rootCAFile, err)
    } else {
        tlsClientConfig.CAFile = rootCAFile
    }
 
    return &Config{
        // TODO: switch to using cluster DNS.
        Host:            "https://" + net.JoinHostPort(host, port),
        TLSClientConfig: tlsClientConfig,
        BearerToken:     string(token),
        BearerTokenFile: tokenFile,
    }, nil
}

2）通过环境变量（包括读取默认配置）获取集群配置：

不管是flag初始化了kubeconfig，还是从环境变量KUBECONFIG读取，或者是读取默认配置（user.HomeDir/.kube/config或user.HomeDir/.kube/config) 都会实例化ClientConfigLoadingRules结构体对象。

本小节源码文件位置：k8s.io/client-go/tools/clientcmd/loader.go

type ClientConfigLoadingRules struct {
    ExplicitPath string
    Precedence   []string
 
    // MigrationRules is a map of destination files to source files.  If a destination file is not present, then the source file is checked.
    // If the source file is present, then it is copied to the destination file BEFORE any further loading happens.
    MigrationRules map[string]string
 
    // DoNotResolvePaths indicates whether or not to resolve paths with respect to the originating files.  This is phrased as a negative so
    // that a default object that doesn't set this will usually get the behavior it wants.
    DoNotResolvePaths bool
 
    // DefaultClientConfig is an optional field indicating what rules to use to calculate a default configuration.
    // This should match the overrides passed in to ClientConfig loader.
    DefaultClientConfig ClientConfig
 
    // WarnIfAllMissing indicates whether the configuration files pointed by KUBECONFIG environment variable are present or not.
    // In case of missing files, it warns the user about the missing files.
    WarnIfAllMissing bool
}　　

其中：

ExplicitPath: 表示kubeconfig的路径，优先级高于Precedence；
Precedence: 表示从KUBECONFIG等环境变量中获取到的路径；
MigrationRules: 表示旧路径到新路径的映射关系；
DoNotResolvePaths: 表示是否需要把相对路径转换成绝对路径；
DefaultClientConfig: 表示默认的配置。

创建ClientConfigLoadingRules结构体对象方法NewDefaultClientConfigLoadingRules()

const (
    RecommendedConfigPathFlag   = "kubeconfig"
    RecommendedConfigPathEnvVar = "KUBECONFIG"
    RecommendedHomeDir          = ".kube"
    RecommendedFileName         = "config"
    RecommendedSchemaName       = "schema"
)
 
var (
    RecommendedConfigDir  = filepath.Join(homedir.HomeDir(), RecommendedHomeDir)
    RecommendedHomeFile   = filepath.Join(RecommendedConfigDir, RecommendedFileName)
    RecommendedSchemaFile = filepath.Join(RecommendedConfigDir, RecommendedSchemaName)
)
 
// NewDefaultClientConfigLoadingRules returns a ClientConfigLoadingRules object with default fields filled in.  You are not required to
// use this constructor
func NewDefaultClientConfigLoadingRules() *ClientConfigLoadingRules {
    chain := []string{}
    warnIfAllMissing := false
 
    envVarFiles := os.Getenv(RecommendedConfigPathEnvVar)
        // 从环境变量KUBECONFIG获取集群配置
    if len(envVarFiles) != 0 {
        fileList := filepath.SplitList(envVarFiles)
        // prevent the same path load multiple times
        chain = append(chain, deduplicate(fileList)...)
        warnIfAllMissing = true
        // 从读取$HOME/kube/config文件中获取集群配置
    } else {
        chain = append(chain, RecommendedHomeFile)
    }
 
    return &ClientConfigLoadingRules{
        Precedence:       chain,
        MigrationRules:   currentMigrationRules(),
        WarnIfAllMissing: warnIfAllMissing,
    }
}

3、总结

controller-runtime框架通过GetConfigOrDie()方法获取kubernetes集群认证配置文件的加载顺序如下：

1、如果flag初始化了kubeconfig，则从kubeconfig中读取集群配置；
2、否则从环境变量KUBECONFIG读取，若没有则从集群内部读取，这种场景适用于部署到kubernetes中的场景，在Pod容器内，通过serviceaccout提供的token值和容器内部集群的环境变量生成*rest.Config对象。；
3、如果通过KUBECONFIG环境变量和集群内部这两个方式都没有找到（包括从集群内部读取集群配置报错），则会读取默认配置$HOME/.kube/config（如果不存在环境变量HOME，则会从user.HomeDir/.kube/config中读取集群配置）。