docker+Rancher+K3S
准备工作3台机器
192.168.88.85 部署Rancher
192.168.88.71 部署K3S server
192.168.88.72 部署K3S agent
1.首先有一个已经安装好docker的机器 192.168.88.85
2.安装Rancher
docker run -d -v /data/docker/rancher-server/var/lib/rancher/:/var/lib/rancher/ --restart=unless-stopped --privileged --name rancher-server -p 8082:80 -p 8443:443 rancher/rancher:stable
查看启动状态
浏览器输入访问地址 192.168.88.85:8082
继续访问
设置好密码
填好可以访问的地址
换成中文
安装K3S server 192.168.88.71
1下载k3s运行的依赖包
cd /
mkdir k3s
cd k3s
wget https://github.com/rancher/k3s/releases/download/v0.10.2/k3s-airgap-images-amd64.tar
wget https://github.com/rancher/k3s/releases/download/v0.10.2/k3s
wget https://github.com/rancher/k3s/blob/master/install.sh
2赋予权限
chmod 755 k3s chmod 755 install.sh chmod 755 k3s-airgap-images-amd64.tar
3修改install.sh文件，添加或设置如下
export INSTALL_K3S_SKIP_DOWNLOAD=true
4将所需镜像文件拷贝
cp k3s /usr/local/bin/k3s mkdir -p /var/lib/rancher/k3s/agent/images/ cp k3s-airgap-images-amd64.tar /var/lib/rancher/k3s/agent/images/
5docker导入镜像
docker load -i k3s-airgap-images-amd64.tar
注：如搭好k3s后，没有docker load -i操作，之后修改k3s.service，添加–docker --no-deploy traefik后，创建demo失败
6启动安装文件
./install.sh
install.sh内容(由于本人操作的手wget获取下来的不对,自己创建)
#!/bin/sh set -e set -o noglob # Usage: # curl ... | ENV_VAR=... sh - # or # ENV_VAR=... ./install.sh # # Example: # Installing a server without traefik: # curl ... | INSTALL_K3S_EXEC="--disable=traefik" sh - # Installing an agent to point at a server: # curl ... | K3S_TOKEN=xxx K3S_URL=https://server-url:6443 sh - # # Environment variables: # - K3S_* # Environment variables which begin with K3S_ will be preserved for the # systemd service to use. Setting K3S_URL without explicitly setting # a systemd exec command will default the command to "agent", and we # enforce that K3S_TOKEN or K3S_CLUSTER_SECRET is also set. # # - INSTALL_K3S_SKIP_DOWNLOAD # If set to true will not download k3s hash or binary. # # - INSTALL_K3S_FORCE_RESTART # If set to true will always restart the K3s service # # - INSTALL_K3S_SYMLINK # If set to 'skip' will not create symlinks, 'force' will overwrite, # default will symlink if command does not exist in path. # # - INSTALL_K3S_SKIP_ENABLE # If set to true will not enable or start k3s service. # # - INSTALL_K3S_SKIP_START # If set to true will not start k3s service. # # - INSTALL_K3S_VERSION # Version of k3s to download from github. Will attempt to download from the # stable channel if not specified. # # - INSTALL_K3S_COMMIT # Commit of k3s to download from temporary cloud storage. # * (for developer & QA use) # # - INSTALL_K3S_BIN_DIR # Directory to install k3s binary, links, and uninstall script to, or use # /usr/local/bin as the default # # - INSTALL_K3S_BIN_DIR_READ_ONLY # If set to true will not write files to INSTALL_K3S_BIN_DIR, forces # setting INSTALL_K3S_SKIP_DOWNLOAD=true # # - INSTALL_K3S_SYSTEMD_DIR # Directory to install systemd service and environment files to, or use # /etc/systemd/system as the default # # - INSTALL_K3S_EXEC or script arguments # Command with flags to use for launching k3s in the systemd service, if # the command is not specified will default to "agent" if K3S_URL is set # or "server" if not. The final systemd command resolves to a combination # of EXEC and script args ($@). # # The following commands result in the same behavior: # curl ... | INSTALL_K3S_EXEC="--disable=traefik" sh -s - # curl ... | INSTALL_K3S_EXEC="server --disable=traefik" sh -s - # curl ... | INSTALL_K3S_EXEC="server" sh -s - --disable=traefik # curl ... | sh -s - server --disable=traefik # curl ... | sh -s - --disable=traefik # # - INSTALL_K3S_NAME # Name of systemd service to create, will default from the k3s exec command # if not specified. If specified the name will be prefixed with 'k3s-'. # # - INSTALL_K3S_TYPE # Type of systemd service to create, will default from the k3s exec command # if not specified. # # - INSTALL_K3S_SELINUX_WARN # If set to true will continue if k3s-selinux policy is not found. # # - INSTALL_K3S_SKIP_SELINUX_RPM # If set to true will skip automatic installation of the k3s RPM. # # - INSTALL_K3S_CHANNEL_URL # Channel URL for fetching k3s download URL. # Defaults to 'https://update.k3s.io/v1-release/channels'. # # - INSTALL_K3S_CHANNEL # Channel to use for fetching k3s download URL. # Defaults to 'stable'. GITHUB_URL=https://github.com/k3s-io/k3s/releases STORAGE_URL=https://storage.googleapis.com/k3s-ci-builds DOWNLOADER= # --- helper functions for logs --- info() { echo '[INFO] ' "$@" } warn() { echo '[WARN] ' "$@" >&2 } fatal() { echo '[ERROR] ' "$@" >&2 exit 1 } # --- fatal if no systemd or openrc --- verify_system() { if [ -x /sbin/openrc-run ]; then HAS_OPENRC=true return fi if [ -d /run/systemd ]; then HAS_SYSTEMD=true return fi fatal 'Can not find systemd or openrc to use as a process supervisor for k3s' } # --- add quotes to command arguments --- quote() { for arg in "$@"; do printf '%s\n' "$arg" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/'/" done } # --- add indentation and trailing slash to quoted args --- quote_indent() { printf ' \\\n' for arg in "$@"; do printf '\t%s \\\n' "$(quote "$arg")" done } # --- escape most punctuation characters, except quotes, forward slash, and space --- escape() { printf '%s' "$@" | sed -e 's/\([][!#$%&()*;<=>?\_`{|}]\)/\\\1/g;' } # --- escape double quotes --- escape_dq() { printf '%s' "$@" | sed -e 's/"/\\"/g' } # --- ensures $K3S_URL is empty or begins with https://, exiting fatally otherwise --- verify_k3s_url() { case "${K3S_URL}" in "") ;; https://*) ;; *) fatal "Only https:// URLs are supported for K3S_URL (have ${K3S_URL})" ;; esac } # --- define needed environment variables --- setup_env() { # --- use command args if passed or create default --- case "$1" in # --- if we only have flags discover if command should be server or agent --- (-*|"") if [ -z "${K3S_URL}" ]; then CMD_K3S=server else if [ -z "${K3S_TOKEN}" ] && [ -z "${K3S_TOKEN_FILE}" ] && [ -z "${K3S_CLUSTER_SECRET}" ]; then fatal "Defaulted k3s exec command to 'agent' because K3S_URL is defined, but K3S_TOKEN, K3S_TOKEN_FILE or K3S_CLUSTER_SECRET is not defined." fi CMD_K3S=agent fi ;; # --- command is provided --- (*) CMD_K3S=$1 shift ;; esac verify_k3s_url CMD_K3S_EXEC="${CMD_K3S}$(quote_indent "$@")" # --- use systemd name if defined or create default --- if [ -n "${INSTALL_K3S_NAME}" ]; then SYSTEM_NAME=k3s-${INSTALL_K3S_NAME} else if [ "${CMD_K3S}" = server ]; then SYSTEM_NAME=k3s else SYSTEM_NAME=k3s-${CMD_K3S} fi fi # --- check for invalid characters in system name --- valid_chars=$(printf '%s' "${SYSTEM_NAME}" | sed -e 's/[][!#$%&()*;<=>?\_`{|}/[:space:]]/^/g;' ) if [ "${SYSTEM_NAME}" != "${valid_chars}" ]; then invalid_chars=$(printf '%s' "${valid_chars}" | sed -e 's/[^^]/ /g') fatal "Invalid characters for system name: ${SYSTEM_NAME} ${invalid_chars}" fi # --- use sudo if we are not already root --- SUDO=sudo if [ $(id -u) -eq 0 ]; then SUDO= fi # --- use systemd type if defined or create default --- if [ -n "${INSTALL_K3S_TYPE}" ]; then SYSTEMD_TYPE=${INSTALL_K3S_TYPE} else if [ "${CMD_K3S}" = server ]; then SYSTEMD_TYPE=notify else SYSTEMD_TYPE=exec fi fi # --- use binary install directory if defined or create default --- if [ -n "${INSTALL_K3S_BIN_DIR}" ]; then BIN_DIR=${INSTALL_K3S_BIN_DIR} else # --- use /usr/local/bin if root can write to it, otherwise use /opt/bin if it exists BIN_DIR=/usr/local/bin if ! $SUDO sh -c "touch ${BIN_DIR}/k3s-ro-test && rm -rf ${BIN_DIR}/k3s-ro-test"; then if [ -d /opt/bin ]; then BIN_DIR=/opt/bin fi fi fi # --- use systemd directory if defined or create default --- if [ -n "${INSTALL_K3S_SYSTEMD_DIR}" ]; then SYSTEMD_DIR="${INSTALL_K3S_SYSTEMD_DIR}" else SYSTEMD_DIR=/etc/systemd/system fi # --- set related files from system name --- SERVICE_K3S=${SYSTEM_NAME}.service UNINSTALL_K3S_SH=${UNINSTALL_K3S_SH:-${BIN_DIR}/${SYSTEM_NAME}-uninstall.sh} KILLALL_K3S_SH=${KILLALL_K3S_SH:-${BIN_DIR}/k3s-killall.sh} # --- use service or environment location depending on systemd/openrc --- if [ "${HAS_SYSTEMD}" = true ]; then FILE_K3S_SERVICE=${SYSTEMD_DIR}/${SERVICE_K3S} FILE_K3S_ENV=${SYSTEMD_DIR}/${SERVICE_K3S}.env elif [ "${HAS_OPENRC}" = true ]; then $SUDO mkdir -p /etc/rancher/k3s FILE_K3S_SERVICE=/etc/init.d/${SYSTEM_NAME} FILE_K3S_ENV=/etc/rancher/k3s/${SYSTEM_NAME}.env fi # --- get hash of config & exec for currently installed k3s --- PRE_INSTALL_HASHES=$(get_installed_hashes) # --- if bin directory is read only skip download --- if [ "${INSTALL_K3S_BIN_DIR_READ_ONLY}" = true ]; then INSTALL_K3S_SKIP_DOWNLOAD=true fi # --- setup channel values INSTALL_K3S_CHANNEL_URL=${INSTALL_K3S_CHANNEL_URL:-'https://update.k3s.io/v1-release/channels'} INSTALL_K3S_CHANNEL=${INSTALL_K3S_CHANNEL:-'stable'} } # --- check if skip download environment variable set --- can_skip_download() { if [ "${INSTALL_K3S_SKIP_DOWNLOAD}" != true ]; then return 1 fi } # --- verify an executable k3s binary is installed --- verify_k3s_is_executable() { if [ ! -x ${BIN_DIR}/k3s ]; then fatal "Executable k3s binary not found at ${BIN_DIR}/k3s" fi } # --- set arch and suffix, fatal if architecture not supported --- setup_verify_arch() { if [ -z "$ARCH" ]; then ARCH=$(uname -m) fi case $ARCH in amd64) ARCH=amd64 SUFFIX= ;; x86_64) ARCH=amd64 SUFFIX= ;; arm64) ARCH=arm64 SUFFIX=-${ARCH} ;; aarch64) ARCH=arm64 SUFFIX=-${ARCH} ;; arm*) ARCH=arm SUFFIX=-${ARCH}hf ;; *) fatal "Unsupported architecture $ARCH" esac } # --- verify existence of network downloader executable --- verify_downloader() { # Return failure if it doesn't exist or is no executable [ -x "$(command -v $1)" ] || return 1 # Set verified executable as our downloader program and return success DOWNLOADER=$1 return 0 } # --- create temporary directory and cleanup when done --- setup_tmp() { TMP_DIR=$(mktemp -d -t k3s-install.XXXXXXXXXX) TMP_HASH=${TMP_DIR}/k3s.hash TMP_BIN=${TMP_DIR}/k3s.bin cleanup() { code=$? set +e trap - EXIT rm -rf ${TMP_DIR} exit $code } trap cleanup INT EXIT } # --- use desired k3s version if defined or find version from channel --- get_release_version() { if [ -n "${INSTALL_K3S_COMMIT}" ]; then VERSION_K3S="commit ${INSTALL_K3S_COMMIT}" elif [ -n "${INSTALL_K3S_VERSION}" ]; then VERSION_K3S=${INSTALL_K3S_VERSION} else info "Finding release for channel ${INSTALL_K3S_CHANNEL}" version_url="${INSTALL_K3S_CHANNEL_URL}/${INSTALL_K3S_CHANNEL}" case $DOWNLOADER in curl) VERSION_K3S=$(curl -w '%{url_effective}' -L -s -S ${version_url} -o /dev/null | sed -e 's|.*/||') ;; wget) VERSION_K3S=$(wget -SqO /dev/null ${version_url} 2>&1 | grep -i Location | sed -e 's|.*/||') ;; *) fatal "Incorrect downloader executable '$DOWNLOADER'" ;; esac fi info "Using ${VERSION_K3S} as release" } # --- download from github url --- download() { [ $# -eq 2 ] || fatal 'download needs exactly 2 arguments' case $DOWNLOADER in curl) curl -o $1 -sfL $2 ;; wget) wget -qO $1 $2 ;; *) fatal "Incorrect executable '$DOWNLOADER'" ;; esac # Abort if download command failed [ $? -eq 0 ] || fatal 'Download failed' } # --- download hash from github url --- download_hash() { if [ -n "${INSTALL_K3S_COMMIT}" ]; then HASH_URL=${STORAGE_URL}/k3s${SUFFIX}-${INSTALL_K3S_COMMIT}.sha256sum else HASH_URL=${GITHUB_URL}/download/${VERSION_K3S}/sha256sum-${ARCH}.txt fi info "Downloading hash ${HASH_URL}" download ${TMP_HASH} ${HASH_URL} HASH_EXPECTED=$(grep " k3s${SUFFIX}$" ${TMP_HASH}) HASH_EXPECTED=${HASH_EXPECTED%%[[:blank:]]*} } # --- check hash against installed version --- installed_hash_matches() { if [ -x ${BIN_DIR}/k3s ]; then HASH_INSTALLED=$(sha256sum ${BIN_DIR}/k3s) HASH_INSTALLED=${HASH_INSTALLED%%[[:blank:]]*} if [ "${HASH_EXPECTED}" = "${HASH_INSTALLED}" ]; then return fi fi return 1 } # --- download binary from github url --- download_binary() { if [ -n "${INSTALL_K3S_COMMIT}" ]; then BIN_URL=${STORAGE_URL}/k3s${SUFFIX}-${INSTALL_K3S_COMMIT} else BIN_URL=${GITHUB_URL}/download/${VERSION_K3S}/k3s${SUFFIX} fi info "Downloading binary ${BIN_URL}" download ${TMP_BIN} ${BIN_URL} } # --- verify downloaded binary hash --- verify_binary() { info "Verifying binary download" HASH_BIN=$(sha256sum ${TMP_BIN}) HASH_BIN=${HASH_BIN%%[[:blank:]]*} if [ "${HASH_EXPECTED}" != "${HASH_BIN}" ]; then fatal "Download sha256 does not match ${HASH_EXPECTED}, got ${HASH_BIN}" fi } # --- setup permissions and move binary to system directory --- setup_binary() { chmod 755 ${TMP_BIN} info "Installing k3s to ${BIN_DIR}/k3s" $SUDO chown root:root ${TMP_BIN} $SUDO mv -f ${TMP_BIN} ${BIN_DIR}/k3s } # --- setup selinux policy --- setup_selinux() { case ${INSTALL_K3S_CHANNEL} in *testing) rpm_channel=testing ;; *latest) rpm_channel=latest ;; *) rpm_channel=stable ;; esac rpm_site="rpm.rancher.io" if [ "${rpm_channel}" = "testing" ]; then rpm_site="rpm-testing.rancher.io" fi policy_hint="please install: yum install -y container-selinux selinux-policy-base yum install -y https://${rpm_site}/k3s/${rpm_channel}/common/centos/7/noarch/k3s-selinux-0.2-1.el7_8.noarch.rpm " policy_error=fatal if [ "$INSTALL_K3S_SELINUX_WARN" = true ] || grep -q 'ID=flatcar' /etc/os-release; then policy_error=warn fi if [ "$INSTALL_K3S_SKIP_SELINUX_RPM" = true ] || can_skip_download; then info "Skipping installation of SELinux RPM" else install_selinux_rpm ${rpm_site} ${rpm_channel} fi if ! $SUDO chcon -u system_u -r object_r -t container_runtime_exec_t ${BIN_DIR}/k3s >/dev/null 2>&1; then if $SUDO grep '^\s*SELINUX=enforcing' /etc/selinux/config >/dev/null 2>&1; then $policy_error "Failed to apply container_runtime_exec_t to ${BIN_DIR}/k3s, ${policy_hint}" fi else if [ ! -f /usr/share/selinux/packages/k3s.pp ]; then $policy_error "Failed to find the k3s-selinux policy, ${policy_hint}" fi fi } # --- if on an el7/el8 system, install k3s-selinux install_selinux_rpm() { if [ -r /etc/redhat-release ] || [ -r /etc/centos-release ] || [ -r /etc/oracle-release ]; then dist_version="$(. /etc/os-release && echo "$VERSION_ID")" maj_ver=$(echo "$dist_version" | sed -E -e "s/^([0-9]+)\.?[0-9]*$/\1/") set +o noglob $SUDO rm -f /etc/yum.repos.d/rancher-k3s-common*.repo set -o noglob if [ -r /etc/redhat-release ]; then case ${maj_ver} in 7) $SUDO yum -y install yum-utils $SUDO yum-config-manager --enable rhel-7-server-extras-rpms ;; 8) : ;; *) return ;; esac fi $SUDO tee /etc/yum.repos.d/rancher-k3s-common.repo >/dev/null << EOF [rancher-k3s-common-${2}] name=Rancher K3s Common (${2}) baseurl=https://${1}/k3s/${2}/common/centos/${maj_ver}/noarch enabled=1 gpgcheck=1 gpgkey=https://${1}/public.key EOF $SUDO yum -y install "k3s-selinux" fi return } # --- download and verify k3s --- download_and_verify() { if can_skip_download; then info 'Skipping k3s download and verify' verify_k3s_is_executable return fi setup_verify_arch verify_downloader curl || verify_downloader wget || fatal 'Can not find curl or wget for downloading files' setup_tmp get_release_version download_hash if installed_hash_matches; then info 'Skipping binary downloaded, installed k3s matches hash' return fi download_binary verify_binary setup_binary } # --- add additional utility links --- create_symlinks() { [ "${INSTALL_K3S_BIN_DIR_READ_ONLY}" = true ] && return [ "${INSTALL_K3S_SYMLINK}" = skip ] && return for cmd in kubectl crictl ctr; do if [ ! -e ${BIN_DIR}/${cmd} ] || [ "${INSTALL_K3S_SYMLINK}" = force ]; then which_cmd=$(command -v ${cmd} 2>/dev/null || true) if [ -z "${which_cmd}" ] || [ "${INSTALL_K3S_SYMLINK}" = force ]; then info "Creating ${BIN_DIR}/${cmd} symlink to k3s" $SUDO ln -sf k3s ${BIN_DIR}/${cmd} else info "Skipping ${BIN_DIR}/${cmd} symlink to k3s, command exists in PATH at ${which_cmd}" fi else info "Skipping ${BIN_DIR}/${cmd} symlink to k3s, already exists" fi done } # --- create killall script --- create_killall() { [ "${INSTALL_K3S_BIN_DIR_READ_ONLY}" = true ] && return info "Creating killall script ${KILLALL_K3S_SH}" $SUDO tee ${KILLALL_K3S_SH} >/dev/null << \EOF #!/bin/sh [ $(id -u) -eq 0 ] || exec sudo $0 $@ for bin in /var/lib/rancher/k3s/data/**/bin/; do [ -d $bin ] && export PATH=$PATH:$bin:$bin/aux done set -x for service in /etc/systemd/system/k3s*.service; do [ -s $service ] && systemctl stop $(basename $service) done for service in /etc/init.d/k3s*; do [ -x $service ] && $service stop done pschildren() { ps -e -o ppid= -o pid= | \ sed -e 's/^\s*//g; s/\s\s*/\t/g;' | \ grep -w "^$1" | \ cut -f2 } pstree() { for pid in $@; do echo $pid for child in $(pschildren $pid); do pstree $child done done } killtree() { kill -9 $( { set +x; } 2>/dev/null; pstree $@; set -x; ) 2>/dev/null } getshims() { ps -e -o pid= -o args= | sed -e 's/^ *//; s/\s\s*/\t/;' | grep -w 'k3s/data/[^/]*/bin/containerd-shim' | cut -f1 } killtree $({ set +x; } 2>/dev/null; getshims; set -x) do_unmount_and_remove() { awk -v path="$1" '$2 ~ ("^" path) { print $2 }' /proc/self/mounts | sort -r | xargs -r -t -n 1 sh -c 'umount "$0" && rm -rf "$0"' } do_unmount_and_remove '/run/k3s' do_unmount_and_remove '/var/lib/rancher/k3s' do_unmount_and_remove '/var/lib/kubelet/pods' do_unmount_and_remove '/var/lib/kubelet/plugins' do_unmount_and_remove '/run/netns/cni-' # Remove CNI namespaces ip netns show 2>/dev/null | grep cni- | xargs -r -t -n 1 ip netns delete # Delete network interface(s) that match 'master cni0' ip link show 2>/dev/null | grep 'master cni0' | while read ignore iface ignore; do iface=${iface%%@*} [ -z "$iface" ] || ip link delete $iface done ip link delete cni0 ip link delete flannel.1 rm -rf /var/lib/cni/ iptables-save | grep -v KUBE- | grep -v CNI- | iptables-restore EOF $SUDO chmod 755 ${KILLALL_K3S_SH} $SUDO chown root:root ${KILLALL_K3S_SH} } # --- create uninstall script --- create_uninstall() { [ "${INSTALL_K3S_BIN_DIR_READ_ONLY}" = true ] && return info "Creating uninstall script ${UNINSTALL_K3S_SH}" $SUDO tee ${UNINSTALL_K3S_SH} >/dev/null << EOF #!/bin/sh set -x [ \$(id -u) -eq 0 ] || exec sudo \$0 \$@ ${KILLALL_K3S_SH} if command -v systemctl; then systemctl disable ${SYSTEM_NAME} systemctl reset-failed ${SYSTEM_NAME} systemctl daemon-reload fi if command -v rc-update; then rc-update delete ${SYSTEM_NAME} default fi rm -f ${FILE_K3S_SERVICE} rm -f ${FILE_K3S_ENV} remove_uninstall() { rm -f ${UNINSTALL_K3S_SH} } trap remove_uninstall EXIT if (ls ${SYSTEMD_DIR}/k3s*.service || ls /etc/init.d/k3s*) >/dev/null 2>&1; then set +x; echo 'Additional k3s services installed, skipping uninstall of k3s'; set -x exit fi for cmd in kubectl crictl ctr; do if [ -L ${BIN_DIR}/\$cmd ]; then rm -f ${BIN_DIR}/\$cmd fi done rm -rf /etc/rancher/k3s rm -rf /run/k3s rm -rf /run/flannel rm -rf /var/lib/rancher/k3s rm -rf /var/lib/kubelet rm -f ${BIN_DIR}/k3s rm -f ${KILLALL_K3S_SH} if type yum >/dev/null 2>&1; then yum remove -y k3s-selinux rm -f /etc/yum.repos.d/rancher-k3s-common*.repo fi EOF $SUDO chmod 755 ${UNINSTALL_K3S_SH} $SUDO chown root:root ${UNINSTALL_K3S_SH} } # --- disable current service if loaded -- systemd_disable() { $SUDO systemctl disable ${SYSTEM_NAME} >/dev/null 2>&1 || true $SUDO rm -f /etc/systemd/system/${SERVICE_K3S} || true $SUDO rm -f /etc/systemd/system/${SERVICE_K3S}.env || true } # --- capture current env and create file containing k3s_ variables --- create_env_file() { info "env: Creating environment file ${FILE_K3S_ENV}" $SUDO touch ${FILE_K3S_ENV} $SUDO chmod 0600 ${FILE_K3S_ENV} env | grep '^K3S_' | $SUDO tee ${FILE_K3S_ENV} >/dev/null env | grep -Ei '^(NO|HTTP|HTTPS)_PROXY' | $SUDO tee -a ${FILE_K3S_ENV} >/dev/null } # --- write systemd service file --- create_systemd_service_file() { info "systemd: Creating service file ${FILE_K3S_SERVICE}" $SUDO tee ${FILE_K3S_SERVICE} >/dev/null << EOF [Unit] Description=Lightweight Kubernetes Documentation=https://k3s.io Wants=network-online.target After=network-online.target [Install] WantedBy=multi-user.target [Service] Type=${SYSTEMD_TYPE} EnvironmentFile=-/etc/default/%N EnvironmentFile=-/etc/sysconfig/%N EnvironmentFile=-${FILE_K3S_ENV} KillMode=process Delegate=yes # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=1048576 LimitNPROC=infinity LimitCORE=infinity TasksMax=infinity TimeoutStartSec=0 Restart=always RestartSec=5s ExecStartPre=/bin/sh -xc '! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service' ExecStartPre=-/sbin/modprobe br_netfilter ExecStartPre=-/sbin/modprobe overlay ExecStart=${BIN_DIR}/k3s \\ ${CMD_K3S_EXEC} EOF } # --- write openrc service file --- create_openrc_service_file() { LOG_FILE=/var/log/${SYSTEM_NAME}.log info "openrc: Creating service file ${FILE_K3S_SERVICE}" $SUDO tee ${FILE_K3S_SERVICE} >/dev/null << EOF #!/sbin/openrc-run depend() { after network-online want cgroups } start_pre() { rm -f /tmp/k3s.* } supervisor=supervise-daemon name=${SYSTEM_NAME} command="${BIN_DIR}/k3s" command_args="$(escape_dq "${CMD_K3S_EXEC}") >>${LOG_FILE} 2>&1" output_log=${LOG_FILE} error_log=${LOG_FILE} pidfile="/var/run/${SYSTEM_NAME}.pid" respawn_delay=5 respawn_max=0 set -o allexport if [ -f /etc/environment ]; then source /etc/environment; fi if [ -f ${FILE_K3S_ENV} ]; then source ${FILE_K3S_ENV}; fi set +o allexport EOF $SUDO chmod 0755 ${FILE_K3S_SERVICE} $SUDO tee /etc/logrotate.d/${SYSTEM_NAME} >/dev/null << EOF ${LOG_FILE} { missingok notifempty copytruncate } EOF } # --- write systemd or openrc service file --- create_service_file() { [ "${HAS_SYSTEMD}" = true ] && create_systemd_service_file [ "${HAS_OPENRC}" = true ] && create_openrc_service_file return 0 } # --- get hashes of the current k3s bin and service files get_installed_hashes() { $SUDO sha256sum ${BIN_DIR}/k3s ${FILE_K3S_SERVICE} ${FILE_K3S_ENV} 2>&1 || true } # --- enable and start systemd service --- systemd_enable() { info "systemd: Enabling ${SYSTEM_NAME} unit" $SUDO systemctl enable ${FILE_K3S_SERVICE} >/dev/null $SUDO systemctl daemon-reload >/dev/null } systemd_start() { info "systemd: Starting ${SYSTEM_NAME}" $SUDO systemctl restart ${SYSTEM_NAME} } # --- enable and start openrc service --- openrc_enable() { info "openrc: Enabling ${SYSTEM_NAME} service for default runlevel" $SUDO rc-update add ${SYSTEM_NAME} default >/dev/null } openrc_start() { info "openrc: Starting ${SYSTEM_NAME}" $SUDO ${FILE_K3S_SERVICE} restart } # --- startup systemd or openrc service --- service_enable_and_start() { [ "${INSTALL_K3S_SKIP_ENABLE}" = true ] && return [ "${HAS_SYSTEMD}" = true ] && systemd_enable [ "${HAS_OPENRC}" = true ] && openrc_enable [ "${INSTALL_K3S_SKIP_START}" = true ] && return POST_INSTALL_HASHES=$(get_installed_hashes) if [ "${PRE_INSTALL_HASHES}" = "${POST_INSTALL_HASHES}" ] && [ "${INSTALL_K3S_FORCE_RESTART}" != true ]; then info 'No change detected so skipping service start' return fi [ "${HAS_SYSTEMD}" = true ] && systemd_start [ "${HAS_OPENRC}" = true ] && openrc_start return 0 } # --- re-evaluate args to include env command --- eval set -- $(escape "${INSTALL_K3S_EXEC}") $(quote "$@") # --- run the install process -- { verify_system setup_env "$@" download_and_verify setup_selinux create_symlinks create_killall create_uninstall systemd_disable create_env_file create_service_file service_enable_and_start }
启动k3s
systemctl daemon-reload
systemctl restart k3s
查看状态
systemctl status k3s
之后kubectl get nodes查看，出现node即成功
kubectl get nodes
安装K3S agent 192.168.88.72
重复server端安装1-5步骤
K3S_TOKEN是server(192.168.88.71)端的，位于/var/lib/rancher/k3s/server/node-token下
cat /var/lib/rancher/k3s/server/node-token
在node节点执行命令，格式为 K3S_URL=${k3s_url} K3S_TOKEN=${k3s_token} ./install.sh
K3S_TOKEN=K10d4548aa68cb990da9ac612ff7742c005d14f75d8bac614648296afe4ac15b36e::node:7927b70419ea785e5c540cf11b919693 K3S_URL=https://192.168.88.71:6443 ./install.sh
服务端再查询下 kubectl get nodes
Rancher和k3s关联
在k3s server 上请求 rancher 服务器地址
如果你一不小心关掉了窗口，可以在首页点击右侧的三个点，然后点击升级，就可以看到上面的集群导入命令。
复制下来命令去执行
出现这个提示说明Rancher已经收到K3S的注册请求，正在完成K3S集群的注册工作，再等数十秒种的时间即可完成K3S集群的导入工作：等待中..........
如何使用:https://rancher.com/docs/rancher/latest/zh/
常用解决问题
1.master节点的防火墙全部关掉，要不然worker可能连不上master
systemctl stop firewalld && systemctl disable firewalld && iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
2.有些.worker可能连不上master，可能有些worker的iptables表混乱了
systemctl stop k3s-agent systemctl stop docker iptables -F && iptables -t nat -F && iptables -t mangle -F systemctl start docker systemctl start k3s-agent
3.因为docker拉取镜像都是用https
解决私库https无法访问问题
若是没有 /etc/docker/daemon.json 这个文件,就新建一个
状况1.这种写法是没有配置Docker加速器的状况下
// 单个私服的写法
{
"insecure-registries": ["registry的IP地址:端口号"]
}
// 多个私服的写法
{
"insecure-registries": ["registry1的IP地址:端口号","registry2的IP地址:端口号"]
}
状况2.这种写法是配置过Docker加速器的状况
// 单个私服的写法
{
"registry-mirrors": ["http://f1361db2.m.daocloud.io"],
"insecure-registries": ["registry的IP地址:端口号"]
}
// 多个私服的写法
{
"registry-mirrors": ["http://f1361db2.m.daocloud.io"],
"insecure-registries": ["registry1的IP地址:端口号","registry2的IP地址:端口号"]
}
修改好以后
systemctl daemon-reload
systemctl restart docker.service
systemctl enable docker.service
4.rancher HTTP response to HTTPS clien
每个节点创建 /etc/rancher/k3s/registries.yaml 文件
registries.yaml 加粗是私库地址
mirrors: "192.168.88.85:5000": endpoint: - "http://192.168.88.85:5000"
每个节点都需要重启
k3s server : systemctl restart k3s
k3s agent : systemctl restart k3s-agent
参考:https://www.cnblogs.com/rancherlabs/p/14324469.html
参考:https://docs.rancher.cn/docs/k3s/installation/private-registry/_index/
如何后续出现
FATA[2021-07-16T15:59:12.102835242+08:00] pulling image failed: rpc error: code = Unknown desc = failed to pull and unpack image "192.168.88.85:5000/jboss:v1": failed to resolve reference "192.168.88.85:5000/jboss:v1": parse :///http;/192.168.88.85:5000/jboss/manifests/v1: missing protocol scheme
测试节点是否能拉取镜像
crictl pull 192.168.88.85:5000/jboss:v1
5.卸载
卸载 K3s#
如果你使用install.sh脚本安装了 K3s，那么在安装过程中会生成一个卸载脚本。该脚本在您的节点上的/usr/local/bin/k3s-uninstall.sh上创建（或者是k3s-agent-uninstall.sh）。
./k3s-uninstall.sh #或是以下命令 ./k3s-agent-uninstall.sh
简单配置教程
首先创建项目
保存好以后选择
选择部署服务
这里可以限制每个容器内存大小
可以自行摸索
