consul v1.10.4 集群部署

一 系统设置

1.1 系统时间同步

~# apt -y install chrony
~# systemctl start chrony
~# systemctl enable chrony

1.2 服务器规划

主机名 IP 角色
consul-01 192.168.174.103 consul server
sonsul-02   192.168.174.104 consul  client
sonsul-03  192.168.174.105 consul  client

1.3 软件清单

consul：1.10.4

1.4 推荐部署架构

官方文档：

https://www.consul.io/docs/install#precompiled-binaries

https://learn.hashicorp.com/tutorials/consul/deployment-guide?in=consul/production-deploy#configure-consul-agents

https://github.com/hashicorp/consul

1.4 服务器配置

Size CPU Memory Disk Capacity Disk IO Disk Throughput
Large 8-16 core 32-64 GB RAM 200+ GB 7500+ IOPS 250+ MB/s

二 安装consul

2.1 下载consul安装包

https://www.consul.io/downloads

~# wget https://releases.hashicorp.com/consul/1.15.2/consul_1.15.2_linux_amd64.zip

2.2 安装consul

root@consul-01:~# unzip consul_1.10.4_linux_amd64.zip -d /usr/local/bin
root@consul-02:~# unzip consul_1.10.4_linux_amd64.zip -d /usr/local/bin
root@consul-03:~# unzip consul_1.10.4_linux_amd64.zip -d /usr/local/bin

2.3 设置属主属组

root@consul-01:~# chown root.root /usr/local/bin/consul

2.4 consul命令补齐

root@consul-01:~# consul -autocomplete-install  # 重新登录可以实现命令补齐功能
root@consul-01:~# complete -C /usr/local/bin/consul sonsul

2.5 创建普通用户consul

root@consul-01:~# useradd --system --home /etc/consul.d --shell /bin/false consul
root@consul-02:~# useradd --system --home /etc/consul.d --shell /bin/false consul
root@consul-03:~# useradd --system --home /etc/consul.d --shell /bin/false consul

2.6 创建数据目录

root@consul-01:~#  mkdir --parents /opt/consul
root@consul-02:~#  mkdir --parents /opt/consul
root@consul-03:~#  mkdir --parents /opt/consul

2.7 设置数据目录权限

root@consul-01:~# chown --recursive consul:consul /opt/consul
root@consul-02:~# chown --recursive consul:consul /opt/consul
root@consul-03:~# chown --recursive consul:consul /opt/consul

2.8 验证版本

root@consul-01:~# consul version
Consul v1.10.4
Revision 7bbad6fe
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

三 准备安全凭证

3.1 生成gossip 加密密钥

gossip是用对称密钥加密的，因为节点之间的gossip是通过 UDP 完成的。所有代理必须具有相同的加密密钥。

root@consul-01:~# consul keygen
VTd56Fa6qg0HVJ1+UqLeub7rGLYn7EncFbyMnSaKZ9o=

3.2 为 RPC 加密生成 TLS 证书

3.2.1 创建证书颁发机构

Start by creating the CA on your admin instance, using the Consul CLI.

root@consul-01:~# consul tls ca create -domain wgs
==> Saved wgs-agent-ca.pem
==> Saved wgs-agent-ca-key.pem

3.2.2 创建证书

接下来创建一组证书，每个 Consul 代理一个。您现在需要为您的主数据中心选择一个名称，以便正确命名证书以及您的 Consul 数据中心的域。

root@consul-01:~# consul tls cert create -server -dc <dc_name> -domain <domain>  #创建格式
root@consul-01:~# consul tls cert create -server -dc wgs -domain  wgs
==> WARNING: Server Certificates grants authority to become a
    server and access all state in the cluster including root keys
    and all ACL tokens. Do not distribute them to production hosts
    that are not server nodes. Store them as securely as CA keys.
==> Using wgs-agent-ca.pem and wgs-agent-ca-key.pem
==> Saved wgs-server-wgs-0.pem
==> Saved wgs-server-wgs-0-key.pem

3.3  将证书分发给agent

root@consul-01:~# scp wgs-agent-ca.pem wgs-server-wgs-0.pem wgs-server-wgs-0-key.pem root@192.168.174.103:/etc/consul.d/certs  
root@consul-01:~# scp wgs-agent-ca.pem wgs-server-wgs-0.pem wgs-server-wgs-0-key.pem root@192.168.174.104:/etc/consul.d/certs                                                                                                                                              
root@consul-01:~# scp wgs-agent-ca.pem wgs-server-wgs-0.pem wgs-server-wgs-0-key.pem root@192.168.174.105:/etc/consul.d/certs

3.4 设置证书权限

chown -R consul.consul /etc/consul.d/

四 配置文件

4.1 创建配置文件

touch /etc/consul.d/consul.hcl
chown --recursive consul:consul /etc/consul.d
chmod 640 /etc/consul.d/consul.hcl

4.2 consul-01配置

查看代码 
root@ceph-node01:~# cat /etc/consul.d/consul.hcl
datacenter = "wgs01"
data_dir = "/opt/consul"
encrypt = "XSsXcaGKyxfa00zAw7Lw0zvlQJ6PcTFlCl+40J9quyw="
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true

ca_file = "/etc/consul.d/certs/consul-agent-ca.pem"
cert_file = "/etc/consul.d/certs/wgs01-server-consul-0.pem"
key_file = "/etc/consul.d/certs/wgs01-server-consul-0-key.pem"

auto_encrypt {
  allow_tls = true
}

performance {
  raft_multiplier = 1
}

server = true
bootstrap_expect = 3
bind_addr = "192.168.174.103"
client_addr = "0.0.0.0"
ui_config {
  enabled = true
}

node_name = "consul-01"
bootstrap_expect = 1

connect {
  enabled = true
}

addresses {
  grpc = "127.0.0.1"
}

ports {
  grpc  = 8502
}

4.3 consul-02配置

查看代码 
root@ceph-node02:~# cat /etc/consul.d/consul.hcl
datacenter = "wgs01"
data_dir = "/opt/consul"
encrypt = "XSsXcaGKyxfa00zAw7Lw0zvlQJ6PcTFlCl+40J9quyw="
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true

ca_file = "/etc/consul.d/certs/consul-agent-ca.pem"

auto_encrypt = {
  tls = true
}

retry_join = ["192.168.174.103"]

performance {
  raft_multiplier = 1
}

bind_addr = "192.168.174.104"
client_addr = "0.0.0.0"
node_name = "consul-02"



connect {
  enabled = true
}

addresses {
  grpc = "127.0.0.1"
}

ports {
  grpc  = 8502
}

4.4 consul-03配置

查看代码 
root@ceph-node03:~# cat /etc/consul.d/consul.hcl
datacenter = "wgs01"
data_dir = "/opt/consul"
encrypt = "XSsXcaGKyxfa00zAw7Lw0zvlQJ6PcTFlCl+40J9quyw="
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true

ca_file = "/etc/consul.d/certs/consul-agent-ca.pem"

auto_encrypt = {
  tls = true
}

retry_join = ["192.168.174.103"]

performance {
  raft_multiplier = 1
}

bind_addr = "192.168.174.105"
client_addr = "0.0.0.0"
node_name = "consul-03"


connect {
  enabled = true
}

addresses {
  grpc = "127.0.0.1"
}

ports {
  grpc  = 8502
}

五 配置详解 

  • ca_file - 指定 CA 公共证书文件的路径。
  • cert_file - 指定代理公共证书文件的路径。
  • key_file - 指定代理证书私钥文件的路径。
  • auto_encrypt: 自动给客户端签发证书。
  • raft_multiplier- Consul 服务器用于缩放关键 Raft 计时参数的整数乘法器。将此值设置为 1 会将 Raft 配置为其最高性能模式，相当于 Consul 0.7 之前的默认时间，建议用于生产 Consul 服务器。
  • server - 此标志用于控制代理处于服务器模式还是客户端模式。
  • bootstrap_expect- 此标志提供数据中心中预期的服务器数量。不应提供此值，或者该值应在数据中心的所有服务器中保持一致。
  • ui - 启用内置的 Web UI。
  • connect.enabled- 控制是否在此代理上启用连接功能。应在集群中的所有服务器上启用，以便 Connect 正常运行。
  • addresses.grpc- Consul 将绑定 gRPC API 的地址。默认为，client_addr但出于安全原因，将其打开localhost/可能很敏感127.0.0.1。
  • ports.grpc- gRPC API 端口。我们建议按惯例使用8502forgrpc因为某些工具会自动使用它。目前 gRPC 仅用于将 Envoy xDS API 暴露给 Envoy 代理。

注意：默认情况下，Consul Connect 服务网格使用嵌入式 CA 为服务生成和签署证书。可以将 Consul 配置为使用不同的 CA。

  • datacenter - 运行代理的数据中心。
  • data_dir - 代理存储状态的数据目录。
  • encrypt - 指定用于 Consul 网络流量的gossip加密密钥。
  • verify_incoming - 如果设置为 true，Consul 要求所有传入连接都使用 TLS。
  • verify_outgoing - 如果设置为 true，Consul 要求来自该代理的所有传出连接都使用 TLS。
  • verify_server_hostname- 如果设置为 true，Consul 将为所有传出 TLS 连接验证服务器提供的 TLS 证书是否与server.<datacenter>.<domain>主机名匹配。
  • raft_multiplier- Consul 服务器用于缩放关键 Raft 计时参数的整数乘法器。将此值设置为 1 会将 Raft 配置为其最高性能模式，相当于 Consul 0.7 之前的默认时间，建议用于生产 Consul 服务器。
  • connect.enabled- 控制是否在此代理上启用连接功能。应在集群中的所有服务器上启用，以便 Connect 正常运行。
  • addresses.grpc- Consul 将绑定 gRPC API 的地址。默认为，client_addr但出于安全原因，将其打开localhost/可能很敏感127.0.0.1。
  • ports.grpc- gRPC API 端口。我们建议按惯例使用8502forgrpc因为某些工具会自动使用它。目前 gRPC 仅用于将 Envoy xDS API 暴露给 Envoy 代理。

注意：默认情况下，Consul Connect 服务网格使用嵌入式 CA 为服务生成和签署证书。可以将 Consul 配置为使用不同的 CA。

六 检查配置文件

~# consul validate /etc/consul.d/
Configuration is valid!

七 节点启动测试

7.1 consul-01节点

root@consul-01:~/consul# /usr/local/bin/consul agent -config-dir=/etc/consul.d/
==> Starting Consul agent...
           Version: '1.10.4'
           Node ID: 'c63043d5-f525-10c6-c244-49bd16e7d261'
         Node name: 'consul-01'
        Datacenter: 'wgs01' (Segment: '<all>')
            Server: true (Bootstrap: true)
       Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, gRPC: 8502, DNS: 8600)
      Cluster Addr: 192.168.174.103 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: true, TLS-Outgoing: true, TLS-Incoming: true, Auto-Encrypt-TLS: true

==> Log data will now stream in as it occurs:

7.2 consul-02节点

root@consul-02:/etc/consul.d# /usr/local/bin/consul agent -config-dir=/etc/consul.d/
==> Starting Consul agent...
           Version: '1.10.4'
           Node ID: '1bd3df65-f957-4ad3-3074-0dbffb9815be'
         Node name: 'consul-02'
        Datacenter: 'wgs01' (Segment: '')
            Server: false (Bootstrap: false)
       Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, gRPC: 8502, DNS: 8600)
      Cluster Addr: 192.168.174.104 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: true, TLS-Outgoing: true, TLS-Incoming: true, Auto-Encrypt-TLS: true

==> Log data will now stream in as it occurs:

7.3 consul-03节点

root@consul-03:/etc/consul.d# /usr/local/bin/consul agent -config-dir=/etc/consul.d/
==> Starting Consul agent...
           Version: '1.10.4'
           Node ID: 'c3ac7a90-c9ec-a7b4-228c-66d5245fb8ae'
         Node name: 'consul-03'
        Datacenter: 'wgs01' (Segment: '')
            Server: false (Bootstrap: false)
       Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, gRPC: 8502, DNS: 8600)
      Cluster Addr: 192.168.174.105 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: true, TLS-Outgoing: true, TLS-Incoming: true, Auto-Encrypt-TLS: true

==> Log data will now stream in as it occurs:

八 添加consul.service文件

root@consul-01:~# cat /lib/systemd/system/consul.service
[Unit]
Description="HashiCorp Consul - A service mesh solution"
Documentation=https://www.consul.io/
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/consul.d/consul.hcl

[Service]
EnvironmentFile=/etc/consul.d/consul.env
User=consul
Group=consul
ExecStart=/usr/bin/consul agent -config-dir=/etc/consul.d/ -log-file=/opt/consul/consul.log
ExecReload=/bin/kill --signal HUP $MAINPID
KillMode=process
KillSignal=SIGTERM
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

九 设置consul开机启动

root@consul-01:~# systemctl enable consul
Created symlink /etc/systemd/system/multi-user.target.wants/consul.service → /lib/systemd/system/consul.service.
root@consul-01:~# systemctl start consul
root@consul-01:~# systemctl status consul
● consul.service - "HashiCorp Consul - A service mesh solution"
     Loaded: loaded (/lib/systemd/system/consul.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2021-11-25 13:43:20 CST; 12s ago
       Docs: https://www.consul.io/
   Main PID: 11179 (consul)
      Tasks: 11 (limit: 2245)
     Memory: 20.3M
     CGroup: /system.slice/consul.service
             └─11179 /usr/local/bin/consul agent -config-dir=/etc/consul.d/ -log-file=/opt/consul/consul.log

Nov 25 13:43:20 consul-01 consul[11179]: 2021-11-25T13:43:20.994+0800 [INFO]  agent: Retry join is supported for the following discovery methods: cluster=LAN discovery_methods="aliyun aws azure digitalocean g>
Nov 25 13:43:20 consul-01 consul[11179]: 2021-11-25T13:43:20.994+0800 [INFO]  agent: Joining cluster...: cluster=LAN
Nov 25 13:43:20 consul-01 consul[11179]: 2021-11-25T13:43:20.994+0800 [INFO]  agent: (LAN) joining: lan_addresses=[192.168.174.103]
Nov 25 13:43:20 consul-01 consul[11179]: 2021-11-25T13:43:20.994+0800 [INFO]  agent: started state syncer
Nov 25 13:43:20 consul-01 consul[11179]: 2021-11-25T13:43:20.994+0800 [INFO]  agent: Consul agent running!
Nov 25 13:43:20 consul-01 consul[11179]: 2021-11-25T13:43:20.994+0800 [WARN]  agent: grpc: addrConn.createTransport failed to connect to {dc1-192.168.174.103:8300 0 consul-01.dc1 <nil>}. Err :connection error>
Nov 25 13:43:20 consul-01 consul[11179]: 2021-11-25T13:43:20.995+0800 [INFO]  agent: (LAN) joined: number_of_nodes=1
Nov 25 13:43:20 consul-01 consul[11179]: 2021-11-25T13:43:20.995+0800 [INFO]  agent: Join cluster completed. Synced with initial agents: cluster=LAN num_agents=1
Nov 25 13:43:22 consul-01 consul[11179]: 2021-11-25T13:43:22.510+0800 [WARN]  agent.server.raft: no known peers, aborting election
Nov 25 13:43:28 consul-01 consul[11179]: 2021-11-25T13:43:28.220+0800 [ERROR] agent.anti_entropy: failed to sync remote state: error="No cluster leader"

十 查看consul集群成员

root@consul-01:~# consul members
Node       Address               Status  Type    Build   Protocol  DC     Segment
consul-01  192.168.174.103:8301  alive   server  1.10.4  2         wgs01  <all>
consul-02  192.168.174.104:8301  alive   client  1.10.4  2         wgs01  <default>
consul-03  192.168.174.105:8301  alive   client  1.10.4  2         wgs01  <default>

root@consul-02:~# consul members
Node       Address               Status  Type    Build   Protocol  DC     Segment
consul-01  192.168.174.103:8301  alive   server  1.10.4  2         wgs01  <all>
consul-02  192.168.174.104:8301  alive   client  1.10.4  2         wgs01  <default>
consul-03  192.168.174.105:8301  alive   client  1.10.4  2         wgs01  <default>

root@consul-03:~# consul members
Node       Address               Status  Type    Build   Protocol  DC     Segment
consul-01  192.168.174.103:8301  alive   server  1.10.4  2         wgs01  <all>
consul-02  192.168.174.104:8301  alive   client  1.10.4  2         wgs01  <default>
consul-03  192.168.174.105:8301  alive   client  1.10.4  2         wgs01  <default>

十一 删除consul集群节点

11.1 查看当前成员信息

root@consul-03:~# consul members
Node       Address               Status  Type    Build   Protocol  DC     Segment
consul-01  192.168.174.103:8301  alive   server  1.10.4  2         wgs01  <all>
consul-02  192.168.174.104:8301  alive   client  1.10.4  2         wgs01  <default>
consul-03  192.168.174.105:8301  alive   client  1.10.4  2         wgs01  <default>

11.2 移除consul-03节点

root@consul-03:~# consul leave  #移除集群并停止服务
Graceful leave complete

root@consul-03:~# consul force-leave <node> #强制删除

11.3 其它节点验证

root@consul-01:~# consul members
Node       Address               Status  Type    Build   Protocol  DC     Segment
consul-01  192.168.174.103:8301  alive   server  1.10.4  2         wgs01  <all>
consul-02  192.168.174.104:8301  alive   client  1.10.4  2         wgs01  <default>
consul-03  192.168.174.105:8301  left    client  1.10.4  2         wgs01  <default>

十二 验证consul web界面

 

    

posted @ 2021-11-24 21:20  小吉猫  阅读(1764)  评论(0)  编辑  收藏  举报