ansible + jinja2
Jinja2 loop:
SDEWAN interface define （Go template）
k8s.plugin.opnfv.org/nfn-network: |- { "type": "ovn4nfv", "interface": [ {{- range .Values.nfn }} {{- with . }} { "defaultGateway": "{{- .defaultGateway -}}", "interface": "{{- .interface -}}", "ipAddress": "{{- .ipAddress -}}", "name": "{{- .name -}}" } {{- .separate -}} {{- end }} {{- end }} ]}
采用ansible 实现：
1. define common/templates/networks-prepare.yaml.j2
{% for item in networks %}
---
apiVersion: k8s.plugin.opnfv.org/v1alpha1
kind: ProviderNetwork
metadata:
name: {{ item.networkname }}
spec:
cniType: ovn4nfv
ipv4Subnets:
- subnet: {{ item.subnname }}
name: subnet
gateway: {{ item.gateway }}
excludeIps: {{ item.excludeIps }}
{% if item.providerInterfaceName is defined %}
value of variable: {{ variable }}
providerNetType: {{ item. providerNetType }}
direct:
providerInterfaceName: {{ item.providerInterfaceName }}
directNodeSelector: all
{% endif %}
{% endfor %}
2. j2生成 common/templates/networks-prepare.yaml
3. ansible task： kubectl apply -f common/templates/networks-prepare.yaml
JinJa2 文档：
https://www.ctolib.com/docs-Jinja-c-153726.html
JinJa2循环的例子：
https://www.ctolib.com/docs-Jinja-c-153735.html
oek/代码，有很多模板的例子：
range：
cd roles/ git grep range kubernetes/dashboard/files/clusterrole.yml: - limitranges telemetry/prometheus/files/node-exporter-daemonset.patch:+ {{- range .Values.nodeExporter.extraSecretMounts }} telemetry/prometheus/files/node-exporter-daemonset.patch: {{- range .Values.nodeExporter.extraConfigmapMounts }} telemetry/prometheus/files/node-exporter-daemonset.patch:+ {{- range .Values.nodeExporter.extraSecretMounts }} video_analytics_services/charts/templates/istio-policies.yaml:{{- range $platform := $.Values.platforms }} video_analytics_services/charts/templates/istio-policies.yaml:{{- range $framework := $.Values.frameworks }} video_analytics_services/charts/templates/istio-policies.yaml: {{- range $instance := $.Values.instances }} video_analytics_services/charts/templates/istio-policies.yaml: {{- range $instance := $.Values.instances }} video_analytics_services/charts/templates/istio-policies.yaml:{{- range $instance := $.Values.instances }} video_analytics_services/charts/templates/video-analytics-serving.yaml:{{- range $platform := $.Values.platforms }} video_analytics_services/charts/templates/video-analytics-serving.yaml:{{- range $framework := $.Values.frameworks }} video_analytics_services/charts/templates/video-analytics-serving.yaml:{{- range $instance := $.Values.instances }}
for item in：
git grep "for item in" kubernetes/cni/kubeovn/controlplane/templates/crd_local.yml.j2:{% for item in groups['edgenode_group'] %} machine_setup/vca_node_setup/templates/install_docker.sh.j2:{% for item in _vca_node_docker_packages_url %} video_analytics_services/templates/values.yaml.j2:{% for item in _frameworks %} video_analytics_services/templates/values.yaml.j2:{% for item in _instances %}
Example
define sdewan network interface
cat jinja2.yml
- hosts: 127.0.0.1 vars: http_port: 80 max_clients: 200 networks: - networkname: "pnet1" # ansible can auto gen?? # NET1 subnname: "pnet1_subnet" # ansible can auto gen?? netname1_subnet subnet: "192.168.1.0/24" # please conform with admin gateway: "192.168.1.1" # please conform with admin excludeIps: "192.168.1.2,192.168.1.9" providerNetType: "DIRECT" providerInterfaceName: "eth0" - networkname: "pnet2" # ansible can auto gen?? subnname: "pnet1_subnet" # ansible can auto gen?? netname1_subnet subnet: "192.168.1.0/24" # please conform with admin gateway: "192.168.1.1" # please conform with admin excludeIps: "192.168.1.2,192.168.1.9" - networkname: "onet1" # ansible can auto gen?? subnname: "pnet1_subnet" # ansible can auto gen?? netname1_subnet subnet: "192.168.1.0/24" # please conform with admin gateway: "192.168.1.1" # please conform with admin excludeIps: "192.168.1.2,192.168.1.9" providerNetType: "DIRECT" providerInterfaceName: "eth1" tasks: - name: Copy Template File template: src: ./motd.j2 dest: /tmp/network.yaml tasks: - name: create some files file: name=/tmp/network-{{ item.networkname }}.yaml state=touch with_items: "{{ networks }}"
cat motd.j2
welcome to {{ ansible_fqdn }} This system total mem is : {{ ansible_memtotal_mb }} MB This system free mem is: {{ ansible_memfree_mb }} MB {% for item in networks %} --- apiVersion: k8s.plugin.opnfv.org/v1alpha1 kind: ProviderNetwork metadata: name: {{ item.networkname }} spec: cniType: ovn4nfv ipv4Subnets: - subnet: {{ item.subnname }} name: subnet gateway: {{ item.gateway }} excludeIps: {{ item.excludeIps }} {% if item.providerInterfaceName is defined %} providerNetType: {{ item. providerNetType }} direct: providerInterfaceName: {{ item.providerInterfaceName }} directNodeSelector: all {% endif %} {% endfor %}
运行
ansible-playbook jinja2.yml
check
ls /tmp/network-*.yaml # /tmp/network-onet1.yaml /tmp/network-pnet1.yaml /tmp/network-pnet2.yaml
check 文件内容
cat /tmp/network.yaml
如下：
welcome to localhost.localdomain This system total mem is : 63909 MB This system free mem is: 52362 MB --- apiVersion: k8s.plugin.opnfv.org/v1alpha1 kind: ProviderNetwork metadata: name: pnet1 spec: cniType: ovn4nfv ipv4Subnets: - subnet: pnet1_subnet name: subnet gateway: 192.168.1.1 excludeIps: 192.168.1.2,192.168.1.9 providerNetType: DIRECT direct: providerInterfaceName: eth0 directNodeSelector: all --- apiVersion: k8s.plugin.opnfv.org/v1alpha1 kind: ProviderNetwork metadata: name: pnet2 spec: cniType: ovn4nfv ipv4Subnets: - subnet: pnet1_subnet name: subnet gateway: 192.168.1.1 excludeIps: 192.168.1.2,192.168.1.9 --- apiVersion: k8s.plugin.opnfv.org/v1alpha1 kind: ProviderNetwork metadata: name: onet1 spec: cniType: ovn4nfv ipv4Subnets: - subnet: pnet1_subnet name: subnet gateway: 192.168.1.1 excludeIps: 192.168.1.2,192.168.1.9 providerNetType: DIRECT direct: providerInterfaceName: eth1 directNodeSelector: all
interfaces for CNF
cat jinja2.yml
- hosts: 127.0.0.1 vars: cnfs: - name: "cnf1" interfaces: - ipAddress: "192.168.1.12" name: "net2" belongto: "pnet1" - ipAddress: "" name: "net3" belongto: "pnet2" - ipAddress: "" name: "net4" belongto: "onet2" tasks: - name: create nfn files # debug: # var: item template: src: ./nfn.j2 dest: /tmp/cnfs-{{ item.name }}-nfn.yaml with_items: "{{ cnfs }}"
cat nfn.j2
nfn: {% for i in item.interfaces %} - defaultGateway: false interface: {{ i.name }} ipAddress: {{ i.ipAddress }} name: {{ i.belongto }} {% if loop.last %} separate: "" {% else %} separate: "," {% endif %} {% endfor %}
运行
ansible-playbook jinja2.yml
check
cat /tmp/cnfs-cnf1-nfn.yaml
内容：
nfn: - defaultGateway: false interface: net2 ipAddress: 192.168.1.12 name: pnet1 separate: "," - defaultGateway: false interface: net3 ipAddress: name: pnet2 separate: "," - defaultGateway: false interface: net4 ipAddress: name: onet2 separate: ""
IPsec host tunnel rule
cat plbook.yaml
- hosts: 127.0.0.1 vars: http_port: 80 max_clients: 200 cnfs: - name: "cnf1" rules: - name: tunnel1 type: tunnelsite local_identifier: 192.168.1.1 remote: local_sourceip: remote_subnet: remote_sourceip: local_subnet: - name: tunnel2 type: tunnelhost local_identifier: 192.168.1.1 remote: local_sourceip: remote_subnet: remote_sourceip: local_subnet: - name: snat1 type: snat network: private: 192.168.1.1 via: - name: dnat1 type: dnat from: 192.168.1.1 ingress: network: tasks: - name: Copy Rules Template File template: src: ./tunnel.j2 dest: /tmp/cnfs-{{ item.name }}-tunnel.yaml with_items: "{{ cnfs }}"
cat tunnel.j2
{% for i in item.rules %}
{% if i.type == "tunnelhost" %}
---
apiVersion: batch.sdewan.akraino.org/v1alpha1
kind: IpsecHost
metadata:
name: ipsechost
namespace: cnf
labels:
sdewanPurpose: {{ item.name }}
spec:
name: {{ i.name }}
remote: {{ i.remote }}
pre_shared_key: test_key
authentication_method: psk
local_identifier: {{ i.local_identifier }}
crypto_proposal:
- ipsecproposal
force_crypto_proposal: "0"
connections:
- name: connA
conn_type: tunnel
mode: start
{% if i.local_sourceip is defined %}
local_sourceip: {{ i.local_sourceip }}
{% else %}
local_sourceip: "%config"
{% endif %}
remote_subnet: {{ i.remote_subnet }}
crypto_proposal:
- ipsecproposal
{% endif %}
{% endfor %}
运行
ansible-playbook plbook.yaml
check
cat /tmp/cnfs-cnf1-tunnel.yaml
内容：
--- apiVersion: batch.sdewan.akraino.org/v1alpha1 kind: IpsecHost metadata: name: ipsechost namespace: cnf labels: sdewanPurpose: cnf1 spec: name: tunnel2 remote: pre_shared_key: test_key authentication_method: psk local_identifier: 192.168.1.1 crypto_proposal: - ipsecproposal force_crypto_proposal: "0" connections: - name: connA conn_type: tunnel mode: start local_sourceip: "%config" remote_subnet: crypto_proposal: - ipsecproposal
IPsec site tunnel rule
cat plbook.yaml
- hosts: 127.0.0.1 vars: http_port: 80 max_clients: 200 cnfs: - name: "cnf1" rules: - name: tunnel1 type: tunnelsite local_identifier: 192.168.1.1 remote: local_sourceip: remote_subnet: remote_sourceip: local_subnet: - name: tunnel2 type: tunnelhost local_identifier: 192.168.1.1 remote: local_sourceip: remote_subnet: remote_sourceip: local_subnet: - name: snat1 type: snat network: private: 192.168.1.1 via: - name: dnat1 type: dnat from: 192.168.1.1 ingress: network: tasks: - name: Copy Rules Template File template: src: ./tunnel.j2 dest: /tmp/cnfs-{{ item.name }}-tunnel.yaml with_items: "{{ cnfs }}"
cat tunnel.j2
{% for i in item.rules %}
{% if i.type == "tunnelsite" %}
---
apiVersion: batch.sdewan.akraino.org/v1alpha1
kind: IpsecSite
metadata:
name: ipsecsite
namespace: cnf
labels:
sdewanPurpose: {{ item.name }}
spec:
name: {{ i.name }}
{% if i.remote is defined %}
remote: {{ i.remote }}
{% else %}
remote: "%any"
{% endif %}
pre_shared_key: test_key
authentication_method: psk
local_identifier: ""
crypto_proposal:
- ipsecproposal
force_crypto_proposal: "0"
connections:
- name: connA
conn_type: tunnel
mode: start
remote_sourceip: {{ i.remote_sourceip }}
local_subnet: {{ i.local_subnet }}
crypto_proposal:
- ipsecproposal
{% endif %}
{% endfor %}
运行
ansible-playbook plbook.yaml
check
cat /tmp/cnfs-cnf1-tunnel.yaml
内容：
--- apiVersion: batch.sdewan.akraino.org/v1alpha1 kind: IpsecSite metadata: name: ipsecsite namespace: cnf labels: sdewanPurpose: cnf1 spec: name: tunnel1 remote: pre_shared_key: test_key authentication_method: psk local_identifier: "" crypto_proposal: - ipsecproposal force_crypto_proposal: "0" connections: - name: connA conn_type: tunnel mode: start remote_sourceip: local_subnet: crypto_proposal: - ipsecproposal
IPsec site snat rule
cat plbook.yaml
- hosts: 127.0.0.1 vars: http_port: 80 max_clients: 200 cnfs: - name: "cnf1" interfaces: - ipAddress: "192.168.1.12" name: "net2" belongto: "pnet1" - ipAddress: "" name: "net3" belongto: "pnet2" - ipAddress: "" name: "net4" belongto: "onet2" rules: - name: tunnel1 type: tunnelsite local_identifier: 192.168.1.1 remote: local_sourceip: remote_subnet: remote_sourceip: local_subnet: - name: tunnel2 type: tunnelhost local_identifier: 192.168.1.1 remote: local_sourceip: remote_subnet: remote_sourceip: local_subnet: - name: snat1 type: snat network: private: 192.168.1.1 via: provider: "pnet1" - name: dnat1 type: dnat from: 192.168.1.1 ingress: network: provider: "pnet2" tasks: - name: Copy Rules Template File template: src: ./tunnel.j2 dest: /tmp/cnfs-{{ item.name }}-tunnel.yaml with_items: "{{ cnfs }}"
cat tunnel.j2
{% for i in item.rules %}
{% if i.type == "snat" %}
---
apiVersion: batch.sdewan.akraino.org/v1alpha1
kind: FirewallSNAT
metadata:
name: firewallsnat
namespace: cnf
labels:
sdewanPurpose: {{ item.name }}
spec:
src_ip: {{ i.private }}
src_dip: {{ i.via }}
dest: {{ i.provider }}
{% if i.network is defined %}
dest_ip: {{ i.network }}
{% endif %}
proto: all
target: SNAT
{% endif %}
{% endfor %}
运行
ansible-playbook plbook.yaml
check：
cat /tmp/cnfs-cnf1-tunnel.yaml
--- apiVersion: batch.sdewan.akraino.org/v1alpha1 kind: FirewallSNAT metadata: name: firewallsnat namespace: cnf labels: sdewanPurpose: cnf1 spec: src_ip: 192.168.1.1 src_dip: dest: pnet1 dest_ip: proto: all target: SNAT
IPsec site dnat rule
cat plbook.yaml
- hosts: 127.0.0.1 vars: http_port: 80 max_clients: 200 cnfs: - name: "cnf1" interfaces: - ipAddress: "192.168.1.12" name: "net2" belongto: "pnet1" - ipAddress: "" name: "net3" belongto: "pnet2" - ipAddress: "" name: "net4" belongto: "onet2" rules: - name: tunnel1 type: tunnelsite local_identifier: 192.168.1.1 remote: local_sourceip: remote_subnet: remote_sourceip: local_subnet: - name: tunnel2 type: tunnelhost local_identifier: 192.168.1.1 remote: local_sourceip: remote_subnet: remote_sourceip: local_subnet: - name: snat1 type: snat network: private: 192.168.1.1 via: provider: "pnet1" - name: dnat1 type: dnat from: 192.168.1.1 ingress: network: provider: "pnet2" tasks: - name: Copy Rules Template File template: src: ./tunnel.j2 dest: /tmp/cnfs-{{ item.name }}-tunnel.yaml with_items: "{{ cnfs }}"
cat tunnel.j2
{% for i in item.rules %}
{% if i.type == "dnat" %}
---
apiVersion: batch.sdewan.akraino.org/v1alpha1
kind: FirewallDNAT
metadata:
name: firewalldnat
namespace: cnf
labels:
sdewanPurpose: {{ item.name }}
spec:
src: {{ i.provider }}
{% if i.network is defined %}
src_ip: {{ i.network }}
{% endif %}
src_dip: {{ i.from }}
dest_ip: {{ i.ingress }}
proto: all
target: DNAT
{% endif %}
{% endfor %}
运行
ansible-playbook plbook.yaml
check：
cat /tmp/cnfs-cnf1-tunnel.yaml
--- apiVersion: batch.sdewan.akraino.org/v1alpha1 kind: FirewallDNAT metadata: name: firewalldnat namespace: cnf labels: sdewanPurpose: cnf1 spec: src: pnet2 src_ip: src_dip: 192.168.1.1 dest_ip: proto: all target: DNAT
------
