|NO.Z.00004|——————————|Deployment|——|Hadoop&Flink计算领域锋利的武器.v04|——|Flink.v01|StandAlone模式部署|
一、Flink安装和部署
### --- Flink支持多种安装模式
~~~ local(本地):单机模式,一般本地开发调试使用
~~~ StandAlone 独立模式:Flink自带集群,自己管理资源调度,生产环境也会有所应用
~~~ Yarn模式:计算资源统一由Hadoop YARN管理,生产环境应用较多
二、环境准备工作
### --- 基础环境
~~~ # jdk1.8及以上【配置JAVA_HOME环境变量】
~~~ # ssh免密码登录【集群内节点之间免密登录】
[root@hadoop01 ~]# java -version
java version "1.8.0_231"
[root@hadoop01 ~]# cat .ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDf1rEa6F/zlreckTPEmVW4gHVi8Q6KUkQH6fN95EiqiEXvoudntlPjq5mdT6VQupuDCJMIbe6pMG276Rn7uDWrY9NZC3uIx3V8PwWbm+mSPJS9uxVYLPxTo4bd8HQrZ5VKsh2NMr+CnVq5Kz5at8z4X3QleeJjCwxbAKHyl4iMsgOTF1/6geNm+0mQiNSvdMDDvmr1wvEtfGMBwKxpCs5QYUuA0ty8nvCQXvoe7AUu3NGNeg494V78uh5gtR+0mB+xbCyVZd61XI0nooPWKpgHyAcsb/mxl8DSDJ2HoZVUWc1D/+KavvOUawAzA7B+/SGjtbRc1RHOMpEmNph6Jk4F root@hadoop01
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfPLjiMFvSWUIDHGybv58EFwpHC6+wg5/kvBy+86pgvzjFTIV4FN4Rr1gTsaV+38YfvXOt+zAD+DAcZPhgzXjhoii32g4s23Bvwdbru2trb/G0pXBM9rnDZ8i/Q2XQN3DPCIzJ3Vk45ab9dMarj1NvMyMt+3pHgut5QtCFlNG1nJVYZuyKPYnYo9LyPxqIdcL9KrJ/y/a8zUdhbm5OUST0WEFx3bRiP4yTjtX6G4s/NM6ajlgmADCB2VzxSYNYsIL0Y2k1YKqLcIluH8kjCPekmRkhnDduEm5DjuaPjuVoEMNQMzV1qbQUhA4l8PUsL9gF1s08culbqdMfYnOqyxJx root@hadoop02
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNSwd/9bKDf4AL01SJ5elFAAd+MMzE/O9YfWrgpas/iAUCadKEPLeaoUF2x23Pn87F5sGk7vssyNjYQPgFxXVJcSc2CI/0s3xKjo/Yi8hGhc4SvCqZawQVFaMd+raE5jzbDNQF/B2n0qrBTt3Wnnz5YDKcw0+cYe7vwUVlshVM5+vdhjuSDMA7FROjGIdjxfxWG7hXcJEmlRHR4TEq9oNSl5xSpblspROIzqfRnNYqplJmkML+AsVOtJfOGFCO/197o4YX79MFyYXBF9rk3Ycklp06dkbI/BB1v+9Du2DGGkiCFmRvmUG/qD0HWLFAoE808JonNoDdICXz6yqeUXzT root@hadoop03
### --- 安装包下载
~~~ 下载地址:https://archive.apache.org/dist/flink/flink-1.11.1/

### --- 下载Flink版本包:
[root@hadoop01 ~]# wget https://archive.apache.org/dist/flink/flink-1.11.1/flink-1.11.1-bin-scala_2.12.tgz
三、集群规划
Hadoop01 | Hadoop02 | Hadoop03 |
JobManager+TaskManager | TaskManager | TaskManager |
一、Flink.Standalone模式部署:StandAlone模式部署
### --- 下载版本包并解压到目标文件夹
~~~ # 下载版本包并解压
[root@hadoop01 ~]# ls /opt/yanqi/software/flink-1.11.1-bin-scala_2.12.tgz
/opt/yanqi/software/flink-1.11.1-bin-scala_2.12.tgz
~~~ # 解压版本包
[root@hadoop01 ~]# cd /opt/yanqi/software/
[root@hadoop01 software]# tar -zxvf flink-1.11.1-bin-scala_2.12.tgz \
-C ../servers/
### --- 修改配置文件:flink-conf.yaml
~~~ # 修改配置文件
[root@hadoop01 ~]# vim /opt/yanqi/servers/flink-1.11.1/conf/flink-conf.yaml
jobmanager.rpc.address: hadoop01 # 设置rpc通信地址设置为hadoop01;flink集群中master地址
taskmanager.numberOfTaskSlots: 1 # 每一个task当中任务槽的数量,默认是1个
parallelism.default: 1 # Flink任务中的并行度
### --- 修改配置文件:master
[root@hadoop01 ~]# vim /opt/yanqi/servers/flink-1.11.1/conf/masters
hadoop01:8081 # 设置master主机地址
### --- 修改配置文件:worker
[root@hadoop01 ~]# vim /opt/yanqi/servers/flink-1.11.1/conf/workers
hadoop01
hadoop02
hadoop03
### --- 修改配置文件:zoo.cnf
~~~ # zoo.cnf:Flink集群的机制是依赖zookeeper的
[root@hadoop01 ~]# vim /opt/yanqi/servers/flink-1.11.1/conf/zoo.cfg
~~~ 配置zk的主机信息
server.1=hadoop01:2888:3888
server.2=hadoop02:2888:3888
server.3=hadoop03:2888:3888
二、分发版本包并启动服务
### --- 分发到其它两台主机上
~~~ # 将Flink版本包发送到其它节点
[root@hadoop01 ~]# rsync-script /opt/yanqi/servers/flink-1.11.1/
### --- standalone模式启动
~~~ # 在Hadoop01上启动Flink整个集群
[root@hadoop01 ~]# cd /opt/yanqi/servers/flink-1.11.1/bin/
[root@hadoop01 bin]# ./start-cluster.sh
~~~ 输出参数
Starting cluster.
Starting standalonesession daemon on host hadoop01.
Starting taskexecutor daemon on host hadoop01.
Starting taskexecutor daemon on host hadoop02.
Starting taskexecutor daemon on host hadoop03.
### --- jps进程查看核实
[root@hadoop00 ~]# jps
~~~ 进程分布
Hadoop01 TaskManagerRunner
Hadoop01 StandaloneSessionClusterEntrypoint
Hadoop02 TaskManagerRunner
Hadoop03 TaskManagerRunner
三、通过web_UI访问Flink:查看Flink的web页面: http://hadoop01:8081/

四、修改任务槽,查看UI展示界面
### --- 修改配置文件:flink-conf.yaml
### --- 查看Flink的web页面: http://hadoop01:8081/
~~~ # 修改配置文件任务槽
[root@hadoop01 ~]# vim /opt/yanqi/servers/flink-1.11.1/conf/flink-conf.yaml
taskmanager.numberOfTaskSlots: 2 # 每一个task当中任务槽的数量,默认是1个
~~~ # 发送到其它节点
[root@hadoop01 ~]# rsync-script /opt/yanqi/servers/flink-1.11.1/conf/flink-conf.yaml
~~~ # 重启Flink服务
[root@hadoop01 ~]# /opt/yanqi/servers/flink-1.11.1/bin/stop-cluster.sh
[root@hadoop01 ~]# /opt/yanqi/servers/flink-1.11.1/bin/start-cluster.sh


Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart
——W.S.Landor
分类:
dov004-flink
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】凌霞软件回馈社区,博客园 & 1Panel & Halo 联合会员上线
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】博客园社区专享云产品让利特惠,阿里云新客6.5折上折
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· 没有源码,如何修改代码逻辑?
· PowerShell开发游戏 · 打蜜蜂
· 在鹅厂做java开发是什么体验
· 百万级群聊的设计实践
· WPF到Web的无缝过渡:英雄联盟客户端的OpenSilver迁移实战