|NO.Z.00004|——————————|Deployment|——|Hadoop&Flink计算领域锋利的武器.v04|——|Flink.v01|StandAlone模式部署|

一、Flink安装和部署
### --- Flink支持多种安装模式

~~~     local(本地):单机模式,一般本地开发调试使用
~~~     StandAlone 独立模式:Flink自带集群,自己管理资源调度,生产环境也会有所应用
~~~     Yarn模式:计算资源统一由Hadoop YARN管理,生产环境应用较多
二、环境准备工作
### --- 基础环境

~~~     # jdk1.8及以上【配置JAVA_HOME环境变量】
~~~     # ssh免密码登录【集群内节点之间免密登录】
[root@hadoop01 ~]# java -version
java version "1.8.0_231"

[root@hadoop01 ~]# cat .ssh/authorized_keys 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDf1rEa6F/zlreckTPEmVW4gHVi8Q6KUkQH6fN95EiqiEXvoudntlPjq5mdT6VQupuDCJMIbe6pMG276Rn7uDWrY9NZC3uIx3V8PwWbm+mSPJS9uxVYLPxTo4bd8HQrZ5VKsh2NMr+CnVq5Kz5at8z4X3QleeJjCwxbAKHyl4iMsgOTF1/6geNm+0mQiNSvdMDDvmr1wvEtfGMBwKxpCs5QYUuA0ty8nvCQXvoe7AUu3NGNeg494V78uh5gtR+0mB+xbCyVZd61XI0nooPWKpgHyAcsb/mxl8DSDJ2HoZVUWc1D/+KavvOUawAzA7B+/SGjtbRc1RHOMpEmNph6Jk4F root@hadoop01
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfPLjiMFvSWUIDHGybv58EFwpHC6+wg5/kvBy+86pgvzjFTIV4FN4Rr1gTsaV+38YfvXOt+zAD+DAcZPhgzXjhoii32g4s23Bvwdbru2trb/G0pXBM9rnDZ8i/Q2XQN3DPCIzJ3Vk45ab9dMarj1NvMyMt+3pHgut5QtCFlNG1nJVYZuyKPYnYo9LyPxqIdcL9KrJ/y/a8zUdhbm5OUST0WEFx3bRiP4yTjtX6G4s/NM6ajlgmADCB2VzxSYNYsIL0Y2k1YKqLcIluH8kjCPekmRkhnDduEm5DjuaPjuVoEMNQMzV1qbQUhA4l8PUsL9gF1s08culbqdMfYnOqyxJx root@hadoop02
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNSwd/9bKDf4AL01SJ5elFAAd+MMzE/O9YfWrgpas/iAUCadKEPLeaoUF2x23Pn87F5sGk7vssyNjYQPgFxXVJcSc2CI/0s3xKjo/Yi8hGhc4SvCqZawQVFaMd+raE5jzbDNQF/B2n0qrBTt3Wnnz5YDKcw0+cYe7vwUVlshVM5+vdhjuSDMA7FROjGIdjxfxWG7hXcJEmlRHR4TEq9oNSl5xSpblspROIzqfRnNYqplJmkML+AsVOtJfOGFCO/197o4YX79MFyYXBF9rk3Ycklp06dkbI/BB1v+9Du2DGGkiCFmRvmUG/qD0HWLFAoE808JonNoDdICXz6yqeUXzT root@hadoop03
### --- 安装包下载

~~~     下载地址:https://archive.apache.org/dist/flink/flink-1.11.1/
### --- 下载Flink版本包:

[root@hadoop01 ~]# wget https://archive.apache.org/dist/flink/flink-1.11.1/flink-1.11.1-bin-scala_2.12.tgz
三、集群规划
Hadoop01 Hadoop02 Hadoop03
JobManager+TaskManager TaskManager TaskManager

一、Flink.Standalone模式部署:StandAlone模式部署
### --- 下载版本包并解压到目标文件夹

~~~     # 下载版本包并解压
[root@hadoop01 ~]# ls /opt/yanqi/software/flink-1.11.1-bin-scala_2.12.tgz 
/opt/yanqi/software/flink-1.11.1-bin-scala_2.12.tgz
~~~     # 解压版本包

[root@hadoop01 ~]# cd /opt/yanqi/software/
[root@hadoop01 software]# tar -zxvf flink-1.11.1-bin-scala_2.12.tgz \
-C ../servers/
### --- 修改配置文件:flink-conf.yaml

~~~     # 修改配置文件
[root@hadoop01 ~]# vim /opt/yanqi/servers/flink-1.11.1/conf/flink-conf.yaml
 
jobmanager.rpc.address: hadoop01            # 设置rpc通信地址设置为hadoop01;flink集群中master地址
taskmanager.numberOfTaskSlots: 1            # 每一个task当中任务槽的数量,默认是1个 
parallelism.default: 1                      # Flink任务中的并行度
### --- 修改配置文件:master

[root@hadoop01 ~]# vim /opt/yanqi/servers/flink-1.11.1/conf/masters
hadoop01:8081                               # 设置master主机地址
### --- 修改配置文件:worker

[root@hadoop01 ~]# vim /opt/yanqi/servers/flink-1.11.1/conf/workers
hadoop01
hadoop02
hadoop03
### --- 修改配置文件:zoo.cnf

~~~     # zoo.cnf:Flink集群的机制是依赖zookeeper的
[root@hadoop01 ~]# vim /opt/yanqi/servers/flink-1.11.1/conf/zoo.cfg
~~~ 配置zk的主机信息
server.1=hadoop01:2888:3888
server.2=hadoop02:2888:3888
server.3=hadoop03:2888:3888
二、分发版本包并启动服务
### --- 分发到其它两台主机上

~~~     # 将Flink版本包发送到其它节点
[root@hadoop01 ~]# rsync-script /opt/yanqi/servers/flink-1.11.1/
### --- standalone模式启动

~~~     # 在Hadoop01上启动Flink整个集群
[root@hadoop01 ~]# cd /opt/yanqi/servers/flink-1.11.1/bin/
[root@hadoop01 bin]# ./start-cluster.sh
~~~ 输出参数
Starting cluster.
Starting standalonesession daemon on host hadoop01.
Starting taskexecutor daemon on host hadoop01.
Starting taskexecutor daemon on host hadoop02.
Starting taskexecutor daemon on host hadoop03.
### --- jps进程查看核实

[root@hadoop00 ~]# jps
~~~ 进程分布
Hadoop01    TaskManagerRunner
Hadoop01    StandaloneSessionClusterEntrypoint

Hadoop02    TaskManagerRunner

Hadoop03    TaskManagerRunner
三、通过web_UI访问Flink:查看Flink的web页面: http://hadoop01:8081/
四、修改任务槽,查看UI展示界面
### --- 修改配置文件:flink-conf.yaml
### --- 查看Flink的web页面: http://hadoop01:8081/

~~~     # 修改配置文件任务槽
[root@hadoop01 ~]# vim /opt/yanqi/servers/flink-1.11.1/conf/flink-conf.yaml
taskmanager.numberOfTaskSlots: 2                 # 每一个task当中任务槽的数量,默认是1个 
~~~     # 发送到其它节点
[root@hadoop01 ~]# rsync-script /opt/yanqi/servers/flink-1.11.1/conf/flink-conf.yaml
~~~     # 重启Flink服务

[root@hadoop01 ~]# /opt/yanqi/servers/flink-1.11.1/bin/stop-cluster.sh 
[root@hadoop01 ~]# /opt/yanqi/servers/flink-1.11.1/bin/start-cluster.sh 

 
 
 
 
 
 
 
 
 

Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart
                                                                                                                                                   ——W.S.Landor

 

posted on   yanqi_vip  阅读(67)  评论(0编辑  收藏  举报

相关博文:
阅读排行:
· 没有源码,如何修改代码逻辑?
· PowerShell开发游戏 · 打蜜蜂
· 在鹅厂做java开发是什么体验
· 百万级群聊的设计实践
· WPF到Web的无缝过渡:英雄联盟客户端的OpenSilver迁移实战
< 2025年2月 >
26 27 28 29 30 31 1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 1
2 3 4 5 6 7 8

导航

统计

点击右上角即可分享
微信分享提示