mesos集群安装，以及spark集群搭建

一. 配置信息：
3台centos7，一台master，两台slave。（无zookeeper安装）

 

二. mesos安装：
yum install http://repos.mesosphere.io/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm
yum install -y mesos

Master:

vim /etc/default/mesos-master
PORT=5050
IP={masterIp}

vim /etc/mesos-master/cluster
ClusterName

vim /etc/mesos-master/hostname
HostName

systemctl start mesos-master.service
systemctl enable mesos-master.service

Slave:

vim /etc/default/mesos-slave
IP={slaveIp}
MASTER={masterIp}:5050/mesos
ISOLATION=cgroups/cpu,cgroups/mem,docker/runtime,filesystem/linux

vim /etc/mesos-slave/hostname
HostName

systemctl start mesos-slave.service
systemctl enable mesos-slave.service

注：更改hostname，rm -rf /var/lib/mesos/* 后，重启。

 

三. 环境变量配置：

vim .bashrc
export SPARK_HOME=/opt/spark-2.4.3-bin-hadoop2.7
export PATH=$PATH:/opt/spark-2.4.3-bin-hadoop2.7/bin
export PATH=$PATH:/opt/spark-2.4.3-bin-hadoop2.7/sbin
export SPARK_LOCAL_IP=10.0.0.11
export LIBPROCESS_IP=10.0.0.11

 

[1] https://www.jianshu.com/p/22a5ed4db6a5
[2] https://blog.csdn.net/qq_35440040/article/details/78427516
[3] https://spark.apache.org/docs/latest/configuration.html
[4] https://stackoverflow.com/questions/24637312/spark-driver-in-apache-spark