cloud-manage

cento os 7 cloud-manage下载地址

http://archive.cloudera.com/cm5/cm/5/

http://archive.cloudera.com/cdh5/parcels/latest/

mysql下载地址

http://mirrors.sohu.com/mysql/MySQL-5.6/

 

卸载mysql

[root@hadoop01 software]# rpm -qa | grep -i mysql
MySQL-shared-compat-5.6.39-1.el7.x86_64
MySQL-client-5.6.39-1.el7.x86_64
MySQL-server-5.6.39-1.el7.x86_64
[root@hadoop01 software]# service mysql stop
Shutting down MySQL.... SUCCESS! 
[root@hadoop01 software]# rpm -e --nodeps MySQL-shared-compat-5.6.39-1.el7.x86_64
[root@hadoop01 software]# rpm -e --nodeps MySQL-client-5.6.39-1.el7.x86_64
[root@hadoop01 software]# rpm -e --nodeps MySQL-server-5.6.39-1.el7.x86_64
[root@hadoop01 software]# 

安装mysql

[root@hadoop01 software]# rpm -ivh MySQL-client-5.6.39-1.el7.x86_64
[root@hadoop01 software]# rpm -ivh MySQL-server-5.6.39-1.el7.x86_64
[root@hadoop01 software]# rpm -ivh MySQL-shared-compat-5.6.39-1.el7.x86_64

添加cloudera-scm用户

useradd --system --home=/opt/cm-5.16.2/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

用户授权

chown -R cloudera-scm:cloudera-scm c*

配置agent

 vim cm-5.16.2/etc/cloudera-scm-agent/config.ini 
 server_host=hadoop01

配置cloudera-scm数据库

mv mysql-connector-java-5.1.27.jar /usr/share/java/mysql-connector-java.jar

cd /opt/cm-5.16.2/share/cmf/schema

　./scm_prepare_database.sh mysql cm -hhadoop01 -uroot -proot --scm-host hadoop01 scm scm scm

分发cm-5.16.2

xsync cm-5.16.2/

重启server

/opt/cm-5.16.2/etc/init.d/cloudera-scm-server restart

重启agent

/opt/cm-5.16.2/etc/init.d/cloudera-scm-agent restart

查看端口7180 代表启动成功

netstat -lntp

启动失败

2020-08-09 08:44:01,623 ERROR SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: The server storage directory [/var/lib/cloudera-scm-server] doesn't exist.
2020-08-09 08:44:01,643 ERROR SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: No read permission to the server storage directory [/var/lib/cloudera-scm-server]
2020-08-09 08:44:01,643 ERROR SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: No write permission to the server storage directory [/var/lib/cloudera-scm-server]
2020-08-09 08:44:03,413 INFO WebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.4
2020-08-09 08:44:03,437 INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:7180
2020-08-09 08:44:03,437 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

网上说缺少server端rpm（但是并不是）很明显没有这个目录/var/lib/cloudera-scm-server

cloudera-manager-server-5.16.2-1.cm5162.p0.7.el7.x86_64.rpm

 下载地址 http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.16.2/RPMS/x86_64/

安装好cm后，需要配置JAVA_HOME，否则使用hdfs命令会显示没有配置java环境变量

[qiunan@hadoop01 bin]$ echo $JAVA_HOME
/opt/module/jdk1.8.0_251

[qiunan@hadoop01 bin]$ sudo mkdir -p /usr/java

[qiunan@hadoop01 bin]$ ln -s /opt/module/jdk1.8.0_251/ /usr/java/default

 flink bin目录提交flink作业到yarn上

[qiunan@hadoop01 bin]$ ./flink run -m yarn-cluster -c org.apache.flink.streaming.examples.wordcount.WordCount /opt/module/flink-1.10.1/examples/streaming/

报错：

Caused by: org.apache.hadoop.security.AccessControlException: Permission denied: user=qiunan, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

 用hdfs用户创建目录并授权

sudo -u hdfs hadoop fs -mkdir /user/qiunan
sudo -u hdfs hadoop fs -chmod 777 /user/qiunan

任务提交成功