cloudera-scm-agent dead but pid file exists

问题一：

错误描述：

/opt/cm-5.7.0/etc/init.d/cloudera-scm-agent status

cloudera-scm-agent dead but pid file exists

 查看日志/opt/cm-5.7.0/log/cloudera-scm-agent/cloudera-scm-agent.log：

No socket could be created on ('testintf.novalocal', 9000) -- [Errno 99] Cannot assign requested address

 

此问题主要是网络问题

1. 

python -c 'import socket; print socket.getfqdn(), socket.gethostbyname(socket.getfqdn())'命令获取/etc/hosts文件中的IP和hostname

正规hosts如下：
127.0.0.1 localhost.xxxx   localhost
111.222.333.444 aa.aa　　aa
555.666.777.888 bb.bb 　　bb

上述命令获取结果为 :111.111.111.111 aa.aa

此IP和ifocnfig中获取的IP相同，（有公网和内网的同学，请自觉选择内网ip）
hostname和hostname命令获取的名称一样。


2. 同在一个内网的几台服务器之间是相互通信的，但是使用公网IP就不可以了，所以CDH集群中需要大量的端口通信，所以在设置ocnfig.ini文件中的server_host时，选择内网IP。

 

 

 

问题二：cm界面安装时，agent服务不起，所在服务器不受管。导致后面agent时界面安装的。在界面安装中出现以下错误提示：

 解决办法：

１.

python -c 'import socket; print socket.getfqdn(), socket.gethostbyname(socket.getfqdn())'

　　和

hostname

　　两种方式得出的主机名不同造成的。

 

 

 

２.　  Ensure that port 7182 is accessible on the Cloudera Manager Server (check firewall rules).

　　　telnet  112.35.23.45 7182

　　　ps -ef |grep PID?　　

 

 

３．　 Ensure that ports 9000 and 9001 are free on the host being added.

　　netstat|grep 9000

　　netstat |grep 9001

 

４．　 Check agent logs in /var/log/cloudera-scm-agent/ on the host being added (some of the logs can be found in the installation details)

这个目录时agent服务起来之后才有的，如果agent 启动失败，则不会有。

 

 

问题三：

[22/Oct/2018 18:49:13 +0000] 3131 MainThread agent ERROR Failed! trying again in 1 second(s)
Traceback (most recent call last):
File "/opt/cm-5.7.1/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.7.1-py2.6.egg/cmf/agent.py", line 2161, in connect_to_new_supervisor
self.get_supervisor_process_info()
File "/opt/cm-5.7.1/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.7.1-py2.6.egg/cmf/agent.py", line 2183, in get_supervisor_process_info
self.identifier = self.supervisor_client.supervisor.getIdentification()
File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
return self.__send(self.__name, args)
File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
verbose=self.__verbose
File "/opt/cm-5.7.1/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0-py2.6.egg/supervisor/xmlrpc.py", line 470, in request
'' )
ProtocolError: <ProtocolError for 127.0.0.1/RPC2: 401 Unauthorized>
[22/Oct/2018 18:49:13 +0000] 3131 MainThread agent ERROR Failed to connect to newly launched supervisor. Agent will exit

解决办法：

kill 掉supervisored的进程，重启，多试几次就好了。