部署Cloudera Manager Server实战案例
部署Cloudera Manager Server实战案例
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.前置准备
1>.大数据集群系统环境配置
博主推荐阅读: https://www.cnblogs.com/yinzhengjie/articles/11019333.html
2>.建立cloudera manager相关组件需要的表
[root@cdh101.yinzhengjie.org.cn ~]# mysql -uroot -pyinzhengjie Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 11 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> SHOW DATABASES; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | +--------------------+ 3 rows in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> CREATE DATABASE cloudera_manager CHARACTER SET = utf8; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> GRANT ALL PRIVILEGES ON cloudera_manager.* TO 'cm'@'172.200.2.%' IDENTIFIED BY 'yinzhengjie' WITH GRANT OPTION; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SHOW DATABASES; +--------------------+ | Database | +--------------------+ | information_schema | | cloudera_manager | | mysql | | performance_schema | +--------------------+ 4 rows in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SELECT user,host,password FROM mysql.user; +------+-------------+-------------------------------------------+ | user | host | password | +------+-------------+-------------------------------------------+ | root | localhost | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | root | 127.0.0.1 | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | root | ::1 | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | cm | 172.200.2.% | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | +------+-------------+-------------------------------------------+ 4 rows in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SHOW GRANTS FOR 'cm'@'172.200.2.%'; +-------------------------------------------------------------------------------------------------------------+ | Grants for cm@172.200.2.% | +-------------------------------------------------------------------------------------------------------------+ | GRANT USAGE ON *.* TO 'cm'@'172.200.2.%' IDENTIFIED BY PASSWORD '*BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7' | | GRANT ALL PRIVILEGES ON `cloudera_manager`.* TO 'cm'@'172.200.2.%' WITH GRANT OPTION | +-------------------------------------------------------------------------------------------------------------+ 2 rows in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> QUIT Bye [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]# mysql -u cm -pyinzhengjie -h cdh101.yinzhengjie.org.cn Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 14 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> SHOW DATABASES; +--------------------+ | Database | +--------------------+ | information_schema | | cloudera_manager | +--------------------+ 2 rows in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> QUIT Bye [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]#
[root@cdh101.yinzhengjie.org.cn ~]# mysql -uroot -pyinzhengjie Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 12 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> CREATE DATABASE hive CHARACTER SET = utf8; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> GRANT ALL PRIVILEGES ON hive.* TO 'hive'@'172.200.2.%' IDENTIFIED BY 'yinzhengjie' WITH GRANT OPTION; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> FLUSH PRIVILEGES; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SELECT user,host,password FROM mysql.user; +------+-------------+-------------------------------------------+ | user | host | password | +------+-------------+-------------------------------------------+ | root | localhost | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | root | 127.0.0.1 | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | root | ::1 | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | hive | 172.200.2.% | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | | cm | 172.200.2.% | *BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7 | +------+-------------+-------------------------------------------+ 5 rows in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> SHOW GRANTS FOR 'hive'@'172.200.2.%'; +---------------------------------------------------------------------------------------------------------------+ | Grants for hive@172.200.2.% | +---------------------------------------------------------------------------------------------------------------+ | GRANT USAGE ON *.* TO 'hive'@'172.200.2.%' IDENTIFIED BY PASSWORD '*BD0B1F48FDC55BD27555FC2F22FF29A68A25A1D7' | | GRANT ALL PRIVILEGES ON `hive`.* TO 'hive'@'172.200.2.%' WITH GRANT OPTION | +---------------------------------------------------------------------------------------------------------------+ 2 rows in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> QUIT Bye [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]# mysql -u hive -pyinzhengjie -h cdh101.yinzhengjie.org.cn Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 13 Server version: 5.5.64-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> MariaDB [(none)]> SHOW DATABASES; +--------------------+ | Database | +--------------------+ | information_schema | | hive | +--------------------+ 2 rows in set (0.00 sec) MariaDB [(none)]> MariaDB [(none)]> QUIT Bye [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]#
二.配置本地软件源
1>.下载cloudera manager(简称CM)软件包
博主推荐阅读: https://archive.cloudera.com/cm6/ https://archive.cloudera.com/cm6/6.2.1/redhat7/yum/ 博主推荐下载地址: https://archive.cloudera.com/cm6/6.2.1/redhat7/yum/RPMS/x86_64/
2>.下载Cloudera’s Distribution Including Apache Hadoop(简称CDH)软件包
博主推荐阅读: https://archive.cloudera.com/cdh6/ https://archive.cloudera.com/cdh6/6.2.1/ 博主推荐下载地址: https://archive.cloudera.com/cdh6/6.2.1/parcels/
3>.制作内网的CM软件源仓库
[root@cdh101.yinzhengjie.org.cn ~]# yum -y install createrepo Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.tuna.tsinghua.edu.cn * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn Resolving Dependencies --> Running transaction check ---> Package createrepo.noarch 0:0.9.9-28.el7 will be installed --> Processing Dependency: python-deltarpm for package: createrepo-0.9.9-28.el7.noarch --> Processing Dependency: libxml2-python for package: createrepo-0.9.9-28.el7.noarch --> Processing Dependency: deltarpm for package: createrepo-0.9.9-28.el7.noarch --> Running transaction check ---> Package deltarpm.x86_64 0:3.6-3.el7 will be installed ---> Package libxml2-python.x86_64 0:2.9.1-6.el7_2.3 will be installed ---> Package python-deltarpm.x86_64 0:3.6-3.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================================================================================================== Installing: createrepo noarch 0.9.9-28.el7 base 94 k Installing for dependencies: deltarpm x86_64 3.6-3.el7 base 82 k libxml2-python x86_64 2.9.1-6.el7_2.3 base 247 k python-deltarpm x86_64 3.6-3.el7 base 31 k Transaction Summary ============================================================================================================================================================================================================================================================================== Install 1 Package (+3 Dependent packages) Total download size: 454 k Installed size: 2.0 M Downloading packages: (1/4): createrepo-0.9.9-28.el7.noarch.rpm | 94 kB 00:00:00 (2/4): libxml2-python-2.9.1-6.el7_2.3.x86_64.rpm | 247 kB 00:00:00 (3/4): deltarpm-3.6-3.el7.x86_64.rpm | 82 kB 00:00:00 (4/4): python-deltarpm-3.6-3.el7.x86_64.rpm | 31 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 691 kB/s | 454 kB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : deltarpm-3.6-3.el7.x86_64 1/4 Installing : python-deltarpm-3.6-3.el7.x86_64 2/4 Installing : libxml2-python-2.9.1-6.el7_2.3.x86_64 3/4 Installing : createrepo-0.9.9-28.el7.noarch 4/4 Verifying : createrepo-0.9.9-28.el7.noarch 1/4 Verifying : libxml2-python-2.9.1-6.el7_2.3.x86_64 2/4 Verifying : deltarpm-3.6-3.el7.x86_64 3/4 Verifying : python-deltarpm-3.6-3.el7.x86_64 4/4 Installed: createrepo.noarch 0:0.9.9-28.el7 Dependency Installed: deltarpm.x86_64 0:3.6-3.el7 libxml2-python.x86_64 0:2.9.1-6.el7_2.3 python-deltarpm.x86_64 0:3.6-3.el7 Complete! [root@cdh101.yinzhengjie.org.cn ~]#
[root@cdh101.yinzhengjie.org.cn ~]# ll /var/www/html/ total 0 [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]# mkdir /var/www/html/cm6.2.1 [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]# cd /var/www/html/cm6.2.1 [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# ll #如下图所示，通过文件传输工具将软件包下载到本地 total 1376144 -rw-r--r-- 1 root root 14041 Mar 7 10:14 allkeys.asc -rw-r--r-- 1 root root 10215608 Mar 7 10:14 cloudera-manager-agent-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 1199718256 Mar 7 10:15 cloudera-manager-daemons-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 9988 Mar 7 10:14 cloudera-manager-server-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 10996 Mar 7 10:14 cloudera-manager-server-db-2-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 14200072 Mar 7 10:14 enterprise-debuginfo-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 184988341 Mar 7 10:14 oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# createrepo . Spawning worker 0 with 3 pkgs Spawning worker 1 with 3 pkgs Workers Finished Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# ll total 1376148 -rw-r--r-- 1 root root 14041 Mar 7 10:14 allkeys.asc -rw-r--r-- 1 root root 10215608 Mar 7 10:14 cloudera-manager-agent-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 1199718256 Mar 7 10:15 cloudera-manager-daemons-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 9988 Mar 7 10:14 cloudera-manager-server-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 10996 Mar 7 10:14 cloudera-manager-server-db-2-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 14200072 Mar 7 10:14 enterprise-debuginfo-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 184988341 Mar 7 10:14 oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm drwxr-xr-x 2 root root 4096 Mar 7 10:19 repodata [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]#
[root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# ll total 1376148 -rw-r--r-- 1 root root 14041 Mar 7 10:14 allkeys.asc -rw-r--r-- 1 root root 10215608 Mar 7 10:14 cloudera-manager-agent-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 1199718256 Mar 7 10:15 cloudera-manager-daemons-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 9988 Mar 7 10:14 cloudera-manager-server-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 10996 Mar 7 10:14 cloudera-manager-server-db-2-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 14200072 Mar 7 10:14 enterprise-debuginfo-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 184988341 Mar 7 10:14 oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm drwxr-xr-x 2 root root 4096 Mar 7 10:19 repodata [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# ll repodata/ total 272 -rw-r--r-- 1 root root 3278 Mar 7 10:19 1f244ed8df75415f4ee8ea560ec72da1c50e9a206984e15c1de7363deda7332a-primary.xml.gz -rw-r--r-- 1 root root 8655 Mar 7 10:19 536d1f06decf2ed5e1e6becdcd9c53d070833472099e0a5628c135017b44e745-primary.sqlite.bz2 -rw-r--r-- 1 root root 1006 Mar 7 10:19 63db56e3214c566de5c6c5b1d205eca0b01e842e845e168a84f76f6033f1b9d7-other.sqlite.bz2 -rw-r--r-- 1 root root 125191 Mar 7 10:19 69d3d483cd320a431671829fa904f0f8ca730a8d5cb3295aa7cd18e4db966909-filelists.xml.gz -rw-r--r-- 1 root root 529 Mar 7 10:19 afd033ea9e4d3005f4508afe7efaba110a716232610fa7131361546d76b94e8c-other.xml.gz -rw-r--r-- 1 root root 120616 Mar 7 10:19 e8f567a04418df17614ccc86b5b3c74d28256e984eecf004e2ee48d0b6a2b6cc-filelists.sqlite.bz2 -rw-r--r-- 1 root root 2979 Mar 7 10:19 repomd.xml [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]#
4>.浏览器访问cm的软件源
[root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# wget https://archive.cloudera.com/cm6/6.2.1/redhat7/yum/RPM-GPG-KEY-cloudera --2020-03-07 10:30:19-- https://archive.cloudera.com/cm6/6.2.1/redhat7/yum/RPM-GPG-KEY-cloudera Resolving archive.cloudera.com (archive.cloudera.com)... 151.101.228.167 Connecting to archive.cloudera.com (archive.cloudera.com)|151.101.228.167|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1718 (1.7K) [binary/octet-stream] Saving to: ‘RPM-GPG-KEY-cloudera’ 100%[=====================================================================================================================================>] 1,718 --.-K/s in 0s 2020-03-07 10:30:20 (306 MB/s) - ‘RPM-GPG-KEY-cloudera’ saved [1718/1718] [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# ll total 1376152 -rw-r--r-- 1 root root 14041 Mar 7 10:14 allkeys.asc -rw-r--r-- 1 root root 10215608 Mar 7 10:14 cloudera-manager-agent-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 1199718256 Mar 7 10:15 cloudera-manager-daemons-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 9988 Mar 7 10:14 cloudera-manager-server-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 10996 Mar 7 10:14 cloudera-manager-server-db-2-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 14200072 Mar 7 10:14 enterprise-debuginfo-6.2.1-1426065.el7.x86_64.rpm -rw-r--r-- 1 root root 184988341 Mar 7 10:14 oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm drwxr-xr-x 2 root root 4096 Mar 7 10:19 repodata -rw-r--r-- 1 root root 1718 Mar 7 10:30 RPM-GPG-KEY-cloudera [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# cat RPM-GPG-KEY-cloudera -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1 mQENBFbGIf4BCAC1sikg19e42rYG9BKy2i/RWw2vlee2MrbgvvJN8jRYE6QGQchx miGWaH97UEHWWSvZ0bHld7c+2Km83Wa71NxsfGUfI837gmQbM3BYEtGWfgTNOiBJ rV6vSfSF64lk8VPdYdbyy3nPWaNm+qMQk5T0iGyOUz3a9+Aqo/4x87WpAy1k55qA fxgY1STcgKhw5MQ30eXJLsNvRUf3m/SiiXwLEc0EbU9ZesrJDZ/DPjOzBEwW4YPJ S8uPWczmd4e/uNxi/ZAEiGMaFBOL88Y2V9NRxAgpGBKPdu3+FaC1Xvtw+gmPHjRf 25irP2UtQdIqkMAyVn9yLmDcyeC6FBVlZPPFABEBAAG0K1BhcmFtZXRlcml6ZWQg QnVpbGQgPHNlY3VyaXR5QGNsb3VkZXJhLmNvbT6JATkEEwECACMFAlsW+osCGwMH CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRBzmF1DsLGcn1JmCACoL27DZEyL /MFHeeKFA5M4SVZ4+qOsg4Lua8kn7wv6YqscRwhjx/HE/6DxEtskpPQjUqml5a1j gUodiS6N7PPF+kB+Lf/D1R+lqRf9uux/efu+VvJOlYcpjvEB0yQuXgHlR59gzrPJ vBZexE9xV0dvzzUKTxA6jmOkaIt1qKv0Y7BrZAMzB97RWMyW8H/Nbx0LxMYhnN6p whb5b993MTEecv6y61d4e6qNeOKavkNUhiWL9r4NZgdqM11v3GZ1C6VSNYYKwohH 35dtCuTYB2Phf5QTXoiWFKqGJKiJi0vHBukV1+6Y2WdgdnKKjiz9/FoeVCdUueWD z21yAQ9vyldluQENBFbGIf4BCADBWH7uZqLHP+s5mxyGFGvIkeohax1bD2sjQh6m +rvjXVfn9CjQofjyAI1q+FCTvcOevC60y1SIitKAPHNhEI22lh0k9jh0ORb5pWms 3pe36H8+2aBLrOj4qVyRRIFXhbLAUZXenhP+nH0yBMqk5LH3cXOQmZZEJgMkQg+q Ft7DLVwR3wse2FFYeL12CEYSKG/4bfQHp9w9MErufKiFGPF/e4lHOVTc7Q2ynxwk GsfNqEZxD+DJWHnsd0GY+sjeLKaj+EaWJIzw1+KSsVw2GATKx0u7LD0r1jTkqYCj oMVPEvAowMeaVYK4QbLysQJlHJLB7IYq2HYtL0ypkFvVdLgBABEBAAGJAR8EGAEC AAkFAlbGIf4CGwwACgkQc5hdQ7CxnJ+juwf9FT9PgfIc5c130eQOnNuOpVPF7p1U c3YJLuwPf60qPQqzk8ZD0pX2BM5DmI8yzmybn/C+PuNRWEPbuaS5SqGYCn6EYj1Q Flbt0iqt4DgP2/ntWz7mdLgWfn7/h8ruOoITF2vRc0d9Vpf3XMXYB1Fy9LbnArxL CJwj60QMsI8cJ0Eajj0wUfMAXztj7gcm+hTMGMh5GMoOZ+7bUcr3HTSPpXKUeAq6 ATEzZ++0ZwAYjCC2bGFjDDQf8R2EdYwF2fZQ6J4t+E46YZo91X4BqM88mCDKzkYD 4KxEX8xwAhW6dAjyCyYXA4Ejo/rdLWBfe0mcOpWbKAINY2Bv+SavbLUMVA== =VXpq -----END PGP PUBLIC KEY BLOCK----- [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]# [root@cdh101.yinzhengjie.org.cn /var/www/html/cm6.2.1]#
浏览器输入如下地址: http://cdh101.yinzhengjie.org.cn/cm6.2.1/
5>.所有节点配置自建的CM软件源仓库
[root@cdh101.yinzhengjie.org.cn ~]# cat /etc/yum.repos.d/cloudera_manager.repo [YinZhengjie_cloudera_manager_repo] name = cm_repo baseurl = http://cdh101.yinzhengjie.org.cn/cm6.2.1/ enabled=1 gpgcheck=1 gpgkey=http://cdh101.yinzhengjie.org.cn/cm6.2.1/RPM-GPG-KEY-cloudera [root@cdh101.yinzhengjie.org.cn ~]#
[root@cdh101.yinzhengjie.org.cn ~]# yum clean all Loaded plugins: fastestmirror Cleaning repos: YinZhengjie_cloudera_manager_repo base extras updates [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]# yum repolist Loaded plugins: fastestmirror Determining fastest mirrors * base: mirrors.aliyun.com * extras: mirrors.huaweicloud.com * updates: mirrors.aliyun.com YinZhengjie_cloudera_manager_repo | 2.9 kB 00:00:00 base | 3.6 kB 00:00:00 extras | 2.9 kB 00:00:00 updates | 2.9 kB 00:00:00 (1/5): YinZhengjie_cloudera_manager_repo/primary_db | 8.5 kB 00:00:00 (2/5): base/7/x86_64/group_gz | 165 kB 00:00:00 (3/5): extras/7/x86_64/primary_db | 164 kB 00:00:00 (4/5): updates/7/x86_64/primary_db | 6.7 MB 00:00:01 (5/5): base/7/x86_64/primary_db | 6.0 MB 00:00:02 repo id repo name status YinZhengjie_cloudera_manager_repo cm_repo 6 base/7/x86_64 CentOS-7 - Base 10,097 extras/7/x86_64 CentOS-7 - Extras 335 updates/7/x86_64 CentOS-7 - Updates 1,487 repolist: 11,925 [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]# cat /etc/yum.repos.d/cloudera_manager.repo [YinZhengjie_cloudera_manager_repo] name = cm_repo baseurl = http://cdh101.yinzhengjie.org.cn/cm6.2.1/ enabled=1 gpgcheck=1 gpgkey=http://cdh101.yinzhengjie.org.cn/cm6.2.1/RPM-GPG-KEY-cloudera [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]#
6>.自定义Parcel 存储库(后续在配置Cloudera Manger Server的WebUI中需要指定该URL)
[root@cdh101.yinzhengjie.org.cn ~]# cd /var/www/html/ [root@cdh101.yinzhengjie.org.cn /var/www/html]# ll total 4 drwxr-xr-x 3 root root 4096 Mar 7 10:30 cm6.2.1 [root@cdh101.yinzhengjie.org.cn /var/www/html]# [root@cdh101.yinzhengjie.org.cn /var/www/html]# [root@cdh101.yinzhengjie.org.cn /var/www/html]# mkdir cdh6.2.1 [root@cdh101.yinzhengjie.org.cn /var/www/html]# [root@cdh101.yinzhengjie.org.cn /var/www/html]# ll total 4 drwxr-xr-x 2 root root 6 Mar 7 17:52 cdh6.2.1 drwxr-xr-x 3 root root 4096 Mar 7 10:30 cm6.2.1 [root@cdh101.yinzhengjie.org.cn /var/www/html]# [root@cdh101.yinzhengjie.org.cn /var/www/html]# cd cdh6.2.1/ [root@cdh101.yinzhengjie.org.cn /var/www/html/cdh6.2.1]# [root@cdh101.yinzhengjie.org.cn /var/www/html/cdh6.2.1]# ll #如下图所示，通过Xshell自带的文件传输工具将已经下载好的CDH文件上传到apache的网页目录中 total 2044316 -rw-r--r-- 1 root root 2093332003 Mar 7 17:53 CDH-6.2.1-1.cdh6.2.1.p0.1425774-el7.parcel -rw-r--r-- 1 root root 40 Mar 7 17:52 CDH-6.2.1-1.cdh6.2.1.p0.1425774-el7.parcel.sha1 -rw-r--r-- 1 root root 64 Mar 7 17:52 CDH-6.2.1-1.cdh6.2.1.p0.1425774-el7.parcel.sha256 -rw-r--r-- 1 root root 33875 Mar 7 17:52 manifest.json [root@cdh101.yinzhengjie.org.cn /var/www/html/cdh6.2.1]# [root@cdh101.yinzhengjie.org.cn /var/www/html/cdh6.2.1]#
7>.浏览器访问cdh的URL
浏览器输入如下地址: http://cdh101.yinzhengjie.org.cn/cdh6.2.1/
三.部署Cloudera Manager Server
1>.安装java环境
[root@cdh101.yinzhengjie.org.cn ~]# yum -y install oracle-j2sdk1.8-1.8.0+update181-1.x86_64
2>.安装Cloudera Manager Server
[root@cdh101.yinzhengjie.org.cn ~]# yum -y install cloudera-manager-server
3>.将下载的CDH的parcel包移动到"/opt/cloudera/parcel-repo"目录下
[root@cdh101.yinzhengjie.org.cn ~]# ll /opt/cloudera/parcel-repo/ total 2044316 -rw-r--r-- 1 root root 2093332003 Mar 7 15:35 CDH-6.2.1-1.cdh6.2.1.p0.1425774-el7.parcel -rw-r--r-- 1 root root 40 Mar 7 15:34 CDH-6.2.1-1.cdh6.2.1.p0.1425774-el7.parcel.sha1 -rw-r--r-- 1 root root 64 Mar 7 15:34 CDH-6.2.1-1.cdh6.2.1.p0.1425774-el7.parcel.sha256 -rw-r--r-- 1 root root 33875 Mar 7 15:34 manifest.json [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]# cd /opt/cloudera/parcel-repo/ [root@cdh101.yinzhengjie.org.cn /opt/cloudera/parcel-repo]# [root@cdh101.yinzhengjie.org.cn /opt/cloudera/parcel-repo]# mv CDH-6.2.1-1.cdh6.2.1.p0.1425774-el7.parcel.sha1 CDH-6.2.1-1.cdh6.2.1.p0.1425774-el7.parcel.sha #切记，此步骤必须得做，否则安装CDH时无法识别parcel包哟~ [root@cdh101.yinzhengjie.org.cn /opt/cloudera/parcel-repo]# [root@cdh101.yinzhengjie.org.cn /opt/cloudera/parcel-repo]# [root@cdh101.yinzhengjie.org.cn /opt/cloudera/parcel-repo]# ll total 2044316 -rw-r--r-- 1 root root 2093332003 Mar 7 15:35 CDH-6.2.1-1.cdh6.2.1.p0.1425774-el7.parcel -rw-r--r-- 1 root root 40 Mar 7 15:34 CDH-6.2.1-1.cdh6.2.1.p0.1425774-el7.parcel.sha -rw-r--r-- 1 root root 64 Mar 7 15:34 CDH-6.2.1-1.cdh6.2.1.p0.1425774-el7.parcel.sha256 -rw-r--r-- 1 root root 33875 Mar 7 15:34 manifest.json [root@cdh101.yinzhengjie.org.cn /opt/cloudera/parcel-repo]# [root@cdh101.yinzhengjie.org.cn /opt/cloudera/parcel-repo]#
4>.初始化数据库(初始化数据库之前一定要确保用于是可以正常登录的哟)
[root@cdh101.yinzhengjie.org.cn ~]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql cloudera_manager cm yinzhengjie -h cdh101.yinzhengjie.org.cn
5>.启动Cloudera Manager Server并设置为开机自启动
[root@cdh101.yinzhengjie.org.cn ~]# systemctl start cloudera-scm-server.service [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]# systemctl enable cloudera-scm-server.service [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep cloudera-scm-server.service cloudera-scm-server.service enabled [root@cdh101.yinzhengjie.org.cn ~]#
[root@cdh101.yinzhengjie.org.cn ~]# systemctl status cloudera-scm-server.service ● cloudera-scm-server.service - Cloudera CM Server Service Loaded: loaded (/usr/lib/systemd/system/cloudera-scm-server.service; enabled; vendor preset: disabled) Active: active (running) since Sat 2020-03-07 15:50:42 CST; 1min 26s ago Main PID: 6347 (java) CGroup: /system.slice/cloudera-scm-server.service └─6347 /usr/java/jdk1.8.0_181-cloudera/bin/java -cp .:/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:lib/* -server -Dlog4j.configuration=file:/etc/cloudera-scm-server/log4j.p... Mar 07 15:50:42 cdh101.yinzhengjie.org.cn systemd[1]: Started Cloudera CM Server Service. Mar 07 15:50:42 cdh101.yinzhengjie.org.cn cm-server[6347]: JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera Mar 07 15:50:42 cdh101.yinzhengjie.org.cn cm-server[6347]: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0 Mar 07 15:50:44 cdh101.yinzhengjie.org.cn cm-server[6347]: ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'org.apache.logging.log4j.simplelog.StatusLogger....ialization logging. Mar 07 15:50:51 cdh101.yinzhengjie.org.cn cm-server[6347]: 15:50:51.181 [main] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Table 'cloudera_manager.CM_VERSION' doesn't exist Hint: Some lines were ellipsized, use -l to show in full. [root@cdh101.yinzhengjie.org.cn ~]# [root@cdh101.yinzhengjie.org.cn ~]#
[root@cdh101.yinzhengjie.org.cn ~]# tail -10f /var/log/cloudera-scm-server/cloudera-scm-server.log 2020-03-07 15:52:14,178 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Generating entities:2020-03-07T07:52:14.178Z 2020-03-07 15:52:14,202 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Num entities:221 2020-03-07 15:52:14,208 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Generating documents:2020-03-07T07:52:14.202Z 2020-03-07 15:52:14,369 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Num docs:234 2020-03-07 15:52:14,395 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Constructing repo:2020-03-07T07:52:14.395Z 2020-03-07 15:52:16,002 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Finished constructing repo:2020-03-07T07:52:16.002Z 2020-03-07 15:52:16,721 INFO WebServerImpl:org.eclipse.jetty.server.Server: jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_181-b13 2020-03-07 15:52:16,790 INFO WebServerImpl:org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@d99cc3a{HTTP/1.1,[http/1.1]}{0.0.0.0:7180} 2020-03-07 15:52:16,794 INFO WebServerImpl:org.eclipse.jetty.server.Server: Started @94336ms 2020-03-07 15:52:16,794 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
6>.访问Cloudera Manager Server的WebUI
如下图所示，打开浏览器访问Cloudera Manager Server的WebUI: http://cdh101.yinzhengjie.org.cn:7180/cmf/login
7>.基于Cloudera Manager Server的WebUI部署CDH集群
博主推荐阅读: https://www.cnblogs.com/yinzhengjie2020/p/12359867.html
当你的才华还撑不起你的野心的时候，你就应该静下心来学习。当你的能力还驾驭不了你的目标的时候，你就应该沉下心来历练。问问自己，想要怎样的人生。 欢迎加入基础架构自动化运维：598432640，大数据SRE进阶之路：959042252，DevOps进阶之路：526991186
