centos7 安装hadoop2.7.6(分布式)
本文只做简单介绍,具体步骤操作请参考centos6.5 安装hadoop1.2.1亲测版
本篇只简单介绍安装步骤
1.安装目录 /usr/local/hadoop (HADOOP_HOME)
2,创建group hadoop
groupadd hadoop
3.创建用户hadoop
useradd -d /home/hadoop -g hadoop hadoop chown -R hadoop:hadoop /usr/local/hadoop
4.每台机器安装jdk(安装在相同的目录)
5.master 到 slave ssh 免密码登录
6.关闭所有机器防火墙。selinux
1)centos 7 以下 防火墙 查看防火墙状态 service iptables status 临时关闭防火墙 service iptables stop 永久关闭防火墙 chkconfig iptables off selinux setenforce 0 #临时关闭 永久关闭需要修改如下文件 vim /etc/selinux/config 将SELINUX=enforcing改为SELINUX=disabled 2)centos 7 firewall-cmd --reload #重启firewall systemctl stop firewalld.service #停止firewall systemctl disable firewalld.service #禁止firewall开机启动 firewall-cmd --state #查看默认防火墙状态(关闭后显示notrunning,开启后显示running) selinux 同上
7.修改hadoop配置文件
core-site.xml
<configuration> <property> <name>fs.defaultFS</name> <value>hdfs://ns1:9000</value> </property> <property> <name>io.file.buffer.size</name> <value>4096</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/home/hadoop/tmp</value> <description>Abase for other temporary directories.</description> </property> </configuration>
hadoop-env.sh
添加
export JAVA_HOME=/usr/local/jdk
hdfs-site.xml
<configuration> <property> <name>dfs.namenode.secondary.http-address</name> <value>ns1:9001</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>file:/home/hadoop/name</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>file:/home/hadoop/data</value> </property> <property> <name>dfs.replication</name> <value>2</value> </property> <property> <name>dfs.webhdfs.enabled</name> <value>true</value> </property> </configuration>
mapred-env.sh
添加
export JAVA_HOME=/usr/local/jdk
mapred-site.xml
<configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <property> <name>mapreduce.jobhistory.address</name> <value>ns1:10020</value> </property> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>ns1:19888</value> </property> </configuration>
slaves
ns1
dn1
yarn-env.sh
添加
export JAVA_HOME=/usr/local/jdk
yarn-site.xml
<configuration> <!-- Site specific YARN configuration properties --> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property> <property> <name>yarn.resourcemanager.address</name> <value>ns1:8032</value> </property> <property> <name>yarn.resourcemanager.scheduler.address</name> <value>ns1:8030</value> </property> <property> <name>yarn.resourcemanager.resource-tracker.address</name> <value>ns1:8031</value> </property> <property> <name>yarn.resourcemanager.admin.address</name> <value>ns1:8033</value> </property> <property> <name>yarn.resourcemanager.webapp.address</name> <value>ns1:8088</value> </property> </configuration>
8.修改环境变量,添加HADOOP_HOME 和JAVA_HOME
9.启动hadoop
1)在master上format namenode hadoop namenode -format 2)启动hadoop sh /user/local/hadoop/sbin/start-all.sh 3)启动jobhistory sh /user/local/hadoop/sbin/mr-jobhistory-daemon.sh start historyserver
10.访问web界面查看
欢迎转载,不必注明出处