hadoop完全分布式安装部署-笔记

规划：

[hadoop@db01 ~]$ cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.100.231         db01        #(namenode，datanode,JobHistoryServer,nodemanager)
192.168.100.232         db02        #(resourcemanager,datanode,nodemanager)
192.168.100.233         db03        #(SecondaryNameNode,datanode,nodemanager)
192.168.100.234         db04        #(nodemanager,datanode)
192.168.100.235         db05        #(nodemanager,datanode)

配置文件：

[hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->

<configuration>
    <property>
            <name>fs.defaultFS</name>
            <value>hdfs://db01:9000</value>
    </property>

        <property>
                <name>hadoop.tmp.dir</name>
                <value>/usr/local/hadoop-2.5.0/data/tmp</value>
        </property>

        <property>
                <name>fs.trash.interval</name>
                <value>7000</value>
        </property>
</configuration>

[hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

        <property>
                <name>dfs.namenode.secondary.http-address</name>
                <value>db03:50090</value>
        </property>

</configuration>

[hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/yarn-site.xml
<?xml version="1.0"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

    <property>
            <name>yarn.nodemanager.aux-services</name>
                <value>mapreduce_shuffle</value>
    </property>

        <property>
                <name>yarn.resourcemanager.hostname</name>
                <value>db02</value>
        </property>

    <property>
                <name>yarn.log-aggregation-enable</name>
                <value>true</value>
        </property>

        <property>
                <name>yarn.log-aggregation.retain-seconds</name>
                <value>600000</value>
        </property>
</configuration>

[hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

    <property>
            <name>mapreduce.framework.name</name>
                <value>yarn</value>
    </property>

        <property>
                <name>mapreduce.jobhistory.address</name>
                <value>db01:10020</value>
        </property>

        <property>
                <name>mapreduce.jobhistory.webapp.address</name>
                <value>db01:19888</value>
        </property>

</configuration>

[hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/slaves
db01
db02
db03
db04
db05

并且在(etc/hadoop/hadoop-env.sh etc/hadoop/mapred-env.sh etc/hadoop/yarn-env.sh)文件中配置相应的JAVA_HOME信息。