Hadoop基础——第一弹：Hadoop介绍

一、基础

1.了解Java、Linux操作系统相关知识

2.如需精进，应为水平要达到一定标准，能够阅读国外相关技术网站，eg：http://hadoop.apache.org/

 

二、什么是Hadoop

照搬官网并略作翻译：

The Apache™ Hadoop® project develops open-source software for reliable, scalable, distributed computing.

The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.

 

The project includes these modules:

• Hadoop Common: The common utilities that support the other Hadoop modules.
• Hadoop Distributed File System (HDFS™): A distributed file system that provides high-throughput access to application data.
• Hadoop YARN: A framework for job scheduling and cluster resource management.
• Hadoop MapReduce: A YARN-based system for parallel processing of large data sets.

 

Hadoop系统是一个稳定、分布式计算相关开源软件系统。

Hadoop框架通过使用集群中的简单程序模型支持大数据的分布式计算，它可以从单台计算机拓展到上千台提供本地计算和存储功能的服务器组成的集群。与以往系统使用硬件保证高可用方式不同，Hadoop在应用层可以检测、处理异常，因此通过集群顶层的服务保证高可用性。

Hadoop主要分为以下模块：

（1）Hadoop Common：支持其他模块的公共工具

（2）HDFS：分布式文件系统，用于提供系统数据存储服务（相当于oracle的存储模块）

（3）Hadoop YARN：工作与资源调度模块，相当于基于HDFS的操作系统

（4）Hadoop MapReduce：基于YARN系统的分布式计算方法

 

三、系统划分