开源的大数据处理工具

1 查询引擎

1.1 phoenix

1.2 stinger

1.3 presto

1.4 shark

1.5 pig

1.6 cloudera impala

1.7 apache drill

1.8 apache tajo

1.9 hive

2 流式计算

2.1 facebook puma

2.2 twitter rainbird

2.3 yahoo s4

2.4 twitter storm

3 迭代计算

3.1 apache hama

3.2 apache Giraph

3.3 Haloop

3.4 twister

4 离线计算

4.1 hadoop mapreduce

4.2 Berkeley spark

4.3 datatorrent

5 键值存储

5.1 levelDB

5.2 RocksDB

5.3 HyperDex

5.4 Tokyo Cabinet

5.5 voldemort

5.6 amazon dynamo

5.7 tair

5.8 apache accumulo

5.9 redis

6 表格存储

6.1 oceanbase

6.2 amazon simpleDB

6.3 vertica

6.4 cassandra

6.5 hypertable

6.6 foundationDB

6.7 HBase

7 文件存储

7.1 couchDB

7.2 mongoDB

7.3 tachyon

7.4 KFS

7.5 HDFS

8 资源管理

8.1 twitter mesos

8.2 hadoop yarn

9 日志收集系统

9.1 facebook scribe

9.2 cloudera flume

9.3 logstash

9.4 kibana

10 消息系统

10.1 stormMQ

10.2 zeroMQ

10.3 RabbitMQ

10.4 Apache activeMQ

10.5 Jafka

10.6 apache jafka

11 分布式协调器

11.1 zookeeper

12 RPC

12.1 apache avro

12.2 facebook thrift

13 集群管理

13.1 nagios

13.2 ganglia

13.3 apache ambari

14 基础设施

14.1 SSTable

14.2 RecordIO

14.3 flat buffers

14.4 protocol buffers

14.5 consistent hashing

14.6 netty

14.7 bloom filter

15 搜索引擎

15.1 nutch

15.2 lucene

15.3 solrcloud

15.4 solr

15.5 elasticsearch

15.6 sphinx

15.7 SenseiDB

16 数据挖掘

16.1 mahout

17 iaas

17.1 openstack

17.2 docker

17.3 Kubernetes

17.4 imctfy