Flume

http://www.iteblog.com/archives/1043

http://www.iteblog.com/archives/908

http://www.iteblog.com/archives/1034

In flume-ng's HBase sink, HbaseEventSerializer implementation is responsible for generating row keys. The default implementation org.apache.flume.sink.hbase.SimpleHbaseEventSerializer already supports generating timestamp row keys in the format prefix + current timestamp, to use it just modify your flume configuration accordinly.

hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
hbase-agent.sinks.sink1.channel = ch1
hbase-agent.sinks.sink1.table = demo
hbase-agent.sinks.sink1.columnFamily = cf
hbase-agent.sinks.sink1.serializer = org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
hbase-agent.sinks.sink1.serializer.payloadColumn = col1
hbase-agent.sinks.sink1.serializer.keyType = timestamp

If the provided timestamp based key generation method is not what you are after then you'll need to provide a custom HbaseEventSerializer implementation to flume, which will require you to:

  1. Create your own row key generator class (the default one is org.apache.flume.sink.hbase.SimpleRowKeyGenerator)
  2. Create your own implementation of the HbaseEventSerializer interface (default implementation is org.apache.flume.sink.hbase.SimpleHbaseEventSerializer) which will use the custom row key generator you created in first step
  3. Modify your flume hbase sink configuration to use the custom HbaseEventSerializer implementation.

http://www.kankanews.com/ICkengine/archives/67816.shtml

http://search-hadoop.com/?q=prefix+salt+key+hotspot&fc_project=HBase

posted @ 2014-08-18 20:00  规格严格-功夫到家  阅读(323)  评论(0编辑  收藏  举报