spring hadoop 访问hbase入门
1、 环境准备:
Maven
Eclipse
Java
Spring 版本 3..2.9
2、 Maven pom.xml配置
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | <!-- Spring hadoop --> < dependency > < groupId >org.apache.hbase</ groupId > < artifactId >hbase-client</ artifactId > < version >0.96.1.1-hadoop2</ version > </ dependency > < dependency > < groupId >org.springframework.data</ groupId > < artifactId >spring-data-jpa</ artifactId > < version >1.6.0.RELEASE</ version > </ dependency > < dependency > < groupId >org.springframework.data</ groupId > < artifactId >spring-data-hadoop</ artifactId > < version >2.0.2.RELEASE</ version > </ dependency > |
3、 Spring和hadoop、hbase相关配置文件
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | <? xml version="1.0" encoding="UTF-8"?> < beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jdbc="http://www.springframework.org/schema/jdbc" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:util="http://www.springframework.org/schema/util" xmlns:aop="http://www.springframework.org/schema/aop" xmlns:context="http://www.springframework.org/schema/context" xmlns:mongo="http://www.springframework.org/schema/data/mongo" xmlns:hdp="http://www.springframework.org/schema/hadoop" xmlns:cache="http://www.springframework.org/schema/cache" xmlns:c="http://www.springframework.org/schema/c" xmlns:p="http://www.springframework.org/schema/p" xsi:schemaLocation=" http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd http://www.springframework.org/schema/data/mongo http://www.springframework.org/schema/data/mongo/spring-mongo.xsd http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd http://www.springframework.org/schema/cache http://www.springframework.org/schema/cache/spring-cache.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd"> |
其中标红的是spring hadoop xml命名空间配置。
Hadoop hbase相关配置文件如下:
<!-- 默认properties-->
<hdp:configuration>fs.default.name=hdfs://192.98.8.224:8010</hdp:configuration>
<hdp:hbase-configuration delete-connection="${delete-connection}" zk-quorum="${hbase.zookeeper.quorum}" zk- port="${hbase.zookeeper.property.clientPort}"/>
对应的properties如下:
hbase.zookeeper.property.clientPort=2181
hbase.zookeeper.quorum=192.98.8.224
hbase.master=192.98.8.224:600000
fs.default.name=hdfs://192.98.8.224:8010
delete-connection=true
#hive jdbc url
hive.url=jdbc:hive://192.98.8.224:10000/default
spring hbasetemplate配置如下:
<bean id="hbaseTemplate" class="org.springframework.data.hadoop.hbase.HbaseTemplate"> <property name="configuration" ref="hbaseConfiguration" />
</bean>
Hbasetemplate使用代码示例:
1 2 3 4 5 6 7 8 9 10 11 | Tile t = hbaseTemplate.get( "GW_TILES" , "0_1_1" , new RowMapper<Tile>() { @Override public Tile mapRow(Result result, int rowNum) throws Exception { // TODO Auto-generated method stub Tile t = new Tile(); t.setData(result.getValue( "T" .getBytes(), "key" .getBytes())); return t; } }); |
Hbasetemplate 常用方法简介:
hbaseTemplate.get("GW_TILES", "0_1_1", new RowMapper 常用于查询,使用示例如下所示:
1 2 3 4 5 6 7 8 9 10 11 | Tile t = hbaseTemplate.get( "GW_TILES" , "0_1_1" , new RowMapper<Tile>() { @Override public Tile mapRow(Result result, int rowNum) throws Exception { // TODO Auto-generated method stub Tile t = new Tile(); t.setData(result.getValue( "T" .getBytes(), "key" .getBytes())); return t; } }); |
hbaseTemplate.execute(dataIdentifier, new TableCallback 常用于更新操作,使用示例如下所示:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | return hbaseTemplate.execute(dataIdentifier, new TableCallback<Boolean>() { @Override public Boolean doInTable(HTableInterface table) throws Throwable { // TODO Auto-generated method stub boolean flag = false ; try { Delete delete = new Delete(key.getBytes()); table.delete(delete); flag = true ; } catch (Exception e){ e.printStackTrace(); } return flag; } }); |
备注:spring hbasetemplate针对hbase接口做了强大的封装,普通功能可以使用它强大的接口,同时复杂的功能,还可以使用hbase原生的接口,如:HTableInterface、Result等。其类方法如下图:
同时hbasetemplate封装了hbase连接池等,它的创建和释放通过配置来自动管理。
文章转载请注明出处:http://www.cnblogs.com/likehua/p/4016257.html
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】凌霞软件回馈社区,博客园 & 1Panel & Halo 联合会员上线
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步