12.27日报

完成大型数据库实验三熟悉常用的HBASE操作，以下为实验内容：

实验3

熟悉常用的HBase操作

 

 

1.实验目的

         （1）理解HBase在Hadoop体系结构中的角色；

         （2）熟练使用HBase操作常用的Shell命令；

         （3）熟悉HBase操作常用的Java API。

2.实验平台

（1）操作系统：Linux（建议Ubuntu16.04或Ubuntu18.04）；

（2）Hadoop版本：3.1.3；

（3）HBase版本：2.2.2；

（4）JDK版本：1.8；

（5）Java IDE：Eclipse。

3. 实验步骤   

（一）编程实现以下指定功能，并用Hadoop提供的HBase Shell命令完成相同任务：

（1）     列出HBase所有的表的相关信息，例如表名；

（2）     在终端打印出指定的表的所有记录数据；

（3）     向已经创建好的表添加和删除指定的列族或列；

（4）     清空指定的表的所有记录数据；

（5）     统计表的行数。

 

（二）HBase数据库操作

1. 现有以下关系型数据库中的表和数据（见表14-3到表14-5），要求将其转换为适合于HBase存储的表并插入数据：

表14-3 学生表（Student）

学号（S_No）	姓名（S_Name）	性别（S_Sex）	年龄（S_Age）
2015001	Zhangsan	male	23
2015002	Mary	female	22
2015003	Lisi	male	24

 

表14-4 课程表（Course）

课程号（C_No）	课程名（C_Name）	学分（C_Credit）
123001	Math	2.0
123002	Computer Science	5.0
123003	English	3.0

 

表14-5 选课表（SC）

学号（SC_Sno）	课程号（SC_Cno）	成绩（SC_Score）
2015001	123001	86
2015001	123003	69
2015002	123002	77
2015002	123003	99
2015003	123001	98
2015003	123002	95

 

2. 请编程实现以下功能：

（1）createTable(String tableName, String[] fields)

        创建表，参数tableName为表的名称，字符串数组fields为存储记录各个字段名称的数组。要求当HBase已经存在名为tableName的表的时候，先删除原有的表，然后再创建新的表。

        （2）addRecord(String tableName, String row, String[] fields, String[] values)

        向表tableName、行row（用S_Name表示）和字符串数组fields指定的单元格中添加对应的数据values。其中，fields中每个元素如果对应的列族下还有相应的列限定符的话，用“columnFamily:column”表示。例如，同时向“Math”、“Computer Science”、“English”三列添加成绩时，字符串数组fields为{“Score:Math”, ”Score:Computer Science”, ”Score:English”}，数组values存储这三门课的成绩。

        （3）scanColumn(String tableName, String column)

        浏览表tableName某一列的数据，如果某一行记录中该列数据不存在，则返回null。要求当参数column为某一列族名称时，如果底下有若干个列限定符，则要列出每个列限定符代表的列的数据；当参数column为某一列具体名称（例如“Score:Math”）时，只需要列出该列的数据。

        （4）modifyData(String tableName, String row, String column)

        修改表tableName，行row（可以用学生姓名S_Name表示），列column指定的单元格的数据。

（5）deleteRow(String tableName, String row)

        删除表tableName中row指定的行的记录。

 

4.实验报告

题目：	熟悉常用的HBase操作	姓名 李健龙		日期 2024/12/10
实验环境：Ubuntu 18.04.6 LTS Hadoop 3.1.3
实验内容与完成情况： 一、    编程实现以下指定功能，并用Hadoop提供的HBase Shell命令完成相同任务 hadoop@hadoop:~/hbase$ hbase shell   HBase Shell Use "help" to get list of supported commands. Use "exit" to quit this interactive shell. For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell Version 2.2.5, rf76a601273e834267b55c0cda12474590283fd4c, 2020年 05月 21日 星期四 18:34:40 CST Took 0.0143 seconds                                                             hbase(main):001:0> hbase(main):002:0* list TABLE                                                                           student                                                                         1 row(s) Took 0.6692 seconds                                                             => ["student"] hbase(main):003:0> scan '表名' ROW                   COLUMN+CELL                                                 ERROR: Unknown table 表名!   For usage try 'help "scan"'   Took 0.1851 seconds                                                             hbase(main):004:0>  scan 'student' ROW                   COLUMN+CELL                                                 2015001              column=info:age, timestamp=1731920741249, value=23         2015001              column=info:name, timestamp=1731920741065, value=Zhangsan  2015001              column=info:sex, timestamp=1731920741183, value=male      1 row(s) Took 0.1098 seconds                                                             hbase(main):005:0> alter 'student', NAME => 'new_family' Updating all regions with the new schema... 1/1 regions updated. Done. Took 2.8537 seconds                                                             hbase(main):006:0> alter 'student', {NAME => 'new_family', METHOD => 'delete'} Updating all regions with the new schema... 1/1 regions updated. Done. Took 2.2191 seconds                                                             hbase(main):007:0> truncate 'student' Truncating 'student' table (it may take a while): Disabling table... Truncating table... Took 2.1639 seconds        二、HBase数据库操作 (1) hadoop@hadoop:~$ cd hbase/ hadoop@hadoop:~/hbase$ bin/start-hbase.sh localhost: running zookeeper, logging to /home/hadoop/hbase/bin/../logs/hbase-hadoop-zookeeper-hadoop.out running master, logging to /home/hadoop/hbase/bin/../logs/hbase-hadoop-master-hadoop.out : running regionserver, logging to /home/hadoop/hbase/bin/../logs/hbase-hadoop-regionserver-hadoop.out hadoop@hadoop:~/hbase$ hbase shell HBase Shell Use "help" to get list of supported commands. Use "exit" to quit this interactive shell. For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell Version 2.2.5, rf76a601273e834267b55c0cda12474590283fd4c, 2020年 05月 21日 星期四 18:34:40 CST Took 0.0163 seconds                                                             hbase(main):001:0> create 'student', 'info' Created table student Took 1.7918 seconds                                                              => Hbase::Table - student hbase(main):002:0> hbase(main):003:0* put 'student', '2015001', 'info:name', 'Zhangsan' Took 0.3184 seconds                                                             hbase(main):004:0> put 'student', '2015001', 'info:sex', 'male' Took 0.0194 seconds                                                             hbase(main):005:0> put 'student', '2015001', 'info:age', '23' Took 0.0080 seconds                                                             hbase(main):006:0> describe 'student' Table student is ENABLED                                                        student                                                                         COLUMN FAMILIES DESCRIPTION                                                     {NAME => 'info', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_ BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'fals e', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLIC ATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE => 'false', IN_ME MORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false', PREFETCH_BLOCKS_ON_OPEN => 'f alse', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}         1 row(s)   QUOTAS                                                                          0 row(s) Took 0.2807 seconds                                                             hbase(main):007:0> scan 'student' ROW                   COLUMN+CELL                                                2015001              column=info:age, timestamp=1731920741249, value=23         2015001              column=info:name, timestamp=1731920741065, value=Zhangsan  2015001              column=info:sex, timestamp=1731920741183, value=male      1 row(s) Took 0.0580 seconds                                                             hbase(main):008:0> exit hadoop@hadoop:~/hbase$             2.编程 pom.xml <dependency>     <groupId>org.apache.hbase</groupId>     <artifactId>hbase-client</artifactId>     <version>2.2.5</version> </dependency> <dependency>     <groupId>org.apache.hbase</groupId>     <artifactId>hbase-common</artifactId>     <version>2.2.5</version> </dependency>   Java import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.TableName; import org.apache.hadoop.hbase.client.*; import org.apache.hadoop.hbase.util.Bytes;   import java.io.IOException;   public class HBaseOperations {     private static Connection connection;       static {         try {             Configuration config = HBaseConfiguration.create();             config.set("hbase.zookeeper.quorum", "localhost");             connection = ConnectionFactory.createConnection(config);         } catch (IOException e) {             e.printStackTrace();         }     }       // 1. 创建表     public static void createTable(String tableName, String[] fields) throws IOException {         Admin admin = connection.getAdmin();         TableName tn = TableName.valueOf(tableName);         if (admin.tableExists(tn)) {             admin.disableTable(tn);             admin.deleteTable(tn);         }           TableDescriptorBuilder builder = TableDescriptorBuilder.newBuilder(tn);         for (String field : fields) {             builder.setColumnFamily(ColumnFamilyDescriptorBuilder.of(field));         }         admin.createTable(builder.build());         admin.close();     }       // 2. 添加记录     public static void addRecord(String tableName, String row, String[] fields, String[] values) throws IOException {         Table table = connection.getTable(TableName.valueOf(tableName));         Put put = new Put(Bytes.toBytes(row));         for (int i = 0; i < fields.length; i++) {             String[] parts = fields[i].split(":");             String columnFamily = parts[0];             String column = parts.length > 1 ? parts[1] : null;             if (column != null) {                 put.addColumn(Bytes.toBytes(columnFamily), Bytes.toBytes(column), Bytes.toBytes(values[i]));             } else {                 put.addColumn(Bytes.toBytes(columnFamily), null, Bytes.toBytes(values[i]));             }         }         table.put(put);         table.close();     }       // 3. 浏览列     public static void scanColumn(String tableName, String column) throws IOException {         Table table = connection.getTable(TableName.valueOf(tableName));         Scan scan = new Scan();         if (column.contains(":")) {             String[] parts = column.split(":");             scan.addColumn(Bytes.toBytes(parts[0]), Bytes.toBytes(parts[1]));         } else {             scan.addFamily(Bytes.toBytes(column));         }           ResultScanner scanner = table.getScanner(scan);         for (Result result : scanner) {             System.out.println(result);         }         scanner.close();         table.close();     }       // 4. 修改数据     public static void modifyData(String tableName, String row, String column, String value) throws IOException {         Table table = connection.getTable(TableName.valueOf(tableName));         String[] parts = column.split(":");         Put put = new Put(Bytes.toBytes(row));         put.addColumn(Bytes.toBytes(parts[0]), Bytes.toBytes(parts[1]), Bytes.toBytes(value));         table.put(put);         table.close();     }       // 5. 删除行     public static void deleteRow(String tableName, String row) throws IOException {         Table table = connection.getTable(TableName.valueOf(tableName));         Delete delete = new Delete(Bytes.toBytes(row));         table.delete(delete);         table.close();     }       public static void main(String[] args) throws IOException {         // 示例：创建表         String[] fields = {"info", "Score"};         createTable("Student", fields);           // 示例：添加记录         String[] recordFields = {"info:name", "info:sex", "info:age"};         String[] recordValues = {"Zhangsan", "male", "23"};         addRecord("Student", "2015001", recordFields, recordValues);           // 示例：浏览列         scanColumn("Student", "info");           // 示例：修改数据         modifyData("Student", "2015001", "info:age", "24");           // 示例：删除行         deleteRow("Student", "2015001");     } }
出现的问题：HBase服务未运行   创建表或执行其他操作时，遇到权限不足的错误
解决方案（列出遇到的问题和解决办法，列出没有解决的问题）：启动HBase服务start-hbase.sh    sudo -u hbase <hbase-command>33.。