pinpoint:查看hbase表和修改数据过期时间
先做个记录,监控数据量过大时可以设置表的数据过期时间来清理数据。
1. 查找本地数据表大小
[root@ZWZF-CWY-LZY-12 ~]# cd /home/pinpoint/hbase/data/default/ [root@ZWZF-CWY-LZY-12 default]# du -sh * 16K AgentEvent 64K AgentInfo 16K AgentLifeCycle 72K AgentStat 632K AgentStatV2 40K ApiMetaData 16K ApplicationIndex 72K ApplicationMapStatisticsCallee_Ver2 72K ApplicationMapStatisticsCaller_Ver2 80K ApplicationMapStatisticsSelf_Ver2 72K ApplicationTraceIndex 20K HostApplicationMap_Ver2 72K SqlMetaData_Ver2 40K StringMetaData 264K Traces 1.1M TraceV2
2. 设置表的数据过期时间
2.1 查看hbase表:list
[root@monitor default]# cd /home/pinpoint/hbase-1.7.1/bin/ [root@monitor bin]# ./hbase shell 2022-10-19 15:03:48,097 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable HBase Shell Use "help" to get list of supported commands. Use "exit" to quit this interactive shell. Version 1.7.1, r2d9273667e418e7023f9104a830cdcb8233b6f25, Fri Jul 16 00:20:26 PDT 2021 hbase(main):001:0>list TABLE AgentEvent AgentInfo AgentLifeCycle AgentStatV2 ApiMetaData ApplicationIndex ApplicationMapStatisticsCallee_Ver2 ApplicationMapStatisticsCaller_Ver2 ApplicationMapStatisticsSelf_Ver2 ApplicationStatAggre ApplicationTraceIndex HostApplicationMap_Ver2 SqlMetaData_Ver2 StringMetaData TraceV2 15 row(s) in 0.1740 seconds => ["AgentEvent", "AgentInfo", "AgentLifeCycle", "AgentStatV2", "ApiMetaData", "ApplicationIndex", "ApplicationMapStatisticsCallee_Ver2", "ApplicationMapStatisticsCaller_Ver2", "ApplicationMapStatisticsSelf_Ver2", "ApplicationStatAggre", "ApplicationTraceIndex", "HostApplicationMap_Ver2", "SqlMetaData_Ver2", "StringMetaData", "TraceV2"]
2.2 查看表描述:desc 'table_name'
hbase(main):002:0> desc 'AgentInfo' Table AgentInfo is ENABLED AgentInfo COLUMN FAMILIES DESCRIPTION {NAME => 'Info', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'PREFIX', TTL => '31536000 SECONDS (365 DAYS)', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => ' 0'} 1 row(s) in 0.1760 seconds
2.3 修改表信息
将AgentInfo表的过期时间(TTL)修改为180天(TTL单位:秒)。
2.3.1 停用表:disable 'table_name'
hbase(main):003:0> disable 'AgentInfo' 0 row(s) in 2.3130 seconds
2.3.2 修改表信息:alter 'table_name' , {NAME=>'xxx',TTL=>'xxx'}
hbase(main):004:0> alter 'AgentInfo', {NAME=>'Info',TTL=>'15552000'} Updating all regions with the new schema... 1/1 regions updated. Done. 0 row(s) in 1.9670 seconds
2.3.3 启用表:enable 'table_name'
hbase(main):005:0> enable 'AgentInfo' 0 row(s) in 1.3030 seconds
2.3.4 查看表信息(确认修改结果)
hbase(main):006:0> desc 'AgentInfo' Table AgentInfo is DISABLED AgentInfo COLUMN FAMILIES DESCRIPTION {NAME => 'Info', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'PREFIX', TTL => '15552000 SECONDS (180 DAYS)', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => ' 0'} 1 row(s) in 0.0310 seconds
3. 修改表过期时间后,清除过期数据:major_compact 'table_name'
定期运行一次Major Compact(通过crontab 任务),用来删除早于给定TTL值的所有数据。
注意:如果只设置TTL而不定期运行 major_compact,只是标记删除,并不会真正从磁盘上删除。
hbase(main):007:0> major_compact 'AgentInfo' 0 row(s) in 0.4290 seconds
本文来自博客园,作者:查拉图斯特拉面条,转载请注明原文链接:https://www.cnblogs.com/n00dle/p/16806643.html