This page describes the JRuby IRB-based HBase Shell. It replaces the SQL-like HQL, the Shell found in HBase versions 0.1.x and previous. Some discussion of new shell requirements can be found in the Shell Replacement document.
To run the shell, do
$ ${HBASE_HOME}/bin/hbase shellYou'll be presented with a prompt like the following:
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.2.0-dev, r670701, Mon Jun 23 17:26:36 PDT 2008
hbase(main):001:0>
Type 'help' followed by a return to get a listing of commands.
Commands
hbase(main):001:0> help
HBASE SHELL COMMANDS:
alter Alter column family schema; pass table name and a dictionary
specifying new column family schema. Dictionaries are described
below in the GENERAL NOTES section. Dictionary must include name
of column family to alter. For example,
To change or add the 'f1' column family in table 't1' from defaults
to instead keep a maximum of 5 cell VERSIONS, do:
hbase> alter 't1', {NAME => 'f1', VERSIONS => 5}
To delete the 'f1' column family in table 't1', do:
hbase> alter 't1', {NAME => 'f1', METHOD => 'delete'}
You can also change table-scope attributes like MAX_FILESIZE
MEMSTORE_FLUSHSIZE and READONLY.
For example, to change the max size of a family to 128MB, do:
hbase> alter 't1', {METHOD => 'table_att', MAX_FILESIZE => '134217728'}
count Count the number of rows in a table. This operation may take a LONG
time (Run '$HADOOP_HOME/bin/hadoop jar hbase.jar rowcount' to run a
counting mapreduce job). Current count is shown every 1000 rows by
default. Count interval may be optionally specified. Examples:
hbase> count 't1'
hbase> count 't1', 100000
create Create table; pass table name, a dictionary of specifications per
column family, and optionally a dictionary of table configuration.
Dictionaries are described below in the GENERAL NOTES section.
Examples:
hbase> create 't1', {NAME => 'f1', VERSIONS => 5}
hbase> create 't1', {NAME => 'f1'}, {NAME => 'f2'}, {NAME => 'f3'}
hbase> # The above in shorthand would be the following:
hbase> create 't1', 'f1', 'f2', 'f3'
hbase> create 't1', {NAME => 'f1', VERSIONS => 1, TTL => 2592000, \
BLOCKCACHE => true}
describe Describe the named table: e.g. "hbase> describe 't1'"
delete Put a delete cell value at specified table/row/column and optionally
timestamp coordinates. Deletes must match the deleted cell's
coordinates exactly. When scanning, a delete cell suppresses older
versions. Takes arguments like the 'put' command described below
deleteall Delete all cells in a given row; pass a table name, row, and optionally
a column and timestamp
disable Disable the named table: e.g. "hbase> disable 't1'"
drop Drop the named table. Table must first be disabled. If table has
more than one region, run a major compaction on .META.:
hbase> major_compact ".META."
enable Enable the named table
exists Does the named table exist? e.g. "hbase> exists 't1'"
exit Type "hbase> exit" to leave the HBase Shell
get Get row or cell contents; pass table name, row, and optionally
a dictionary of column(s), timestamp and versions. Examples:
hbase> get 't1', 'r1'
hbase> get 't1', 'r1', {COLUMN => 'c1'}
hbase> get 't1', 'r1', {COLUMN => ['c1', 'c2', 'c3']}
hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1}
hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1, \
VERSIONS => 4}
list List all tables in hbase
put Put a cell 'value' at specified table/row/column and optionally
timestamp coordinates. To put a cell value into table 't1' at
row 'r1' under column 'c1' marked with the time 'ts1', do:
hbase> put 't1', 'r1', 'c1', 'value', ts1
tools Listing of hbase surgery tools
scan Scan a table; pass table name and optionally a dictionary of scanner
specifications. Scanner specifications may include one or more of
the following: LIMIT, STARTROW, STOPROW, TIMESTAMP, or COLUMNS. If
no columns are specified, all columns will be scanned. To scan all
members of a column family, leave the qualifier empty as in
'col_family:'. Examples:
hbase> scan '.META.'
hbase> scan '.META.', {COLUMNS => 'info:regioninfo'}
hbase> scan 't1', {COLUMNS => ['c1', 'c2'], LIMIT => 10, \
STARTROW => 'xyz'}
For experts, there is an additional option -- CACHE_BLOCKS -- which
switches block caching for the scanner on (true) or off (false). By
default it is enabled. Examples:
hbase> scan 't1', {COLUMNS => ['c1', 'c2'], CACHE_BLOCKS => false}
status Show cluster status. Can be 'summary', 'simple', or 'detailed'. The
default is 'summary'. Examples:
hbase> status
hbase> status 'simple'
hbase> status 'summary'
hbase> status 'detailed'
shutdown Shut down the cluster.
truncate Disables, drops and recreates the specified table.
version Output this HBase version
GENERAL NOTES:
Quote all names in the hbase shell such as table and column names. Don't
forget commas delimit command parameters. Type <RETURN> after entering a
command to run it. Dictionaries of configuration used in the creation and
alteration of tables are ruby Hashes. They look like this:
{'key1' => 'value1', 'key2' => 'value2', ...}
They are opened and closed with curley-braces. Key/values are delimited by
the '=>' character combination. Usually keys are predefined constants such as
NAME, VERSIONS, COMPRESSION, etc. Constants do not need to be quoted. Type
'Object.constants' to see a (messy) list of all constants in the environment.
In case you are using binary keys or values and need to enter them into the
shell then use double-quotes to make use of hexadecimal or octal notations,
for example:
hbase> get 't1', "key\x03\x3f\xcd"
hbase> get 't1', "key\003\023\011"
hbase> put 't1', "test\xef\xff", 'f1:', "\x01\x33\x40"
Using the double-quote notation you can directly use the values output by the
shell for example during a "scan" call.
This HBase shell is the JRuby IRB with the above HBase-specific commands added.
For more on the HBase Shell, see http://wiki.apache.org/hadoop/Hbase/Shell
General Notes
GENERAL NOTES: Quote all names in the hbase shell such as table and column names. Don't forget commas delimit command parameters. Type <RETURN> after entering a command to run it. Dictionaries of configuration used in the creation and alteration of tables are ruby Hashes. They look like this:
{'key1' => 'value1', 'key2' => 'value2', ...}
They are opened and closed with curley-braces. Key/values are delimited by the '=>' character combination. Usually keys are predefined constants such as NAME, VERSIONS, COMPRESSION, etc. Constants do not need to be quoted. Type 'Object.constants' to see a (messy) list of all constants in the environment.
This HBase shell is the JRuby IRB with the above HBase-specific commands added.
Anatomy Lesson
The HBase Shell is a ruby script, ${HBASE_HOME}/bin/hirb.rb, that is passed to the JRuby class org.jruby.Main (See ${HBASE_HOME}/bin/shell and search for org.jruby.Main to see how its done).
The hirb.rb' script defines a set of hbase methods (get, set, scan, etc.), sets some environment variables, imports a few hbase-particular modules -- a results Formatter and a ruby wrapper around the HBaseAdmin and HTable java classes that can be found at ${HBASE_HOME}/bin/HBase.rb -- and then starts up a subclass of IRB, named HIRB (IRB is the name of the interactive Ruby interpreter; for an untainted irb experience, type 'irb' on a system that has ruby installed). The subclassing is done to alter slightly some of the usual IRB behaviors. For instance, every command invocation returns a result and the result is printed to the terminal even if the result is nil. The subclass intercepts nil emissions and just suppresses them.
Anything you can do in irb, or at least in its JRuby equivalent, jirb, you should be able to do in HBase Shell.
Scripting
You can pass scripts to the HBase Shell by doing the following:
durruti:~ stack$ ${HBASE_HOME}/bin/hbase shell PATH_TO_SCRIPTYour script can lean on the methods provided by the HBase Shell.
For more control, you may prefer manipulating HTable and HBaseAdmin methods directly. For example, if you would do your own formatting or you are storing structures that are mangled when manipulated in HBase Shell. In this case, you might pass your scripts directly to JRuby by doing:
durruti:~ stack$ ${HBASE_HOME}/bin/hbase org.jruby.Main PATH_TO_SCRIPTFor examples writing straight ruby manhandling hbase client instances, see the scripts on this page: Hbase/JRuby (Ignore the sections that have you fetching the jruby jar and the prefacing your scripts with the '#!' bang magic).
Useful Tricks
irbrc
Create an .irbrc file for yourself in your home directory. Add HBase Shell customizations. A useful one is command history:
durruti:~ stack$ more .irbrc
require 'irb/ext/save-history'
IRB.conf[:SAVE_HISTORY] = 100
IRB.conf[:HISTORY_FILE] = "#{ENV['HOME']}/.irb-save-history"
Log date to timestamp
To convert the date '08/08/16 20:56:29' from an hbase log into a timestamp, do:
hbase(main):021:0> import java.text.SimpleDateFormat
hbase(main):022:0> import java.text.ParsePosition
hbase(main):023:0> SimpleDateFormat.new("yy/MM/dd HH:mm:ss").parse("08/08/16 20:56:29", ParsePosition.new(0)).getTime() => 1218920189000
To go the other direction, do:
hbase(main):021:0> import java.util.DateTo output in a format that is exactly like hbase log format is a pain messing with SimpleDateFormat.
hbase(main):022:0> Date.new(1218920189000).toString() => "Sat Aug 16 20:56:29 UTC 2008"
