Storing RDF data into HBase?

login | faq

• Questions
• Tags
• Users
• Badges
• Unanswered

• Ask Question

Storing RDF data into HBase?

3

7

What would be the best way for storing and quering RDF data (triples or quads) using HBase?

Update:

I've just found this:

• http://cs.utdallas.edu/semanticweb/HBase-Extension/

Unfortunately, the .tar.gz has been removed from the utdallas.edu website.

rdf triplestore hbase query storage

flag

edited Dec 14 at 16:37

asked Apr 12 at 20:19 castagna 944●2●14

1

Storing is easy, isn't the complex part the querying especially joins? – Ian Davis Apr 12 at 22:08

Changed the question to: "for storing and queryng" :-) But, storing would be the first step and it would have the advantage of being able to easily use MapReduce over it. For querying I would look at Pig and how to use HBase as input/output source/destination for PigLatin scripts (see: semanticoverflow.com/questions/715/…). – castagna Apr 15 at 6:27

3 Answers

oldest newest votes

7

I can't comment specifically on HBase, but I have implemented RDF storage for Cassandra which has a very similar BigTable-inspired data model.

You basically have two options in how to store RDF data in wide-column databases like HBase and Cassandra: the resource-centric approach and the statement-centric approach.

In the statement-oriented approach, each RDF statement corresponds to a row key (for instance, a UUID) and contains subject, predicate and object columns. In Cassandra, each of these would be supercolumns that would then contain subcolumns such as type and value, to differentiate between RDF literals, blank nodes and URIs. If you needed to support named graphs, each row could also have a context column that would contain a list of the named graphs that the statement was part of.

The above is a relatively simple mapping to implement but suffers from some problems, notably the fact that preventing the creation of duplicate statements (an important RDF semantic) means having to do a read before doing a write, which at least in Cassandra quickly becomes a performance bottleneck as writes are much faster than reads.

There are ways to work around this problem, in particular by using content-addressable statement identifiers (e.g. the SHA-1 of the canonicalized N-Triples representation of each statement) as the row keys, but this in turn introduces other trade-offs such as no longer being able to version statement data: every time statement data changes, the old row needs to be deleted and a new one inserted with the new row key.

In view of the previous considerations, the resource-oriented approach is generally a better natural fit for storing RDF data in wide-column databases. In this approach, each RDF subject/resource corresponds to a row key, and each RDF predicate/property corresponds to a column or supercolumn. Keyspaces can be used to represent RDF repositories, and column families can be used to represent named graphs.

The main trade-off with the resource-based approach is that some statement-oriented operations become more complex and/or slower, notably those counting the total number of statements or querying for predicate or object terms without specifying a subject. To support performant basic graph pattern matching, additional POS, OPS, etc. indices may need to be created and maintained.

See RDF::Cassandra, my Cassandra storage adapter for RDF.rb, for a more detailed example of a resource-centric mapping from the RDF data model to a wide-column data model.

link|flag

edited Apr 25 at 19:21

answered Apr 25 at 19:13 Arto Bendiken 202●2●5

	Thank you for your detailed answer. In a "resource-oriented" approach, what do you do when you have multiple objects with the same property? – castagna Apr 25 at 20:20
	I take it you mean multiple object values for a given predicate on a particular subject/resource? With Cassandra this is relatively easy since predicates are represented by supercolumns, and each predicate value can be stored in a subcolumn of the supercolumn. With HBase, I suspect you'd have to look at making use of the column's timestamp value (which can be any arbitrary integer, apparently) to differentiate between multiple object term values. This means that the resource-centric storage approach may be less appropriate for HBase than it is for Cassandra... – Arto Bendiken Apr 25 at 21:43
	Are you using this just for storage of RDF or are you actively querying RDF with this? My concern with Cassandra is the need to create my own indexes if I then want to be able to do real querying over the data – Rob Vesse Oct 7 at 13:04

0

Given that HBase is designed for row based storage of data the simplest thing would be to have a GSPO layout i.e.

Graph | Subject | Predicate | Object

In terms of actually reading and writing data you'll most likely need to look at whatever API you want to use to manipulate the RDF and implement the necessary interfaces/classes that allows your chosen API to get data in and out of HBase.

As for querying then you'll need to implement a SPARQL engine for your database which could be rather complex or use an API that does SPARQL in-memory but this will have the overhead of needing to read out some/all of the data from HBase first.

Personally I don't know enough about HBase to say whether such an approach is viable or sensible, if you have hardware on which you can install and run HBase then you would most likely be better off installing and running a Triple Store for your RDF data.

link|flag

answered Apr 13 at 9:41 Rob Vesse 1,766●3●11

0

Hey Arto or Vesse, Why not use Cassandra/HBase as a secondary storage mechanism and the Jena GraphMemFaster as a Write-Through Cache? In the Cassandra case you have to use twice as much memory but it would solve your performance problems. Alternatively you could implement a Hexastore in Cassandra if you can afford the memory costs.

link|flag

edited Aug 26 at 7:23

answered Aug 26 at 7:08 Alexi 1●1

	Such solutions don't scale to datasets larger than memory, nor beyond a single node, negating much of the point of using Cassandra or HBase. And, as you say, it would use at least twice the memory even for use cases where you could fit it all on a single node. As I outlined above, the resource-centric approach is simpler in many ways, at least for Cassandra. – Arto Bendiken Aug 27 at 0:51
	You're making the assumption here that everyone uses Jena. Main issue with widetable nosql stores is that you have to do value indexing yourself which means they may be very good for just/reading RDF but they aren't so good for querying – Rob Vesse Oct 7 at 13:09