By Lars Hofhansl Updated (again) Wednesday, January 25th, 2012. As I painfully worked through HBASE-5229 I realized that HBase already has all the building blocks needed for complex (local) transactions. What’s important here is that (see
By Lars Hofhansl
Updated (again) Wednesday, January 25th, 2012.
As I painfully worked through HBASE-5229 I realized that HBase already has all the building blocks needed for complex (local) transactions.
What’s important here is that (see my introduction to HBase):
- HBase ensures atomicity for operations for the same row key
- HBase keys have internal structure: (row-key, column family, column, …)
The missing piece was ColumnRangeFilter. With this filter it is possible to retrieve all columns whose identifier starts with “abc”, or all columns whose identifier sorts > “test”. For example:
// all columns whose identifier starts with “abc”
Filter f = new ColumnRangeFilter(Bytes.toBytes(“abc”), true,
Bytes.toBytes(“abd“), false);
// all columns whose identifier sorts after “test”
Filter f = new ColumnRangeFilter(Bytes.toBytes(“test”), true,
null, true);
So this allows to search (scan) inside a row by column identifier just as HBase allows searching by row key.
A client application can exploit this to achieve transactions by grouping all entities that can participate in the same transaction into a single row (and single column family).
Then using prefixes of the column identifiers can be used to define rows inside that group. Basically the search criteria for keys was moved one level down to the column identifier.
Say we wanted to implement a store with transactional tables that contain rows and columns. One way to doing this with HBase as follows:
- the HBase row-key/column-family maps to a “table”
- a prefix of the HBase column identifier maps to a “row”
- the rest of the HBase column identifier identifies the “column”
This is in fact similar to what Google’s 本文来源gaodai#ma#com搞*!代#%^码$网*Megastore (pdf) does.
This leads to potentially wide HBase rows with many columns. The missing piece is allowing a Scan to efficiently retrieve a slice of a wide row.
This where ColumnRangeFilter comes into play. This filter seeks efficiently into the row by seeking ahead to the first HBase block that contains the first KeyValue (or cell) for that column.
Let’s model a table “pets” this way. And let’s say a pet has a name and a species. The HBase key for entries would look like this:
(table, CF1, rowA|column1) -> value for column1 in rowA
The code would look something like this:
(apologies for the initial incorrect code that I had posted here)
HTable t = …;
Scan s = …;
s.setStartRow(“pets”);
s.setStopRow(“pets”);
// get all columns for my pet “fluffy”.
Filter f = new ColumnRangeFilter(Bytes.toBytes(“fluffy”), true,
Bytes.toBytes(“fluffz“), false);
s.setFilter(f);
s.setBatch(20); // avoid getting all columns for the HBase row
ResultScanner rs = t.getScanner(s);
for (Result r = rs.next(); r != null; r = rs.next()) {
// r will now have all HBase columns that start with “fluffy”,
// which would represent a single row
for (KeyValue kv : r.raw()) {
// each kv represent – the latest version of – a column
}
}
The downside of this is that HBase achieves atomicity by collocating all cells with the same row-key, so it has to be hosted by a single region server.
原文地址：HBase intra row scanning, 感谢原作者分享。
转载请注明原文链接：HBase intra row scanning
