HBase filter

Posted by slug58 on Fri, 19 Jul 2019 05:07:15 +0200

Filter


There are many types of filters, but they can be divided into two categories - Comparative filters and special filters.
The function of the filter is to judge whether the data satisfy the condition at the server side, and then return the data satisfying the condition to the client side only.

The basic API s of HBase include addition, deletion, modification, check and so on.
Addition and deletion are relatively simple operations. Compared with traditional RDBMS, the query operations here are slightly pale, and can only be queried according to the characteristics of the row keys (Get) or according to the range of the row keys (Scan).
HBase not only provides these simple queries, but also provides more advanced filters to query.

 

Abstract operator (comparison operator)

LESS <

LESS_OR_EQUAL <=

EQUAL =

NOT_EQUAL <>

GREATER_OR_EQUAL >=

GREATER >

NO_OP Exclude all

Comparator (specifying a comparison mechanism)

BinaryComparator compares specified byte arrays in byte index order, using Bytes.compareTo(byte [])

The BinaryPrefixComparator is the same as the previous one, except that the data on the left side is the same.

Null Comparator judges whether a given is empty

Bit Comparator Bit Comparator Bit Comparator Bit Comparator Bit Comparator Bit Comparator Bit Comparator Bit Comparator Bit Comparator Bit Comparator Bit Comparator

RegexString Comparator provides a regular comparator that supports only EQUAL and non-EQUAL

Substring Comparator determines whether the supplied substring appears in value

Comparing filters

1. RowFilter (line key filter or rowkey filter)

Filter rowFilter = new RowFilter(CompareOp.GREATER, new BinaryComparator("95007".getBytes()));
scan.setFilter(rowFilter);

2. Column Cluster Filter FamilyFilter

Filter familyFilter = new FamilyFilter(CompareOp.EQUAL, new BinaryComparator("info".getBytes()));
scan.setFilter(familyFilter);

3. Column filter Qualifier Filter

Filter qualifierFilter = new QualifierFilter(CompareOp.EQUAL, new BinaryComparator("name".getBytes()));
scan.setFilter(qualifierFilter);

4. Value Filter

Filter valueFilter = new ValueFilter(CompareOp.EQUAL, new SubstringComparator("male"));
scan.setFilter(valueFilter);

5. Timestamps Filter

List<Long> list = new ArrayList<>();
list.add(1522469029503l);
TimestampsFilter timestampsFilter = new TimestampsFilter(list);
scan.setFilter(timestampsFilter);

Special filter

1. Single column value filter SingleColumnValueFilter -- returns the entire line that satisfies the criteria

SingleColumnValueFilter singleColumnValueFilter = new SingleColumnValueFilter(
                "info".getBytes(), //Column Cluster
                "name".getBytes(), //column
                CompareOp.EQUAL, 
                new SubstringComparator("Liu Chen"));
//If not set to true, rows that do not contain the specified column will also return
singleColumnValueFilter.setFilterIfMissing(true);
scan.setFilter(singleColumnValueFilter);

2. Single Column Value Exclude Filter

SingleColumnValueExcludeFilter singleColumnValueExcludeFilter = new SingleColumnValueExcludeFilter(
                "info".getBytes(), 
                "name".getBytes(), 
                CompareOp.EQUAL, 
                new SubstringComparator("Liu Chen"));
singleColumnValueExcludeFilter.setFilterIfMissing(true);
        
scan.setFilter(singleColumnValueExcludeFilter);

3. Prefix Filter - For row keys

PrefixFilter prefixFilter = new PrefixFilter("9501".getBytes());
scan.setFilter(prefixFilter);

4. Column Prefix Filter

ColumnPrefixFilter columnPrefixFilter = new ColumnPrefixFilter("name".getBytes());
scan.setFilter(columnPrefixFilter);

Page Filter

 scan.setStartRow(startRow.getBytes());
 scan.setMaxResultSize(pageSize);//Set how many pieces of data we scan
 PageFilter filter1 = new PageFilter(pageSize);

 

Topics: HBase less