Category Archives: HBase

Delete snapshots older than 7 days

Posted on 2023 年 11 月 16 日 by cowman.chiang

To avoid running out of disk space in our test environment, we developed a plan to regularly execute shell scripts to clean up unnecessary snapshots. #!/bin/bash week=`date –date=’7 days ago’ +’%Y%m%d’` echo “list_snapshots” | /home/webuser/hbase-1.2.9/bin/hbase shell | grep “pattern ” … Continue reading →

Posted in HBase, Shell Script | Comments Off on Delete snapshots older than 7 days

HBase Client 2.5.5 in JDK 11

Posted on 2023 年 10 月 02 日 by cowman.chiang

Because of the vulnerability scan results of the project, the JDK version must be upgraded from 8 to 11 to complete the repair. The original HBase Client version 1.2 we used was incompatible with JDK 11, so we had to … Continue reading →

Posted in HBase, Java | Comments Off on HBase Client 2.5.5 in JDK 11

[HBase] security issue

Posted on 2019 年 03 月 18 日 by cowman.chiang

error message: com.a.b.c.exception.BaseDAOException: org.apache.hadoop.hbase.security.AccessDeniedException: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user ‘Cowman’ (table=ABC, action=READ) in jvm conf. add “-DHADOOP_USER_NAME=webuser”

Posted in HBase, Java | Leave a comment

[HBase] convert long value to bytes

Posted on 2018 年 11 月 13 日 by cowman.chiang

Java code: public static void main(String[] args) { String stringValue = Bytes.toStringBinary(Bytes.toBytes(1532080782183l)); System.out.println(stringValue); Long longValue = Bytes.toLong(Bytes.toBytesBinary(stringValue)); System.out.println(longValue); } HBase shell command: hbase(main):056:0> Bytes.toStringBinary(Bytes.to_bytes(1532080782183)) => “\\x00\\x00\\x01d\\xB7!{g” hbase(main):057:0> Bytes.toLong(“\x00\x00\x01d\xB7!{g”.to_java_bytes) => 1532080782183

Posted in HBase, Java | Leave a comment

[HBase] Filters not working for negative integers

Posted on 2018 年 07 月 12 日 by cowman.chiang

[stackoverflow] HBase: Filters not working for negative integers Easility say: Since, Hbase has only BinaryComparators and not other ‘typed’ comparators, it fails to filter on Negative integers as it stores the 2’s compliment of the negative number. Further, the binary … Continue reading →

Posted in HBase | Leave a comment

[HBase] get the scan result without specific cq

Posted on 2018 年 06 月 06 日 by cowman.chiang

scan ‘TableName’, FILTER=>”QualifierFilter(!=,’binary:QUALIFY2′)”

Posted in HBase | Leave a comment

Install Hbase 5.9 in Mac OS X

Posted on 2017 年 03 月 10 日 by cowman.chiang

Download hbase package tar.gz file from https://www.cloudera.com/documentation/enterprise/release-notes/topics/cdh_vd_cdh_package_tarball_59.html Untar tar.gz file edit conf/hbase-env.sh export JAVA_HOME={{JAVA_HOME Directory path}} edit conf/hbase-site.xml <property> <name>hbase.rootdir</name> <value>file:///{{location}}/data</value> </property> <property> <name>hbase.zookeeper.property.dataDir</name> <value>{{location}}/zookeeper</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>{{hostname}}</value> </property> start hbase service bin/start-hbase.sh run hbase shell bin/hbase shell stop … Continue reading →

Posted in HBase, Mac | Leave a comment

HBase: put byte value

Posted on 2016 年 10 月 04 日 by cowman.chiang

put “TableName”, “rowkey”, “cf:fieldname”, [0].pack(“N”) N => byte array put “TableName”, “rowkey”, “cf:fieldname”, [0].pack(“Q>”) Q => 64-bit unsigned, native endian => change endian to big endian

Posted in HBase | Leave a comment

[HBase] Scan with filter by partial rowkey using HBase shell

Posted on 2016 年 05 月 26 日 by cowman.chiang

If we have these rowkey list data. xx_abc xx_abc_def xx_abd xx_abd_rfg To get rowkey which contain “abc” scan ‘TableName’, FILTER => “RowFilter(=, ‘substring:abc’)” We will get these. xx_abc xx_abc_def But if we want only get “xx_abc” scan ‘TableName’, FILTER => … Continue reading →

Posted in HBase | Leave a comment

[HBase] get/scan String or Integer value

Posted on 2016 年 04 月 22 日 by cowman.chiang

get String Value get ‘Table_Name’, ‘RowKey’, {COLUMNS=>’Column_Family:Qualify:toString’} get Integer Value get ‘Table_Name’, ‘RowKey’, {COLUMNS=>’Column_Family:Qualify:toInt’} scan and get String Value scan ‘Table_Name’, {COLUMNS=>’Column_Family:Qualify:toString’} scan and get Integer Value scan ‘Table_Name’, {COLUMNS=>’Column_Family:Qualify:toInt’}

Posted in HBase | Leave a comment