Category Archives: 資料庫

[HBase] Scan with filter by partial rowkey using HBase shell

Posted on 2016 年 05 月 26 日 by cowman.chiang

If we have these rowkey list data. xx_abc xx_abc_def xx_abd xx_abd_rfg To get rowkey which contain “abc” scan ‘TableName’, FILTER => “RowFilter(=, ‘substring:abc’)” We will get these. xx_abc xx_abc_def But if we want only get “xx_abc” scan ‘TableName’, FILTER => … Continue reading →

Posted in HBase | Leave a comment

[HBase] get/scan String or Integer value

Posted on 2016 年 04 月 22 日 by cowman.chiang

get String Value

get 'Table_Name', 'RowKey', {COLUMNS=&gt;'Column_Family:Qualify:toString'}

1 2	get 'Table_Name', 'RowKey', {COLUMNS=>'Column_Family:Qualify:toString'}

get Integer Value

get 'Table_Name', 'RowKey', {COLUMNS=&gt;'Column_Family:Qualify:toInt'}

1 2	get 'Table_Name', 'RowKey', {COLUMNS=>'Column_Family:Qualify:toInt'}

scan and get String Value

scan 'Table_Name', {COLUMNS=&gt;'Column_Family:Qualify:toString'}

1 2	scan 'Table_Name', {COLUMNS=>'Column_Family:Qualify:toString'}

scan and get Integer Value

scan 'Table_Name', {COLUMNS=&gt;'Column_Family:Qualify:toInt'}

1 2	scan 'Table_Name', {COLUMNS=>'Column_Family:Qualify:toInt'}

Posted in HBase | Leave a comment

[HBase] run hbase command by ruby

Posted on 2016 年 04 月 22 日 by cowman.chiang

echo 'list.each {|t| truncate t}; quit;' | hbase shell

1 2	echo 'list.each {\|t\| truncate t}; quit;' \| hbase shell

or Drop Table

echo 'list.each {|t| disable t; drop t}; quit;' | hbase shell

1 2	echo 'list.each {\|t\| disable t; drop t}; quit;' \| hbase shell

Scan Table

list.each {|t| print t; scan t, {LIMIT=>1};}

1 2	list.each {\|t\| print t; scan t, {LIMIT=>1};}

Posted in HBase, Ruby | Leave a comment

[MySQL] Resolve duplicate entry error in MySQL Replication

Posted on 2015 年 12 月 22 日 by cowman.chiang

while [ 1 ]; do if [ $(mysql -uroot -ppassword -e"show slave status G;" | grep "Duplicate entry" | wc -l) -eq 1 ] ; then mysql -uroot -ppassword -e"stop slave; set global sql_slave_skip_counter=1; start slave;"; fi; sleep 1; mysql -uroot -ppassword -e"show slave statusG"; done

1 2	while [ 1 ]; do if [ $(mysql -uroot -ppassword -e"show slave status G;" \| grep "Duplicate entry" \| wc -l) -eq 1 ] ; then mysql -uroot -ppassword -e"stop slave; set global sql_slave_skip_counter=1; start slave;"; fi; sleep 1; mysql -uroot -ppassword -e"show slave statusG"; done

Ref. MySQL Replication Error 1062: ‘Duplicate entry for PRIMARY key’

Posted in MySQL | Leave a comment

[Java, HBase] Hostname is very important!

Posted on 2015 年 12 月 01 日 by cowman.chiang

If the connection is failed, try to check the hdfs uri setting, it might be used hostname.

Posted in HBase, Java | Leave a comment

[HBase] Use SingleColumnFilter in shell.

Posted on 2015 年 11 月 03 日 by cowman.chiang

scan 'TABLENAME', {FILTER=>"(SingleColumnValueFilter('CF', ‘CQ', =, 'binary:VALUE', true, true))”}

1 2	scan 'TABLENAME', {FILTER=>"(SingleColumnValueFilter('CF', ‘CQ', =, 'binary:VALUE', true, true))”}

the first true means filterIfColumnMissing the second true means setLatestVersionOnly Ref. Spinning Thoughts: Applying Filters in HBase shell

Posted in HBase | 2 Comments

[Java、HBase] Retrieving timestamp form HBase row

Posted on 2015 年 08 月 11 日 by cowman.chiang

Ref. stackoverflow: retrieving timestamp from hbase row

HTableInterface hTableInterface = CONNECTION.getTable(Table.SCHEMA_NAME);
Result result = hTableInterface.get(get);
Date date = new Date(result.raw()[0].getTimestamp());
System.out.println("DateTime: " + sdf.format(date));

HTableInterface hTableInterface = CONNECTION.getTable(Table.SCHEMA_NAME);

Result result = hTableInterface.get(get);

Date date = new Date(result.raw()[0].getTimestamp());

System.out.println("DateTime: " + sdf.format(date));

we must edit HBaseSettings to set hbaseConf.set(HBASE_ENV_KEY_ZOOKEEPER_QUORUM, “target_zookeeper_address”);

Posted in HBase, Java | Tagged HBase, Java | Leave a comment

HBase get

Posted on 2015 年 05 月 27 日 by cowman.chiang

get 'TableName', ‘RowKey'

get 'TableName', ‘RowKey’, 'cf:qualifier'

get 'TableName', 'RowKey', {FILTER => "ValueFilter(=, 'binary:Search Key')"}

get 'TableName', ‘RowKey'

get 'TableName', ‘RowKey’, 'cf:qualifier'

get 'TableName', 'RowKey', {FILTER => "ValueFilter(=, 'binary:Search Key')"}

Posted in HBase | Leave a comment

HBase scan

Posted on 2015 年 05 月 27 日 by cowman.chiang

scan 'TableName', FILTER => "ValueFilter(=, ‘binary:Search key’)"

scan 'TableName', {FILTER => "ValueFilter(=, 'binary:Search key1') OR ValueFilter(=, 'binary:Search key2’)"}

scan 'TableName', {COLUMNS => 'cf:qualifer', FILTER =&gt; "ValueFilter(&gt;=, 'binary:Search key1') AND ValueFilter(&lt;, 'binary:Search key1~’)"}

scan 'TableName', FILTER => "ValueFilter(=, 'binaryprefix:+8869’)"
//符合+8869開頭的

scan 'TableName', FILTER => "ValueFilter(=, 'substring:345678')"
//部分字串符合

scan 'TableName', FILTER => "RowFilter(=, 'substring:k3S')"
//Rowkey部分字串符合

scan 'TableName', FILTER => "ValueFilter(=, 'regexstring:^.*610')"
//符合610結尾

scan 'TableName', FILTER => "ValueFilter(=, ‘binary:Search key’)"

scan 'TableName', {FILTER => "ValueFilter(=, 'binary:Search key1') OR ValueFilter(=, 'binary:Search key2’)"}

scan 'TableName', {COLUMNS => 'cf:qualifer', FILTER => "ValueFilter(>=, 'binary:Search key1') AND ValueFilter(<, 'binary:Search key1~’)"}

scan 'TableName', FILTER => "ValueFilter(=, 'binaryprefix:+8869’)"

//符合+8869開頭的

scan 'TableName', FILTER => "ValueFilter(=, 'substring:345678')"

//部分字串符合

scan 'TableName', FILTER => "RowFilter(=, 'substring:k3S')"

//Rowkey部分字串符合

scan 'TableName', FILTER => "ValueFilter(=, 'regexstring:^.*610')"

//符合610結尾

Posted in HBase | Leave a comment

[Apache HBase] Install Apache HBase in Ubuntu 14.04

Posted on 2015 年 03 月 22 日 by cowman.chiang

Java 1.7

add-apt-repository ppa:webupd8team/java
apt-get update
apt-get install oracle-java7-installer
apt-get install oracle-java7-set-default

add-apt-repository ppa:webupd8team/java

apt-get update

apt-get install oracle-java7-installer

apt-get install oracle-java7-set-default

Cloudera HBase Ref. installing-apache-hbase-on-ubuntu-for-standalone-mode Note. hbase tar file choose http://archive.apache.org/dist/hbase/hbase-0.98.5/hbase-0.98.5-hadoop1-bin.tar.gz (or http://ftp.mirror.tw/pub/apache/hbase/hbase-0.98.11/hbase-0.98.11-hadoop1-bin.tar.gz )

Posted in HBase, Ubuntu | Leave a comment

Category Archives: 資料庫

Categories