Category Archives: Logstash

[Logstash] Pattern set in grok.

Posted on 2015 年 02 月 16 日 by cowman.chiang

The original warning message is this.

"You are using a deprecated config setting "pattern" set in grok. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. You should use this instead: match =&gt; { "message" =&gt; "your pattern here" } If you have any questions about this, please visit the #logstash channel on freenode irc.""

"You are using a deprecated config setting "pattern" set in grok. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. You should use this instead: match => { "message" => "your pattern here" } If you have any questions about this, please visit the #logstash channel on freenode irc.""

And here is the original server config.

if [type] == "nginx-access" {
  grok {
    type =&gt; "nginx-access"
    pattern =&gt; "%{NGINXACCESS}"
    patterns_dir =&gt; ["/opt/logstash/patterns"]
  }
}

if [type] == "nginx-access" {

grok {

type => "nginx-access"

pattern => "%{NGINXACCESS}"

patterns_dir => ["/opt/logstash/patterns"]

}

So what we should do is modify the config. Using the match function to replace the pattern, like this.

if [type] == "nginx-access" {
  grok {
    patterns_dir =&gt; "/opt/logstash/patterns"
    type =&gt; "nginx-access"
    match =&gt; ["message", "%{NGINXACCESS}"]
  }
}

if [type] == "nginx-access" {

grok {

patterns_dir => "/opt/logstash/patterns"

type => "nginx-access"

match => ["message", "%{NGINXACCESS}"]

}

Posted in Logstash | Leave a comment

[Logstash] LoadError: no such file to load — i18n

Posted on 2015 年 01 月 27 日 by cowman.chiang

In CentOS 5.5, I want to install the logstash agent to forward logs. Because it was older system and no one would upgrade it, the original java version is 1.6. I always got the error about “LoadError: no such file … Continue reading →

Posted in Logstash | Leave a comment

Logstash + Redis + Nginx + Kibana + ElasticSearch

Posted on 2014 年 12 月 26 日 by cowman.chiang

環境: Ubuntu 14.04 x64 server 安裝Java環境

sudo apt-get install openjdk-7-jdk

1	sudo apt-get install openjdk-7-jdk

下載Logstash壓縮檔

cd /tmp
curl -O https://download.elasticsearch.org/logstash/logstash/logstash-1.4.0.tar.gz
tar zxvf logstash-1.4.0.tar.gz -C /opt/logstash

cd /tmp

curl -O https://download.elasticsearch.org/logstash/logstash/logstash-1.4.0.tar.gz

tar zxvf logstash-1.4.0.tar.gz -C /opt/logstash

基本測試

cd /opt/logstash
bin/logstash -e 'input { stdin { } } output { stdout {} }
# 試著打幾個字看看

cd /opt/logstash

bin/logstash -e 'input { stdin { } } output { stdout {} }

# 試著打幾個字看看

安裝Elasticsearch

cd /tmp
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.1.1.deb
sudo dpkg -i elasticsearch-1.1.1.deb

cd /tmp

wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.1.1.deb

sudo dpkg -i elasticsearch-1.1.1.deb

設定Elasticsearch 編輯/etc/elasticsearch/elasticsearch.yml 取消cluster.name、node.name的注解並改成自己想要的重新啟動Elasticsearch

sudo service elastic search restart

1	sudo service elastic search restart

安裝Redis

sudo apt-get install redis-server

1	sudo apt-get install redis-server

設定Redis 編輯/etc/redis/redis.conf，使用

bind 0.0.0.0
取代
bind 127.0.0.1

bind 0.0.0.0

取代

bind 127.0.0.1

重新啟動Redis

sudo service redis-server restart

1	sudo service redis-server restart

下載Kibana

cd /tmp
wget https://download.elasticsearch.org/kibana/kibana/kibana-3.0.0.tar.gz
mkdir -p /var/www/kibana
tar xvfz kibana-3.0.0.tar.gz -C /var/www/kibana

cd /tmp

wget https://download.elasticsearch.org/kibana/kibana/kibana-3.0.0.tar.gz

mkdir -p /var/www/kibana

tar xvfz kibana-3.0.0.tar.gz -C /var/www/kibana

編輯Kibana設定檔編輯/var/www/kubana/config.js，使用

elasticsearch: "http://本機IP:9200"
取代
elasticsearch: "http://"+window.location.hostname+":9200"

elasticsearch: "http://本機IP:9200"

取代

elasticsearch: "http://"+window.location.hostname+":9200"

設定Logstash Server設定檔編輯/etc/logstash/server.conf

input {
  redis {
    host => "本機IP"
    type => "redis"
    data_type => "list"
    key => "logstash"
  }
}
output {
stdout { }
  elasticsearch {
    cluster => "elasticsearch"
  }
}

input {

redis {

host => "本機IP"

type => "redis"

data_type => "list"

key => "logstash"

}

output {

stdout { }

elasticsearch {

cluster => "elasticsearch"

}

測試Logstash能否以Server啟動

/opt/logstash/bin/logstash --verbose -f /etc/logstash/server.conf

1	/opt/logstash/bin/logstash --verbose -f /etc/logstash/server.conf

安裝Nginx

sudo apt-get install nginx

1	sudo apt-get install nginx

設定Nginx 編輯/etc/nginx/sites-enabled/default，使用 … Continue reading →

Posted in Elasticsearch, Kibana, Logstash, Redis | Leave a comment

Category Archives: Logstash

[Logstash] Pattern set in grok.

[Logstash] LoadError: no such file to load — i18n

Logstash + Redis + Nginx + Kibana + ElasticSearch

Categories