Logstash + Redis + Nginx + Kibana + ElasticSearch

環境: Ubuntu 14.04 x64 server

安裝Java環境

sudo apt-get install openjdk-7-jdk

下載Logstash壓縮檔

cd /tmp
curl -O https://download.elasticsearch.org/logstash/logstash/logstash-1.4.0.tar.gz
tar zxvf logstash-1.4.0.tar.gz -C /opt/logstash

基本測試

cd /opt/logstash
bin/logstash -e 'input { stdin { } } output { stdout {} }
# 試著打幾個字看看

安裝Elasticsearch

cd /tmp
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.1.1.deb
sudo dpkg -i elasticsearch-1.1.1.deb

設定Elasticsearch
編輯/etc/elasticsearch/elasticsearch.yml
取消cluster.name、node.name的注解並改成自己想要的

重新啟動Elasticsearch

sudo service elastic search restart

安裝Redis

sudo apt-get install redis-server

設定Redis
編輯/etc/redis/redis.conf，使用

bind 0.0.0.0
取代
bind 127.0.0.1

重新啟動Redis

sudo service redis-server restart

下載Kibana

cd /tmp
wget https://download.elasticsearch.org/kibana/kibana/kibana-3.0.0.tar.gz
mkdir -p /var/www/kibana
tar xvfz kibana-3.0.0.tar.gz -C /var/www/kibana

編輯Kibana設定檔
編輯/var/www/kubana/config.js，使用

elasticsearch: "http://本機IP:9200"
取代
elasticsearch: "http://"+window.location.hostname+":9200"

設定Logstash Server設定檔
編輯/etc/logstash/server.conf

input {
  redis {
    host => "本機IP"
    type => "redis"
    data_type => "list"
    key => "logstash"
  }
}
output {
stdout { }
  elasticsearch {
    cluster => "elasticsearch"
  }
}

測試Logstash能否以Server啟動

/opt/logstash/bin/logstash --verbose -f /etc/logstash/server.conf

安裝Nginx

sudo apt-get install nginx

設定Nginx
編輯/etc/nginx/sites-enabled/default，使用

root /var/www/kibana
取代
root /usr/share/nginx/www

重新啟動Nginx

service nginx restart

設定Logstash agent設定檔
編輯/etc/logstash/agent.conf

input {
  file {
    type => "nginx-access"
    path => "/var/log/nginx/access.log"
  }
 
  file {
    type => "nginx-error"
    path => "/var/log/nginx/error.log"
  }

  tcp {
    port => 514
    type => syslog
  }

  udp {
    port => 514
    type => syslog
  }
}
 
output {
  stdout { }
  redis {
    host => "本機IP"
    data_type => "list"
    key => "logstash"
  }
}

測試Logstash能否以agent啟動

/opt/logstash/bin/logstash agent --verbose -f /etc/logstash/agent.conf

設定Logstash Upstart Service
(1) 編輯/etc/init/logstash-server.conf

description "Logstash Server"

#start on started networking
start on started redis-server
stop on runlevel [!2345]

env LogstashHome=/opt/logstash
env LogstashLogs=/var/log/logstash

respawn

pre-start script
    cd $LogstashHome
    mkdir $LogstashLogs                        ||true
end script

script
  cd $LogstashHome
  ./bin/logstash --verbose -f /etc/logstash/server.conf 
                        >> $LogstashLogs/access.log 
                        2>> $LogstashLogs/error.log
end script

(2) 編輯/etc/init/logstash-agent.conf

description "Logstash Agent"

#start on started networking
start on runlevel [2345]
stop on runlevel [!2345]

env LogstashHome=/opt/logstash
env LogstashLogs=/var/log/logstash

respawn

pre-start script
    cd $LogstashHome
    mkdir $LogstashLogs                        ||true
end script

script
  cd $LogstashHome
  ./bin/logstash agent --verbose -f /etc/logstash/agent.conf 
                        >> $LogstashLogs/agent-access.log 
                        2>> $LogstashLogs/agent-error.log
end script

使用Upstart Service測試啟動Logstash

start logstash-server
start logstash-agent

備註:
1. 測試Logstash設定檔

logstash --configtest -f 檔案

Logstash + Redis + Nginx + Kibana + ElasticSearch

Categories