Category Archives: Splunk

[Splunk] Get the quota exceeded count

Posted on 2015 年 02 月 03 日 by cowman.chiang

When we want to find some way to get the count of the quota exceed event, we might find this page “Splunk > Wiki : Community:TroubleshootingIndexedDataVolume Troubleshooting Indexed Data Volume”. Then we could use the following to get the count. … Continue reading →

Posted in Splunk | Leave a comment

Splunk : 計算log檔案在Splunk所佔的大小

Posted on 2014 年 10 月 20 日 by cowman.chiang

計算檔案空間 Splunk 語法： source=”/home/splunk_input_data/imss/2014/log.imss.20141019.0002″ | eval esize=len(_raw) | stats sum(esize) AS sum_esize, count | eval fsize=sum_esize + count | fields fsize Linux指令： grep -v -e “^s*$” /home/splunk_input_data/imss/2014/log.imss.20141019.0002 | wc -c 計算筆數 Splunk 語法： source=”/home/splunk_input_data/imss/2014/log.imss.20141019.0002″ | stats sum(linecount) Linux指令： grep -v … Continue reading →

Posted in Linux, Splunk | Leave a comment

Splunk 安裝

Posted on 2013 年 09 月 30 日 by cowman.chiang

至 Splunk 安裝程式下載網頁依據系統下載對應的安裝程式 (此步驟需要Splunk上的帳號) <—-後續以 CentOS 6.x 32bit為例—->

Posted in Splunk | Leave a comment

Splunk Subsearch ( 或是一次執行兩個有關係的Search )

Posted on 2013 年 05 月 31 日 by cowman.chiang

sourcetype=”mail_pop3″ [search sourcetype=”mail_pop3″ | top 100 pop3_loginfailed_ip | rename pop3_loginfailed_ip as pop3_signingon_ip | fields pop3_signingon_ip | outputcsv pop3_loginfailed_ip_top10.csv] [| inputcsv pop3_loginfailed_ip_top10.csv | return 10 pop3_signingon_ip] | stats count by pop3_signingon_ip, pop3_signingon_id 綠色：這個搜尋主要是將POP3_loginfailed_ip的前10名輸出至pop3_loginfailed_ip_top10.csv檔案並且同時將pop3_loginfailed_ip的欄位名稱以pop3_signingon_ip取代，以便後續input的時候使用藍色：接著就輸入pop3_loginfailed_ip_top10.csv檔案，並且依據pop3_signingon_ip這個欄位產生10組搜尋語法假設pop3_login_failed_ip_top10.csv的內容為

"pop3_signingon_ip",       "_tc"
"206.54.106.82",           61296
"119.42.150.122",          61296
"211.79.36.249",           61296
"219.68.232.164",          61296
"59.124.81.159",           61296

"pop3_signingon_ip", "_tc"

"206.54.106.82", 61296

"119.42.150.122", 61296

"211.79.36.249", 61296

"219.68.232.164", 61296

"59.124.81.159", 61296

則”return 10 … Continue reading →

Posted in Splunk | Leave a comment

FTP Download Script

Posted on 2013 年 05 月 22 日 by cowman.chiang

< pre lang=”bash”> !/bin/sh 程式說明：每日將 Mail Log 匯入 Splunk 之 Script 版本時間：2013/05/30 10:40 開發維護：cowman.chiang@udngroup.com 檢查檔案是否存在，存在則移除檔案避免ftp下載時無法覆蓋檔案以致無法取得新版檔案 function checkfile(){ if test -f “${local_dir}/${1}” then echo “File ${1} exist, remove!” rm “${local_dir}/${1}” else echo “File ${1} not exist” a=1 fi } 設定 … Continue reading →

Posted in Linux, Splunk | Leave a comment

Splunk 指定Web服務Port

Posted on 2013 年 05 月 22 日 by cowman.chiang

方法1 登入 Web 後進管理員更改方法2 修改 web.conf (但是我還是不知道在哪….) 方法3 指令直接更改 (狂推) /opt/splunk/bin/splunk set web-port 443

Posted in Splunk | Leave a comment

Splunk搬移到新主機

Posted on 2013 年 05 月 17 日 by cowman.chiang

{Old-Splunk-server}：service splunk stop {Old-Splunk-server}：tar cvf {檔案名稱} /opt/splunk/ {Old-Splunk-server}：scp {檔案名稱} {帳號}@{New-Splunk-server}:/tmp {New-Splunk-server}：tar xvf /tmp/{檔案名稱} -C / {New-Splunk-server}：下載Splunk Tar安裝檔案 {New-Splunk-server}：tar xvzf {Splunk-Tar-安裝檔案} -C /opt {New-Splunk-server}：/opt/splunk/bin/splunk enable boot-start 就可以依造 {Old-Splunk-server} 的連接設定進行使用但需要注意因為IP名稱應該會更動，有關IP的部分都要重新設定包含Clinet端Foward的目標、Syslog server的設定等

Posted in Splunk | Leave a comment

Splunk – Transaction 查詢一段相近時間之累積資訊

Posted on 2013 年 02 月 21 日 by cowman.chiang

sourcetype=”來源型態” | rex “(?i) .? User (?P<pop3_failed_id>[^.]+) login failed.” | search pop3_failed_id=”” | transaction 判斷資訊 maxevents=2000 keepevicted=true | concurrency duration=duration | eval duration=tostring(duration,”duration”) 原理有點像是將所有事件跑過一次，然後把相近時間的判斷資訊放在同一筆事件中一般預設單筆事件僅顯示500筆資訊但是有時候可能會是好幾萬筆資訊就會被拆成好幾筆事件，將造成事件數字判讀上的不便因此可以進行下面的動作將

/opt/splunk/etc/apps/search/default/data/ui/views/flashtimeline.xml

1	/opt/splunk/etc/apps/search/default/data/ui/views/flashtimeline.xml

檔案複製到

/opt/splunk/etc/apps/search/local/data/ui/views/flashtimeline.xml

1	/opt/splunk/etc/apps/search/local/data/ui/views/flashtimeline.xml

編輯

/opt/splunk/etc/apps/search/local/data/ui/views/flashtimeline.xml

1	/opt/splunk/etc/apps/search/local/data/ui/views/flashtimeline.xml

搜尋

<module name="EventsViewer" layoutPanel="resultsAreaLeft">

1	<module name="EventsViewer" layoutPanel="resultsAreaLeft">

在下面插入

<param name="maxLinesConstraint">數字</param>

1	<param name="maxLinesConstraint">數字</param>

會變成

<module name="EventsViewer" layoutPanel="resultsAreaLeft">
        
        <param name="maxLinesConstraint">數字</param>

        <param name="segmentation">full</param>

        <param name="reportFieldLink">report_builder_format_report</param>

</module>

<param name="reportFieldLink">report_builder_format_report</param>

</module>

原廠建議數字不要超過1000，以免造成系統負擔過重，但實際還是請視需要調整 … Continue reading →

Posted in Splunk | Leave a comment

Splunk Lincese Warning 次數

Posted on 2013 年 02 月 19 日 by cowman.chiang

除了可以到 “管理員=>授權” 去查看外還可以搜尋 “index=_internal licensemanager” 根據搜尋的結果看 Audit:[quotaExceededCount=1 的數量所以便可以擷取欄位將 (?i) Audit:[quotaExceededCount=(?P[^,]+) 存為記得的名稱以便後續搜尋

Posted in Splunk | Leave a comment

Splunk CLI With Script

Posted on 2013 年 02 月 08 日 by cowman.chiang

因為 Splunk CLI 一樣需樣登入所以可以先在 Script 檔案中預先輸入帳號、密碼

    export SPLUNK_USERNAME=帳號
    export SPLUNK_PASSWORD=密碼

1 2	export SPLUNK_USERNAME=帳號 export SPLUNK_PASSWORD=密碼

然後在下 Splunk 的登入、查詢等指令

    /opt/splunk/bin/splunk login
    /opt/splunk/bin/splunk search 'index=_internal source=*metrics.log group=per_index_thruput NOT (series=_* OR series=*summary) starttime=02/07/2013:00:00:00 | timechart span=1d sum(eval(kb/1024)) AS "MB indexed" by series | fields + main | fields - _* | outputcsv test_usage.csv'

/opt/splunk/bin/splunk login

/opt/splunk/bin/splunk search 'index=_internal source=*metrics.log group=per_index_thruput NOT (series=_* OR series=*summary) starttime=02/07/2013:00:00:00 | timechart span=1d sum(eval(kb/1024)) AS "MB indexed" by series | fields + main | fields - _* | outputcsv test_usage.csv'

這邊主要是將結果輸出到 test_usage.csv ，預設存放路徑為 “/opt/splunk/var/run/splunk/” (檔案格式為CSV) 並且輸出欄位僅留下 “main” 順帶一提的是 Splunk CLI 指令中不能包含常用的 Script 變數所以可以先用 Script 將變數搭配 Splunk 指令輸出到一個文字檔中再去執行該文字檔，下面是個例子

/bin/sh /tmp/test_usage.sh > /tmp/test_usage2.sh
/bin/sh /tmp/test_usage2.sh

1 2	/bin/sh /tmp/test_usage.sh > /tmp/test_usage2.sh /bin/sh /tmp/test_usage2.sh

而第一個 Script 檔案要注意保留字元的部分

/bin/echo "export SPLUNK_USERNAME=帳號"
/bin/echo "export SPLUNK_PASSWORD=密碼"
/bin/echo "/opt/splunk/bin/splunk login"
/bin/echo "/opt/splunk/bin/splunk search 'index=_internal source=*metrics.log group=per_index_thruput NOT (series=_* OR series=*summary) starttime=$(date '+%m/%d/%Y'):00:00:00 | timechart span=1d sum(eval(kb/1024)) AS "MB indexed" by series | fields + main | fields - _* | outputcsv test_usage.csv'"
/bin/echo "/bin/echo "已使用額度""
/bin/echo "/bin/echo $(/bin/sed '1d;s/^.//;s/.$//' /opt/splunk/var/run/splunk/test_usage.csv)"

/bin/echo "export SPLUNK_USERNAME=帳號"

/bin/echo "export SPLUNK_PASSWORD=密碼"

/bin/echo "/opt/splunk/bin/splunk login"

/bin/echo "/opt/splunk/bin/splunk search 'index=_internal source=*metrics.log group=per_index_thruput NOT (series=_* OR series=*summary) starttime=$(date '+%m/%d/%Y'):00:00:00 | timechart span=1d sum(eval(kb/1024)) AS "MB indexed" by series | fields + main | fields - _* | outputcsv test_usage.csv'"

/bin/echo "/bin/echo "已使用額度""

/bin/echo "/bin/echo $(/bin/sed '1d;s/^.//;s/.$//' /opt/splunk/var/run/splunk/test_usage.csv)"

Posted in Splunk | Leave a comment

Category Archives: Splunk

Categories