Tips
Key=Value
to get Key=Value out from and input, I use this in a filter:
..... if [type] == "pinglog" { kv {} mutate { convert => { "pingtime" => "float" } } } else { if [type] == "syslog" { kv { include_keys => ["OUT","IN","SRC","SPT","DST","DPT","PROTO","ACTION"] trim => "<>\[\]," trimkey => "<>\[\]," } .....
According to http://logz.io/blog/5-logstash-pitfalls-and-how-to-avoid-them/ there a danger in using "kv" without adding specific fields
As my "pingtime" is a value I want to visualize and use for an Y-Axis, it must be numeric - hence the mutate to a float:
Drop something
THis is how I drop part of the syslog (google DNS lookup) after it has been "kv" (:
if [DST] == "8.8.8.8" { drop {} }
Adding fields/metadata
FRom the syslog, I would like to add an owner of the Source (Identified by Source IP "SRC") and a Device (also identified by Source IP "SRC")
if [SRC] == "10.0.0.102" { mutate { add_field => { "IPOwner" => "Toke" "Device" => "iPad" } } } ... ... if [SRC] == "10.0.0.109" { mutate { add_field => { "IPOwner" => "Christopher" "Device" => "Laptop LAN" } } }
These metadata give me a possibility to Visualize traffic by IPOwner:
Trobleshooting
Cant send data to Elasticsearch - Elasticsearch wont recieve
My Elasticseach was dowm see -
Can send data to Elasticsearch - Congestion
The /var/log/logstash/logstash.log shows:
{:timestamp=>"2016-04-08T08:32:32.217000+0200", :message=>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect_backoff_sleep=>0.5, :level=>:warn} {:timestamp=>"2016-04-08T08:32:32.721000+0200", :message=>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect_backoff_sleep=>0.5, :level=>:warn} {:timestamp=>"2016-04-08T08:32:33.232000+0200", :message=>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect_backoff_sleep=>0.5, :level=>:warn} {:timestamp=>"2016-04-08T08:32:33.733000+0200", :message=>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect_backoff_sleep=>0.5, :level=>:warn} {:timestamp=>"2016-04-08T08:32:34.282000+0200", :message=>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect_backoff_sleep=>0.5, :level=>:warn} {:timestamp=>"2016-04-08T08:32:34.783000+0200", :message=>"Beats input: the pipeline is blocked, temporary refusing new connection.", :reconnect_backoff_sleep=>0.5, :level=>:warn}
I have seen this twice, the first time setting congestion_threshold in the beats input to more than 5 (I use 25) - https://github.com/elastic/logstash/issues/4368 - helped
input { beats { port => 5044 ssl => true ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt" ssl_key => "/etc/pki/tls/private/logstash-forwarder.key" congestion_threshold => 25 } }
And restart logstash