Configurar filebeat em JBOSS/Wildfly


1. Instalar o filebeat
> curl -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-6.3.2-amd64.deb
> dpkg -i filebeat-oss-6.3.2-amd64.deb

2. Configurar o filebeat
> /etc/filebeat/filebeat.yml

filebeat.inputs:

- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /usr/local/wildfly/standalone/log/server.log


  #Multiline pattern wildfly Timestamp
  #referencia https://www.elastic.co/guide/en/beats/filebeat/master/_examples_of_multiline_configuration.html
  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after


#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 2
  index.number_of_replicas: 0
 
#================================ Outputs =====================================

# Configure what output to use when sending the data collected by the beat.

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["localhost:9200"]
  pipeline: "wildfly-pipeline-parser"



3. Configurar o injest pipeline no kibana

#referencias
  # grok => https://www.elastic.co/guide/en/elasticsearch/reference/current/grok-processor.html
  # date => https://www.elastic.co/guide/en/elasticsearch/reference/current/date-processor.html
  
  
put _ingest/pipeline/wildfly-pipeline-parser
{
    "description": "Parse log lines",
    "processors": [
        {
            "grok": {
                "field": "message",
                "patterns": [
                    "(?<data_wildfly>\\d+-\\d+-\\d+ \\d+:\\d+:\\d+,\\d+) +(?<severity>\\S+) +\\[(?<thread>[^:]+)\\] +(?<class>\\S+) (?<message>(.|\r|\n)*)",
                    "(?<message>.+)"
                ]
            }
        },
        {
            "date": {
                "field": "data_wildfly",
                "formats": [
                    "yyyy-MM-dd HH:mm:ss,SSS"
                ],
                "timezone":"-0300"
            }
        }
    ]
}


* Teste do pipeline => https://www.elastic.co/guide/en/elasticsearch/reference/master/simulate-pipeline-api.html
POST _ingest/pipeline/wildfly-pipeline-parser/_simulate
{
  "docs" : [
    { "_source": {"message":"2018-07-26 15:30:03,984 INFO  [MSC service thread 1-1] org.jboss.weld.Version WELD-000900: 2.3.5 (Final)" }}
  ]
}