Skip to content

Instantly share code, notes, and snippets.

@gerardorochin

gerardorochin/php_error_logstash.conf

Created June 2, 2014 15:39
Show Gist options
  • Save gerardorochin/36d2b1be8b65ca0c7373 to your computer and use it in GitHub Desktop.
Save gerardorochin/36d2b1be8b65ca0c7373 to your computer and use it in GitHub Desktop.
Download ZIP
php error logging into logstash + elasticsearch and trace errors on single line and root path hidden
Raw
php_error_logstash.conf
input {
file {
type => "php-error"
path => "/var/www/error_log"
sincedb_path => "/opt/logstash/sincedb-access"
}
}
filter {
mutate {
gsub => [
"message", "/var/www", "",
"message", "/var/www", "",
"path", "/var/www", ""
]
}
if [type] == "php-error" {
grok {
match => [ "message", "\[%{MONTHDAY:day}-%{MONTH:month}-%{YEAR:year} %{TIME:time} %{WORD:zone}/%{WORD:country}\] PHP %{DATA:level}\: %{GREEDYDATA:error}" ]
add_field => { "timestamp" => "%{day}-%{month}-%{year} %{time} %{zone}/%{country}" }
add_tag => [ "%{level}" ]
remove_field => [ "day", "month", "year", "time", "zone", "country" ]
}
multiline {
pattern => "(Stack trace:)|(^#.+)|(^\"\")|( thrown+)|(^\s)"
what => "previous"
}
date {
timezone => "America/Mexico_City"
match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss", "dd-MMM-yyyy HH:mm:ss ZZZ" ]
target => "@timestamp"
remove_field => "timestamp"
}
}
mutate {
uppercase => [ "level" ]
lowercase => [ "tags" ]
gsub => [
"tags", " ", "_",
"level", " ", "_"
]
}
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
host => "localhost"
}
}
@gerardorochin
Copy link
Author

Single line

{
       "message" => "[31-May-2014 20:44:29 America/Mexico_City] PHP Fatal error:  Call to a member function detach() on a non-object in clickbalance/lib/phpexcel/PHPExcel/CachedObjectStorage/MemorySerialized.php on line 47",
      "@version" => "1",
    "@timestamp" => "2014-06-01T01:44:29.000Z",
          "type" => "php-error",
          "host" => "clickbalance.vm",
          "path" => "/clickbalance/error_log",
          "tags" => [
        [0] "fatal_error"
    ],
         "level" => "FATAL_ERROR",
         "error" => "Call to a member function detach() on a non-object in clickbalance/lib/phpexcel/PHPExcel/CachedObjectStorage/MemorySerialized.php on line 47"
}

Multiline with trace error

{
       "message" => "[14-May-2014 16:46:41 America/Mexico_City] PHP Fatal error:  Uncaught exception 'DOMPDF_Exception' with message 'Requested HTML document contains no data.' in clickbalance/lib/dompdf/include/frame_tree.cls.php:146\nStack trace:\n#0 clickbalance/lib/dompdf/include/dompdf.cls.php(451): Frame_Tree->build_tree()\n#1 clickbalance/lib/dompdf/include/dompdf.cls.php(591): DOMPDF->_process_html()\n#2 clickbalance/core/Dispatcher.php(608): DOMPDF->render()\n#3 clickbalance/index.php(21): Dispatcher->go('/clickbalance/i...')\n#4 {main}\n  thrown in clickbalance/lib/dompdf/include/frame_tree.cls.php on line 146",
      "@version" => "1",
    "@timestamp" => "2014-05-14T21:46:41.000Z",
          "type" => "php-error",
          "host" => "clickbalance.vm",
          "path" => "/public_html/staging/clickbalance/error_log",
          "tags" => [
        [0] "fatal_error",
        [1] "multiline"
    ],
         "level" => "FATAL_ERROR",
         "error" => "Uncaught exception 'DOMPDF_Exception' with message 'Requested HTML document contains no data.' in clickbalance/lib/dompdf/include/frame_tree.cls.php:146"
}

@friedlysol
Copy link

Thank you

@jmkgreen
Copy link

So our logs have a different date format - instead of Region/CIty we get the timezone (UTC).

The following works for us:

    match => [ "message", "\[%{MONTHDAY:day}-%{MONTH:month}-%{YEAR:year} %{TIME:time} %{WORD:zone}\] PHP %{DATA:level}\:  %{GREEDYDATA:error}" ]
    add_field    => { "timestamp" => "%{day}-%{month}-%{year} %{time} %{zone}" }

@mathiasaerts
Copy link

Any reason why the "message", "/var/www", "", line is listed twice within mutate?

@gerardorochin
Copy link
Author

For security reasons, hide your path

@kamermans
Copy link

Since Logstash has now deprecated the multiline filter, I've written something similar using the multiline input codec: https://gist.github.com/kamermans/f53aa58bbd14a3fff9541b76064efa1a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment