2016/03/31 11:56:20 Diagnostics: 30 packs have been idle more than 120 seconds. 2016/03/31 11:56:20 Diagnostics: (inject) Plugin names and quantities found on idle packs: 2016/03/31 11:56:20 Diagnostics: influxdb_accumulator_filter: 30 2016/03/31 11:56:20 2016/03/31 11:56:20 Diagnostics: 32 packs have been idle more than 120 seconds. 2016/03/31 11:56:20 Diagnostics: (input) Plugin names and quantities found on idle packs: 2016/03/31 11:56:20 Diagnostics: elasticsearch_output: 32 2016/03/31 11:56:20 Diagnostics: http_metrics_filter: 32 2016/03/31 11:56:20 2016/03/31 11:56:23 Queue report initiated. 2016/03/31 11:56:23 ========[heka.all-report]======== ====Globals==== inputRecycleChan: InChanCapacity: 100 InChanLength: 0 injectRecycleChan: InChanCapacity: 100 InChanLength: 0 Router: InChanCapacity: 30 InChanLength: 30 ProcessMessageCount: 34157 ====Inputs==== collectd_httplisten: mysql_logstreamer: ovs_logstreamer: openstack_error_amqp: aggregator_tcpinput: openstack_7_0_logstreamer: pacemaker_logstreamer: rabbitmq_logstreamer: system_logstreamer: openstack_info_amqp: openstack_warn_amqp: http-check_httplisten: keystone_7_0_logstreamer: keystone_wsgi_logstreamer: openstack_dashboard_logstreamer: swift_logstreamer: ====Splitters==== keystone_7_0_logstreamer-openstack_splitter-1: rabbitmq_logstreamer-rabbitmq_splitter-1: swift_logstreamer-TokenSplitter-1: keystone_wsgi_logstreamer-openstack_splitter-1: ovs_logstreamer-TokenSplitter-1: pacemaker_logstreamer-TokenSplitter-1: rabbitmq_logstreamer-rabbitmq_splitter-2: system_logstreamer-TokenSplitter-1: ovs_logstreamer-TokenSplitter-2: openstack_7_0_logstreamer-openstack_splitter-1: openstack_error_amqp-NullSplitter: openstack_info_amqp-NullSplitter: openstack_warn_amqp-NullSplitter: system_logstreamer-TokenSplitter-2: openstack_7_0_logstreamer-openstack_splitter-2: system_logstreamer-TokenSplitter-3: openstack_7_0_logstreamer-openstack_splitter-3: system_logstreamer-TokenSplitter-4: openstack_7_0_logstreamer-openstack_splitter-4: system_logstreamer-TokenSplitter-5: openstack_7_0_logstreamer-openstack_splitter-5: system_logstreamer-TokenSplitter-6: system_logstreamer-TokenSplitter-7: collectd_httplisten-NullSplitter-127.0.0.1:49578: ====Decoders==== keystone_7_0_logstreamer-keystone_7_0_decoder-1: InChanCapacity: 30 InChanLength: 30 ProcessMessageCount: 15919 Memory: 477116 MaxMemory: 581511 MaxInstructions: 2341 MaxOutput: 747 ProcessMessageAvgDuration: 461828 rabbitmq_logstreamer-rabbitmq_decoder-1: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 0 Memory: 129446 MaxMemory: 179984 MaxInstructions: 1271 MaxOutput: 0 ProcessMessageAvgDuration: 0 keystone_wsgi_logstreamer-keystone_wsgi_decoder-1: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 99 Memory: 488300 MaxMemory: 557289 MaxInstructions: 2214 MaxOutput: 781 ProcessMessageAvgDuration: 861615 rabbitmq_logstreamer-rabbitmq_decoder-2: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 2 Memory: 133733 MaxMemory: 179984 MaxInstructions: 1271 MaxOutput: 312 ProcessMessageAvgDuration: 177754 ovs_logstreamer-ovs_decoder-1: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 0 Memory: 330239 MaxMemory: 411797 MaxInstructions: 1875 MaxOutput: 0 ProcessMessageAvgDuration: 0 swift_logstreamer-swift_decoder-1: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 17 Memory: 476408 MaxMemory: 593525 MaxInstructions: 2478 MaxOutput: 374 ProcessMessageAvgDuration: 2.4724358e+07 system_logstreamer-system_decoder-1: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 2 Memory: 442584 MaxMemory: 593525 MaxInstructions: 2478 MaxOutput: 340 ProcessMessageAvgDuration: 4.802986e+06 openstack_7_0_logstreamer-openstack_decoder-1: InChanCapacity: 30 InChanLength: 5 ProcessMessageCount: 91 Memory: 420709 MaxMemory: 591444 MaxInstructions: 2382 MaxOutput: 795 ProcessMessageAvgDuration: 215947 pacemaker_logstreamer-pacemaker_decoder-1: InChanCapacity: 30 InChanLength: 1 ProcessMessageCount: 242 Memory: 290196 MaxMemory: 569573 MaxInstructions: 2255 MaxOutput: 1894 ProcessMessageAvgDuration: 652347 ovs_logstreamer-ovs_decoder-2: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 0 Memory: 330239 MaxMemory: 411797 MaxInstructions: 1875 MaxOutput: 0 ProcessMessageAvgDuration: 0 system_logstreamer-system_decoder-2: InChanCapacity: 30 InChanLength: 4 ProcessMessageCount: 124 Memory: 424829 MaxMemory: 593525 MaxInstructions: 2478 MaxOutput: 346 ProcessMessageAvgDuration: 438067 openstack_7_0_logstreamer-openstack_decoder-2: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 48 Memory: 519006 MaxMemory: 584263 MaxInstructions: 2382 MaxOutput: 2084 ProcessMessageAvgDuration: 290406 system_logstreamer-system_decoder-3: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 1 Memory: 440715 MaxMemory: 593525 MaxInstructions: 2478 MaxOutput: 444 ProcessMessageAvgDuration: 1.661955e+06 openstack_7_0_logstreamer-openstack_decoder-3: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 46 Memory: 482411 MaxMemory: 584263 MaxInstructions: 2382 MaxOutput: 1223 ProcessMessageAvgDuration: 3.9395568e+07 system_logstreamer-system_decoder-4: InChanCapacity: 30 InChanLength: 1 ProcessMessageCount: 42 Memory: 536817 MaxMemory: 593525 MaxInstructions: 2478 MaxOutput: 588 ProcessMessageAvgDuration: 479831 openstack_7_0_logstreamer-openstack_decoder-4: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 23 Memory: 448524 MaxMemory: 584263 MaxInstructions: 2382 MaxOutput: 842 ProcessMessageAvgDuration: 5.612098e+06 system_logstreamer-system_decoder-5: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 76 Memory: 350673 MaxMemory: 593525 MaxInstructions: 2478 MaxOutput: 821 ProcessMessageAvgDuration: 416208 openstack_7_0_logstreamer-openstack_decoder-5: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 83 Memory: 329897 MaxMemory: 591504 MaxInstructions: 2382 MaxOutput: 571 ProcessMessageAvgDuration: 265289 system_logstreamer-system_decoder-6: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 0 Memory: 425080 MaxMemory: 593525 MaxInstructions: 2478 MaxOutput: 0 ProcessMessageAvgDuration: 0 system_logstreamer-system_decoder-7: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 74 Memory: 318635 MaxMemory: 593525 MaxInstructions: 2478 MaxOutput: 465 ProcessMessageAvgDuration: 2.238353e+06 collectd_httplisten-collectd_decoder: InChanCapacity: 30 InChanLength: 0 ProcessMessageCount: 85 Memory: 273448 MaxMemory: 336625 MaxInstructions: 3876 MaxOutput: 664 ProcessMessageAvgDuration: 9.62174e+06 ====Filters==== afd_api_endpoints_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 2318 ProcessMessageCount: 0 InjectMessageCount: 0 Memory: 176693 MaxMemory: 219223 MaxInstructions: 1280 MaxOutput: 0 ProcessMessageAvgDuration: 0 TimerEventAvgDuration: 0 afd_service_apache_worker_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 740 ProcessMessageCount: 1 InjectMessageCount: 0 Memory: 166232 MaxMemory: 274569 MaxInstructions: 1687 MaxOutput: 570 ProcessMessageAvgDuration: 118394 TimerEventAvgDuration: 167695 gse_global_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 9208 ProcessMessageCount: 1 InjectMessageCount: 0 Memory: 176244 MaxMemory: 272572 MaxInstructions: 9739 MaxOutput: 0 ProcessMessageAvgDuration: 11781 TimerEventAvgDuration: 249126 instance_state_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 761 ProcessMessageCount: 0 InjectMessageCount: 0 Memory: 122692 MaxMemory: 193772 MaxInstructions: 1199 MaxOutput: 0 ProcessMessageAvgDuration: 0 TimerEventAvgDuration: 0 afd_node_controller_cpu_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 728 ProcessMessageCount: 4 InjectMessageCount: 0 Memory: 181182 MaxMemory: 275522 MaxInstructions: 1994 MaxOutput: 1217 ProcessMessageAvgDuration: 128632 TimerEventAvgDuration: 199067 afd_service_nova-api_http_errors_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 90659 ProcessMessageCount: 32 InjectMessageCount: 0 Memory: 165598 MaxMemory: 273621 MaxInstructions: 1690 MaxOutput: 623 ProcessMessageAvgDuration: 91455 TimerEventAvgDuration: 242376 heka_monitoring_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 467 ProcessMessageCount: 0 InjectMessageCount: 0 Memory: 179112 MaxMemory: 219026 MaxInstructions: 1206 MaxOutput: 0 ProcessMessageAvgDuration: 0 TimerEventAvgDuration: 0 influxdb_annotation_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 453 ProcessMessageCount: 0 InjectMessageCount: 0 Memory: 129449 MaxMemory: 219819 MaxInstructions: 1296 MaxOutput: 0 ProcessMessageAvgDuration: 0 TimerEventAvgDuration: 0 afd_service_neutron-api_http_errors_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 653 ProcessMessageCount: 32 InjectMessageCount: 0 Memory: 165611 MaxMemory: 273407 MaxInstructions: 1690 MaxOutput: 629 ProcessMessageAvgDuration: 122188 TimerEventAvgDuration: 1.730834e+06 afd_service_swift-api_http_errors_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 783 ProcessMessageCount: 32 InjectMessageCount: 0 Memory: 165599 MaxMemory: 273397 MaxInstructions: 1690 MaxOutput: 625 ProcessMessageAvgDuration: 5.351447e+07 TimerEventAvgDuration: 702093 afd_workers_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 1283 ProcessMessageCount: 48 InjectMessageCount: 11 Memory: 197730 MaxMemory: 221230 MaxInstructions: 1281 MaxOutput: 380 ProcessMessageAvgDuration: 46636 TimerEventAvgDuration: 0 influxdb_accumulator_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 2099 ProcessMessageCount: 17110 InjectMessageCount: 169 Memory: 143023 MaxMemory: 395635 MaxInstructions: 1320 MaxOutput: 13849 ProcessMessageAvgDuration: 1.242292e+06 TimerEventAvgDuration: 5.365546e+06 afd_api_backends_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 743 ProcessMessageCount: 64 InjectMessageCount: 32 Memory: 221146 MaxMemory: 221402 MaxInstructions: 1281 MaxOutput: 387 ProcessMessageAvgDuration: 13278 TimerEventAvgDuration: 0 afd_service_glance-api_http_errors_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 1240 ProcessMessageCount: 32 InjectMessageCount: 0 Memory: 165605 MaxMemory: 273402 MaxInstructions: 1690 MaxOutput: 627 ProcessMessageAvgDuration: 8.027391e+06 TimerEventAvgDuration: 641181 afd_service_keystone-public-api_http_errors_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 887 ProcessMessageCount: 32 InjectMessageCount: 0 Memory: 165659 MaxMemory: 273447 MaxInstructions: 1690 MaxOutput: 645 ProcessMessageAvgDuration: 159202 TimerEventAvgDuration: 459675 afd_service_rabbitmq_disk_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 1120 ProcessMessageCount: 1 InjectMessageCount: 0 Memory: 171732 MaxMemory: 274503 MaxInstructions: 1839 MaxOutput: 813 ProcessMessageAvgDuration: 101983 TimerEventAvgDuration: 2.612459e+06 gse_node_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 1483 ProcessMessageCount: 1 InjectMessageCount: 0 Memory: 157270 MaxMemory: 258878 MaxInstructions: 3481 MaxOutput: 0 ProcessMessageAvgDuration: 27381 TimerEventAvgDuration: 315577 service_heartbeat_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 2343 ProcessMessageCount: 1157 InjectMessageCount: 2 Memory: 121237 MaxMemory: 210095 MaxInstructions: 1293 MaxOutput: 365 ProcessMessageAvgDuration: 22895 TimerEventAvgDuration: 364950 afd_node_mysql-nodes_mysql-fs_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 735 ProcessMessageCount: 5 InjectMessageCount: 0 Memory: 172511 MaxMemory: 274554 MaxInstructions: 1840 MaxOutput: 886 ProcessMessageAvgDuration: 65544 TimerEventAvgDuration: 394385 afd_service_heat-api_http_errors_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 663 ProcessMessageCount: 32 InjectMessageCount: 0 Memory: 165598 MaxMemory: 273621 MaxInstructions: 1690 MaxOutput: 623 ProcessMessageAvgDuration: 1.603797e+06 TimerEventAvgDuration: 459755 afd_service_keystone-admin-api_http_errors_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 712 ProcessMessageCount: 32 InjectMessageCount: 0 Memory: 165653 MaxMemory: 273442 MaxInstructions: 1690 MaxOutput: 643 ProcessMessageAvgDuration: 118282 TimerEventAvgDuration: 177474 afd_service_mysql_node-status_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 1526 ProcessMessageCount: 2 InjectMessageCount: 0 Memory: 171920 MaxMemory: 274516 MaxInstructions: 1843 MaxOutput: 805 ProcessMessageAvgDuration: 197406 TimerEventAvgDuration: 649836 afd_service_rabbitmq_memory_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 717 ProcessMessageCount: 1 InjectMessageCount: 0 Memory: 171742 MaxMemory: 274509 MaxInstructions: 1839 MaxOutput: 819 ProcessMessageAvgDuration: 132631 TimerEventAvgDuration: 177091 afd_service_rabbitmq_queue_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 708 ProcessMessageCount: 1 InjectMessageCount: 0 Memory: 166592 MaxMemory: 273277 MaxInstructions: 1687 MaxOutput: 573 ProcessMessageAvgDuration: 72504 TimerEventAvgDuration: 839358 resource_creation_time_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 606 ProcessMessageCount: 0 InjectMessageCount: 0 Memory: 112924 MaxMemory: 174275 MaxInstructions: 1206 MaxOutput: 0 ProcessMessageAvgDuration: 0 TimerEventAvgDuration: 0 afd_service_cinder-api_http_errors_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 721 ProcessMessageCount: 32 InjectMessageCount: 0 Memory: 165605 MaxMemory: 273631 MaxInstructions: 1690 MaxOutput: 627 ProcessMessageAvgDuration: 135598 TimerEventAvgDuration: 186075 gse_service_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 1011 ProcessMessageCount: 1 InjectMessageCount: 0 Memory: 196321 MaxMemory: 293907 MaxInstructions: 27940 MaxOutput: 0 ProcessMessageAvgDuration: 12659 TimerEventAvgDuration: 325466 afd_node_controller_log-fs_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 845 ProcessMessageCount: 5 InjectMessageCount: 0 Memory: 172433 MaxMemory: 274408 MaxInstructions: 1840 MaxOutput: 869 ProcessMessageAvgDuration: 47964 TimerEventAvgDuration: 225126 afd_node_controller_root-fs_filter: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 2765 ProcessMessageCount: 5 InjectMessageCount: 0 Memory: 172384 MaxMemory: 274581 MaxInstructions: 1840 MaxOutput: 854 ProcessMessageAvgDuration: 92554 TimerEventAvgDuration: 177984 http_metrics_filter: InChanCapacity: 30 InChanLength: 30 MatchChanCapacity: 30 MatchChanLength: 30 MatchAvgDuration: 843 ProcessMessageCount: 15702 InjectMessageCount: 15701 Memory: 144442 MaxMemory: 238871 MaxInstructions: 1201 MaxOutput: 491 ProcessMessageAvgDuration: 114036 TimerEventAvgDuration: 0 ====Outputs==== influxdb_output: InChanCapacity: 2 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 1509 nagios_afd_nodes_output: InChanCapacity: 2 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 683 nagios_gse_global_clusters_output: InChanCapacity: 2 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 443 nagios_gse_node_clusters_output: InChanCapacity: 2 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 344 aggregator_tcpoutput: InChanCapacity: 2 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 6381 ProcessMessageCount: 45 DashboardOutput: InChanCapacity: 30 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 679 elasticsearch_output: InChanCapacity: 2 InChanLength: 0 MatchChanCapacity: 30 MatchChanLength: 0 MatchAvgDuration: 754 ====Encoders==== nagios_gse_node_clusters_output-nagios_gse_node_clusters_encoder: ProcessMessageCount: 0 Memory: 181631 MaxMemory: 227465 MaxInstructions: 1346 MaxOutput: 0 ProcessMessageAvgDuration: 0 aggregator_tcpoutput-ProtobufEncoder: ProcessMessageCount: 49 ProcessMessageAvgDuration: 2150 elasticsearch_output-elasticsearch_encoder: influxdb_output-influxdb_encoder: nagios_afd_nodes_output-nagios_afd_nodes_encoder: ProcessMessageCount: 0 Memory: 181623 MaxMemory: 227457 MaxInstructions: 1346 MaxOutput: 0 ProcessMessageAvgDuration: 0 nagios_gse_global_clusters_output-nagios_gse_global_clusters_encoder: ProcessMessageCount: 0 Memory: 181633 MaxMemory: 227467 MaxInstructions: 1346 MaxOutput: 0 ProcessMessageAvgDuration: 0 ========