vikumar-ciena

RSyslog Forwarding Setup

Overview

I'm looking to centralize logging for our dev team into Elasticsearch via Logstash. The wrinkle is that we aren't a Java shop, so installing java on our hosts just to ship logs back to a central Logstash indexer is something we'd like to avoid. So, I'm approaching things as a chance to understand RSyslog and its capabilities as a log shipper.

Procedure

Set up TCP listening on the log index host

Uncomment the following lines in /etc/rsyslog.conf. This will enable the rsyslog daemon to listen for incoming requests on TCP port 514. We're using TCP here so that we can have some confidence that the messages from the agent hosts reach the indexer. (More on this below)

#Use pprof debug docker daemon

Use pprof debug docker daemon
pprof debug entrypoint
Start docker daemon in debug mode
Run socat to make docker sock available via tcp port
Access debug url entrypoint

USE AT OWN RISK

This was only tested on a partial SentinelOne installation on the High Sierra beta, where SentinelOne was never allowed to enable it's kernel extension.

	#!/usr/bin/env bash
	# postfix-wrapper.sh, version 0.1.0
	#
	# You cannot start postfix in some foreground mode and
	# it's more or less important that docker doesn't kill
	# postfix and its chilren if you stop the container.
	#
	# Use this script with supervisord and it will take
	# care about starting and stopping postfix correctly.
	#

	HOME = .
	RANDFILE = .rnd

	[ ca ]
	default_ca = CA_default # The default ca section

	[ CA_default ]
	dir = . # Where everything is kept
	certs = $dir/cert # Where the issued certs are kept
	crl_dir = $dir # Where the issued crl are kept

	package main

	import (
	"fmt"
	)

	type Node struct {
	Value int
	}

	package com.devadmin.utils.dropwizard;

	import ch.qos.logback.classic.LoggerContext;
	import ch.qos.logback.classic.util.ContextInitializer;
	import ch.qos.logback.core.joran.spi.JoranException;
	import com.codahale.metrics.MetricRegistry;
	import com.fasterxml.jackson.annotation.JsonIgnore;
	import io.dropwizard.logging.LoggingFactory;
	import io.dropwizard.logging.LoggingUtil;

	#!/usr/bin/env bash

	function __tc_encode {
	# Only unicode characters are not supported
	echo -n "$1" \| sed "s/\([\|']\)/\\|\1/g; s/\[/\\|\[/g; s/\]/\\|\]/g; s/\r/\\|r/g;" \| sed ':a;N;$!ba;s/\n/\|n/g'
	}
	function __tc_message {
	echo "##teamcity[message text='$(__tc_encode "$2")' status='${1:-NORMAL}']"
	}
	function __tc_simple {

	ssh-keygen -t rsa -b 4096 -N '' -C "[email protected]" -f ~/.ssh/id_rsa
	ssh-keygen -t rsa -b 4096 -N '' -C "[email protected]" -f ~/.ssh/github_rsa
	ssh-keygen -t rsa -b 4096 -N '' -C "[email protected]" -f ~/.ssh/mozilla_rsa

	import java.io.File;
	import java.util.*;
	import java.lang.management.BufferPoolMXBean;
	import java.lang.management.ManagementFactory;
	import javax.management.MBeanServerConnection;
	import javax.management.ObjectName;
	import javax.management.remote.*;

	import com.sun.tools.attach.VirtualMachine; // Attach API