Skip to content

Instantly share code, notes, and snippets.

@rjmanlawe

rjmanlawe/fluentd_centos.md

Forked from zzz0123/fluentd_centos.md
Created August 3, 2020 15:11
Show Gist options
  • Save rjmanlawe/03608b9fc9595d75c90cabe469c6eb28 to your computer and use it in GitHub Desktop.
Save rjmanlawe/03608b9fc9595d75c90cabe469c6eb28 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @zzz0123 zzz0123 revised this gist Jul 21, 2014. No changes.
  2. @zzz0123 zzz0123 revised this gist Jul 21, 2014. No changes.
  3. @zzz0123 zzz0123 revised this gist Jul 21, 2014. 1 changed file with 8 additions and 4 deletions.
    12 changes: 8 additions & 4 deletions fluentd_centos.md
    View file
    Original file line number Diff line number Diff line change
    @@ -6,13 +6,14 @@
    ## install
    * http://help.treasuredata.com/customer/portal/articles/1246904-installing-td-agent-on-rhel-and-centos

    ## setup
    ## edit td-agent.conf
    ### server#1
    ```
    $ sudo vi /etc/td-agent/td-agent.conf
    <source>
    type tail
    format nginx
    format ltsv
    time_key time_local # formatがltsvのときに必要
    path /var/log/nginx/access.log
    tag nginx.access
    pos_file /var/log/td-agent/nginx_access.pos
    @@ -50,10 +51,13 @@ $ sudo service td-agent stop
    $ sudo service td-agent start
    ```

    ## setup iptables
    ## edit iptables
    ```
    $ sudo cat /etc/sysconfig/iptables
    + -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 24224 -j ACCEPT
    + -A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 24224 -j ACCEPT
    $ sudo service iptables restart
    ```
    ```

    ## reference
    * http://blog.excale.net/index.php/archives/2682
  4. @zzz0123 zzz0123 revised this gist Jul 21, 2014. 1 changed file with 8 additions and 7 deletions.
    15 changes: 8 additions & 7 deletions fluentd_centos.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,12 +1,13 @@
    # about
    * source: nginx_access.log
    * dist: mongodb
    # setup td-agent
    ## about
    * source: server#1 nginx_access.log
    * dist: server#2 mongodb

    # install
    ## install
    * http://help.treasuredata.com/customer/portal/articles/1246904-installing-td-agent-on-rhel-and-centos

    # setup
    ## source
    ## setup
    ### server#1
    ```
    $ sudo vi /etc/td-agent/td-agent.conf
    <source>
    @@ -29,7 +30,7 @@ $ sudo vi /etc/td-agent/td-agent.conf
    </server>
    </match>
    ```
    ## dist
    ### server#2
    ```
    <source>
    type forward
  5. @zzz0123 zzz0123 renamed this gist Jul 21, 2014. 1 changed file with 0 additions and 0 deletions.
    0 gistfile1.txt → fluentd_centos.md
    View file
    File renamed without changes.
  6. @zzz0123 zzz0123 created this gist Jul 21, 2014.
    58 changes: 58 additions & 0 deletions gistfile1.txt
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,58 @@
    # about
    * source: nginx_access.log
    * dist: mongodb

    # install
    * http://help.treasuredata.com/customer/portal/articles/1246904-installing-td-agent-on-rhel-and-centos

    # setup
    ## source
    ```
    $ sudo vi /etc/td-agent/td-agent.conf
    <source>
    type tail
    format nginx
    path /var/log/nginx/access.log
    tag nginx.access
    pos_file /var/log/td-agent/nginx_access.pos
    </source>

    <match nginx.*>
    type forward
    buffer_type memory
    buffer_chunk_limit 256m
    buffer_queue_limit 128
    flush_interval 5s
    <server>
    host $host
    port $port
    </server>
    </match>
    ```
    ## dist
    ```
    <source>
    type forward
    port $port
    </source>
    <match nginx.access>
    type mongo
    database $db_name
    collection $collection_name
    </match>
    ```

    ## restart td-agent
    ```
    // pidが残る or 2重起動するためstop, startで反映
    $ sudo service td-agent stop
    $ sudo service td-agent start
    ```

    ## setup iptables
    ```
    $ sudo cat /etc/sysconfig/iptables
    + -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 24224 -j ACCEPT
    + -A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 24224 -j ACCEPT
    $ sudo service iptables restart
    ```