Skip to content

Instantly share code, notes, and snippets.

@jamesbjackson

jamesbjackson/Fix redis server startup warnings, redis tuning performance on Ubuntu 16.04.md

Forked from shinebayar-g/Fix redis server startup warnings, redis tuning performance on Ubuntu 16.04.md
Created August 2, 2019 11:09
Show Gist options
  • Save jamesbjackson/b9800f25bb0aa7244ba7ad403d178f1c to your computer and use it in GitHub Desktop.
Save jamesbjackson/b9800f25bb0aa7244ba7ad403d178f1c to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion Fix redis server startup warnings, redis tuning performance on Ubuntu 16.04.md
    View file
    Original file line number Diff line number Diff line change
    @@ -93,4 +93,4 @@ This require you to **reboot**, backup your data or do anything you need before

    Now check you redis log again, you should have a redis server without any errors or warnings.

    ![Analytics](https://ga-beacon.appspot.com/UA-46262276-2/3145e0c6ace8aabb70cf756924c6463a?pixel&useReferer)
    ![Analytics](https://ga-beacon.appspot.com/UA-46262276-2/gist-3145e0c6ace8aabb70cf756924c6463a-redis?pixel&useReferer)
  2. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 1 addition and 3 deletions.
    4 changes: 1 addition & 3 deletions Fix redis server startup warnings, redis tuning performance on Ubuntu 16.04.md
    View file
    Original file line number Diff line number Diff line change
    @@ -93,6 +93,4 @@ This require you to **reboot**, backup your data or do anything you need before

    Now check you redis log again, you should have a redis server without any errors or warnings.

    [![Analytics](https://ga-beacon.appspot.com/UA-46262276-2/redis-tuning)](https://gist.github.com/shinebayar-g/3145e0c6ace8aabb70cf756924c6463a)

    ![Analytics](https://ga-beacon.appspot.com/UA-46262276-2/3145e0c6ace8aabb70cf756924c6463a?pixel)
    ![Analytics](https://ga-beacon.appspot.com/UA-46262276-2/3145e0c6ace8aabb70cf756924c6463a?pixel&useReferer)
  3. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 5 additions and 1 deletion.
    6 changes: 5 additions & 1 deletion Fix redis server startup warnings, redis tuning performance on Ubuntu 16.04.md
    View file
    Original file line number Diff line number Diff line change
    @@ -91,4 +91,8 @@ This require you to **reboot**, backup your data or do anything you need before

    `sudo reboot`

    Now check you redis log again, you should have a redis server without any errors or warnings.
    Now check you redis log again, you should have a redis server without any errors or warnings.

    [![Analytics](https://ga-beacon.appspot.com/UA-46262276-2/redis-tuning)](https://gist.github.com/shinebayar-g/3145e0c6ace8aabb70cf756924c6463a)

    ![Analytics](https://ga-beacon.appspot.com/UA-46262276-2/3145e0c6ace8aabb70cf756924c6463a?pixel)
  4. @shinebayar-g shinebayar-g renamed this gist Jun 29, 2018. 1 changed file with 0 additions and 0 deletions.
    0 redis.md → ...dis tuning performance on Ubuntu 16.04.md
    View file
    File renamed without changes.
  5. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions redis.md
    View file
    Original file line number Diff line number Diff line change
    @@ -75,6 +75,8 @@ Now for these configs to work, you need to reload the config
    WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
    ```
    *Solution:*
    To permanently solve this, follow the log's suggestion, and modify rc.local

    `sudo vi /etc/rc.local`
  6. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 5 additions and 1 deletion.
    6 changes: 5 additions & 1 deletion redis.md
    View file
    Original file line number Diff line number Diff line change
    @@ -36,7 +36,9 @@ and you will see
    Max open files 65536 65536 files
    Max locked memory 65536 65536 bytes
    ```
    ***
    - **Socket Maximum Connection**
    ```Shell
    @@ -65,6 +67,8 @@ Now for these configs to work, you need to reload the config
    `sudo sysctl -p`
    ***
    - **Transparent Huge Pages**
    ```Shell
  7. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 24 additions and 1 deletion.
    25 changes: 24 additions & 1 deletion redis.md
    View file
    Original file line number Diff line number Diff line change
    @@ -62,4 +62,27 @@ net.core.somaxconn=1024
    ```
    Now for these configs to work, you need to reload the config
    `sudo sysctl -p`
    `sudo sysctl -p`
    - **Transparent Huge Pages**
    ```Shell
    WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
    ```
    To permanently solve this, follow the log's suggestion, and modify rc.local

    `sudo vi /etc/rc.local`

    ```Shell
    if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
    echo never > /sys/kernel/mm/transparent_hugepage/enabled
    fi
    ```

    This require you to **reboot**, backup your data or do anything you need before you actually do it!!

    `sudo reboot`

    Now check you redis log again, you should have a redis server without any errors or warnings.
  8. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 6 additions and 5 deletions.
    11 changes: 6 additions & 5 deletions redis.md
    View file
    Original file line number Diff line number Diff line change
    @@ -32,17 +32,18 @@ To check if it works, try to cat proc limits
    `cat /proc/PID/limits`
    and you will see
    `Max open files 65536 65536 files`
    `Max locked memory 65536 65536 bytes`
    ```
    Max open files 65536 65536 files
    Max locked memory 65536 65536 bytes
    ```
    **Socket Maximum Connection**
    - **Socket Maximum Connection**
    ```Shell
    WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
    ```
    **Memory Overcommit**
    - **Memory Overcommit**
    ```Shell
    WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
  9. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 7 additions and 2 deletions.
    9 changes: 7 additions & 2 deletions redis.md
    View file
    Original file line number Diff line number Diff line change
    @@ -24,13 +24,17 @@ LimitNOFILE=65536
    Then you must daemon reload and restart the service
    `sudo systemctl daemon-reload`
    `sudo systemctl restart redis.service`
    To check if it works, try to cat proc limits
    `cat /proc/PID/limits`
    and you will see
    `Max open files 65536 65536 files
    Max locked memory 65536 65536 bytes`
    `Max open files 65536 65536 files`
    `Max locked memory 65536 65536 bytes`
    **Socket Maximum Connection**
    @@ -55,5 +59,6 @@ Since these two are related, we will solve it at once.
    vm.overcommit_memory = 1
    net.core.somaxconn=1024
    ```
    Now for these configs to work, you need to reload the config
    `sudo sysctl -p`
  10. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion redis.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,5 +1,5 @@

    **Maximum Open Files**
    - **Maximum Open Files**


    ```Shell
  11. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 5 additions and 6 deletions.
    11 changes: 5 additions & 6 deletions redis.md
    View file
    Original file line number Diff line number Diff line change
    @@ -32,19 +32,19 @@ and you will see
    `Max open files 65536 65536 files
    Max locked memory 65536 65536 bytes`
    **Socket Maximum Connection
    **Socket Maximum Connection**
    ```Shell
    WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
    ```
    **Memory Overcommit
    **Memory Overcommit**
    ```Shell
    WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
    ```
    **Solution:
    *Solution:*
    Since these two are related, we will solve it at once.
    @@ -55,6 +55,5 @@ Since these two are related, we will solve it at once.
    vm.overcommit_memory = 1
    net.core.somaxconn=1024
    ```
    Now for these configs to work, you need to reload the config
    `sudo sysctl -p`
  12. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions redis.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,5 +1,5 @@

    **Maximum Open Files
    **Maximum Open Files**


    ```Shell
    @@ -8,7 +8,7 @@ Server can't set maximum open files to 10032 because of OS error: Operation not
    Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
    ```
    *Solution:
    *Solution:*
    Edit systemd service file
    `sudo vim /etc/systemd/system/redis.service`
  13. @shinebayar-g shinebayar-g renamed this gist Jun 29, 2018. 1 changed file with 0 additions and 0 deletions.
    0 fix redis startup warnings.md → redis.md
    View file
    File renamed without changes.
  14. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions fix redis startup warnings.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,7 @@

    **Maximum Open Files


    ```Shell
    You requested maxclients of 10000 requiring at least 10032 max file descriptors.
    Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
  15. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions fix redis startup warnings.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,3 +1,4 @@

    **Maximum Open Files

    ```Shell
  16. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 44 additions and 44 deletions.
    88 changes: 44 additions & 44 deletions fix redis startup warnings.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,58 +1,58 @@
    - **Maximum Open Files
    **Maximum Open Files

    ```Shell
    You requested maxclients of 10000 requiring at least 10032 max file descriptors.
    Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
    Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
    ```
    **Solution:
    Edit systemd service file
    `sudo vim /etc/systemd/system/redis.service`
    ```
    [Service]
    ...
    User=redis
    Group=redis
    # should be fine as long as you add it under [Service] block
    LimitNOFILE=65536
    ```
    Then you must daemon reload and restart the service
    ```Shell
    You requested maxclients of 10000 requiring at least 10032 max file descriptors.
    Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
    Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
    ```
    *Solution:
    Edit systemd service file
    `sudo vim /etc/systemd/system/redis.service`
    ```
    [Service]
    ...
    User=redis
    Group=redis
    # should be fine as long as you add it under [Service] block
    LimitNOFILE=65536
    ```
    Then you must daemon reload and restart the service
    `sudo systemctl daemon-reload`
    `sudo systemctl restart redis.service`
    `sudo systemctl daemon-reload`
    `sudo systemctl restart redis.service`
    To check if it works, try to cat proc limits
    `cat /proc/PID/limits`
    and you will see
    `Max open files 65536 65536 files
    Max locked memory 65536 65536 bytes`
    To check if it works, try to cat proc limits
    `cat /proc/PID/limits`
    and you will see
    `Max open files 65536 65536 files
    Max locked memory 65536 65536 bytes`
    - **Socket Maximum Connection
    **Socket Maximum Connection
    ```Shell
    WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
    ```
    ```Shell
    WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
    ```
    **Memory Overcommit
    **Memory Overcommit
    ```Shell
    WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
    ```
    ```Shell
    WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
    ```
    **Solution:
    **Solution:
    Since these two are related, we will solve it at once.
    Since these two are related, we will solve it at once.
    `sudo vim /etc/sysctl.conf`
    `sudo vim /etc/sysctl.conf`
    ```
    # Add at the bottom of file
    vm.overcommit_memory = 1
    net.core.somaxconn=1024
    ```
    ```Shell
    # Add at the bottom of file
    vm.overcommit_memory = 1
    net.core.somaxconn=1024
    ```
  17. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 40 additions and 7 deletions.
    47 changes: 40 additions & 7 deletions fix redis startup warnings.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    - Warning #1
    - **Maximum Open Files

    ```Shell
    You requested maxclients of 10000 requiring at least 10032 max file descriptors.
    @@ -7,19 +7,52 @@
    ```
    **Solution:
    Edit systemd service file
    `sudo vim /etc/systemd/system/redis.service`
    ```
    [Service]
    ...
    User=redis
    Group=redis
    #*should be fine as long as you add it under [Service] block
    # should be fine as long as you add it under [Service] block
    LimitNOFILE=65536
    ```
    sudo systemctl daemon-reload
    sudo systemctl restart redis.service
    Then you must daemon reload and restart the service
    `sudo systemctl daemon-reload`
    `sudo systemctl restart redis.service`
    Then you must daemon reload and restart the service
    To check if it works, try to cat proc limits
    `cat /proc/PID/limits`
    and you will see
    `Max open files 65536 65536 files
    Max locked memory 65536 65536 bytes`
    - **Socket Maximum Connection
    ```Shell
    WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
    ```
    **Memory Overcommit
    ```Shell
    WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
    ```
    **Solution:
    Since these two are related, we will solve it at once.
    `sudo vim /etc/sysctl.conf`
    ```
    # Add at the bottom of file
    vm.overcommit_memory = 1
    net.core.somaxconn=1024
    ```
  18. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 21 additions and 18 deletions.
    39 changes: 21 additions & 18 deletions fix redis startup warnings.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,22 +1,25 @@
    - Warning #1
    - Warning #1

    ```Shell
    You requested maxclients of 10000 requiring at least 10032 max file descriptors.
    Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
    Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
    ```
    ```Shell
    You requested maxclients of 10000 requiring at least 10032 max file descriptors.
    Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
    Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
    ```
    **Solution:
    **Solution:
    sudo vim /etc/systemd/system/redis.service
    [Service]
    ...
    User=redis
    Group=redis
    #*should be fine as long as you add it under [Service] block
    LimitNOFILE=65536
    `sudo vim /etc/systemd/system/redis.service`
    ```
    [Service]
    ...
    User=redis
    Group=redis
    #*should be fine as long as you add it under [Service] block
    LimitNOFILE=65536
    ```
    sudo systemctl daemon-reload
    sudo systemctl restart redis.service
    sudo systemctl daemon-reload
    sudo systemctl restart redis.service
    Then you must daemon reload and restart the service
    Then you must daemon reload and restart the service
  19. @shinebayar-g shinebayar-g revised this gist Jun 29, 2018. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions fix redis startup warnings.md
    View file
    Original file line number Diff line number Diff line change
    @@ -6,14 +6,14 @@
    Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
    ```
    *Solution:*
    **Solution:
    sudo vim /etc/systemd/system/redis.service
    [Service]
    ...
    User=redis
    Group=redis
    # should be fine as long as you add it under [Service] block
    #*should be fine as long as you add it under [Service] block
    LimitNOFILE=65536
    sudo systemctl daemon-reload
  20. @shinebayar-g shinebayar-g renamed this gist Jun 29, 2018. 1 changed file with 0 additions and 0 deletions.
    0 fix redis startup warnings → fix redis startup warnings.md
    View file
    File renamed without changes.
  21. @shinebayar-g shinebayar-g created this gist Jun 29, 2018.
    22 changes: 22 additions & 0 deletions fix redis startup warnings
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,22 @@
    - Warning #1

    ```Shell
    You requested maxclients of 10000 requiring at least 10032 max file descriptors.
    Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
    Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
    ```

    *Solution:*

    sudo vim /etc/systemd/system/redis.service
    [Service]
    ...
    User=redis
    Group=redis
    # should be fine as long as you add it under [Service] block
    LimitNOFILE=65536

    sudo systemctl daemon-reload
    sudo systemctl restart redis.service

    Then you must daemon reload and restart the service