Splunk User Resource Limits Setup

References:

1. https://docs.splunk.com/Documentation/Splunk/9.4.0/Admin/Limitsconf

2. https://www.splunk.com/en_us/blog/tips-and-tricks/whats-your-ulimit.html

3. https://docs.splunk.com/Documentation/Forwarder/9.4.0/Forwarder/Installanixuniversalforwarder#Install_the_universal_forwarder_on_Linux

1. Create Splunk User

useradd -m -s /bin/bash splunk

2. Configure PAM Limits

# Ref: https://linux.die.net/man/5/limits.conf
# Controls user limits in /etc/security/limits.conf
# Note: Provides basic resource control but has limitations
sudo tee -a /etc/security/limits.conf << 'EOF'
splunk soft nproc 4096
splunk hard nproc 8192
splunk soft nofile 10240
splunk hard nofile 20240
splunk soft memlock 64000
splunk hard memlock 64000
splunk soft as 16G
splunk hard as 16G
EOF

3. Configure Systemd Slice

# Ref: https://linux.die.net/man/5/systemd.slice
# Systemd slice provides more precise control:
# - CPU quota control (percentage based)
# - Real-time CPU limiting
# - Modern cgroups integration
# Note: 100% = 1 CPU core, 200% = 2 CPU cores
sudo tee /etc/systemd/system/splunk.slice << 'EOF'
[Unit]
Description=Slice for Splunk User
Before=slices.target

[Slice]
CPUQuota=200%
MemoryMax=16G
TasksMax=8192
EOF

4. Apply User Service Limits

sudo tee /etc/systemd/system/[email protected]/override.conf << 'EOF'
[Service]
Slice=splunk.slice
EOF

5. System Reload

systemctl daemon-reload