# 🛠️ How-To: Granular System Monitoring with sysstat on AlmaLinux 9

## 🎯 Objective

Deploy `sysstat` to collect system performance data every 60 seconds, enabling real-time visibility and historical analysis.

---

## 1. 🔧 Install `sysstat`

Install the `sysstat` package (includes `sar`, `sadc`, etc.):

```bash
sudo dnf install -y sysstat
```

---

## 2. ⏱️ Configure 60-Second Data Collection

Modify cron to collect every minute instead of 10:

```bash
sudo sed -i 's|^\*/10 \* \* \* \* root /usr/lib64/sa/sa1 1 1|* * * * * root /usr/lib64/sa/sa1 1 1|' /etc/cron.d/sysstat
```

> This ensures per-minute data capture.

---

## 3. ▶️ Enable and Start sysstat

Enable and start the sysstat service:

```bash
sudo systemctl enable --now sysstat
```

---

## 4. ⚙️ Customize Advanced Settings

Edit `/etc/sysconfig/sysstat` for retention and metric control:

```bash
sudo vi /etc/sysconfig/sysstat
```

Suggested changes:

```ini
HISTORY=90             # Retain 90 days of logs
COMPRESSAFTER=31       # Compress old logs after 31 days
SADC_OPTIONS=" -S ALL" # Collect all system stats (CPU, disk, memory, etc.)
```

---

## 5. ✅ Verify Data Collection

Check log presence in `/var/log/sa/`:

```bash
ls -lh /var/log/sa/
```

Use `sar` to review collected data:

```bash
sar -u        # CPU
sar -r        # Memory
sar -b        # Disk I/O
sar -n DEV    # Network
```

View specific day's data:

```bash
sar -u -f /var/log/sa/sa24
```

---

## 6. 🤖 Automate with Ansible

### `deploy_sysstat.yml`

```yaml
---
- name: Configure sysstat for 1-minute monitoring on AlmaLinux 9
  hosts: almalinux_nodes
  become: true

  tasks:
    - name: Ensure sysstat package is installed
      ansible.builtin.package:
        name: sysstat
        state: present

    - name: Set sysstat data collection interval to 60 seconds
      ansible.builtin.lineinfile:
        path: /etc/cron.d/sysstat
        regexp: '^\*/10 \* \* \* \* root /usr/lib64/sa/sa1 1 1'
        line: '* * * * * root /usr/lib64/sa/sa1 1 1'
        backup: yes

    - name: Ensure sysstat service is enabled and started
      ansible.builtin.service:
        name: sysstat
        state: started
        enabled: true

    - name: Set comprehensive data collection
      ansible.builtin.lineinfile:
        path: /etc/sysconfig/sysstat
        regexp: '^SADC_OPTIONS='
        line: 'SADC_OPTIONS=" -S ALL"'
        backup: yes

    - name: Set data retention to 90 days
      ansible.builtin.lineinfile:
        path: /etc/sysconfig/sysstat
        regexp: '^HISTORY='
        line: 'HISTORY=90'
        backup: yes
```

### Execute:

```bash
ansible-playbook deploy_sysstat.yml -l <your_host_or_group>
```

---

## 7. 📊 Analyze Metrics with `sar`

### 🖥️ CPU Utilization

```bash
sar -u        # Daily summary
sar -u 1 5    # Real-time stats (5 samples every 1s)
```

- `%user`: App usage
- `%iowait`: Disk bottlenecks
- `%steal`: VM CPU contention

---

### 🧠 Memory Utilization

```bash
sar -r
```

- `kbavail`, `%memused`, `%swpused` indicate memory pressure

---

### 💽 Disk I/O

```bash
sar -b        # Global I/O stats
sar -d        # Per-device I/O
```

- Look at `tps`, `await`, `%util` for bottlenecks

---

### 🌐 Network Stats

```bash
sar -n DEV     # Network traffic
sar -n EDEV    # Network errors
```

- High `rxerr/s`, `txerr/s`, `coll/s` → NIC or network problem

---

## 🧠 Pro Tips

- Use `-S DISK`, `-S XALL`, or `-S ALL` depending on server role
- Integrate with Grafana or SIEM for full-stack visibility

---

## 👤 Author

**Prepared by:** Harisfazillah Jamel  
🗓️ 24 July 2025

---