Intrusion_detection

Chapter 34: Intrusion Detection and Fail2Ban

Comprehensive Guide to Linux Intrusion Detection

---

33.1 Understanding Intrusion Detection

Types of IDS/IPS

                 Intrusion Detection Systems
+------------------------------------------------------------------+
|                                                                   |
|    Network-based IDS (NIDS):                                       |
|    +----------------------------------------------------------+   |
|    | • Monitors network traffic                                |   |
|    | • Examples: Snort, Suricata                             |   |
|    | • Passive monitoring                                    |   |
|    +----------------------------------------------------------+   |
|                                                                   |
|    Host-based IDS (HIDS):                                         |
|    +----------------------------------------------------------+   |
|    | • Monitors system activity                               |   |
|    | • Examples: OSSEC, AIDE                                 |   |
|    | • File integrity, process monitoring                    |   |
|    +----------------------------------------------------------+   |
|                                                                   |
|    Intrusion Prevention System (IPS):                              |
|    +----------------------------------------------------------+   |
|    | • Active blocking of attacks                            |   |
|    | • Examples: Fail2Ban, Snort-inline                     |   |
|    +----------------------------------------------------------+   |
|                                                                   |
|    Detection Methods:                                              |
|    +----------------------------------------------------------+   |
|    | Signature-based | Known attack patterns                   |   |
|    | Anomaly-based   | Deviations from normal                |   |
|    | Stateful        | Protocol state analysis               |   |
|    +----------------------------------------------------------+   |
|                                                                   |
+------------------------------------------------------------------+

---

33.2 Fail2Ban Configuration

Installation and Setup

# Install
sudo apt install fail2ban          # Debian/Ubuntu
sudo yum install fail2ban         # RHEL/CentOS
sudo pacman -S fail2ban           # Arch

# Start service
sudo systemctl enable --now fail2ban

# Configuration file
# /etc/fail2ban/jail.local (recommended)
# /etc/fail2ban/jail.conf (default)

jail.local Configuration

/etc/fail2ban/jail.local

[DEFAULT]
# Ban time (duration)
bantime = 1h

# Time window for retries
findtime = 10m

# Max retries before ban
maxretry = 3

# Email notifications
destemail = admin@example.com
sender = fail2ban@example.com
action = %(action_mwl)s

# Whitelist (never ban)
ignoreip = 127.0.0.1/8 10.0.0.0/8 192.168.0.0/16

SSH Jail

/etc/fail2ban/jail.local

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 5
bantime = 1h
findtime = 10m
action = %(action_mwl)s

# Custom port
[sshd-custom]
enabled = true
port = 2222
filter = sshd
logpath = /var/log/auth.log
maxretry = 3

Web Server Jails

# Nginx HTTP Auth
[nginx-http-auth]
enabled = true
port = http,https
filter = nginx-http-auth
logpath = /var/log/nginx/error.log
maxretry = 5

# Nginx Bot Search
[nginx-badrequests]
enabled = true
port = http,https
filter = nginx-badrequests
logpath = /var/log/nginx/error.log

# Apache Auth
[apache-auth]
enabled = true
port = http,https
filter = apache-auth
logpath = /var/log/apache2/error.log
maxretry = 5

# Apache Badbots
[apache-badbots]
enabled = true
port = http,https
filter = apache-badbots
logpath = /var/log/apache2/error.log

Mail Server Jails

# Postfix
[postfix]
enabled = true
port = smtp,submission,imap3,imaps,pop3,pop3s
filter = postfix
logpath = /var/log/mail.log

# Dovecot
[dovecot]
enabled = true
port = smtp,submission,imap3,imaps,pop3,pop3s
filter = dovecot
logpath = /var/log/mail.log

# SASL
[sasl]
enabled = true
port = smtp,submission,imap3,imaps,pop3,pop3s
filter = sasl
logpath = /var/log/mail.log
maxretry = 3

Custom Jail Example

# Block repeated login attempts to API
[api-rate-limit]
enabled = true
port = http,https
filter = api-rate-limit
logpath = /var/log/api/access.log
maxretry = 10
bantime = 30m
findtime = 1m
action = %(action_mwl)s

# Create filter
# /etc/fail2ban/filter.d/api-rate-limit.conf
[Definition]
failregex = ^<HOST> .* "POST /api/login
ignoreregex =

Fail2Ban Commands

# Status
sudo fail2ban-client status
sudo fail2ban-client status sshd

# Ban/Unban
sudo fail2ban-client set sshd banip 1.2.3.4
sudo fail2ban-client set sshd unbanip 1.2.3.4

# Reload
sudo fail2ban-client reload
sudo fail2ban-client reload sshd

# Check logs
tail -f /var/log/fail2ban.log

# Test filter
fail2ban-regex /var/log/auth.log /etc/fail2ban/filter.d/sshd.conf

---

33.3 OSSEC (Host-based IDS)

Installation

# Install
sudo apt install ossec-hids           # Debian/Ubuntu
sudo yum install ossec-hids           # RHEL/CentOS
sudo pacman -S ossec-hids            # Arch

# Configure
sudo /var/ossec/bin/ossec-configure

Configuration

/var/ossec/etc/ossec.conf

# Local rules
# /var/ossec/rules/local_rules.xml

# Commands
# View alerts
tail -f /var/ossec/logs/alerts/alerts.log

# Manage
/var/ossec/bin/ossec-control start
/var/ossec/bin/ossec-control stop
/var/ossec/bin/ossec-control restart

# Client sync
/var/ossec/bin/manage_agents

---

33.4 AIDE (File Integrity)

Installation

# Install
sudo apt install aide           # Debian/Ubuntu
sudo yum install aide          # RHEL/CentOS
sudo pacman -S aide           # Arch

Initialization and Usage

# Initialize database
sudo aide --init

# Rename database
sudo mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db

# Check integrity
sudo aide --check

# Update database
sudo aide --update

# Check specific file
sudo aide --check /etc/passwd

# Daily check (cron)
sudo crontab -e
0 5 * * * /usr/bin/aide --check

Configuration

/etc/aide/aide.conf

# Database location
database=file:/var/lib/aide/aide.db
database_out=file:/var/lib/aide/aide.db.new

# Groups
/etc/pam.d R
/etc/passwd R
/etc/shadow R
/etc/group R
/etc/sudoers R
/var/log R
/bin R
/sbin R
/usr/bin R
/usr/sbin R
/etc/cron R
/etc/hosts R

---

33.5 RKHunter (Rootkit Detection)

Installation and Usage

# Install
sudo apt install rkhunter          # Debian/Ubuntu
sudo yum install rkhunter          # RHEL/CentOS

# Update
sudo rkhunter --update

# Check system
sudo rkhunter --check
sudo rkhunter --check --sk

# Properties test
sudo rkhunter --propupd

---

33.6 Lynis (Security Auditing)

Installation and Usage

# Install
sudo apt install lynis            # Debian/Ubuntu
sudo yum install lynis           # RHEL/CentOS
sudo pacman -S lynis            # Arch

# Run audit
sudo lynis audit system
sudo lynis audit system --cronjob

# Check hardening
sudo lynis audit system --details

# Quick scan
sudo lynis quick

---

33.7 Interview Questions

Basic Questions

1. What is Fail2Ban?

   • Intrusion prevention system that bans IPs after failed attempts

2. What is the difference between IDS and IPS?

   • IDS detects; IPS prevents (active blocking)

3. What is AIDE?

   • File integrity monitoring tool

4. How does Fail2Ban work?

   • Monitors log files, regex matching, bans via iptables

5. What is OSSEC?

   • Host-based intrusion detection system

Intermediate Questions

6. What are the main components of Fail2Ban?

   • Filters, Actions, Jails

7. How do you whitelist an IP in Fail2Ban?

   • Add to ignoreip in jail.local

8. What is the purpose of findtime in Fail2Ban?

   • Time window for counting retries

9. How do you create a custom Fail2Ban jail?

   • Create filter in /etc/fail2ban/filter.d/ and jail in jail.local

10. What is rootkit detection?

    • Detecting hidden malicious software

---

Summary

                 Quick Reference
+------------------------------------------------------------------+
|                                                                   |
|    Fail2Ban:                                                      |
|    +----------------------------------------------------------+  |
|    | sudo fail2ban-client status           | Status             |  |
|    | sudo fail2ban-client set sshd banip  | Ban IP             |  |
|    | sudo fail2ban-client set sshd unbanip | Unban IP           |  |
|    +----------------------------------------------------------+  |
|                                                                   |
|    AIDE:                                                          |
|    +----------------------------------------------------------+  |
|    | sudo aide --init                  | Initialize            |  |
|    | sudo aide --check                 | Check integrity      |  |
|    | sudo aide --update                | Update DB            |  |
|    +----------------------------------------------------------+  |
|                                                                   |
|    OSSEC:                                                         |
|    +----------------------------------------------------------+  |
|    | /var/ossec/bin/ossec-control start| Start                |  |
|    | tail -f /var/ossec/logs/alerts/ | View alerts          |  |
|    +----------------------------------------------------------+  |
|                                                                   |
+------------------------------------------------------------------+