Skip to content

Linux_Practical_Interview_1001 1250

Linux Practical Interview Questions (1001-1250)

Section titled “Linux Practical Interview Questions (1001-1250)”

Q1001: How do Linux kernel and user space interact?

Section titled “Q1001: How do Linux kernel and user space interact?”

Answer: The Linux kernel and user space interact through system calls, which are the primary interface between user applications and the kernel.

Terminal window
# System call interface
# User space -> libc -> system call wrapper -> kernel
# Example: read() system call
ssize_t read(int fd, void *buf, size_t count);
# View system calls
strace -e trace=read,write cat /etc/passwd
# List system calls
man syscalls
# or
cat /usr/include/asm/unistd_64.h | head -30

Q1002: What is the role of the init system in Linux?

Section titled “Q1002: What is the role of the init system in Linux?”

Answer: The init system is the first process started by the kernel (PID 1) and is responsible for:

  • Starting system services in the correct order
  • Managing runlevels/targets
  • Handling system shutdown and restart
  • Being the parent of all orphaned processes
Terminal window
# Systemd (modern)
systemctl status
systemctl list-units --type=service
# SysVinit (legacy)
runlevel
ls -la /etc/rc.d/
# Runit
sv status /service/*

Q1003: How does Linux handle process scheduling?

Section titled “Q1003: How does Linux handle process scheduling?”

Answer: Linux uses Completely Fair Scheduler (CFS) which provides fair CPU time distribution:

Terminal window
# View process priority and nice value
ps -eo pid,ni,pri,pcpu,comm
renice 10 -p 1234
# Set real-time priority
chrt -f 50 -p 1234
chrt -r -p 50 1234
# View scheduler
cat /proc/1234/sched

Q1004: Explain Linux virtual memory management.

Section titled “Q1004: Explain Linux virtual memory management.”

Answer: Linux uses demand paging with virtual memory:

Terminal window
# View memory info
cat /proc/meminfo
free -h
vmstat 1
# View process memory maps
pmap -x 1234
cat /proc/1234/maps
# Memory zones
cat /proc/buddyinfo

Q1005: What are Linux namespaces and how are they used?

Section titled “Q1005: What are Linux namespaces and how are they used?”

Answer: Namespaces provide process isolation:

Terminal window
# View namespaces
ls -la /proc/$$/ns/
# Create a new namespace (user namespace requires kernel 3.8+)
unshare --mount --pid --fork --user --map-root-user bash
# Network namespace
ip netns add mynet
ip netns exec mynet ip link list

Q1006: Describe the Linux boot process from power on.

Section titled “Q1006: Describe the Linux boot process from power on.”

Answer:

  1. BIOS/UEFI POST
  2. Boot loader (GRUB2) loads kernel
  3. Kernel initializes and loads initrd/initramfs
  4. Kernel mounts root filesystem
  5. Init system (systemd) starts
  6. Runlevel targets reached
Terminal window
# View boot messages
dmesg | less
journalctl -b
# Boot time analysis
systemd-analyze time
systemd-analyze blame | head -20

Q1007: How do you troubleshoot boot issues in Linux?

Section titled “Q1007: How do you troubleshoot boot issues in Linux?”

Answer:

Terminal window
# Check boot logs
journalctl -b -1 # Previous boot
journalctl -b --priority=err
# Emergency mode
# At GRUB menu, add 'systemd.unit=emergency.target'
# Recovery mode
# At GRUB menu, add 'systemd.unit=rescue.target'
# Check filesystem
fsck /dev/sda1
mount -o remount,rw /

Q1008: What is GRUB2 and how do you configure it?

Section titled “Q1008: What is GRUB2 and how do you configure it?”

Answer: GRUB2 (Grand Unified Boot Loader) is the default boot loader:

Terminal window
# Edit GRUB config
vim /etc/default/grub
# Common settings
GRUB_TIMEOUT=5
GRUB_DEFAULT=saved
GRUB_CMDLINE_LINUX="quiet splash"
# Regenerate config
update-grub # Debian/Ubuntu
grub2-mkconfig -o /boot/grub2/grub.cfg # RHEL/CentOS
# GRUB command line
# Press 'e' at boot menu to edit

Answer: Initial RAM filesystem contains modules needed before root filesystem is mounted:

Terminal window
# Rebuild initramfs
dracut -f # RHEL/CentOS
update-initramfs -u # Debian/Ubuntu
# View contents
lsinitramfs /boot/initrd.img-$(uname -r)
zcat /boot/initrd.img | cpio -id
# Custom initramfs
# Add to /etc/initramfs-tools/modules

Answer: Systemd uses unit files instead of init scripts:

/etc/systemd/system/myservice.service
# Service unit example
[Unit]
Description=My Service
After=network.target
[Service]
Type=simple
ExecStart=/usr/bin/myservice
Restart=on-failure
[Install]
WantedBy=multi-user.target
# Manage service
systemctl enable myservice
systemctl start myservice
systemctl status myservice

Answer: Logical Volume Manager provides flexible storage:

Terminal window
# Physical volume
pvcreate /dev/sdb1
pvdisplay
pvscan
# Volume group
vgcreate vg_data /dev/sdb1
vgdisplay
vgextend vg_data /dev/sdc1
# Logical volume
lvcreate -L 10G -n lv_mydata vg_data
lvcreate -l 100%FREE -n lv_backup vg_data
# Format and mount
mkfs.xfs /dev/vg_data/lv_mydata
mount /dev/vg_data/lv_mydata /mnt/data
# Resize
lvextend -L +5G /dev/vg_data/lv_mydata
resize2fs /dev/vg_data/lv_mydata

Q1012: What is the difference between ext4 and XFS?

Section titled “Q1012: What is the difference between ext4 and XFS?”

Answer:

Featureext4XFS
Max file size16TB8EB
Max volume1EB8EB
JournalingYesYes
Online resizeLimitedYes
PerformanceSmall filesLarge files
Terminal window
# Create ext4
mkfs.ext4 -j /dev/sdb1
# Create XFS
mkfs.xfs -f /dev/sdb1
# Tune ext4
tune2fs -o journal_data /dev/sda1
tune2fs -O dir_index /dev/sda1

Answer:

/etc/fstab
# Enable quota
/dev/sda1 /home ext4 usrquota,grpquota 0 2
# Initialize quota
quotacheck -cug /home
quotaon /home
# Set quotas
edquota -u username
# or
setquota -u username 10000 15000 1000 2000 /home
# Check quota
quota -u username
repquota -a

Q1014: What is RAID and how do you configure it in Linux?

Section titled “Q1014: What is RAID and how do you configure it in Linux?”

Answer: Redundant Array of Independent Disks:

Terminal window
# Software RAID with mdadm
mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1
# Manage RAID
mdadm --detail /dev/md0
cat /proc/mdstat
# Add spare
mdadm /dev/md0 --add /dev/sde1
# Monitor
mdadm --monitor --mail=admin@example.com --daemonize /dev/md0

Q1015: How do you troubleshoot filesystem errors?

Section titled “Q1015: How do you troubleshoot filesystem errors?”

Answer:

Terminal window
# Check filesystem
fsck -n /dev/sda1 # dry run
fsck -y /dev/sda1 # auto repair
# XFS specific
xfs_repair /dev/sda1
xfs_info /dev/sda1
# View inode usage
df -i
tune2fs -l /dev/sda1 | grep -i inode
# Recover deleted files (ext3/4)
extundelete /dev/sda1 --inode 2

Q1016: How do you configure network bonding?

Section titled “Q1016: How do you configure network bonding?”

Answer: Combine multiple NICs for redundancy/performance:

Terminal window
# Load bonding module
modprobe bonding mode=active-backup
# Create bond interface
cat > /etc/sysconfig/network-scripts/ifcfg-bond0 << EOF
DEVICE=bond0
TYPE=Bond
IPADDR=192.168.1.10
NETMASK=255.255.255.0
ONBOOT=yes
BONDING_OPTS="mode=active-backup miimon=100"
EOF
# Add slave interfaces
echo "MASTER=bond0" >> /etc/sysconfig/network-scripts/ifcfg-eth0
echo "MASTER=bond0" >> /etc/sysconfig/network-scripts/ifcfg-eth1
# View bond status
cat /proc/net/bonding/bond0

Q1017: How do you configure VLANs in Linux?

Section titled “Q1017: How do you configure VLANs in Linux?”

Answer:

Terminal window
# Enable 8021q module
modprobe 8021q
# Create VLAN interface
ip link add link eth0 name eth0.100 type vlan id 100
ip addr add 192.168.100.1/24 dev eth0.100
ip link set eth0.100 up
# Or using vconfig
vconfig add eth0 100
ifconfig eth0.100 192.168.100.1 netmask 255.255.255.0 up
# Persistent VLAN config (RHEL)
# /etc/sysconfig/network-scripts/ifcfg-eth0.100

Q1018: What is iptables and how do you configure basic rules?

Section titled “Q1018: What is iptables and how do you configure basic rules?”

Answer:

Terminal window
# List rules
iptables -L -n -v
iptables -t nat -L -n -v
# Basic rules
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -j DROP
# NAT rules
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080
# Save rules
iptables-save > /etc/iptables/rules.v4
# or
service iptables save

Q1019: How do you configure network bridging?

Section titled “Q1019: How do you configure network bridging?”

Answer:

Terminal window
# Create bridge
brctl addbr br0
ip addr add 192.168.1.1/24 dev br0
ip link set br0 up
# Add interfaces
brctl addif br0 eth0
brctl addif br0 eth1
# View bridge
brctl show
ip link show type bridge
# Delete bridge
ip link set br0 down
brctl delbr br0

Q1020: How do you troubleshoot network connectivity issues?

Section titled “Q1020: How do you troubleshoot network connectivity issues?”

Answer:

Terminal window
# Check interface status
ip link show
ip addr show
ethtool eth0
# Test connectivity
ping -c 4 8.8.8.8
traceroute 8.8.8.8
mtr 8.8.8.8
# Check DNS
dig example.com
nslookup example.com
cat /etc/resolv.conf
# Check ports
netstat -tulpn
ss -tulpn
lsof -i :80
# Capture packets
tcpdump -i eth0 port 80
tcpdump -i eth0 host 192.168.1.1

Answer: Security-Enhanced Linux provides mandatory access control:

Terminal window
# Check SELinux status
getenforce
sestatus
# Set mode
setenforce 1 # Enforcing
setenforce 0 # Permissive
# Configure /etc/selinux/config
# SELINUX=enforcing
# SELINUXTYPE=targeted
# Manage contexts
chcon -t httpd_sys_content_t /var/www/html/index.html
semanage fcontext -a -t httpd_sys_content_t "/web(/.*)?"
restorecon -Rv /web
# Boolean values
getsebool -a
setsebool -P httpd_can_network_connect on

Q1022: How do you configure a firewall with firewalld?

Section titled “Q1022: How do you configure a firewall with firewalld?”

Answer:

Terminal window
# Check status
firewall-cmd --state
firewall-cmd --list-all
# Add services
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-service=https
# Add ports
firewall-cmd --permanent --add-port=8080/tcp
# Zones
firewall-cmd --permanent --zone=public --add-service=ssh
firewall-cmd --set-default-zone=trusted
# Reload
firewall-cmd --reload

Answer:

/etc/ssh/sshd_config
Port 2222
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
ClientAliveInterval 300
AllowUsers user1 user2
# Generate key pair
ssh-keygen -t ed25519 -C "work laptop"
# Copy key
ssh-copy-id user@server
# Fail2ban
apt install fail2ban
# or
yum install fail2ban

Answer:

Terminal window
# Install audit
apt install auditd
# or
yum install audit
# Configure /etc/audit/auditd.conf
max_log_file = 50
max_log_file_action = rotate
# Add rules
auditctl -w /etc/passwd -p wa -k passwd_changes
auditctl -w /usr/bin/rm -p x -k file_delete
# View logs
ausearch -k passwd_changes
aureport --file
# Make rules persistent
# /etc/audit/rules.d/audit.rules

Q1025: How do you implement user authentication with PAM?

Section titled “Q1025: How do you implement user authentication with PAM?”

Answer:

Terminal window
# PAM configuration files
ls -la /etc/pam.d/
# Example: password policy
# /etc/pam.d/common-password
password required pam_pwhistory.so remember=5
password [default=1] pam_permit.so
# Limit resources
# /etc/security/limits.conf
username soft nofile 4096
username hard nofile 8192
# Time-based login
# /etc/security/time.conf
login;ts001;users;Al0900-1700

Q1026: How do you configure Docker networking?

Section titled “Q1026: How do you configure Docker networking?”

Answer:

Terminal window
# List networks
docker network ls
docker network inspect bridge
# Create network
docker network create --driver bridge mynetwork
docker network create --driver overlay myoverlay
# Connect container
docker run -d --network mynetwork --name web nginx
# Port mapping
docker run -d -p 8080:80 --name web2 nginx
# DNS resolution
docker run -d --network-alias db --network mynetwork mysql

Answer:

Terminal window
# Create volume
docker volume create mydata
# Mount volume
docker run -v mydata:/data mysql
# Bind mount
docker run -v /host/path:/container/path nginx
# Volume driver
docker volume create --driver local myvolume
# View volumes
docker volume ls
docker volume inspect mydata

Q1028: How do you configure Docker Compose?

Section titled “Q1028: How do you configure Docker Compose?”

Answer:

docker-compose.yml
version: '3.8'
services:
web:
build: .
ports:
- "8080:80"
environment:
- NODE_ENV=production
volumes:
- ./data:/data
depends_on:
- db
networks:
- frontend
- backend
db:
image: postgres:14
volumes:
- db-data:/var/lib/postgresql/data
networks:
- backend
volumes:
db-data:
networks:
frontend:
backend:

Q1029: How do you secure Docker containers?

Section titled “Q1029: How do you secure Docker containers?”

Answer:

Terminal window
# Run as non-root
docker run -u 1000:1000 nginx
# Read-only filesystem
docker run --read-only nginx
# Limit capabilities
docker run --cap-drop ALL --cap-add NET_BIND_SERVICE nginx
# Disable networking
docker run --network none nginx
# Scan images
docker scan nginx
trivy image nginx
# Best practices in Dockerfile
FROM ubuntu:22.04
RUN useradd -m appuser
USER appuser

Q1030: How do you troubleshoot container issues?

Section titled “Q1030: How do you troubleshoot container issues?”

Answer:

Terminal window
# View logs
docker logs container_id
docker logs -f container_id
docker logs --tail 100 container_id
# Inspect container
docker inspect container_id
docker inspect --format='{{.NetworkSettings.IPAddress}}' container_id
# Execute commands
docker exec -it container_id /bin/bash
docker exec container_id ps aux
# Resource usage
docker stats container_id
docker stats --no-stream container_id
# Network debugging
docker run --rm -it --network container:name nicolaka/netshoot

Answer:

Terminal window
# View CPU info
lscpu
cat /proc/cpuinfo
# CPU frequency
cpupower frequency-info
cpupower frequency-set -g performance
# CPU affinity
taskset -c 0-3 myapp
taskset -p 0xF myapp
# Process priority
nice -n 10 myapp
renice 5 -p 1234
# View scheduler
cat /proc/sched_debug

Q1032: How do you tune memory performance?

Section titled “Q1032: How do you tune memory performance?”

Answer:

Terminal window
# View memory
free -h
cat /proc/meminfo
# Clear cache
sync && echo 3 > /proc/sys/vm/drop_caches
# Swappiness
cat /proc/sys/vm/swappiness
sysctl vm.swappiness=10
# Add to /etc/sysctl.conf
# vm.swappiness=10
# Huge pages
echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

Answer:

Terminal window
# I/O scheduler
cat /sys/block/sda/queue/scheduler
echo cfq > /sys/block/sda/queue/scheduler
# or
echo deadline > /sys/block/sda/queue/scheduler
# I/O priority
ionice -c 2 -n 0 -p 1234
# Block device settings
echo 4096 > /sys/block/sda/queue/read_ahead_kb
echo 0 > /sys/block/sda/queue/rotational
# Filesystem options
mount -o noatime,nodiratime /dev/sda1 /mnt

Q1034: How do you tune network performance?

Section titled “Q1034: How do you tune network performance?”

Answer:

Terminal window
# Network buffer sizes
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
# TCP tuning
sysctl -w net.ipv4.tcp_window_scaling=1
sysctl -w net.ipv4.tcp_congestion_control=cubic
# Make persistent
# /etc/sysctl.conf
net.core.rmem_max=16777216
net.core.wmem_max=16777216
# Ring buffer (NIC)
ethtool -G eth0 rx 4096 tx 4096
ethtool -g eth0

Q1035: How do you use performance monitoring tools?

Section titled “Q1035: How do you use performance monitoring tools?”

Answer:

Terminal window
# System performance
top
htop
atop
# Process monitoring
pidstat -p 1234 1
prstat -p 1234
# I/O monitoring
iostat -xz 1
iotop
# Network monitoring
nethogs
iftop
sar -n DEV 1
# Full system analysis
perf record -g ./myapp
perf report

Q1036: How do you write efficient bash scripts?

Section titled “Q1036: How do you write efficient bash scripts?”

Answer:

#!/bin/bash
set -euo pipefail
# Use functions
log() {
echo "[$(date +'%Y-%m-%d %H:%M:%S')] $*"
}
# Parse arguments
while getopts "hvf:" opt; do
case $opt in
h) help; exit 0 ;;
v) VERBOSE=1 ;;
f) FILE="$OPTARG" ;;
*) exit 1 ;;
esac
done
# Use arrays
files=("file1" "file2" "file3")
for file in "${files[@]}"; do
process "$file"
done
# Error handling
trap 'echo "Error on line $LINENO"' ERR

Q1037: How do you use awk for text processing?

Section titled “Q1037: How do you use awk for text processing?”

Answer:

Terminal window
# Basic usage
awk '{print $1}' file.txt
awk -F: '{print $1, $6}' /etc/passwd
# Conditional
awk '$3 > 1000 {print $1, $3}' /etc/passwd
# Calculations
awk '{sum+=$1} END {print sum}' numbers.txt
# Multiple fields
awk '{print "User:" $1 " UID:" $3 " Home:" $6}' /etc/passwd
# Patterns
awk '/^root/ {print}' /etc/passwd
awk 'NR==5 {print}' file.txt
# Field separator
awk -F: '{print NF}' /etc/passwd

Q1038: How do you use sed for text manipulation?

Section titled “Q1038: How do you use sed for text manipulation?”

Answer:

Terminal window
# Replace text
sed 's/old/new/' file.txt
sed 's/old/new/g' file.txt
sed 's/old/new/2' file.txt
# In-place editing
sed -i 's/old/new/g' file.txt
# Delete lines
sed '/pattern/d' file.txt
sed '1,5d' file.txt
sed '5d' file.txt
# Insert text
sed '1i\Header line' file.txt
sed '1a\New line' file.txt
# Regex
sed -E 's/[0-9]+/[REDACTED]/g' file.txt

Answer:

Terminal window
# Using jq
cat data.json | jq '.name'
cat data.json | jq '.items[].id'
cat data.json | jq 'map(select(.age > 25))'
cat data.json | jq '.items | length'
# Create JSON
jq -n '{name: "test", value: 42}'
# Modify JSON
cat data.json | jq '.name = "new_name"'
cat data.json | jq '.items += [{"id": 3}]'
# Filter
cat data.json | jq '.items[] | select(.id == 2)'

Q1040: How do you write Python scripts for Linux administration?

Section titled “Q1040: How do you write Python scripts for Linux administration?”

Answer:

#!/usr/bin/env python3
import subprocess
import os
import json
# Run shell commands
result = subprocess.run(['df', '-h'], capture_output=True, text=True)
print(result.stdout)
# Work with files
with open('/etc/passwd', 'r') as f:
for line in f:
if 'admin' in line:
print(line.strip())
# Parse JSON
with open('config.json', 'r') as f:
config = json.load(f)
print(config.get('database', {}))
# System info
import platform
import socket
print(platform.uname())
print(socket.gethostname())

Answer:

Terminal window
# Install
apt install apache2
# or
yum install httpd
# Configuration
# /etc/apache2/apache2.conf (Debian)
# /etc/httpd/conf/httpd.conf (RHEL)
# Virtual hosts
# /etc/apache2/sites-available/example.conf
<VirtualHost *:80>
ServerName example.com
DocumentRoot /var/www/html
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
# Enable site
a2ensite example
systemctl reload apache2
# Modules
a2enmod ssl rewrite proxy

Answer:

Terminal window
# Install
apt install nginx
# or
yum install nginx
# Configuration
# /etc/nginx/nginx.conf
# Server block
# /etc/nginx/sites-available/default
server {
listen 80;
server_name example.com;
root /var/www/html;
index index.html;
location / {
try_files $uri $uri/ =404;
}
location /api {
proxy_pass http://localhost:3000;
}
}
# Test config
nginx -t
# Reload
systemctl reload nginx

Q1043: How do you configure MySQL/MariaDB?

Section titled “Q1043: How do you configure MySQL/MariaDB?”

Answer:

Terminal window
# Install
apt install mysql-server
# or
yum install mariadb-server
# Secure installation
mysql_secure_installation
# Create database and user
mysql -u root -p
CREATE DATABASE myapp;
CREATE USER 'myuser'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON myapp.* TO 'myuser'@'localhost';
FLUSH PRIVILEGES;
# Configuration
# /etc/mysql/mariadb.conf.d/50-server.cnf
# key_buffer_size = 256M
# max_connections = 200
# Backup
mysqldump -u root -p myapp > backup.sql
mysql -u root -p myapp < backup.sql

Answer:

Terminal window
# Install
apt install postgresql
# or
yum install postgresql-server
# Initialize
postgresql-setup --initdb
# or
pg_ctl -D /var/lib/pgsql/data initdb
# Create user and database
sudo -u postgres createuser myuser
sudo -u postgres createdb mydb
sudo -u postgres psql
ALTER USER myuser WITH PASSWORD 'password';
GRANT ALL PRIVILEGES ON DATABASE mydb TO myuser;
# Configuration
# /var/lib/pgsql/data/postgresql.conf
# max_connections = 100
# shared_buffers = 128MB
# Backup
pg_dump -U myuser mydb > backup.sql

Answer:

Terminal window
# Install
apt install redis-server
# or
yum install redis
# Configuration
# /etc/redis/redis.conf
bind 127.0.0.1
port 6379
maxmemory 256mb
maxmemory-policy allkeys-lru
appendonly yes
# Commands
redis-cli PING
redis-cli SET mykey "value"
redis-cli GET mykey
redis-cli KEYS "*"
# Cluster mode
redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 --cluster-replicas 1

Q1046: How do you set up Prometheus monitoring?

Section titled “Q1046: How do you set up Prometheus monitoring?”

Answer:

Terminal window
# Install
apt install prometheus
# or
tar -xzf prometheus-*.tar.gz
# Configuration
# /etc/prometheus/prometheus.yml
global:
scrape_interval: 15s
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090']
- job_name: 'node'
static_configs:
- targets: ['localhost:9100']
# Node exporter
./node_exporter --collector.filesystem.mount-points-exclude="^/(sys|proc|dev|run)($|/)"
# View metrics
curl http://localhost:9100/metrics

Answer:

Terminal window
# Install
apt install grafana
# or
yum install grafana
# Start
systemctl start grafana-server
# Default: admin/admin
# Add data source
# HTTP URL: http://localhost:9090 (Prometheus)
# Create dashboard (JSON)
{
"dashboard": {
"title": "System Monitoring",
"panels": [
{
"title": "CPU Usage",
"type": "graph",
"targets": [
{
"expr": "100 - (avg by (instance) (irate(node_cpu_seconds_total{mode='idle'}[5m])) * 100)"
}
]
}
]
}
}

Answer:

Terminal window
# Install Elasticsearch
apt install elasticsearch
systemctl enable elasticsearch
# Configure
# /etc/elasticsearch/elasticsearch.yml
cluster.name: mycluster
network.host: 0.0.0.0
discovery.type: single-node
# Install Kibana
apt install kibana
# Install Logstash
apt install logstash
# Filebeat configuration
# /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
paths:
- /var/log/*.log
output.logstash:
hosts: ["localhost:5044"]

Answer:

Terminal window
# Install
apt install nagios4
# or
yum install nagios
# Create check script
#!/bin/bash
# /usr/local/nagios/lib/check_disk.sh
USAGE=$(df -h / | tail -1 | awk '{print $5}' | sed 's/%//')
if [ "$USAGE" -gt 90 ]; then
echo "CRITICAL - Disk usage is ${USAGE}%"
exit 2
elif [ "$USAGE" -gt 80 ]; then
echo "WARNING - Disk usage is ${USAGE}%"
exit 1
else
echo "OK - Disk usage is ${USAGE}%"
exit 0
fi
# Define service
# /etc/nagios4/conf.d/services.cfg
define service{
host_name localhost
service_description Disk Usage
check_command check_disk
check_interval 5
}

Answer:

Terminal window
# Install Zabbix server
apt install zabbix-server-mysql zabbix-frontend-php
# Create database
mysql -u root -p
CREATE DATABASE zabbix CHARACTER SET utf8 COLLATE utf8_bin;
CREATE USER 'zabbix'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON zabbix.* TO 'zabbix'@'localhost';
FLUSH PRIVILEGES;
quit;
# Import schema
zcat /usr/share/zabbix-sql-scripts/mysql/server.sql.gz | mysql -u zabbix -p zabbix
# Configure server
# /etc/zabbix/zabbix_server.conf
DBPassword=password
# Install agent
apt install zabbix-agent
# Configure agent
# /etc/zabbix/zabbix_agentd.conf
Server=127.0.0.1
ServerActive=127.0.0.1
Hostname=Zabbix server

Answer:

Terminal window
# Basic sync
rsync -av /source/ /destination/
rsync -avz -e ssh user@remote:/remote/ /local/
# Delete files not in source
rsync -av --delete /source/ /destination/
# Exclude patterns
rsync -av --exclude='*.log' --exclude='tmp/' /source/ /destination/
# Progress and bandwidth
rsync -av --progress --bwlimit=1000 /source/ /destination/
# Dry run
rsync -avn /source/ /destination/
# Script for automated backup
#!/bin/bash
rsync -avz --delete -e ssh /data/ backup@server:/backup/$(date +%Y%m%d)/

Answer:

Terminal window
# Create archive
tar -cvf backup.tar /data
tar -cvzf backup.tar.gz /data
tar -cvjf backup.tar.bz2 /data
# Extract
tar -xvf backup.tar
tar -xvzf backup.tar.gz
# List contents
tar -tvf backup.tar
# Incremental backup
tar -g /var/log/backup.snap -cvzf backup-incremental.tar.gz /data
# With date
tar -cvzf backup-$(date +%Y%m%d).tar.gz /data

Q1053: How do you configure Bacula for backups?

Section titled “Q1053: How do you configure Bacula for backups?”

Answer:

Terminal window
# Install
apt install bacula-server bacula-client
# Configure Director
# /etc/bacula/bacula-dir.conf
Director {
Name = bacula-dir
DIRport = 9101
QueryFile = "/etc/bacula/query.sql"
WorkingDirectory = "/var/lib/bacula"
PidDirectory = "/var/run/bacula"
}
# FileSet
FileSet {
Name = "Full Set"
Include {
Options {
Signature = MD5
}
File = /data
}
Exclude {
File = /var/lib/bacula
}
}
# Schedule
Schedule {
Name = "WeeklyCycle"
Run = Full 1st sun at 23:05
Run = Incremental mon-sat at 23:05
}

Q1054: How do you use Duplicity for backups?

Section titled “Q1054: How do you use Duplicity for backups?”

Answer:

Terminal window
# Install
apt install duplicity
# Backup to local
duplicity /data file:///backup/
# Backup to remote
duplicity /data sftp://user@remote//backup/
# Encrypted backup
duplicity --encrypt-key ABC123 /data sftp://user@remote//backup/
# Restore
duplicity file:///backup/ /restore/
duplicity --time 2024-01-01 sftp://user@remote//backup/ /restore/
# List files
duplicity list-current-files file:///backup/
# Verify
duplicity verify file:///backup/ /data/

Q1055: How do you create a disaster recovery plan?

Section titled “Q1055: How do you create a disaster recovery plan?”

Answer:

Terminal window
# 1. Document current system
# Hardware inventory
lshw > hardware_inventory.txt
lspci >> hardware_inventory.txt
# Software inventory
dpkg -l > installed_packages.txt
# or
rpm -qa > installed_packages.txt
# Network configuration
ip addr show > network_config.txt
route -n >> network_config.txt
iptables-save > iptables_rules.txt
# Configuration files
tar -cvzf configs.tar.gz /etc/
# 2. Test recovery
# In VM/test environment
# 1. Restore configs
# 2. Verify services
# 3. Test applications

Answer:

Terminal window
# Install
apt install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils
# Check support
kvm-ok
# Create VM
virt-install \
--name webserver \
--ram 2048 \
--disk path=/var/lib/libvirt/images/webserver.qcow2,size=20 \
--vcpus 2 \
--os-type linux \
--os-variant ubuntu22.04 \
--network bridge=virbr0 \
--graphics vnc \
--console pty,target_type=serial \
--location 'http://archive.ubuntu.com/ubuntu/dists/jammy/main/installer-amd64/' \
--extra-args 'console=ttyS0'
# Manage VMs
virsh list --all
virsh start webserver
virsh shutdown webserver
virsh undefine webserver

Answer:

Terminal window
# Connect
virsh --connect qemu:///system
# Create network
# /tmp/network.xml
<network>
<name>internal</name>
<forward mode='nat'/>
<ip address='192.168.100.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.100.128' end='192.168.100.254'/>
</dhcp>
</ip>
</network>
virsh net-define /tmp/network.xml
virsh net-start internal
# Snapshot
virsh snapshot-create-as webserver --name "before-update"
virsh snapshot-list webserver
virsh snapshot-revert webserver before-update
# Pool management
virsh pool-list
virsh pool-info default

Answer:

Terminal window
# Run QEMU
qemu-system-x86_64 \
-m 2048 \
-hda disk.img \
-cdrom ubuntu.iso \
-boot d \
-enable-kvm
# With network
qemu-system-x86_64 \
-m 2048 \
-hda disk.img \
-netdev user,id=net0,hostfwd=tcp::2222-:22 \
-device e1000,netdev=net0
# Snapshots
qemu-img create -f qcow2 -b base.img snapshot.img
qemu-img info snapshot.img
# Resize disk
qemu-img resize disk.img +10G
# Convert image
qemu-img convert -O vmdk img.qcow2 img.vmdk

Q1059: How do you troubleshoot virtualization issues?

Section titled “Q1059: How do you troubleshoot virtualization issues?”

Answer:

Terminal window
# Check KVM
lsmod | grep kvm
cat /proc/cpuinfo | grep vmx
virt-host-validate
# View logs
journalctl -u libvirtd
dmesg | grep -i kvm
# Debug VM
virsh console webserver
virsh dump webserver /tmp/core.qemu
# Network issues
brctl show
ip link show virbr0
# Performance
virsh dominfo webserver
virsh cpu-stats webserver

Q1060: How do you configure LXC containers?

Section titled “Q1060: How do you configure LXC containers?”

Answer:

Terminal window
# Install
apt install lxc
# Create container
lxc-create -n mycontainer -t ubuntu
# Start container
lxc-start -n mycontainer
lxc-attach -n mycontainer
# Configuration
# /var/lib/lxc/mycontainer/config
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.arch = amd64
lxc.rootfs.path = dir:/var/lib/lxc/mycontainer/rootfs
lxc.uts.name = mycontainer
lxc.network.type = veth
lxc.network.link = lxcbr0
lxc.network.hwaddr = 00:16:3e:xx:xx:xx
# Clone
lxc-copy -n mycontainer -N mycontainer2
# Snapshots
lxc-snapshot -n mycontainer
lxc-snapshot -n mycontainer -L

Answer:

/etc/keepalived/keepalived.conf
# Install
apt install keepalived
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass mysecret
}
virtual_ipaddress {
192.168.1.100
}
track_script {
check_apache
}
}
vrrp_script check_apache {
script "pkill -0 apache2"
interval 2
weight 2
}
# On backup server
# priority 90
# state BACKUP

Answer:

/etc/haproxy/haproxy.cfg
# Install
apt install haproxy
global
log /dev/log local0
maxconn 4096
user haproxy
group haproxy
defaults
log global
mode http
option httplog
option dontlognull
option redispatch
retries 3
timeout connect 5000
timeout client 50000
timeout server 50000
frontend http-in
bind *:80
default_backend app-servers
backend app-servers
balance roundrobin
server app1 192.168.1.10:8080 check
server app2 192.168.1.11:8080 check
server app3 192.168.1.12:8080 check backup

Q1063: How do you configure Corosync/Pacemaker?

Section titled “Q1063: How do you configure Corosync/Pacemaker?”

Answer:

Terminal window
# Install
apt install pacemaker corosync pcs
# Configure corosync
# /etc/corosync/corosync.conf
totem {
version: 2
cluster_name: mycluster
transport: udpu
interface {
ringnumber: 0
bindnetaddr: 192.168.1.0
mcastport: 5405
}
}
nodelist {
node {
ring0_addr: node1.example.com
nodeid: 1
}
node {
ring0_addr: node2.example.com
nodeid: 2
}
}
quorum {
provider: corosync_votequorum
expected_votes: 2
}
# Configure cluster
pcs host auth node1 node2
pcs cluster setup mycluster node1 node2
pcs cluster start --all
pcs cluster enable --all
# Add resources
pcs resource create VirtualIP ocf:heartbeat:IPaddr2 \
ip=192.168.1.100 cidr_netmask=24 op monitor interval=30s
pcs resource create WebService systemd:apache2 \
op monitor interval=30s
pcs constraint colocation add WebService VirtualIP

Answer:

/etc/drbd.d/web.res
# Install
apt install drbd-utils
resource web {
protocol C;
on node1 {
device /dev/drbd0;
disk /dev/sdb1;
address 192.168.1.10:7788;
meta-disk internal;
}
on node2 {
device /dev/drbd0;
disk /dev/sdb1;
address 192.168.1.11:7788;
meta-disk internal;
}
}
# Initialize
drbdadm create-md web
drbdadm up web
# Primary
drbdadm primary --force web
# Filesystem
mkfs.xfs /dev/drbd0
mount /dev/drbd0 /var/www
# Status
cat /proc/drbd
drbdadm status

Answer:

Terminal window
# Test failover
# Node 1: Primary
pcs status
# Stop cluster on node 1
pcs cluster stop node1
# Verify IP moved to node 2
ip addr show
pcs status
# Test resource
pcs resource move WebService node2
pcs status
# Failover test script
#!/bin/bash
echo "Starting failover test..."
CURRENT=$(crm_mon -1 | grep -A1 "Master/Slave Set:" | tail -1 | awk '{print $3}')
echo "Current primary: $CURRENT"
if [ "$CURRENT" == "node1" ]; then
pcs cluster stop node1
sleep 10
echo "Checking failover..."
pcs status | grep "VirtualIP"
else
pcs cluster stop node2
sleep 10
pcs status | grep "VirtualIP"
fi
echo "Failover test completed"

Answer:

/etc/bind/named.conf.options
# Install
apt install bind9
options {
directory "/var/cache/bind";
forwarders {
8.8.8.8;
8.8.4.4;
};
dnssec-validation auto;
listen-on { any; };
};
# Zone file
# /etc/bind/db.example.com
$TTL 604800
@ IN SOA ns1.example.com. admin.example.com. (
2024010101 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
;
@ IN NS ns1.example.com.
@ IN A 192.168.1.10
ns1 IN A 192.168.1.10
www IN A 192.168.1.10
# Test configuration
named-checkconf
named-checkzone example.com /etc/bind/db.example.com

Answer:

/etc/dhcp/dhcpd.conf
# Install
apt install isc-dhcp-server
default-lease-time 600;
max-lease-time 7200;
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.100 192.168.1.200;
option routers 192.168.1.1;
option subnet-mask 255.255.255.0;
option domain-name-servers 8.8.8.8, 8.8.4.4;
option domain-name "example.com";
}
host static-client {
hardware ethernet 00:11:22:33:44:55;
fixed-address 192.168.1.50;
}
# Restart
systemctl restart isc-dhcp-server
# View leases
cat /var/lib/dhcp/dhcpd.leases

Answer:

/etc/dnsmasq.conf
# Install
apt install dnsmasq
interface=eth0
bind-interfaces
domain=example.com
local=/example.com/
# DNS
address=/example.com/192.168.1.10
address=/www.example.com/192.168.1.10
# DHCP
dhcp-range=192.168.1.100,192.168.1.200,12h
dhcp-option=option:router,192.168.1.1
dhcp-option=option:dns-server,192.168.1.1
# Static DHCP
dhcp-host=00:11:22:33:44:55,192.168.1.50
# DNS forwarding
server=8.8.8.8
server=8.8.4.4

Q1069: How do you troubleshoot DNS issues?

Section titled “Q1069: How do you troubleshoot DNS issues?”

Answer:

Terminal window
# Query DNS
dig example.com
dig @8.8.8.8 example.com
nslookup example.com
host example.com
# DNS zone transfer
dig axfr example.com @ns1.example.com
# Reverse lookup
dig -x 192.168.1.10
# Trace DNS resolution
dig +trace example.com
# Check DNS server
dig +short myip.opendns.com @resolver1.opendns.com
# Flush DNS cache
systemd-resolve --flush-caches
# or
/etc/init.d/nscd restart
# or
resolvectl flush-caches

Answer:

# BIND configuration for split DNS
# Internal view
view "internal" {
match-clients { 192.168.0.0/16; };
zone "example.com" {
type master;
file "db.internal.example.com";
};
};
# External view
view "external" {
match-clients { any; };
zone "example.com" {
type master;
file "db.external.example.com";
};
};
# Using different A records
# Internal: 192.168.1.10
# External: 203.0.113.10
# Test from internal
dig @internal-dns.example.com www.example.com +short
# Test from external
dig @external-dns.example.com www.example.com +short

Answer:

/etc/postfix/main.cf
# Install
apt install postfix
myhostname = mail.example.com
mydomain = example.com
myorigin = $mydomain
inet_interfaces = all
mydestination = $myhostname, localhost, localhost.localdomain
home_mailbox = Maildir/
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_recipient_restrictions = permit_sasl_authenticated, reject
# Virtual aliases
# /etc/postfix/virtual
admin@example.com admin
support@example.com support@company.com
# Master.cf
submission inet n - y - - smtpd
-o syslog_name=postfix/submission
# Test
postfix check
postfix reload

Answer:

/etc/dovecot/dovecot.conf
# Install
apt install dovecot-imapd dovecot-pop3d
protocols = imap pop3
listen = *
base_dir = /var/run/dovecot/
# Authentication
# /etc/dovecot/conf.d/10-auth.conf
disable_plaintext_auth = yes
auth_mechanisms = plain login
# User database
# /etc/dovecot/conf.d/10-mail.conf
mail_location = maildir:~/Maildir
# PAM
passdb {
driver = pam
args = session_failure_delay=finite_secs
}
# SSL
# /etc/dovecot/conf.d/10-ssl.conf
ssl = required
ssl_cert = </etc/ssl/certs/dovecot.pem
ssl_key = </etc/ssl/private/dovecot.pem

Q1073: How do you configure spam filtering?

Section titled “Q1073: How do you configure spam filtering?”

Answer:

Terminal window
# Install SpamAssassin
apt install spamassassin spamc
# Configure Postfix
# /etc/postfix/master.cf
spamassassin unix - n n - - pipe
user=spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient}
# /etc/postfix/main.cf
content_filter = spamassassin
# Configure SpamAssassin
# /etc/spamassassin/local.cf
required_score 5.0
rewrite_header Subject [SPAM]
report_safe 0
# Add headers
add_header all Status _YESNO_, score=_SCORE_ required=_REQD_ tests=_TESTS_
add_header all Reply-To _ADDR_
# Train bayes
sa-learn --spam /var/virusmails/*
sa-learn --ham /var/mail/*

Answer:

/etc/postfix/main.cf
# Postfix relay configuration
relayhost = [smtp.example.com]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_tls_security_level = encrypt
smtp_tls_security_level = encrypt
smtp_tls_wrappermode = no
# /etc/postfix/sasl_passwd
[smtp.example.com]:587 username:password
# Secure the password file
chmod 600 /etc/postfix/sasl_passwd
postmap /etc/postfix/sasl_passwd
# Relay for specific domains
# /etc/postfix/transport
example.com :[smtp.example.com]
* smtp:relay.other.com
# Test
echo "Test email" | sendmail -v user@example.com

Q1075: How do you troubleshoot mail issues?

Section titled “Q1075: How do you troubleshoot mail issues?”

Answer:

Terminal window
# Check mail queue
mailq
postqueue -p
# View mail log
tail -f /var/log/mail.log
journalctl -u postfix -f
# Flush queue
postfix flush
postqueue -f
# Check specific mail
postcat -q <queue_id>
# Remove stuck mail
postsuper -d <queue_id>
postsuper -d ALL deferred
# Test SMTP
telnet mail.example.com 25
nc -C mail.example.com 25
# Check delivery
echo "Test" | mail -v user@example.com

Answer:

Terminal window
# Install
apt install slapd ldap-utils
# Reconfigure
dpkg-reconfigure slapd
# Add entries
# add_user.ldif
dn: uid=john,ou=people,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
uid: john
cn: John Doe
sn: Doe
givenName: John
uidNumber: 1000
gidNumber: 1000
homeDirectory: /home/john
mail: john@example.com
# Add entry
ldapadd -x -D "cn=admin,dc=example,dc=com" -W -f add_user.ldif
# Search
ldapsearch -x -b "dc=example,dc=com" "(uid=john)"
ldapsearch -x -H ldap://localhost -D "cn=admin,dc=example,dc=com" -W

Q1077: How do you integrate Linux with LDAP?

Section titled “Q1077: How do you integrate Linux with LDAP?”

Answer:

Terminal window
# Install client
apt install libnss-ldap libpam-ldap ldap-utils
# Configure NSS
# /etc/nsswitch.conf
passwd: compat ldap
group: compat ldap
shadow: compat ldap
hosts: files dns ldap
networks: files ldap
# Configure PAM
# /etc/pam.d/common-session
session optional pam_mkhomedir.so skel=/etc/skel umask=077
# Configure LDAP client
# /etc/ldap.conf
base dc=example,dc=com
uri ldap://ldap.example.com
ldap_version 3
rootbinddn cn=admin,dc=example,dc=com
# Test
getent passwd john
id john

Q1078: How do you configure LDAP replication?

Section titled “Q1078: How do you configure LDAP replication?”

Answer:

/etc/ldap/slapd.conf
# Master (provider) configuration
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 100
# Enable syncprov module
modulepath /usr/lib/ldap
moduleload syncprov
# Consumer (replica) configuration
# /etc/ldap/slapd.conf
syncrepl rid=123
provider=ldap://master.example.com:389
type=refreshAndPersist
searchbase="dc=example,dc=com"
bindmethod=simple
binddn="cn=admin,dc=example,dc=com"
credentials=secret
retry="60 +"
# Or use LDAP URL
syncrepl rid=123
provider=ldap://master.example.com:389
searchbase="dc=example,dc=com"
binddn="cn=syncuser,dc=example,dc=com"
credentials=password

Answer:

/etc/ldap/ldap.conf
# TLS/SSL configuration
TLS_CACERT /etc/ssl/certs/ca-certificates.crt
TLS_REQCERT demand
# Enable TLS in slapd
# /etc/default/slapd
SLAPD_SERVICES="ldap://localhost/ ldap://localhost:7389/ ldaps://localhost:636/"
# Generate certificates
openssl req -new -x509 -nodes -days 365 \
-keyout /etc/ldap/tls/ldap.key \
-out /etc/ldap/tls/ldap.crt
# Restrict access
# /etc/ldap/slapd.conf
access to dn.base=""
by * read
access to *
by self write
by dn="cn=admin,dc=example,dc=com" write
by * read

Q1080: How do you backup and restore LDAP?

Section titled “Q1080: How do you backup and restore LDAP?”

Answer:

Terminal window
# Backup database
slapcat -n 1 > backup.ldif
# or using ldapsearch
ldapsearch -x -LLL -b "dc=example,dc=com" > backup.ldif
# Restore
# Stop slapd
systemctl stop slapd
# Remove database
rm -rf /var/backups/ldap/*
# Restore
slapadd -l backup.ldif
# Set permissions
chown -R openldap:openldap /var/lib/ldap/
# Start slapd
systemctl start slapd
# Automate backup
#!/bin/bash
DATE=$(date +%Y%m%d)
slapcat -n 1 > /backup/ldap-$DATE.ldif
gzip /backup/ldap-$DATE.ldif
find /backup -mtime +30 -delete

Answer:

Terminal window
# Install (initiator)
apt install open-iscsi
# Discover targets
iscsiadm -m discovery -t st -p 192.168.1.10
# Login
iscsiadm -m node --targetname iqn.2010-01.com.example:storage.target1 --login
# Configure auto-login
iscsiadm -m node -p 192.168.1.10 -o update -n node.startup -v automatic
# View sessions
iscsiadm -m session
iscsiadm -m session -P 3
# Use device
fdisk -l /dev/sdb
mkfs.xfs /dev/sdb1
# Logout
iscsiadm -m node --targetname iqn.2010-01.com.example:storage.target1 --logout

Answer:

/etc/exports
# Install
apt install nfs-kernel-server
/data 192.168.1.0/24(rw,sync,no_subtree_check,no_root_squash)
/backup 192.168.1.10(rw,sync,all_squash,anonuid=1000,anongid=1000)
*(ro,async,no_subtree_check)
# Export
exportfs -a
exportfs -r
# Client mount
mount -t nfs 192.168.1.10:/data /mnt/data
# /etc/fstab
192.168.1.10:/data /mnt/data nfs defaults,_netdev 0 0
# Options
# rw/sync - read-write/synchronous
# no_subtree_check - disable subtree checking
# no_root_squash - allow root access

Answer:

/etc/samba/smb.conf
# Install
apt install samba
[global]
workgroup = WORKGROUP
server string = File Server
security = user
map to guest = bad user
dns proxy = no
[shared]
path = /srv/samba/shared
browsable = yes
writable = yes
guest ok = yes
read only = no
[private]
path = /srv/samba/private
valid users = @smbgroup
writable = yes
browseable = no
# Create user
useradd -m smbuser
smbpasswd -a smbuser
# Test config
testparm
# Client
mount -t cifs //server/share /mnt -o user=smbuser

Answer:

# Install
apt install ceph-mon ceph-osd ceph-mds
# Create cluster
ceph-deploy new mon1 osd1 osd2
# Deploy monitors
ceph-deploy mon create mon1
# Deploy OSDs
ceph-deploy osd create --data /dev/sdb1 mon1
ceph-deploy osd create --data /dev/sdb1 osd1
# Create filesystem
ceph osd pool create cephfs_data 128
ceph osd pool create cephfs_metadata 128
ceph fs new cephfs cephfs_metadata cephfs_data
# Mount
# Kernel
mount -t ceph mon1:6789:/ /mnt/ceph
# FUSE
ceph-fuse -n client.admin /mnt/ceph

Answer:

Terminal window
# Using lvmcache
lvcreate --type cache --cachevol cachevol --pool vg_data/lv_data vg_data/cachepool
# Using btrfs
mkfs.btrfs -d single -m single /dev/sda1 /dev/sdb1
# Using mdadm with SSD cache
mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/sd[a-c]1
mdadm --manage /dev/md0 --add /dev/sdd1
# ZFS
zpool create data mirror /dev/sda1 /dev/sdb1 cache /dev/sdc1
#查看状态
zpool status
zpool list
zfs get all

Q1086: How do you compile a custom Linux kernel?

Section titled “Q1086: How do you compile a custom Linux kernel?”

Answer:

Terminal window
# Install build dependencies
apt build-dep linux
apt install git bc bison flex libssl-dev
# Download source
git clone --depth 1 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
cd linux
# Configure
make menuconfig
# or
make xconfig
# or copy current config
cp /boot/config-$(uname -r) .config
make olddefconfig
# Build
make -j$(nproc)
make modules_install
make install
# Update boot loader
update-grub
# Reboot to new kernel
reboot

Answer:

Terminal window
# View parameters
sysctl -a
cat /proc/sys/.../...
# Set temporarily
sysctl -w net.ipv4.ip_forward=1
# Set permanently
# /etc/sysctl.conf
net.ipv4.ip_forward=1
vm.swappiness=10
net.core.somaxconn=1024
# Apply
sysctl -p
# For specific interface
sysctl -w net.ipv4.tcp_congestion_control=bbr
# Network performance
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.ipv4.tcp_rmem="4096 87380 16777216"
net.ipv4.tcp_wmem="4096 65536 16777216"

Answer:

Terminal window
# List modules
lsmod
modinfo module_name
# Load module
modprobe module_name
# Unload module
modprobe -r module_name
# Module parameters
modprobe module_name parameter=value
# Persistent configuration
# /etc/modprobe.d/blacklist.conf
blacklist module_name
# /etc/modprobe.d/module.conf
options module_name parameter=value
# Create module dependency
depmod -a
# View module info
modinfo -p module_name

Answer:

Terminal window
# Kernel messages
dmesg
dmesg | tail
dmesg -T | grep -i error
# Kernel panic
# Check /var/log/kern.log
tail -f /var/log/kern.log
# Kernel configuration
zcat /proc/config.gz
# or
cat /boot/config-$(uname -r)
# System calls
strace -c ./program
strace -e openat ls
# Kernel debugging
echo 1 > /proc/sys/kernel/debug/earlyprintk
echo "debug" > /sys/power/state
# OOM killer
dmesg | grep -i "out of memory"
cat /var/log/syslog | grep -i oom

Q1090: How do you secure the Linux kernel?

Section titled “Q1090: How do you secure the Linux kernel?”

Answer:

/etc/sysctl.conf
# Kernel hardening
kernel.dmesg_restrict=1
kernel.kptr_restrict=2
kernel.yama.ptrace_scope=2
kernel.sysrq=0
net.ipv4.conf.all.rp_filter=1
net.ipv4.conf.default.rp_filter=1
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.conf.all.accept_redirects=0
net.ipv4.conf.default.accept_redirects=0
# Disable unused filesystems
# /etc/modprobe.d/disable-filesystems.conf
install squashfs /bin/true
install udf /bin/true
# Disable IPv6 if not needed
# /etc/sysctl.conf
net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.default.disable_ipv6=1

Answer:

ansible-playbook.yml
---
- name: Configure webserver
hosts: webservers
become: yes
vars:
http_port: 80
tasks:
- name: Install Apache
apt:
name: apache2
state: present
when: ansible_os_family == "Debian"
- name: Start Apache
service:
name: apache2
state: started
enabled: yes
- name: Copy config
template:
src: templates/httpd.conf.j2
dest: /etc/apache2/apache2.conf
notify: restart apache
handlers:
- name: restart apache
service:
name: apache2
state: restarted

Q1092: How do you use Terraform for Linux infrastructure?

Section titled “Q1092: How do you use Terraform for Linux infrastructure?”

Answer:

main.tf
terraform {
required_version = ">= 1.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
provider "aws" {
region = "us-east-1"
}
resource "aws_instance" "web" {
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t3.micro"
subnet_id = aws_subnet.main.id
tags = {
Name = "webserver"
}
user_data = <<-EOF
#!/bin/bash
yum update -y
yum install -y httpd
systemctl start httpd
systemctl enable httpd
EOF
}
resource "aws_security_group" "web" {
name = "web-sg"
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}

Answer:

cookbook/recipes/default.rb
package 'httpd' do
package_name case node['platform']
when 'centos', 'redhat', 'amazon' then 'httpd'
when 'debian', 'ubuntu' then 'apache2'
end
action :install
end
service 'httpd' do
case node['platform']
when 'centos', 'redhat', 'amazon' then service_name 'httpd'
when 'debian', 'ubuntu' then service_name 'apache2'
end
action [:enable, :start]
end
template '/var/www/html/index.html' do
source 'index.html.erb'
mode '0644'
owner 'root'
group 'root'
end
# Run chef
chef-client --local-mode recipe.rb
# or
knife solo bootstrap user@server

Answer:

manifests/site.pp
node 'webserver.example.com' {
# Install package
package { 'apache2':
ensure => installed,
}
# Manage service
service { 'apache2':
ensure => running,
enable => true,
require => Package['apache2'],
}
# Manage file
file { '/var/www/html/index.html':
ensure => file,
content => template('webserver/index.html.erb'),
mode => '0644',
require => Service['apache2'],
}
}
# Run
puppet agent --test
# or
puppet apply manifests/site.pp

Answer:

# Vagrantfile
Vagrant.configure("2") do |config|
config.vm.box = "ubuntu/jammy64"
config.vm.network "private_network", ip: "192.168.33.10"
config.vm.network "forwarded_port", guest: 80, host: 8080
config.vm.synced_folder "./data", "/vagrant_data"
config.vm.provider "virtualbox" do |vb|
vb.memory = "2048"
vb.cpus = 2
end
config.vm.provision "shell", inline: <<-SHELL
apt update
apt install -y apache2
SHELL
end
# Commands
vagrant up
vagrant ssh
vagrant halt
vagrant destroy
vagrant provision

Q1096: How do you troubleshoot slow systems?

Section titled “Q1096: How do you troubleshoot slow systems?”

Answer:

Terminal window
# CPU usage
top
htop
ps aux --sort=-%cpu | head
# Memory
free -h
vmstat 1
pmap -x <pid>
# I/O
iostat -xz 1
iotop
sar -b 1
# Network
netstat -i
ss -s
# Process analysis
strace -c <command>
perf top
# System resources
uptime
cat /proc/loadavg

Q1097: How do you troubleshoot disk space issues?

Section titled “Q1097: How do you troubleshoot disk space issues?”

Answer:

Terminal window
# Disk usage
df -h
df -i
# Largest directories
du -sh /*
du -sh /var/*
du -shx /var/* | sort -rh | head
# Largest files
find / -type f -size +100M -exec ls -lh {} \; 2>/dev/null | sort -k5 -h
# Log files
journalctl --disk-usage
du -sh /var/log/*
find /var/log -type f -mtime +7 -delete
# Deleted but open files
lsof +L1
ls -l /proc/*/fd/* | grep deleted

Q1098: How do you troubleshoot network issues?

Section titled “Q1098: How do you troubleshoot network issues?”

Answer:

Terminal window
# Interface status
ip link
ip addr
ethtool eth0
# Routing
ip route
ip route get 8.8.8.8
# DNS
getent hosts example.com
dig +trace example.com
# Connectivity
ping -c 4 8.8.8.8
traceroute 8.8.8.8
mtr 8.8.8.8
# Ports
ss -tulpn
netstat -tulpn
# Firewall
iptables -L -n -v
firewall-cmd --list-all
# Traffic capture
tcpdump -i eth0 host 192.168.1.1
tcpdump -i eth0 port 80

Q1099: How do you troubleshoot service failures?

Section titled “Q1099: How do you troubleshoot service failures?”

Answer:

Terminal window
# Service status
systemctl status service-name
journalctl -u service-name -n 50
journalctl -u service-name --since "1 hour ago"
# Service logs
cat /var/log/service/name.log
tail -f /var/log/syslog | grep service
# Configuration test
apache2ctl configtest
nginx -t
named-checkconf
# Check permissions
ls -la /etc/service/
ls -l /var/run/service/
# Dependencies
systemctl list-dependencies service-name
systemctl daemon-reload
# Process issues
ps aux | grep service
lsof -p <pid>
strace -p <pid>

Q1100: How do you troubleshoot performance bottlenecks?

Section titled “Q1100: How do you troubleshoot performance bottlenecks?”

Answer:

Terminal window
# Overall system
top
htop
atop
sar -A 1 5
# CPU
mpstat -P ALL 1
pidstat -p <pid> 1
# Memory
pmap -x <pid>
cat /proc/<pid>/status
# I/O
iostat -xz 1
pidstat -d 1
# Network
nethogs
iftop
sar -n DEV 1
# Application
perf record -g -p <pid>
perf report
# System calls
strace -c -p <pid>
strace -tt -p <pid>

Q1101: How do you configure AWS EC2 instance?

Section titled “Q1101: How do you configure AWS EC2 instance?”

Answer:

Terminal window
# Install AWS CLI
apt install awscli
# or
pip install awscli
# Configure
aws configure
# AWS Access Key ID: ***
# AWS Secret Access Key: ***
# Default region name: us-east-1
# Default output format: json
# EC2 commands
aws ec2 describe-instances
aws ec2 start-instances --instance-ids i-1234567890abcdef0
aws ec2 stop-instances --instance-ids i-1234567890abcdef0
# Instance metadata
curl http://169.254.169.254/latest/meta-data/
curl http://169.254.169.254/latest/user-data/
# Install SSM Agent
apt install amazon-ssm-agent

answer:

cloud-config.yaml
#cloud-config
package_update: true
packages:
- nginx
- curl
write_files:
- path: /var/www/html/index.html
content: |
<html>
<h1>Hello from Cloud-Init</h1>
</html>
owner: root:root
permissions: '0644'
runcmd:
- systemctl enable nginx
- systemctl start nginx
- echo "192.168.1.10 webserver" >> /etc/hosts
users:
- name: admin
sudo: ALL=(ALL) NOPASSWD:ALL
ssh_authorized_keys:
- ssh-rsa AAAA...

Q1103: How do you configure Docker on cloud?

Section titled “Q1103: How do you configure Docker on cloud?”

Answer:

Terminal window
# Install Docker
curl -fsSL https://get.docker.com | sh
# Configure Docker daemon
# /etc/docker/daemon.json
{
"storage-driver": "overlay2",
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"live-restore": true,
"default-address-pools": [
{"base": "172.17.0.0/16", "size": 24}
]
}
# Enable Docker service
systemctl enable docker
systemctl start docker
# Docker swarm (for multi-host)
docker swarm init
docker node ls

Q1104: How do you configure Kubernetes on Linux?

Section titled “Q1104: How do you configure Kubernetes on Linux?”

Answer:

Terminal window
# Install kubeadm
apt-get update && apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee /etc/apt/sources.list.d/kubernetes.list
apt-get update
apt-get install -y kubelet kubeadm kubectl
# Initialize cluster
kubeadm init --pod-network-cidr=10.244.0.0/16
# Join nodes
kubeadm join <master-ip>:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>
# Install network plugin
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
# Deploy application
kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80 --type=LoadBalancer

Q1105: How do you configure load balancer in cloud?

Section titled “Q1105: How do you configure load balancer in cloud?”

Answer:

Terminal window
# AWS Application Load Balancer
aws elbv2 create-load-balancer \
--name my-alb \
--subnets subnet-12345678 subnet-87654321 \
--security-groups sg-12345678
# Target group
aws elv2 create-target-group \
--name my-targets \
--protocol HTTP \
--port 80 \
--vpc vpc-12345678
# Register targets
aws elv2 register-targets \
--target-group-arn arn:aws:elasticloadbalancing:... \
--targets Id=i-12345678
# Listener
aws elv2 create-listener \
--load-balancer-arn arn:aws:elasticloadbalancing:... \
--protocol HTTP \
--port 80 \
--default-actions Type=forward,TargetGroupArn=arn:aws:...
# HAProxy load balancer (on-premise)
# See HAProxy configuration earlier

Q1106: How do you automate security updates?

Section titled “Q1106: How do you automate security updates?”

Answer:

Terminal window
# Install unattended-upgrades (Debian/Ubuntu)
apt install unattended-upgrades
# Configure
# /etc/apt/apt.conf.d/50unattended-upgrades
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}-security";
};
Unattended-Upgrade::Package-blacklist {
"vim";
"apache2";
};
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "02:00";
# Enable
dpkg-reconfigure -plow unattended-upgrades
# RHEL/CentOS
yum install yum-cron
# /etc/yum/yum-cron.conf
apply_updates = yes
download_updates = yes
# Test
unattended-upgrades --dry-run --debug

Answer:

Terminal window
# Using kpatch (RHEL/CentOS)
yum install kpatch
kpatch install
# Build patch
kpatch build patch.diff
# Apply
kpatch load kpatch-mypatch.ko
# Check
kpatch list
# Using livepatch (Ubuntu)
snap install canonical-livepatch
canonical-livepatch enable <token>
# Check status
canonical-livepatch status

Q1108: How do you manage package repositories?

Section titled “Q1108: How do you manage package repositories?”

Answer:

/etc/apt/sources.list
# Debian/Ubuntu
deb http://archive.ubuntu.com/ubuntu/ jammy main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ jammy-updates main restricted universe multiverse
# Add repository
add-apt-repository ppa:nginx/stable
# Update
apt update
# RHEL/CentOS
# /etc/yum.repos.d/*.repo
[baseos]
name=BaseOS
baseurl=https://dl.fedoraproject.org/pub/epel/$releasever/Everything/$basearch
enabled=1
# Clean cache
apt clean
yum clean all
# List repos
apt-cache policy
yum repolist

Answer:

Terminal window
# Debian/Ubuntu
# Hold package
apt-mark hold apache2
# View installed versions
apt-cache policy nginx
# Downgrade
apt install nginx=1.18.0-*
# Snapshots with apt
apt install apt-clone
apt-clone clone myserver-packages
apt-clone restore myserver-packages.tar.gz
# RPM rollback (RHEL)
# yum history
yum history
yum history undo <transaction-id>
# Transactional updates (openSUSE)
transactional-update

Q1110: How do you test updates in staging?

Section titled “Q1110: How do you test updates in staging?”

Answer:

Terminal window
# Create test environment
vagrant up staging
# Run tests
# In staging environment
apt update
DEBIAN_FRONTEND=noninteractive apt upgrade -y
# Test application
curl http://localhost
systemctl status myapp
journalctl -u myapp -n 50
# Check logs
tail -f /var/log/syslog
# If issues
# Rollback
vagrant destroy staging
vagrant up staging
# Production update with backup
# Before update
tar -czf /backup/$(hostname)-$(date +%Y%m%d).tar.gz /etc /var/www
# Run update
apt update && apt upgrade -y
# If failed
tar -xzf /backup/$(hostname)-backup.tar.gz -C /

Q1111: How do you implement CIS benchmarks?

Section titled “Q1111: How do you implement CIS benchmarks?”

Answer:

Terminal window
# Install CIS benchmark tool
apt install lynis
# Run audit
lynis audit system
lynis audit --profile cis-ubuntu-22.04
# Key CIS controls
# 1.1.1 Disable unused filesystems
echo "install cramfs /bin/true" > /etc/modprobe.d/cramfs.conf
echo "install squashfs /bin/true" > /etc/modprobe.d/squashfs.conf
# 1.5.1 Set bootloader password
grub-mkpasswd-pbkdf2
# Add to /etc/grub.d/40_custom
# 3.1 Enable syncookies
sysctl -w net.ipv4.tcp_syncookies=1
# 4.1 Configure auditd
# See auditd configuration earlier
# Generate report
lynis audit system --html > report.html

Answer:

Terminal window
# Install
apt install auditd
# Configure rules
# /etc/audit/audit.rules
# Watch files
-w /etc/passwd -p wa -k passwd_changes
-w /etc/shadow -p wa -k shadow_changes
-w /etc/sudoers -p wa -k sudoers_changes
# Watch commands
-a always,exit -F arch=b64 -S execve -F path=/usr/bin/wget -k network-download
-a always,exit -F arch=b64 -S execve -F path=/usr/bin/curl -k network-download
# Watch directories
-w /etc/httpd/conf/ -p wa -k httpd_conf
# Services
-w /usr/sbin/service -p x -k service_management
# Generate report
aureport --summary
aureport --file
aureport --terminal

Answer:

Terminal window
# Enable FIPS (RHEL/CentOS)
fips-mode-setup --enable
# Configure OpenSSL for FIPS
# /etc/ssl/openssl.cnf
openssl_conf = openssl_init
[openssl_init]
ssl_conf = ssl_sect
[ssl_sect]
system_default = system_default_sect
[system_default_sect]
CipherString = DEFAULT@SECLEVEL=2
# Check FIPS status
cat /proc/sys/crypto/fips_enabled
# Use FIPS certified algorithms
# OpenSSL
openssl ciphers -v 'FIPS'
# SSH
# /etc/ssh/sshd_config
Ciphers aes256-ctr,aes192-ctr,aes128-ctr
MACs hmac-sha2-512,hmac-sha2-256
KexAlgorithms diffie-hellman-group-exchange-sha256

Answer:

Terminal window
# Install
apt install aide
# Configure
# /etc/aide/aide.conf
# Database location
database=file:/var/lib/aide/aide.db
database_out=file:/var/lib/aide/aide.db.new
# Rules
Full = p+i+n+u+g+s+m+c+md5+sha256
Lsof = p+u+g+i+n+S
# Files to monitor
/etc p+inode+u+g+i+m+c+md5+sha256
/bin p+inode+u+g+i+m+c+md5+sha256
/sbin p+inode+u+g+i+m+c+md5+sha256
/usr p+inode+u+g+i+m+c+md5+sha256
# Initialize database
aideinit
# Check integrity
aide --check
aide --update
# Schedule
# /etc/cron.d/aide
0 5 * * * root /usr/bin/aide --check

Q1115: How do you implement network segmentation?

Section titled “Q1115: How do you implement network segmentation?”

Answer:

Terminal window
# VLAN isolation
ip link add link eth0 name eth0.100 type vlan id 100
ip addr add 192.168.100.1/24 dev eth0.100
# iptables zones
iptables -N DMZ
iptables -A DMZ -j DROP
iptables -A FORWARD -i eth0 -o eth1 -j ACCEPT
# Network namespaces for isolation
ip netns add isolated
ip netns exec isolated ip link set lo up
# Cgroups for process isolation
# /etc/cgconfig.conf
group web {
cpu {
cpu.shares=512;
}
memory {
memory.limit_in_bytes=512M;
}
}
# AppArmor/SELinux
apparmor_parser -r /etc/apparmor.d/*

Q1116: How do you configure caching proxy?

Section titled “Q1116: How do you configure caching proxy?”

Answer:

/etc/squid/squid.conf
# Install Squid
apt install squid
# Basic config
http_port 3128
cache_dir ufs /var/spool/squid 1000 16 256
# Access control
acl localnet src 192.168.0.0/16
http_access allow localnet
http_access deny all
# Cache rules
refresh_pattern -i \.jpg$ 10080 90% 43200
refresh_pattern -i \.html$ 1440 90% 3600
refresh_pattern -i \.css$ 10080 90% 43200
refresh_pattern -i \.js$ 10080 90% 43200
# Transparent proxy
http_port 3128 transparent
# Test
squid -k parse
systemctl restart squid

Q1117: How do you configure reverse proxy?

Section titled “Q1117: How do you configure reverse proxy?”

Answer:

/etc/nginx/sites-available/reverse-proxy
# Nginx reverse proxy
server {
listen 80;
server_name api.example.com;
location / {
proxy_pass http://backend:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /admin {
proxy_pass http://admin-backend:8081;
auth_basic "Admin Area";
auth_basic_user_file /etc/nginx/.htpasswd;
}
}
# Cache with nginx
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=api_cache:10m max_size=1g inactive=60m;
location / {
proxy_cache api_cache;
proxy_cache_valid 200 60m;
proxy_pass http://backend:8080;
}

Q1118: How do you configure caching server?

Section titled “Q1118: How do you configure caching server?”

Answer:

/etc/varnish/default.vcl
# Install Varnish
apt install varnish
vcl 4.1;
backend default {
.host = "127.0.0.1";
.port = "8080";
.probe = {
.url = "/health";
.timeout = 5s;
.interval = 10s;
}
}
sub vcl_recv {
# Don't cache admin pages
if (req.url ~ "^/admin") {
return (pass);
}
# Don't cache POST requests
if (req.method == "POST") {
return (pass);
}
}
sub vcl_backend_response {
# Cache static files
if (bereq.url ~ "\.(jpg|jpeg|png|gif|ico|css|js)$") {
set beresp.ttl = 24h;
}
}
# Commands
varnishd -F -f /etc/varnish/default.vcl
varnishstat
varnishlog

Q1119: How do you configure web server tuning?

Section titled “Q1119: How do you configure web server tuning?”

Answer:

/etc/apache2/mods-enabled/mpm_prefork.conf
# Apache tuning
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxRequestWorkers 150
MaxConnectionsPerChild 0
</IfModule>
# Nginx tuning
# /etc/nginx/nginx.conf
worker_processes auto;
worker_rlimit_nofile 65535;
events {
worker_connections 65535;
multi_accept on;
use epoll;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
keepalive_requests 1000;
open_file_cache max=10000 inactive=30s;
open_file_cache_valid 60s;
open_file_cache_min_uses 2;
gzip on;
gzip_types text/plain text/css application/json application/javascript;
}

Q1120: How do you configure load balancing algorithms?

Section titled “Q1120: How do you configure load balancing algorithms?”

Answer:

Terminal window
# HAProxy algorithms
# Round Robin (default)
backend servers
balance roundrobin
server s1 192.168.1.10:80 check
server s2 192.168.1.11:80 check
# Least Connections
backend servers
balance leastconn
server s1 192.168.1.10:80 check
server s2 192.168.1.11:80 check
# Source IP Hash
backend servers
balance source
server s1 192.168.1.10:80 check
server s2 192.168.1.11:80 check
# URI Hash
backend servers
balance uri
server s1 192.168.1.10:80 check
server s2 192.168.1.11:80 check
# Weighted
backend servers
balance roundrobin
server s1 192.168.1.10:80 weight 3 check
server s2 192.168.1.11:80 weight 1 check

Answer:

/etc/sysctl.conf
# TCP buffer sizes
net.core.rmem_default=262144
net.core.rmem_max=16777216
net.core.wmem_default=262144
net.core.wmem_max=16777216
# TCP settings
net.ipv4.tcp_rmem=4096 87380 16777216
net.ipv4.tcp_wmem=4096 65536 16777216
net.ipv4.tcp_congestion_control=cubic
net.ipv4.tcp_fastopen=3
net.ipv4.tcp_max_syn_backlog=8192
# TCP performance
net.core.netdev_max_backlog=65535
net.ipv4.tcp_fin_timeout=15
# TCP keepalive
net.ipv4.tcp_keepalive_time=600
net.ipv4.tcp_keepalive_intvl=60
net.ipv4.tcp_keepalive_probes=5
# Apply
sysctl -p

Answer:

/etc/sysctl.conf
# Swappiness
vm.swappiness=10
vm.vfs_cache_pressure=50
# Memory management
vm.dirty_ratio=15
vm.dirty_background_ratio=5
vm.dirty_expire_centisecs=3000
vm.dirty_writeback_centisecs=500
# Overcommit
vm.overcommit_memory=1
vm.overcommit_ratio=50
# Huge pages
vm.nr_hugepages=512
# Apply
sysctl -p
# Configure huge pages
echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
# Transparent huge pages
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

Answer:

Terminal window
# I/O Scheduler
# Check current
cat /sys/block/sda/queue/scheduler
# Set deadline scheduler
echo deadline > /sys/block/sda/queue/scheduler
echo cfq > /sys/block/sda/queue/scheduler
# Make permanent
# /etc/udev/rules.d/60-ioschedulers.rules
ACTION=="add|change", KERNEL=="sda", SUBSYSTEM=="block", ATTR{queue/scheduler}="deadline"
# Block device settings
echo 4096 > /sys/block/sda/queue/read_ahead_kb
echo 0 > /sys/block/sda/queue/rotational
echo 2 > /sys/block/sda/queue/rq_affinity
# Filesystem options
# /etc/fstab
/dev/sda1 / ext4 noatime,nodiratime,errors=remount-ro 0 1

Answer:

/etc/security/limits.conf
# Max open files
* soft nofile 65535
* hard nofile 65535
root soft nofile 65535
root hard nofile 65535
# Max processes
* soft nproc 4096
* hard nproc 8192
# Core dumps
* soft core 0
* hard core unlimited
# Locked memory
* soft memlock unlimited
* hard memlock unlimited
# Apply without logout
ulimit -n 65535
# View limits
ulimit -a
cat /proc/<pid>/limits

Q1125: How do you optimize network throughput?

Section titled “Q1125: How do you optimize network throughput?”

Answer:

# Network card offloading
ethtool -K eth0 tso on
ethtool -K eth0 gso on
ethtool -K eth0 gro on
ethtool -K eth0 rx on
ethtool -K eth0 tx on
# Ring buffer
ethtool -G eth0 rx 4096 tx 4096
# Interrupt coalescing
ethtool -C eth0 rx-usecs 100 tx-usecs 100
# Flow control
ethtool -A eth0 rx on tx on
# Bonding for throughput
# See earlier section on network bonding
# Multi-queue
# Check
cat /sys/class/net/eth0/queues/rx-0/rps_cpus
# Set
echo ffffff > /sys/class/net/eth0/queues/rx-0/rps_cpus

Answer:

/etc/rsyslog.conf
# Modules
$ModLoad imudp
$UDPServerRun 514
$ModLoad imtcp
$InputTCPServerRun 514
# Templates
$template RemoteLogs,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log"
$template DetailedLogs,"/var/log/%HOSTNAME%/%$year%/%$month%/%$day%/detail.log"
# Rules
# Log everything to remote server
*.* @@remote-server:514
# Local logging
*.info;mail.none;authpriv.none /var/log/messages
authpriv.* /var/log/secure
mail.* -/var/log/maillog
cron.* /var/log/cron
# Filter by program
:programname, isequal, "apache" /var/log/apache.log
# Stop processing
& stop

Answer:

/etc/systemd/journald.conf
[Journal]
Storage=persistent
Compress=yes
SystemMaxUse=500M
SystemMaxFileSize=50M
MaxRetentionSec=30day
# Forward to syslog
ForwardToSyslog=yes
ForwardToKMsg=no
ForwardToWall=no
# Rate limiting
RateLimitIntervalSec=30s
RateLimitBurst=1000
# View logs
journalctl
journalctl -u nginx
journalctl --since "2024-01-01"
journalctl --since "1 hour ago"
journalctl -p err
journalctl -f
# Persistent storage
mkdir -p /var/log/journal
systemd-tmpfiles --create --prefix /var/log/journal

Answer:

/etc/filebeat/filebeat.yml
# ELK Stack
# Filebeat on clients
filebeat.inputs:
- type: log
paths:
- /var/log/*.log
fields:
type: syslog
output.logstash:
hosts: ["logstash:5044"]
# Logstash config
# /etc/logstash/conf.d/01-input.conf
input {
beats {
port => 5044
}
}
# Filter
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
}
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}

Answer:

/etc/logrotate.conf
# Global settings
daily
rotate 14
compress
delaycompress
missingok
notifempty
create 0640 root adm
# Include configs
include /etc/logrotate.d/
# Specific config
# /etc/logrotate.d/nginx
/var/log/nginx/*.log {
daily
missingok
rotate 52
compress
delaycompress
notifempty
create 0640 www-data adm
sharedscripts
postrotate
invoke-rc.d nginx rotate >/dev/null 2>&1
endscript
}
# Test
logrotate -d /etc/logrotate.conf
logrotate -f /etc/logrotate.d/nginx

Q1130: How do you analyze logs efficiently?

Section titled “Q1130: How do you analyze logs efficiently?”

Answer:

Terminal window
# Using grep
grep -i error /var/log/syslog | head -20
grep -v "INFO" /var/log/app.log
# Using awk
awk '/ERROR/ {print $1, $5}' /var/log/app.log
awk '{print $NF}' /var/log/access.log | sort | uniq -c | sort -rn
# Using cut
cut -d' ' -f1 /var/log/access.log | sort | uniq -c
# Using logrotate with logwatch
apt install logwatch
# Using GoAccess
goaccess /var/log/nginx/access.log -o /var/www/html/report.html
# Using lnav
lnav /var/log/syslog
lnav /var/log/*.log

Q1131: How do you configure incremental backups?

Section titled “Q1131: How do you configure incremental backups?”

Answer:

incremental-backup.sh
#!/bin/bash
SOURCE="/data"
BACKUP="/backup"
DATE=$(date +%Y%m%d)
# Full backup on Sunday
if [ $(date +%w) -eq 0 ]; then
echo "Full backup"
rm -rf $BACKUP/full
cp -al $SOURCE $BACKUP/full
else
# Incremental backup
echo "Incremental backup"
rm -rf $BACKUP/incremental_$DATE
cp -al $BACKUP/full $BACKUP/incremental_$DATE
cd $BACKUP/full
rsync -a --delete --link-dest=../incremental_$DATE $SOURCE/ .
fi
# Restore
# rsync -a --delete incremental_20240115/ /data/

Answer:

backup-mysql.sh
#!/bin/bash
DB_NAME="mydb"
DB_USER="backup"
DB_PASS="password"
BACKUP_DIR="/backup/mysql"
DATE=$(date +%Y%m%d_%H%M%S)
# MySQL backup
mysqldump -u$DB_USER -p$DB_PASS --single-transaction --routines --triggers $DB_NAME | gzip > $BACKUP_DIR/${DB_NAME}_${DATE}.sql.gz
# PostgreSQL backup
pg_dump -U $DB_USER -F c -b -v -f $BACKUP_DIR/${DB_NAME}_${DATE}.dump $DB_NAME
# Retention
find $BACKUP_DIR -name "*.sql.gz" -mtime +7 -delete
find $BACKUP_DIR -name "*.dump" -mtime +7 -delete
# Verify
zcat $BACKUP_DIR/${DB_NAME}_${DATE}.sql.gz | head -5
# Restore
# mysql -u$DB_USER -p$DB_PASS $DB_NAME < backup.sql
# pg_restore -U $DB_USER -d $DB_NAME backup.dump

Q1133: How do you configure remote backup?

Section titled “Q1133: How do you configure remote backup?”

Answer:

remote-backup.sh
#!/bin/bash
SOURCE="/data"
REMOTE="backup@remote.server:/backups/$(hostname)"
DATE=$(date +%Y%m%d)
# Rsync over SSH
rsync -avz --delete \
-e "ssh -i /root/.ssh/backup_key" \
--exclude='*.tmp' \
$SOURCE/ $REMOTE/daily/
# With compression
rsync -avz --delete -e ssh $SOURCE/ user@remote:/backup/
# Incremental with link-dest
rsync -avz --delete --link-dest=../last $SOURCE/ user@remote:/backup/$DATE/
# Verify
rsync -avnc --delete $SOURCE/ user@remote:/backup/

Answer:

Terminal window
# Check backup file integrity
# Compressed files
gzip -t backup.tar.gz
bzip2 -t backup.tar.bz2
# Checksums
sha256sum backup.tar.gz > backup.sha256
sha256sum -c backup.sha256
# Verify MySQL backup
mysqlcheck -u root -p --all-databases
# or
mysql -u root -p -e "source backup.sql"
# Verify PostgreSQL backup
pg_restore --list backup.dump | head
# Test restore in VM
vagrant up test
vagrant ssh test -c "mysql -u root -p mydb < /vagrant/backup.sql"
vagrant ssh test -c "curl localhost"
vagrant destroy test
# Automated verification
#!/bin/bash
if ! tar -tzf /backup/backup.tar.gz >/dev/null 2>&1; then
echo "Backup is corrupted!"
mail -s "Backup Failed" admin@example.com
fi

Answer:

Terminal window
# Document everything
# 1. Hardware inventory
lshw > inventory/$(hostname)-hardware.txt
lspci >> inventory/$(hostname)-hardware.txt
# 2. Software inventory
dpkg -l > inventory/$(hostname)-packages.txt
# 3. Network configuration
ip addr show > network/$(hostname)-interfaces.txt
route -n >> network/$(hostname)-routes.txt
# 4. Services configuration
tar -czf configs-$(hostname).tar.gz /etc/
# 5. Create recovery runbook
# Step 1: Boot from rescue media
# Step 2: Verify hardware
# Step 3: Recreate partitions
# Step 4: Restore OS
# Step 5: Install packages
# Step 6: Restore configurations
# Step 7: Restore data
# Step 8: Start services
# Test in DR site
vagrant up dr-test
# Run recovery procedures

Answer:

Terminal window
# Traffic shaping with tc
# Limit outgoing bandwidth
tc qdisc add dev eth0 root handle 1: htb default 10
tc class add dev eth0 parent 1: classid 1:10 htb rate 100mbit ceil 100mbit
tc class add dev eth0 parent 1: classid