One_liners

Chapter 34: Common Bash One-liners

Overview

This chapter covers essential bash one-liners that every DevOps engineer should know. These commands are invaluable for system administration, log analysis, and automation.

---

File Operations

Basic File Operations

# Count lines in file
wc -l file.txt

# Count words in file
wc -w file.txt

# Count characters
wc -c file.txt

# Get file size in human-readable format
ls -lh file.txt

# Find largest files in directory
ls -lS | head -10

# Find files modified in last N days
find /path -mtime -N

# Find files larger than N MB
find /path -size +NM

# Remove empty files/directories
find /path -type f -empty -delete

---

Text Processing

grep and Search

# Search in files
grep -r "pattern" /path

# Case-insensitive search
grep -ri "pattern" /path

# Show line numbers
grep -n "pattern" file.txt

# Show context (before, after, around)
grep -B3 -A3 "pattern" file.txt

# Count matches
grep -c "pattern" file.txt

# Show only matching part
grep -o "pattern" file.txt

# Use regex
grep -E "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" file.txt

sed One-liners

# Replace text
sed -i 's/old/new/g' file.txt

# Delete lines containing pattern
sed -i '/pattern/d' file.txt

# Insert line before pattern
sed -i '/pattern/i\New line' file.txt

# Print specific lines
sed -n '5p' file.txt
sed -n '5,10p' file.txt

# Remove trailing whitespace
sed -i 's/[[:space:]]*$//' file.txt

awk One-liners

# Print specific column
awk '{print $1}' file.txt

# Print multiple columns
awk '{print $1, $3}' file.txt

# Print last column
awk '{print $NF}' file.txt

# Sum column
awk '{sum+=$1} END {print sum}' file.txt

# Filter by condition
awk '$3 > 100' file.txt

# Print with format
awk '{printf "%-10s %5d\n", $1, $2}' file.txt

---

System Information

Hardware and OS

# Show system information
uname -a

# Show kernel version
uname -r

# Show CPU info
lscpu

# Show memory info
free -h

# Show disk usage
df -h

# Show disk usage of current directory
du -sh .

# Show disk usage of subdirectories
du -h --max-depth=1

Process Management

# Show processes
ps aux

# Show processes by name
pgrep -a process_name

# Kill process by name
pkill process_name

# Kill process by ID
kill $(pgrep process_name)

# Show top processes by memory
ps aux --sort=-%mem | head -10

# Show top processes by CPU
ps aux --sort=-%cpu | head -10

---

Network Operations

Network Diagnostics

# Show IP address
ip addr show

# Show routing table
ip route

# Test connectivity
ping -c 4 google.com

# Check open ports
ss -tulpn

# Show network connections
netstat -ant

# DNS lookup
dig example.com

# Check port connectivity
nc -zv host port

---

Log Analysis

Common Log Patterns

# Count HTTP status codes
awk '{print $9}' access.log | sort | uniq -c | sort -rn

# Find 5xx errors
grep " 5[0-9][0-9] " access.log

# Find specific IP
grep "192.168.1.100" access.log

# Extract timestamps
awk '{print $4, $5}' access.log

# Count unique IPs
awk '{print $1}' access.log | sort -u | wc -l

# Show most visited URLs
awk '{print $7}' access.log | sort | uniq -c | sort -rn | head -10

---

Package Management (Arch Linux)

pacman Commands

# Sync databases
sudo pacman -Sy

# Upgrade system
sudo pacman -Syu

# Install package
sudo pacman -S package_name

# Remove package
sudo pacman -R package_name

# Remove with dependencies
sudo pacman -Rns package_name

# Search for package
pacman -Ss search_term

# List installed packages
pacman -Q

# List orphan packages
pacman -Qdt

AUR Helpers

# Using yay
yay -S package_name
yay -R package_name
yay -Ss search_term

# Using paru
paru -S package_name

---

Docker Commands

Common Docker Operations

# List containers
docker ps -a

# List images
docker images

# Remove stopped containers
docker container prune -f

# Remove unused images
docker image prune -f

# Remove unused volumes
docker volume prune -f

# Show container logs
docker logs -f container_name

# Show container stats
docker stats --no-stream

# Execute command in container
docker exec -it container_name /bin/bash

---

Git Commands

Common Git Operations

# Show recent commits
git log --oneline -10

# Show changed files
git diff --name-only

# Show branch
git branch

# Show remote
git remote -v

# Show status
git status

# Add all changes
git add -A

# Create branch
git checkout -b branch_name

---

Text Manipulation

Useful One-liners

# Sort and get unique
sort file.txt | uniq

# Count unique lines
sort -u file.txt | wc -l

# Reverse lines
tac file.txt

# Randomize lines
shuf file.txt

# Extract column from CSV
cut -d',' -f1 file.csv

# Merge two files
paste file1.txt file2.txt

# Split file into chunks
split -l 1000 file.txt chunk_

# Convert to uppercase
tr '[:lower:]' '[:upper:]' < file.txt

# Remove duplicate lines
awk '!seen[$0]++' file.txt

---

Date and Time

Date Operations

# Current date
date "+%Y-%m-%d"

# Current time
date "+%H:%M:%S"

# Current timestamp
date +%s

# Date from timestamp
date -d @timestamp

# Yesterday
date -d "yesterday"

# Tomorrow
date -d "tomorrow"

# Add days
date -d "+7 days"

# Format for filenames
date +%Y%m%d_%H%M%S

---

Variables and Arrays

Variable Manipulation

# Get string length
${#variable}

# Extract substring
${variable:position:length}

# Replace pattern
${variable//old/new}

# Remove pattern
${variable#pattern}
${variable##pattern}

---

Loop Examples

For Loops

# Loop through files
for f in *.txt; do echo "$f"; done

# Loop through numbers
for i in {1..10}; do echo "$i"; done

# Loop through array
for item in "${array[@]}"; do echo "$item"; done

# Loop through command output
for line in $(command); do echo "$line"; done

---

Summary

In this chapter, you learned:

• ✅ File operation one-liners
• ✅ Text processing (grep, sed, awk)
• ✅ System information commands
• ✅ Process management
• ✅ Network operations
• ✅ Log analysis
• ✅ Package management (Arch Linux)
• ✅ Docker commands
• ✅ Git commands
• ✅ Text manipulation
• ✅ Date and time operations
• ✅ Variable manipulation
• ✅ Loop examples

---

Next Steps

Continue to the next chapter to learn about Code Style and Organization.

---

Previous Chapter: Best Practices Next Chapter: Code Style and Organization