Network_troubleshooting
Chapter 24: Network Troubleshooting
Section titled “Chapter 24: Network Troubleshooting”Overview
Section titled “Overview”Network troubleshooting is an essential skill for Linux system administrators and DevOps engineers. This chapter covers comprehensive diagnostic tools, systematic troubleshooting approaches, common network issues and their solutions, and practical workflows for diagnosing and fixing network problems in production environments.
24.1 Troubleshooting Tools
Section titled “24.1 Troubleshooting Tools”Network Diagnostic Tools
Section titled “Network Diagnostic Tools”┌─────────────────────────────────────────────────────────────────────────┐│ NETWORK TROUBLESHOOTING TOOLS │├─────────────────────────────────────────────────────────────────────────┤│ ││ Connectivity Tests: ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ ping - ICMP echo test │ ││ │ ping6 - IPv6 ping │ ││ │ arping - ARP ping (layer 2) │ ││ │ fping - Multiple hosts │ ││ │ hping3 - Advanced packet crafting │ ││ └─────────────────────────────────────────────────────────────────┘ ││ ││ Path Analysis: ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ traceroute - Trace route (UDP/ICMP) │ ││ │ tracepath - Trace with MTU discovery │ ││ │ mtr - Combined ping + traceroute │ ││ │ tcptraceroute - TCP traceroute │ ││ │ nmap - Port scanning + more │ ││ └─────────────────────────────────────────────────────────────────┘ ││ ││ DNS Tools: ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ dig - DNS lookup utility │ ││ │ nslookup - Interactive DNS client │ ││ │ host - Simple DNS queries │ ││ │ resolvectl - systemd-resolved control │ ││ │ dig +trace - Full DNS resolution chain │ ││ └─────────────────────────────────────────────────────────────────┘ ││ ││ Socket/Port Tools: ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ ss - Socket statistics (modern) │ ││ │ netstat - Network statistics (legacy) │ ││ │ lsof - List open files (network) │ ││ │ nc/netcat - Network swiss army knife │ ││ │ socat - Socket relay │ ││ └─────────────────────────────────────────────────────────────────┘ ││ ││ Interface Tools: ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ ip - Modern interface management │ ││ │ ifconfig - Legacy interface config │ ││ │ ethtool - Ethernet device parameters │ ││ │ iwconfig - Wireless interface config │ ││ │ iwlist - Wireless device scanning │ ││ └─────────────────────────────────────────────────────────────────┘ ││ │└─────────────────────────────────────────────────────────────────────────┘Basic Diagnostics
Section titled “Basic Diagnostics”# ============================================================# CONNECTIVITY TESTS# ============================================================
# Basic pingping -c 4 google.com
# Continuous pingping -i 0.2 google.com
# Flood ping (requires root)ping -f google.com
# Ping from specific interfaceping -I eth0 google.com
# Packet sizeping -s 1000 google.com
# IPv6ping6 -I eth0 ipv6.google.com
# ============================================================# PATH ANALYSIS# ============================================================
# UDP traceroute (default)traceroute google.com
# ICMP traceroutetraceroute -I google.com
# TCP SYN traceroute (for firewalls)traceroute -T -p 80 google.com
# MTU path discoverytracepath google.com
# Continuous traceroutemtr google.com
# Report modemtr -r -c 5 google.com
# ============================================================# DNS LOOKUPS# ============================================================
# Basic lookupdig google.comdig google.com +short
# Query specific record typesdig google.com Adig google.com AAAAdig google.com MXdig google.com TXTdig google.com NSdig google.com CNAME
# Reverse DNS lookupdig -x 8.8.8.8host 8.8.8.8
# Query specific DNS serverdig @8.8.8.8 google.com
# Trace DNS resolutiondig google.com +trace
# DNS zone transfer (if allowed)dig axfr example.com @ns1.example.comPort and Service Testing
Section titled “Port and Service Testing”# ============================================================# PORT SCANNING WITH NMAP# ============================================================
# Basic port scannmap target.com
# Service version detectionnmap -sV target.com
# OS detectionnmap -O target.com
# Scan specific portsnmap -p 80,443,8080 target.com
# All portsnmap -p- target.com
# TCP SYN scan (requires root)sudo nmap -sS target.com
# UDP scansudo nmap -sU target.com
# Quick scannmap -T4 -F target.com
# ============================================================# PORT CONNECTIVITY TESTING# ============================================================
# Test port connectivitync -zv target.com 80nc -zv target.com 80-90
# Banner grabbingnc -zv target.com 80
# Telnet (legacy but useful)telnet target.com 80
# Bash /dev/tcp (no external tools)timeout 2 bash -c 'cat < /dev/tcp/target.com/80'
# HTTP request with curlcurl -v http://target.comcurl -I http://target.com
# Check listening portsss -tunaplnetstat -tunapl
# Process using portlsof -i :80fuser 80/tcp24.2 Network Interface Configuration
Section titled “24.2 Network Interface Configuration”Interface Information
Section titled “Interface Information”# ============================================================# INTERFACE INFORMATION# ============================================================
# Show all interfacesip addr showip link show
# Statisticsip -s linkip -s link show eth0
# Detailed ethtool infosudo ethtool eth0sudo ethtool -S eth0 # Statisticssudo ethtool -k eth0 # Offload featuressudo ethtool -g eth0 # Ring parameterssudo ethtool -i eth0 # Driver info
# Wirelessiwconfigiwlist scan
# Bonding infocat /proc/net/bonding/bond024.3 Common Issues and Solutions
Section titled “24.3 Common Issues and Solutions”Troubleshooting Workflow
Section titled “Troubleshooting Workflow”┌─────────────────────────────────────────────────────────────────────────┐│ NETWORK TROUBLESHOOTING WORKFLOW │├─────────────────────────────────────────────────────────────────────────┤│ ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ Step 1: Identify the Problem │ ││ │ - What doesn't work? (ping, DNS, port, etc.) │ ││ │ - When did it start? │ ││ │ - What changed? │ ││ └─────────────────────────────────────────────────────────────────┘ ││ │ ││ ▼ ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ Step 2: Isolate the Layer │ ││ │ Layer 1: Physical (cable, link light) │ ││ │ Layer 2: MAC, ARP │ ││ │ Layer 3: IP, Routing │ ││ │ Layer 4: TCP/UDP, Ports │ ││ │ Layer 7: Application │ ││ └─────────────────────────────────────────────────────────────────┘ ││ │ ││ ▼ ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ Step 3: Verify Each Layer │ ││ │ - L1: ip link, ethtool │ ││ │ - L2: arp, bridge show │ ││ │ - L3: ip addr, ip route │ ││ │ - L4: ss, iptables │ ││ │ - L7: curl, telnet │ ││ └─────────────────────────────────────────────────────────────────┘ ││ │ ││ ▼ ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ Step 4: Fix and Test │ ││ │ - Apply fix │ ││ │ - Verify with same tests │ ││ │ - Monitor for recurrence │ ││ └─────────────────────────────────────────────────────────────────┘ ││ │└─────────────────────────────────────────────────────────────────────────┘No Connectivity
Section titled “No Connectivity”# ============================================================# NO CONNECTIVITY TROUBLESHOOTING# ============================================================
# Step 1: Check IP addressip addr show# Should have IP in correct subnet
# Step 2: Check default routeip routeip route get 8.8.8.8
# Step 3: Check DNScat /etc/resolv.confresolvectl status
# Step 4: Test basic connectivityping -c 3 8.8.8.8 # Gateway firstping -c 3 google.com # DNS test
# Step 5: Check firewalliptables -L -n -vss -tunapl | grep -i reject
# Step 6: Check DNS resolutionnslookup google.comdig google.com
# Common fixes:# Add IP: ip addr add 192.168.1.100/24 dev eth0# Add route: ip route add default via 192.168.1.1# Set DNS: echo "nameserver 8.8.8.8" > /etc/resolv.confDNS Issues
Section titled “DNS Issues”# ============================================================# DNS TROUBLESHOOTING# ============================================================
# Check current DNS serverscat /etc/resolv.confresolvectl status
# Test specific DNS serverdig @8.8.8.8 google.comdig @1.1.1.1 google.com
# Test resolution chaindig +trace google.com
# Flush DNS cache# systemd-resolvedsudo resolvectl flush-caches
# nscd (Name Service Cache Daemon)sudo systemctl restart nscd
# Check for DNSSEC issuesdig +cd google.com # Checking disabled
# Local DNS issues# /etc/hosts overridecat /etc/hosts
# Troubleshooting steps:# 1. ping <hostname> fails, ping <IP> works -> DNS issue# 2. Check /etc/resolv.conf# 3. Try different DNS server# 4. Check firewall blocking DNS (port 53)Firewall Blocking
Section titled “Firewall Blocking”# ============================================================# FIREWALL TROUBLESHOOTING# ============================================================
# List all rulesiptables -L -n -vip6tables -L -n -v
# Check NAT rulesiptables -t nat -L -n -v
# Check for dropped packetsiptables -L -n -v | grep DROP
# Check recent droppediptables -L INPUT -n -v --line-numbers
# ss to find listening servicesss -tunapl
# Check if port is listeningss -tlnp | grep :80
# Common issues:# - OUTPUT chain blocking (allow established)# - Docker manipulating iptables# - Cloud security groups# - ufw/ufw status24.4 Interview Questions
Section titled “24.4 Interview Questions”┌─────────────────────────────────────────────────────────────────────────┐│ NETWORK TROUBLESHOOTING INTERVIEW QUESTIONS │├─────────────────────────────────────────────────────────────────────────┤ │Q1: How do you troubleshoot no network connectivity? │ │A1: │1. Check IP: ip addr show │2. Check route: ip route, ip route get 8.8.8.8 │3. Check DNS: cat /etc/resolv.conf │4. Test ping: first gateway, then external IP, then hostname │5. Check firewall: iptables -L │6. Check interface: ip link, ethtool │ │─────────────────────────────────────────────────────────────────────────┤ │Q2: What is the difference between ping and traceroute? │ │A2: │- ping: Tests connectivity, measures RTT │- traceroute: Shows path (each hop), identifies where packets fail │- ping uses ICMP echo request │- traceroute uses varying TTL to probe each hop │- mtr combines both continuously │ │─────────────────────────────────────────────────────────────────────────┤ │Q3: How do you check if a specific port is open? │ │A3: │- nc -zv host port │- telnet host port │- ss -tlnp | grep port │- nmap -p port host │- lsof -i :port │ │─────────────────────────────────────────────────────────────────────────┤ │Q4: What is the difference between ss and netstat? │ │A4: │- ss: Modern tool, faster, more detailed │- netstat: Legacy, slower │- ss uses netlink, netstat reads /proc/net │- ss -tunapl shows all socket info │ │─────────────────────────────────────────────────────────────────────────┤ │Q5: How do you troubleshoot DNS issues? │ │A5: │1. Check /etc/resolv.conf │2. Test with specific DNS: dig @8.8.8.8 │3. Check if ping to IP works │4. Flush cache: resolvectl flush-caches │5. Check /etc/hosts for overrides │6. Use dig +trace to see full resolution │ │─────────────────────────────────────────────────────────────────────────┤ │Q6: How does traceroute work? │ │A6: │- Sends packets with incrementing TTL │- TTL=1 reaches first hop (router), returns ICMP time exceeded │- TTL=2 reaches second hop, etc. │- When reaches destination, returns ICMP port unreachable │- Shows IP and hostname of each hop │- Can use UDP, ICMP, or TCP SYN │ │─────────────────────────────────────────────────────────────────────────┤ │Q7: What could cause intermittent network issues? │ │A7: │- Network congestion │- Duplex mismatch (half/full) │- Cable/connector issues │- Driver bugs │- Firewall (stateful) issues │- DNS resolution problems │- MTU/fragmentation issues │- Hardware issues (NIC, switch, router) │- Rate limiting │ │─────────────────────────────────────────────────────────────────────────┤ │Q8: How do you check if network interface is up? │ │A8: │- ip link show │- ip addr show │- cat /sys/class/net/eth0/operstate │- ethtool eth0 │- mii-tool eth0 (legacy) │ │─────────────────────────────────────────────────────────────────────────┤ │Q9: What is the difference between TCP and UDP traceroute? │ │A9: │- UDP: Default, uses high-numbered ports │- ICMP (traceroute -I): Uses ICMP echo │- TCP SYN (traceroute -T): Uses TCP SYN to port 80/443 │- TCP often gets through firewalls better │- Different tools: tcptraceroute for TCP │ │─────────────────────────────────────────────────────────────────────────┤ │Q10: How do you measure network bandwidth? │ │A10: │- iperf/iperf3: Point-to-point bandwidth test │- speedtest-cli: Internet speed test │- curl with timing: Measures download speed │- iftop: Per-connection bandwidth │- nethogs: Per-process bandwidth │ │└─────────────────────────────────────────────────────────────────────────┘Quick Reference
Section titled “Quick Reference”# Connectivityping -c 4 hosttraceroute hostmtr host
# DNSdig hostnslookup hosthost host
# Portsnc -zv host portss -tlnp
# Interfaceip addr showip route showethtool eth0
# Troubleshooting flowping gateway → ping 8.8.8.8 → ping google.com → check DNSSummary
Section titled “Summary”- Tools: ping, traceroute, mtr, dig, nmap, ss, ethtool
- Workflow: Identify → Isolate → Verify → Fix
- Layers: Physical → Data Link → Network → Transport → Application
- Common issues: IP, routing, DNS, firewall
Next Chapter
Section titled “Next Chapter”Chapter 25: Advanced Networking
Last Updated: February 2026