TCP/IP Stack Tuning & Network Performance

Chapter 11: TCP/IP Stack Tuning & Network Performance

Learning Objectives

• Understand the Linux network stack architecture
• Know how to tune TCP for high-throughput and low-latency workloads
• Diagnose network performance problems
• Understand network queuing and packet processing

Prerequisites

• Chapter 10: sysctl Tuning
• Networking fundamentals (see Networking Guide)

---

What is this? (Beginner On-Ramp)

Network tuning involves tweaking the kernel’s network settings to handle massive amounts of traffic. By default, Linux is configured for general-purpose use (like a laptop). On a busy server, you have to adjust memory buffers and connection queues so it doesn’t drop traffic under heavy load.

11.1 Linux Network Stack Architecture

    Linux Network Stack (RX Path)
    ──────────────────────────────

    Packet arrives at NIC hardware
          │ DMA transfer to ring buffer
          ▼
    ┌─────────────────────────────────────────────────────────┐
    │                NIC Driver (e.g., e1000e, ixgbe)         │
    │  Hardware interrupt → NAPI poll (softirq)               │
    └──────────────────────────┬──────────────────────────────┘
                               │ sk_buff (socket buffer)
                               ▼
    ┌─────────────────────────────────────────────────────────┐
    │              netfilter hooks (iptables/nftables)         │
    │  PREROUTING → FORWARD → POSTROUTING                     │
    └──────────────────────────┬──────────────────────────────┘
                               │
                               ▼
    ┌─────────────────────────────────────────────────────────┐
    │              IP Layer (routing decision)                 │
    │  For local delivery: INPUT netfilter hook               │
    └──────────────────────────┬──────────────────────────────┘
                               │
                               ▼
    ┌─────────────────────────────────────────────────────────┐
    │              TCP/UDP Layer                               │
    │  Connection tracking, checksums, reassembly             │
    └──────────────────────────┬──────────────────────────────┘
                               │ copy to socket receive buffer
                               ▼
    ┌─────────────────────────────────────────────────────────┐
    │              Socket Receive Buffer (SO_RCVBUF)           │
    └──────────────────────────┬──────────────────────────────┘
                               │ application read()
                               ▼
                         Application

---

11.2 TCP Connection States

    TCP State Machine (critical for network tuning)
    ─────────────────────────────────────────────────

    Client                              Server
    ──────                              ──────
    CLOSED                              LISTEN

    SYN_SENT     ──── SYN ──────────►  SYN_RCVD
    ESTABLISHED  ◄──SYN+ACK ──────────
                 ──── ACK ──────────►  ESTABLISHED

    [Data exchange]

    FIN_WAIT_1   ──── FIN ──────────►  CLOSE_WAIT
    FIN_WAIT_2   ◄──  ACK ──────────
    TIME_WAIT    ◄──  FIN ──────────   LAST_ACK
                 ──── ACK ──────────►  CLOSED
    CLOSED (after 2*MSL)

    Key States to Monitor in Production:
    ─────────────────────────────────────
    ESTABLISHED: Active connections (normal)
    TIME_WAIT:   Waiting after FIN (normal, lasts 2*MSL ≈ 60s)
    CLOSE_WAIT:  Waiting for local close (bug if many!)
    SYN_RCVD:    Half-open connections (SYN flood if many!)

# Monitor connection states
ss -s                              # Summary by state
ss -ant | awk '{print $1}' | sort | uniq -c   # Count by state

# Detailed view
ss -tnp                            # TCP connections with process info
ss -tnp state established          # Only established
ss -tnp state time-wait            # Only TIME_WAIT

# If CLOSE_WAIT count is growing:
# → Application bug: server-side not closing connections properly
# Fix: Check for missing socket.close() calls

# If SYN_RCVD count is very high:
# → SYN flood attack
# Fix: Ensure tcp_syncookies=1, rate limit with iptables

---

11.3 TCP Buffer Sizing and Throughput

    TCP Throughput Formula
    ──────────────────────

    Maximum TCP Throughput = TCP Window Size / RTT

    Example:
    Window size = 65,535 bytes (64KB, old default)
    RTT = 100ms (100ms latency)

    Throughput = 65,535 / 0.1 = 655,350 bytes/s ≈ 5 Mbps

    With large window (Window Scaling extension, RFC 7323):
    Window size = 16MB
    RTT = 100ms

    Throughput = 16,777,216 / 0.1 = 167,772,160 bytes/s ≈ 1.3 Gbps!

    BDP (Bandwidth-Delay Product) = Bandwidth × RTT
    Set buffers to at least BDP:

    1 Gbps × 100ms = 100MB  ← Need 100MB buffer for full utilization!
    10 Gbps × 10ms = 12.5MB

/etc/sysctl.d/99-tcp-tuning.conf

# For high-bandwidth long-distance connections:
# Enable window scaling (should already be on)
net.ipv4.tcp_window_scaling = 1

# Buffer sizes (min, default, max in bytes)
net.ipv4.tcp_rmem = 4096 87380 134217728     # 128MB max receive
net.ipv4.tcp_wmem = 4096 65536 134217728     # 128MB max send

# Socket buffer maximums
net.core.rmem_max = 134217728
net.core.wmem_max = 134217728

# Memory pressure thresholds (in pages, page=4KB)
net.ipv4.tcp_mem = 1048576 4194304 8388608   # 4GB, 16GB, 32GB

# Enable BBR congestion control (better for modern networks)
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr

BBR vs CUBIC Congestion Control

    Congestion Control Algorithms
    ──────────────────────────────

    CUBIC (default):
    - Loss-based: reduces window when packets drop
    - Works well in LAN environments
    - Can underutilize high-bandwidth paths

    BBR (Bottleneck Bandwidth and RTT):
    - Model-based: measures bandwidth and RTT
    - Doesn't need packet loss to probe bandwidth
    - Much better on high-BDP paths (WAN, satellite)
    - Significantly better throughput in practice

    Results from Google production deployment:
    - 2-25% throughput improvement
    - 15-25% retransmit rate reduction

# Check available congestion control algorithms
cat /proc/sys/net/ipv4/tcp_available_congestion_control

# Check current
cat /proc/sys/net/ipv4/tcp_congestion_control

# Enable BBR
modprobe tcp_bbr
echo "tcp_bbr" >> /etc/modules-load.d/bbr.conf
echo "net.core.default_qdisc=fq" >> /etc/sysctl.d/99-bbr.conf
echo "net.ipv4.tcp_congestion_control=bbr" >> /etc/sysctl.d/99-bbr.conf
sysctl -p /etc/sysctl.d/99-bbr.conf

---

11.4 Network Interface Tuning

# ── NIC Queue Tuning ──────────────────────────────────────
# View NIC settings
ethtool eth0
ethtool -g eth0     # Ring buffer sizes
ethtool -k eth0     # Offload features
ethtool -c eth0     # Interrupt coalescing

# Increase ring buffer (prevent packet drops under load)
ethtool -G eth0 rx 4096 tx 4096    # Increase to 4096 (default 256)

# Enable/disable offload features
ethtool -K eth0 gro on             # Generic Receive Offload
ethtool -K eth0 gso on             # Generic Segmentation Offload
ethtool -K eth0 tso on             # TCP Segmentation Offload

# ── Interrupt Coalescing ──────────────────────────────────
# Delay interrupt for higher throughput (batching)
ethtool -C eth0 rx-usecs 50        # Wait 50μs before interrupt

# For low-latency: reduce coalescing
ethtool -C eth0 rx-usecs 0         # Interrupt on every packet
ethtool -C eth0 adaptive-rx on     # Let driver decide

# ── Multi-Queue NICs (RSS) ────────────────────────────────
# Receive Side Scaling: spread packets across CPU cores
# Each queue is processed by a different CPU

# Check number of queues
ethtool -l eth0

# Set number of queues (match CPU count)
ethtool -L eth0 combined $(nproc)

# View IRQ-CPU affinity
cat /proc/interrupts | grep eth0

# Use irqbalance to automatically balance
systemctl enable --now irqbalance

---

11.5 Network Performance Monitoring

# ── Interface Statistics ──────────────────────────────────
# Real-time interface stats
ip -s link show eth0
cat /proc/net/dev

# Per-second stats
sar -n DEV 1          # Network interface stats per second
sar -n EDEV 1         # Network error stats

# Watch for packet drops!
watch -n 1 'ip -s link show eth0 | grep -A2 RX'
# If RX drops growing → ring buffer full (increase -G rx), or CPU slow

# ── TCP Statistics ────────────────────────────────────────
ss -s                 # Socket summary
netstat -s            # Detailed protocol stats (retransmits, etc.)
nstat -a              # Kernel network statistics

# Check TCP retransmissions (high = congestion or packet loss)
nstat | grep -i retran
# Or:
cat /proc/net/netstat | grep -i retran

# ── Network Latency ───────────────────────────────────────
ping -c 100 <target>   # Basic latency check
# Check for jitter (variation in latency)

# MTR: combines ping + traceroute
mtr --report --report-cycles 100 <target>

# ── Bandwidth Testing ─────────────────────────────────────
# Between two hosts:
# Server: iperf3 -s
# Client: iperf3 -c <server_ip> -t 30 -P 8   # 8 parallel streams

# Check NIC maximum:
ethtool eth0 | grep Speed

---

11.6 Production Scenarios

Scenario 1: High TIME_WAIT Count

# Alert: ss -s shows 50,000 connections in TIME_WAIT

# Understanding the problem:
# TIME_WAIT holds the port for 2*MSL (60s) after connection close
# Heavy web server making outbound connections can exhaust ports

# Fix 1: Enable tcp_tw_reuse (reuse TIME_WAIT sockets for new connections)
sysctl -w net.ipv4.tcp_tw_reuse=1
# Note: Only works for outbound connections (client side)

# Fix 2: Reduce fin_timeout
sysctl -w net.ipv4.tcp_fin_timeout=15

# Fix 3: Expand ephemeral port range
sysctl -w net.ipv4.ip_local_port_range="1024 65535"

# Fix 4 (application level): Use HTTP keep-alive to reuse connections
# Don't close connections after every request

Scenario 2: Network Packet Drops Under Load

# Problem: Server drops packets at high load despite low CPU

# Check where drops happen:
# Interface level drops:
ip -s link show eth0 | grep -A3 "RX:"
# If dropped > 0: NIC ring buffer too small
ethtool -G eth0 rx 4096

# Softnet drops (kernel network queue):
cat /proc/net/softnet_stat
# Column 3 (throttle/drops) growing = NET_RX_SOFTIRQ can't keep up
# Fix: Increase netdev_max_backlog
sysctl -w net.core.netdev_max_backlog=250000

# Socket buffer drops (application not reading fast enough):
ss -nt | grep -v "Recv-Q: 0"
# If Recv-Q is large for many sockets: application is bottleneck

---

11.7 Interview Questions

Q1: Explain what BDP (Bandwidth-Delay Product) is and why it matters for TCP tuning.

Answer: BDP = Bandwidth × RTT. It represents the amount of data that can be “in flight” on the network at any time. TCP’s window size must be at least as large as the BDP to keep the pipe full. For example, a 1Gbps link with 100ms RTT has BDP = 12.5MB. If the TCP window is only 64KB (old default), the connection can only achieve 64KB/0.1s = 5Mbps — a 200x reduction. This is why high-bandwidth long-distance connections (cloud to cloud, international data transfers) need large TCP buffers. The fix: increase tcp_rmem and tcp_wmem max values, ensure window scaling is enabled.

Q2: What is the purpose of TCP SYN cookies and when should they be enabled?

Answer: SYN cookies (enabled by net.ipv4.tcp_syncookies=1) protect against SYN flood attacks. Normally, when a SYN arrives, the kernel allocates state in the SYN backlog queue and responds with SYN+ACK. An attacker can exhaust this queue with fake SYNs (never sending the final ACK), preventing legitimate connections. With SYN cookies, the kernel encodes connection state in the SYN+ACK’s sequence number instead of storing it. When the legitimate ACK arrives with the proper sequence number (proving the cookie), the connection is established. Should always be enabled (tcp_syncookies=1) in production — the overhead is negligible and the protection is significant.

Q3: What is BBR and how does it differ from CUBIC congestion control?

Answer: CUBIC (default) is loss-based: it grows the window until a packet is lost, then halves it. It assumes loss = congestion, which works in LAN but underperforms on WAN (where buffer bloat causes packets to sit in router queues without dropping). BBR (Bottleneck Bandwidth and RTT) is model-based: it continuously measures the actual bottleneck bandwidth and RTT, and sets the window to match. It doesn’t need loss as a signal. In practice, BBR achieves 25-200% higher throughput on long-distance connections, while also reducing latency by not filling up intermediate router buffers. Google deployed BBR across their infrastructure and saw significant improvements.

---

11.8 Summary

Parameter	Purpose	Production Value
tcp_syncookies	SYN flood protection	1 (always)
tcp_tw_reuse	TIME_WAIT reuse	1
tcp_fin_timeout	FIN_WAIT2 timeout	15s
tcp_keepalive_time	Keepalive interval	300s
somaxconn	Listen backlog	65535
tcp_rmem/wmem	Buffer sizes	min,def,128MB
tcp_congestion	Algorithm	bbr

---

Next Chapter: Chapter 12: HugePages, THP, NUMA & OOM Killer

Prerequisites

Chapter 10 (sysctl), networking fundamentals from your existing Networking guide.

---

Advantages & Disadvantages

Advantages: Can handle millions of connections (C10M), prevents dropped packets during spikes. Disadvantages: Tuning buffers too high wastes RAM (bufferbloat), highly specific to the workload.

---

Common Mistakes

• Increasing TCP buffer sizes blindly — doubling buffer size doesn’t double throughput if the bottleneck is CPU or NIC.
• Enabling tcp_tw_reuse on servers — it’s safe for clients (outbound); on servers with NAT it can cause connection confusion.
• Not setting SO_REUSEPORT for multi-process servers — without it, all connections funnel through one accept queue (Linux 3.9+ supports per-socket queues).
• Ignoring NIC queue depth — ring buffer size (ethtool -g eth0) is often more impactful than sysctl buffer tuning for high-throughput workloads.
• Forgetting interrupt affinity — all NIC interrupts on CPU 0 by default; spread with irqbalance or manual /proc/irq/N/smp_affinity.

---

Troubleshooting

Symptom	Cause	Diagnosis	Fix
High packet drop rate (RX drops)	NIC ring buffer overflow; app too slow to consume	`ethtool -S eth0	grep drop; netstat -s
TCP connections slow to establish	SYN backlog overflow under load	`netstat -s	grep ‘SYNs to LISTEN’; ss -lnt`
TIME_WAIT accumulation causing port exhaustion	Short-lived connections not recycled fast enough	`ss -s	grep TIME-WAIT; netstat -an
Intermittent packet loss on 10G link	NIC interrupt coalescing too aggressive	ethtool -c eth0; check coalesce settings	Tune: ethtool -C eth0 rx-usecs 100 tx-usecs 100

---

Hands-on Lab

Objective: Tune TCP stack for high-concurrency workload.

# 1. Baseline network stats
ss -s                            # Connection summary
netstat -s | head -50           # Protocol stats
cat /proc/net/dev                # Per-interface counters

# 2. Current buffer sizes
sysctl net.core.rmem_default net.core.rmem_max
sysctl net.ipv4.tcp_rmem net.ipv4.tcp_wmem

# 3. Apply network tuning
sudo tee /etc/sysctl.d/99-net.conf << 'EOF'
net.core.somaxconn = 65535
net.core.rmem_max = 134217728
net.core.wmem_max = 134217728
net.ipv4.tcp_rmem = 4096 87380 134217728
net.ipv4.tcp_wmem = 4096 65536 134217728
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 1024 65535
EOF
sudo sysctl -p /etc/sysctl.d/99-net.conf

# 4. NIC ring buffer
ethtool -g eth0     # current ring buffer sizes
sudo ethtool -G eth0 rx 4096 tx 4096

# 5. Load test before/after
# ab -n 10000 -c 200 http://localhost/

---

Exercises

1. Use iperf3 to benchmark TCP throughput before and after doubling the TCP buffer sizes. Document the improvement.
2. Set up SO_REUSEPORT on an nginx server and verify multiple worker processes each get a dedicated accept queue.
3. Configure interrupt affinity manually: assign NIC IRQs to specific CPUs using /proc/irq/N/smp_affinity_list.
4. Monitor netstat -s | grep -i 'retransmit\|reset\|error' every second during a load test. Identify the bottleneck.
5. Compare CLOSE_WAIT accumulation with and without tcp_keepalive_time=60. Explain the connection lifecycle difference.

---

Revision Notes

• TCP buffer sizing: net.ipv4.tcp_rmem = min/default/max per socket; net.core.rmem_max = system ceiling.
• Accept queue: somaxconn = max connections waiting for accept(); tcp_max_syn_backlog = half-open connections.
• TIME_WAIT: normal state after connection closes; lasts 2×MSL (60s). Thousands accumulate under high traffic.
• tcp_tw_reuse=1 allows reusing TIME_WAIT sockets for new outbound connections — safe for clients.
• NIC RSS (Receive Side Scaling): spreads interrupt processing across CPUs — requires multi-queue NIC.
• Ephemeral ports: ip_local_port_range — default 32768-60999; expand to 1024-65535 for high-throughput clients.

---

Further Reading

• TCP tuning: https://fasterdata.es.net/host-tuning/linux/
• Cloudflare TCP tuning: https://blog.cloudflare.com/optimizing-tcp-for-high-throughput-and-low-latency/
• man 7 tcp — complete TCP socket option reference
• Linux NIC tuning: https://www.kernel.org/doc/html/latest/networking/scaling.html

---

Related Chapters

• Chapter 10 — sysctl reference and patterns
• Chapter 25 — Network performance analysis
• Chapter 40 — Load balancing TCP connections

---

Last Updated: July 2026