Systemd
Chapter 17: systemd Deep Dive
Section titled “Chapter 17: systemd Deep Dive”Comprehensive System Management in Modern Linux
Section titled “Comprehensive System Management in Modern Linux”17.1 Understanding systemd Architecture
Section titled “17.1 Understanding systemd Architecture”What is systemd?
Section titled “What is systemd?”systemd is the init system and service manager for modern Linux distributions. It provides a comprehensive framework for managing system services, processes, resources, and system state. Originally designed by Lennart Poettering, it has become the default init system for most major Linux distributions including RHEL, Fedora, Debian, Ubuntu, and Arch Linux.
systemd Architecture Overview+------------------------------------------------------------------+| || systemd System Stack || || +-------------------------------------------------------------+|| | User Space ||| | +------------------------------------------------------+ ||| | | systemd Manager (PID 1) | ||| | | +------------------------------------------------+ | ||| | | | Unit Management | | ||| | | | - Service Units | | ||| | | | - Socket Units | | ||| | | | - Target Units | | ||| | | | - Mount Units | | ||| | | | - Timer Units | | ||| | | | - Path Units | | ||| | | +------------------------------------------------+ | ||| | | | | ||| | | +-----------------v------------------+ | ||| | | | Service Dependencies | | ||| | | | - Dependency Resolution | | ||| | | | - Ordering | | ||| | | | - Parallelization | | ||| | | +-------------------------------------+ | ||| | +------------------------------------------------------+ ||| | | ||| | +-------------------------v--------------------------+ ||| | | SystemD Components | || | | +-------------+ +-------------+ +-----------+ | || | | | journald | | logind | | networkd | | || | | | (Logging) | |(Login Mgmt) | | (Network) | | || | | +-------------+ +-------------+ +-----------+ | || | | +-------------+ +-------------+ +-----------+ | || | | | udevd | | timedatectl| | resolved | | || | | | (Device Mgmt)| | (Time Mgmt) | | (DNS) | | || | | +-------------+ +-------------+ +-----------+ | || | +------------------------------------------------------+ ||| +------------------------------------------------------------+|| || Kernel Space || +------------------------------------------------------------+|| | cgroups | namespaces | selinux | audit | kmod ||| +------------------------------------------------------------+|| |+------------------------------------------------------------------+systemd Components and Their Roles
Section titled “systemd Components and Their Roles” systemd Components Detailed+------------------------------------------------------------------+| || Component | Binary | Purpose || ---------------|-------------|--------------------------------|| systemd | /sbin/init | PID 1, init system || journald | journald | Centralized logging || systemd-logind | logind | User login management || systemd-networkd | networkd | Network configuration || systemd-resolved | resolved | DNS resolution || systemd-udevd | udevd | Device management || systemd-timedated| timedatectl| Time management || systemd-coredump| coredump | Core dump handling || systemd-hostnamed| hostnamed | Hostname management || systemd-localed | localed | Locale management || systemd-machine-id-setup| | Machine ID setup || systemd-fsck | fsck | Filesystem checking || systemd-hwdb | hwdb | Hardware database || || Key Features: || +----------------------------------------------------------+ || | • Parallel service startup using cgroups | || | • Socket-based activation (on-demand starting) | || | • D-Bus based service communication | || | • Resource control with cgroups | || | • Transaction-based dependency resolution | || | • Automatic service restarting | || | • Integrated journal logging | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+17.2 systemd Unit Types
Section titled “17.2 systemd Unit Types”Understanding Units
Section titled “Understanding Units”Units are the fundamental objects that systemd manages. Each unit is defined by a configuration file and represents a resource or service that systemd knows how to activate, manage, and track.
systemd Unit Types+------------------------------------------------------------------+| || Unit Type | Extension | Description || --------------|-------------|----------------------------------|| Service | .service | Managed application process || Socket | .socket | IPC/network socket || Target | .target | Group of units (like runlevels)|| Device | .device | Hardware device || Mount | .mount | Filesystem mount point || Automount | .automount | Auto-mount point || Swap | .swap | Swap device || Timer | .timer | Scheduled task (cron-like) || Path | .path | Path-based activation || Slice | .slice | Resource control group || Scope | .scope | External process group || || Unit File Locations: || +----------------------------------------------------------+ || | /etc/systemd/system/ - System-wide units (user-custom) | || | /usr/lib/systemd/system/ - Vendor-provided units | || | ~/.config/systemd/user/ - User units | || | /run/systemd/system/ - Runtime units | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+Unit File Structure
Section titled “Unit File Structure” Generic Unit File Structure+------------------------------------------------------------------+| || [Unit] || Description= # Human-readable description || Documentation= # URL or man page || After= # Units to start after || Before= # Units to start before || Requires= # Hard dependencies || Wants= # Soft dependencies || BindsTo= # Tight binding || PartOf= # Stop/start together || Conflicts= # Mutually exclusive || ConditionArchitecture= # Architecture check || ConditionVirtualization= # VM/container check || AssertArchitecture= # Fail if not matching || SourcePath= # Originating config file || || [Service] (for .service units) || Type= # Service type (simple, forking, etc) || ExecStart= # Command to start || ExecStop= # Command to stop || ExecReload= # Command to reload || ExecStartPre= # Pre-start commands || ExecStartPost= # Post-start commands || ExecStopPost= # Post-stop commands || Restart= # Restart policy || RestartSec= # Seconds to wait before restart || TimeoutStartSec= # Start timeout || TimeoutStop= # Stop timeout || User= # User to run as || Group= # Group to run as || WorkingDirectory= # Working directory || Environment= # Environment variables || EnvironmentFile= # File with env vars || PIDFile= # PID file for forking services || StandardOutput= # stdout handling || StandardError= # stderr handling || || [Install] || WantedBy= # Units that want this || RequiredBy= # Units that require this || Also= # Units to install/uninstall together || Alias= # Alternative names || || [Socket] (for .socket units) || ListenStream= # TCP socket || ListenDatagram= # UDP socket || ListenFIFO= # FIFO file || SocketProtocol= # Protocol (tcp, udp) || BindIPv6Only= # IPv6 binding || Backlog= # Connection backlog || BindToDevice= # Bind to specific device || SocketUser= # Socket owner || SocketGroup= # Socket group || DirectoryMode= # Directory permissions || || [Timer] (for .timer units) || OnActiveSec= # Time after activation || OnBootSec= # Time after boot || OnStartupSec= # Time after startup || OnUnitActiveSec= # Time after unit last active || OnUnitInactiveSec= # Time after unit last inactive || OnCalendar= # Calendar-based trigger || AccuracySec= # Accuracy needed || RandomizedDelaySec= # Random delay to prevent storms || Unit= # Unit to activate || Persistent= # Persist across reboots || WakeSystem= | Wake system from suspend || RemainAfterExit= | Keep active after exit || |+------------------------------------------------------------------+17.3 systemctl Command Reference
Section titled “17.3 systemctl Command Reference”Service Management Commands
Section titled “Service Management Commands”# Start a servicesudo systemctl start nginxsudo systemctl start nginx.service
# Stop a servicesudo systemctl stop nginx
# Restart a service (stop then start)sudo systemctl restart nginx
# Reload configuration (without restart)sudo systemctl reload nginx
# Reload or restart (try reload, fallback to restart)sudo systemctl reload-or-restart nginx
# Check service statussudo systemctl status nginxsudo systemctl status nginx.servicesudo systemctl is-active nginx # Check if running (returns active/inactive)sudo systemctl is-enabled nginx # Check if enabled (returns enabled/disabled)sudo systemctl is-failed nginx # Check if failed (returns active/failed)
# Enable service (start at boot)sudo systemctl enable nginxsudo systemctl enable --now nginx # Enable and start now
# Disable service (don't start at boot)sudo systemctl disable nginxsudo systemctl disable --now nginx # Disable and stop
# Mask service (prevent all activation)sudo systemctl mask nginx # Creates symlink to /dev/nullsudo systemctl unmask nginx # Remove mask
# View service definitionsudo systemctl cat nginx # Show unit filesudo systemctl show nginx # Show all properties
# Edit service unitsudo systemctl edit nginx # Opens editor for drop-insudo systemctl edit --full nginx # Edit full unit fileViewing System State
Section titled “Viewing System State”# List all unitssystemctl list-units # All active unitssystemctl list-units --type=service # Only servicessystemctl list-units --type=service --state=activesystemctl list-units --type=service --state=failedsystemctl list-units --type=socketsystemctl list-units --type=timersystemctl list-units --type=mount
# List unit filessystemctl list-unit-files # All unit filessystemctl list-unit-files --state=enabledsystemctl list-unit-files --state=disabled
# Show dependenciessystemctl list-dependencies nginx # What nginx needssystemctl list-dependencies --all nginx # Full treesystemctl list-dependencies --reverse nginx # What needs nginx
# Show reverse dependenciessystemctl list-dependencies --reverse nginx
# View failed unitssystemctl --failedsystemctl list-units --state=failedSystem Control Commands
Section titled “System Control Commands”# Reboot systemsudo systemctl rebootsudo systemctl reboot -i # Force immediate rebootsudo systemctl --message="Upgrading" reboot # Message for users
# Power off systemsudo systemctl poweroffsudo systemctl --message="Shutting down" poweroff
# Suspend systemsudo systemctl suspend
# Hibernate systemsudo systemctl hibernate
# Hybrid sleepsudo systemctl hybrid-sleep
# Switch to different target (runlevel)sudo systemctl multi-user.target # Text modesudo systemctl graphical.target # GUI modesudo systemctl emergency.target # Emergency modesudo systemctl rescue # Rescue mode (single user)
# Get current targetsystemctl get-default
# Set default targetsudo systemctl set-default graphical.target
# Isolate target (switch immediately)sudo systemctl isolate multi-user.targetDaemon Management
Section titled “Daemon Management”# Reload systemd configuration (after changing unit files)sudo systemctl daemon-reload
# Reload systemd manager configurationsudo systemctl daemon-reexec
# Kill a service (send signal)sudo systemctl kill nginx # Send SIGTERM (default)sudo systemctl kill -s SIGKILL nginx # Send specific signalsudo systemctl kill --kill-who=main nginx # Kill main process only
# Reset failed statesudo systemctl reset-failed
# Show environment variables of a servicesystemctl show nginx -P Environmentsystemctl show nginx -P EnvironmentFile17.4 Understanding Service Types
Section titled “17.4 Understanding Service Types”Service Type Comparison
Section titled “Service Type Comparison” Service Types Detailed+------------------------------------------------------------------+| || Type: simple || +----------------------------------------------------------+ || | • Main process is the service | || | • systemd starts the process and considers it active | || | • If process exits, service is considered failed | || | • Most common type for custom services | || | • Example: nginx with type=simple | || +----------------------------------------------------------+ || || Type: forking || +----------------------------------------------------------+ || | • Service forks child processes | || | • Parent exits after forking | || | • systemd waits for parent to exit | || | • Requires PIDFile= directive | || | • Example: traditional daemons | || +----------------------------------------------------------+ || || Type: oneshot || +----------------------------------------------------------+ || | • Runs once and exits | || | • Must be used with RemainAfterExit=yes for status | || | • Used for one-time initialization tasks | || | • Example: system setup scripts | || +----------------------------------------------------------+ || || Type: notify || +----------------------------------------------------------+ || | • Similar to simple but sends notification | || | • Service sends READY=1 via sd_notify() | || | • systemd waits for notification before marking active | || | • Example: nginx (with systemd notification support) | || +----------------------------------------------------------+ || || Type: dbus || +----------------------------------------------------------+ || | • Service acquires a D-Bus name | || | • systemd waits for BusName= to appear | || | • Service marked as running when name is acquired | || | • Example: D-Bus activated services | || +----------------------------------------------------------+ || || Type: idle || +----------------------------------------------------------+ || | • Similar to simple but delays execution | || | • Waits until system is idle | || | • Useful for delaying non-critical services | || | • Example: optional GUI services | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+17.5 Service Restart and Dependency Logic
Section titled “17.5 Service Restart and Dependency Logic”Restart Policies
Section titled “Restart Policies”# Restart= optionsRestart=no # Never restartRestart=on-failure # Restart only on failure (non-zero exit)Restart=on-success # Restart only on clean exit (for oneshot)Restart=always # Always restart (for Type=oneshot)Restart=on-abnormal # Restart on any error (timeout, crash, non-zero)
# RestartSec - delay between stop and startRestartSec=5 # Wait 5 seconds (default)
# StartLimitInterval and StartLimitBurst - rate limiting# Example: max 5 starts per 10 secondsStartLimitIntervalSec=10StartLimitBurst=5
# StartLimitAction - what to do when limit reachedStartLimitAction=reboot # Reboot the systemStartLimitAction=reboot-force # Force rebootStartLimitAction=reboot-immediate # Immediate rebootDependency Types
Section titled “Dependency Types” systemd Dependency Types+------------------------------------------------------------------+| || Dependency | Behavior || ---------------|-----------------------------------------------|| Requires= | Hard dependency - stop if dependency fails || Wants= | Soft dependency - start if dependency starts || BindsTo= | Tight binding - stop if dependency stops || PartOf= | Stop/start together - like bindsTo but less || Conflicts= | Mutual exclusion - cannot run together || || Ordering (these are separate from dependencies): || +----------------------------------------------------------+ || | After= | Start after dependency starts | || | Before= | Start before dependent starts | || | PropagatesReloadTo= | Reload propagates to dependent | || | JoinsNamespaceOf= | Join namespace of dependent | || +----------------------------------------------------------+ || || Best Practices: || +----------------------------------------------------------+ || | • Use Wants= instead of Requires= for most cases | || | • Always use After= with Wants= to control order | || | • Avoid circular dependencies | || | • Use BindsTo= sparingly (very tight coupling) | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+Common Dependency Patterns
Section titled “Common Dependency Patterns”# Service that needs network[Unit]Description=My Web ServiceAfter=network.targetWants=network.target
# Service that needs database[Unit]Description=My AppAfter=network.target postgresql.serviceWants=postgresql.service
# Service that needs to start after certain services[Unit]Description=Backup ServiceAfter=network.target remote-fs.targetRequiresMountsFor=/backup
# Service that conflicts with another[Unit]Description=IPv6 FirewallConflicts=ip6tables.service
# Service that triggers another[Unit]Description=Logging ServiceWants=web-app-logger.service
# Mount units dependencies[Unit]Description=Mount for Data DriveAfter=local-fs-pre.targetRequiresMountsFor=/mnt/data17.6 systemd Targets (Runlevels)
Section titled “17.6 systemd Targets (Runlevels)”Understanding Targets
Section titled “Understanding Targets”Targets in systemd are similar to runlevels in SysV init, but more flexible. They group units together to achieve a specific system state.
systemd Targets+------------------------------------------------------------------+| || Target | Description | Equivalent || ---------------|------------------------|---------------------|| graphical.target| GUI with login | Runlevel 5 || multi-user.target| Text mode, multi-user | Runlevel 3 || rescue.target | Single user mode | Runlevel 1 || emergency.target| Emergency shell | Runlevel 1 (sbin/init) || shutdown.target| System shutdown | - || halt.target | System halt | Runlevel 0 || reboot.target | System reboot | Runlevel 6 || suspend.target | Suspend to RAM | - || hibernate.target| Suspend to disk | - || default.target | Default target | - || || Legacy Runlevel Mapping: || +----------------------------------------------------------+ || | Runlevel 0 = poweroff.target | || | Runlevel 1 = rescue.target | || | Runlevel 2 = multi-user.target (Debian) | || | Runlevel 3 = multi-user.target (Red Hat) | || | Runlevel 4 = multi-user.target (Debian) | || | Runlevel 5 = graphical.target | || | Runlevel 6 = reboot.target | || +----------------------------------------------------------+ || || Common Targets and Their Dependencies: || +----------------------------------------------------------+ || | graphical.target requires: | || | + multi-user.target | || | + basic.target | || | + sysinit.target | || | + local-fs.target | || | + swap.target | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+Working with Targets
Section titled “Working with Targets”# View default targetsystemctl get-default
# Set default targetsudo systemctl set-default graphical.targetsudo systemctl set-default multi-user.target
# Switch to target immediatelysudo systemctl isolate multi-user.targetsudo systemctl isolate graphical.target
# List target dependenciessystemctl list-dependencies graphical.targetsystemctl list-dependencies multi-user.target
# List targetssystemctl list-units --type=target
# View target statesystemctl status graphical.target
# Mask/unmask targetssudo systemctl mask emergency.targetsudo systemctl unmask emergency.target17.7 systemd Journal
Section titled “17.7 systemd Journal”Journalctl Commands
Section titled “Journalctl Commands”# View all logsjournalctl
# View logs for specific servicejournalctl -u nginxjournalctl --unit=nginxjournalctl -u nginx.service
# View logs since specific timejournalctl --since "2024-01-01 00:00:00"journalctl --since "1 hour ago"journalctl --since "yesterday"
# View logs until specific timejournalctl --until "2024-01-01 23:59:59"journalctl --until "1 minute ago"
# Time rangejournalctl --since "2024-01-01" --until "2024-01-02"
# Follow logs (like tail -f)journalctl -fjournalctl -u nginx -f
# Number of linesjournalctl -n 100 # Last 100 linesjournalctl -n 100 -u nginx
# Show only errors and worsejournalctl -p errjournalctl -p warningjournalctl -p infojournalctl -p debug
# Priority levels: emerg(0), alert(1), crit(2), err(3), warning(4), notice(5), info(6), debug(7)
# Show kernel messagesjournalctl -kjournalctl --dmesg
# Show systemd boot messagesjournalctl -bjournalctl -b -1 # Previous bootjournalctl -b -2 # Two boots agojournalctl --list-boots # List available boots
# Disk usagejournalctl --disk-usage
# Rotate logssudo journalctl --rotate
# Vacuum old logssudo journalctl --vacuum-size=500Msudo journalctl --vacuum-time=2weeks17.8 systemd Resources and Limits
Section titled “17.8 systemd Resources and Limits”Resource Management with systemd
Section titled “Resource Management with systemd”# Memory limits (in service unit)[Service]MemoryMax=1GMemoryHigh=512M
# CPU limits (in percentage)[Service]CPUQuota=50%
# CPU affinity (specific cores)[Service]CPUAffinity=0 1 2 3
# IO limits[Service]IOReadBandwidthMax=/var/lib/mysql 1GIOWriteBandwidthMax=/var/lib/mysql 1GIOPSReadMax=10000IOPSWriteMax=5000
# Process limits[Service]TasksMax=100LimitNPROC=50LimitNOFILE=1000
# Working set limits[Service]MemorySwapMax=100M17.9 systemd Timers
Section titled “17.9 systemd Timers”Timer Units
Section titled “Timer Units”# Example timer unit - /etc/systemd/system/backup.timer[Unit]Description=Daily Backup TimerRequires=backup.service
[Timer]OnCalendar=dailyPersistent=trueRandomizedDelaySec=1h
[Install]WantedBy=timers.target
# Corresponding service - /etc/systemd/system/backup.service[Unit]Description=Daily Backup Service
[Service]Type=oneshotExecStart=/usr/local/bin/backup.shUser=backupuser
# Timer calendar specificationsOnCalendar=daily # Every day at midnightOnCalendar=weekly # Every Monday at midnightOnCalendar=Mon *-*-* 02:00:00 # Every Monday at 2 AMOnCalendar=*-*-* 12:00:00 # Every day at noonOnCalendar=*:0/15 # Every 15 minutes
# Time intervals (from activation)OnActiveSec=1h # 1 hour after timer activatesOnBootSec=30min # 30 minutes after bootOnUnitActiveSec=1h # 1 hour after last runManaging Timers
Section titled “Managing Timers”# List active timerssystemctl list-timerssystemctl list-timers --all
# Start/stop timersudo systemctl start backup.timersudo systemctl stop backup.timer
# Enable/disable timersudo systemctl enable backup.timersudo systemctl disable backup.timer
# Check timer statussystemctl status backup.timer
# View timer next runsystemctl list-timers --all | grep backup17.10 systemd Networking Features
Section titled “17.10 systemd Networking Features”Network-related systemd Components
Section titled “Network-related systemd Components”# systemd-resolved - DNS resolversystemctl status systemd-resolvedsystemctl restart systemd-resolved
# View DNS cacheresolvectl statusresolvectl query example.com
# Flush DNS cachesudo resolvectl flush-cachessudo systemd-resolve --flush-caches
# systemd-networkd - network managementsystemctl status systemd-networkdsystemctl restart systemd-networkd
# Network configuration in /etc/systemd/network/# *.network files for network interfaces# *.netdev files for virtual devices# *.link files for link configuration
# Example *.network file[Match]Name=eth0
[Network]Address=192.168.1.10/24Gateway=192.168.1.1DNS=8.8.8.8DNS=8.8.4.4IPForward=ipv4
# Example *.netdev for VLAN[NetDev]Name=vlan100Kind=vlan
[VLAN]Id=10017.11 systemd in Containers and Virtualization
Section titled “17.11 systemd in Containers and Virtualization”Container Considerations
Section titled “Container Considerations”# In container environment, systemd may not be PID 1# Use --privileged container or configure init properly
# For Docker with systemd:# Dockerfile entrypointENTRYPOINT ["/usr/sbin/init"]
# Or run with --privileged and initdocker run --privileged myimage /sbin/init
# systemd slice management for containers# Create slice with resource limits[Slice]CPUQuota=50%MemoryLimit=1GTasksMax=5017.12 Interview Questions
Section titled “17.12 Interview Questions”Basic Questions
Section titled “Basic Questions”-
What is systemd and what does it replace?
- systemd is an init system that replaces SysV init and Upstart
- It provides service management, logging, network management, and more
-
What is the first process started by the Linux kernel?
- systemd runs as PID 1 (or the container init process)
- It is the first user-space process
-
What are the main unit types in systemd?
- Service, Socket, Target, Device, Mount, Automount, Swap, Timer, Path, Slice, Scope
-
What is the difference between systemctl start and systemctl enable?
- start: starts the service immediately
- enable: configures service to start at boot
-
What does systemctl daemon-reload do?
- Reloads systemd unit files without restarting services
- Required after creating or modifying unit files
Intermediate Questions
Section titled “Intermediate Questions”-
Explain the difference between Requires and Wants
- Requires: hard dependency - if the dependency fails, this unit also fails
- Wants: soft dependency - if the dependency fails, this unit continues
-
What is the purpose of After= in a unit file?
- Defines ordering - this unit will start after the specified units
- Different from dependency - doesn’t actually require the unit
-
What are the different service types in systemd?
- simple, forking, oneshot, notify, dbus, idle
-
How do you view logs for a specific service?
journalctl -u servicenamejournalctl -u servicename --since "1 hour ago"
-
What is a target in systemd?
- A grouping mechanism similar to runlevels
- Groups multiple units together for a specific system state
Advanced Questions
Section titled “Advanced Questions”-
How does systemd achieve parallel service startup?
- Uses socket-based activation
- Uses D-Bus for inter-process communication
- Dependencies are analyzed to start independent services in parallel
-
Explain the systemd boot process
- Boot → systemd PID1 → target.target → activates dependencies
- Systemd starts basic.target, then multi-user.target or graphical.target
-
How do you troubleshoot a failing service?
systemctl status servicenamejournalctl -u servicename -xe- Check unit file syntax:
systemd-analyze verify servicename.service - Check dependencies:
systemctl list-dependencies servicename
-
What is the difference between socket activation and regular service startup?
- Socket activation starts the service on-demand when a connection is made
- Improves boot time and resource usage
-
How do you set resource limits for a service in systemd?
- MemoryMax, MemoryHigh for memory limits
- CPUQuota for CPU limits
- IOReadBandwidthMax for I/O limits
- TasksMax for process/thread limits
Summary
Section titled “Summary”systemd is the modern init system for Linux that provides comprehensive service management. Key commands and concepts:
Quick Reference+------------------------------------------------------------------+| || Common Commands: || +----------------------------------------------------------+ || | systemctl start <service> | Start service | || | systemctl stop <service> | Stop service | || | systemctl restart <service> | Restart service | || | systemctl status <service> | Check status | || | systemctl enable <service> | Enable at boot | || | systemctl disable <service> | Disable at boot | || | systemctl list-units | List active units | || | systemctl list-dependencies | Show dependencies | || | journalctl -u <service> | View service logs | || | systemctl daemon-reload | Reload unit files | || +----------------------------------------------------------+ || || Key Concepts: || +----------------------------------------------------------+ || | Units: Service, Socket, Target, Timer, Path, Mount | || | Dependencies: Requires, Wants, After, Before | || | Service Types: simple, forking, notify, oneshot | || | Targets: graphical, multi-user, rescue, emergency | || | Logging: journalctl | || +----------------------------------------------------------+ || |+------------------------------------------------------------------+