Linux Command Line & Bash Scripting Master Guide: Systems Administration, Log Parsing & Production Automation

Over 90% of the world's public cloud instances, supercomputers, and modern container runtimes execute on top of the Linux Kernel. Whether you are troubleshooting cloud web servers, deploying microservices via Kubernetes, or automating daily infrastructure maintenance, command-line fluency and Bash scripting are non-negotiable core skills.

This master guide breaks down the Linux Filesystem Hierarchy, permission bit arithmetic (chmod/chown), high-performance text pipelines (grep, awk, sed), systemd service management, and concludes with a production-grade Bash backup script.

1. Linux Filesystem Hierarchy (FHS) Explained

Unlike Windows, which organizes storage volumes under distinct drive letters (C:\, D:\), Linux organizes everything as a single inverted tree originating from the Root directory (/):

• /bin & /usr/bin: Essential executable command binaries (e.g., bash, ls, grep).
• /etc: System-wide configuration files (e.g., network settings, SSH daemon config, password hashes).
• /var/log: Persistent application and system runtime log files (e.g., auth.log, syslog, nginx/access.log).
• /home: User personal workspaces and home directories.
• /proc & /sys: Virtual filesystems exposed directly by the Linux kernel representing running processes and hardware state.

2. File Permissions & Ownership Mechanics

Every file and directory in Linux has three permission classes: User (u), Group (g), and Others (o). Permissions are calculated using octal binary values:

• Read (r) = Value 4
• Write (w) = Value 2
• Execute (x) = Value 1

# Make script executable by owner, read-only by group/others (755)
$ chmod 755 backup_script.sh

# Restrict confidential private key to owner read-only (400 or 600)
$ chmod 600 id_rsa

# Recursively change directory ownership to 'devops' user and group
$ chown -R devops:devops /var/www/html

# Inspect permissions and timestamps
$ ls -la

3. High-Performance Text Processing & Log Diagnostics

System administrators spend significant time parsing megabytes of access and error logs. The Linux pipeline trifecta—grep, awk, and sed—allows you to extract actionable intelligence in seconds without launching an editor:

# 1. Search for 500 Internal Server Errors in Nginx logs
$ grep "HTTP/1.1\" 500" /var/log/nginx/access.log

# 2. Extract unique client IP addresses and count frequency using AWK
$ awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -n 10

# 3. Stream real-time log changes while filtering for 'Failed password'
$ tail -f /var/log/auth.log | grep --color=auto "Failed password"

# 4. Stream edit: Replace obsolete database host IP across config files
$ sed -i 's/192.168.1.50/10.0.0.12/g' /etc/app/database.env

4. Systemd Service & Process Management

Modern enterprise Linux distributions (Ubuntu, Debian, RHEL, Rocky Linux) utilize systemd as the init system and service supervisor:

# Start, stop, and restart background daemons
$ sudo systemctl restart nginx

# Enable service to start automatically upon server reboot
$ sudo systemctl enable docker

# Inspect live service health, memory usage, and recent log traces
$ systemctl status postgresql

# Query systemd journal logs for kernel or service failures
$ sudo journalctl -u nginx -n 50 --no-pager

5. Production-Grade Automated Backup Script

Below is a production-ready Bash script that archives a critical web directory, compresses it using tar, creates a sha256 checksum, and purges archives older than 7 days:

#!/usr/bin/env bash
# ==============================================================================
# Automated Server Directory Backup & Retention Policy Script
# Author: IT Hub Official Engineering Practice
# ==============================================================================
set -euo pipefail

BACKUP_SRC="/var/www/html"
BACKUP_DEST="/backup/storage"
TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
ARCHIVE_NAME="site_backup_${TIMESTAMP}.tar.gz"
RETENTION_DAYS=7

# Ensure backup destination exists
mkdir -p "${BACKUP_DEST}"

echo "🚀 [$(date)] Initiating backup of ${BACKUP_SRC}..."

# Create gzip compressed archive
tar -czf "${BACKUP_DEST}/${ARCHIVE_NAME}" "${BACKUP_SRC}"

# Generate SHA256 integrity hash
sha256sum "${BACKUP_DEST}/${ARCHIVE_NAME}" > "${BACKUP_DEST}/${ARCHIVE_NAME}.sha256"

echo "✅ [$(date)] Backup completed: ${BACKUP_DEST}/${ARCHIVE_NAME}"

# Enforce 7-day retention policy to conserve storage
find "${BACKUP_DEST}" -type f -name "site_backup_*.tar.gz*" -mtime +"${RETENTION_DAYS}" -delete
echo "🧹 Retention policy enforced. Archives older than ${RETENTION_DAYS} days removed."

Summary & Daily CLI Checklist

1. Always test destructive operations like rm -rf by echoing parameters first (e.g., echo rm -rf $DIR).
2. Use set -euo pipefail at the top of all production Bash scripts to halt execution immediately when any pipeline fails.
3. Leverage SSH keys (ssh-keygen -t ed25519) and disable password-based SSH authentication in /etc/ssh/sshd_config.