Chapter 13: Real-World Command Line Use Cases

The fluorescent lights hummed softly in the server room as Sarah, a seasoned system administrator, walked between the towering racks of servers. Each machine represented countless hours of configuration, monitoring, and maintenance—all orchestrated through the elegant simplicity of the command line. Tonight, she would be training her new colleague, Marcus, on the real-world applications that separate novice users from command line masters.

"The beauty of the Linux command line," Sarah began, settling into her workstation, "isn't just in knowing individual commands. It's in understanding how to combine them to solve real problems efficiently." The glow of multiple terminal windows reflected in her glasses as she prepared to demonstrate the practical scenarios that every Linux professional encounters.

System Administration Scenarios

Server Health Monitoring

The first lesson began with a scenario familiar to every system administrator: monitoring server health during peak hours. Sarah's fingers danced across the keyboard as she explained the situation.

"Imagine it's Black Friday, and our e-commerce servers are under heavy load. We need to monitor system resources in real-time and identify potential bottlenecks before they become critical issues."

# Real-time system monitoring command

top -d 1 -p $(pgrep -d',' nginx)

Command Breakdown:

"But that's just the beginning," Sarah continued, opening another terminal window. "Let's create a comprehensive monitoring script that checks multiple system parameters."

#!/bin/bash

# comprehensive_monitor.sh - Real-time system health checker

echo "=== System Health Monitor - $(date) ==="

echo

# CPU Usage

echo "CPU Usage:"

grep 'cpu ' /proc/stat | awk '{usage=($2+$4)*100/($2+$4+$5)} END {print usage "%"}'

# Memory Usage

echo -e "\nMemory Usage:"

free -h | awk 'NR==2{printf "Memory Usage: %s/%s (%.2f%%)\n", $3,$2,$3*100/$2 }'

# Disk Usage

echo -e "\nDisk Usage:"

df -h | awk '$NF=="/"{printf "Disk Usage: %d/%dGB (%s)\n", $3,$2,$5}'

# Network Connections

echo -e "\nActive Network Connections:"

netstat -tuln | grep LISTEN | wc -l | awk '{print $1 " listening ports"}'

# Load Average

echo -e "\nLoad Average:"

uptime | awk -F'load average:' '{print $2}'

# Top 5 CPU consuming processes

echo -e "\nTop 5 CPU Consumers:"

ps aux --sort=-%cpu | head -6 | tail -5 | awk '{print $11 " - " $3"%"}'

Script Notes:

Marcus watched intently as the script executed, displaying a clean, formatted report of the system's current state. "This is exactly what we need during high-traffic periods," he noted.

Log Analysis and Troubleshooting

Sarah's next demonstration focused on log analysis—a critical skill for identifying and resolving system issues. She navigated to the /var/log directory, where countless log files chronicled the server's activities.

"Last week, we experienced intermittent connection issues. Let's investigate using command line tools to analyze our web server logs."

# Analyze Apache access logs for error patterns

grep "$(date --date='yesterday' '+%d/%b/%Y')" /var/log/apache2/access.log | \

awk '$9 >= 400 {print $1, $7, $9}' | \

sort | uniq -c | sort -nr | head -10

Command Analysis:

"Now, let's create a more sophisticated log analysis tool," Sarah said, opening her text editor.

#!/bin/bash

# log_analyzer.sh - Advanced web server log analysis

LOG_FILE="/var/log/apache2/access.log"

DATE_FILTER=${1:-$(date '+%d/%b/%Y')}

echo "=== Web Server Log Analysis for $DATE_FILTER ==="

echo

# Total requests

TOTAL_REQUESTS=$(grep "$DATE_FILTER" "$LOG_FILE" | wc -l)

echo "Total Requests: $TOTAL_REQUESTS"

# Requests by status code

echo -e "\nRequests by Status Code:"

grep "$DATE_FILTER" "$LOG_FILE" | awk '{print $9}' | sort | uniq -c | sort -nr

# Top 10 IP addresses

echo -e "\nTop 10 IP Addresses:"

grep "$DATE_FILTER" "$LOG_FILE" | awk '{print $1}' | sort | uniq -c | sort -nr | head -10

# Top 10 requested pages

echo -e "\nTop 10 Requested Pages:"

grep "$DATE_FILTER" "$LOG_FILE" | awk '{print $7}' | sort | uniq -c | sort -nr | head -10

# Error analysis (4xx and 5xx status codes)

echo -e "\nError Analysis:"

grep "$DATE_FILTER" "$LOG_FILE" | awk '$9 >= 400 {print $9}' | sort | uniq -c | sort -nr

# Bandwidth usage (approximate)

echo -e "\nBandwidth Usage:"

TOTAL_BYTES=$(grep "$DATE_FILTER" "$LOG_FILE" | awk '{sum+=$10} END {print sum}')

echo "Total Bytes Transferred: $(echo "$TOTAL_BYTES" | awk '{printf "%.2f MB", $1/1024/1024}')"

# Peak hour analysis

echo -e "\nPeak Hour Analysis:"

grep "$DATE_FILTER" "$LOG_FILE" | awk '{print substr($4, 13, 2)}' | sort | uniq -c | sort -nr | head -5

Advanced Features:

Automated Backup Solutions

The conversation turned to backup strategies—a topic close to every administrator's heart. Sarah demonstrated how command line tools could create robust, automated backup solutions.

"Backups are only as good as their reliability and recoverability," she emphasized, crafting a comprehensive backup script.

#!/bin/bash

# intelligent_backup.sh - Smart backup system with rotation

# Configuration

BACKUP_SOURCE="/var/www/html"

BACKUP_DEST="/backup/web"

RETENTION_DAYS=30

LOG_FILE="/var/log/backup.log"

EMAIL_ALERT="admin@company.com"

# Function to log messages

log_message() {

echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"

}

# Function to send email alerts

send_alert() {

echo "$1" | mail -s "Backup Alert - $(hostname)" "$EMAIL_ALERT"

}

# Create backup directory structure

BACKUP_DATE=$(date '+%Y%m%d_%H%M%S')

BACKUP_PATH="$BACKUP_DEST/$BACKUP_DATE"

log_message "Starting backup process"

# Verify source directory exists

if [[ ! -d "$BACKUP_SOURCE" ]]; then

log_message "ERROR: Source directory $BACKUP_SOURCE does not exist"

send_alert "Backup failed: Source directory missing"

exit 1

fi

# Create backup destination

mkdir -p "$BACKUP_PATH"

# Perform backup with compression and verification

log_message "Creating compressed archive"

tar -czf "$BACKUP_PATH/backup.tar.gz" -C "$(dirname "$BACKUP_SOURCE")" "$(basename "$BACKUP_SOURCE")" 2>/dev/null

# Verify backup integrity

if tar -tzf "$BACKUP_PATH/backup.tar.gz" >/dev/null 2>&1; then

BACKUP_SIZE=$(du -sh "$BACKUP_PATH/backup.tar.gz" | cut -f1)

log_message "Backup completed successfully - Size: $BACKUP_SIZE"

else

log_message "ERROR: Backup verification failed"

send_alert "Backup failed: Archive verification error"

exit 1

fi

# Cleanup old backups

log_message "Cleaning up old backups (older than $RETENTION_DAYS days)"

find "$BACKUP_DEST" -type d -mtime +$RETENTION_DAYS -exec rm -rf {} + 2>/dev/null

# Generate backup report

BACKUP_COUNT=$(find "$BACKUP_DEST" -maxdepth 1 -type d | wc -l)

TOTAL_SIZE=$(du -sh "$BACKUP_DEST" | cut -f1)

log_message "Backup report: $BACKUP_COUNT backups, Total size: $TOTAL_SIZE"

# Create symbolic link to latest backup

ln -sf "$BACKUP_PATH" "$BACKUP_DEST/latest"

log_message "Backup process completed"

Key Features:

Development Workflows

Git Repository Management

Sarah shifted focus to development scenarios, demonstrating how command line mastery accelerates software development workflows. She opened a terminal in a sample project directory.

"Effective Git usage goes beyond basic add-commit-push cycles. Let's explore advanced Git operations that streamline development."

#!/bin/bash

# git_workflow_helper.sh - Advanced Git workflow automation

# Function to display current branch status

show_status() {

echo "=== Repository Status ==="

echo "Current branch: $(git branch --show-current)"

echo "Uncommitted changes: $(git status --porcelain | wc -l)"

echo "Commits ahead of origin: $(git rev-list --count HEAD ^origin/$(git branch --show-current) 2>/dev/null || echo "0")"

echo

}

# Function to create feature branch with naming convention

create_feature_branch() {

local feature_name="$1"

local branch_name="feature/$(date +%Y%m%d)-$feature_name"

git checkout -b "$branch_name"

git push -u origin "$branch_name"

echo "Created and pushed feature branch: $branch_name"

}

# Function to perform intelligent commit

smart_commit() {

# Check for common issues before committing

if grep -r "TODO\|FIXME\|console\.log\|debugger" . --exclude-dir=.git --exclude-dir=node_modules; then

echo "Warning: Found TODO/FIXME/debug statements. Continue? (y/N)"

read -r response

[[ "$response" != "y" ]] && return 1

fi

# Stage all changes and create commit with detailed message

git add .

git commit -m "$1" -m "$(git diff --cached --stat)"

}

# Function to sync with remote repository

sync_repository() {

local current_branch=$(git branch --show-current)

# Fetch latest changes

git fetch origin

# Check if behind remote

local behind_count=$(git rev-list --count HEAD..origin/"$current_branch" 2>/dev/null || echo "0")

if [[ "$behind_count" -gt 0 ]]; then

echo "Branch is $behind_count commits behind origin. Pulling changes..."

git pull origin "$current_branch"

fi

# Push local commits

local ahead_count=$(git rev-list --count origin/"$current_branch"..HEAD 2>/dev/null || echo "0")

if [[ "$ahead_count" -gt 0 ]]; then

echo "Pushing $ahead_count local commits..."

git push origin "$current_branch"

fi

}

# Main script logic

case "$1" in

"status")

show_status

;;

"feature")

create_feature_branch "$2"

;;

"commit")

smart_commit "$2"

;;

"sync")

sync_repository

;;

*)

echo "Usage: $0 {status|feature|commit|sync}"

echo " status - Show detailed repository status"

echo " feature <name> - Create feature branch with naming convention"

echo " commit <msg> - Perform intelligent commit with checks"

echo " sync - Synchronize with remote repository"

;;

esac

Advanced Git Features:

Automated Testing and Deployment

The final development scenario focused on continuous integration and deployment—areas where command line automation truly shines.

#!/bin/bash

# ci_cd_pipeline.sh - Continuous Integration and Deployment Pipeline

# Configuration

PROJECT_DIR="/var/www/myapp"

TEST_DIR="$PROJECT_DIR/tests"

DEPLOY_DIR="/var/www/production"

LOG_DIR="/var/log/deployment"

# Create log directory

mkdir -p "$LOG_DIR"

DEPLOY_LOG="$LOG_DIR/deploy_$(date +%Y%m%d_%H%M%S).log"

# Function to log with timestamp

log() {

echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$DEPLOY_LOG"

}

# Function to run tests

run_tests() {

log "Starting test suite"

cd "$PROJECT_DIR" || exit 1

# Run different types of tests

if [[ -f "package.json" ]]; then

log "Running npm tests"

npm test 2>&1 | tee -a "$DEPLOY_LOG"

TEST_RESULT=${PIPESTATUS[0]}

elif [[ -f "requirements.txt" ]]; then

log "Running Python tests"

python -m pytest tests/ -v 2>&1 | tee -a "$DEPLOY_LOG"

TEST_RESULT=${PIPESTATUS[0]}

else

log "Running custom test script"

if [[ -x "$TEST_DIR/run_tests.sh" ]]; then

"$TEST_DIR/run_tests.sh" 2>&1 | tee -a "$DEPLOY_LOG"

TEST_RESULT=${PIPESTATUS[0]}

else

log "No test framework detected"

TEST_RESULT=0

fi

fi

return $TEST_RESULT

}

# Function to build application

build_application() {

log "Building application"

cd "$PROJECT_DIR" || exit 1

# Build based on project type

if [[ -f "package.json" ]]; then

npm run build 2>&1 | tee -a "$DEPLOY_LOG"

return ${PIPESTATUS[0]}

elif [[ -f "Makefile" ]]; then

make build 2>&1 | tee -a "$DEPLOY_LOG"

return ${PIPESTATUS[0]}

else

log "No build process required"

return 0

fi

}

# Function to deploy application

deploy_application() {

log "Starting deployment"

# Create backup of current deployment

if [[ -d "$DEPLOY_DIR" ]]; then

BACKUP_DIR="/backup/deployment_$(date +%Y%m%d_%H%M%S)"

log "Creating backup at $BACKUP_DIR"

cp -r "$DEPLOY_DIR" "$BACKUP_DIR"

fi

# Deploy new version

log "Copying files to production"

rsync -av --delete "$PROJECT_DIR/" "$DEPLOY_DIR/" \

--exclude='.git' \

--exclude='node_modules' \

--exclude='tests' \

--exclude='*.log' 2>&1 | tee -a "$DEPLOY_LOG"

# Restart services if needed

if systemctl is-active --quiet apache2; then

log "Restarting Apache"

systemctl restart apache2

fi

if systemctl is-active --quiet nginx; then

log "Restarting Nginx"

systemctl restart nginx

fi

log "Deployment completed"

}

# Function to rollback deployment

rollback_deployment() {

log "Starting rollback process"

LATEST_BACKUP=$(find /backup -name "deployment_*" -type d | sort | tail -1)

if [[ -n "$LATEST_BACKUP" ]]; then

log "Rolling back to $LATEST_BACKUP"

rsync -av --delete "$LATEST_BACKUP/" "$DEPLOY_DIR/"

# Restart services

systemctl restart apache2 nginx

log "Rollback completed"

else

log "ERROR: No backup found for rollback"

return 1

fi

}

# Main pipeline execution

main() {

log "=== Starting CI/CD Pipeline ==="

# Run tests

if run_tests; then

log "Tests passed"

else

log "Tests failed - aborting deployment"

exit 1

fi

# Build application

if build_application; then

log "Build successful"

else

log "Build failed - aborting deployment"

exit 1

fi

# Deploy application

if deploy_application; then

log "Deployment successful"

# Send success notification

echo "Deployment completed successfully at $(date)" | \

mail -s "Deployment Success - $(hostname)" admin@company.com

else

log "Deployment failed - initiating rollback"

rollback_deployment

exit 1

fi

log "=== CI/CD Pipeline Completed ==="

}

# Script execution

case "$1" in

"deploy")

main

;;

"test")

run_tests

;;

"build")

build_application

;;

"rollback")

rollback_deployment

;;

*)

echo "Usage: $0 {deploy|test|build|rollback}"

echo " deploy - Run full CI/CD pipeline"

echo " test - Run tests only"

echo " build - Build application only"

echo " rollback - Rollback to previous deployment"

;;

esac

Pipeline Features:

Data Processing Tasks

Large File Analysis

As the evening progressed, Sarah demonstrated how command line tools excel at processing large datasets—a common requirement in modern data-driven environments.

"Last month, we needed to analyze 50GB of server logs to identify performance patterns. Traditional tools would have crashed, but the command line handled it elegantly."

#!/bin/bash

# big_data_processor.sh - Efficient large file processing

INPUT_FILE="$1"

OUTPUT_DIR="./analysis_results"

# Validate input

if [[ ! -f "$INPUT_FILE" ]]; then

echo "Error: File $INPUT_FILE not found"

exit 1

fi

# Create output directory

mkdir -p "$OUTPUT_DIR"

echo "Processing large file: $INPUT_FILE"

echo "File size: $(du -sh "$INPUT_FILE" | cut -f1)"

echo "Processing started at: $(date)"

# Split large file into manageable chunks for parallel processing

split -l 100000 "$INPUT_FILE" "$OUTPUT_DIR/chunk_"

# Process chunks in parallel

process_chunk() {

local chunk_file="$1"

local chunk_name=$(basename "$chunk_file")

# Extract unique IP addresses

awk '{print $1}' "$chunk_file" | sort | uniq > "$OUTPUT_DIR/ips_$chunk_name"

# Extract status codes

awk '{print $9}' "$chunk_file" | sort | uniq -c | sort -nr > "$OUTPUT_DIR/status_$chunk_name"

# Extract hourly patterns

awk '{print substr($4, 13, 2)}' "$chunk_file" | sort | uniq -c > "$OUTPUT_DIR/hours_$chunk_name"

}

# Export function for parallel execution

export -f process_chunk

export OUTPUT_DIR

# Process all chunks in parallel

find "$OUTPUT_DIR" -name "chunk_*" | xargs -P 4 -I {} bash -c 'process_chunk "$@"' _ {}

# Combine results from all chunks

echo "Combining results..."

# Combine IP addresses

cat "$OUTPUT_DIR"/ips_chunk_* | sort | uniq > "$OUTPUT_DIR/unique_ips.txt"

IP_COUNT=$(wc -l < "$OUTPUT_DIR/unique_ips.txt")

# Combine status codes

cat "$OUTPUT_DIR"/status_chunk_* | awk '{a[$2]+=$1} END {for(i in a) print a[i], i}' | sort -nr > "$OUTPUT_DIR/status_summary.txt"

# Combine hourly patterns

cat "$OUTPUT_DIR"/hours_chunk_* | awk '{a[$2]+=$1} END {for(i in a) print a[i], i}' | sort -k2n > "$OUTPUT_DIR/hourly_pattern.txt"

# Generate final report

cat > "$OUTPUT_DIR/analysis_report.txt" << EOF

=== Large File Analysis Report ===

Generated: $(date)

Source File: $INPUT_FILE

File Size: $(du -sh "$INPUT_FILE" | cut -f1)

Total Lines: $(wc -l < "$INPUT_FILE")

Unique IP Addresses: $IP_COUNT

Top 10 Status Codes:

$(head -10 "$OUTPUT_DIR/status_summary.txt")

Hourly Traffic Pattern:

$(cat "$OUTPUT_DIR/hourly_pattern.txt")

EOF

# Cleanup temporary files

rm "$OUTPUT_DIR"/chunk_* "$OUTPUT_DIR"/ips_chunk_* "$OUTPUT_DIR"/status_chunk_* "$OUTPUT_DIR"/hours_chunk_*

echo "Analysis completed at: $(date)"

echo "Results saved in: $OUTPUT_DIR"

Performance Optimizations:

As Sarah saved the final script, Marcus reflected on the evening's lessons. The command line had transformed from a collection of individual tools into a powerful ecosystem for solving complex, real-world problems.

"The key insight," Sarah concluded, "is that mastery comes not from memorizing every command option, but from understanding how to combine simple tools to create sophisticated solutions. Each script we've written tonight solves actual problems we face daily—monitoring systems, analyzing logs, managing deployments, and processing data."

The server room's ambient hum seemed to underscore her point. Behind every successful Linux environment lay countless hours of thoughtful automation, careful monitoring, and efficient problem-solving—all orchestrated through the elegant simplicity of the command line.

Through these real-world scenarios, the true power of Linux command line mastery becomes apparent: the ability to transform complex operational challenges into elegant, automated solutions that scale with your infrastructure and adapt to your organization's evolving needs.