Apache Security and Traffic Monitor

This guide explains how to implement the Apache Security and Traffic Monitor, a powerful log-analysis tool that provides a 24-hour summary of your server's health and visitor patterns. Unlike real-time trackers, this script performs a deep dive into your historical logs to separate human traffic from bots and identify malicious scanning attempts.

Audit Your Traffic: How to Install the Apache Security & Traffic Monitor

To maintain a secure server, you need to understand who is visiting your site and what they are looking for. While real-time alerts are great for immediate events, a daily summary is essential for spotting long-term patterns, such as rising bot traffic or persistent security scans.

This guide uses the logic from the apache-daily-report.sh framework found in our infrastructure sources to build a comprehensive auditing tool.

How the Script Works

  • Log Aggregation: It finds all relevant Apache access logs, including "rotated" (older) compressed files, ensuring no data from the last 24 hours is missed.
  • Time-Based Filtering: Using awk, it extracts only the log entries that have occurred within the last 24 hours.
  • Traffic Segmentation: It uses regular expressions to categorize visitors as "Likely Humans" (based on browser signatures) or "Known Bots" (based on crawler signatures).
  • Security Analysis: It counts specific HTTP error codes (401, 403, 404) and searches for "Suspicious" requests targeting sensitive paths like .env or wp-admin.
  • Risk Assessment: It calculates a traffic quality score and security status based on the volume of bots and suspicious attempts detected.

Step 1: Prepare the Script

Create a new file (e.g., /usr/local/bin/apache_audit.sh) and paste the following generic version of the monitor. Replace the placeholders with your actual server details.    

#!/bin/bash
### Apache Security and Traffic Monitor (24h Summary)

# --- Configuration ---
DOMAIN="example.com"
LOG_DIR="/var/log/apache2"
TELEGRAM_BOT_TOKEN="YOUR_BOT_TOKEN"
CHAT_ID="YOUR_CHAT_ID"
HOSTNAME=$(hostname)
DATE=$(date '+%Y-%m-%d')

# --- Helper: Aggregate Logs ---
get_last_24h_logs() {
    local since_timestamp=$(date --date='24 hours ago' '+%d/%b/%Y:%H:%M:%S')
    # Find all access logs for the domain and pipe them through awk for time filtering
    find "$LOG_DIR" -name "${DOMAIN}*access.log*" -type f | xargs cat 2>/dev/null | \
    awk -v since="$since_timestamp" '{ log_date = substr($4, 2); if (log_date >= since) { print } }'
}

# --- 1. Collect and Filter Data ---
LOG_DATA=$(get_last_24h_logs)
# Remove your own internal health-checks or specific IPs to keep data clean
FILTERED_DATA=$(echo "$LOG_DATA" | grep -v "YOUR_OFFICE_IP")

# --- 2. Build Statistics ---
TOTAL=$(echo "$FILTERED_DATA" | wc -l)
UNIQUE_IPS=$(echo "$FILTERED_DATA" | awk '{print $1}' | sort | uniq | wc -l)
KNOWN_BOTS=$(echo "$FILTERED_DATA" | grep -Ei "bot|spider|crawl|scanner" | wc -l)
LIKELY_HUMANS=$(echo "$FILTERED_DATA" | grep -Ei "chrome|firefox|safari|edge" | grep -v -Ei "bot|spider" | wc -l)

# Security metrics
ERROR_403=$(echo "$FILTERED_DATA" | grep ' 403 ' | wc -l)
SUSPICIOUS=$(echo "$FILTERED_DATA" | grep -Ei "wp-admin|.env|phpmyadmin|.git" | wc -l)

# --- 3. Format Telegram Message ---
MESSAGE="🛡️ *Security Report for ${DOMAIN}*
📊 *Traffic Overview*
- Visitors: ${UNIQUE_IPS} Unique IPs
- Requests: ${TOTAL} Total
- Humans: ~${LIKELY_HUMANS}
- Bots: ~$(($TOTAL - $LIKELY_HUMANS))

🚨 *Security Alerts*
- 403 Forbidden: ${ERROR_403}
- Scan Attempts: ${SUSPICIOUS}"

# Add a simple risk indicator
if [ $SUSPICIOUS -gt 10 ]; then
    MESSAGE="${MESSAGE}\n🔴 *Status: Attention Needed*"
else
    MESSAGE="${MESSAGE}\n🟢 *Status: All Clear*"
fi

# --- 4. Send to Telegram ---
curl -s -X POST "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \
     -d chat_id="${CHAT_ID}" -d parse_mode="Markdown" -d text="$MESSAGE"

 

   Step 2: Installation and Execution

   

  •        Permissions: Make the script executable: sudo chmod +x /usr/local/bin/apache_audit.sh
  • Scheduling: This script is designed to run once a day to summarize the previous 24 hours. Based on standard server practices, schedule it for just before midnight. Open your crontab (sudo crontab -e) and add: 58 23 * * * /usr/local/bin/apache_audit.sh >/dev/null 2>&1

Customizing for Different Scenarios

The filtering and categorization logic used here can be adapted to monitor many other types of activity:

  • API Endpoint Performance: If you run an API, you can modify the script to track the number of 200 OK responses versus 500 Internal Server Error responses. This gives you a daily "Success Rate" report for your developers.
  • Media Hotlinking Tracker: You can customize the grep filters to look specifically for image or video file extensions (.jpg|.mp4) requested by external referrers. This helps identify if other sites are "stealing" your bandwidth by embedding your files.
  • Legacy Content Auditing: If you have moved or deleted old sections of your site, you can use the script to track which specific URLs are still generating 404 Not Found errors. This provides a clear list of pages that need 301 redirects to maintain your SEO.

By using this daily reporting framework, you gain a deep understanding of your server's exposure and traffic quality, allowing you to make data-driven decisions about your security configurations.