Skip to main content

Bash Script Untuk Cek Log Based By Spesific Time

Berikut FItur bash script nya

  • Highlight IP yang hit-nya melebihi threshold (default: >100)
  • Deteksi pola serangan umum, seperti:
    • Bot flood (User-Agent kosong / suspicious)
    • 404 spike
    • Request berulang ke path yang sama
  • Export hasil ke CSV (opsional)
  • Filter berdasarkan User-Agent berbahaya (curl, python-requests, libweb, wget, bot murahan)
  • bisa detek HTTP Status
    • Banyak 5xx → server error (down)
    • Banyak 4xx → client/bot error
    • Banyak 502/504 → upstream/downstream error
    • Banyak 499 → client close connection (biasanya karena lambat → server overload)
    • Dan lain-lain.

Arti Warna BOT

  • [DANGEROUS BOT] → MERAH TERANG

  • [SUSPICIOUS BOT] → KUNING

  • [LEGIT BOT] → HIJAU

  • [UNKNOWN] → NORMAL

Arti Warna HTTP Status

  • Warna status code (2xx hijau, 4xx kuning, 5xx merah)

Berikut Bash Scriptnya

#!/bin/bash

# ======================================================
#   Usage:
#       ./top-ip-range-advanced.sh <logfile> "<start>" "<end>"
#
#   Example:
#       ./top-ip-range-advanced.sh /var/log/nginx/access.log \
#       "18/Nov/2025:09:00" "18/Nov/2025:09:15"
# ======================================================

# ANSI COLORS
RED="\033[1;31m"
YELLOW="\033[1;33m"
GREEN="\033[1;32m"
CYAN="\033[1;36m"
MAGENTA="\033[1;35m"
RESET="\033[0m"

if [ $# -ne 3 ]; then
    echo "Usage: $0 <logfile> <start 'dd/Mon/yyyy:HH:MM'> <end 'dd/Mon/yyyy:HH:MM'>"
    exit 1
fi

LOGFILE="$1"
START="$2"
END="$3"

if [ ! -f "$LOGFILE" ]; then
    echo "Error: File log '$LOGFILE' tidak ditemukan!"
    exit 2
fi

echo "=================================================="
echo " ANALYZING LOG RANGE"
echo " File  : $LOGFILE"
echo " Start : $START"
echo " End   : $END"
echo "=================================================="
echo

TMPFILE=$(mktemp)

# Filter timestamp
awk -v start="[$START" -v end="[$END" '
    $4 >= start && $4 <= end { print }
' "$LOGFILE" > "$TMPFILE"

echo ">> Total entries in range : $(wc -l < $TMPFILE)"
echo


# ======================================================
# HTTP STATUS BREAKDOWN (COLORED)
# ======================================================
echo "=================================================="
echo " HTTP STATUS BREAKDOWN (COLORED)"
echo "=================================================="

awk -v RED="$RED" -v YELLOW="$YELLOW" -v GREEN="$GREEN" -v RESET="$RESET" '
{
    status=$9
    if (status ~ /^5/) color=RED
    else if (status ~ /^4/) color=YELLOW
    else if (status ~ /^2/) color=GREEN
    else color=RESET
    print color status RESET
}' "$TMPFILE" | sort | uniq -c | sort -nr | head -20
echo


# ======================================================
# STATUS SUMMARY (COLORED)
# ======================================================
echo "=================================================="
echo " STATUS SUMMARY (GROUPED + COLORED)"
echo "=================================================="

echo -e "2xx Success:   ${GREEN}$(awk '$9 ~ /^2/ {c++} END {print c+0}' $TMPFILE)${RESET}"
echo -e "3xx Redirect:  ${CYAN}$(awk '$9 ~ /^3/ {c++} END {print c+0}' $TMPFILE)${RESET}"
echo -e "4xx ClientErr: ${YELLOW}$(awk '$9 ~ /^4/ {c++} END {print c+0}' $TMPFILE)${RESET}"
echo -e "5xx ServerErr: ${RED}$(awk '$9 ~ /^5/ {c++} END {print c+0}' $TMPFILE)${RESET}"
echo


# ======================================================
# TOP IP WITH 5xx ERRORS
# ======================================================
echo "=================================================="
echo " TOP IP WITH SERVER ERRORS (5xx)"
echo "=================================================="
awk '$9 ~ /^5/ {print $1}' "$TMPFILE" | sort | uniq -c | sort -nr | head -15
echo


# ======================================================
# TOP IP (HIGHLIGHT TOP 1)
# ======================================================
echo "=================================================="
echo " TOP IP (Top 20, Highlight Highest)"
echo "=================================================="

TOP_IPS=$(awk '{print $1}' "$TMPFILE" | sort | uniq -c | sort -nr | head -20)
TOP1=$(echo "$TOP_IPS" | head -1)

echo "$TOP_IPS" | while read count ip; do
    if [[ "$count $ip" == "$TOP1" ]]; then
        echo -e "${GREEN}[TOP] $count $ip${RESET}"
    else
        echo "$count $ip"
    fi
done
echo


# ======================================================
# BOT DETECTION + CLASSIFICATION + COLOR
# ======================================================
echo "=================================================="
echo " BOT DETECTION (Classified + Highlight)"
echo "=================================================="

LEGIT='googlebot|bingbot|baiduspider|yandexbot|duckduckbot|applebot|slurp'
SUSPICIOUS='curl|wget|spider|crawler|httpclient|node|libwww|java|scrapy'
DANGEROUS='sqlmap|zerodium|dirbuster|acunetix|nmap|nikto|fuzz|attack|python-requests|python|masscan|zgrab|hydra|wpscan|metasploit|bruteforce|Nexus'

grep -Ei "$LEGIT|$SUSPICIOUS|$DANGEROUS" "$TMPFILE" | \
awk -v legit="$LEGIT" -v susp="$SUSPICIOUS" -v danger="$DANGEROUS" \
    -v RED="$RED" -v YELLOW="$YELLOW" -v GREEN="$GREEN" -v RESET="$RESET" '
{
    ip=$1
    ua=""
    for (i=12;i<=NF;i++) ua=ua" "$i

    if (ua ~ danger) label=RED"[DANGEROUS BOT]"RESET
    else if (ua ~ susp) label=YELLOW"[SUSPICIOUS BOT]"RESET
    else if (ua ~ legit) label=GREEN"[LEGIT BOT]"RESET
    else label="[UNKNOWN]"

    print label, ip, ua
}' | sort | uniq -c | sort -nr | head -30
echo


# ======================================================
# GROUP IP + STATUS + PATH
# ======================================================
echo "=================================================="
echo " GROUPED BY IP + STATUS + PATH"
echo "=================================================="

awk '{
    ip=$1;
    status=$9;
    path=$7;
    key=ip" "status" "path;
    count[key]++;
}
END {
    for (k in count) print count[k], k;
}' "$TMPFILE" | sort -nr | head -50
echo


# ======================================================
# 404 SPIKE (HIGHLIGHT TOP 1)
# ======================================================
echo "=================================================="
echo " TOP 404 PATH (Highlight Spike)"
echo "=================================================="

DATA_404=$(grep ' 404 ' "$TMPFILE" | awk '{print $7}' | sort | uniq -c | sort -nr | head -15)
TOP_404=$(echo "$DATA_404" | head -1)

echo "$DATA_404" | while read count path; do
    if [[ "$count $path" == "$TOP_404" ]]; then
        echo -e "${RED}[SPIKE] $count $path${RESET}"
    else
        echo "$count $path"
    fi
done
echo


# ======================================================
# MOST ACCESSED PATH
# ======================================================
echo "=================================================="
echo " MOST ACCESSED PATH (Top 20)"
echo "=================================================="
awk '{print $7}' "$TMPFILE" | sort | uniq -c | sort -nr | head -20

rm -f "$TMPFILE"

Berikut Contoh Hasil nya

sudo ./top-ip-range-advanced.sh /var/log/nginx/middle-east.cotecna.com.access.log "21/Nov/2025:02:45" "21/Nov/2025:03:15"

==================================================
 ANALYZING LOG RANGE
 File  : /var/log/nginx/middle-east.cotecna.com.access.log
 Start : 21/Nov/2025:02:45
 End   : 21/Nov/2025:03:15
==================================================

>> Total entries in range : 76

==================================================
 HTTP STATUS BREAKDOWN (COLORED)
==================================================
     30 200
     23 302
     20 301
      3 404

==================================================
 STATUS SUMMARY (GROUPED + COLORED)
==================================================
2xx Success:   30
3xx Redirect:  43
4xx ClientErr: 3
5xx ServerErr: 0

==================================================
 TOP IP WITH SERVER ERRORS (5xx)
==================================================

==================================================
 TOP IP (Top 20, Highlight Highest)
==================================================
6 64.227.150.179
6 167.99.201.22
3 45.32.141.163
3 34.96.139.85
3 34.95.163.121
3 34.92.171.119
3 34.65.97.251
3 34.65.146.108
3 34.34.97.213
3 34.165.80.145
3 178.62.85.105
3 167.172.163.238
3 143.110.177.254
3 138.68.252.209
3 111.39.249.249
2 43.130.72.40
2 35.197.232.17
2 167.172.171.150
1 89.246.241.14
1 66.249.68.4

==================================================
 BOT DETECTION (Classified + Highlight)
==================================================
      1 [LEGIT BOT] 40.77.167.130  "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/116.0.1938.76 Safari/537.36"
      1 [DANGEROUS BOT] 66.249.68.4  "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.7390.122 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

==================================================
 GROUPED BY IP + STATUS + PATH
==================================================
2 43.130.72.40 200 /en
2 167.99.201.22 302 /
2 167.99.201.22 301 /en/
2 167.99.201.22 200 /en
1 89.246.241.14 302 /
1 66.249.68.4 301 /en/
1 64.227.150.179 404 /en/uploads
1 64.227.150.179 404 /en/.well-known
1 64.227.150.179 302 /uploads/
1 64.227.150.179 302 /.well-known/
1 64.227.150.179 301 /en/uploads/
1 64.227.150.179 301 /en/.well-known/
1 49.51.141.76 200 /en
1 45.32.141.163 302 /
1 45.32.141.163 301 /en/
1 45.32.141.163 200 /en
1 43.164.197.209 200 /en
1 43.163.104.54 302 /en
1 43.157.62.101 200 /en
1 43.157.22.109 200 /en
1 43.153.192.98 200 /en
1 43.131.45.213 200 /en
1 40.77.167.130 404 /en/news/tea-export-from-sri-lanka-to-saudi-arabia
1 35.197.232.17 302 /
1 35.197.232.17 301 /en/
1 34.96.139.85 302 /
1 34.96.139.85 301 /en/
1 34.96.139.85 200 /en
1 34.95.163.121 302 /
1 34.95.163.121 301 /en/
1 34.95.163.121 200 /en
1 34.92.171.119 302 /
1 34.92.171.119 301 /en/
1 34.92.171.119 200 /en
1 34.65.97.251 302 /
1 34.65.97.251 301 /en/
1 34.65.97.251 200 /en
1 34.65.146.108 302 /
1 34.65.146.108 301 /en/
1 34.65.146.108 200 /en
1 34.34.97.213 302 /
1 34.34.97.213 301 /en/
1 34.34.97.213 200 /en
1 34.165.80.145 302 /
1 34.165.80.145 301 /en/
1 34.165.80.145 200 /en
1 220.181.51.92 200 /ar/cookie-policy
1 178.62.85.105 302 /
1 178.62.85.105 301 /en/
1 178.62.85.105 200 /en

==================================================
 TOP 404 PATH (Highlight Spike)
==================================================
1 /en/uploads
1 /en/news/tea-export-from-sri-lanka-to-saudi-arabia
1 /en/.well-known

==================================================
 MOST ACCESSED PATH (Top 20)
==================================================
     31 /en
     18 /en/
     18 /
      1 /uploads/
      1 /en/uploads/
      1 /en/uploads
      1 /en/news/tea-export-from-sri-lanka-to-saudi-arabia
      1 /en/.well-known/
      1 /en/.well-known
      1 /cookie/api/v1/CookieMaker/Generate?locale=ar
      1 /ar/cookie-policy
      1 /.well-known/

Berikut potongan hasil nya

image.png

Berikut keuntungan yang paling penting dari script tersebut

✅ 1. Troubleshooting Lebih Cepat

Saat incident, biasanya kamu tau kira-kira jam berapa masalah terjadi.

Misal:

  • Web down jam 09.05 — 09.10

  • CPU spike jam 15.41 — 15.45

  • Banyak client komplain jam 11.00

Logging yang ditarik hanya dari range waktu itu bikin kamu:

  • langsung lihat IP apa yang traffic-nya naik

  • lihat apakah ada lonjakan 404, 500, atau timeout

  • langsung tahu siapa yang melakukan flood request

➡️ Menghemat waktu sangat besar.

✅ 2. Mendeteksi Serangan Lebih Akurat

Serangan (bot, crawling, bruteforce, scanning) biasanya terjadi dalam burst waktu tertentu.

Dengan time-range checking, kamu bisa tahu:

  • siapa yang spamming request dalam 3 menit terakhir

  • path apa yang di-scan oleh attacker

  • apakah ada bot berbahaya seperti
    sqlmap, dirbuster, wpscan, masscan, zgrab

➡️ Kamu bisa ambil tindakan cepat (block IP / firewall / rate-limit).

✅ 3. Analisis Performa & Bottleneck

Misal kamu deploy update jam 14.00, lalu web lemot jam 14.10.

Dengan time-range filter:

  • langsung terlihat apakah banyak 500 (server error)

  • apakah banyak request ke API tertentu

  • apakah access log naik drastis

➡️ Bisa fokus ke waktu setelah deploy, bukan keseluruhan log.

✅ 4. Memudahkan Korelasi Dengan Alert dari Monitoring

Biasanya Prometheus / Grafana / Datadog / CloudWatch kasih warning:

Response time increased at 10:32

Kalau kamu filter log range 10:30 – 10:35, kamu bisa lihat:

  • IP flooding?

  • Path mana lonjakannya?

  • Ada bot akses aneh?

  • Ada 5xx yang tiba-tiba muncul?

➡️ Korelasi cepat antara monitoring dan log.

✅ 5. File Log Jadi “Ringkas”, Mudah Dibaca

Daripada buka log harian (misal 500 MB), kamu cukup lihat 2–10 menit saja.

Lebih ringan, lebih cepat, lebih enak dibaca.

✅ 6. Mempermudah Audit & Keamanan

Saat audit security atau compliance, kadang ditanya:

"Apa yang terjadi pada jam 03.15 — 03.17 saat kami lihat anomalous login?"

Dengan script time-range:

  • kamu bisa reconstruct kejadian

  • lihat client IP, user-agent, path yang diakses

➡️ Tool wajib buat forensic.

🔥 Intinya

Cek log berdasarkan waktu spesifik itu membuat analisis:

  • lebih presisi

  • lebih cepat

  • lebih akurat

  • lebih aman

  • lebih mudah korelasi dengan kejadian lain




No Comments
Back to top