aiwaf 0.1.0__tar.gz → 0.1.3__tar.gz

aiwaf-0.1.3/PKG-INFO

@@ -0,0 +1,181 @@
+Metadata-Version: 2.4
+Name: aiwaf
+Version: 0.1.3
+Summary: AI-powered Web Application Firewall
+Author: Aayush Gauba
+Author-email: Aayush Gauba <gauba.aayush@gmail.com>
+License: MIT
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Dynamic: author
+
+# AI‑WAF
+
+> A self‑learning, Django‑friendly Web Application Firewall  
+> with rate‑limiting, anomaly detection, honeypots, UUID‑tamper protection, dynamic keyword extraction, file‑extension probing detection, and daily retraining.
+
+---
+
+## Package Structure
+
+```
+aiwaf/
+├── __init__.py
+├── blacklist_manager.py
+├── middleware.py
+├── trainer.py                   # exposes train()
+├── utils.py
+├── template_tags/
+│   └── aiwaf_tags.py
+├── resources/
+│   ├── model.pkl                # pre‑trained base model
+│   └── dynamic_keywords.json    # evolves daily
+├── management/
+│   └── commands/
+│       └── detect_and_train.py  # `python manage.py detect_and_train`
+└── LICENSE
+```
+
+---
+
+## Features
+
+- **IP Blocklist**  
+  Instantly blocks suspicious IPs (supports CSV fallback or Django model).
+
+- **Rate Limiting**  
+  Sliding‑window blocks flooders (> `AIWAF_RATE_MAX` per `AIWAF_RATE_WINDOW`), then blacklists them.
+
+- **AI Anomaly Detection**  
+  IsolationForest on features:
+  - Path length  
+  - Keyword hits (static + dynamic)  
+  - Response time  
+  - Status‑code index  
+  - Burst count  
+  - Total 404s  
+
+- **Dynamic Keyword Extraction**  
+  Every retrain: top 10 most frequent “words” from 4xx/5xx paths are appended to your malicious keyword set.
+
+- **File‑Extension Probing Detection**  
+  Tracks repeated 404s on common web‑extensions (e.g. `.php`, `.asp`) and auto‑blocks after a burst.
+
+- **Honeypot Field**  
+  Hidden form field (via template tag) that bots fill → instant block.
+
+- **UUID Tampering Protection**  
+  Any `<uuid:…>` URL that doesn’t map to **any** model in its Django app gets blocked.
+
+- **Daily Retraining**  
+  Reads rotated/gzipped logs, auto‑blocks 404 floods (≥6), retrains the model, updates `model.pkl` + `dynamic_keywords.json`.
+
+---
+
+## Installation
+
+```bash
+# From PyPI
+pip install aiwaf
+
+# Or for local development
+git clone https://github.com/aayushgauba/aiwaf.git
+cd aiwaf
+pip install -e .
+```
+
+---
+
+## ⚙️ Configuration (`settings.py`)
+
+```python
+INSTALLED_APPS += ["aiwaf"]
+
+# Required
+AIWAF_ACCESS_LOG = "/var/log/nginx/access.log"
+
+# Optional (defaults shown)
+AIWAF_MODEL_PATH         = BASE_DIR / "aiwaf" / "resources" / "model.pkl"
+AIWAF_HONEYPOT_FIELD     = "hp_field"
+AIWAF_RATE_WINDOW        = 10         # seconds
+AIWAF_RATE_MAX           = 20         # max reqs/window
+AIWAF_RATE_FLOOD         = 10         # flood threshold
+AIWAF_WINDOW_SECONDS     = 60         # anomaly window
+AIWAF_FILE_EXTENSIONS    = [".php", ".asp", ".jsp"]  # 404‑burst tracked extensions
+```
+
+> **Note:** You no longer need to define `AIWAF_MALICIOUS_KEYWORDS` or `AIWAF_STATUS_CODES` in your settings — they’re built in and evolve dynamically.
+
+---
+
+## Middleware Setup
+
+Add in **this** order to your `MIDDLEWARE` list:
+
+```python
+MIDDLEWARE = [
+    "aiwaf.middleware.IPBlockMiddleware",
+    "aiwaf.middleware.RateLimitMiddleware",
+    "aiwaf.middleware.AIAnomalyMiddleware",
+    "aiwaf.middleware.HoneypotMiddleware",
+    "aiwaf.middleware.UUIDTamperMiddleware",
+    # ... other middleware ...
+]
+```
+
+---
+
+## Honeypot Field (in your template)
+
+```django
+{% load aiwaf_tags %}
+
+<form method="post">
+  {% csrf_token %}
+  {% honeypot_field %}
+  <!-- your real fields -->
+</form>
+```
+
+> Renders a hidden `<input name="hp_field" style="display:none">`.  
+> Any non‑empty submission → IP blacklisted.
+
+---
+
+## Running Detection & Training
+
+```bash
+python manage.py detect_and_train
+```
+
+**What happens:**
+1. Read access logs
+2. Auto‑block IPs with ≥ 6 total 404s
+3. Extract features & train IsolationForest
+4. Save `model.pkl`
+5. Extract top 10 dynamic keywords from 4xx/5xx
+
+---
+
+## How It Works
+
+| Middleware               | Purpose                                                         |
+|--------------------------|------------------------------------------------------------------|
+| IPBlockMiddleware        | Blocks requests from known blacklisted IPs                      |
+| RateLimitMiddleware      | Enforces burst & flood thresholds                               |
+| AIAnomalyMiddleware      | ML‑driven behavior analysis + block on anomaly                  |
+| HoneypotMiddleware       | Detects bots filling hidden inputs in forms                     |
+| UUIDTamperMiddleware     | Blocks guessed/nonexistent UUIDs across all models in an app    |
+
+---
+
+## License
+
+This project is licensed under the **MIT License**. See the [LICENSE](LICENSE) file for details.
+
+---
+
+## Credits
+
+**AI‑WAF** by [Aayush Gauba](https://github.com/aayushgauba)  
+> “Let your firewall learn and evolve — keep your site a fortress.”

aiwaf-0.1.3/README.md

@@ -0,0 +1,170 @@
+# AI‑WAF
+
+> A self‑learning, Django‑friendly Web Application Firewall  
+> with rate‑limiting, anomaly detection, honeypots, UUID‑tamper protection, dynamic keyword extraction, file‑extension probing detection, and daily retraining.
+
+---
+
+## Package Structure
+
+```
+aiwaf/
+├── __init__.py
+├── blacklist_manager.py
+├── middleware.py
+├── trainer.py                   # exposes train()
+├── utils.py
+├── template_tags/
+│   └── aiwaf_tags.py
+├── resources/
+│   ├── model.pkl                # pre‑trained base model
+│   └── dynamic_keywords.json    # evolves daily
+├── management/
+│   └── commands/
+│       └── detect_and_train.py  # `python manage.py detect_and_train`
+└── LICENSE
+```
+
+---
+
+## Features
+
+- **IP Blocklist**  
+  Instantly blocks suspicious IPs (supports CSV fallback or Django model).
+
+- **Rate Limiting**  
+  Sliding‑window blocks flooders (> `AIWAF_RATE_MAX` per `AIWAF_RATE_WINDOW`), then blacklists them.
+
+- **AI Anomaly Detection**  
+  IsolationForest on features:
+  - Path length  
+  - Keyword hits (static + dynamic)  
+  - Response time  
+  - Status‑code index  
+  - Burst count  
+  - Total 404s  
+
+- **Dynamic Keyword Extraction**  
+  Every retrain: top 10 most frequent “words” from 4xx/5xx paths are appended to your malicious keyword set.
+
+- **File‑Extension Probing Detection**  
+  Tracks repeated 404s on common web‑extensions (e.g. `.php`, `.asp`) and auto‑blocks after a burst.
+
+- **Honeypot Field**  
+  Hidden form field (via template tag) that bots fill → instant block.
+
+- **UUID Tampering Protection**  
+  Any `<uuid:…>` URL that doesn’t map to **any** model in its Django app gets blocked.
+
+- **Daily Retraining**  
+  Reads rotated/gzipped logs, auto‑blocks 404 floods (≥6), retrains the model, updates `model.pkl` + `dynamic_keywords.json`.
+
+---
+
+## Installation
+
+```bash
+# From PyPI
+pip install aiwaf
+
+# Or for local development
+git clone https://github.com/aayushgauba/aiwaf.git
+cd aiwaf
+pip install -e .
+```
+
+---
+
+## ⚙️ Configuration (`settings.py`)
+
+```python
+INSTALLED_APPS += ["aiwaf"]
+
+# Required
+AIWAF_ACCESS_LOG = "/var/log/nginx/access.log"
+
+# Optional (defaults shown)
+AIWAF_MODEL_PATH         = BASE_DIR / "aiwaf" / "resources" / "model.pkl"
+AIWAF_HONEYPOT_FIELD     = "hp_field"
+AIWAF_RATE_WINDOW        = 10         # seconds
+AIWAF_RATE_MAX           = 20         # max reqs/window
+AIWAF_RATE_FLOOD         = 10         # flood threshold
+AIWAF_WINDOW_SECONDS     = 60         # anomaly window
+AIWAF_FILE_EXTENSIONS    = [".php", ".asp", ".jsp"]  # 404‑burst tracked extensions
+```
+
+> **Note:** You no longer need to define `AIWAF_MALICIOUS_KEYWORDS` or `AIWAF_STATUS_CODES` in your settings — they’re built in and evolve dynamically.
+
+---
+
+## Middleware Setup
+
+Add in **this** order to your `MIDDLEWARE` list:
+
+```python
+MIDDLEWARE = [
+    "aiwaf.middleware.IPBlockMiddleware",
+    "aiwaf.middleware.RateLimitMiddleware",
+    "aiwaf.middleware.AIAnomalyMiddleware",
+    "aiwaf.middleware.HoneypotMiddleware",
+    "aiwaf.middleware.UUIDTamperMiddleware",
+    # ... other middleware ...
+]
+```
+
+---
+
+## Honeypot Field (in your template)
+
+```django
+{% load aiwaf_tags %}
+
+<form method="post">
+  {% csrf_token %}
+  {% honeypot_field %}
+  <!-- your real fields -->
+</form>
+```
+
+> Renders a hidden `<input name="hp_field" style="display:none">`.  
+> Any non‑empty submission → IP blacklisted.
+
+---
+
+## Running Detection & Training
+
+```bash
+python manage.py detect_and_train
+```
+
+**What happens:**
+1. Read access logs
+2. Auto‑block IPs with ≥ 6 total 404s
+3. Extract features & train IsolationForest
+4. Save `model.pkl`
+5. Extract top 10 dynamic keywords from 4xx/5xx
+
+---
+
+## How It Works
+
+| Middleware               | Purpose                                                         |
+|--------------------------|------------------------------------------------------------------|
+| IPBlockMiddleware        | Blocks requests from known blacklisted IPs                      |
+| RateLimitMiddleware      | Enforces burst & flood thresholds                               |
+| AIAnomalyMiddleware      | ML‑driven behavior analysis + block on anomaly                  |
+| HoneypotMiddleware       | Detects bots filling hidden inputs in forms                     |
+| UUIDTamperMiddleware     | Blocks guessed/nonexistent UUIDs across all models in an app    |
+
+---
+
+## License
+
+This project is licensed under the **MIT License**. See the [LICENSE](LICENSE) file for details.
+
+---
+
+## Credits
+
+**AI‑WAF** by [Aayush Gauba](https://github.com/aayushgauba)  
+> “Let your firewall learn and evolve — keep your site a fortress.”

{aiwaf-0.1.0 → aiwaf-0.1.3}/aiwaf/middleware.py

@@ -1,23 +1,40 @@
+# aiwaf/middleware.py
+
 import time
+import re
+import os
 import numpy as np
 import joblib
+
 from collections import defaultdict
 from django.utils.deprecation import MiddlewareMixin
 from django.http import JsonResponse
 from django.conf import settings
 from django.core.cache import cache
-from django.urls import resolve
+from django.db.models import F
 from django.apps import apps
-from .blacklist_manager import BlacklistManager
 
-try:
-    MODEL_PATH = settings.AIWAF_MODEL_PATH
-except AttributeError:
-    import importlib.resources
-    MODEL_PATH = importlib.resources.files("aiwaf").joinpath("resources/model.pkl")
+from .blacklist_manager import BlacklistManager
+from .models import DynamicKeyword
 
+# ─── Model loading with fallback ────────────────────────────────────────────
+MODEL_PATH = getattr(
+    settings,
+    "AIWAF_MODEL_PATH",
+    os.path.join(os.path.dirname(__file__), "resources", "model.pkl")
+)
 MODEL = joblib.load(MODEL_PATH)
 
+# ─── Static keywords default ────────────────────────────────────────────────
+STATIC_KW = getattr(
+    settings,
+    "AIWAF_MALICIOUS_KEYWORDS",
+    [
+        ".php", "xmlrpc", "wp-", ".env", ".git", ".bak",
+        "conflg", "shell", "filemanager"
+    ]
+)
+
 def get_ip(request):
     xff = request.META.get("HTTP_X_FORWARDED_FOR")
     if xff:
@@ -37,18 +54,21 @@ class IPBlockMiddleware:
 
 
 class RateLimitMiddleware:
-    WINDOW = getattr(settings, "AIWAF_RATE_WINDOW", 10)
-    MAX = getattr(settings, "AIWAF_RATE_MAX", 20)
-    FLOOD = getattr(settings, "AIWAF_RATE_FLOOD", 10)
+    WINDOW = 10
+    MAX    = 20
+    FLOOD  = 10
+
     def __init__(self, get_response):
         self.get_response = get_response
         self.logs = defaultdict(list)
+
     def __call__(self, request):
-        ip = get_ip(request)
+        ip  = get_ip(request)
         now = time.time()
         recs = [t for t in self.logs[ip] if now - t < self.WINDOW]
         recs.append(now)
         self.logs[ip] = recs
+
         if len(recs) > self.MAX:
             return JsonResponse({"error": "too_many_requests"}, status=429)
         if len(recs) > self.FLOOD:
@@ -59,40 +79,57 @@ class RateLimitMiddleware:
 
 
 class AIAnomalyMiddleware(MiddlewareMixin):
-    WINDOW_SECONDS = getattr(settings, "AIWAF_WINDOW_SECONDS", 60)
+    WINDOW = getattr(settings, "AIWAF_WINDOW_SECONDS", 60)
+    TOP_N  = getattr(settings, "AIWAF_DYNAMIC_TOP_N", 10)
+
     def process_request(self, request):
         ip = get_ip(request)
         if BlacklistManager.is_blocked(ip):
             return JsonResponse({"error": "blocked"}, status=403)
+
         now = time.time()
         key = f"aiwaf:{ip}"
         data = cache.get(key, [])
+        # TODO: you may want to capture real status & response_time in process_response
         data.append((now, request.path, 0, 0.0))
-        data = [d for d in data if now - d[0] < self.WINDOW_SECONDS]
-        cache.set(key, data, timeout=self.WINDOW_SECONDS)
+        data = [d for d in data if now - d[0] < self.WINDOW]
+        cache.set(key, data, timeout=self.WINDOW)
+
+        # update dynamic‐keyword counts
+        for seg in re.split(r"\W+", request.path.lower()):
+            if len(seg) > 3:
+                obj, _ = DynamicKeyword.objects.get_or_create(keyword=seg)
+                DynamicKeyword.objects.filter(pk=obj.pk).update(count=F("count") + 1)
+
         if len(data) < 5:
             return None
-        total = len(data)
-        ratio_404 = sum(1 for (_, _, st, _) in data if st == 404) / total
-        hits = sum(
-            any(k in path.lower() for k in settings.AIWAF_MALICIOUS_KEYWORDS)
-            for (_, path, _, _) in data
+
+        # pull top‐N dynamic tokens
+        top_dynamic = list(
+            DynamicKeyword.objects
+            .order_by("-count")
+            .values_list("keyword", flat=True)[: self.TOP_N]
         )
-        avg_rt = np.mean([rt for (_, _, _, rt) in data]) if data else 0.0
-        intervals = [
-            data[i][0] - data[i-1][0] for i in range(1, total)
-        ]
-        avg_iv = np.mean(intervals) if intervals else 0.0
-        X = np.array([[total, ratio_404, hits, avg_rt, avg_iv]], dtype=float)
+        ALL_KW = set(STATIC_KW) | set(top_dynamic)
+
+        total    = len(data)
+        ratio404 = sum(1 for (_, _, st, _) in data if st == 404) / total
+        hits     = sum(any(kw in path.lower() for kw in ALL_KW) for (_, path, _, _) in data)
+        avg_rt   = np.mean([rt for (_, _, _, rt) in data]) if data else 0.0
+        ivs      = [data[i][0] - data[i - 1][0] for i in range(1, total)]
+        avg_iv   = np.mean(ivs) if ivs else 0.0
+
+        X = np.array([[total, ratio404, hits, avg_rt, avg_iv]], dtype=float)
         if MODEL.predict(X)[0] == -1:
             BlacklistManager.block(ip, "AI anomaly")
             return JsonResponse({"error": "blocked"}, status=403)
+
         return None
 
 
 class HoneypotMiddleware(MiddlewareMixin):
     def process_view(self, request, view_func, view_args, view_kwargs):
-        trap = request.POST.get(settings.AIWAF_HONEYPOT_FIELD, "")
+        trap = request.POST.get(getattr(settings, "AIWAF_HONEYPOT_FIELD", "hp_field"), "")
         if trap:
             ip = get_ip(request)
             BlacklistManager.block(ip, "HONEYPOT triggered")
@@ -105,11 +142,13 @@ class UUIDTamperMiddleware(MiddlewareMixin):
         uid = view_kwargs.get("uuid")
         if not uid:
             return None
+
         ip = get_ip(request)
-        app_label = view_kwargs.get("app_label") or view_func.__module__.split('.')[0]
-        app_config = apps.get_app_config(app_label)
-        for Model in app_config.get_models():
+        app_label = view_func.__module__.split(".")[0]
+        app_cfg   = apps.get_app_config(app_label)
+        for Model in app_cfg.get_models():
             if Model.objects.filter(pk=uid).exists():
                 return None
+
         BlacklistManager.block(ip, "UUID tampering")
-        return JsonResponse({"error": "blocked"}, status=403)
+        return JsonResponse({"error": "blocked"}, status=403)

{aiwaf-0.1.0 → aiwaf-0.1.3}/aiwaf/models.py

@@ -26,3 +26,11 @@ class BlacklistEntry(models.Model):
 
     def __str__(self):
         return f"{self.ip_address} ({self.reason})"
+
+class DynamicKeyword(models.Model):
+    keyword      = models.CharField(max_length=100, unique=True)
+    count        = models.PositiveIntegerField(default=0)
+    last_updated = models.DateTimeField(auto_now=True)
+
+    class Meta:
+        ordering = ['-count']

aiwaf-0.1.3/aiwaf/trainer.py

@@ -0,0 +1,175 @@
+import os
+import glob
+import gzip
+import re
+import json
+import joblib
+
+from datetime import datetime
+from collections import defaultdict, Counter
+
+import pandas as pd
+from sklearn.ensemble import IsolationForest
+
+from django.conf import settings
+from django.apps import apps
+
+# ─── CONFIG ────────────────────────────────────────────────────────────────
+
+# Where to read your access logs (and rotated/.gz siblings)
+LOG_PATH = settings.AIWAF_ACCESS_LOG
+
+# Where we save our trained model
+MODEL_PATH = os.path.join(
+    os.path.dirname(__file__),
+    "resources",
+    "model.pkl"
+)
+
+# Static “malicious” path keywords & file extensions
+MALICIOUS_KEYWORDS = [
+    ".php", "xmlrpc", "wp-", ".env", ".git", ".bak",
+    "conflg", "shell", "filemanager"
+]
+STATUS_CODES = ["200", "403", "404", "500"]
+
+# Regex for combined log with response-time=…
+_LOG_RX = re.compile(
+    r'(\d+\.\d+\.\d+\.\d+).*\[(.*?)\].*"(?:GET|POST) (.*?) HTTP/.*?" '
+    r'(\d{3}).*?"(.*?)" "(.*?)".*?response-time=(\d+\.\d+)'
+)
+
+# Your Django model for storing blocked IPs
+BlacklistEntry = apps.get_model("aiwaf", "BlacklistEntry")
+
+
+# ─── READ & PARSE LOG LINES ─────────────────────────────────────────────────
+
+def _read_all_logs():
+    lines = []
+    if LOG_PATH and os.path.exists(LOG_PATH):
+        with open(LOG_PATH, "r", errors="ignore") as f:
+            lines += f.readlines()
+    for path in sorted(glob.glob(LOG_PATH + ".*")):
+        opener = gzip.open if path.endswith(".gz") else open
+        try:
+            with opener(path, "rt", errors="ignore") as f:
+                lines += f.readlines()
+        except OSError:
+            continue
+    return lines
+
+def _parse(line):
+    m = _LOG_RX.search(line)
+    if not m:
+        return None
+    ip, ts_str, path, status, ref, ua, rt = m.groups()
+    try:
+        ts = datetime.strptime(ts_str.split()[0], "%d/%b/%Y:%H:%M:%S")
+    except ValueError:
+        return None
+    return {
+        "ip": ip,
+        "timestamp": ts,
+        "path": path,
+        "status": status,
+        "ua": ua,
+        "response_time": float(rt),
+    }
+
+
+# ─── TRAIN ENTRYPOINT ───────────────────────────────────────────────────────
+
+def train():
+    raw = _read_all_logs()
+    if not raw:
+        print("❌ No log lines found – check settings.AIWAF_ACCESS_LOG")
+        return
+
+    parsed  = []
+    ip_404   = defaultdict(int)
+    ip_times = defaultdict(list)
+
+    # parse + accumulate timestamps & 404 counts
+    for ln in raw:
+        rec = _parse(ln)
+        if not rec:
+            continue
+        parsed.append(rec)
+        ip_times[rec["ip"]].append(rec["timestamp"])
+        if rec["status"] == "404":
+            ip_404[rec["ip"]] += 1
+
+    # auto-block IPs with >=6 total 404s
+    newly_blocked = []
+    for ip, cnt in ip_404.items():
+        if cnt >= 6:
+            obj, created = BlacklistEntry.objects.get_or_create(
+                ip_address=ip,
+                defaults={"reason": "Excessive 404s (≥6)"}
+            )
+            if created:
+                newly_blocked.append(ip)
+    if newly_blocked:
+        print(f"🔒 Blocked {len(newly_blocked)} IPs for 404 flood: {newly_blocked}")
+
+    # build feature vectors
+    rows = []
+    for r in parsed:
+        ip         = r["ip"]
+        burst      = sum(
+            1 for t in ip_times[ip]
+            if (r["timestamp"] - t).total_seconds() <= 10
+        )
+        total404   = ip_404[ip]
+        kw_hits    = sum(k in r["path"].lower() for k in MALICIOUS_KEYWORDS)
+        status_idx = STATUS_CODES.index(r["status"]) if r["status"] in STATUS_CODES else -1
+
+        rows.append([
+            len(r["path"]),
+            kw_hits,
+            r["response_time"],
+            status_idx,
+            burst,
+            total404
+        ])
+
+    if not rows:
+        print("⚠️ No entries to train on.")
+        return
+
+    df = pd.DataFrame(
+        rows,
+        columns=[
+            "path_len", "kw_hits", "resp_time",
+            "status_idx", "burst_count", "total_404"
+        ]
+    ).fillna(0).astype(float)
+
+    # train & save
+    clf = IsolationForest(contamination=0.01, random_state=42)
+    clf.fit(df.values)
+    os.makedirs(os.path.dirname(MODEL_PATH), exist_ok=True)
+    joblib.dump(clf, MODEL_PATH)
+    print(f"✅ Model trained on {len(df)} samples → {MODEL_PATH}")
+
+    # extract top‑10 dynamic keywords from 4xx/5xx paths
+    tokens = Counter()
+    for r in parsed:
+        if r["status"].startswith(("4", "5")):
+            segs = re.split(r"\W+", r["path"].lower())
+            for seg in segs:
+                if len(seg) > 3 and seg not in MALICIOUS_KEYWORDS:
+                    tokens[seg] += 1
+
+    new_kw = [kw for kw, _ in tokens.most_common(10)]
+    DK_FILE = os.path.join(os.path.dirname(__file__), "resources", "dynamic_keywords.json")
+    try:
+        existing = set(json.load(open(DK_FILE)))
+    except FileNotFoundError:
+        existing = set()
+    updated = sorted(existing | set(new_kw))
+    with open(DK_FILE, "w") as f:
+        json.dump(updated, f, indent=2)
+
+    print(f"📝 Updated dynamic keywords: {new_kw}")

aiwaf-0.1.3/aiwaf.egg-info/PKG-INFO

@@ -0,0 +1,181 @@
+Metadata-Version: 2.4
+Name: aiwaf
+Version: 0.1.3
+Summary: AI-powered Web Application Firewall
+Author: Aayush Gauba
+Author-email: Aayush Gauba <gauba.aayush@gmail.com>
+License: MIT
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Dynamic: author
+
+# AI‑WAF
+
+> A self‑learning, Django‑friendly Web Application Firewall  
+> with rate‑limiting, anomaly detection, honeypots, UUID‑tamper protection, dynamic keyword extraction, file‑extension probing detection, and daily retraining.
+
+---
+
+## Package Structure
+
+```
+aiwaf/
+├── __init__.py
+├── blacklist_manager.py
+├── middleware.py
+├── trainer.py                   # exposes train()
+├── utils.py
+├── template_tags/
+│   └── aiwaf_tags.py
+├── resources/
+│   ├── model.pkl                # pre‑trained base model
+│   └── dynamic_keywords.json    # evolves daily
+├── management/
+│   └── commands/
+│       └── detect_and_train.py  # `python manage.py detect_and_train`
+└── LICENSE
+```
+
+---
+
+## Features
+
+- **IP Blocklist**  
+  Instantly blocks suspicious IPs (supports CSV fallback or Django model).
+
+- **Rate Limiting**  
+  Sliding‑window blocks flooders (> `AIWAF_RATE_MAX` per `AIWAF_RATE_WINDOW`), then blacklists them.
+
+- **AI Anomaly Detection**  
+  IsolationForest on features:
+  - Path length  
+  - Keyword hits (static + dynamic)  
+  - Response time  
+  - Status‑code index  
+  - Burst count  
+  - Total 404s  
+
+- **Dynamic Keyword Extraction**  
+  Every retrain: top 10 most frequent “words” from 4xx/5xx paths are appended to your malicious keyword set.
+
+- **File‑Extension Probing Detection**  
+  Tracks repeated 404s on common web‑extensions (e.g. `.php`, `.asp`) and auto‑blocks after a burst.
+
+- **Honeypot Field**  
+  Hidden form field (via template tag) that bots fill → instant block.
+
+- **UUID Tampering Protection**  
+  Any `<uuid:…>` URL that doesn’t map to **any** model in its Django app gets blocked.
+
+- **Daily Retraining**  
+  Reads rotated/gzipped logs, auto‑blocks 404 floods (≥6), retrains the model, updates `model.pkl` + `dynamic_keywords.json`.
+
+---
+
+## Installation
+
+```bash
+# From PyPI
+pip install aiwaf
+
+# Or for local development
+git clone https://github.com/aayushgauba/aiwaf.git
+cd aiwaf
+pip install -e .
+```
+
+---
+
+## ⚙️ Configuration (`settings.py`)
+
+```python
+INSTALLED_APPS += ["aiwaf"]
+
+# Required
+AIWAF_ACCESS_LOG = "/var/log/nginx/access.log"
+
+# Optional (defaults shown)
+AIWAF_MODEL_PATH         = BASE_DIR / "aiwaf" / "resources" / "model.pkl"
+AIWAF_HONEYPOT_FIELD     = "hp_field"
+AIWAF_RATE_WINDOW        = 10         # seconds
+AIWAF_RATE_MAX           = 20         # max reqs/window
+AIWAF_RATE_FLOOD         = 10         # flood threshold
+AIWAF_WINDOW_SECONDS     = 60         # anomaly window
+AIWAF_FILE_EXTENSIONS    = [".php", ".asp", ".jsp"]  # 404‑burst tracked extensions
+```
+
+> **Note:** You no longer need to define `AIWAF_MALICIOUS_KEYWORDS` or `AIWAF_STATUS_CODES` in your settings — they’re built in and evolve dynamically.
+
+---
+
+## Middleware Setup
+
+Add in **this** order to your `MIDDLEWARE` list:
+
+```python
+MIDDLEWARE = [
+    "aiwaf.middleware.IPBlockMiddleware",
+    "aiwaf.middleware.RateLimitMiddleware",
+    "aiwaf.middleware.AIAnomalyMiddleware",
+    "aiwaf.middleware.HoneypotMiddleware",
+    "aiwaf.middleware.UUIDTamperMiddleware",
+    # ... other middleware ...
+]
+```
+
+---
+
+## Honeypot Field (in your template)
+
+```django
+{% load aiwaf_tags %}
+
+<form method="post">
+  {% csrf_token %}
+  {% honeypot_field %}
+  <!-- your real fields -->
+</form>
+```
+
+> Renders a hidden `<input name="hp_field" style="display:none">`.  
+> Any non‑empty submission → IP blacklisted.
+
+---
+
+## Running Detection & Training
+
+```bash
+python manage.py detect_and_train
+```
+
+**What happens:**
+1. Read access logs
+2. Auto‑block IPs with ≥ 6 total 404s
+3. Extract features & train IsolationForest
+4. Save `model.pkl`
+5. Extract top 10 dynamic keywords from 4xx/5xx
+
+---
+
+## How It Works
+
+| Middleware               | Purpose                                                         |
+|--------------------------|------------------------------------------------------------------|
+| IPBlockMiddleware        | Blocks requests from known blacklisted IPs                      |
+| RateLimitMiddleware      | Enforces burst & flood thresholds                               |
+| AIAnomalyMiddleware      | ML‑driven behavior analysis + block on anomaly                  |
+| HoneypotMiddleware       | Detects bots filling hidden inputs in forms                     |
+| UUIDTamperMiddleware     | Blocks guessed/nonexistent UUIDs across all models in an app    |
+
+---
+
+## License
+
+This project is licensed under the **MIT License**. See the [LICENSE](LICENSE) file for details.
+
+---
+
+## Credits
+
+**AI‑WAF** by [Aayush Gauba](https://github.com/aayushgauba)  
+> “Let your firewall learn and evolve — keep your site a fortress.”

{aiwaf-0.1.0 → aiwaf-0.1.3}/aiwaf.egg-info/SOURCES.txt

@@ -1,4 +1,5 @@
 README.md
+pyproject.toml
 setup.py
 aiwaf/__init__.py
 aiwaf/apps.py
@@ -11,8 +12,6 @@ aiwaf/utils.py
 aiwaf.egg-info/PKG-INFO
 aiwaf.egg-info/SOURCES.txt
 aiwaf.egg-info/dependency_links.txt
-aiwaf.egg-info/entry_points.txt
-aiwaf.egg-info/requires.txt
 aiwaf.egg-info/top_level.txt
 aiwaf/management/__init__.py
 aiwaf/management/commands/__init__.py

aiwaf-0.1.3/pyproject.toml

@@ -0,0 +1,9 @@
+[project]
+name = "aiwaf"
+version = "0.1.3"
+description = "AI-powered Web Application Firewall"
+readme = "README.md"
+requires-python = ">=3.8"
+license = {text = "MIT"}
+authors = [{ name = "Aayush Gauba", email = "gauba.aayush@gmail.com" }]
+dependencies = [ ]

{aiwaf-0.1.0 → aiwaf-0.1.3}/setup.py

@@ -1,9 +1,15 @@
 from setuptools import setup, find_packages
+from pathlib import Path
+
+this_directory = Path(__file__).parent
+long_description = (this_directory / "README.md").read_text(encoding="utf-8")
 
 setup(
     name="aiwaf",
-    version="0.1.0",
+    version="0.1.3",
     description="AI‑driven pluggable Web Application Firewall for Django (CSV or DB storage)",
+    long_description=long_description,
+    long_description_content_type="text/markdown",  # <- required for markdown support
     author="Aayush Gauba",
     packages=find_packages(),
     package_data={

aiwaf-0.1.0/PKG-INFO

@@ -1,13 +0,0 @@
-Metadata-Version: 2.4
-Name: aiwaf
-Version: 0.1.0
-Summary: AI‑driven pluggable Web Application Firewall for Django (CSV or DB storage)
-Author: Aayush Gauba
-Requires-Dist: django>=3.0
-Requires-Dist: scikit-learn
-Requires-Dist: numpy
-Requires-Dist: pandas
-Requires-Dist: joblib
-Dynamic: author
-Dynamic: requires-dist
-Dynamic: summary

aiwaf-0.1.0/README.md

@@ -1,176 +0,0 @@
-# AI‑WAF
-
-> A self-learning, Django-friendly Web Application Firewall  
-> with rate-limiting, anomaly detection, honeypots, UUID-tamper protection, and daily retraining.
-
----
-
-## Package Structure
-
-```
-aiwaf/
-├── __init__.py
-├── blacklist_manager.py
-├── middleware.py
-├── trainer.py                   # exposes detect_and_train()
-├── utils.py
-├── template_tags/
-│   └── aiwaf_tags.py
-├── resources/
-│   └── model.pkl                # pre-trained base model
-├── management/
-│   └── commands/
-│       └── detect_and_train.py  # python manage.py detect_and_train
-└── LICENSE
-```
-
----
-
-## Features
-
-- **IP Blocklist**  
-  Automatically blocks suspicious IPs; optionally backed by CSV or Django model.
-
-- **Rate Limiting**  
-  Sliding window logic blocks IPs exceeding a threshold of requests per second.
-
-- **AI Anomaly Detection**  
-  IsolationForest trained on real logs with features like:
-  - Path length
-  - Keyword hits
-  - Response time
-  - Status code index
-  - Burst count
-  - Total 404s
-
-- **Honeypot Field**  
-  Hidden form field that bots are likely to fill — if triggered, the IP is blocked.
-
-- **UUID Tampering Protection**  
-  Detects if someone is probing by injecting random/nonexistent UUIDs into URLs.
-
-- **Daily Retraining**  
-  A single command retrains your model every day based on your logs.
-
----
-
-## Installation
-
-Install locally or from PyPI:
-
-```bash
-pip install aiwaf
-```
-
-Or for local dev:
-
-```bash
-git clone https://github.com/aayushgauba/aiwaf.git
-cd aiwaf
-pip install -e .
-```
-
----
-
-## ⚙️ Configuration (`settings.py`)
-
-```python
-INSTALLED_APPS += [
-    "aiwaf",
-]
-
-# Required
-AIWAF_ACCESS_LOG = "/var/log/nginx/access.log"
-
-# Optional (defaults included)
-AIWAF_MODEL_PATH = BASE_DIR / "aiwaf" / "resources" / "model.pkl"
-AIWAF_MALICIOUS_KEYWORDS = [".php", "xmlrpc", "wp-", ".env", ".git", ".bak", "conflg", "shell", "filemanager"]
-AIWAF_STATUS_CODES = ["200", "403", "404", "500"]
-AIWAF_HONEYPOT_FIELD = "hp_field"
-```
-
----
-
-## Middleware Setup
-
-Add to `MIDDLEWARE` in order:
-
-```python
-MIDDLEWARE = [
-    "aiwaf.middleware.IPBlockMiddleware",
-    "aiwaf.middleware.RateLimitMiddleware",
-    "aiwaf.middleware.AIAnomalyMiddleware",
-    "aiwaf.middleware.HoneypotMiddleware",
-    "aiwaf.middleware.UUIDTamperMiddleware",
-    ...
-]
-```
-
----
-
-## Honeypot Field (in template)
-
-```html
-{% load aiwaf_tags %}
-
-<form method="post">
-  {% csrf_token %}
-  {% honeypot_field %}
-  <!-- other fields -->
-</form>
-```
-
-The hidden field will be `<input type="hidden" name="hp_field">`.  
-If it’s ever filled → IP gets blocked.
-
----
-
-##  Run Detection + Training
-
-```bash
-python manage.py detect_and_train
-```
-
-What it does:
-
-- Reads logs (supports `.gz` and rotated logs).
-- Detects excessive 404s (≥6) → instant block.
-- Builds feature vectors from logs.
-- Trains IsolationForest and saves `model.pkl`.
-
-Schedule it to run daily via `cron`, `Celery beat`, or systemd timer.
-
----
-
-## How It Works (Simplified)
-
-| Middleware             | Functionality                                                |
-|------------------------|--------------------------------------------------------------|
-| IPBlockMiddleware      | Blocks requests from known blacklisted IPs                   |
-| RateLimitMiddleware    | Blocks flooders (>20/10s) and blacklists them (>10/10s)      |
-| AIAnomalyMiddleware    | Uses ML to detect suspicious behavior in request patterns    |
-| HoneypotMiddleware     | Detects bots filling hidden inputs in forms                  |
-| UUIDTamperMiddleware   | Detects guessing/probing by checking invalid UUID access     |
-
----
-
-## Development Roadmap
-
-- [ ] Add CSV blocklist fallback
-- [ ] Admin dashboard integration
-- [ ] Auto-pruning of old block entries
-- [ ] Real-time log streaming compatibility
-- [ ] Docker/Helm deployment guide
-
----
-
-##  License
-
-This project is licensed under the **MIT License** — see `LICENSE` for details.
-
----
-
-## Credits
-
-**AIWAF** by [Aayush Gauba](https://github.com/aayushgauba)  
-> "Let your firewall learn and evolve with your logs. Make your site a fortress."

aiwaf-0.1.0/aiwaf/trainer.py

@@ -1,123 +0,0 @@
-# aiwaf/trainer.py
-
-import os
-import glob
-import gzip
-import re
-import joblib
-from datetime import datetime
-from collections import defaultdict
-from .models import BlacklistEntry
-import pandas as pd
-from sklearn.ensemble import IsolationForest
-from django.conf import settings
-from django.apps import apps
-
-LOG_PATH   = settings.AIWAF_ACCESS_LOG
-MODEL_PATH = os.path.join(
-    os.path.dirname(__file__),
-    "resources",
-    "model.pkl"
-)
-MALICIOUS_KEYWORDS = [".php", "xmlrpc", "wp-", ".env", ".git", ".bak", "conflg", "shell", "filemanager"]
-STATUS_CODES       = ["200", "403", "404", "500"]
-_LOG_RX = re.compile(
-    r'(\d+\.\d+\.\d+\.\d+).*\[(.*?)\].*"(?:GET|POST) (.*?) HTTP/.*?" (\d{3}).*?"(.*?)" "(.*?)".*?response-time=(\d+\.\d+)'
-)
-BlacklistedIP = BlacklistEntry.objects.all()
-def _read_all_logs():
-    lines = []
-    if LOG_PATH and os.path.exists(LOG_PATH):
-        with open(LOG_PATH, "r", errors="ignore") as f:
-            lines += f.readlines()
-    for path in sorted(glob.glob(LOG_PATH + ".*")):
-        opener = gzip.open if path.endswith(".gz") else open
-        try:
-            with opener(path, "rt", errors="ignore") as f:
-                lines += f.readlines()
-        except OSError:
-            continue
-    return lines
-
-def _parse(line):
-    m = _LOG_RX.search(line)
-    if not m:
-        return None
-    ip, ts_str, path, status, ref, ua, rt = m.groups()
-    try:
-        ts = datetime.strptime(ts_str.split()[0], "%d/%b/%Y:%H:%M:%S")
-    except ValueError:
-        return None
-    return {
-        "ip": ip,
-        "timestamp": ts,
-        "path": path,
-        "status": status,
-        "ua": ua,
-        "response_time": float(rt),
-    }
-
-
-def train():
-    raw = _read_all_logs()
-    if not raw:
-        print("No log lines found – check AIWAF_ACCESS_LOG")
-        return
-    parsed = []
-    ip_404   = defaultdict(int)
-    ip_times = defaultdict(list)
-    for ln in raw:
-        rec = _parse(ln)
-        if not rec:
-            continue
-        parsed.append(rec)
-        ip_times[rec["ip"]].append(rec["timestamp"])
-        if rec["status"] == "404":
-            ip_404[rec["ip"]] += 1
-    blocked = []
-    for ip, count in ip_404.items():
-        if count >= 6:
-            obj, created = BlacklistEntry.objects.get_or_create(
-                ip_address=ip,
-                defaults={"reason": "Excessive 404s (≥6)"}
-            )
-            if created:
-                blocked.append(ip)
-    if blocked:
-        print(f"Auto‑blocked {len(blocked)} IPs for ≥6 404s: {', '.join(blocked)}")
-    rows = []
-    for r in parsed:
-        ip        = r["ip"]
-        burst     = sum(
-            1 for t in ip_times[ip]
-            if (r["timestamp"] - t).total_seconds() <= 10
-        )
-        total404  = ip_404[ip]
-        kw_hits   = sum(k in r["path"].lower() for k in MALICIOUS_KEYWORDS)
-        status_idx = STATUS_CODES.index(r["status"]) if r["status"] in STATUS_CODES else -1
-        rows.append([
-            len(r["path"]),
-            kw_hits,
-            r["response_time"],
-            status_idx,
-            burst,
-            total404
-        ])
-
-    if not rows:
-        print("No entries to train on!")
-        return
-
-    df = pd.DataFrame(
-        rows,
-        columns=[
-            "path_len", "kw_hits", "resp_time",
-            "status_idx", "burst_count", "total_404"
-        ]
-    ).fillna(0).astype(float)
-    clf = IsolationForest(contamination=0.01, random_state=42)
-    clf.fit(df.values)
-    os.makedirs(os.path.dirname(MODEL_PATH), exist_ok=True)
-    joblib.dump(clf, MODEL_PATH)
-    print(f"Model trained on {len(df)} samples and saved to {MODEL_PATH}")
-

aiwaf-0.1.0/aiwaf.egg-info/PKG-INFO

@@ -1,13 +0,0 @@
-Metadata-Version: 2.4
-Name: aiwaf
-Version: 0.1.0
-Summary: AI‑driven pluggable Web Application Firewall for Django (CSV or DB storage)
-Author: Aayush Gauba
-Requires-Dist: django>=3.0
-Requires-Dist: scikit-learn
-Requires-Dist: numpy
-Requires-Dist: pandas
-Requires-Dist: joblib
-Dynamic: author
-Dynamic: requires-dist
-Dynamic: summary

aiwaf-0.1.0/aiwaf.egg-info/entry_points.txt

@@ -1,2 +0,0 @@
-[console_scripts]
-aiwaf-detect = aiwaf.trainer:detect_and_train

aiwaf-0.1.0/aiwaf.egg-info/requires.txt

@@ -1,5 +0,0 @@
-django>=3.0
-scikit-learn
-numpy
-pandas
-joblib