aiwaf 0.1.7__tar.gz → 0.1.7.2__tar.gz

{aiwaf-0.1.7 → aiwaf-0.1.7.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: aiwaf
-Version: 0.1.7
+Version: 0.1.7.2
 Summary: AI-powered Web Application Firewall
 Home-page: https://github.com/aayushgauba/aiwaf
 Author: Aayush Gauba
@@ -15,14 +15,14 @@ Dynamic: license-file
 Dynamic: requires-python
 
 
-# AI‑WAF 
+# AI‑WAF
 
 > A self‑learning, Django‑friendly Web Application Firewall  
-> with rate‑limiting, anomaly detection, honeypots, UUID‑tamper protection, dynamic keyword extraction, file‑extension probing detection, and daily retraining.
+> with rate‑limiting, anomaly detection, honeypots, UUID‑tamper protection, dynamic keyword extraction, file‑extension probing detection, exempt path awareness, and daily retraining.
 
 ---
 
-## Package Structure
+## 📁 Package Structure
 
 ```
 aiwaf/
@@ -44,7 +44,7 @@ aiwaf/
 
 ---
 
-## Features
+## 🚀 Features
 
 - **IP Blocklist**  
   Instantly blocks suspicious IPs (supports CSV fallback or Django model).
@@ -53,7 +53,7 @@ aiwaf/
   Sliding‑window blocks flooders (> `AIWAF_RATE_MAX` per `AIWAF_RATE_WINDOW`), then blacklists them.
 
 - **AI Anomaly Detection**  
-  IsolationForest on features:
+  IsolationForest trained on:
   - Path length  
   - Keyword hits (static + dynamic)  
   - Response time  
@@ -61,34 +61,28 @@ aiwaf/
   - Burst count  
   - Total 404s  
 
-- **Dynamic Keyword Extraction**  
-  Every retrain: top 10 most frequent “words” from 4xx/5xx paths are appended to your malicious keyword set.
+- **Dynamic Keyword Extraction & Cleanup**  
+  - Every retrain adds top 10 keyword segments from 4xx/5xx paths  
+  - **If a path is added to `AIWAF_EXEMPT_PATHS`, its keywords are automatically removed from the database**
 
 - **File‑Extension Probing Detection**  
-  Tracks repeated 404s on common web‑extensions (e.g. `.php`, `.asp`) and auto‑blocks after a burst.
+  Tracks repeated 404s on common extensions (e.g. `.php`, `.asp`) and blocks IPs.
 
 - **Honeypot Field**  
-  Hidden form field (via template tag) that bots fill → instant block.
+  Hidden field for bot detection → IP blacklisted on fill.
 
 - **UUID Tampering Protection**  
-  Any `<uuid:…>` URL that doesn’t map to **any** model in its Django app gets blocked.
+  Blocks guessed or invalid UUIDs that don’t resolve to real models.
 
-- **Daily Retraining**  
-  Reads rotated/gzipped logs, auto‑blocks 404 floods (≥6), retrains the model, updates `model.pkl` + `dynamic_keywords.json`.
-
----
-
-## Installation
-
-```bash
-# From PyPI
-pip install aiwaf
+- **Exempt Path Awareness**  
+  Fully respects `AIWAF_EXEMPT_PATHS` across all modules — exempt paths are:
+  - Skipped from keyword learning
+  - Immune to AI blocking
+  - Ignored in log training
+  - Cleaned from `DynamicKeyword` model automatically
 
-# Or for local development
-git clone https://github.com/aayushgauba/aiwaf.git
-cd aiwaf
-pip install -e .
-```
+- **Daily Retraining**  
+  Reads rotated logs, auto‑blocks 404 floods, retrains the IsolationForest, updates `model.pkl`, and evolves the keyword DB.
 
 ---
 
@@ -96,33 +90,51 @@ pip install -e .
 
 ```python
 INSTALLED_APPS += ["aiwaf"]
+```
 
 ### Database Setup
 
-After adding `aiwaf` to your `INSTALLED_APPS`, create the necessary tables for the IP‐blacklist and dynamic‐keyword models:
+After adding `aiwaf` to your `INSTALLED_APPS`, run the following to create the necessary tables:
 
 ```bash
 python manage.py makemigrations aiwaf
 python manage.py migrate
+```
 
-# Required
+---
+
+### Required
+
+```python
 AIWAF_ACCESS_LOG = "/var/log/nginx/access.log"
+```
+
+---
+
+### Optional (defaults shown)
 
-# Optional (defaults shown)
+```python
 AIWAF_MODEL_PATH         = BASE_DIR / "aiwaf" / "resources" / "model.pkl"
 AIWAF_HONEYPOT_FIELD     = "hp_field"
 AIWAF_RATE_WINDOW        = 10         # seconds
-AIWAF_RATE_MAX           = 20         # max reqs/window
+AIWAF_RATE_MAX           = 20         # max requests per window
 AIWAF_RATE_FLOOD         = 10         # flood threshold
-AIWAF_WINDOW_SECONDS     = 60         # anomaly window
-AIWAF_FILE_EXTENSIONS    = [".php", ".asp", ".jsp"]  # 404‑burst tracked extensions
+AIWAF_WINDOW_SECONDS     = 60         # anomaly detection window
+AIWAF_FILE_EXTENSIONS    = [".php", ".asp", ".jsp"]
+AIWAF_EXEMPT_PATHS       = [          # optional but highly recommended
+    "/favicon.ico",
+    "/robots.txt",
+    "/static/",
+    "/media/",
+    "/health/",
+]
 ```
 
-> **Note:** You no longer need to define `AIWAF_MALICIOUS_KEYWORDS` or `AIWAF_STATUS_CODES` in your settings — they’re built in and evolve dynamically.
+> **Note:** You no longer need to define `AIWAF_MALICIOUS_KEYWORDS` or `AIWAF_STATUS_CODES` — they evolve dynamically.
 
 ---
 
-## Middleware Setup
+## 🧱 Middleware Setup
 
 Add in **this** order to your `MIDDLEWARE` list:
 
@@ -139,7 +151,7 @@ MIDDLEWARE = [
 
 ---
 
-## Honeypot Field (in your template)
+## 🕵️ Honeypot Field (in your template)
 
 ```django
 {% load aiwaf_tags %}
@@ -156,22 +168,23 @@ MIDDLEWARE = [
 
 ---
 
-## Running Detection & Training
+## 🔁 Running Detection & Training
 
 ```bash
 python manage.py detect_and_train
 ```
 
-**What happens:**
-1. Read access logs
+### What happens:
+1. Read access logs (incl. rotated or gzipped)
 2. Auto‑block IPs with ≥ 6 total 404s
 3. Extract features & train IsolationForest
 4. Save `model.pkl`
 5. Extract top 10 dynamic keywords from 4xx/5xx
+6. Remove any keywords associated with newly exempt paths
 
 ---
 
-## How It Works
+## 🧠 How It Works
 
 | Middleware                         | Purpose                                                         |
 |------------------------------------|-----------------------------------------------------------------|
@@ -180,15 +193,16 @@ python manage.py detect_and_train
 | AIAnomalyMiddleware                | ML‑driven behavior analysis + block on anomaly                  |
 | HoneypotMiddleware                 | Detects bots filling hidden inputs in forms                     |
 | UUIDTamperMiddleware               | Blocks guessed/nonexistent UUIDs across all models in an app    |
+
 ---
 
-## License
+## 📄 License
 
 This project is licensed under the **MIT License**. See the [LICENSE](LICENSE) file for details.
 
 ---
 
-## Credits
+## 👤 Credits
 
 **AI‑WAF** by [Aayush Gauba](https://github.com/aayushgauba)  
 > “Let your firewall learn and evolve — keep your site a fortress.”

{aiwaf-0.1.7 → aiwaf-0.1.7.2}/README.md

@@ -1,12 +1,12 @@
 
-# AI‑WAF 
+# AI‑WAF
 
 > A self‑learning, Django‑friendly Web Application Firewall  
-> with rate‑limiting, anomaly detection, honeypots, UUID‑tamper protection, dynamic keyword extraction, file‑extension probing detection, and daily retraining.
+> with rate‑limiting, anomaly detection, honeypots, UUID‑tamper protection, dynamic keyword extraction, file‑extension probing detection, exempt path awareness, and daily retraining.
 
 ---
 
-## Package Structure
+## 📁 Package Structure
 
 ```
 aiwaf/
@@ -28,7 +28,7 @@ aiwaf/
 
 ---
 
-## Features
+## 🚀 Features
 
 - **IP Blocklist**  
   Instantly blocks suspicious IPs (supports CSV fallback or Django model).
@@ -37,7 +37,7 @@ aiwaf/
   Sliding‑window blocks flooders (> `AIWAF_RATE_MAX` per `AIWAF_RATE_WINDOW`), then blacklists them.
 
 - **AI Anomaly Detection**  
-  IsolationForest on features:
+  IsolationForest trained on:
   - Path length  
   - Keyword hits (static + dynamic)  
   - Response time  
@@ -45,34 +45,28 @@ aiwaf/
   - Burst count  
   - Total 404s  
 
-- **Dynamic Keyword Extraction**  
-  Every retrain: top 10 most frequent “words” from 4xx/5xx paths are appended to your malicious keyword set.
+- **Dynamic Keyword Extraction & Cleanup**  
+  - Every retrain adds top 10 keyword segments from 4xx/5xx paths  
+  - **If a path is added to `AIWAF_EXEMPT_PATHS`, its keywords are automatically removed from the database**
 
 - **File‑Extension Probing Detection**  
-  Tracks repeated 404s on common web‑extensions (e.g. `.php`, `.asp`) and auto‑blocks after a burst.
+  Tracks repeated 404s on common extensions (e.g. `.php`, `.asp`) and blocks IPs.
 
 - **Honeypot Field**  
-  Hidden form field (via template tag) that bots fill → instant block.
+  Hidden field for bot detection → IP blacklisted on fill.
 
 - **UUID Tampering Protection**  
-  Any `<uuid:…>` URL that doesn’t map to **any** model in its Django app gets blocked.
+  Blocks guessed or invalid UUIDs that don’t resolve to real models.
 
-- **Daily Retraining**  
-  Reads rotated/gzipped logs, auto‑blocks 404 floods (≥6), retrains the model, updates `model.pkl` + `dynamic_keywords.json`.
-
----
-
-## Installation
-
-```bash
-# From PyPI
-pip install aiwaf
+- **Exempt Path Awareness**  
+  Fully respects `AIWAF_EXEMPT_PATHS` across all modules — exempt paths are:
+  - Skipped from keyword learning
+  - Immune to AI blocking
+  - Ignored in log training
+  - Cleaned from `DynamicKeyword` model automatically
 
-# Or for local development
-git clone https://github.com/aayushgauba/aiwaf.git
-cd aiwaf
-pip install -e .
-```
+- **Daily Retraining**  
+  Reads rotated logs, auto‑blocks 404 floods, retrains the IsolationForest, updates `model.pkl`, and evolves the keyword DB.
 
 ---
 
@@ -80,33 +74,51 @@ pip install -e .
 
 ```python
 INSTALLED_APPS += ["aiwaf"]
+```
 
 ### Database Setup
 
-After adding `aiwaf` to your `INSTALLED_APPS`, create the necessary tables for the IP‐blacklist and dynamic‐keyword models:
+After adding `aiwaf` to your `INSTALLED_APPS`, run the following to create the necessary tables:
 
 ```bash
 python manage.py makemigrations aiwaf
 python manage.py migrate
+```
 
-# Required
+---
+
+### Required
+
+```python
 AIWAF_ACCESS_LOG = "/var/log/nginx/access.log"
+```
+
+---
+
+### Optional (defaults shown)
 
-# Optional (defaults shown)
+```python
 AIWAF_MODEL_PATH         = BASE_DIR / "aiwaf" / "resources" / "model.pkl"
 AIWAF_HONEYPOT_FIELD     = "hp_field"
 AIWAF_RATE_WINDOW        = 10         # seconds
-AIWAF_RATE_MAX           = 20         # max reqs/window
+AIWAF_RATE_MAX           = 20         # max requests per window
 AIWAF_RATE_FLOOD         = 10         # flood threshold
-AIWAF_WINDOW_SECONDS     = 60         # anomaly window
-AIWAF_FILE_EXTENSIONS    = [".php", ".asp", ".jsp"]  # 404‑burst tracked extensions
+AIWAF_WINDOW_SECONDS     = 60         # anomaly detection window
+AIWAF_FILE_EXTENSIONS    = [".php", ".asp", ".jsp"]
+AIWAF_EXEMPT_PATHS       = [          # optional but highly recommended
+    "/favicon.ico",
+    "/robots.txt",
+    "/static/",
+    "/media/",
+    "/health/",
+]
 ```
 
-> **Note:** You no longer need to define `AIWAF_MALICIOUS_KEYWORDS` or `AIWAF_STATUS_CODES` in your settings — they’re built in and evolve dynamically.
+> **Note:** You no longer need to define `AIWAF_MALICIOUS_KEYWORDS` or `AIWAF_STATUS_CODES` — they evolve dynamically.
 
 ---
 
-## Middleware Setup
+## 🧱 Middleware Setup
 
 Add in **this** order to your `MIDDLEWARE` list:
 
@@ -123,7 +135,7 @@ MIDDLEWARE = [
 
 ---
 
-## Honeypot Field (in your template)
+## 🕵️ Honeypot Field (in your template)
 
 ```django
 {% load aiwaf_tags %}
@@ -140,22 +152,23 @@ MIDDLEWARE = [
 
 ---
 
-## Running Detection & Training
+## 🔁 Running Detection & Training
 
 ```bash
 python manage.py detect_and_train
 ```
 
-**What happens:**
-1. Read access logs
+### What happens:
+1. Read access logs (incl. rotated or gzipped)
 2. Auto‑block IPs with ≥ 6 total 404s
 3. Extract features & train IsolationForest
 4. Save `model.pkl`
 5. Extract top 10 dynamic keywords from 4xx/5xx
+6. Remove any keywords associated with newly exempt paths
 
 ---
 
-## How It Works
+## 🧠 How It Works
 
 | Middleware                         | Purpose                                                         |
 |------------------------------------|-----------------------------------------------------------------|
@@ -164,15 +177,16 @@ python manage.py detect_and_train
 | AIAnomalyMiddleware                | ML‑driven behavior analysis + block on anomaly                  |
 | HoneypotMiddleware                 | Detects bots filling hidden inputs in forms                     |
 | UUIDTamperMiddleware               | Blocks guessed/nonexistent UUIDs across all models in an app    |
+
 ---
 
-## License
+## 📄 License
 
 This project is licensed under the **MIT License**. See the [LICENSE](LICENSE) file for details.
 
 ---
 
-## Credits
+## 👤 Credits
 
 **AI‑WAF** by [Aayush Gauba](https://github.com/aayushgauba)  
 > “Let your firewall learn and evolve — keep your site a fortress.”

{aiwaf-0.1.7 → aiwaf-0.1.7.2}/aiwaf/middleware.py

@@ -18,6 +18,14 @@ from django.urls import get_resolver
 from .blacklist_manager import BlacklistManager
 from .models import DynamicKeyword
 
+def is_exempt_path(path):
+    path = path.lower()
+    exempt_paths = getattr(settings, "AIWAF_EXEMPT_PATHS", [])
+    for exempt in exempt_paths:
+        if path == exempt or path.startswith(exempt.rstrip("/") + "/"):
+            return True
+    return False
+
 MODEL_PATH = getattr(
     settings,
     "AIWAF_MODEL_PATH",
@@ -64,8 +72,11 @@ class IPAndKeywordBlockMiddleware:
         return prefixes
 
     def __call__(self, request):
+        raw_path = request.path.lower()
+        if is_exempt_path(raw_path):
+            return self.get_response(request)
         ip = get_ip(request)
-        path = request.path.lower().lstrip("/")
+        path = raw_path.lstrip("/")
         if BlacklistManager.is_blocked(ip):
             return JsonResponse({"error": "blocked"}, status=403)
         segments = [seg for seg in re.split(r"\W+", path) if len(seg) > 3]
@@ -90,27 +101,29 @@ class IPAndKeywordBlockMiddleware:
 
 
 class RateLimitMiddleware:
-    WINDOW = 10
-    MAX    = 20
-    FLOOD  = 10
+    WINDOW = 10  # seconds
+    MAX = 20     # soft limit
+    FLOOD = 40   # hard limit
 
     def __init__(self, get_response):
         self.get_response = get_response
-        self.logs = defaultdict(list)
 
     def __call__(self, request):
-        ip  = get_ip(request)
-        now = time.time()
-        recs = [t for t in self.logs[ip] if now - t < self.WINDOW]
-        recs.append(now)
-        self.logs[ip] = recs
+        if is_exempt_path(request.path):
+            return self.get_response(request)
 
-        if len(recs) > self.MAX:
-            return JsonResponse({"error": "too_many_requests"}, status=429)
-        if len(recs) > self.FLOOD:
+        ip = get_ip(request)
+        key = f"ratelimit:{ip}"
+        now = time.time()
+        timestamps = cache.get(key, [])
+        timestamps = [t for t in timestamps if now - t < self.WINDOW]
+        timestamps.append(now)
+        cache.set(key, timestamps, timeout=self.WINDOW)
+        if len(timestamps) > self.FLOOD:
             BlacklistManager.block(ip, "Flood pattern")
             return JsonResponse({"error": "blocked"}, status=403)
-
+        if len(timestamps) > self.MAX:
+            return JsonResponse({"error": "too_many_requests"}, status=429)
         return self.get_response(request)
 
 
@@ -119,6 +132,8 @@ class AIAnomalyMiddleware(MiddlewareMixin):
     TOP_N  = getattr(settings, "AIWAF_DYNAMIC_TOP_N", 10)
 
     def process_request(self, request):
+        if is_exempt_path(request.path):
+            return None
         ip = get_ip(request)
         if BlacklistManager.is_blocked(ip):
             return JsonResponse({"error": "blocked"}, status=403)
@@ -160,6 +175,8 @@ class AIAnomalyMiddleware(MiddlewareMixin):
 
 class HoneypotMiddleware(MiddlewareMixin):
     def process_view(self, request, view_func, view_args, view_kwargs):
+        if is_exempt_path(request.path):
+            return None
         trap = request.POST.get(getattr(settings, "AIWAF_HONEYPOT_FIELD", "hp_field"), "")
         if trap:
             ip = get_ip(request)
@@ -170,6 +187,8 @@ class HoneypotMiddleware(MiddlewareMixin):
 
 class UUIDTamperMiddleware(MiddlewareMixin):
     def process_view(self, request, view_func, view_args, view_kwargs):
+        if is_exempt_path(request.path):
+            return None
         uid = view_kwargs.get("uuid")
         if not uid:
             return None

{aiwaf-0.1.7 → aiwaf-0.1.7.2}/aiwaf/trainer.py

@@ -14,7 +14,6 @@ from django.apps import apps
 from django.db.models import F
 from django.urls import get_resolver
 
-# ─── CONFIG ────────────────────────────────────────────────────────────────
 
 LOG_PATH = settings.AIWAF_ACCESS_LOG
 MODEL_PATH = os.path.join(os.path.dirname(__file__), "resources", "model.pkl")
@@ -30,7 +29,13 @@ _LOG_RX = re.compile(
 BlacklistEntry = apps.get_model("aiwaf", "BlacklistEntry")
 DynamicKeyword = apps.get_model("aiwaf", "DynamicKeyword")
 
-
+def is_exempt_path(path):
+    path = path.lower()
+    exempt_paths = getattr(settings, "AIWAF_EXEMPT_PATHS", [])
+    for exempt in exempt_paths:
+        if path == exempt or path.startswith(exempt.rstrip("/") + "/"):
+            return True
+    return False
 
 def path_exists_in_django(path):
     from django.urls import get_resolver
@@ -51,6 +56,18 @@ def path_exists_in_django(path):
                 return True
     return False
 
+def remove_exempt_keywords():
+    exempt_paths = getattr(settings, "AIWAF_EXEMPT_PATHS", [])
+    exempt_tokens = set()
+
+    for path in exempt_paths:
+        path = path.strip("/").lower()
+        segments = re.split(r"\W+", path)
+        exempt_tokens.update(seg for seg in segments if len(seg) > 3)
+
+    if exempt_tokens:
+        deleted_count, _ = DynamicKeyword.objects.filter(keyword__in=exempt_tokens).delete()
+        print(f"Removed {deleted_count} dynamic keywords that are now exempt: {list(exempt_tokens)}")
 
 def _read_all_logs():
     lines = []
@@ -88,6 +105,7 @@ def _parse(line):
 
 
 def train():
+    remove_exempt_keywords()
     raw_lines = _read_all_logs()
     if not raw_lines:
         print("No log lines found – check AIWAF_ACCESS_LOG setting.")
@@ -125,7 +143,7 @@ def train():
         total404 = ip_404[ip]
         is_known_path = path_exists_in_django(r["path"])
         kw_hits = 0
-        if not is_known_path:
+        if not is_known_path and not is_exempt_path(r["path"]):
             kw_hits = sum(k in r["path"].lower() for k in STATIC_KW)
         status_idx = STATUS_IDX.index(r["status"]) if r["status"] in STATUS_IDX else -1
         feature_dicts.append({
@@ -178,4 +196,4 @@ def train():
 
 
 if __name__ == "__main__":
-    train()
+    train()

{aiwaf-0.1.7 → aiwaf-0.1.7.2}/aiwaf.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: aiwaf
-Version: 0.1.7
+Version: 0.1.7.2
 Summary: AI-powered Web Application Firewall
 Home-page: https://github.com/aayushgauba/aiwaf
 Author: Aayush Gauba
@@ -15,14 +15,14 @@ Dynamic: license-file
 Dynamic: requires-python
 
 
-# AI‑WAF 
+# AI‑WAF
 
 > A self‑learning, Django‑friendly Web Application Firewall  
-> with rate‑limiting, anomaly detection, honeypots, UUID‑tamper protection, dynamic keyword extraction, file‑extension probing detection, and daily retraining.
+> with rate‑limiting, anomaly detection, honeypots, UUID‑tamper protection, dynamic keyword extraction, file‑extension probing detection, exempt path awareness, and daily retraining.
 
 ---
 
-## Package Structure
+## 📁 Package Structure
 
 ```
 aiwaf/
@@ -44,7 +44,7 @@ aiwaf/
 
 ---
 
-## Features
+## 🚀 Features
 
 - **IP Blocklist**  
   Instantly blocks suspicious IPs (supports CSV fallback or Django model).
@@ -53,7 +53,7 @@ aiwaf/
   Sliding‑window blocks flooders (> `AIWAF_RATE_MAX` per `AIWAF_RATE_WINDOW`), then blacklists them.
 
 - **AI Anomaly Detection**  
-  IsolationForest on features:
+  IsolationForest trained on:
   - Path length  
   - Keyword hits (static + dynamic)  
   - Response time  
@@ -61,34 +61,28 @@ aiwaf/
   - Burst count  
   - Total 404s  
 
-- **Dynamic Keyword Extraction**  
-  Every retrain: top 10 most frequent “words” from 4xx/5xx paths are appended to your malicious keyword set.
+- **Dynamic Keyword Extraction & Cleanup**  
+  - Every retrain adds top 10 keyword segments from 4xx/5xx paths  
+  - **If a path is added to `AIWAF_EXEMPT_PATHS`, its keywords are automatically removed from the database**
 
 - **File‑Extension Probing Detection**  
-  Tracks repeated 404s on common web‑extensions (e.g. `.php`, `.asp`) and auto‑blocks after a burst.
+  Tracks repeated 404s on common extensions (e.g. `.php`, `.asp`) and blocks IPs.
 
 - **Honeypot Field**  
-  Hidden form field (via template tag) that bots fill → instant block.
+  Hidden field for bot detection → IP blacklisted on fill.
 
 - **UUID Tampering Protection**  
-  Any `<uuid:…>` URL that doesn’t map to **any** model in its Django app gets blocked.
+  Blocks guessed or invalid UUIDs that don’t resolve to real models.
 
-- **Daily Retraining**  
-  Reads rotated/gzipped logs, auto‑blocks 404 floods (≥6), retrains the model, updates `model.pkl` + `dynamic_keywords.json`.
-
----
-
-## Installation
-
-```bash
-# From PyPI
-pip install aiwaf
+- **Exempt Path Awareness**  
+  Fully respects `AIWAF_EXEMPT_PATHS` across all modules — exempt paths are:
+  - Skipped from keyword learning
+  - Immune to AI blocking
+  - Ignored in log training
+  - Cleaned from `DynamicKeyword` model automatically
 
-# Or for local development
-git clone https://github.com/aayushgauba/aiwaf.git
-cd aiwaf
-pip install -e .
-```
+- **Daily Retraining**  
+  Reads rotated logs, auto‑blocks 404 floods, retrains the IsolationForest, updates `model.pkl`, and evolves the keyword DB.
 
 ---
 
@@ -96,33 +90,51 @@ pip install -e .
 
 ```python
 INSTALLED_APPS += ["aiwaf"]
+```
 
 ### Database Setup
 
-After adding `aiwaf` to your `INSTALLED_APPS`, create the necessary tables for the IP‐blacklist and dynamic‐keyword models:
+After adding `aiwaf` to your `INSTALLED_APPS`, run the following to create the necessary tables:
 
 ```bash
 python manage.py makemigrations aiwaf
 python manage.py migrate
+```
 
-# Required
+---
+
+### Required
+
+```python
 AIWAF_ACCESS_LOG = "/var/log/nginx/access.log"
+```
+
+---
+
+### Optional (defaults shown)
 
-# Optional (defaults shown)
+```python
 AIWAF_MODEL_PATH         = BASE_DIR / "aiwaf" / "resources" / "model.pkl"
 AIWAF_HONEYPOT_FIELD     = "hp_field"
 AIWAF_RATE_WINDOW        = 10         # seconds
-AIWAF_RATE_MAX           = 20         # max reqs/window
+AIWAF_RATE_MAX           = 20         # max requests per window
 AIWAF_RATE_FLOOD         = 10         # flood threshold
-AIWAF_WINDOW_SECONDS     = 60         # anomaly window
-AIWAF_FILE_EXTENSIONS    = [".php", ".asp", ".jsp"]  # 404‑burst tracked extensions
+AIWAF_WINDOW_SECONDS     = 60         # anomaly detection window
+AIWAF_FILE_EXTENSIONS    = [".php", ".asp", ".jsp"]
+AIWAF_EXEMPT_PATHS       = [          # optional but highly recommended
+    "/favicon.ico",
+    "/robots.txt",
+    "/static/",
+    "/media/",
+    "/health/",
+]
 ```
 
-> **Note:** You no longer need to define `AIWAF_MALICIOUS_KEYWORDS` or `AIWAF_STATUS_CODES` in your settings — they’re built in and evolve dynamically.
+> **Note:** You no longer need to define `AIWAF_MALICIOUS_KEYWORDS` or `AIWAF_STATUS_CODES` — they evolve dynamically.
 
 ---
 
-## Middleware Setup
+## 🧱 Middleware Setup
 
 Add in **this** order to your `MIDDLEWARE` list:
 
@@ -139,7 +151,7 @@ MIDDLEWARE = [
 
 ---
 
-## Honeypot Field (in your template)
+## 🕵️ Honeypot Field (in your template)
 
 ```django
 {% load aiwaf_tags %}
@@ -156,22 +168,23 @@ MIDDLEWARE = [
 
 ---
 
-## Running Detection & Training
+## 🔁 Running Detection & Training
 
 ```bash
 python manage.py detect_and_train
 ```
 
-**What happens:**
-1. Read access logs
+### What happens:
+1. Read access logs (incl. rotated or gzipped)
 2. Auto‑block IPs with ≥ 6 total 404s
 3. Extract features & train IsolationForest
 4. Save `model.pkl`
 5. Extract top 10 dynamic keywords from 4xx/5xx
+6. Remove any keywords associated with newly exempt paths
 
 ---
 
-## How It Works
+## 🧠 How It Works
 
 | Middleware                         | Purpose                                                         |
 |------------------------------------|-----------------------------------------------------------------|
@@ -180,15 +193,16 @@ python manage.py detect_and_train
 | AIAnomalyMiddleware                | ML‑driven behavior analysis + block on anomaly                  |
 | HoneypotMiddleware                 | Detects bots filling hidden inputs in forms                     |
 | UUIDTamperMiddleware               | Blocks guessed/nonexistent UUIDs across all models in an app    |
+
 ---
 
-## License
+## 📄 License
 
 This project is licensed under the **MIT License**. See the [LICENSE](LICENSE) file for details.
 
 ---
 
-## Credits
+## 👤 Credits
 
 **AI‑WAF** by [Aayush Gauba](https://github.com/aayushgauba)  
 > “Let your firewall learn and evolve — keep your site a fortress.”

{aiwaf-0.1.7 → aiwaf-0.1.7.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "aiwaf"
-version = "0.1.7"
+version = "0.1.7.2"
 description = "AI-powered Web Application Firewall"
 readme = "README.md"
 requires-python = ">=3.8"

{aiwaf-0.1.7 → aiwaf-0.1.7.2}/setup.py

@@ -9,7 +9,7 @@ long_description = (HERE / "README.md").read_text(encoding="utf-8")
 
 setup(
     name="aiwaf",
-    version="0.1.7",
+    version="0.1.7.2",
     description="AI‑driven, self‑learning Web Application Firewall for Django",
     long_description=long_description,
     long_description_content_type="text/markdown",