wimotek-edge-agent 1.0.13 → 1.0.15

package/README.md

@@ -14,6 +14,9 @@ Commercial-grade edge access control, service bridge, and tunnel agent featuring
 - [Programmatic usage](#programmatic-usage)
 - [Reverse tunnel / NAT4](#reverse-tunnel--nat4)
 - [Audit log](#audit-log)
+  - [Storage Mechanics & Format](#1-storage-mechanics--format)
+  - [Risks of Unmanaged Log Growth](#2-risks-of-unmanaged-log-growth)
+  - [Log Rotation & Cleanup Solutions](#3-log-rotation--cleanup-solutions)
 - [Security model](#security-model)
 - [Commercial deployment checklist](#commercial-deployment-checklist)
 - [License](#license)
@@ -241,6 +244,8 @@ npx wimotek-edge-agent --help
 | `EDGE_AGENT_SESSION_TICKET_TTL_SECONDS` | no | `300` | Session ticket TTL (seconds) |
 | `EDGE_AGENT_TOFU_HEARTBEAT_SECONDS` | no | `10` | TOFU quarantine heartbeat interval |
 | `EDGE_AGENT_REQUIRE_LOCAL_BYPASS_TOKEN` | no | `true` | Require local bypass token for loopback |
+| `EDGE_AGENT_AUDIT_MAX_BYTES` | no | `10485760` (10MB) | Max audit log size before auto-rotation |
+| `EDGE_AGENT_AUDIT_BACKUP_COUNT` | no | `5` | Number of rotated audit log files to keep |
 | `EDGE_AGENT_REVERSE_TUNNEL` | no | auto | Force reverse tunnel (`true`/`false`) |
 | `EDGE_AGENT_NAT_TYPE` | no | `unknown` | Network NAT type (`nat4`, `symmetric`, etc.) |
 | `EDGE_AGENT_NODE_UUID` | reverse only | - | Agent node UUID |
@@ -310,6 +315,16 @@ Read or generate the local bypass token:
 wimotek-edge-agent bypass-token
 ```
 
+Clean or truncate the audit log (keep recent N lines or 0 for complete truncate):
+
+```bash
+# Keep latest 500 lines:
+wimotek-edge-agent clean-audit --keep 500
+
+# Completely wipe audit log:
+wimotek-edge-agent clean-audit --keep 0
+```
+
 ## Programmatic usage
 
 ```python
@@ -364,13 +379,76 @@ await agent.run_reverse_tunnel(
 
 ## Audit log
 
-Every `AccessDecision` is written to `EDGE_AGENT_DATA_DIR/audit.jsonl`:
+### 1. Storage Mechanics & Format
+
+Every `AccessDecision` and administrative action is written to `$EDGE_AGENT_DATA_DIR/audit.jsonl` (default: `./edge_agent_data/audit.jsonl`).
+Lines are formatted as standard **JSON Lines (JSONL)** and protected by a thread-safe mutex lock:
 
 ```json
 {"ts":"2026-11-06T12:00:00+00:00","event":"access_decision","node_uuid":"node-xxx","source_ip":"203.0.113.10","is_loopback":false,"decision":"allow_token","reason":"edge_agent_token_verified","network_profile":"direct","nat_type":"unknown","fingerprint":"sha256:...","node_status":"trusted"}
 ```
 
-Ship this structured log file to your central logging pipeline (Loki, ELK, CloudWatch, OpenTelemetry). Raw join tokens are never logged.
+Raw join tokens are never logged.
+
+---
+
+### 2. Risks of Unmanaged Log Growth
+
+In high-throughput edge environments (hundreds of handshakes/sec):
+1. **Disk Exhaustion (ENOSPC)**: Without rotation, `audit.jsonl` can grow to tens of gigabytes, exhausting device storage.
+2. **Database Failure**: When disk fills up, SQLite (`trust_store.sqlite3`) fails during WAL commits, causing trust state and replay tracking to crash.
+3. **Memory & I/O Spikes**: Reading massive un-rotated log files causes severe memory consumption and potential OOM kills.
+4. **Compliance & SIEM Delay**: Retaining only raw local logs without shipping violates enterprise zero-trust audit policies.
+
+---
+
+### 3. Log Rotation & Cleanup Solutions
+
+#### A. Built-in Automatic Size Rotation (Recommended)
+`AuditLogger` features built-in auto-rotation when file size reaches the threshold:
+- Automatically rolls into `audit.jsonl.1` through `audit.jsonl.5`, purging the oldest file.
+- Configurable via environment variables:
+  ```bash
+  # Maximum file size before rotation (default: 10485760 = 10MB)
+  export EDGE_AGENT_AUDIT_MAX_BYTES=10485760
+  # Number of rotated archives to retain (default: 5)
+  export EDGE_AGENT_AUDIT_BACKUP_COUNT=5
+  ```
+
+#### B. CLI Manual Truncate & Clear
+```bash
+# Keep latest 500 lines and safely truncate history:
+wimotek-edge-agent clean-audit --keep 500
+
+# Completely wipe audit log:
+wimotek-edge-agent clean-audit --keep 0
+```
+
+#### C. Web Console 1-Click Clean
+In the Edge Agent Web Console:
+- Navigate to **"安全审计 (Audit Log)"** tab.
+- Click the red **"清理日志 (Clear Logs)"** button to truncate safely with confirmation.
+
+#### D. Production Linux Logrotate Configuration
+For systemd/daemon deployments, configure `/etc/logrotate.d/wimotek-edge-agent`:
+
+```text
+/path/to/edge_agent_data/audit.jsonl {
+    daily
+    missingok
+    rotate 7
+    compress
+    delaycompress
+    notifempty
+    copytruncate
+}
+```
+*(The `copytruncate` directive allows safe truncation without interrupting running agent processes).*
+
+#### E. SIEM Central Log Shipping
+Stream JSONL lines via Vector, FluentBit, or Filebeat directly to Loki, Elasticsearch, or AWS CloudWatch.
+
+---
 
 ## Security model
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "wimotek-edge-agent",
-  "version": "1.0.13",
+  "version": "1.0.15",
   "description": "Universal Edge Reverse Proxy & Service Bridge Agent CLI Runner",
   "bin": {
     "wimotek-edge-agent": "./bin/cli.js",

package/pyproject.toml

@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "wimotek-edge-agent"
-version = "1.0.13"
+version = "1.0.15"
 description = "Universal Edge Reverse Proxy & Service Bridge Agent"
 readme = "README.md"
 requires-python = ">=3.9"

package/wimotek_edge_agent/agent.py

@@ -40,7 +40,11 @@ class EdgeAgent:
         self.config.ensure_dirs()
         self.data_handler = data_handler
 
-        self.audit = AuditLogger(config.audit_log_path)
+        self.audit = AuditLogger(
+            config.audit_log_path,
+            max_bytes=config.audit_max_bytes,
+            backup_count=config.audit_backup_count,
+        )
         self.trust_store = TrustStore(config.trust_db_path)
         self.local_bypass = LocalBypassManager(
             config.local_bypass_token_path,
@@ -451,6 +455,15 @@ def build_parser() -> argparse.ArgumentParser:
     p_sim.add_argument("--reverse-tunnel", action="store_true")
     p_sim.add_argument("--nat-type", default="unknown")
 
+    # 13. clean-audit
+    p_clean = subparsers.add_parser("clean-audit", help="Truncate or clear audit log")
+    p_clean.add_argument(
+        "--keep",
+        type=int,
+        default=0,
+        help="Keep last N records (default 0, clears all records)",
+    )
+
     return parser
 
 
@@ -491,6 +504,16 @@ def main(argv: Optional[list[str]] = None) -> int:
         }, indent=2))
         return 0
 
+    if args.command == "clean-audit":
+        retained = agent.audit.clear(keep_last_lines=args.keep)
+        agent.audit.emit("audit_log_cleaned", retained_lines=retained, keep_requested=args.keep)
+        print(json.dumps({
+            "status": "ok",
+            "message": f"Audit log cleaned (retained {retained} lines)",
+            "retained_lines": retained,
+        }, indent=2))
+        return 0
+
     if args.command == "stats":
         stats = agent.trust_store.get_stats()
         print(json.dumps(stats, indent=2))

package/wimotek_edge_agent/audit.py

@@ -9,14 +9,45 @@ from typing import Any
 
 class AuditLogger:
     """
-    结构化审计日志。商业部署建议再接入中心端日志平台。
+    结构化审计日志管理器。
+    支持自动按大小轮转 (Auto-Rotation) 与安全截断清理，防止未清理导致磁盘耗尽。
     """
 
-    def __init__(self, path: Path) -> None:
+    def __init__(
+        self,
+        path: Path,
+        max_bytes: int = 10 * 1024 * 1024,
+        backup_count: int = 5,
+    ) -> None:
         self.path = path
         self.path.parent.mkdir(parents=True, exist_ok=True)
+        self.max_bytes = max_bytes
+        self.backup_count = backup_count
         self._lock = threading.Lock()
 
+    def _rotate_if_needed(self) -> None:
+        if self.max_bytes <= 0 or self.backup_count <= 0:
+            return
+        try:
+            if not self.path.exists() or self.path.stat().st_size < self.max_bytes:
+                return
+
+            for i in range(self.backup_count - 1, 0, -1):
+                sfn = self.path.with_name(f"{self.path.name}.{i}")
+                dfn = self.path.with_name(f"{self.path.name}.{i + 1}")
+                if sfn.exists():
+                    if dfn.exists():
+                        dfn.unlink()
+                    sfn.rename(dfn)
+
+            dfn = self.path.with_name(f"{self.path.name}.1")
+            if dfn.exists():
+                dfn.unlink()
+            self.path.rename(dfn)
+        except Exception:
+            # 保证主线程写日志不因归档异常而崩溃
+            pass
+
     def emit(self, event: str, **fields: Any) -> None:
         record = {
             "ts": datetime.now(timezone.utc).isoformat(),
@@ -25,5 +56,31 @@ class AuditLogger:
         }
         line = json.dumps(record, ensure_ascii=False, default=str, sort_keys=True)
         with self._lock:
+            self._rotate_if_needed()
             with self.path.open("a", encoding="utf-8") as f:
-                f.write(line + "\n")
+                f.write(line + "\n")
+
+    def clear(self, keep_last_lines: int = 0) -> int:
+        """
+        清理或截断审计日志。
+        - keep_last_lines = 0: 完全清空
+        - keep_last_lines > 0: 保留最近 N 行
+        返回清理后保留的行数。
+        """
+        with self._lock:
+            if not self.path.exists():
+                return 0
+            if keep_last_lines <= 0:
+                self.path.write_text("", encoding="utf-8")
+                return 0
+            try:
+                content = self.path.read_text(encoding="utf-8")
+                lines = [l for l in content.splitlines() if l.strip()]
+                remaining = lines[-keep_last_lines:]
+                self.path.write_text(
+                    "\n".join(remaining) + ("\n" if remaining else ""),
+                    encoding="utf-8",
+                )
+                return len(remaining)
+            except Exception:
+                return 0

package/wimotek_edge_agent/config.py

@@ -44,6 +44,10 @@ class Config:
     session_ticket_ttl_seconds: int = 300
     tofu_heartbeat_seconds: int = 10
 
+    # 审计日志自动轮转配置
+    audit_max_bytes: int = 10 * 1024 * 1024  # 默认 10MB
+    audit_backup_count: int = 5              # 默认保留 5 个归档
+
     # None 表示自动判断；True 表示强制反向隧道；False 表示强制直连。
     reverse_tunnel: Optional[bool] = None
     nat_type: Optional[str] = None
@@ -99,6 +103,12 @@ class Config:
             tofu_heartbeat_seconds=int(
                 os.getenv("EDGE_AGENT_TOFU_HEARTBEAT_SECONDS", "10")
             ),
+            audit_max_bytes=int(
+                os.getenv("EDGE_AGENT_AUDIT_MAX_BYTES", str(10 * 1024 * 1024))
+            ),
+            audit_backup_count=int(
+                os.getenv("EDGE_AGENT_AUDIT_BACKUP_COUNT", "5")
+            ),
             reverse_tunnel=_env_opt_bool("EDGE_AGENT_REVERSE_TUNNEL"),
             nat_type=os.getenv("EDGE_AGENT_NAT_TYPE") or None,
             tls_server_name=os.getenv("EDGE_AGENT_TLS_SERVER_NAME") or None,