@topy-ai/maggie 0.7.56 → 0.7.58
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README-zh-TW.md +6 -2
- package/README.md +18 -4
- package/bundled-skills/maggie-security/SKILL.md +2 -0
- package/bundled-skills/maggie-security/references/commands.md +15 -0
- package/bundled-skills/maggie-security/registry/security-tools.registry.json +27 -1
- package/bundled-tools/clis/maggie_security_host.py +12 -6
- package/bundled-tools/clis/maggie_security_mcp.py +12 -1
- package/bundled-tools/clis/maggie_security_report.py +188 -14
- package/bundled-tools/clis/maggie_security_web.py +311 -0
- package/bundled-tools/clis/security_data/tools.lock.json +7 -1
- package/package.json +1 -1
package/README-zh-TW.md
CHANGED
|
@@ -29,7 +29,11 @@ npx @topy-ai/maggie init --scope project --agent all
|
|
|
29
29
|
npx @topy-ai/maggie doctor --scope project --project .
|
|
30
30
|
```
|
|
31
31
|
|
|
32
|
-
目前 npm 版本為 `0.7.
|
|
32
|
+
目前 npm 版本為 `0.7.58`:新增 `maggie security fix-plan`(由發現整理出的修復 runbook:檔案權限、防火牆、套件/apt 更新、入侵防護、Web server 設定、EOL;產生一份你自己在 host lock 下、備份後執行的 shell script,Maggie 從不代為套用),並把 lynis tarball 以 sha256 pin 住,讓 `host-audit` 能透過 SSH 做深度主機稽核。
|
|
33
|
+
|
|
34
|
+
0.7.57 新增 `maggie security public-check`,對你**無法控制**的站點做被動、等同瀏覽器的檢查(安全 headers、TLS、首頁洩漏的版本與其已知 CVE、SPF/DMARC、security.txt),讓你能通知站主;只限真人互動執行,絕不探測、提交或掃描,主動掃描(web-scan、zap-baseline)與主機檢查仍須先驗證域名。
|
|
35
|
+
|
|
36
|
+
0.7.56 新增 Maggie Security(`maggie-security` skill 與 `maggie security` 指令,共 46 個指令、29 個 MCP 工具),涵蓋套件、runtime、資料庫、Web server、Linux 發行版與 WordPress 的漏洞公告與 end-of-life 日期,以及 secrets、程式碼、網站與主機檢查、報告、CI gate 與每日監控。
|
|
33
37
|
|
|
34
38
|
0.7.55 完成 feedback batch 31,共 16 條,來自一個以排程 AI agent 操作自家行銷產品、同時維護 PHP 主機的團隊,並同步發布 MaggieDash 0.2.15。
|
|
35
39
|
|
|
@@ -139,7 +143,7 @@ structure drift 或重複 treatment product 時 fail closed。
|
|
|
139
143
|
deployment、memory、feedback 和 MaggieDash。內容先 draft/review,外部寫入、
|
|
140
144
|
publish 與 production deployment 需要明確確認。
|
|
141
145
|
|
|
142
|
-
目前 npm 版本為 `0.7.
|
|
146
|
+
目前 npm 版本為 `0.7.58`,包含 fix-plan、public-check、Maggie Security、batch 31 排程 agent routines、報告收件匣與無人值守 agent 限制、PHP runtime 盤點與主機鎖、pull 網站改寫、query 點子與社群佇列檢查,batch 30 MaggieDash 先裝核心、release privacy gate、feedback 備援、社群連結與圖片檢查、大量頁面產生與 app 檢查,batch 29 預約規則、部署、app QA、助理與社群功能,以及 batch 28 產品 app、assistant tools、mobile、product release、skills catalog 與 bridge host 功能、batch 26、27 feedback 修正與 idempotent `maggie update`與 Booking email-template 合約/CLI validator 同步。本版完成 2026-09-24 booking feedback batch:Calendar grid/free-slot entry 與 host modal focus-restoration evidence。本版新增 Starter marketplace provisioning plan;本版完成 feedback batches 22、23:加入 declarative browser interaction evidence、design markup/style/content-key gate、compiler source anchoring、entity double-escape protection、`update --help` read-only、noindex hypothetical indexability、可過期 baseline acknowledgement、DOM-only link extraction regression,以及 MaggieDash 0.2.10 的 content override、短期 browser session、CDN-safe public-session cache contracts。authentication 和 isolated dev server 仍由 host 負責。`maggie-booking` workflow、傳統 email/password、Stripe、Resend、public booking flow 及既有安全邊界維持不變;provider-bound video generation 與 transactional delivery 仍記錄在 issue #39,前一批延後功能仍在 issue #38,不會直接成為 active memory。0.7.33 記錄並審查一筆更正回饋;確認 0.7.32 已正確處理 batch index,沒有修改已驗證的 shared behavior。0.7.32 加入 feedback 一基索引支援、MaggieDash panel 的 source/freshness/error evidence、Consent Mode 與實際 write/no-op reconciliation、SEO baseline recapture、sitemap origin rebasing、content-diff scope、migration ledger reconciliation、deployment credential preflight、release provenance/runtime preflight、icon release gate、opt-in blog auto-publish gate,以及 least-privilege VPS deployer。也修復 top-level `maggie feedback` dispatcher parity。0.7.31 加入 update 差異摘要、atomic installer copy、subset install manifest merge,以及 inventory drift 時 doctor non-zero。0.7.30 補上 feedback tracking issue closeout gate 與 batch 14/15 audit ledger。0.7.29 修正 npm package landing page 使用英文 README;0.7.28 加入 versioned Gemini model policy、明確 fallback 與 provenance、bundled-first CLI
|
|
143
147
|
dispatch、`maggie --version`、feedback batch review 聚合與重複偵測、marketplace enrichment evidence、booking
|
|
144
148
|
worker/resolver evidence,以及 read-only migration preflight。它也包含 host-owned mobile app surface contract、signed-in
|
|
145
149
|
camera-state QA、直接 Astro route resolution、correlated feedback batch,以及
|
package/README.md
CHANGED
|
@@ -89,8 +89,22 @@ Maggie keeps the existing project foundation and asks for decisions before
|
|
|
89
89
|
shared routes, analytics, or publishing boundaries change. The current
|
|
90
90
|
package ships 23 installable skills and a local-first MaggieDash foundation.
|
|
91
91
|
|
|
92
|
-
This package is `0.7.
|
|
93
|
-
|
|
92
|
+
This package is `0.7.58`. It adds `maggie security fix-plan` — a
|
|
93
|
+
reviewed remediation runbook built from the open findings (file permissions,
|
|
94
|
+
firewall, package/apt updates, intrusion prevention, web-server config, end of
|
|
95
|
+
life). It writes a shell script you run yourself, under a host lock, after a
|
|
96
|
+
backup; Maggie never applies it. This release also pins the `lynis` tarball
|
|
97
|
+
(sha256) so `host-audit` can run the deep host audit over SSH.
|
|
98
|
+
|
|
99
|
+
The 0.7.57 release added `maggie security public-check`: passive,
|
|
100
|
+
browser-equivalent checks of sites you do **not** control (security headers,
|
|
101
|
+
TLS, versions disclosed on the home page and their known CVEs, SPF/DMARC,
|
|
102
|
+
`security.txt`), so you can tell an owner about a visible problem. It is
|
|
103
|
+
interactive-only and never probes, submits or scans; the active scanners
|
|
104
|
+
(`web-scan`, `zap-baseline`) and host checks stay verified-only.
|
|
105
|
+
|
|
106
|
+
The 0.7.56 release added Maggie Security, a new
|
|
107
|
+
`maggie-security` skill and `maggie security` command family (48 commands, 30
|
|
94
108
|
MCP tools, 7 contracts under `contracts/maggie-security/`).
|
|
95
109
|
|
|
96
110
|
- **Advisories for the whole stack.** Packages (npm, pnpm, Yarn, Bun,
|
|
@@ -837,8 +851,8 @@ artifact schemas.
|
|
|
837
851
|
Recommended upgrade sequence for the current release:
|
|
838
852
|
|
|
839
853
|
```bash
|
|
840
|
-
npx @topy-ai/maggie@0.7.
|
|
841
|
-
npx @topy-ai/maggie@0.7.
|
|
854
|
+
npx @topy-ai/maggie@0.7.58 update --scope project --project . --force
|
|
855
|
+
npx @topy-ai/maggie@0.7.58 cleanup --scope project --project .
|
|
842
856
|
```
|
|
843
857
|
|
|
844
858
|
Maintainers should pass npm credentials through the repository helper, never
|
|
@@ -46,9 +46,11 @@ Follow [Maggie Memory Hook](../../references/memory-hook.md) at invocation and c
|
|
|
46
46
|
| "are my dependencies safe?" | `maggie security deps` (every lockfile ecosystem + JavaScript copied into the site) | `security_deps_scan` |
|
|
47
47
|
| "is my site configured safely?" | verify first: `maggie security domain-verify start --domain example.com` (the person adds the TXT record) then `maggie security domain-verify check --domain example.com`; then `maggie security headers --url https://example.com` and `maggie security tls --host example.com`; weekly `maggie security web-scan --url https://example.com` | `security_domain_verify_start`, `security_domain_verify_check`, `security_headers_check`, `security_tls_check`, `security_web_scan` |
|
|
48
48
|
| "check my code before I commit" | `maggie security code-scan --changed-since HEAD` (full: `maggie security code-scan`) | `security_code_scan` |
|
|
49
|
+
| "check a site we don't own (so we can tell them)" | `maggie security public-check --url https://example.com --outreach notes/` — passive only (headers, TLS, visible versions + known CVEs, SPF/DMARC, security.txt); interactive-only; never scan or probe a site you don't control (that's `web-scan`/`zap-baseline`, verified domains only) | `security_public_check` |
|
|
49
50
|
| "are we being attacked / were we hacked?" | `maggie security logs-traffic --file access.log`, `maggie security logs-sigma --file access.log`, `maggie security logs-ioc --file access.log`; servers: `maggie security logs-remote --ssh web1=user@host` | `security_logs_check` |
|
|
50
51
|
| "is my server hardened?" | `maggie security host-quick --ssh web1=user@host` (or `--print-collector` for the person to run), `maggie security host-nginx-config --ssh web1=user@host`, deep: `maggie security host-audit --ssh web1=user@host` | `security_host_check` |
|
|
51
52
|
| "protect my server" (CrowdSec / fail2ban / Cloudflare) | `maggie security protect-plan --host web1 [--engine crowdsec\|fail2ban]` → the person reviews and runs it under the host lock; afterwards `maggie security protect-plan verify --plan FILE --ssh web1=user@host` | `security_protect_plan` |
|
|
53
|
+
| "give me the fixes to run" | after the checks: `maggie security fix-plan --script fix.sh --output RUNBOOK.md` — a reviewed runbook (file permissions, firewall, apt updates, intrusion prevention, nginx, EOL) built from the findings; the person reviews it and runs it themselves under the host lock. Maggie never applies it. | — |
|
|
52
54
|
| "monitor uptime / tell me when the site is down" | `maggie security uptime-init` (verified domains) → `maggie security uptime-deploy --hostname status.example.com` (plan) → `maggie security uptime-status --gatus URL`; jobs report with `maggie security uptime-push` | `security_uptime_status` |
|
|
53
55
|
| "how do we compare to OWASP?" / "threat model" | `maggie security checklist --level 1`, `maggie security threat-model --include-findings` | `security_checklist_get`, `security_threat_model` |
|
|
54
56
|
| "is this npm package safe to add?" | `maggie security preinstall PKG[@VERSION]` before installing (W5); never install on `block` | `security_package_vet` |
|
|
@@ -53,6 +53,20 @@ findings never leave the machine.
|
|
|
53
53
|
| `maggie security web-scan --url U [--profile safe\|full-safe]` | httpx fingerprint, then nuclei with the safe profile (no intrusive, fuzz, DoS, brute-force, OAST templates; ≤ 10 req/s; one scan per domain at a time) | the site |
|
|
54
54
|
| `maggie security zap-baseline --url U` | weekly passive ZAP baseline in Docker; header rules skipped (headers covers them) | the site |
|
|
55
55
|
|
|
56
|
+
## Public check (sites you do NOT control — passive only)
|
|
57
|
+
|
|
58
|
+
`public-check` is the one web command that does not need domain verification,
|
|
59
|
+
because it only does what any visitor already does: one GET of the home page, a
|
|
60
|
+
TLS handshake, and public DNS. It never probes paths, never submits anything,
|
|
61
|
+
and never runs an active scanner. Use it to spot a visible problem on someone
|
|
62
|
+
else's site and tell them. It is interactive-only (refused under `--ci` or in a
|
|
63
|
+
routine) and never writes to the ledger. `web-scan`, `zap-baseline` and the host
|
|
64
|
+
checks stay verified-only — do not run them against a site you do not control.
|
|
65
|
+
|
|
66
|
+
| Command | What it does | Contacted |
|
|
67
|
+
|---|---|---|
|
|
68
|
+
| `maggie security public-check --url U` / `--domain D` (repeatable) / `--domains FILE` `[--outreach DIR] [--rate N] [--max-sites N]` | per site: security headers + cookies, TLS certificate and protocols, versions disclosed on the home page (`generator` meta, `Server`/`X-Powered-By`, JS libraries) matched to known CVEs (OSV, WPVulnerability, KEV, EPSS), missing/weak SPF and DMARC, and missing `security.txt`. `--outreach` writes a disclosure-ready note per domain (what is visible, no exploit detail). | the site, system resolver, cloudflare-dns.com, api.osv.dev, wpvulnerability.net, KEV, api.first.org |
|
|
69
|
+
|
|
56
70
|
## Code
|
|
57
71
|
|
|
58
72
|
| Command | What it does |
|
|
@@ -108,4 +122,5 @@ Formats: nginx/Apache combined and vcombined, Caddy JSON, Cloudflare Logpush JSO
|
|
|
108
122
|
| Command | What it does |
|
|
109
123
|
|---|---|
|
|
110
124
|
| `maggie security report --kind weekly\|daily_ops\|incident\|gate [--audience owner\|technical] [--save]` | Markdown + JSON report from the latest runs; `--save` writes the `report_save` payload |
|
|
125
|
+
| `maggie security fix-plan [--priority p0..p4] [--script fix.sh] [--output RUNBOOK.md]` | a reviewed remediation runbook from the open findings: grouped by category (file permissions, firewall, package/apt updates, intrusion prevention, web-server config, EOL), each step the commands the findings themselves carry (never invented). Writes a shell script you run yourself, under `maggie deployment host-lock`, after a backup and with a second SSH session open. Maggie never applies it. |
|
|
111
126
|
| `maggie security mcp [--tools p0\|all] [--print-config claude\|codex]` | local stdio MCP server |
|
|
@@ -12,6 +12,10 @@
|
|
|
12
12
|
"project-files": {
|
|
13
13
|
"label": "Reads your project files",
|
|
14
14
|
"modelNote": "Local server only."
|
|
15
|
+
},
|
|
16
|
+
"interactive-only": {
|
|
17
|
+
"label": "A person must run it",
|
|
18
|
+
"modelNote": "Refused under --ci or in a scheduled routine; it checks sites you do not control, so a person decides to run it."
|
|
15
19
|
}
|
|
16
20
|
},
|
|
17
21
|
"cantDo": [
|
|
@@ -189,7 +193,7 @@
|
|
|
189
193
|
},
|
|
190
194
|
{
|
|
191
195
|
"name": "security_secrets_scan",
|
|
192
|
-
"description": "Find API keys, passwords and tokens in git history, project files and served build output (dist/, public/
|
|
196
|
+
"description": "Find API keys, passwords and tokens in git history, project files and served build output (dist/, public/ …), including .env values copied into the build. Values are never returned: only the file, line and a 4+4 character preview. Reads project files only; contacts nothing.",
|
|
193
197
|
"risk": "read",
|
|
194
198
|
"scope": "workspace",
|
|
195
199
|
"channels": [
|
|
@@ -353,6 +357,28 @@
|
|
|
353
357
|
"verified-domain"
|
|
354
358
|
]
|
|
355
359
|
},
|
|
360
|
+
{
|
|
361
|
+
"name": "security_public_check",
|
|
362
|
+
"description": "Passively check sites you do NOT control, the way a visitor's browser does: grade security headers, the TLS certificate, versions disclosed on the home page and their known CVEs, SPF/DMARC and security.txt. One home-page GET, a TLS handshake and public DNS only; never probes paths, submits anything or runs an active scanner. For telling an owner about a visible problem; interactive use only.",
|
|
363
|
+
"risk": "read",
|
|
364
|
+
"scope": "workspace",
|
|
365
|
+
"channels": [
|
|
366
|
+
"in-app",
|
|
367
|
+
"mcp",
|
|
368
|
+
"claude",
|
|
369
|
+
"chatgpt",
|
|
370
|
+
"cli"
|
|
371
|
+
],
|
|
372
|
+
"idempotent": true,
|
|
373
|
+
"openWorld": true,
|
|
374
|
+
"user": {
|
|
375
|
+
"label": "Check another site passively",
|
|
376
|
+
"detail": "Browser-equivalent checks of a site you do not own, so you can tell its owner. No scanning."
|
|
377
|
+
},
|
|
378
|
+
"gates": [
|
|
379
|
+
"interactive-only"
|
|
380
|
+
]
|
|
381
|
+
},
|
|
356
382
|
{
|
|
357
383
|
"name": "security_code_scan",
|
|
358
384
|
"description": "Look for unsafe code patterns in the project (unescaped HTML, server-side requests to user URLs, SQL built from strings, open redirects, insecure cookies, eval, secrets in public build variables, root containers) with semgrep and Maggie's own rules. Reads project files only; no rules are downloaded.",
|
|
@@ -329,9 +329,14 @@ def remote_lynis(ctx: core.Context, alias: str, target: str) -> dict:
|
|
|
329
329
|
subprocess.run(["ssh", "-o", "BatchMode=yes", target, f"umask 077 && mkdir -p {remote_dir}"], check=True, timeout=60)
|
|
330
330
|
subprocess.run(["scp", "-q", "-o", "BatchMode=yes", str(tarball), f"{target}:{remote_dir}/lynis.tar.gz"], check=True, timeout=300)
|
|
331
331
|
profile = shlex.quote(LYNIS_PROFILE)
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
|
|
332
|
+
# Run lynis with sudo when available (a full audit needs root); read the
|
|
333
|
+
# report back with the SAME sudo, because a root run writes report.dat
|
|
334
|
+
# root-owned and 0600 — a plain `cat` as the login user reads nothing.
|
|
335
|
+
script = (f"SUDO=$( [ $(id -u) != 0 ] && sudo -n true 2>/dev/null && echo 'sudo -n' ); "
|
|
336
|
+
f"cd {remote_dir} && tar xzf lynis.tar.gz && printf %s {profile} > maggie.prf && cd lynis && "
|
|
337
|
+
f"$SUDO ./lynis audit system --quick --no-colors --no-log "
|
|
338
|
+
f"--profile {remote_dir}/maggie.prf --report-file {remote_dir}/report.dat --logfile /dev/null >/dev/null 2>&1; "
|
|
339
|
+
f"$SUDO cat {remote_dir}/report.dat")
|
|
335
340
|
proc = subprocess.run(["ssh", "-o", "BatchMode=yes", target, script], capture_output=True, text=True, timeout=1800, check=False)
|
|
336
341
|
report = proc.stdout
|
|
337
342
|
finally:
|
|
@@ -385,7 +390,8 @@ def nginx_findings(ctx: core.Context, nginx: str, php: str, key_perms: str, host
|
|
|
385
390
|
findings = []
|
|
386
391
|
seen = set()
|
|
387
392
|
|
|
388
|
-
def add(fid: str, title: str, severity: str, summary: str, snippet: str | None = None, where: str = "", asvs: list[str] | None = None
|
|
393
|
+
def add(fid: str, title: str, severity: str, summary: str, snippet: str | None = None, where: str = "", asvs: list[str] | None = None,
|
|
394
|
+
steps: list[str] | None = None) -> None:
|
|
389
395
|
key = (fid, where)
|
|
390
396
|
if key in seen:
|
|
391
397
|
return
|
|
@@ -393,7 +399,7 @@ def nginx_findings(ctx: core.Context, nginx: str, php: str, key_perms: str, host
|
|
|
393
399
|
findings.append(core.make_finding(kind="misconfiguration", command="host-nginx-config", primary_id=fid, site=ctx.site,
|
|
394
400
|
title=title + (f" ({where})" if where else ""), severity=severity, severity_source="maggie",
|
|
395
401
|
confidence="likely", target={"host": host, "path": where or None}, asvs_ids=asvs or [], rule_id=fid,
|
|
396
|
-
remediation={"summary": summary, "steps": ["sudo nginx -t && sudo systemctl reload nginx"],
|
|
402
|
+
remediation={"summary": summary, "steps": steps or ["sudo nginx -t && sudo systemctl reload nginx"],
|
|
397
403
|
"snippets": [{"kind": "nginx", "text": snippet}] if snippet else [],
|
|
398
404
|
"references": ["https://nginx.org/en/docs/"], "cheat_sheet": None}))
|
|
399
405
|
|
|
@@ -471,7 +477,7 @@ def nginx_findings(ctx: core.Context, nginx: str, php: str, key_perms: str, host
|
|
|
471
477
|
parts = line.split(None, 2)
|
|
472
478
|
if len(parts) == 3 and (int(parts[0][-1]) > 0 or int(parts[0][-2]) > 4 or parts[1] != "root"):
|
|
473
479
|
add("ngx-tls-key-perms", f"TLS private key is readable by others ({parts[0]} {parts[1]})", "high", "chmod 600 the key and chown root.",
|
|
474
|
-
where=parts[2], asvs=["V13.3.1"])
|
|
480
|
+
where=parts[2], asvs=["V13.3.1"], steps=[f"sudo chmod 600 {shlex.quote(parts[2])} && sudo chown root:root {shlex.quote(parts[2])}"])
|
|
475
481
|
return findings
|
|
476
482
|
|
|
477
483
|
|
|
@@ -64,6 +64,9 @@ SCHEMAS: dict[str, dict] = {
|
|
|
64
64
|
"host": {"type": "string", "pattern": DOMAIN}, "port": {"type": "integer", "minimum": 1, "maximum": 65535}}},
|
|
65
65
|
"security_web_scan": {"type": "object", "required": ["url"], "additionalProperties": False, "properties": {
|
|
66
66
|
"url": {"type": "string", "pattern": r"^https?://[A-Za-z0-9.-]+(:\d+)?(/[^\s]*)?$"}, "profile": {"enum": ["safe", "full-safe"]}}},
|
|
67
|
+
"security_public_check": {"type": "object", "additionalProperties": False, "properties": {
|
|
68
|
+
"url": {"type": "string", "pattern": r"^https?://[A-Za-z0-9.-]+(:\d+)?(/[^\s]*)?$"},
|
|
69
|
+
"domains": {"type": "array", "items": {"type": "string", "pattern": DOMAIN}, "minItems": 1, "maxItems": 50}}},
|
|
67
70
|
"security_code_scan": {"type": "object", "additionalProperties": False, "properties": {
|
|
68
71
|
"changed_since": {"type": "string", "pattern": r"^[A-Za-z0-9._/~^-]{1,80}$"}}},
|
|
69
72
|
"security_logs_check": {"type": "object", "required": ["mode"], "additionalProperties": False, "properties": {
|
|
@@ -94,7 +97,8 @@ SCHEMAS: dict[str, dict] = {
|
|
|
94
97
|
"confirm": {"type": "boolean"}, "confirmation": {"type": "string", "maxLength": 80}}},
|
|
95
98
|
}
|
|
96
99
|
NETWORK_TOOLS = {"security_vuln_watch", "security_osv_query", "security_wp_check", "security_eol_check", "security_deps_scan",
|
|
97
|
-
"security_domain_verify_check", "security_headers_check", "security_tls_check", "security_web_scan", "
|
|
100
|
+
"security_domain_verify_check", "security_headers_check", "security_tls_check", "security_web_scan", "security_public_check",
|
|
101
|
+
"security_uptime_status",
|
|
98
102
|
"security_checklist_get", "security_package_vet", "security_exposure_check", "security_image_scan"}
|
|
99
103
|
|
|
100
104
|
|
|
@@ -207,6 +211,13 @@ class Server:
|
|
|
207
211
|
return ["tls", "--host", args.get("host", "x.invalid"), "--port", str(args.get("port", 443))]
|
|
208
212
|
if name == "security_web_scan":
|
|
209
213
|
return ["web-scan", "--url", args.get("url", "https://x.invalid"), "--profile", args.get("profile", "safe")]
|
|
214
|
+
if name == "security_public_check":
|
|
215
|
+
argv = ["public-check"]
|
|
216
|
+
if args.get("url"):
|
|
217
|
+
argv += ["--url", args["url"]]
|
|
218
|
+
for domain in args.get("domains") or []:
|
|
219
|
+
argv += ["--domain", domain]
|
|
220
|
+
return argv
|
|
210
221
|
if name == "security_code_scan":
|
|
211
222
|
return ["code-scan"] + (["--changed-since", args["changed_since"]] if args.get("changed_since") else [])
|
|
212
223
|
if name == "security_logs_check":
|
|
@@ -153,6 +153,9 @@ def why(finding: dict) -> str:
|
|
|
153
153
|
return "Anyone who sees the file can use this key."
|
|
154
154
|
if kev:
|
|
155
155
|
return "Attackers are already using this flaw (on the KEV list)."
|
|
156
|
+
if kind in ("misconfiguration", "header", "tls", "exposure", "attack-signal"):
|
|
157
|
+
sev = {"critical": "A serious", "high": "A serious", "medium": "A moderate", "low": "A minor", "info": "A minor"}[finding["severity"]]
|
|
158
|
+
return f"{sev} configuration weakness an attacker could use."
|
|
156
159
|
severity = {"critical": "A serious", "high": "A serious", "medium": "A moderate", "low": "A minor", "info": "A minor"}[finding["severity"]]
|
|
157
160
|
return f"{severity} known flaw (CVE) affects the version you run."
|
|
158
161
|
|
|
@@ -184,8 +187,10 @@ def group_actions(findings: list[dict]) -> list[dict]:
|
|
|
184
187
|
items = core.sort_findings(groups[key])
|
|
185
188
|
lead = items[0]
|
|
186
189
|
if key[0] == "single" or len(items) == 1:
|
|
190
|
+
la = lead.get("affected") or {}
|
|
187
191
|
actions.append({"priority": lead["priority"], "title": action_title(lead), "why": why(lead), "how": how(lead),
|
|
188
|
-
"ids": [lead["id"]], "count": 1, "lead": lead
|
|
192
|
+
"ids": [lead["id"]], "count": 1, "lead": lead,
|
|
193
|
+
"package": la.get("name"), "ecosystem": la.get("ecosystem"), "fix_version": (lead.get("fix") or {}).get("recommended")})
|
|
189
194
|
continue
|
|
190
195
|
affected = lead["affected"]
|
|
191
196
|
eco = affected.get("ecosystem")
|
|
@@ -205,7 +210,8 @@ def group_actions(findings: list[dict]) -> list[dict]:
|
|
|
205
210
|
+ ("; attackers are using at least one (on the KEV list)." if exploited else "."))
|
|
206
211
|
step_from = next((f for f in items if (f.get("fix") or {}).get("recommended") == target_version), lead)
|
|
207
212
|
actions.append({"priority": lead["priority"], "title": title, "why": reason, "how": how(step_from),
|
|
208
|
-
"ids": [f["id"] for f in items], "count": len(items), "lead": lead
|
|
213
|
+
"ids": [f["id"] for f in items], "count": len(items), "lead": lead,
|
|
214
|
+
"package": affected.get("name"), "ecosystem": eco, "fix_version": target_version})
|
|
209
215
|
return sorted(actions, key=lambda a: (core.PRIORITIES.index(a["priority"]), -a["count"]))
|
|
210
216
|
|
|
211
217
|
|
|
@@ -219,19 +225,25 @@ def explain_jargon(text: str, used: set[str]) -> str:
|
|
|
219
225
|
return re.sub(r"\b(" + "|".join(JARGON) + r")\b", sub, text)
|
|
220
226
|
|
|
221
227
|
|
|
222
|
-
def
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
if args.kind == "gate" and since in (None, "last-report"):
|
|
226
|
-
since = "6h" # the gate reads this deploy's runs only
|
|
228
|
+
def resolve_since(ctx: core.Context, since: str | None, kind: str = "weekly") -> str:
|
|
229
|
+
if kind == "gate" and since in (None, "last-report"):
|
|
230
|
+
return "6h" # the gate reads this deploy's runs only
|
|
227
231
|
if since in (None, "last-report"):
|
|
228
232
|
last = last_report_time(ctx.state)
|
|
229
233
|
if last:
|
|
230
234
|
hours = max(1, int((core.utcnow() - core.parse_time(last)).total_seconds() // 3600) + 1)
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
+
return f"{hours}h"
|
|
236
|
+
return "7d"
|
|
237
|
+
return since
|
|
238
|
+
|
|
239
|
+
|
|
240
|
+
def collect_findings(ctx: core.Context, since: str, run_ids: list[str] | None) -> tuple[list[dict], list[dict], set[str]]:
|
|
241
|
+
"""The current findings for a site: the latest run per command merged with the ledger.
|
|
242
|
+
|
|
243
|
+
Returns (findings with their derived status, the runs used, the set of fixed ids).
|
|
244
|
+
Shared by `report` and `fix-plan` so both see exactly the same state.
|
|
245
|
+
"""
|
|
246
|
+
runs = latest_runs(ctx.state, since, run_ids)
|
|
235
247
|
if not runs:
|
|
236
248
|
raise core.UsageError(f"no security runs in the last {since}; run e.g. `maggie security vuln-watch` first")
|
|
237
249
|
ledger = core.Ledger(ctx.state.path("ledger.jsonl"))
|
|
@@ -254,7 +266,13 @@ def build(ctx: core.Context) -> dict:
|
|
|
254
266
|
first_seen = (state or {}).get("first_seen") or finding.get("first_seen")
|
|
255
267
|
finding = dict(finding, status="new" if first_seen and core.parse_time(first_seen) >= floor and status in ("new", "open") else status)
|
|
256
268
|
findings.append(finding)
|
|
257
|
-
|
|
269
|
+
return core.sort_findings(findings), runs, fixed_ids
|
|
270
|
+
|
|
271
|
+
|
|
272
|
+
def build(ctx: core.Context) -> dict:
|
|
273
|
+
args = ctx.args
|
|
274
|
+
since = resolve_since(ctx, args.since, args.kind)
|
|
275
|
+
findings, runs, fixed_ids = collect_findings(ctx, since, args.runs)
|
|
258
276
|
actionable = [f for f in findings if f["status"] in ("new", "open")]
|
|
259
277
|
grouped = group_actions([f for f in actionable if f["priority"] in ("p0", "p1")])
|
|
260
278
|
p0 = sum(1 for a in grouped if a["priority"] == "p0")
|
|
@@ -271,7 +289,7 @@ def build(ctx: core.Context) -> dict:
|
|
|
271
289
|
|
|
272
290
|
built = set(maggie_security.area_commands())
|
|
273
291
|
missing = [c for c in EXPECTED.get(site_type, EXPECTED["unknown"]) if c not in ran and c in built]
|
|
274
|
-
period = {"from": core.iso(
|
|
292
|
+
period = {"from": core.iso(core.utcnow() - core.parse_duration(since)), "to": core.iso()}
|
|
275
293
|
actions = [f for f in actionable if f["priority"] in ("p0", "p1")]
|
|
276
294
|
report_id = f"{core.iso()[:10]}-{args.kind}-{core.sha256_bytes(json.dumps(sorted(f['id'] for f in findings)).encode())[:6]}"
|
|
277
295
|
data = {
|
|
@@ -490,6 +508,160 @@ def incident_cmd(ctx: core.Context) -> dict:
|
|
|
490
508
|
return data
|
|
491
509
|
|
|
492
510
|
|
|
511
|
+
# --------------------------------------------------------------------------- fix-plan (a reviewed remediation runbook)
|
|
512
|
+
|
|
513
|
+
# A fix-plan turns the open findings into an ordered runbook of the fixes the
|
|
514
|
+
# findings themselves already carry. It never connects to a host and never runs
|
|
515
|
+
# anything — the operator reviews it and runs it under a host lock. It only
|
|
516
|
+
# emits remediation text the checks authored; it does not invent commands.
|
|
517
|
+
|
|
518
|
+
FIX_CATEGORIES = [
|
|
519
|
+
("secrets-permissions", "Secrets and file permissions", "Quick, high-value fixes — do these first."),
|
|
520
|
+
("firewall", "Firewall", "Close everything except the ports you actually serve."),
|
|
521
|
+
("packages", "Package updates", "Apply in a maintenance window; reboot if the kernel or libc changed."),
|
|
522
|
+
("intrusion-prevention", "Intrusion prevention", "Use `maggie security protect-plan` for a reviewed fail2ban/CrowdSec setup."),
|
|
523
|
+
("web-server", "Web server configuration", "Edit the server/vhost files, then `sudo nginx -t && sudo systemctl reload nginx`."),
|
|
524
|
+
("end-of-life", "End-of-life upgrades", "Plan a move to a supported release line."),
|
|
525
|
+
("other", "Other", "Review and apply as appropriate."),
|
|
526
|
+
]
|
|
527
|
+
|
|
528
|
+
|
|
529
|
+
def fix_category(finding: dict) -> str:
|
|
530
|
+
kind = finding.get("kind")
|
|
531
|
+
rule = (finding.get("source") or {}).get("rule_id") or ""
|
|
532
|
+
command = (finding.get("source") or {}).get("command") or ""
|
|
533
|
+
title = finding.get("title", "").lower()
|
|
534
|
+
if kind == "secret" or "readable by others" in title or "world-readable" in title or "permission" in title:
|
|
535
|
+
return "secrets-permissions"
|
|
536
|
+
if "firewall" in title or rule.startswith("hq-firewall"):
|
|
537
|
+
return "firewall"
|
|
538
|
+
if "fail2ban" in title or "crowdsec" in title or "intrusion" in title:
|
|
539
|
+
return "intrusion-prevention"
|
|
540
|
+
if kind == "vulnerability":
|
|
541
|
+
return "packages"
|
|
542
|
+
if kind == "eol":
|
|
543
|
+
return "end-of-life"
|
|
544
|
+
if command in ("host-nginx-config", "headers", "tls") or kind in ("header", "tls"):
|
|
545
|
+
return "web-server"
|
|
546
|
+
return "other"
|
|
547
|
+
|
|
548
|
+
|
|
549
|
+
def fix_commands(action: dict) -> tuple[list[str], list[tuple[str, str]], list[str]]:
|
|
550
|
+
"""(shell commands, (kind, snippet) paste-ins, TODO notes) faithfully from the lead finding."""
|
|
551
|
+
lead = action["lead"]
|
|
552
|
+
rem = lead.get("remediation") or {}
|
|
553
|
+
steps = [s for s in (rem.get("steps") or []) if not s.startswith(("Listed by CISA", "Check you can log in"))]
|
|
554
|
+
snippets = [(s.get("kind") or "text", s.get("text") or "") for s in (rem.get("snippets") or []) if s.get("text")]
|
|
555
|
+
commands = [s for s in steps if re.match(r"^\s*(sudo |apt|chmod|chown|ufw|systemctl|nginx |visudo|sed |install )", s)]
|
|
556
|
+
todos = [] if (commands or snippets) else [rem.get("summary") or action["title"]]
|
|
557
|
+
return commands, snippets, todos
|
|
558
|
+
|
|
559
|
+
|
|
560
|
+
def fix_plan(ctx: core.Context) -> dict:
|
|
561
|
+
args = ctx.args
|
|
562
|
+
since = resolve_since(ctx, args.since)
|
|
563
|
+
findings, _runs, _fixed = collect_findings(ctx, since, args.runs)
|
|
564
|
+
actionable = [f for f in findings if f["status"] in ("new", "open")]
|
|
565
|
+
if args.priority:
|
|
566
|
+
floor = core.PRIORITIES.index(args.priority)
|
|
567
|
+
actionable = [f for f in actionable if core.PRIORITIES.index(f["priority"]) <= floor]
|
|
568
|
+
actions = group_actions(actionable)
|
|
569
|
+
buckets: dict[str, list[dict]] = {key: [] for key, _, _ in FIX_CATEGORIES}
|
|
570
|
+
for action in actions:
|
|
571
|
+
buckets[fix_category(action["lead"])].append(action)
|
|
572
|
+
|
|
573
|
+
host = next((f.get("target", {}).get("host") for f in actionable if (f.get("target") or {}).get("host")), None)
|
|
574
|
+
lock_target = host or "<user@host>"
|
|
575
|
+
sh: list[str] = [
|
|
576
|
+
"#!/usr/bin/env bash",
|
|
577
|
+
f"# Remediation runbook for {ctx.site}" + (f" ({host})" if host else "") + f" — generated {core.iso()[:10]} by maggie security fix-plan.",
|
|
578
|
+
"#",
|
|
579
|
+
"# REVIEW EVERY LINE BEFORE RUNNING. Maggie did not and will not run any of this.",
|
|
580
|
+
"# Run it yourself, as the operator, after:",
|
|
581
|
+
"# 1. taking a snapshot/backup of the server,",
|
|
582
|
+
"# 2. opening a SECOND SSH session (so a bad sshd/firewall change cannot lock you out),",
|
|
583
|
+
f"# 3. taking the change lock: maggie deployment host-lock acquire --host {lock_target} --purpose fix-plan",
|
|
584
|
+
"# Package upgrades belong in a maintenance window. Test nginx with `nginx -t` before every reload.",
|
|
585
|
+
"set -euo pipefail",
|
|
586
|
+
"",
|
|
587
|
+
]
|
|
588
|
+
md: list[str] = [f"# Remediation runbook — {ctx.site}" + (f" ({host})" if host else ""), "",
|
|
589
|
+
"Generated by `maggie security fix-plan`. **Review before running — Maggie never runs these.** "
|
|
590
|
+
"Take a backup, keep a second SSH session open, and work under "
|
|
591
|
+
f"`maggie deployment host-lock acquire --host {lock_target} --purpose fix-plan`.", ""]
|
|
592
|
+
apt_packages: list[str] = []
|
|
593
|
+
total = 0
|
|
594
|
+
for key, label, note in FIX_CATEGORIES:
|
|
595
|
+
items = buckets.get(key) or []
|
|
596
|
+
if not items:
|
|
597
|
+
continue
|
|
598
|
+
sh += [f"# =================== {label} ===================", f"# {note}", ""]
|
|
599
|
+
md += [f"## {label}", f"_{note}_", ""]
|
|
600
|
+
for action in items:
|
|
601
|
+
total += 1
|
|
602
|
+
sh.append(f"# [{action['priority']}] {action['title']} — {action['why']}")
|
|
603
|
+
md.append(f"- **{action['title']}** — {action['why']}")
|
|
604
|
+
if key == "packages":
|
|
605
|
+
# Build the fix command ourselves; never copy the finding's step text
|
|
606
|
+
# (it reads "apt-get install --only-upgrade X (fixed in Y)" — not valid shell).
|
|
607
|
+
name = action.get("package")
|
|
608
|
+
osfam = (action.get("ecosystem") or "").split(":")[0]
|
|
609
|
+
if name and osfam in ("Ubuntu", "Debian") and action.get("fix_version"):
|
|
610
|
+
apt_packages.append(name)
|
|
611
|
+
sh.append(f"# {name} -> {action['fix_version']} (in the apt batch above)")
|
|
612
|
+
else:
|
|
613
|
+
sh.append(f"# TODO (manual): {action['how']}")
|
|
614
|
+
md.append(f" - {action['how']}")
|
|
615
|
+
sh.append("")
|
|
616
|
+
continue
|
|
617
|
+
commands, snippets, todos = fix_commands(action)
|
|
618
|
+
for cmd in commands:
|
|
619
|
+
sh.append(cmd)
|
|
620
|
+
for kind, text in snippets:
|
|
621
|
+
sh.append(f"# {kind} config — paste into the right block:")
|
|
622
|
+
sh += [f"# {line}" for line in text.splitlines()]
|
|
623
|
+
md.append(f" - `{kind}`: `{text}`")
|
|
624
|
+
for todo in todos:
|
|
625
|
+
sh.append(f"# TODO (manual): {todo}")
|
|
626
|
+
md.append(f" - {todo}")
|
|
627
|
+
if commands:
|
|
628
|
+
md.append(" - run: " + "; ".join(f"`{c}`" for c in commands))
|
|
629
|
+
sh.append("")
|
|
630
|
+
md.append("")
|
|
631
|
+
if apt_packages:
|
|
632
|
+
pkgs = " ".join(sorted(set(apt_packages)))
|
|
633
|
+
insert = ["# Apply available Ubuntu/Debian security updates for the affected packages:",
|
|
634
|
+
"sudo apt-get update",
|
|
635
|
+
f"sudo apt-get install --only-upgrade {pkgs}",
|
|
636
|
+
"# Or apply all pending security updates at once:",
|
|
637
|
+
"# sudo unattended-upgrade -d # or: sudo apt-get upgrade", ""]
|
|
638
|
+
anchor = sh.index("# =================== Package updates ===================") + 3
|
|
639
|
+
sh[anchor:anchor] = insert
|
|
640
|
+
md.append("### apt batch")
|
|
641
|
+
md.append(f"```\nsudo apt-get update && sudo apt-get install --only-upgrade {pkgs}\n```\n")
|
|
642
|
+
runbook = "\n".join(md).rstrip() + "\n"
|
|
643
|
+
script = "\n".join(sh).rstrip() + "\n"
|
|
644
|
+
if args.script:
|
|
645
|
+
path = Path(args.script)
|
|
646
|
+
path.parent.mkdir(parents=True, exist_ok=True)
|
|
647
|
+
path.write_text(script, encoding="utf-8")
|
|
648
|
+
path.chmod(0o755)
|
|
649
|
+
if getattr(args, "output", None):
|
|
650
|
+
out = Path(args.output)
|
|
651
|
+
out.parent.mkdir(parents=True, exist_ok=True)
|
|
652
|
+
out.write_text(runbook, encoding="utf-8")
|
|
653
|
+
return {"schema": "maggie-security-fix-plan.v1", "site": ctx.site, "host": host, "generated_at": core.iso(),
|
|
654
|
+
"actions": total, "apt_packages": sorted(set(apt_packages)),
|
|
655
|
+
"script_path": str(args.script) if args.script else None,
|
|
656
|
+
"note": "Review before running. Maggie never applies these changes.", "markdown": runbook}
|
|
657
|
+
|
|
658
|
+
|
|
659
|
+
def _fix(p: argparse.ArgumentParser) -> None:
|
|
660
|
+
p.add_argument("--priority", choices=["p0", "p1", "p2", "p3", "p4"], help="only include findings at or above this priority")
|
|
661
|
+
p.add_argument("--script", help="write the runnable shell runbook to this path (chmod 755)")
|
|
662
|
+
p.add_argument("--runs", action="append", help="use these run ids instead of the latest per command")
|
|
663
|
+
|
|
664
|
+
|
|
493
665
|
def _incident(p: argparse.ArgumentParser) -> None:
|
|
494
666
|
p.add_argument("action", choices=["start", "log", "close", "show", "list"])
|
|
495
667
|
p.add_argument("--id")
|
|
@@ -503,4 +675,6 @@ def _incident(p: argparse.ArgumentParser) -> None:
|
|
|
503
675
|
COMMANDS = [core.Command("report", "one per-site report from the latest runs and the ledger (Markdown + JSON, report_save payload)", [],
|
|
504
676
|
configure, run, findings=False),
|
|
505
677
|
core.Command("incident", "timeline for a CVE response or other incident (local records; used by report --kind incident)", [], _incident,
|
|
506
|
-
incident_cmd, findings=False)
|
|
678
|
+
incident_cmd, findings=False),
|
|
679
|
+
core.Command("fix-plan", "a reviewed remediation runbook from the open findings (Markdown + a shell script); never applied by Maggie",
|
|
680
|
+
[], _fix, fix_plan, findings=False, routine_safe=False)]
|
|
@@ -30,6 +30,7 @@ import ssl
|
|
|
30
30
|
import struct
|
|
31
31
|
import subprocess
|
|
32
32
|
import tempfile
|
|
33
|
+
import time
|
|
33
34
|
import urllib.error
|
|
34
35
|
import urllib.parse
|
|
35
36
|
import urllib.request
|
|
@@ -883,6 +884,303 @@ def zap_baseline(ctx: core.Context):
|
|
|
883
884
|
return ctx.finish(findings, covered=[f"url:{url}"])
|
|
884
885
|
|
|
885
886
|
|
|
887
|
+
# --------------------------------------------------------------------------- public-check (passive, browser-equivalent)
|
|
888
|
+
|
|
889
|
+
# A public check only looks at what any visitor already sees: one GET of the
|
|
890
|
+
# home page, a TLS handshake, and public DNS. It never probes paths, never
|
|
891
|
+
# submits anything and never runs an active scanner (that is `web-scan` /
|
|
892
|
+
# `zap-baseline`, which stay verified-only). The point is to spot a problem on
|
|
893
|
+
# a site you do NOT control so you can tell its owner.
|
|
894
|
+
|
|
895
|
+
# <meta name="generator"> product -> how to record it in a stack.
|
|
896
|
+
GENERATORS = [
|
|
897
|
+
(re.compile(r"WordPress\s+([0-9]+(?:\.[0-9]+){1,2})", re.I), "wordpress-core"),
|
|
898
|
+
(re.compile(r"Drupal\s+([0-9]+(?:\.[0-9]+){0,2})", re.I), ("npm", "drupal")),
|
|
899
|
+
(re.compile(r"Joomla!?\s*-?\s*([0-9]+(?:\.[0-9]+){1,2})", re.I), ("npm", "joomla")),
|
|
900
|
+
]
|
|
901
|
+
# Server / X-Powered-By token -> runtime or service name used by the matchers.
|
|
902
|
+
SERVER_RUNTIMES = [
|
|
903
|
+
(re.compile(r"nginx/([0-9]+(?:\.[0-9]+){1,2})", re.I), "nginx"),
|
|
904
|
+
(re.compile(r"Apache/([0-9]+(?:\.[0-9]+){1,2})", re.I), "apache"),
|
|
905
|
+
(re.compile(r"PHP/([0-9]+(?:\.[0-9]+){1,2})", re.I), "php"),
|
|
906
|
+
(re.compile(r"OpenSSL/([0-9]+(?:\.[0-9]+){1,2}[a-z]?)", re.I), "openssl"),
|
|
907
|
+
]
|
|
908
|
+
ASSET_RE = re.compile(r"""(?:src|href)\s*=\s*['"]([^'"]+?\.(?:js|css))(?:\?[^'"]*)?['"]""", re.I)
|
|
909
|
+
GENERATOR_RE = re.compile(r"""<meta[^>]+name\s*=\s*['"]generator['"][^>]+content\s*=\s*['"]([^'"]+)['"]""", re.I)
|
|
910
|
+
|
|
911
|
+
|
|
912
|
+
def resolve_txt(ctx: core.Context, name: str) -> list[str]:
|
|
913
|
+
"""Best-effort TXT lookup: the system resolver, then public DoH. Never raises."""
|
|
914
|
+
values: list[str] = []
|
|
915
|
+
try:
|
|
916
|
+
servers = system_resolvers()
|
|
917
|
+
if servers:
|
|
918
|
+
values = dns_txt_udp(name, servers[0])
|
|
919
|
+
except OSError:
|
|
920
|
+
values = []
|
|
921
|
+
if not values:
|
|
922
|
+
try:
|
|
923
|
+
values = dns_txt_doh(ctx, name) or []
|
|
924
|
+
except (core.DegradedError, OSError, ValueError):
|
|
925
|
+
values = []
|
|
926
|
+
return values
|
|
927
|
+
|
|
928
|
+
|
|
929
|
+
def remote_fingerprint(host: str, response: dict, site: str) -> dict:
|
|
930
|
+
"""A maggie-security-stack.v1 built only from the home page a visitor receives."""
|
|
931
|
+
body = response.get("body") or b""
|
|
932
|
+
text = body.decode("utf-8", errors="replace") if isinstance(body, (bytes, bytearray)) else str(body)
|
|
933
|
+
headers = response.get("headers") or {}
|
|
934
|
+
components: list[dict] = []
|
|
935
|
+
runtimes: list[dict] = []
|
|
936
|
+
wordpress: dict = {}
|
|
937
|
+
seen: set[tuple] = set()
|
|
938
|
+
|
|
939
|
+
generator = ""
|
|
940
|
+
match = GENERATOR_RE.search(text)
|
|
941
|
+
if match:
|
|
942
|
+
generator = match.group(1)
|
|
943
|
+
for rx, target in GENERATORS:
|
|
944
|
+
got = rx.search(generator)
|
|
945
|
+
if not got:
|
|
946
|
+
continue
|
|
947
|
+
if target == "wordpress-core":
|
|
948
|
+
wordpress = {"core": got.group(1), "plugins": [], "themes": []}
|
|
949
|
+
else:
|
|
950
|
+
eco, cname = target
|
|
951
|
+
key = (eco, cname, got.group(1))
|
|
952
|
+
if key not in seen:
|
|
953
|
+
seen.add(key)
|
|
954
|
+
components.append({"purl": core.make_purl(eco, cname, got.group(1)), "ecosystem": eco, "name": cname,
|
|
955
|
+
"version": got.group(1), "scope": "runtime", "direct": True, "location": f"https://{host}/ (generator)"})
|
|
956
|
+
break
|
|
957
|
+
|
|
958
|
+
for header in ("server", "x-powered-by"):
|
|
959
|
+
value = " ".join(headers.get(header) or [])
|
|
960
|
+
for rx, name in SERVER_RUNTIMES:
|
|
961
|
+
got = rx.search(value)
|
|
962
|
+
if got and ("runtime", name) not in seen:
|
|
963
|
+
seen.add(("runtime", name))
|
|
964
|
+
runtimes.append({"name": name, "version": got.group(1), "source": f"{header} header", "host": None})
|
|
965
|
+
|
|
966
|
+
import maggie_security_inventory as inventory
|
|
967
|
+
|
|
968
|
+
extractors = inventory._retire_extractors()
|
|
969
|
+
assets = 0
|
|
970
|
+
for raw in ASSET_RE.findall(text):
|
|
971
|
+
if assets >= 40:
|
|
972
|
+
break
|
|
973
|
+
assets += 1
|
|
974
|
+
basename = raw.rstrip("/").rsplit("/", 1)[-1]
|
|
975
|
+
for lib, (name_res, _content_res, npm) in extractors.items():
|
|
976
|
+
version = None
|
|
977
|
+
for rx in name_res:
|
|
978
|
+
got = rx.search(basename)
|
|
979
|
+
if got:
|
|
980
|
+
version = got.group(1)
|
|
981
|
+
break
|
|
982
|
+
if not version:
|
|
983
|
+
continue
|
|
984
|
+
version = re.sub(r"([.-](min|slim|pack|prod|production|umd|esm|bundle|js))+$", "", version).rstrip(".-_")
|
|
985
|
+
key = ("npm", npm, version)
|
|
986
|
+
if key in seen:
|
|
987
|
+
break
|
|
988
|
+
seen.add(key)
|
|
989
|
+
components.append({"purl": core.make_purl("npm", npm, version), "ecosystem": "npm", "name": npm, "version": version,
|
|
990
|
+
"scope": "runtime", "direct": None, "location": f"https://{host}/{basename}"})
|
|
991
|
+
break
|
|
992
|
+
|
|
993
|
+
site_type = "wordpress" if wordpress else "custom"
|
|
994
|
+
digest = core.sha256_bytes(core.canonical_json([components, runtimes, wordpress]).encode("utf-8"))
|
|
995
|
+
return {"schema": "maggie-security-stack.v1", "site": {"id": site, "type": site_type, "domains": [host], "hosting": None},
|
|
996
|
+
"collected_at": core.iso(), "hash": f"sha256:{digest}", "sources": [{"kind": "homepage", "path": f"https://{host}/"}],
|
|
997
|
+
"components": components, "runtimes": runtimes, "wordpress": wordpress, "images": [], "hosts": [],
|
|
998
|
+
"watch": {"products": []}, "generator": generator}
|
|
999
|
+
|
|
1000
|
+
|
|
1001
|
+
def email_dns_findings(ctx: core.Context, host: str, site: str) -> list[dict]:
|
|
1002
|
+
"""Passive email-spoofing hygiene from public DNS: SPF and DMARC on the registrable domain."""
|
|
1003
|
+
domain = registrable(host)
|
|
1004
|
+
findings: list[dict] = []
|
|
1005
|
+
spf = [v for v in resolve_txt(ctx, domain) if v.lower().startswith("v=spf1")]
|
|
1006
|
+
if not spf:
|
|
1007
|
+
findings.append(core.make_finding(
|
|
1008
|
+
kind="misconfiguration", command="public-check", primary_id="email-spf-missing", site=site, title=f"{domain} has no SPF record",
|
|
1009
|
+
severity="low", severity_source="maggie", confidence="confirmed", target={"host": domain}, asvs_ids=[],
|
|
1010
|
+
remediation={"summary": "Publish an SPF TXT record so receivers can reject mail forged from this domain.", "steps": [],
|
|
1011
|
+
"snippets": [{"kind": "dns", "text": f'{domain}. TXT "v=spf1 -all" # tighten to your real senders'}],
|
|
1012
|
+
"references": ["https://datatracker.ietf.org/doc/html/rfc7208"], "cheat_sheet": None}))
|
|
1013
|
+
elif any("+all" in v.lower() for v in spf):
|
|
1014
|
+
findings.append(core.make_finding(
|
|
1015
|
+
kind="misconfiguration", command="public-check", primary_id="email-spf-permissive", site=site,
|
|
1016
|
+
title=f"{domain} SPF ends in +all (anyone may send as this domain)", severity="medium", severity_source="maggie",
|
|
1017
|
+
confidence="confirmed", target={"host": domain}, evidence=[{"path": "SPF", "excerpt": core.clean_excerpt(spf[0], 200)}],
|
|
1018
|
+
remediation={"summary": "Replace +all with -all (or ~all) so forged senders are rejected.", "steps": [], "snippets": [],
|
|
1019
|
+
"references": [], "cheat_sheet": None}))
|
|
1020
|
+
dmarc = [v for v in resolve_txt(ctx, f"_dmarc.{domain}") if v.lower().startswith("v=dmarc1")]
|
|
1021
|
+
if not dmarc:
|
|
1022
|
+
findings.append(core.make_finding(
|
|
1023
|
+
kind="misconfiguration", command="public-check", primary_id="email-dmarc-missing", site=site, title=f"{domain} has no DMARC record",
|
|
1024
|
+
severity="low", severity_source="maggie", confidence="confirmed", target={"host": f"_dmarc.{domain}"}, asvs_ids=[],
|
|
1025
|
+
remediation={"summary": "Publish a DMARC record so spoofed mail is reported and (later) rejected.", "steps": [],
|
|
1026
|
+
"snippets": [{"kind": "dns", "text": f'_dmarc.{domain}. TXT "v=DMARC1; p=none; rua=mailto:dmarc@{domain}"'}],
|
|
1027
|
+
"references": ["https://datatracker.ietf.org/doc/html/rfc7489"], "cheat_sheet": None}))
|
|
1028
|
+
elif any(re.search(r"\bp\s*=\s*none\b", v, re.I) for v in dmarc):
|
|
1029
|
+
findings.append(core.make_finding(
|
|
1030
|
+
kind="misconfiguration", command="public-check", primary_id="email-dmarc-none", site=site,
|
|
1031
|
+
title=f"{domain} DMARC policy is p=none (monitoring only, no enforcement)", severity="info", severity_source="maggie",
|
|
1032
|
+
confidence="confirmed", target={"host": f"_dmarc.{domain}"}, evidence=[{"path": "DMARC", "excerpt": core.clean_excerpt(dmarc[0], 200)}],
|
|
1033
|
+
remediation={"summary": "Once reports look clean, move the policy to quarantine then reject.", "steps": [], "snippets": [],
|
|
1034
|
+
"references": [], "cheat_sheet": None}))
|
|
1035
|
+
return findings
|
|
1036
|
+
|
|
1037
|
+
|
|
1038
|
+
def securitytxt_finding(ctx: core.Context, base: str, host: str, site: str) -> dict | None:
|
|
1039
|
+
"""One GET of /.well-known/security.txt; a missing file is an info nudge, not a risk."""
|
|
1040
|
+
try:
|
|
1041
|
+
response = fetch(urllib.parse.urljoin(base + "/", "/.well-known/security.txt"), follow=False, timeout=10)
|
|
1042
|
+
except (OSError, ssl.SSLError):
|
|
1043
|
+
return None
|
|
1044
|
+
if response.get("status") == 200 and b"contact" in (response.get("body") or b"").lower():
|
|
1045
|
+
return None
|
|
1046
|
+
return core.make_finding(
|
|
1047
|
+
kind="misconfiguration", command="public-check", primary_id="no-security-txt", site=site,
|
|
1048
|
+
title="No security.txt: no published way to report a vulnerability", severity="info", severity_source="maggie",
|
|
1049
|
+
confidence="confirmed", target={"url": f"https://{host}/.well-known/security.txt", "host": host}, asvs_ids=[],
|
|
1050
|
+
remediation={"summary": "Publish /.well-known/security.txt with a Contact: line so researchers can reach you.", "steps": [],
|
|
1051
|
+
"snippets": [{"kind": "text", "text": f"Contact: mailto:security@{registrable(host)}\nExpires: <date>"}],
|
|
1052
|
+
"references": ["https://www.rfc-editor.org/rfc/rfc9116"], "cheat_sheet": None})
|
|
1053
|
+
|
|
1054
|
+
|
|
1055
|
+
def advisory_findings_remote(ctx: core.Context, stack: dict) -> tuple[list[dict], list[str]]:
|
|
1056
|
+
"""Run the same advisory engine as vuln-watch over the passively-built remote stack."""
|
|
1057
|
+
import maggie_security_intel as intel
|
|
1058
|
+
|
|
1059
|
+
findings: list[dict] = []
|
|
1060
|
+
notes: list[str] = []
|
|
1061
|
+
try:
|
|
1062
|
+
osv, ok = intel.match_osv(ctx, stack, "public-check")
|
|
1063
|
+
findings += osv
|
|
1064
|
+
if not ok:
|
|
1065
|
+
notes.append("OSV lookup was degraded; version advisories may be incomplete")
|
|
1066
|
+
soft, _cov = intel.software_findings(ctx, stack, "public-check")
|
|
1067
|
+
findings += soft
|
|
1068
|
+
if stack.get("wordpress", {}).get("core"):
|
|
1069
|
+
wp, wp_ok = intel.wordpress_findings(ctx, stack, "public-check", budget=1)
|
|
1070
|
+
findings += wp
|
|
1071
|
+
if not wp_ok:
|
|
1072
|
+
notes.append("WordPress advisory lookup was degraded")
|
|
1073
|
+
kev = intel.load_kev(ctx)
|
|
1074
|
+
intel.apply_kev(findings, kev, stack)
|
|
1075
|
+
intel.join_epss(ctx, findings)
|
|
1076
|
+
except core.DegradedError as error:
|
|
1077
|
+
notes.append(f"advisory lookup degraded: {core.clean_excerpt(str(error), 160)}")
|
|
1078
|
+
return findings, notes
|
|
1079
|
+
|
|
1080
|
+
|
|
1081
|
+
SEVERITY_ORDER = {"critical": 0, "high": 1, "medium": 2, "low": 3, "info": 4}
|
|
1082
|
+
|
|
1083
|
+
|
|
1084
|
+
def outreach_markdown(host: str, findings: list[dict]) -> str:
|
|
1085
|
+
"""A disclosure-ready note for the site owner: what is visible, not how to exploit it."""
|
|
1086
|
+
by_sev: dict[str, list[dict]] = {}
|
|
1087
|
+
for f in findings:
|
|
1088
|
+
by_sev.setdefault(f["severity"], []).append(f)
|
|
1089
|
+
lines = [f"# Security notes for {host}", "",
|
|
1090
|
+
"We ran only passive checks — the same requests any visitor's browser makes "
|
|
1091
|
+
"(your home page, a TLS handshake, and public DNS). We did not probe, log in, "
|
|
1092
|
+
"or scan. Here is what is publicly visible, so you can decide what to fix.", ""]
|
|
1093
|
+
actionable = [s for s in ("critical", "high", "medium", "low") if by_sev.get(s)]
|
|
1094
|
+
if not actionable:
|
|
1095
|
+
lines += ["Nothing notable was visible from the outside. ", ""]
|
|
1096
|
+
for sev in ("critical", "high", "medium", "low", "info"):
|
|
1097
|
+
items = sorted(by_sev.get(sev) or [], key=lambda f: f["title"])
|
|
1098
|
+
if not items:
|
|
1099
|
+
continue
|
|
1100
|
+
lines.append(f"## {sev.title()}")
|
|
1101
|
+
for f in items:
|
|
1102
|
+
fix = (f.get("remediation") or {}).get("summary") or ""
|
|
1103
|
+
lines.append(f"- **{f['title']}**" + (f" — {fix}" if fix else ""))
|
|
1104
|
+
lines.append("")
|
|
1105
|
+
lines += ["---",
|
|
1106
|
+
"_Sent by <your name / company>. We have no affiliation with this site and "
|
|
1107
|
+
"hold no further details beyond the public observations above. If you would "
|
|
1108
|
+
"rather not hear from us again, just reply and we will stop._"]
|
|
1109
|
+
return "\n".join(lines)
|
|
1110
|
+
|
|
1111
|
+
|
|
1112
|
+
def _public_targets(args) -> list[str]:
|
|
1113
|
+
"""Base URLs to check, de-duplicated; a bare domain becomes https://domain/."""
|
|
1114
|
+
raw: list[str] = []
|
|
1115
|
+
if getattr(args, "url", None):
|
|
1116
|
+
raw.append(args.url)
|
|
1117
|
+
raw += list(getattr(args, "domain", None) or [])
|
|
1118
|
+
if getattr(args, "domains", None):
|
|
1119
|
+
for line in Path(args.domains).read_text(encoding="utf-8").splitlines():
|
|
1120
|
+
line = line.strip()
|
|
1121
|
+
if line and not line.startswith("#"):
|
|
1122
|
+
raw.append(line)
|
|
1123
|
+
bases = [(item if "://" in item else f"https://{item}") for item in raw]
|
|
1124
|
+
return list(dict.fromkeys(b for b in bases if host_of(b)))
|
|
1125
|
+
|
|
1126
|
+
|
|
1127
|
+
def public_check(ctx: core.Context):
|
|
1128
|
+
args = ctx.args
|
|
1129
|
+
ctx.args.no_ledger = True # these are other people's sites: never touch the local ledger
|
|
1130
|
+
bases = _public_targets(args)
|
|
1131
|
+
if not bases:
|
|
1132
|
+
raise core.UsageError("give --url, one or more --domain, or --domains FILE")
|
|
1133
|
+
if len(bases) > args.max_sites:
|
|
1134
|
+
raise core.PolicyError(f"{len(bases)} domains exceeds --max-sites {args.max_sites}; raise it deliberately")
|
|
1135
|
+
outreach_dir = Path(args.outreach) if args.outreach else None
|
|
1136
|
+
if outreach_dir:
|
|
1137
|
+
outreach_dir.mkdir(parents=True, exist_ok=True)
|
|
1138
|
+
all_findings: list[dict] = []
|
|
1139
|
+
sites: list[dict] = []
|
|
1140
|
+
for index, base in enumerate(bases):
|
|
1141
|
+
host = host_of(base)
|
|
1142
|
+
if index:
|
|
1143
|
+
time.sleep(max(0.0, 1.0 / max(args.rate, 0.2))) # be a polite visitor between sites
|
|
1144
|
+
require_ownership(ctx, host, public_check=True) # interactive-only; refuses in CI/routines
|
|
1145
|
+
entry = {"host": host, "notes": []}
|
|
1146
|
+
try:
|
|
1147
|
+
response = fetch(urllib.parse.urljoin(base + "/", "/"), timeout=15)
|
|
1148
|
+
except (OSError, ssl.SSLError) as error:
|
|
1149
|
+
entry["notes"].append(f"home page not reachable: {core.clean_excerpt(str(error), 160)}")
|
|
1150
|
+
sites.append(entry)
|
|
1151
|
+
continue
|
|
1152
|
+
if response.get("left_domain"):
|
|
1153
|
+
entry["notes"].append(f"redirects off-domain to {response['left_domain']}; checked only the first response")
|
|
1154
|
+
# The remote site is unrelated to the local project, so detect the serving
|
|
1155
|
+
# layer from its response headers alone, never from files on this machine.
|
|
1156
|
+
layer = detect_serving(Path("/nonexistent-maggie-prospect"), response["headers"], "auto")
|
|
1157
|
+
findings = grade_response(ctx, response["url"], response, layer, response["url"].startswith("https://"))
|
|
1158
|
+
findings += native_tls(ctx, host, 443, None)
|
|
1159
|
+
stack = remote_fingerprint(host, response, ctx.site)
|
|
1160
|
+
entry["detected"] = {"generator": stack.get("generator") or None,
|
|
1161
|
+
"components": [f"{c['name']} {c['version']}" for c in stack["components"]],
|
|
1162
|
+
"runtimes": [f"{r['name']} {r['version']}" for r in stack["runtimes"]]}
|
|
1163
|
+
advisories, notes = advisory_findings_remote(ctx, stack)
|
|
1164
|
+
findings += advisories
|
|
1165
|
+
entry["notes"] += notes
|
|
1166
|
+
findings += email_dns_findings(ctx, host, ctx.site)
|
|
1167
|
+
sec = securitytxt_finding(ctx, base, host, ctx.site)
|
|
1168
|
+
if sec:
|
|
1169
|
+
findings.append(sec)
|
|
1170
|
+
for f in findings:
|
|
1171
|
+
f.setdefault("target", {}).setdefault("host", host)
|
|
1172
|
+
entry["counts"] = {s: sum(1 for f in findings if f["severity"] == s) for s in SEVERITY_ORDER if any(x["severity"] == s for x in findings)}
|
|
1173
|
+
all_findings += findings
|
|
1174
|
+
if outreach_dir:
|
|
1175
|
+
report = outreach_dir / f"{registrable(host)}.md"
|
|
1176
|
+
report.write_text(outreach_markdown(host, findings), encoding="utf-8")
|
|
1177
|
+
entry["outreach"] = str(report)
|
|
1178
|
+
sites.append(entry)
|
|
1179
|
+
all_findings.sort(key=lambda f: (SEVERITY_ORDER.get(f["severity"], 9), f["title"]))
|
|
1180
|
+
ctx.run["inputs"].update({"mode": "public-check (passive, unverified)", "sites": sites, "label": "unverified domains (public check)"})
|
|
1181
|
+
return ctx.finish(all_findings, covered=[f"url:{host_of(b)}" for b in bases])
|
|
1182
|
+
|
|
1183
|
+
|
|
886
1184
|
# --------------------------------------------------------------------------- registration
|
|
887
1185
|
|
|
888
1186
|
|
|
@@ -923,6 +1221,15 @@ def _zap(p: argparse.ArgumentParser) -> None:
|
|
|
923
1221
|
p.add_argument("--image", help="pinned ZAP image reference (default ghcr.io/zaproxy/zaproxy:stable)")
|
|
924
1222
|
|
|
925
1223
|
|
|
1224
|
+
def _public(p: argparse.ArgumentParser) -> None:
|
|
1225
|
+
p.add_argument("--url", help="one site to check")
|
|
1226
|
+
p.add_argument("--domain", action="append", help="a domain to check (repeatable)")
|
|
1227
|
+
p.add_argument("--domains", help="file with one domain per line")
|
|
1228
|
+
p.add_argument("--outreach", help="write a disclosure-ready note per domain into this directory")
|
|
1229
|
+
p.add_argument("--rate", type=float, default=0.5, help="sites per second between domains (default 0.5)")
|
|
1230
|
+
p.add_argument("--max-sites", type=int, default=50, help="refuse a run with more domains than this")
|
|
1231
|
+
|
|
1232
|
+
|
|
926
1233
|
COMMANDS = [
|
|
927
1234
|
core.Command("domain-verify", "prove you control a domain (DNS TXT or a token file) before any active check", ["doh", "dns", "target-site"], _dv,
|
|
928
1235
|
domain_verify, findings=False),
|
|
@@ -933,4 +1240,8 @@ COMMANDS = [
|
|
|
933
1240
|
_scan, web_scan),
|
|
934
1241
|
core.Command("zap-baseline", "weekly passive ZAP baseline scan of your verified site (Docker)", ["target-site", "doh", "dns"], _zap, zap_baseline,
|
|
935
1242
|
routine_safe=False),
|
|
1243
|
+
core.Command("public-check",
|
|
1244
|
+
"passive, browser-equivalent checks of sites you do not control (headers, TLS, visible versions and their known CVEs, SPF/DMARC, "
|
|
1245
|
+
"security.txt) so you can tell the owner; never probes or scans",
|
|
1246
|
+
["target-site", "doh", "dns", "osv-api", "wpvulnerability", "kev", "epss"], _public, public_check, routine_safe=False),
|
|
936
1247
|
]
|
|
@@ -202,7 +202,13 @@
|
|
|
202
202
|
"forbidden_flags": [
|
|
203
203
|
"--upload"
|
|
204
204
|
],
|
|
205
|
-
"notes": "GPL:
|
|
205
|
+
"notes": "GPL: downloaded on use, sha256-verified, uploaded to a private /tmp dir on the host and removed after; never vendored into the package.",
|
|
206
|
+
"assets": {
|
|
207
|
+
"any": {
|
|
208
|
+
"url": "https://downloads.cisofy.com/lynis/lynis-3.1.5.tar.gz",
|
|
209
|
+
"sha256": "8d2c6652ba60116a82514522b666ca77293f4bfc69f1e581028769f7ebb52ba4"
|
|
210
|
+
}
|
|
211
|
+
}
|
|
206
212
|
},
|
|
207
213
|
"goaccess": {
|
|
208
214
|
"version": "1.9.4",
|