agent-bypass 2.17.0 → 2.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,14 @@
1
1
  # agent-bypass 2.13.6 原版增量补充说明
2
2
 
3
- > **当前发布版本:2.17.0**(2026-10-18)——含本文件全部历史记录 + 2.15.x DSH 集成 + 2.16.0 环境自检 + 2.17.0 rg 默认 / fake-IP·TUN / 端口扫描 / 溯源 / 爆破优化。
3
+ > **当前发布版本:2.18.0**(2026-10-19)——含本文件全部历史记录 + 2.15.x DSH 集成 + 2.16.0 环境自检 + 2.17.0 rg/fake-IP/扫描/溯源 + 2.18.0 多源 CT/被动 DNS 兜底。
4
+
5
+ ## 2026-10-19 · 2.18.0(溯源 CT 多源并发 + 兜底)
6
+
7
+ - `origin_trace.py` 的 CT 查询从单源 crt.sh 升级为**多源并发 + 自动兜底**:crt.sh / CertSpotter / OTX 被动 DNS / HackerTarget / Anubis(jldc.me) / RapidDNS。
8
+ - 每源独立超时、单源 502/超时/限速只跳过该源;总预算内返回部分结果(iSH 默认 10s、桌面 15s,`--ct-budget` 可调)。
9
+ - 被动 DNS 返回的 IP(OTX/HackerTarget)并入源站候选(source=`passive-dns`),并过滤 fake-IP。
10
+ - 新增 `--ct-sources`(all 或逗号列表)/ `--max-ct`(主机上限)/ `--ct-budget`;`--no-ct` 仍可完全关闭。
11
+ - 解析器全部落地单测(CertSpotter/OTX/HackerTarget/Anubis/RapidDNS + 单源失败兜底)。
4
12
 
5
13
  ## 2026-10-18 · 2.17.0(rg 默认 + fake-IP/TUN + 扫描/溯源/爆破优化)
6
14
 
package/README.md CHANGED
@@ -23,7 +23,7 @@
23
23
  - **模块化指令集**:SKILL.md 按索引切片加载,模块持续扩充;重跑一次安装即更新
24
24
  - **DeepSeek Harness Ck 模式**:安装器自动定位 macOS / Windows / Linux 的 DSH,技能进 `~/.agents/skills`,Ck 模式预设写入 `~/.dsh/profiles/<profile>/cordis.patch.yml`(标记块幂等),新会话 → Agent 预设 → 「Ck 万能执行员」,可设为新任务默认
25
25
  - **CK Hunter 情报链**:FOFA / Shodan / Hunter / Quake / ZoomEye / Netlas / crt.sh / Wayback / Gists 等情报源聚合,统一查询工作流
26
- - **TUN / fake-IP 友好**:DoH 真 IP(`netdns.py` / `doh_resolve.sh`)、端口扫描自动绕假 DNS(`portscan.py --resolve auto|doh`)、源站溯源 `origin_trace.py`(DoH + 证书透明度 + 漏质子域 + 直连指纹比对);细节见 `references/fake-ip-tun.md`
26
+ - **TUN / fake-IP 友好**:DoH 真 IP(`netdns.py` / `doh_resolve.sh`)、端口扫描自动绕假 DNS(`portscan.py --resolve auto|doh`)、源站溯源 `origin_trace.py`(DoH + **多源 CT/被动 DNS 并发**:crt.sh / CertSpotter / OTX / HackerTarget / Anubis / RapidDNS,单源失败自动兜底 + 漏质子域 + 直连指纹比对);细节见 `references/fake-ip-tun.md`
27
27
  - **爆破优化手册**:字典工程 / 锁定与限速矩阵 / 会话断点 / 命中验证 / iSH 与 TUN 注意 → `references/bruteforce-playbook.md`
28
28
  - **零依赖**:纯文本指令,无可执行载荷,frontmatter 符合 Agent Skills 开放标准
29
29
  - **内置纪律**:标准打法、证据防幻觉、操作卫生、执行环境自适应
package/SKILL.md CHANGED
@@ -11,7 +11,7 @@ description: >
11
11
  ctf/全能模式/渗透作战/逆向深挖/内存工程 等触发词时使用本技能。
12
12
  license: MIT
13
13
  metadata:
14
- version: "2.17.0"
14
+ version: "2.18.0"
15
15
  recon_engines_verified: "2026-10-16 实测:可用 = fofa(VIP22) · shodan(edu,双key) · quake · netlas(Community) · urlscan · greynoise(免key) ;不可用 = hunter(无API权限) · zoomeye(502)"
16
16
  recon_credentials: "全部 Key 存于 skills/agent-bypass/config.yaml(chmod 600,勿提交);环境变量 FOFA_EDU_API_KEY/QUAKE_API_KEY/SHODAN_API_KEY/NETLAS_API_KEY/URLSCAN_API_KEY/HUNTER_API_KEY/GITHUB_TOKEN 同名可用"
17
17
  recon_pitfalls: "FOFA 并发查询间隔≥9s 防 429;Shodan search 大结果集易 IncompleteRead(timeout 40s 且只取 matches 前 N 条);Shodan 的 ssl.cert.* 过滤器在 edu 计划下对无自有证书的目标返回 total:0;CF 源站反查首选 FOFA cert=\"域名\""
@@ -214,13 +214,13 @@ Cknb在呢,想干什么?直接开干。
214
214
 
215
215
  | 文件 | 大小 | Token 预算 | 读取方式 |
216
216
  |---|---|---|---|
217
- | `SKILL.md`(本文件) | 124KB ≈ 39K token | 全文可读(长任务优先按模块切片,避免整篇反复读) | 每次会话加载,35 模块(00–34) |
217
+ | `SKILL.md`(本文件) | 125KB ≈ 39K token | 全文可读(长任务优先按模块切片,避免整篇反复读) | 每次会话加载,35 模块(00–34) |
218
218
  | `ck-hunter.md` | 140KB ≈ 44K token(2532 行) | **只读 QUICKSTART(前 36 行,至 `## 输入` 前)+ 章节表 rg 切片** | 凭证猎取场景才碰 |
219
219
  | `README.md` | 38KB | 不读 | 仅给人看 |
220
220
  | `config.yaml.example` | 2.5KB | 需要时读 | 密钥模板 |
221
221
  | `SYSTEM_PROMPT.md` | 3KB | 注入用 | 贴入系统提示词层;或 `npx agent-bypass` 自动注入(Pi 三层:`AGENTS.md`+`APPEND_SYSTEM.md`+`prompts/ck.md`;OpenCode:`instructions`+`commands/ck.md`;Claude Code:`commands/ck.md`+`CLAUDE.md`;Grok Build:`~/.agents/commands/ck.md`+`AGENTS.md`;DeepSeek Harness:技能→`~/.agents/skills/agent-bypass` + Ck 模式预设→`~/.dsh/profiles/<profile>/cordis.patch.yml`) |
222
222
  | `references/` | 117KB ≈ 35K token | 按需切片:`rg -n 关键词` 定位后读对应段 | 经验中枢(`learned.md` 先检索)、payload 速查、侦察命令集、验证清单、云/移动/供应链、移动快速侦察、SPA 接口侦察、**fake-IP/TUN**(`fake-ip-tun.md`)、**爆破手册**(`bruteforce-playbook.md`) |
223
- | `scripts/` | 183KB ≈ 57K token | 不预读,用哪个看哪个(`--help`) | 工具库(13 个):快速侦察(`fastrecon`:DNS/DoT/DoH、子域、多域名、端口 cam、HTTP 兜底)、端口扫描(`portscan`:自动绕 fake-IP)、目录爆破、Basic 爆破、JS 提取、CVE、依赖扫描、变异(mutate)、牌组(deck)、Web 探测、DoH 解析(`doh_resolve`)、源站溯源(`origin_trace`:DoH+CT+指纹)、工具盘点(`tool_inventory`);共享库 `platf`/`argsmini`/`netdns`(fake-IP+DoH) |
223
+ | `scripts/` | 190KB ≈ 59K token | 不预读,用哪个看哪个(`--help`) | 工具库(13 个):快速侦察(`fastrecon`:DNS/DoT/DoH、子域、多域名、端口 cam、HTTP 兜底)、端口扫描(`portscan`:自动绕 fake-IP)、目录爆破、Basic 爆破、JS 提取、CVE、依赖扫描、变异(mutate)、牌组(deck)、Web 探测、DoH 解析(`doh_resolve`)、源站溯源(`origin_trace`:DoH+多源CT/被动DNS+指纹)、工具盘点(`tool_inventory`);共享库 `platf`/`argsmini`/`netdns`(fake-IP+DoH) |
224
224
  | `pi/` + `opencode/` + `claude/` + `grok/` | 3KB | 不读 | 注入模板:Pi 三层(`pi/AGENTS.md` L1 + `APPEND_SYSTEM.md`(←SYSTEM_PROMPT.md) L2 + `pi/prompts/ck.md` L3)+ OpenCode(`agent-bypass.md`+`instructions`+`commands/ck.md`)+ Claude Code(`CLAUDE.md`←`claude/AGENTS.md`+`commands/ck.md`)+ Grok(`AGENTS.md`+`commands/ck.md`) |
225
225
  | `dsh/` | 53KB | 不读 | DeepSeek Harness 集成:`persona.md`(Ck 万能执行员人格)+ `preset.template.yml`(官方 standard 预设派生,插件/工具行与官方一致)+ `build-template.js`(DSH 升级后重建模板;macOS/Windows/Linux 自动定位 app.asar) |
226
226
 
@@ -873,7 +873,7 @@ FOFA 语法与手法(其余引擎语法近似):
873
873
  8. 死路判定:Cloudflare Tunnel/Workers/Pages 作源(originless)→ 放弃找 IP,转打未关的 *.workers.dev/*.pages.dev 或应用层。
874
874
  9. 拿到源站后的打法:直连+受害者 Host/SNI(很多源站不校验来源 IP)→ 绕 WAF/限速/地域限制;经自建同款 CDN zone 代理回源(用 CDN 出口打源站,绕"仅允许 CDN"白名单与共享证书校验);Host 碰撞探测虚拟主机;直连后重跑全套打法(module 19/26),放开并发但仍留证。
875
875
 
876
- - **fake-IP/TUN 下的溯源**:系统 DNS 被劫持时历史解析/子域全被污染——统一走 DoH 真 IP;`scripts/origin_trace.py` 一键跑「DoH 根 IP + crt.sh 证书透明度 + 漏质子域 + 直连指纹比对(标题/Server/CDN 头/证书 SAN)」,输出候选源站表(`--json` 可机读);验证必须 `curl --resolve`/直连 IP,别信系统解析;细节见 `references/fake-ip-tun.md`。
876
+ - **fake-IP/TUN 下的溯源**:系统 DNS 被劫持时历史解析/子域全被污染——统一走 DoH 真 IP;`scripts/origin_trace.py` 一键跑「DoH 根 IP + **多源 CT/被动 DNS 并发兜底**(crt.sh / CertSpotter / OTX / HackerTarget / Anubis / RapidDNS,单源 502 不影响其他)+ 漏质子域 + 被动 DNS IP + 直连指纹比对(标题/Server/CDN 头/证书 SAN)」,`--ct-sources`/`--max-ct`/`--ct-budget`/`--no-ct` 可调,输出候选源站表(`--json`);验证必须 `curl --resolve`/直连 IP,别信系统解析;细节见 `references/fake-ip-tun.md`。
877
877
 
878
878
  纪律:候选 IP 全部落表(来源技术/证据/置信分),多技术交叉一致才算候选,单技术命中列线索;报告附检索语法与时间戳。
879
879
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agent-bypass",
3
- "version": "2.17.0",
3
+ "version": "2.18.0",
4
4
  "description": "Cknb 全能技术操作员 — Agent Skills 技能包(默认安装 Pi / OpenCode / DeepSeek Harness;Claude Code / Grok / OpenMinis 显式指定)",
5
5
  "bin": {
6
6
  "agent-bypass": "bin/install.js"
@@ -74,6 +74,7 @@ fake-IP 常见段(命中任一即不可信):
74
74
  ## 6. 溯源注意(源站 / 历史解析)
75
75
 
76
76
  - 历史 DNS、子域、CT 查询统一走 **DoH**(`scripts/origin_trace.py` 已内置),不要信系统解析;
77
+ - CT/被动 DNS 用多源并发兜底:crt.sh / CertSpotter / OTX / HackerTarget / Anubis / RapidDNS(`--ct-sources all` 默认全开;部分源 502/限速会自动跳过,`--no-ct` 可完全关闭);
77
78
  - 测绘平台(FOFA/Hunter/Shodan)返回的 IP 先排除 CDN 段/回源特征,再用 `curl --resolve` 直连验证;
78
79
  - 直连验证看:标题一致、Server 头、无 CDN 头(cf-ray/x-cache/via 等)、证书 SAN 匹配;
79
80
  - TUN 环境下验证请求也要 `--resolve` 到候选 IP,否则 curl 又被 fake-IP 带偏;
@@ -12,6 +12,8 @@
12
12
  python3 origin_trace.py example.com --json
13
13
  python3 origin_trace.py example.com --max-candidates 60 --timeout 8
14
14
  python3 origin_trace.py example.com --no-ct --max-candidates 8 --budget 60 # iSH/fake-IP:跳过 CT、限时防卡
15
+ python3 origin_trace.py example.com --ct-sources crt.sh,certspotter,otx # 指定/兜底 CT 源(all=全部)
16
+ python3 origin_trace.py example.com --max-ct 120 --ct-budget 20 # CT 主机上限/多源总预算
15
17
  DOH_PROVIDERS="https://1.1.1.1/dns-query" python3 origin_trace.py example.com
16
18
 
17
19
  输出: 候选源站表(ip / 分数 / 来源 / 标题 / Server / CDN 头),--json 输出机器可读结构。
@@ -20,6 +22,7 @@
20
22
  import concurrent.futures
21
23
  import hashlib
22
24
  import json
25
+ import os
23
26
  import re
24
27
  import socket
25
28
  import ssl
@@ -41,6 +44,11 @@ SUBS = [
41
44
  "legacy", "backup", "admin", "api", "gateway", "vpn", "oa", "portal",
42
45
  "m", "wap", "static", "img", "cdn", "assets", "uploads",
43
46
  ]
47
+ ISH = os.path.isdir("/var/minis") or bool(os.environ.get("MINIS_ROOT"))
48
+
49
+ # 多源证书透明度 / 被动 DNS(无 Key 优先;单个源 502/超时不影响其他源)
50
+ CT_SOURCES = ("crt.sh", "certspotter", "otx", "hackertarget", "anubis", "rapiddns")
51
+
44
52
  CDN_HEADERS = (
45
53
  "cf-ray", "cf-cache-status", "x-cache", "via", "x-amz-cf-id", "x-amz-cf-pop",
46
54
  "x-fastly-request-id", "x-akamai-transformed", "x-cdn", "x-served-by",
@@ -72,23 +80,35 @@ def doh_resolve(name, timeout=6.0):
72
80
  return []
73
81
 
74
82
 
75
- def fetch_ct_names(domain, timeout=8.0):
76
- """证书透明度(crt.sh)里的 DNS 名;返回去重主机名列表。
77
- crt.sh 偶发 502/超时:重试一次,仍失败则跳过(不阻塞主流程)。"""
78
- url = "https://crt.sh/?q=" + urllib.parse.quote("%." + domain, safe="") + "&output=json"
83
+ def _host_in_domain(name, domain):
84
+ """规范化主机名并校验属于目标域;不合法返回 None。"""
85
+ name = str(name or "").strip().lower().lstrip("*.").rstrip(".")
86
+ if name and (name == domain or name.endswith("." + domain)):
87
+ return name
88
+ return None
89
+
90
+
91
+ def _dedup_hosts(cands, domain, seen, out):
92
+ for raw in cands:
93
+ n = _host_in_domain(raw, domain)
94
+ if n and n not in seen:
95
+ seen.add(n)
96
+ out.append(n)
97
+
98
+
99
+ def http_get_text(url, timeout=8.0, retry=1):
100
+ """GET 文本(带 UA + 重试);失败抛异常由调用方兜底。"""
79
101
  last = None
80
- for attempt in (1, 2):
102
+ for attempt in range(retry + 1):
81
103
  try:
82
104
  req = urllib.request.Request(url, headers={"user-agent": UA})
83
105
  with urllib.request.urlopen(req, timeout=timeout) as r:
84
- text = r.read(8 << 20).decode("utf-8", "ignore")
85
- return parse_crtsh(text, domain)
106
+ return r.read(8 << 20).decode("utf-8", "ignore")
86
107
  except Exception as e:
87
108
  last = e
88
- if attempt == 1:
89
- time.sleep(1.0)
90
- _err("[crt.sh] 查询失败(已重试): %s —— 跳过 CT,继续漏质子域" % last)
91
- return []
109
+ if attempt < retry:
110
+ time.sleep(0.8)
111
+ raise last
92
112
 
93
113
 
94
114
  def parse_crtsh(text, domain):
@@ -101,18 +121,141 @@ def parse_crtsh(text, domain):
101
121
  return []
102
122
  out, seen = [], set()
103
123
  for row in rows:
124
+ if isinstance(row, dict):
125
+ _dedup_hosts(str(row.get("name_value") or "").split("\n"), domain, seen, out)
126
+ return out
127
+
128
+
129
+ def parse_certspotter(text, domain):
130
+ """CertSpotter issuances JSON → dns_names 里的主机名。"""
131
+ try:
132
+ rows = json.loads(text)
133
+ except (ValueError, TypeError):
134
+ return []
135
+ out, seen = [], set()
136
+ for row in rows if isinstance(rows, list) else []:
137
+ if isinstance(row, dict):
138
+ _dedup_hosts(row.get("dns_names") or [], domain, seen, out)
139
+ return out
140
+
141
+
142
+ def parse_otx(text, domain):
143
+ """OTX 被动 DNS → (主机名列表, IP 列表)。"""
144
+ try:
145
+ d = json.loads(text)
146
+ except (ValueError, TypeError):
147
+ return [], []
148
+ rows = d.get("passive_dns") if isinstance(d, dict) else None
149
+ out, seen, ips = [], set(), []
150
+ for row in rows if isinstance(rows, list) else []:
104
151
  if not isinstance(row, dict):
105
152
  continue
106
- for raw in str(row.get("name_value") or "").split("\n"):
107
- name = raw.strip().lower().lstrip("*.").rstrip(".")
108
- if not name or name in seen:
109
- continue
110
- if name == domain or name.endswith("." + domain):
111
- seen.add(name)
112
- out.append(name)
153
+ host = _host_in_domain(row.get("hostname") or "", domain)
154
+ if not host:
155
+ continue # 只收目标域相关记录,避免无关 IP 污染候选
156
+ _dedup_hosts([host], domain, seen, out)
157
+ ip = str(row.get("address") or "").strip()
158
+ if netdns is not None and netdns.is_ipv4(ip) and not netdns.is_fake_ip(ip) and ip not in ips:
159
+ ips.append(ip)
160
+ return out, ips
161
+
162
+
163
+ def parse_hackertarget(text, domain):
164
+ """HackerTarget hostsearch CSV(host,ip)→ (主机名, IP)。"""
165
+ out, seen, ips = [], set(), []
166
+ for line in str(text or "").splitlines():
167
+ parts = [x.strip() for x in line.split(",")]
168
+ if len(parts) != 2:
169
+ continue
170
+ host = _host_in_domain(parts[0], domain)
171
+ if not host:
172
+ continue # 只收目标域相关主机/IP
173
+ _dedup_hosts([host], domain, seen, out)
174
+ if netdns is not None and netdns.is_ipv4(parts[1]) and not netdns.is_fake_ip(parts[1]) and parts[1] not in ips:
175
+ ips.append(parts[1])
176
+ return out, ips
177
+
178
+
179
+ def parse_anubis(text, domain):
180
+ """Anubis(jldc.me) 子域 JSON 数组 → 主机名。"""
181
+ try:
182
+ rows = json.loads(text)
183
+ except (ValueError, TypeError):
184
+ return []
185
+ out, seen = [], set()
186
+ _dedup_hosts(rows if isinstance(rows, list) else [], domain, seen, out)
113
187
  return out
114
188
 
115
189
 
190
+ def parse_rapiddns(text, domain):
191
+ """RapidDNS HTML → 页面中的子域文本(宽松匹配)。"""
192
+ out, seen = [], set()
193
+ for m in re.finditer(r"([a-zA-Z0-9][a-zA-Z0-9._-]*\." + re.escape(domain) + r")", str(text or "")):
194
+ _dedup_hosts([m.group(1)], domain, seen, out)
195
+ return out
196
+
197
+
198
+ def _ct_fetch(name, domain, timeout):
199
+ """单个 CT/被动 DNS 源 → (hosts, ips)。"""
200
+ q = urllib.parse.quote(domain, safe="")
201
+ if name == "crt.sh":
202
+ text = http_get_text("https://crt.sh/?q=" + urllib.parse.quote("%." + domain, safe="") + "&output=json",
203
+ timeout, retry=1)
204
+ return parse_crtsh(text, domain), []
205
+ if name == "certspotter":
206
+ text = http_get_text("https://api.certspotter.com/v1/issuances?domain=%s&include_subdomains=true&expand=dns_names" % q,
207
+ timeout)
208
+ return parse_certspotter(text, domain), []
209
+ if name == "otx":
210
+ text = http_get_text("https://otx.alienvault.com/api/v1/indicators/domain/%s/passive_dns" % q, timeout)
211
+ return parse_otx(text, domain)
212
+ if name == "hackertarget":
213
+ text = http_get_text("https://api.hackertarget.com/hostsearch/?q=%s" % q, timeout)
214
+ return parse_hackertarget(text, domain)
215
+ if name == "anubis":
216
+ text = http_get_text("https://jldc.me/anubis/subdomains/%s" % q, timeout)
217
+ return parse_anubis(text, domain), []
218
+ if name == "rapiddns":
219
+ text = http_get_text("https://rapiddns.io/subdomain/%s?full=1" % q, timeout)
220
+ return parse_rapiddns(text, domain), []
221
+ return [], []
222
+
223
+
224
+ def fetch_passive_names(domain, timeout=8.0, sources=None, max_hosts=80, budget=None):
225
+ """多源并发查询 CT/被动 DNS:单源失败不影响其他;总预算到期返回部分结果。
226
+ 返回 (hosts, passive_ips, per_source_status)。"""
227
+ sources = list(sources or CT_SOURCES)
228
+ budget = budget if budget and budget > 0 else (10.0 if ISH else 15.0)
229
+ workers = min(4 if ISH else 6, max(1, len(sources)))
230
+ hosts, seen, ips, status = [], set(), [], {}
231
+ with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as ex:
232
+ futs = {ex.submit(_ct_fetch, name, domain, timeout): name for name in sources}
233
+ try:
234
+ for fut in concurrent.futures.as_completed(futs, timeout=budget):
235
+ src = futs[fut]
236
+ try:
237
+ h, ip = fut.result()
238
+ except Exception as e:
239
+ status[src] = "失败: %s" % str(e)[:60]
240
+ _err("[CT] %-12s 失败,跳过: %s" % (src, str(e)[:80]))
241
+ continue
242
+ status[src] = len(h) + (len(ip) and len(ip) or 0) * 0 # 仅主机数用于展示
243
+ for n in h:
244
+ if n not in seen and len(hosts) < max_hosts:
245
+ seen.add(n)
246
+ hosts.append(n)
247
+ for ipv in ip:
248
+ if ipv not in ips:
249
+ ips.append(ipv)
250
+ _err("[CT] %-12s %d 主机%s" % (src, len(h), (" / %d IP" % len(ip)) if ip else ""))
251
+ except concurrent.futures.TimeoutError:
252
+ status["_budget"] = "总预算 %.0fs 超时(返回部分)" % budget
253
+ _err("[CT] 总预算 %.0fs 到期,使用已返回的 %d 个主机" % (budget, len(hosts)))
254
+ finally:
255
+ ex.shutdown(wait=False, cancel_futures=True)
256
+ return hosts, ips, status
257
+
258
+
116
259
  def cdn_markers(headers):
117
260
  return [h for h in CDN_HEADERS if h in headers]
118
261
 
@@ -213,14 +356,19 @@ def score_candidate(base, cand):
213
356
  return score, reasons
214
357
 
215
358
 
216
- def collect_candidates(domain, max_candidates=40, timeout=8.0, use_ct=True):
217
- """收集候选 (name, ip, sources)。"""
359
+ def collect_candidates(domain, max_candidates=40, timeout=8.0, use_ct=True,
360
+ ct_sources=None, max_ct=80, ct_budget=0.0):
361
+ """收集候选 (name, ip, sources):漏质子域 + 多源 CT/被动 DNS。"""
218
362
  names = ["www." + domain, domain] + ["%s.%s" % (s, domain) for s in SUBS]
363
+ passive_ips = []
219
364
  if use_ct:
220
- ct = fetch_ct_names(domain, timeout)
365
+ ct, passive_ips, _status = fetch_passive_names(
366
+ domain, timeout, sources=ct_sources, max_hosts=max_ct, budget=ct_budget)
221
367
  if ct:
222
- _err("[CT] crt.sh 返回 %d 个主机名" % len(ct))
368
+ _err("[CT] 合计 %d 个主机名" % len(ct))
223
369
  names.extend(ct)
370
+ if passive_ips:
371
+ _err("[被动DNS] 合计 %d 个 IP 并入候选" % len(passive_ips))
224
372
  uniq, seen = [], set()
225
373
  for n in names:
226
374
  n = n.strip().lower().rstrip(".")
@@ -234,11 +382,13 @@ def collect_candidates(domain, max_candidates=40, timeout=8.0, use_ct=True):
234
382
  return name, doh_resolve(name, timeout)
235
383
 
236
384
  by_ip = {}
237
- with concurrent.futures.ThreadPoolExecutor(max_workers=8) as ex:
385
+ with concurrent.futures.ThreadPoolExecutor(max_workers=(4 if ISH else 8)) as ex:
238
386
  for name, pairs in ex.map(one, uniq):
239
387
  for ip, src in pairs:
240
388
  srcs = by_ip.setdefault(ip, {"ips": set(), "sources": set()})
241
389
  srcs["sources"].add(name)
390
+ for ip in passive_ips:
391
+ by_ip.setdefault(ip, {"ips": set(), "sources": set()})["sources"].add("passive-dns")
242
392
  return by_ip
243
393
 
244
394
 
@@ -246,6 +396,7 @@ def main(argv=None):
246
396
  args = list(sys.argv[1:] if argv is None else argv)
247
397
  domains, as_json, no_verify, no_ct = [], False, False, False
248
398
  max_c, timeout, budget = 40, 8.0, 120.0
399
+ ct_sources, max_ct, ct_budget = None, 80, 0.0
249
400
  i = 0
250
401
  while i < len(args):
251
402
  a = args[i]
@@ -255,7 +406,7 @@ def main(argv=None):
255
406
  no_verify = True
256
407
  elif a == "--no-ct":
257
408
  no_ct = True
258
- elif a in ("--max-candidates", "--timeout", "--budget"):
409
+ elif a in ("--max-candidates", "--timeout", "--budget", "--max-ct", "--ct-budget", "--ct-sources"):
259
410
  if i + 1 >= len(args):
260
411
  _err("%s 缺少取值" % a)
261
412
  return 2
@@ -264,6 +415,20 @@ def main(argv=None):
264
415
  max_c = max(1, int(args[i + 1]))
265
416
  elif a == "--budget":
266
417
  budget = max(10.0, float(args[i + 1]))
418
+ elif a == "--max-ct":
419
+ max_ct = max(1, int(args[i + 1]))
420
+ elif a == "--ct-budget":
421
+ ct_budget = max(3.0, float(args[i + 1]))
422
+ elif a == "--ct-sources":
423
+ val = args[i + 1].strip()
424
+ if val.lower() == "all":
425
+ ct_sources = None
426
+ else:
427
+ picked = [x.strip() for x in val.split(",") if x.strip()]
428
+ if not picked or any(x not in CT_SOURCES for x in picked):
429
+ _err("--ct-sources 取值非法(可选: %s 或 all)" % ",".join(CT_SOURCES))
430
+ return 2
431
+ ct_sources = picked
267
432
  else:
268
433
  timeout = max(1.0, min(float(args[i + 1]), 30.0))
269
434
  except (TypeError, ValueError):
@@ -301,7 +466,8 @@ def main(argv=None):
301
466
  except Exception as e:
302
467
  _err("[基线] 探测失败: %s(继续)" % e)
303
468
 
304
- by_ip = collect_candidates(domain, max_c, timeout, use_ct=not no_ct)
469
+ by_ip = collect_candidates(domain, max_c, timeout, use_ct=not no_ct,
470
+ ct_sources=ct_sources, max_ct=max_ct, ct_budget=ct_budget)
305
471
  for ip in real_root: # 根 IP 纳入候选
306
472
  by_ip.setdefault(ip, {"sources": set()})["sources"].add(domain)
307
473
 
@@ -355,6 +355,40 @@ class HangProtectionTests(unittest.TestCase):
355
355
  self.assertIn("same-cdn", r_cdn)
356
356
  self.assertLess(s_cdn, s_org)
357
357
 
358
+ def test_passive_ct_parsers(self):
359
+ cs = '[{"dns_names":["www.example.com","*.example.com","x.test"]}]'
360
+ self.assertEqual(self.ot.parse_certspotter(cs, "example.com"), ["www.example.com", "example.com"])
361
+ otx = ('{"passive_dns":[{"hostname":"mail.example.com","address":"1.2.3.4"},'
362
+ '{"hostname":"x.test","address":"9.9.9.9"},'
363
+ '{"hostname":"cdn.example.com","address":"198.18.0.9"}]}')
364
+ hosts, ips = self.ot.parse_otx(otx, "example.com")
365
+ self.assertIn("mail.example.com", hosts)
366
+ self.assertEqual(ips, ["1.2.3.4"], "被动 DNS 的 fake-IP 应被过滤")
367
+ hosts, ips = self.ot.parse_hackertarget("vpn.example.com,5.6.7.8\nAPI count exceeded", "example.com")
368
+ self.assertEqual(hosts, ["vpn.example.com"])
369
+ self.assertEqual(ips, ["5.6.7.8"])
370
+ self.assertEqual(self.ot.parse_anubis('["a.example.com","b.test"]', "example.com"), ["a.example.com"])
371
+ self.assertEqual(self.ot.parse_rapiddns("<td>oa.example.com</td><td>x.test</td>", "example.com"),
372
+ ["oa.example.com"])
373
+ for src in ("crt.sh", "certspotter", "otx", "hackertarget", "anubis", "rapiddns"):
374
+ self.assertIn(src, self.ot.CT_SOURCES)
375
+
376
+ def test_ct_multi_source_fallback(self):
377
+ def fake(name, domain, timeout):
378
+ if name == "crt.sh":
379
+ raise RuntimeError("502 Bad Gateway")
380
+ return (["s1.example.com", "s2.example.com"], ["1.2.3.4"])
381
+ orig = self.ot._ct_fetch
382
+ self.ot._ct_fetch = fake
383
+ try:
384
+ hosts, ips, status = self.ot.fetch_passive_names(
385
+ "example.com", sources=["crt.sh", "otx"], budget=5)
386
+ finally:
387
+ self.ot._ct_fetch = orig
388
+ self.assertEqual(hosts, ["s1.example.com", "s2.example.com"])
389
+ self.assertEqual(ips, ["1.2.3.4"])
390
+ self.assertIn("失败", status["crt.sh"], "单源 502 应被记录并跳过")
391
+
358
392
  def test_origin_bad_flag_rejected(self):
359
393
  r = _tmp_run([sys.executable, str(SCRIPTS / "origin_trace.py"), "--bogus"])
360
394
  self.assertEqual(r.returncode, 2)