phantomprobe 0.9.1__tar.gz → 0.9.3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (53) hide show
  1. phantomprobe-0.9.3/CHANGELOG.md +203 -0
  2. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/PKG-INFO +36 -11
  3. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/README.md +35 -10
  4. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/scripts/audit_cpe_mapping.py +14 -1
  5. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/active.py +191 -99
  6. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/cli.py +34 -7
  7. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/constants.py +1 -1
  8. phantomprobe-0.9.3/src/phantomprobe/cpe_tables.py +285 -0
  9. phantomprobe-0.9.3/src/phantomprobe/cve.py +413 -0
  10. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/dashboard.py +13 -1
  11. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/js.py +29 -0
  12. phantomprobe-0.9.3/src/phantomprobe/nvd.py +256 -0
  13. phantomprobe-0.9.3/src/phantomprobe/ports.py +171 -0
  14. phantomprobe-0.9.3/tests/test_active.py +350 -0
  15. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/test_audit_script.py +26 -1
  16. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/test_cli.py +61 -0
  17. phantomprobe-0.9.3/tests/test_cve.py +1049 -0
  18. phantomprobe-0.9.3/tests/test_ports.py +253 -0
  19. phantomprobe-0.9.1/CHANGELOG.md +0 -94
  20. phantomprobe-0.9.1/src/phantomprobe/cve.py +0 -622
  21. phantomprobe-0.9.1/tests/test_active.py +0 -107
  22. phantomprobe-0.9.1/tests/test_cve.py +0 -496
  23. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/.gitignore +0 -0
  24. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/CONTRIBUTING.md +0 -0
  25. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/LICENSE +0 -0
  26. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/pyproject.toml +0 -0
  27. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/__init__.py +0 -0
  28. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/__main__.py +0 -0
  29. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/aggressive.py +0 -0
  30. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/asgi.py +0 -0
  31. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/burp.py +0 -0
  32. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/cookies.py +0 -0
  33. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/dns_security.py +0 -0
  34. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/doh.py +0 -0
  35. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/http_checks.py +0 -0
  36. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/http_client.py +0 -0
  37. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/models.py +0 -0
  38. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/passive.py +0 -0
  39. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/report.py +0 -0
  40. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/screenshot.py +0 -0
  41. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/takeover.py +0 -0
  42. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/src/phantomprobe/waf.py +0 -0
  43. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/__init__.py +0 -0
  44. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/conftest.py +0 -0
  45. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/test_aggressive.py +0 -0
  46. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/test_burp.py +0 -0
  47. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/test_cookies.py +0 -0
  48. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/test_dashboard.py +0 -0
  49. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/test_dns_security.py +0 -0
  50. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/test_http_checks.py +0 -0
  51. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/test_http_client.py +0 -0
  52. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/test_takeover.py +0 -0
  53. {phantomprobe-0.9.1 → phantomprobe-0.9.3}/tests/test_waf.py +0 -0
@@ -0,0 +1,203 @@
1
+ # Changelog
2
+
3
+ All notable changes to PhantomProbe are recorded here. The format follows
4
+ [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project
5
+ follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
+
7
+ ## [0.9.3] - 2026-11-17
8
+
9
+ Each entry below was found by checking the scanner against a known answer, and
10
+ confirmed afterwards against live NVD, CISA KEV and real DNS.
11
+
12
+ ### Added
13
+
14
+ - **Service banners in phase 2.** Where a service speaks first (FTP, SSH,
15
+ SMTP, POP3, IMAP and the MySQL handshake), the port scan now records what it
16
+ sends on connect, and `--cve` correlates the version in it. Nothing is sent
17
+ to draw the banner out. The CPE table already listed OpenSSH, ProFTPD,
18
+ vsftpd, Exim, MySQL and MariaDB, but a scan only recorded that their ports
19
+ were open, so those entries could never match. A port number alone still
20
+ names no product: "Port 3306/MySQL" without a banner is not looked up.
21
+ - **A caution on distribution builds.** A distribution fixes a vulnerability
22
+ in the release it ships and keeps the upstream version number, so a version
23
+ match can be a patched system. When the banner a version was read from names
24
+ a distribution, every CVE matched on it says so in the Markdown and JSON
25
+ reports, the console summary and the dashboard, and the CVE section of the
26
+ report opens by stating that a version match is a lead to confirm.
27
+
28
+ ### Fixed
29
+
30
+ - **A lettered version lost its letter.** OpenSSL 1.1.1w and ProFTPD 1.3.5e
31
+ were looked up as 1.1.1 and 1.3.5, the oldest release of each line. NVD
32
+ compares the letter: it returned 63 CVEs for OpenSSL 1.1.1 and 33 for
33
+ 1.1.1w, so fixed vulnerabilities were reported as open. A single trailing
34
+ letter is now kept; the `p1` of `OpenSSH_8.9p1` is still left out.
35
+
36
+ - **Wildcard DNS filled the report with subdomains that do not exist.** Behind
37
+ a wildcard every name resolves, so one real scan reported all 198 wordlist
38
+ names, nine tenths of its findings, although phase 1 had flagged the
39
+ wildcard. Enumeration now learns the wildcard's answer first and lists only
40
+ names that answer differently, with one note accounting for the rest.
41
+ - **CVE correlation missed most current nginx CVEs.** `f5:nginx` returns 41
42
+ results and passed the audit, but NVD files the newer nginx CVEs under
43
+ `f5:nginx_open_source`. For nginx 1.24.0 the scanner saw 2 CVEs of 17, and
44
+ showed 1 of the 10 scored 7.0 or higher. A technology can now carry further
45
+ identifiers (`CPE_SIBLINGS`), queried together and merged.
46
+ - **The same gap affected 18 more products.** 42 candidate second identifiers
47
+ were looked up in NVD and 36 exist. 32 of those were asked about a real
48
+ version alongside the identifier in the table, and 25 returned CVEs the
49
+ table's identifier did not; they are now queried too, across 19 products.
50
+ For five products the table's identifier returned nothing at all, which
51
+ looks exactly like a clean target: Joomla 3.9.0 (125 CVEs under
52
+ `joomla:joomla\!`), Docker 20.10.5 (16), Confluence 7.13.0 (14), RabbitMQ
53
+ 3.8.9 (8) and IIS 7.5 (3). MySQL 8.0.30 had 147 CVEs under `oracle:mysql`
54
+ and a further 155 under `oracle:mysql_server`, and Magento 2.4.3 had 8
55
+ against 126 under `adobe:magento`. Each addition carries its measurement in
56
+ `cpe_tables.py`, with the identifiers that were measured and left out listed
57
+ beside them. The measurements themselves, with the scripts that produced
58
+ them, are in `docs/research/nvd-identifier-audit-2026-11-16/`.
59
+ - **An exploited CVE scoring under 7.0 was dropped before KEV was consulted.**
60
+ The severity cut ran first, so the promise that an exploited CVE outranks a
61
+ dormant one could not hold for CVE-2020-11023 (jQuery, scored 6.1 to 6.9, in
62
+ KEV since 2025). Exploitation data is now fetched before the cut, and a CVE in
63
+ KEV is kept at any score.
64
+ - **NVD results were read as one short page.** The client took the first 20 and
65
+ stopped, and NVD sorts oldest first: for `f5:nginx` that page ran from 2009
66
+ to 2014. Pages are now followed to the end, and a cut at the cap is announced.
67
+ - **Client-side library versions never reached CVE matching.** Script URLs were
68
+ not recorded as findings, and JavaScript analysis ran after correlation.
69
+ Versioned script assets are now reported and analysis runs first. A script URL
70
+ is read for client-side libraries only, so `spring-1.2.0.js` is not taken for
71
+ the Spring Framework.
72
+ - **A busy NVD made a product look clean.** A 503, a 429 or a timeout returned
73
+ nothing for that product. They are now retried, and a timeout while reading
74
+ the response body no longer ends the scan.
75
+ - **Technology fingerprinting reported words, not technologies.** Signatures
76
+ were substrings searched across the headers and the page text together, so
77
+ "java" matched any page containing "javascript", ".net" any link to a .net
78
+ domain, and "passenger" a travel site. One real scan credited a WordPress
79
+ site with Java and Node.js. Each signature now names the header, cookie or
80
+ markup it may be found in and matches whole words, and each finding states
81
+ the signal that identified it. Cookie values and request ids are kept out of
82
+ the evidence.
83
+ - **The full Docker image stopped building.** The `playwright` dependency has
84
+ no upper bound and resolved to 1.65, whose bundled Node driver needs
85
+ `libatomic`. The slim base image does not ship it, so `playwright install`
86
+ could not start. The full image now installs `libatomic1`.
87
+ - **CI could not run Python 3.8 or 3.9.** `ubuntu-latest` moved to Ubuntu
88
+ 26.04, which has no build of either version, so both jobs failed before any
89
+ test ran. The test job is pinned to `ubuntu-24.04`.
90
+
91
+ ### Changed
92
+
93
+ - The CPE audit checks sibling identifiers alongside their technology, and its
94
+ documentation states what a passing count does and does not prove.
95
+ - `cve.py` is split by responsibility into `cve.py` (correlation), `nvd.py`
96
+ (the NVD client) and `cpe_tables.py` (the identifier tables), each under 500
97
+ lines. `CVEMatcher` keeps its interface.
98
+ - IIS is reported as `iis` rather than folded into `asp.net`, and a WordPress
99
+ generator tag carries its version through to CVE correlation.
100
+
101
+ ## [0.9.2] - 2026-09-11
102
+
103
+ ### Fixed
104
+
105
+ - **A skipped dashboard was silent.** On a plain `pip install phantomprobe`
106
+ without the `[dashboard]` extra, `--dashboard` correctly skipped the web UI,
107
+ but the only notice printed once at the start and scrolled away during the
108
+ scan, so the run looked like it did nothing. The scan always wrote its reports
109
+ to the current directory, where a user expecting a dashboard would not look.
110
+ The end-of-run summary now always states where the reports were written, and
111
+ says plainly when `--dashboard` was requested without its dependencies, with
112
+ the one command that installs them.
113
+
114
+ ## [0.9.1] - 2026-09-04
115
+
116
+ ### Fixed
117
+
118
+ - **The CI was red on every run since the 0.9.0 dashboard redesign.** The docker
119
+ job's reachability check grepped the served page for a string the redesign
120
+ removed; it now matches a stable section heading. Local pytest stayed green,
121
+ which is why it went unnoticed.
122
+ - **The release workflow could swallow itself**: PyPI publish ran first and was
123
+ mandatory, so a missing token failed the whole run and no GitHub Release was
124
+ created. The Release is now created first and always; the PyPI publish is last
125
+ and runs only when a token is configured.
126
+
127
+ ### Changed
128
+
129
+ - The README leads with `pip install phantomprobe` now that the package is on
130
+ PyPI, and carries a PyPI version badge.
131
+
132
+ ## [0.9.0] - 2026-09-03
133
+
134
+ The first packaged release. It consolidates the earlier single-file prototypes
135
+ into a modular, passive-first scanner, so several entries below read as fixes
136
+ against that prototype. Every data table and severity calibration was checked
137
+ against its live source before shipping.
138
+
139
+ ### Added
140
+
141
+ - **Subdomain takeover detection** (`--phase2`). A two-signal check: the CNAME
142
+ must point at a known service *and* the service must report the resource
143
+ unclaimed, using the can-i-take-over-xyz fingerprints. Resolves CNAMEs over
144
+ DNS-over-HTTPS so the core stays dependency-free.
145
+ - **Email security** over DoH: SPF, DMARC and DKIM, with `+all` treated as
146
+ worse than a missing record, and DKIM reporting only the selectors it finds.
147
+ - **CAA and DNSSEC** checks, reported as hardening.
148
+ - **Cookie security** analysis (`Secure`, `HttpOnly`, `SameSite`), weighted by
149
+ whether a cookie looks like a session or a tracker.
150
+ - **WAF / CDN fingerprinting** of 35 WAFs from the wafw00f signature set.
151
+ - **HSTS preload eligibility**, judged against hstspreload.org's current
152
+ one-year `max-age` floor; **redirect-chain** analysis; and **security.txt**
153
+ discovery.
154
+ - **CVE exploitation intelligence**: every match is enriched with CISA KEV
155
+ (exploited in the wild, with a ransomware marker) and EPSS (probability of
156
+ exploitation), and an exploited CVE ranks above a higher-scored dormant one.
157
+ - **Phase 3 active vulnerability probing** (`--aggressive`, opt-in): CORS, open
158
+ redirect, HTTP parameter pollution and S3 bucket exposure. Non-destructive;
159
+ request smuggling deliberately excluded.
160
+ - **CPE audit script** (`scripts/audit_cpe_mapping.py`) to re-check the CVE
161
+ table against live NVD, and a **CI job** that builds and exercises every
162
+ Docker stage.
163
+ - **`requirements-dev.txt`** and a `--no-takeover` flag.
164
+
165
+ ### Changed
166
+
167
+ - **Restructured** the 2270-line single scanner file into a proper
168
+ `src/phantomprobe/` package of focused modules, each under 500 lines.
169
+ - **Redesigned the dashboard**: a dense, calibrated dark theme that scores zero
170
+ on the impeccable design detector, down from 41 anti-patterns.
171
+ - **Expanded the CVE CPE table** from 21 to 74 products, each vendor/product
172
+ pair verified against NVD before being added (nginx corrected to `f5`, express
173
+ to `openjsf`).
174
+ - **Version is single-sourced** from `constants.py` via hatchling, so
175
+ distribution metadata cannot drift from the running code.
176
+ - Trimmed the `burp` extra to `requests` alone, and rewrote the README.
177
+
178
+ ### Fixed
179
+
180
+ - **Packaging and tests were broken**: the wheel shipped only a shim reaching
181
+ the real code through a `sys.path` hack, and all five tests failed on imports.
182
+ - **Docker**: the dashboard bound to loopback inside the container (unreachable
183
+ via the published port), the `full` image could not take a screenshot
184
+ (browsers installed as root), and several compose profiles were misconfigured.
185
+ - **Console encoding** could abort a whole scan: an unencodable byte in output
186
+ raised on a legacy code page, discarding every finding.
187
+ - **Burp integration** was rewritten against the API Burp actually exposes; the
188
+ old code sent the key as a header instead of a path prefix and read fields
189
+ that do not exist.
190
+ - **CVE correlation returned nothing**: a padded CPE matched zero results in
191
+ NVD, two vendor mappings were wrong, and CVSS v4.0 scores were unread.
192
+
193
+ ### Security
194
+
195
+ - Fixed **stored XSS** in the dashboard: target- and finding-derived fields are
196
+ now HTML-escaped before rendering.
197
+ - All outbound fetches go through a helper enforcing an **http/https scheme
198
+ allowlist**, so a hostile page cannot point the scanner at `file://`.
199
+
200
+ [0.9.3]: https://github.com/Ravel226/PhantomProbe/releases/tag/v0.9.3
201
+ [0.9.2]: https://github.com/Ravel226/PhantomProbe/releases/tag/v0.9.2
202
+ [0.9.1]: https://github.com/Ravel226/PhantomProbe/releases/tag/v0.9.1
203
+ [0.9.0]: https://github.com/Ravel226/PhantomProbe/releases/tag/v0.9.0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: phantomprobe
3
- Version: 0.9.1
3
+ Version: 0.9.3
4
4
  Summary: Passive-first reconnaissance scanner for penetration testers and bug bounty hunters
5
5
  Project-URL: Homepage, https://github.com/Ravel226/PhantomProbe
6
6
  Project-URL: Repository, https://github.com/Ravel226/PhantomProbe
@@ -54,7 +54,7 @@ Description-Content-Type: text/markdown
54
54
  [![PyPI](https://img.shields.io/pypi/v/phantomprobe.svg)](https://pypi.org/project/phantomprobe/)
55
55
  [![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
56
56
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
57
- [![Tests](https://img.shields.io/badge/tests-264%20passing-brightgreen.svg)](tests/)
57
+ [![Tests](https://img.shields.io/badge/tests-377%20passing-brightgreen.svg)](tests/)
58
58
 
59
59
  PhantomProbe maps a target's attack surface and reports what is actually wrong
60
60
  with it. The core runs on the Python standard library alone, so a default
@@ -145,9 +145,15 @@ Reads what the target already sends and resolves DNS. No probing.
145
145
 
146
146
  ### Phase 2: active reconnaissance (`--phase2`)
147
147
 
148
- - **Port scan** - common service ports, concurrently
149
- - **Subdomain enumeration** - common-name discovery
150
- - **Technology fingerprinting** - server and framework detection
148
+ - **Port scan** - common service ports, concurrently. Where a service speaks
149
+ first (FTP, SSH, SMTP, POP3, IMAP and the MySQL handshake), the banner it
150
+ sends on connect is recorded. Nothing is sent to draw it out
151
+ - **Subdomain enumeration** - common-name discovery. Under wildcard DNS, where
152
+ every name resolves, only names that answer differently from the wildcard are
153
+ listed, and one note says how many were held back
154
+ - **Technology fingerprinting** - server and framework detection from response
155
+ headers, cookie names and CMS markup, each reported with the signal that
156
+ identified it
151
157
  - **Subdomain takeover** - dangling-CNAME detection against a two-signal check
152
158
  (the CNAME must point at a known service *and* the service must report the
153
159
  resource unclaimed), using the [can-i-take-over-xyz](https://github.com/EdOverflow/can-i-take-over-xyz)
@@ -165,8 +171,23 @@ and enriches every match with:
165
171
  - **CISA KEV** - whether the CVE is exploited in the wild, with a ransomware marker
166
172
  - **EPSS** - the probability of exploitation in the next 30 days
167
173
 
168
- An exploited CVE is ranked above a higher-scored dormant one. Keep the CPE table
169
- honest as vendors change hands:
174
+ An exploited CVE is ranked above a higher-scored dormant one. The report keeps
175
+ CVEs scored 7.0 or higher, plus any CVE in CISA KEV whatever its score. With
176
+ `--js`, versions carried in script URLs (`jquery-3.6.0.min.js`) are correlated
177
+ too, and with `--phase2` so are the versions in service banners
178
+ (`SSH-2.0-OpenSSH_8.9p1`).
179
+
180
+ A match is made on the version a service reports, which is a lead to confirm
181
+ and not a finding. Distributions fix vulnerabilities without changing that
182
+ version, so a match read from a banner that names one (Ubuntu, Debian, the Red
183
+ Hat family and others) is reported with that caution attached.
184
+
185
+ A pair that returns results can still be incomplete, because NVD files many
186
+ products under more than one identifier. For 19 of the 74 products a second
187
+ identifier returned CVEs the first did not: MySQL 8.0.30 has 147 under one and
188
+ 155 more under another, and Joomla 3.9.0 has none under one and 125 under the
189
+ other. Every identifier a product has is queried and the results merged. Keep
190
+ the CPE table honest as vendors change hands:
170
191
 
171
192
  ```bash
172
193
  python scripts/audit_cpe_mapping.py # re-check every mapping against NVD
@@ -192,7 +213,8 @@ dedicated tool under a scoped engagement.
192
213
  ### Other features
193
214
 
194
215
  - **JavaScript analysis** (`--js`) - extracts API endpoints, exposed secrets
195
- (API keys, tokens, AWS keys) and hidden paths from linked scripts
216
+ (API keys, tokens, AWS keys) and hidden paths from linked scripts, and lists
217
+ versioned script assets for `--cve` to check
196
218
  - **Screenshot** (`--screenshot`) - a full-page capture via headless Chromium
197
219
  - **Burp Professional** (`--burp`) - runs a Burp scan and imports its issues; see below
198
220
 
@@ -302,13 +324,16 @@ src/phantomprobe/
302
324
  ├── dns_security.py SPF, DMARC, DKIM, CAA, DNSSEC
303
325
  ├── http_checks.py HSTS preload, redirect chain, security.txt
304
326
  │
305
- ├── active.py Phase 2: ports, subdomains, fingerprinting
327
+ ├── active.py Phase 2: subdomains, fingerprinting, orchestration
328
+ ├── ports.py Port scan and service banners
306
329
  ├── takeover.py Subdomain takeover (two-signal)
307
330
  ├── waf.py WAF/CDN fingerprinting
308
331
  │
309
332
  ├── aggressive.py Phase 3: CORS, open redirect, HPP, S3 (opt-in)
310
333
  │
311
- ├── cve.py NVD correlation + KEV/EPSS enrichment
334
+ ├── cve.py CVE correlation + KEV/EPSS enrichment
335
+ ├── nvd.py NVD API client: paging, retries, record parsing
336
+ ├── cpe_tables.py What NVD calls each technology (data only)
312
337
  ├── js.py JavaScript endpoint/secret discovery
313
338
  ├── screenshot.py Playwright capture (extra: screenshot)
314
339
  ├── burp.py Burp REST integration (extra: burp)
@@ -317,7 +342,7 @@ src/phantomprobe/
317
342
  └── asgi.py Standalone ASGI entry point (extra: dashboard)
318
343
 
319
344
  scripts/audit_cpe_mapping.py Re-check the CVE table against live NVD
320
- tests/ 264 tests, network stubbed
345
+ tests/ 377 tests, network stubbed
321
346
  ```
322
347
 
323
348
  ## Development
@@ -5,7 +5,7 @@
5
5
  [![PyPI](https://img.shields.io/pypi/v/phantomprobe.svg)](https://pypi.org/project/phantomprobe/)
6
6
  [![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
7
7
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
8
- [![Tests](https://img.shields.io/badge/tests-264%20passing-brightgreen.svg)](tests/)
8
+ [![Tests](https://img.shields.io/badge/tests-377%20passing-brightgreen.svg)](tests/)
9
9
 
10
10
  PhantomProbe maps a target's attack surface and reports what is actually wrong
11
11
  with it. The core runs on the Python standard library alone, so a default
@@ -96,9 +96,15 @@ Reads what the target already sends and resolves DNS. No probing.
96
96
 
97
97
  ### Phase 2: active reconnaissance (`--phase2`)
98
98
 
99
- - **Port scan** - common service ports, concurrently
100
- - **Subdomain enumeration** - common-name discovery
101
- - **Technology fingerprinting** - server and framework detection
99
+ - **Port scan** - common service ports, concurrently. Where a service speaks
100
+ first (FTP, SSH, SMTP, POP3, IMAP and the MySQL handshake), the banner it
101
+ sends on connect is recorded. Nothing is sent to draw it out
102
+ - **Subdomain enumeration** - common-name discovery. Under wildcard DNS, where
103
+ every name resolves, only names that answer differently from the wildcard are
104
+ listed, and one note says how many were held back
105
+ - **Technology fingerprinting** - server and framework detection from response
106
+ headers, cookie names and CMS markup, each reported with the signal that
107
+ identified it
102
108
  - **Subdomain takeover** - dangling-CNAME detection against a two-signal check
103
109
  (the CNAME must point at a known service *and* the service must report the
104
110
  resource unclaimed), using the [can-i-take-over-xyz](https://github.com/EdOverflow/can-i-take-over-xyz)
@@ -116,8 +122,23 @@ and enriches every match with:
116
122
  - **CISA KEV** - whether the CVE is exploited in the wild, with a ransomware marker
117
123
  - **EPSS** - the probability of exploitation in the next 30 days
118
124
 
119
- An exploited CVE is ranked above a higher-scored dormant one. Keep the CPE table
120
- honest as vendors change hands:
125
+ An exploited CVE is ranked above a higher-scored dormant one. The report keeps
126
+ CVEs scored 7.0 or higher, plus any CVE in CISA KEV whatever its score. With
127
+ `--js`, versions carried in script URLs (`jquery-3.6.0.min.js`) are correlated
128
+ too, and with `--phase2` so are the versions in service banners
129
+ (`SSH-2.0-OpenSSH_8.9p1`).
130
+
131
+ A match is made on the version a service reports, which is a lead to confirm
132
+ and not a finding. Distributions fix vulnerabilities without changing that
133
+ version, so a match read from a banner that names one (Ubuntu, Debian, the Red
134
+ Hat family and others) is reported with that caution attached.
135
+
136
+ A pair that returns results can still be incomplete, because NVD files many
137
+ products under more than one identifier. For 19 of the 74 products a second
138
+ identifier returned CVEs the first did not: MySQL 8.0.30 has 147 under one and
139
+ 155 more under another, and Joomla 3.9.0 has none under one and 125 under the
140
+ other. Every identifier a product has is queried and the results merged. Keep
141
+ the CPE table honest as vendors change hands:
121
142
 
122
143
  ```bash
123
144
  python scripts/audit_cpe_mapping.py # re-check every mapping against NVD
@@ -143,7 +164,8 @@ dedicated tool under a scoped engagement.
143
164
  ### Other features
144
165
 
145
166
  - **JavaScript analysis** (`--js`) - extracts API endpoints, exposed secrets
146
- (API keys, tokens, AWS keys) and hidden paths from linked scripts
167
+ (API keys, tokens, AWS keys) and hidden paths from linked scripts, and lists
168
+ versioned script assets for `--cve` to check
147
169
  - **Screenshot** (`--screenshot`) - a full-page capture via headless Chromium
148
170
  - **Burp Professional** (`--burp`) - runs a Burp scan and imports its issues; see below
149
171
 
@@ -253,13 +275,16 @@ src/phantomprobe/
253
275
  ├── dns_security.py SPF, DMARC, DKIM, CAA, DNSSEC
254
276
  ├── http_checks.py HSTS preload, redirect chain, security.txt
255
277
  │
256
- ├── active.py Phase 2: ports, subdomains, fingerprinting
278
+ ├── active.py Phase 2: subdomains, fingerprinting, orchestration
279
+ ├── ports.py Port scan and service banners
257
280
  ├── takeover.py Subdomain takeover (two-signal)
258
281
  ├── waf.py WAF/CDN fingerprinting
259
282
  │
260
283
  ├── aggressive.py Phase 3: CORS, open redirect, HPP, S3 (opt-in)
261
284
  │
262
- ├── cve.py NVD correlation + KEV/EPSS enrichment
285
+ ├── cve.py CVE correlation + KEV/EPSS enrichment
286
+ ├── nvd.py NVD API client: paging, retries, record parsing
287
+ ├── cpe_tables.py What NVD calls each technology (data only)
263
288
  ├── js.py JavaScript endpoint/secret discovery
264
289
  ├── screenshot.py Playwright capture (extra: screenshot)
265
290
  ├── burp.py Burp REST integration (extra: burp)
@@ -268,7 +293,7 @@ src/phantomprobe/
268
293
  └── asgi.py Standalone ASGI entry point (extra: dashboard)
269
294
 
270
295
  scripts/audit_cpe_mapping.py Re-check the CVE table against live NVD
271
- tests/ 264 tests, network stubbed
296
+ tests/ 377 tests, network stubbed
272
297
  ```
273
298
 
274
299
  ## Development
@@ -11,6 +11,13 @@ Vendors do drift. NVD files nginx under f5 because F5 acquired it, so a mapping
11
11
  that was right when written can quietly stop matching. Re-run this now and then,
12
12
  and after adding anything to CPE_MAPPING.
13
13
 
14
+ A count above zero proves a pair exists, not that it is complete. f5:nginx
15
+ returned 41 and passed here while NVD was filing the newer nginx CVEs under
16
+ f5:nginx_open_source, so nginx 1.24.0 was shown 2 CVEs out of 17. Further
17
+ identifiers for the same software go in CPE_SIBLINGS and are checked here with
18
+ their technology. Finding them still takes a known answer: pick a version, look
19
+ up a CVE its vendor says affects it, and confirm the scanner returns that CVE.
20
+
14
21
  python scripts/audit_cpe_mapping.py # check the whole table
15
22
  python scripts/audit_cpe_mapping.py --tech nginx php # check a few entries
16
23
  python scripts/audit_cpe_mapping.py --check f5:nginx # try a candidate pair
@@ -158,7 +165,13 @@ def main(argv: Optional[List[str]] = None) -> int:
158
165
  unknown = [n for n in names if n not in mapping]
159
166
  if unknown:
160
167
  parser.error(f"not in CPE_MAPPING: {', '.join(unknown)}")
161
- pairs = [(n, mapping[n]["vendor"], mapping[n]["product"]) for n in names]
168
+ pairs = []
169
+ for n in names:
170
+ pairs.append((n, mapping[n]["vendor"], mapping[n]["product"]))
171
+ # A sibling that stops existing fails as quietly as a primary does.
172
+ for extra in CVEMatcher.CPE_SIBLINGS.get(n, []):
173
+ pairs.append((f"{n}+{extra['product']}",
174
+ extra["vendor"], extra["product"]))
162
175
 
163
176
  seconds = len(pairs) * interval
164
177
  eta = f"{seconds / 60:.0f} min" if seconds >= 90 else f"{seconds:.0f}s"