@fanboynz/network-scanner 2.0.58 → 2.0.60
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +970 -0
- package/README.md +31 -0
- package/lib/adblock.js +215 -179
- package/lib/cdp.js +17 -169
- package/lib/compare.js +19 -32
- package/lib/domain-cache.js +9 -7
- package/lib/fingerprint.js +9 -1
- package/lib/ghost-cursor.js +258 -0
- package/lib/grep.js +9 -13
- package/lib/nettools.js +177 -42
- package/lib/output.js +17 -30
- package/nwss.js +214 -73
- package/package.json +5 -1
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,970 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to the Network Scanner (nwss.js) project.
|
|
4
|
+
|
|
5
|
+
## [2.0.60] - 2026-03-16
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
- `--dns-cache` flag for persistent dig/whois disk caching between runs (`.digcache`, `.whoiscache`)
|
|
9
|
+
- `--load-extension <path>` flag to load unpacked Chrome extensions (supports multiple)
|
|
10
|
+
- `--block-ads` now supports comma-separated list files (`--block-ads=easylist.txt,easyprivacy.txt`)
|
|
11
|
+
- `disable_ad_tagging` config option to control Chrome AdTagging (default: true)
|
|
12
|
+
- DNS cache hit/miss statistics in scan summary output with fresh domain names listed
|
|
13
|
+
- Concurrent dig/whois deduplication — multiple pages requesting the same domain share one lookup
|
|
14
|
+
- SIGINT/SIGTERM handlers for `--keep-open` to prevent orphaned Chrome processes
|
|
15
|
+
|
|
16
|
+
### Fixed
|
|
17
|
+
- Adblock pipe (`|`) character handling — mid-pattern pipes were incorrectly treated as anchors, causing broad false positives on EasyList rules like `/addyn|*|adtech;`
|
|
18
|
+
- Domain Map fast path was skipping resource type checks — `$ping`, `$script` etc. now correctly enforced
|
|
19
|
+
- Domain extraction for `||domain.com/path` rules — path was incorrectly included in domain name
|
|
20
|
+
- `--keep-open` now skips extension-blocking Chrome flags so Chrome Web Store and extensions work
|
|
21
|
+
- Corrupt disk cache files are deleted instead of persisted
|
|
22
|
+
- `getBaseDomain()` now uses `psl` for correct multi-part TLD handling (`.co.uk`, `.com.au`)
|
|
23
|
+
- Merged 7 separate `--disable-features` flags into one — Chrome only reads the last occurrence
|
|
24
|
+
|
|
25
|
+
### Improved
|
|
26
|
+
- `$document` rules treated as full domain blocks (matches all resource types)
|
|
27
|
+
- `adblock.js`: regex cache for compiled patterns, Set for resource type lookups, lazy parentDomains, two-level result cache with LRU eviction (32K), hoisted constants, freed parsed options after rule parsing
|
|
28
|
+
- `output.js`: capped wildcard regex cache at 500, simplified `*.domain.com` suffix matching, hoisted resource type map
|
|
29
|
+
- `compare.js`: pre-compiled and deduplicated 6 normalization regexes
|
|
30
|
+
- `grep.js`: build grep args once outside pattern loop
|
|
31
|
+
- `domain-cache.js`: use Set iterator for eviction instead of full array copy
|
|
32
|
+
- `nettools.js`: hoisted ANSI strip regex, disk cache flushes once on exit instead of per-lookup
|
|
33
|
+
- Dig/whois cache: 14-hour TTL, 1000 entry limit, pretty-printed JSON files
|
|
34
|
+
|
|
35
|
+
## [2.0.59] - 2026-03-15
|
|
36
|
+
|
|
37
|
+
### Added
|
|
38
|
+
- `--keep-open` flag to keep browser and all tabs open after scan completes (use with `--headful` for debugging)
|
|
39
|
+
- `--use-puppeteer-core` flag to use `puppeteer-core` with system Chrome instead of bundled Chromium
|
|
40
|
+
- `puppeteer-core` as optional dependency in package.json
|
|
41
|
+
- Ghost-cursor integration for Bezier-based mouse movements (`--ghost-cursor` flag)
|
|
42
|
+
- Help text entries for `--keep-open`, `--use-puppeteer-core`
|
|
43
|
+
|
|
44
|
+
### Fixed
|
|
45
|
+
- Simulated mouse events now include `pageX`/`pageY`/`screenX`/`screenY` properties — scripts reading `event.pageX`/`pageY` for bot detection (e.g. dkitac.js) previously saw zero movement
|
|
46
|
+
- Stale comment reference to removed function
|
|
47
|
+
- CDP timeout leaks and dead code in `cdp.js`
|
|
48
|
+
|
|
49
|
+
### Improved
|
|
50
|
+
- Mouse interaction runs concurrently with post-load delay for better performance
|
|
51
|
+
- `maxTouchPoints` hardcoded to 0 for desktop Linux Chrome consistency
|
|
52
|
+
|
|
53
|
+
## [2.0.58] - 2026-03-14
|
|
54
|
+
|
|
55
|
+
### Fixed
|
|
56
|
+
- Race condition: re-check `isProcessing` before `page.close()` in realtime cleanup
|
|
57
|
+
- Page tracker stale entries during concurrent execution (added `untrackPage()`)
|
|
58
|
+
- ElementHandle leak in `interaction.js` — dispose body handle in `finally` block
|
|
59
|
+
|
|
60
|
+
### Improved
|
|
61
|
+
- macOS compatibility: add Chrome path detection and use `os.tmpdir()` for cross-platform temp dirs
|
|
62
|
+
- Harden `interaction.js` with page lifecycle checks to prevent mid-close errors
|
|
63
|
+
- Fingerprint interaction-gated trigger with scroll/keydown events and readyState check
|
|
64
|
+
- Low-impact optimisations across 6 modules (grep, flowproxy, dry-run, adblock, interaction, openvpn_vpn)
|
|
65
|
+
- Remove redundant `fs.existsSync()` guards in openvpn_vpn.js, compress.js, compare.js, validate_rules.js, output.js
|
|
66
|
+
- Hoist regex constants in `validate_rules.js`, cache wildcard regex in `output.js`
|
|
67
|
+
- Optimise `browserexit.js`: replace shell spawns with native fs operations
|
|
68
|
+
- Deduplicate session-closed error checks in `fingerprint.js`
|
|
69
|
+
- Remove dead code (`performMinimalInteraction`, unused `filteredArgs`)
|
|
70
|
+
- Migrate `.clauderc` to `CLAUDE.md`
|
|
71
|
+
|
|
72
|
+
## [2.0.57] - 2026-03-14
|
|
73
|
+
|
|
74
|
+
### Improved
|
|
75
|
+
- Optimise `ignore_similar.js`
|
|
76
|
+
|
|
77
|
+
## [2.0.56] - 2026-03-13
|
|
78
|
+
|
|
79
|
+
### Fixed
|
|
80
|
+
- Cloudflare challenge/solver scanning issues
|
|
81
|
+
- Browser health monitoring improvements
|
|
82
|
+
|
|
83
|
+
### Improved
|
|
84
|
+
- Cloudflare detection reliability and performance
|
|
85
|
+
- Chrome/Puppeteer performance tuning
|
|
86
|
+
- Smart cache optimisations in `smart-cache.js`
|
|
87
|
+
- Post-processing optimisations
|
|
88
|
+
|
|
89
|
+
## [2.0.55] - 2026-03-12
|
|
90
|
+
|
|
91
|
+
### Fixed
|
|
92
|
+
- Browser cleanup missing `com.google.Chrome` temp files
|
|
93
|
+
- Interaction.js reload interaction issues
|
|
94
|
+
|
|
95
|
+
### Improved
|
|
96
|
+
- Fingerprint.js improvements
|
|
97
|
+
- Interaction.js cleanup
|
|
98
|
+
|
|
99
|
+
## [2.0.54] - 2026-03-11
|
|
100
|
+
|
|
101
|
+
### Improved
|
|
102
|
+
- WebGL fingerprinting improvements, revert to `--disable-gpu`
|
|
103
|
+
- Reduce DIG and Whois request volume with domain caching
|
|
104
|
+
- Update user agents
|
|
105
|
+
|
|
106
|
+
## [2.0.53] - 2026-03-10
|
|
107
|
+
|
|
108
|
+
### Fixed
|
|
109
|
+
- Headless/GPU crash issues
|
|
110
|
+
- Fingerprint protection hardening
|
|
111
|
+
|
|
112
|
+
### Added
|
|
113
|
+
- Screenshot support using `force` option
|
|
114
|
+
|
|
115
|
+
### Improved
|
|
116
|
+
- Fingerprint protection improvements
|
|
117
|
+
|
|
118
|
+
## [2.0.52] - 2026-03-10
|
|
119
|
+
|
|
120
|
+
### Fixed
|
|
121
|
+
- Headless/GPU crash and fingerprint improvements
|
|
122
|
+
|
|
123
|
+
## [2.0.51] - 2026-02-24
|
|
124
|
+
|
|
125
|
+
### Added
|
|
126
|
+
- SOCKS/HTTP/HTTPS proxy support (`proxy.js`)
|
|
127
|
+
|
|
128
|
+
### Improved
|
|
129
|
+
- Update packages
|
|
130
|
+
- Compatibility improvements
|
|
131
|
+
|
|
132
|
+
## [2.0.50] - 2026-02-17
|
|
133
|
+
|
|
134
|
+
### Fixed
|
|
135
|
+
- Fingerprint `random` mode improvements
|
|
136
|
+
- CDP round-trips reduced to 1, cache bodyText
|
|
137
|
+
- `safeClick`/`safeWaitForNavigation` timeout leaks
|
|
138
|
+
- Redundant context validation removed
|
|
139
|
+
- Shadowroot compatibility on `cloudflare.js`
|
|
140
|
+
|
|
141
|
+
### Improved
|
|
142
|
+
- `interact: true` performance
|
|
143
|
+
- Canvas noise optimisation for large canvases
|
|
144
|
+
- Fingerprint consistency fixes (mousemove WeakMap, human simulation timing)
|
|
145
|
+
- `measureText` read-only property fix
|
|
146
|
+
- Support for larger lists
|
|
147
|
+
- `ignoreDomains` improvements
|
|
148
|
+
- Hot path performance optimisations (indexed loops, single-pass regex matching, URL parsing)
|
|
149
|
+
- Adblock domain matcher precomputation
|
|
150
|
+
|
|
151
|
+
## [2.0.49] - 2026-02-17
|
|
152
|
+
|
|
153
|
+
### Improved
|
|
154
|
+
- Fingerprint protection `random` mode enhancements
|
|
155
|
+
|
|
156
|
+
## [2.0.48] - 2026-02-17
|
|
157
|
+
|
|
158
|
+
### Improved
|
|
159
|
+
- Adblock rule parser: V8 optimisations, cached hostname split, Map-based lookups
|
|
160
|
+
- Precompute parent domains for whitelist and block checks
|
|
161
|
+
- Remove dead code in `grep.js`
|
|
162
|
+
|
|
163
|
+
## [2.0.47] - 2026-02-17
|
|
164
|
+
|
|
165
|
+
### Added
|
|
166
|
+
- Support for `$counter` adblock rules
|
|
167
|
+
- Support for `$1p`, `$~third-party`, `$first-party` adblock options
|
|
168
|
+
|
|
169
|
+
### Fixed
|
|
170
|
+
- Missing variable fix
|
|
171
|
+
- More adblock rule format support
|
|
172
|
+
|
|
173
|
+
## [2.0.46] - 2026-02-16
|
|
174
|
+
|
|
175
|
+
### Fixed
|
|
176
|
+
- Potential memory leaks
|
|
177
|
+
- Timing range miscalculation
|
|
178
|
+
- `TEXT_PREVIEW_LENGTH` unreachable inside `page.evaluate()`
|
|
179
|
+
- Unused variables cleanup
|
|
180
|
+
|
|
181
|
+
### Improved
|
|
182
|
+
- Processing termination to avoid stale processes
|
|
183
|
+
|
|
184
|
+
## [2.0.45] - 2026-02-16
|
|
185
|
+
|
|
186
|
+
### Improved
|
|
187
|
+
- Processing termination reliability
|
|
188
|
+
|
|
189
|
+
## [2.0.44] - 2026-02-16
|
|
190
|
+
|
|
191
|
+
### Added
|
|
192
|
+
- OpenVPN support (`openvpn_vpn.js`) — [#45](https://github.com/ryanbr/network-scanner/issues/45)
|
|
193
|
+
|
|
194
|
+
## [2.0.43] - 2026-02-16
|
|
195
|
+
|
|
196
|
+
### Added
|
|
197
|
+
- Initial WireGuard VPN support (`wireguard_vpn.js`) — [#45](https://github.com/ryanbr/network-scanner/issues/45)
|
|
198
|
+
|
|
199
|
+
## [2.0.42] - 2026-02-16
|
|
200
|
+
|
|
201
|
+
### Fixed
|
|
202
|
+
- `maxTouchPoints` potentially overridden twice
|
|
203
|
+
- Duplicate `console.error` overrides in fingerprint
|
|
204
|
+
- `hardwareConcurrency` returning different values on every read
|
|
205
|
+
- Brave UA getter infinite recursion
|
|
206
|
+
|
|
207
|
+
### Improved
|
|
208
|
+
- General cleanups and unused function removal
|
|
209
|
+
|
|
210
|
+
## [2.0.41] - 2026-02-16
|
|
211
|
+
|
|
212
|
+
### Fixed
|
|
213
|
+
- Binary issue with `smart-cache.js`
|
|
214
|
+
|
|
215
|
+
## [2.0.40] - 2026-02-16
|
|
216
|
+
|
|
217
|
+
### Fixed
|
|
218
|
+
- Missing `requestCache` in smart-cache clear/destroy
|
|
219
|
+
- Duplicate `totalCacheEntries` in `getStats`
|
|
220
|
+
- Undefined `forceDebug` reference in `cacheRequest`
|
|
221
|
+
- Missing `normalizedUrl` declaration in `cacheRequest`
|
|
222
|
+
|
|
223
|
+
## [2.0.39] - 2026-02-16
|
|
224
|
+
|
|
225
|
+
### Improved
|
|
226
|
+
- Nettools: buffered log writer instead of `fs.appendFileSync`
|
|
227
|
+
|
|
228
|
+
## [2.0.38] - 2026-02-16
|
|
229
|
+
|
|
230
|
+
### Fixed
|
|
231
|
+
- Catch-and-rethrow doing nothing in nettools
|
|
232
|
+
- Double timeout in `createNetToolsHandler`
|
|
233
|
+
|
|
234
|
+
### Improved
|
|
235
|
+
- Replace global whois server index with module-level variable
|
|
236
|
+
- Hoist `execSync` and move `tldServers` to module scope
|
|
237
|
+
|
|
238
|
+
## [2.0.37] - 2026-02-16
|
|
239
|
+
|
|
240
|
+
### Improved
|
|
241
|
+
- Browser health: store timestamp in page creation tracker
|
|
242
|
+
- Cleanup `formatMemory` redefinition
|
|
243
|
+
- Hoist `require('child_process')` in `checkBrowserMemory`
|
|
244
|
+
- Replace `Page.prototype` monkey-patch with explicit tracker cleanup
|
|
245
|
+
|
|
246
|
+
## [2.0.36] - 2026-02-16
|
|
247
|
+
|
|
248
|
+
### Improved
|
|
249
|
+
- `browserexit.js`: remove duplicate pattern, hoist requires
|
|
250
|
+
|
|
251
|
+
## [2.0.35] - 2026-02-16
|
|
252
|
+
|
|
253
|
+
### Improved
|
|
254
|
+
- Buffer log writes, pre-compile regexes, deduplicate request handler
|
|
255
|
+
|
|
256
|
+
## [2.0.34] - 2026-02-16
|
|
257
|
+
|
|
258
|
+
### Improved
|
|
259
|
+
- General cleanup
|
|
260
|
+
|
|
261
|
+
## [2.0.33] - 2025-11-14
|
|
262
|
+
|
|
263
|
+
### Added
|
|
264
|
+
- Adblock list support for blocking URLs during scanning
|
|
265
|
+
- V8 optimised adblock parser with LRU cache and Map-based domain lookups
|
|
266
|
+
|
|
267
|
+
### Improved
|
|
268
|
+
- Bump Firefox user agent
|
|
269
|
+
- Rename `adblock_rules.js` to `adblock.js`
|
|
270
|
+
|
|
271
|
+
## [2.0.32] - 2025-11-08
|
|
272
|
+
|
|
273
|
+
### Fixed
|
|
274
|
+
- Race conditions: atomic `checkAndMark()` in domain cache
|
|
275
|
+
- Performance improvements and V8 optimisations
|
|
276
|
+
|
|
277
|
+
### Improved
|
|
278
|
+
- `referrer.js` V8 optimisations
|
|
279
|
+
- Update packages
|
|
280
|
+
|
|
281
|
+
## [2.0.31] - 2025-10-31
|
|
282
|
+
|
|
283
|
+
### Added
|
|
284
|
+
- `referrer_disable` support
|
|
285
|
+
- `referrer_headers` support
|
|
286
|
+
|
|
287
|
+
### Fixed
|
|
288
|
+
- `url is not defined` errors
|
|
289
|
+
- Referrer.js incorrectly added to nwss.js
|
|
290
|
+
- Page state checks before reload, network idle, CSS blocking evaluation
|
|
291
|
+
|
|
292
|
+
### Improved
|
|
293
|
+
- `grep.js` improvements
|
|
294
|
+
|
|
295
|
+
## [2.0.30] - 2025-10-29
|
|
296
|
+
|
|
297
|
+
### Added
|
|
298
|
+
- Location URL masking
|
|
299
|
+
- Additional automation property hiding
|
|
300
|
+
|
|
301
|
+
### Improved
|
|
302
|
+
- Font enumeration protection
|
|
303
|
+
- Fingerprint platform matching
|
|
304
|
+
- Bump Chrome to 142.x
|
|
305
|
+
|
|
306
|
+
## [2.0.29] - 2025-10-21
|
|
307
|
+
|
|
308
|
+
### Improved
|
|
309
|
+
- Hang check loop and browser restart on hang
|
|
310
|
+
- Chrome launch arguments
|
|
311
|
+
- Permissions API fingerprinting
|
|
312
|
+
- Realistic Chrome browser behaviour simulation
|
|
313
|
+
- Chrome runtime simulation strengthening
|
|
314
|
+
|
|
315
|
+
## [2.0.28] - 2025-10-11
|
|
316
|
+
|
|
317
|
+
### Improved
|
|
318
|
+
- Page method caching optimisations
|
|
319
|
+
- Consistent return objects in health checks
|
|
320
|
+
- CDP.js V8 optimisations
|
|
321
|
+
- Bump overall timeout from 30s to 65s
|
|
322
|
+
- Nettools optimisations
|
|
323
|
+
|
|
324
|
+
## [2.0.27] - 2025-10-07
|
|
325
|
+
|
|
326
|
+
### Improved
|
|
327
|
+
- Whois retry on TIMEOUT/FAIL to avoid throttling
|
|
328
|
+
|
|
329
|
+
## [2.0.26] - 2025-10-06
|
|
330
|
+
|
|
331
|
+
### Improved
|
|
332
|
+
- V8 optimisations: `Object.freeze()`, destructuring, pre-allocated arrays, Maps
|
|
333
|
+
- Bump Chrome version
|
|
334
|
+
|
|
335
|
+
## [2.0.25] - 2025-10-05
|
|
336
|
+
|
|
337
|
+
### Fixed
|
|
338
|
+
- Frame handling `frameUrl is not defined` errors
|
|
339
|
+
- `activeFrames.add is not a function`
|
|
340
|
+
- `spoofNavigatorProperties is not defined`
|
|
341
|
+
|
|
342
|
+
### Improved
|
|
343
|
+
- Frame URL improvements
|
|
344
|
+
- Allow grep without curl
|
|
345
|
+
|
|
346
|
+
## [2.0.24] - 2025-10-04
|
|
347
|
+
|
|
348
|
+
### Improved
|
|
349
|
+
- Fingerprint.js V8 performance: pre-compiled mocks, monomorphic object shapes, cached descriptors
|
|
350
|
+
- Address [#41](https://github.com/ryanbr/network-scanner/issues/41)
|
|
351
|
+
|
|
352
|
+
## [2.0.23] - 2025-10-01
|
|
353
|
+
|
|
354
|
+
### Improved
|
|
355
|
+
- Whois retry enabled by default with tuned retries/delay
|
|
356
|
+
|
|
357
|
+
## [2.0.22] - 2025-10-01
|
|
358
|
+
|
|
359
|
+
### Added
|
|
360
|
+
- `--dry-run` split into separate module (`dry-run.js`)
|
|
361
|
+
|
|
362
|
+
## [2.0.21] - 2025-09-30
|
|
363
|
+
|
|
364
|
+
### Added
|
|
365
|
+
- Domain-based `forcereload` support (`forcereload=domain.com,domain2.com`)
|
|
366
|
+
- Input validation and domain cleaning for forcereload
|
|
367
|
+
|
|
368
|
+
### Improved
|
|
369
|
+
- Update man page and `--help` args
|
|
370
|
+
|
|
371
|
+
## [2.0.20] - 2025-09-29
|
|
372
|
+
|
|
373
|
+
### Improved
|
|
374
|
+
- `--localhost` now configurable (`--localhost=x.x.x.x`)
|
|
375
|
+
|
|
376
|
+
## [2.0.19] - 2025-09-27
|
|
377
|
+
|
|
378
|
+
### Improved
|
|
379
|
+
- `--remove-dupes` reliability
|
|
380
|
+
|
|
381
|
+
## [2.0.18] - 2025-09-27
|
|
382
|
+
|
|
383
|
+
### Fixed
|
|
384
|
+
- Whois logic occasionally missing records
|
|
385
|
+
|
|
386
|
+
## [2.0.17] - 2025-09-27
|
|
387
|
+
|
|
388
|
+
### Improved
|
|
389
|
+
- `window_cleanup` realtime less aggressive, added validation checks
|
|
390
|
+
|
|
391
|
+
## [2.0.16] - 2025-09-25
|
|
392
|
+
|
|
393
|
+
### Fixed
|
|
394
|
+
- `tar-fs` security vulnerability
|
|
395
|
+
|
|
396
|
+
## [2.0.15] - 2025-09-25
|
|
397
|
+
|
|
398
|
+
### Improved
|
|
399
|
+
- Font, canvas, WebGL, permission, hardware concurrency, plugin fingerprinting
|
|
400
|
+
|
|
401
|
+
## [2.0.14] - 2025-09-24
|
|
402
|
+
|
|
403
|
+
### Improved
|
|
404
|
+
- Fingerprinting updates
|
|
405
|
+
- Bump Firefox version
|
|
406
|
+
- Wrap errors in `--debug`
|
|
407
|
+
|
|
408
|
+
## [2.0.13] - 2025-09-24
|
|
409
|
+
|
|
410
|
+
### Improved
|
|
411
|
+
- Bump Firefox version
|
|
412
|
+
|
|
413
|
+
## [2.0.12] - 2025-09-23
|
|
414
|
+
|
|
415
|
+
### Fixed
|
|
416
|
+
- Navigator.brave checks
|
|
417
|
+
- Fingerprint.js error handling
|
|
418
|
+
|
|
419
|
+
## [2.0.11] - 2025-09-23
|
|
420
|
+
|
|
421
|
+
### Improved
|
|
422
|
+
- Bump timeouts, make delay a const
|
|
423
|
+
|
|
424
|
+
## [2.0.10] - 2025-09-23
|
|
425
|
+
|
|
426
|
+
### Fixed
|
|
427
|
+
- Occasional detach issues during scanning
|
|
428
|
+
|
|
429
|
+
## [2.0.9] - 2025-09-21
|
|
430
|
+
|
|
431
|
+
### Added
|
|
432
|
+
- `cdp_specific` support for per-URL CDP without global `cdp: true`
|
|
433
|
+
|
|
434
|
+
## [2.0.8] - 2025-09-20
|
|
435
|
+
|
|
436
|
+
### Added
|
|
437
|
+
- User agents for Linux and macOS
|
|
438
|
+
|
|
439
|
+
## [2.0.7] - 2025-09-20
|
|
440
|
+
|
|
441
|
+
### Added
|
|
442
|
+
- `clear_sitedata.js` for CDP fixes
|
|
443
|
+
|
|
444
|
+
### Improved
|
|
445
|
+
- Bump Cloudflare version
|
|
446
|
+
|
|
447
|
+
## [2.0.6] - 2025-09-19
|
|
448
|
+
|
|
449
|
+
### Improved
|
|
450
|
+
- CDP.js reliability with retry support
|
|
451
|
+
|
|
452
|
+
## [2.0.5] - 2025-09-19
|
|
453
|
+
|
|
454
|
+
### Fixed
|
|
455
|
+
- Race condition with `window_cleanup=realtime` and Cloudflare
|
|
456
|
+
|
|
457
|
+
## [2.0.4] - 2025-09-17
|
|
458
|
+
|
|
459
|
+
### Improved
|
|
460
|
+
- Cloudflare.js v2.6.1
|
|
461
|
+
|
|
462
|
+
## [2.0.3] - 2025-09-17
|
|
463
|
+
|
|
464
|
+
### Fixed
|
|
465
|
+
- Frame detach errors — [#38](https://github.com/ryanbr/network-scanner/issues/38)
|
|
466
|
+
|
|
467
|
+
### Improved
|
|
468
|
+
- Cloudflare.js v2.6.0
|
|
469
|
+
|
|
470
|
+
## [2.0.2] - 2025-09-15
|
|
471
|
+
|
|
472
|
+
### Improved
|
|
473
|
+
- Cloudflare.js v2.5.0
|
|
474
|
+
|
|
475
|
+
## [2.0.1] - 2025-09-15
|
|
476
|
+
|
|
477
|
+
### Fixed
|
|
478
|
+
- Pi-hole regex slash handling
|
|
479
|
+
- Allow latest Puppeteer version
|
|
480
|
+
|
|
481
|
+
## [2.0.0] - 2025-09-15
|
|
482
|
+
|
|
483
|
+
### Changed
|
|
484
|
+
- Major version bump — Puppeteer compatibility and architecture updates
|
|
485
|
+
|
|
486
|
+
## [1.0.99] - 2025-09-13
|
|
487
|
+
|
|
488
|
+
### Improved
|
|
489
|
+
- Bump user agents
|
|
490
|
+
- Increase browser health thresholds
|
|
491
|
+
|
|
492
|
+
## [1.0.98] - 2025-09-09
|
|
493
|
+
|
|
494
|
+
### Added
|
|
495
|
+
- Realtime `window_cleanup` for larger URL lists
|
|
496
|
+
|
|
497
|
+
## [1.0.97] - 2025-09-06
|
|
498
|
+
|
|
499
|
+
### Added
|
|
500
|
+
- `window_cleanup` to close old tabs, releasing memory on larger URL lists
|
|
501
|
+
|
|
502
|
+
## [1.0.96] - 2025-09-05
|
|
503
|
+
|
|
504
|
+
### Improved
|
|
505
|
+
- CDP timeout improvements
|
|
506
|
+
|
|
507
|
+
## [1.0.95] - 2025-09-05
|
|
508
|
+
|
|
509
|
+
### Fixed
|
|
510
|
+
- Persistent failure recovery — move to next URL instead of error
|
|
511
|
+
|
|
512
|
+
## [1.0.94] - 2025-09-04
|
|
513
|
+
|
|
514
|
+
### Fixed
|
|
515
|
+
- ForceReload fallback for Puppeteer v23.x compatibility
|
|
516
|
+
|
|
517
|
+
## [1.0.93] - 2025-09-03
|
|
518
|
+
|
|
519
|
+
### Improved
|
|
520
|
+
- Health checks and fallback on `evaluateOnNewDocument` failure
|
|
521
|
+
|
|
522
|
+
## [1.0.92] - 2025-09-03
|
|
523
|
+
|
|
524
|
+
### Improved
|
|
525
|
+
- Interaction.js tweaks
|
|
526
|
+
|
|
527
|
+
## [1.0.91] - 2025-09-03
|
|
528
|
+
|
|
529
|
+
### Improved
|
|
530
|
+
- Minor version bumps
|
|
531
|
+
|
|
532
|
+
## [1.0.88] - 2025-09-01
|
|
533
|
+
|
|
534
|
+
### Added
|
|
535
|
+
- Split curl functions from `grep.js` — [#33](https://github.com/ryanbr/network-scanner/issues/33)
|
|
536
|
+
|
|
537
|
+
### Improved
|
|
538
|
+
- Cloudflare.js v2.4.1
|
|
539
|
+
|
|
540
|
+
## [1.0.86] - 2025-08-31
|
|
541
|
+
|
|
542
|
+
### Fixed
|
|
543
|
+
- Puppeteer 24.x compatibility and browser health issues — [#28](https://github.com/ryanbr/network-scanner/issues/28)
|
|
544
|
+
|
|
545
|
+
## [1.0.85] - 2025-08-31
|
|
546
|
+
|
|
547
|
+
### Improved
|
|
548
|
+
- Post-processing first-party item checks
|
|
549
|
+
|
|
550
|
+
## [1.0.83] - 2025-08-30
|
|
551
|
+
|
|
552
|
+
### Fixed
|
|
553
|
+
- ForceReload logic to apply after each reload
|
|
554
|
+
|
|
555
|
+
## [1.0.82] - 2025-08-29
|
|
556
|
+
|
|
557
|
+
### Improved
|
|
558
|
+
- Cloudflare.js v2.4
|
|
559
|
+
|
|
560
|
+
## [1.0.81] - 2025-08-28
|
|
561
|
+
|
|
562
|
+
### Fixed
|
|
563
|
+
- Endless loops caused by some sites
|
|
564
|
+
|
|
565
|
+
## [1.0.80] - 2025-08-27
|
|
566
|
+
|
|
567
|
+
### Fixed
|
|
568
|
+
- Performance issues with interact and resource cleanup
|
|
569
|
+
|
|
570
|
+
## [1.0.78] - 2025-08-27
|
|
571
|
+
|
|
572
|
+
### Added
|
|
573
|
+
- INSTALL suggestions
|
|
574
|
+
|
|
575
|
+
## [1.0.77] - 2025-08-26
|
|
576
|
+
|
|
577
|
+
### Improved
|
|
578
|
+
- Cloudflare.js v2.3
|
|
579
|
+
|
|
580
|
+
## [1.0.76] - 2025-08-21
|
|
581
|
+
|
|
582
|
+
### Added
|
|
583
|
+
- Cached network requests for duplicate URLs in same JSON
|
|
584
|
+
|
|
585
|
+
### Fixed
|
|
586
|
+
- Duplicate function removal
|
|
587
|
+
|
|
588
|
+
## [1.0.75] - 2025-08-19
|
|
589
|
+
|
|
590
|
+
### Fixed
|
|
591
|
+
- Nettools not firing
|
|
592
|
+
|
|
593
|
+
## [1.0.74] - 2025-08-19
|
|
594
|
+
|
|
595
|
+
### Fixed
|
|
596
|
+
- Nettools being ignored
|
|
597
|
+
|
|
598
|
+
## [1.0.73] - 2025-08-19
|
|
599
|
+
|
|
600
|
+
### Added
|
|
601
|
+
- `regex_and` to apply AND logic on filterRegex
|
|
602
|
+
|
|
603
|
+
## [1.0.72] - 2025-08-18
|
|
604
|
+
|
|
605
|
+
### Improved
|
|
606
|
+
- Cloudflare.js v2.2
|
|
607
|
+
|
|
608
|
+
## [1.0.70] - 2025-08-17
|
|
609
|
+
|
|
610
|
+
### Improved
|
|
611
|
+
- Regex tool GitHub compatibility
|
|
612
|
+
|
|
613
|
+
## [1.0.69] - 2025-08-17
|
|
614
|
+
|
|
615
|
+
### Improved
|
|
616
|
+
- Convert magic numbers to constants in nwss.js
|
|
617
|
+
|
|
618
|
+
## [1.0.68] - 2025-08-15
|
|
619
|
+
|
|
620
|
+
### Fixed
|
|
621
|
+
- URL popup protection — don't treat main URL changes as third-party
|
|
622
|
+
|
|
623
|
+
## [1.0.67] - 2025-08-14
|
|
624
|
+
|
|
625
|
+
### Fixed
|
|
626
|
+
- `third-party: true` never matches root URL
|
|
627
|
+
|
|
628
|
+
## [1.0.66] - 2025-08-12
|
|
629
|
+
|
|
630
|
+
### Improved
|
|
631
|
+
- Interaction.js performance
|
|
632
|
+
- Fingerprint.js refactor
|
|
633
|
+
- Puppeteer 23 compatibility
|
|
634
|
+
|
|
635
|
+
## [1.0.63] - 2025-08-11
|
|
636
|
+
|
|
637
|
+
### Fixed
|
|
638
|
+
- Occasional interaction.js delays
|
|
639
|
+
- Security vulnerabilities in `tar-fs` and `ws`
|
|
640
|
+
|
|
641
|
+
### Improved
|
|
642
|
+
- Puppeteer 23.x support
|
|
643
|
+
|
|
644
|
+
## [1.0.60] - 2025-08-10
|
|
645
|
+
|
|
646
|
+
### Changed
|
|
647
|
+
- Pin to Puppeteer 20.x for stability
|
|
648
|
+
|
|
649
|
+
## [1.0.59] - 2025-08-08
|
|
650
|
+
|
|
651
|
+
### Improved
|
|
652
|
+
- Searchstring improvements
|
|
653
|
+
- Update dependencies for Node.js 20+
|
|
654
|
+
|
|
655
|
+
## [1.0.58] - 2025-08-08
|
|
656
|
+
|
|
657
|
+
### Added
|
|
658
|
+
- `--clear-cache` / `--ignore-cache` options
|
|
659
|
+
|
|
660
|
+
### Improved
|
|
661
|
+
- Smart cache memory management
|
|
662
|
+
|
|
663
|
+
## [1.0.57] - 2025-08-06
|
|
664
|
+
|
|
665
|
+
### Added
|
|
666
|
+
- Smart caching system (`smart-cache.js`)
|
|
667
|
+
|
|
668
|
+
## [1.0.53] - 2025-08-06
|
|
669
|
+
|
|
670
|
+
### Added
|
|
671
|
+
- Automated npm publishing workflow
|
|
672
|
+
|
|
673
|
+
## [1.0.49] - 2025-08-04
|
|
674
|
+
|
|
675
|
+
### Added
|
|
676
|
+
- ESLint configuration
|
|
677
|
+
|
|
678
|
+
### Improved
|
|
679
|
+
- CDP functionality separated into own module
|
|
680
|
+
|
|
681
|
+
## [1.0.47] - 2025-08-03
|
|
682
|
+
|
|
683
|
+
### Improved
|
|
684
|
+
- Mouse simulator made more modular
|
|
685
|
+
- Cloudflare and FlowProxy skip non-HTTP URLs
|
|
686
|
+
|
|
687
|
+
### Fixed
|
|
688
|
+
- Regression on `subDomains=1`
|
|
689
|
+
|
|
690
|
+
## [1.0.46] - 2025-08-01
|
|
691
|
+
|
|
692
|
+
### Improved
|
|
693
|
+
- Skip previously detected domains
|
|
694
|
+
- Magic numbers converted to constants
|
|
695
|
+
- Cloudflare.js documentation
|
|
696
|
+
|
|
697
|
+
## [1.0.45] - 2025-07-31
|
|
698
|
+
|
|
699
|
+
### Added
|
|
700
|
+
- Whois and dig result caching
|
|
701
|
+
|
|
702
|
+
### Improved
|
|
703
|
+
- Dig/nettools with multiple URLs
|
|
704
|
+
|
|
705
|
+
## [1.0.44] - 2025-07-30
|
|
706
|
+
|
|
707
|
+
### Added
|
|
708
|
+
- User-configurable `maxConcurrentSites` and `cleanup-interval`
|
|
709
|
+
|
|
710
|
+
## [1.0.43] - 2025-07-20
|
|
711
|
+
|
|
712
|
+
### Fixed
|
|
713
|
+
- Browser restart on `protocolTimeout`
|
|
714
|
+
|
|
715
|
+
### Improved
|
|
716
|
+
- Cloudflare wait times and timeouts
|
|
717
|
+
|
|
718
|
+
## [1.0.42] - 2025-07-16
|
|
719
|
+
|
|
720
|
+
### Added
|
|
721
|
+
- Referrer options support
|
|
722
|
+
|
|
723
|
+
### Improved
|
|
724
|
+
- Redirecting domains compatibility
|
|
725
|
+
- Fingerprint.js improvements
|
|
726
|
+
|
|
727
|
+
## [1.0.41] - 2025-07-14
|
|
728
|
+
|
|
729
|
+
### Added
|
|
730
|
+
- `ignore_similar` domains feature
|
|
731
|
+
|
|
732
|
+
## [1.0.40] - 2025-07-02
|
|
733
|
+
|
|
734
|
+
### Added
|
|
735
|
+
- `even_blocked` option
|
|
736
|
+
|
|
737
|
+
### Fixed
|
|
738
|
+
- Puppeteer old headless deprecation warnings
|
|
739
|
+
|
|
740
|
+
### Improved
|
|
741
|
+
- Domain validation — [#27](https://github.com/ryanbr/network-scanner/issues/27)
|
|
742
|
+
- `--append` output support
|
|
743
|
+
|
|
744
|
+
## [1.0.39] - 2025-06-24
|
|
745
|
+
|
|
746
|
+
### Added
|
|
747
|
+
- `--dry-run` option with file output
|
|
748
|
+
|
|
749
|
+
### Improved
|
|
750
|
+
- `ignoreDomains` fallback removal
|
|
751
|
+
|
|
752
|
+
## [1.0.38] - 2025-06-21
|
|
753
|
+
|
|
754
|
+
### Fixed
|
|
755
|
+
- First-party/third-party and ignoreDomains prioritisation
|
|
756
|
+
|
|
757
|
+
## [1.0.37] - 2025-06-17
|
|
758
|
+
|
|
759
|
+
### Added
|
|
760
|
+
- `--remove-tempfiles` option
|
|
761
|
+
|
|
762
|
+
## [1.0.36] - 2025-06-16
|
|
763
|
+
|
|
764
|
+
### Added
|
|
765
|
+
- FlowProxy DDoS protection support — [#24](https://github.com/ryanbr/network-scanner/issues/24)
|
|
766
|
+
|
|
767
|
+
### Improved
|
|
768
|
+
- Browser health checks and restart on degradation
|
|
769
|
+
- Chrome process killing
|
|
770
|
+
- Insecure site loading support
|
|
771
|
+
|
|
772
|
+
## [1.0.35] - 2025-06-15
|
|
773
|
+
|
|
774
|
+
### Added
|
|
775
|
+
- Whois and dig debug file output with ANSI stripping
|
|
776
|
+
|
|
777
|
+
### Improved
|
|
778
|
+
- Whois reliability
|
|
779
|
+
|
|
780
|
+
## [1.0.34] - 2025-06-13
|
|
781
|
+
|
|
782
|
+
### Fixed
|
|
783
|
+
- Out-of-space issues from `puppeteer_dev_chrome_profile` temp files
|
|
784
|
+
- Error handling crash
|
|
785
|
+
- `about:srcdoc`, `data:`, `about:`, `chrome:`, `blob:` URL handling — [#21](https://github.com/ryanbr/network-scanner/issues/21)
|
|
786
|
+
|
|
787
|
+
## [1.0.33] - 2025-06-11
|
|
788
|
+
|
|
789
|
+
### Added
|
|
790
|
+
- Pi-hole output format (`--pihole`)
|
|
791
|
+
- Privoxy output format
|
|
792
|
+
- Comments value in JSON config
|
|
793
|
+
|
|
794
|
+
## [1.0.32] - 2025-06-10
|
|
795
|
+
|
|
796
|
+
### Added
|
|
797
|
+
- `whois_server_mode` (random/cycle)
|
|
798
|
+
- Configurable whois delay
|
|
799
|
+
- Whois error logging to `logs/debug`
|
|
800
|
+
- Coloured console output
|
|
801
|
+
|
|
802
|
+
### Improved
|
|
803
|
+
- Browser detection with custom userAgent
|
|
804
|
+
|
|
805
|
+
## [1.0.31] - 2025-06-09
|
|
806
|
+
|
|
807
|
+
### Changed
|
|
808
|
+
- Rename `scanner-script.js` to `nwss.js`
|
|
809
|
+
|
|
810
|
+
## [1.0.30] - 2025-06-08
|
|
811
|
+
|
|
812
|
+
### Added
|
|
813
|
+
- Searchstring AND logic
|
|
814
|
+
- Unbound, DNSMasq output formats
|
|
815
|
+
|
|
816
|
+
### Fixed
|
|
817
|
+
- Iframe debug errors
|
|
818
|
+
|
|
819
|
+
## [1.0.29] - 2025-06-06
|
|
820
|
+
|
|
821
|
+
### Added
|
|
822
|
+
- Custom whois servers with retry/fallback
|
|
823
|
+
|
|
824
|
+
### Improved
|
|
825
|
+
- WSL compatibility
|
|
826
|
+
|
|
827
|
+
## [1.0.28] - 2025-06-05
|
|
828
|
+
|
|
829
|
+
### Added
|
|
830
|
+
- Global blocked domains support
|
|
831
|
+
- `goto_options` config
|
|
832
|
+
|
|
833
|
+
### Improved
|
|
834
|
+
- Scanning method improvements
|
|
835
|
+
|
|
836
|
+
## [1.0.27] - 2025-06-04
|
|
837
|
+
|
|
838
|
+
### Added
|
|
839
|
+
- `--compare` with `--titles` support — [#1](https://github.com/ryanbr/network-scanner/issues/1)
|
|
840
|
+
- `--remove-dupes` alias
|
|
841
|
+
|
|
842
|
+
### Improved
|
|
843
|
+
- Resource management with service restarts
|
|
844
|
+
|
|
845
|
+
## [1.0.26] - 2025-06-02
|
|
846
|
+
|
|
847
|
+
### Added
|
|
848
|
+
- Whois/dig support — [#18](https://github.com/ryanbr/network-scanner/issues/18)
|
|
849
|
+
- `--debug` and `--dumpurls` file output
|
|
850
|
+
|
|
851
|
+
## [1.0.25] - 2025-05-31
|
|
852
|
+
|
|
853
|
+
### Added
|
|
854
|
+
- Curl and grep alternative scan method
|
|
855
|
+
- Adblock rules output format
|
|
856
|
+
|
|
857
|
+
### Improved
|
|
858
|
+
- Cloudflare bypass split to own module
|
|
859
|
+
- Output split to `output.js`
|
|
860
|
+
- Fingerprinting split to own module
|
|
861
|
+
|
|
862
|
+
## [1.0.24] - 2025-05-30
|
|
863
|
+
|
|
864
|
+
### Added
|
|
865
|
+
- Searchstring support (search within regex-matched content)
|
|
866
|
+
- `--remove-dupes` on output
|
|
867
|
+
- Wildcard support in ignored domains
|
|
868
|
+
|
|
869
|
+
## [1.0.23] - 2025-05-27
|
|
870
|
+
|
|
871
|
+
### Added
|
|
872
|
+
- `--debug` logging improvements
|
|
873
|
+
|
|
874
|
+
### Improved
|
|
875
|
+
- Graceful exit handling
|
|
876
|
+
- Module split: CDP, interact, evaluateOnNewDocument, Cloudflare, CSS blocking, fingerprint
|
|
877
|
+
|
|
878
|
+
## [1.0.22] - 2025-05-26
|
|
879
|
+
|
|
880
|
+
### Added
|
|
881
|
+
- Cloudflare phishing warning bypass
|
|
882
|
+
- CSS blocking support — [#2](https://github.com/ryanbr/network-scanner/issues/2)
|
|
883
|
+
|
|
884
|
+
### Improved
|
|
885
|
+
- Concurrent site scanning resource management
|
|
886
|
+
|
|
887
|
+
## [1.0.21] - 2025-05-23
|
|
888
|
+
|
|
889
|
+
### Added
|
|
890
|
+
- Multithread/concurrent support
|
|
891
|
+
- CDP logging improvements
|
|
892
|
+
- Address [#14](https://github.com/ryanbr/network-scanner/issues/14), [#15](https://github.com/ryanbr/network-scanner/issues/15)
|
|
893
|
+
|
|
894
|
+
## [1.0.20] - 2025-05-21
|
|
895
|
+
|
|
896
|
+
### Added
|
|
897
|
+
- Per-site verbose output with matching regex
|
|
898
|
+
- Scan timer
|
|
899
|
+
- Scan counter
|
|
900
|
+
|
|
901
|
+
### Improved
|
|
902
|
+
- First-party/third-party detection
|
|
903
|
+
|
|
904
|
+
## [1.0.19] - 2025-05-19
|
|
905
|
+
|
|
906
|
+
### Added
|
|
907
|
+
- `package.json` for npm
|
|
908
|
+
|
|
909
|
+
### Fixed
|
|
910
|
+
- Sandboxing issue on Linux
|
|
911
|
+
|
|
912
|
+
## [1.0.18] - 2025-05-03
|
|
913
|
+
|
|
914
|
+
### Added
|
|
915
|
+
- JSON manual (`JSONMANUAL.md`)
|
|
916
|
+
|
|
917
|
+
### Improved
|
|
918
|
+
- Scanner methods — [#3](https://github.com/ryanbr/network-scanner/issues/3)
|
|
919
|
+
- `--plain` unformatted domain output
|
|
920
|
+
- Global blocked items
|
|
921
|
+
|
|
922
|
+
## [1.0.17] - 2025-05-01
|
|
923
|
+
|
|
924
|
+
### Added
|
|
925
|
+
- Headful browser mode
|
|
926
|
+
- Screenshot option for debugging
|
|
927
|
+
- Custom JSON file support
|
|
928
|
+
|
|
929
|
+
### Fixed
|
|
930
|
+
- Regex crash on undefined `.replace()`
|
|
931
|
+
|
|
932
|
+
## [1.0.16] - 2025-04-29
|
|
933
|
+
|
|
934
|
+
### Added
|
|
935
|
+
- Fingerprinting support — [#7](https://github.com/ryanbr/network-scanner/issues/7)
|
|
936
|
+
- Multiple URL support and `--no-interact`
|
|
937
|
+
- HTML source output — [#12](https://github.com/ryanbr/network-scanner/issues/12)
|
|
938
|
+
|
|
939
|
+
### Fixed
|
|
940
|
+
- Execution context destroyed crash in Puppeteer
|
|
941
|
+
|
|
942
|
+
## [1.0.15] - 2025-04-28
|
|
943
|
+
|
|
944
|
+
### Added
|
|
945
|
+
- Localhost JSON configs — [#11](https://github.com/ryanbr/network-scanner/issues/11)
|
|
946
|
+
|
|
947
|
+
## [1.0.14] - 2025-04-27
|
|
948
|
+
|
|
949
|
+
### Added
|
|
950
|
+
- SubDomains support
|
|
951
|
+
- Delay option — [#8](https://github.com/ryanbr/network-scanner/issues/8)
|
|
952
|
+
- UserAgent support — [#6](https://github.com/ryanbr/network-scanner/issues/6)
|
|
953
|
+
- Mouse interaction — [#5](https://github.com/ryanbr/network-scanner/issues/5)
|
|
954
|
+
|
|
955
|
+
### Fixed
|
|
956
|
+
- Blocked JSON requests — [#4](https://github.com/ryanbr/network-scanner/issues/4)
|
|
957
|
+
- Subdomain and localhost output
|
|
958
|
+
|
|
959
|
+
## [1.0.0] - 2025-04-27
|
|
960
|
+
|
|
961
|
+
### Added
|
|
962
|
+
- Initial release of network scanner
|
|
963
|
+
- Puppeteer-based browser automation for network request analysis
|
|
964
|
+
- JSON configuration for site-specific scanning rules
|
|
965
|
+
- Regex-based URL matching with domain extraction
|
|
966
|
+
- First-party/third-party request classification
|
|
967
|
+
- Multiple output formats (hosts, adblock)
|
|
968
|
+
- `--dumpurls` matched URL logging
|
|
969
|
+
- `--debug` mode
|
|
970
|
+
- `--localhost` format output
|