@fanboynz/network-scanner 2.0.58 → 2.0.60

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,970 @@
1
+ # Changelog
2
+
3
+ All notable changes to the Network Scanner (nwss.js) project.
4
+
5
+ ## [2.0.60] - 2026-03-16
6
+
7
+ ### Added
8
+ - `--dns-cache` flag for persistent dig/whois disk caching between runs (`.digcache`, `.whoiscache`)
9
+ - `--load-extension <path>` flag to load unpacked Chrome extensions (supports multiple)
10
+ - `--block-ads` now supports comma-separated list files (`--block-ads=easylist.txt,easyprivacy.txt`)
11
+ - `disable_ad_tagging` config option to control Chrome AdTagging (default: true)
12
+ - DNS cache hit/miss statistics in scan summary output with fresh domain names listed
13
+ - Concurrent dig/whois deduplication — multiple pages requesting the same domain share one lookup
14
+ - SIGINT/SIGTERM handlers for `--keep-open` to prevent orphaned Chrome processes
15
+
16
+ ### Fixed
17
+ - Adblock pipe (`|`) character handling — mid-pattern pipes were incorrectly treated as anchors, causing broad false positives on EasyList rules like `/addyn|*|adtech;`
18
+ - Domain Map fast path was skipping resource type checks — `$ping`, `$script` etc. now correctly enforced
19
+ - Domain extraction for `||domain.com/path` rules — path was incorrectly included in domain name
20
+ - `--keep-open` now skips extension-blocking Chrome flags so Chrome Web Store and extensions work
21
+ - Corrupt disk cache files are deleted instead of persisted
22
+ - `getBaseDomain()` now uses `psl` for correct multi-part TLD handling (`.co.uk`, `.com.au`)
23
+ - Merged 7 separate `--disable-features` flags into one — Chrome only reads the last occurrence
24
+
25
+ ### Improved
26
+ - `$document` rules treated as full domain blocks (matches all resource types)
27
+ - `adblock.js`: regex cache for compiled patterns, Set for resource type lookups, lazy parentDomains, two-level result cache with LRU eviction (32K), hoisted constants, freed parsed options after rule parsing
28
+ - `output.js`: capped wildcard regex cache at 500, simplified `*.domain.com` suffix matching, hoisted resource type map
29
+ - `compare.js`: pre-compiled and deduplicated 6 normalization regexes
30
+ - `grep.js`: build grep args once outside pattern loop
31
+ - `domain-cache.js`: use Set iterator for eviction instead of full array copy
32
+ - `nettools.js`: hoisted ANSI strip regex, disk cache flushes once on exit instead of per-lookup
33
+ - Dig/whois cache: 14-hour TTL, 1000 entry limit, pretty-printed JSON files
34
+
35
+ ## [2.0.59] - 2026-03-15
36
+
37
+ ### Added
38
+ - `--keep-open` flag to keep browser and all tabs open after scan completes (use with `--headful` for debugging)
39
+ - `--use-puppeteer-core` flag to use `puppeteer-core` with system Chrome instead of bundled Chromium
40
+ - `puppeteer-core` as optional dependency in package.json
41
+ - Ghost-cursor integration for Bezier-based mouse movements (`--ghost-cursor` flag)
42
+ - Help text entries for `--keep-open`, `--use-puppeteer-core`
43
+
44
+ ### Fixed
45
+ - Simulated mouse events now include `pageX`/`pageY`/`screenX`/`screenY` properties — scripts reading `event.pageX`/`pageY` for bot detection (e.g. dkitac.js) previously saw zero movement
46
+ - Stale comment reference to removed function
47
+ - CDP timeout leaks and dead code in `cdp.js`
48
+
49
+ ### Improved
50
+ - Mouse interaction runs concurrently with post-load delay for better performance
51
+ - `maxTouchPoints` hardcoded to 0 for desktop Linux Chrome consistency
52
+
53
+ ## [2.0.58] - 2026-03-14
54
+
55
+ ### Fixed
56
+ - Race condition: re-check `isProcessing` before `page.close()` in realtime cleanup
57
+ - Page tracker stale entries during concurrent execution (added `untrackPage()`)
58
+ - ElementHandle leak in `interaction.js` — dispose body handle in `finally` block
59
+
60
+ ### Improved
61
+ - macOS compatibility: add Chrome path detection and use `os.tmpdir()` for cross-platform temp dirs
62
+ - Harden `interaction.js` with page lifecycle checks to prevent mid-close errors
63
+ - Fingerprint interaction-gated trigger with scroll/keydown events and readyState check
64
+ - Low-impact optimisations across 6 modules (grep, flowproxy, dry-run, adblock, interaction, openvpn_vpn)
65
+ - Remove redundant `fs.existsSync()` guards in openvpn_vpn.js, compress.js, compare.js, validate_rules.js, output.js
66
+ - Hoist regex constants in `validate_rules.js`, cache wildcard regex in `output.js`
67
+ - Optimise `browserexit.js`: replace shell spawns with native fs operations
68
+ - Deduplicate session-closed error checks in `fingerprint.js`
69
+ - Remove dead code (`performMinimalInteraction`, unused `filteredArgs`)
70
+ - Migrate `.clauderc` to `CLAUDE.md`
71
+
72
+ ## [2.0.57] - 2026-03-14
73
+
74
+ ### Improved
75
+ - Optimise `ignore_similar.js`
76
+
77
+ ## [2.0.56] - 2026-03-13
78
+
79
+ ### Fixed
80
+ - Cloudflare challenge/solver scanning issues
81
+ - Browser health monitoring improvements
82
+
83
+ ### Improved
84
+ - Cloudflare detection reliability and performance
85
+ - Chrome/Puppeteer performance tuning
86
+ - Smart cache optimisations in `smart-cache.js`
87
+ - Post-processing optimisations
88
+
89
+ ## [2.0.55] - 2026-03-12
90
+
91
+ ### Fixed
92
+ - Browser cleanup missing `com.google.Chrome` temp files
93
+ - Interaction.js reload interaction issues
94
+
95
+ ### Improved
96
+ - Fingerprint.js improvements
97
+ - Interaction.js cleanup
98
+
99
+ ## [2.0.54] - 2026-03-11
100
+
101
+ ### Improved
102
+ - WebGL fingerprinting improvements, revert to `--disable-gpu`
103
+ - Reduce DIG and Whois request volume with domain caching
104
+ - Update user agents
105
+
106
+ ## [2.0.53] - 2026-03-10
107
+
108
+ ### Fixed
109
+ - Headless/GPU crash issues
110
+ - Fingerprint protection hardening
111
+
112
+ ### Added
113
+ - Screenshot support using `force` option
114
+
115
+ ### Improved
116
+ - Fingerprint protection improvements
117
+
118
+ ## [2.0.52] - 2026-03-10
119
+
120
+ ### Fixed
121
+ - Headless/GPU crash and fingerprint improvements
122
+
123
+ ## [2.0.51] - 2026-02-24
124
+
125
+ ### Added
126
+ - SOCKS/HTTP/HTTPS proxy support (`proxy.js`)
127
+
128
+ ### Improved
129
+ - Update packages
130
+ - Compatibility improvements
131
+
132
+ ## [2.0.50] - 2026-02-17
133
+
134
+ ### Fixed
135
+ - Fingerprint `random` mode improvements
136
+ - CDP round-trips reduced to 1, cache bodyText
137
+ - `safeClick`/`safeWaitForNavigation` timeout leaks
138
+ - Redundant context validation removed
139
+ - Shadowroot compatibility on `cloudflare.js`
140
+
141
+ ### Improved
142
+ - `interact: true` performance
143
+ - Canvas noise optimisation for large canvases
144
+ - Fingerprint consistency fixes (mousemove WeakMap, human simulation timing)
145
+ - `measureText` read-only property fix
146
+ - Support for larger lists
147
+ - `ignoreDomains` improvements
148
+ - Hot path performance optimisations (indexed loops, single-pass regex matching, URL parsing)
149
+ - Adblock domain matcher precomputation
150
+
151
+ ## [2.0.49] - 2026-02-17
152
+
153
+ ### Improved
154
+ - Fingerprint protection `random` mode enhancements
155
+
156
+ ## [2.0.48] - 2026-02-17
157
+
158
+ ### Improved
159
+ - Adblock rule parser: V8 optimisations, cached hostname split, Map-based lookups
160
+ - Precompute parent domains for whitelist and block checks
161
+ - Remove dead code in `grep.js`
162
+
163
+ ## [2.0.47] - 2026-02-17
164
+
165
+ ### Added
166
+ - Support for `$counter` adblock rules
167
+ - Support for `$1p`, `$~third-party`, `$first-party` adblock options
168
+
169
+ ### Fixed
170
+ - Missing variable fix
171
+ - More adblock rule format support
172
+
173
+ ## [2.0.46] - 2026-02-16
174
+
175
+ ### Fixed
176
+ - Potential memory leaks
177
+ - Timing range miscalculation
178
+ - `TEXT_PREVIEW_LENGTH` unreachable inside `page.evaluate()`
179
+ - Unused variables cleanup
180
+
181
+ ### Improved
182
+ - Processing termination to avoid stale processes
183
+
184
+ ## [2.0.45] - 2026-02-16
185
+
186
+ ### Improved
187
+ - Processing termination reliability
188
+
189
+ ## [2.0.44] - 2026-02-16
190
+
191
+ ### Added
192
+ - OpenVPN support (`openvpn_vpn.js`) — [#45](https://github.com/ryanbr/network-scanner/issues/45)
193
+
194
+ ## [2.0.43] - 2026-02-16
195
+
196
+ ### Added
197
+ - Initial WireGuard VPN support (`wireguard_vpn.js`) — [#45](https://github.com/ryanbr/network-scanner/issues/45)
198
+
199
+ ## [2.0.42] - 2026-02-16
200
+
201
+ ### Fixed
202
+ - `maxTouchPoints` potentially overridden twice
203
+ - Duplicate `console.error` overrides in fingerprint
204
+ - `hardwareConcurrency` returning different values on every read
205
+ - Brave UA getter infinite recursion
206
+
207
+ ### Improved
208
+ - General cleanups and unused function removal
209
+
210
+ ## [2.0.41] - 2026-02-16
211
+
212
+ ### Fixed
213
+ - Binary issue with `smart-cache.js`
214
+
215
+ ## [2.0.40] - 2026-02-16
216
+
217
+ ### Fixed
218
+ - Missing `requestCache` in smart-cache clear/destroy
219
+ - Duplicate `totalCacheEntries` in `getStats`
220
+ - Undefined `forceDebug` reference in `cacheRequest`
221
+ - Missing `normalizedUrl` declaration in `cacheRequest`
222
+
223
+ ## [2.0.39] - 2026-02-16
224
+
225
+ ### Improved
226
+ - Nettools: buffered log writer instead of `fs.appendFileSync`
227
+
228
+ ## [2.0.38] - 2026-02-16
229
+
230
+ ### Fixed
231
+ - Catch-and-rethrow doing nothing in nettools
232
+ - Double timeout in `createNetToolsHandler`
233
+
234
+ ### Improved
235
+ - Replace global whois server index with module-level variable
236
+ - Hoist `execSync` and move `tldServers` to module scope
237
+
238
+ ## [2.0.37] - 2026-02-16
239
+
240
+ ### Improved
241
+ - Browser health: store timestamp in page creation tracker
242
+ - Cleanup `formatMemory` redefinition
243
+ - Hoist `require('child_process')` in `checkBrowserMemory`
244
+ - Replace `Page.prototype` monkey-patch with explicit tracker cleanup
245
+
246
+ ## [2.0.36] - 2026-02-16
247
+
248
+ ### Improved
249
+ - `browserexit.js`: remove duplicate pattern, hoist requires
250
+
251
+ ## [2.0.35] - 2026-02-16
252
+
253
+ ### Improved
254
+ - Buffer log writes, pre-compile regexes, deduplicate request handler
255
+
256
+ ## [2.0.34] - 2026-02-16
257
+
258
+ ### Improved
259
+ - General cleanup
260
+
261
+ ## [2.0.33] - 2025-11-14
262
+
263
+ ### Added
264
+ - Adblock list support for blocking URLs during scanning
265
+ - V8 optimised adblock parser with LRU cache and Map-based domain lookups
266
+
267
+ ### Improved
268
+ - Bump Firefox user agent
269
+ - Rename `adblock_rules.js` to `adblock.js`
270
+
271
+ ## [2.0.32] - 2025-11-08
272
+
273
+ ### Fixed
274
+ - Race conditions: atomic `checkAndMark()` in domain cache
275
+ - Performance improvements and V8 optimisations
276
+
277
+ ### Improved
278
+ - `referrer.js` V8 optimisations
279
+ - Update packages
280
+
281
+ ## [2.0.31] - 2025-10-31
282
+
283
+ ### Added
284
+ - `referrer_disable` support
285
+ - `referrer_headers` support
286
+
287
+ ### Fixed
288
+ - `url is not defined` errors
289
+ - Referrer.js incorrectly added to nwss.js
290
+ - Page state checks before reload, network idle, CSS blocking evaluation
291
+
292
+ ### Improved
293
+ - `grep.js` improvements
294
+
295
+ ## [2.0.30] - 2025-10-29
296
+
297
+ ### Added
298
+ - Location URL masking
299
+ - Additional automation property hiding
300
+
301
+ ### Improved
302
+ - Font enumeration protection
303
+ - Fingerprint platform matching
304
+ - Bump Chrome to 142.x
305
+
306
+ ## [2.0.29] - 2025-10-21
307
+
308
+ ### Improved
309
+ - Hang check loop and browser restart on hang
310
+ - Chrome launch arguments
311
+ - Permissions API fingerprinting
312
+ - Realistic Chrome browser behaviour simulation
313
+ - Chrome runtime simulation strengthening
314
+
315
+ ## [2.0.28] - 2025-10-11
316
+
317
+ ### Improved
318
+ - Page method caching optimisations
319
+ - Consistent return objects in health checks
320
+ - CDP.js V8 optimisations
321
+ - Bump overall timeout from 30s to 65s
322
+ - Nettools optimisations
323
+
324
+ ## [2.0.27] - 2025-10-07
325
+
326
+ ### Improved
327
+ - Whois retry on TIMEOUT/FAIL to avoid throttling
328
+
329
+ ## [2.0.26] - 2025-10-06
330
+
331
+ ### Improved
332
+ - V8 optimisations: `Object.freeze()`, destructuring, pre-allocated arrays, Maps
333
+ - Bump Chrome version
334
+
335
+ ## [2.0.25] - 2025-10-05
336
+
337
+ ### Fixed
338
+ - Frame handling `frameUrl is not defined` errors
339
+ - `activeFrames.add is not a function`
340
+ - `spoofNavigatorProperties is not defined`
341
+
342
+ ### Improved
343
+ - Frame URL improvements
344
+ - Allow grep without curl
345
+
346
+ ## [2.0.24] - 2025-10-04
347
+
348
+ ### Improved
349
+ - Fingerprint.js V8 performance: pre-compiled mocks, monomorphic object shapes, cached descriptors
350
+ - Address [#41](https://github.com/ryanbr/network-scanner/issues/41)
351
+
352
+ ## [2.0.23] - 2025-10-01
353
+
354
+ ### Improved
355
+ - Whois retry enabled by default with tuned retries/delay
356
+
357
+ ## [2.0.22] - 2025-10-01
358
+
359
+ ### Added
360
+ - `--dry-run` split into separate module (`dry-run.js`)
361
+
362
+ ## [2.0.21] - 2025-09-30
363
+
364
+ ### Added
365
+ - Domain-based `forcereload` support (`forcereload=domain.com,domain2.com`)
366
+ - Input validation and domain cleaning for forcereload
367
+
368
+ ### Improved
369
+ - Update man page and `--help` args
370
+
371
+ ## [2.0.20] - 2025-09-29
372
+
373
+ ### Improved
374
+ - `--localhost` now configurable (`--localhost=x.x.x.x`)
375
+
376
+ ## [2.0.19] - 2025-09-27
377
+
378
+ ### Improved
379
+ - `--remove-dupes` reliability
380
+
381
+ ## [2.0.18] - 2025-09-27
382
+
383
+ ### Fixed
384
+ - Whois logic occasionally missing records
385
+
386
+ ## [2.0.17] - 2025-09-27
387
+
388
+ ### Improved
389
+ - `window_cleanup` realtime less aggressive, added validation checks
390
+
391
+ ## [2.0.16] - 2025-09-25
392
+
393
+ ### Fixed
394
+ - `tar-fs` security vulnerability
395
+
396
+ ## [2.0.15] - 2025-09-25
397
+
398
+ ### Improved
399
+ - Font, canvas, WebGL, permission, hardware concurrency, plugin fingerprinting
400
+
401
+ ## [2.0.14] - 2025-09-24
402
+
403
+ ### Improved
404
+ - Fingerprinting updates
405
+ - Bump Firefox version
406
+ - Wrap errors in `--debug`
407
+
408
+ ## [2.0.13] - 2025-09-24
409
+
410
+ ### Improved
411
+ - Bump Firefox version
412
+
413
+ ## [2.0.12] - 2025-09-23
414
+
415
+ ### Fixed
416
+ - Navigator.brave checks
417
+ - Fingerprint.js error handling
418
+
419
+ ## [2.0.11] - 2025-09-23
420
+
421
+ ### Improved
422
+ - Bump timeouts, make delay a const
423
+
424
+ ## [2.0.10] - 2025-09-23
425
+
426
+ ### Fixed
427
+ - Occasional detach issues during scanning
428
+
429
+ ## [2.0.9] - 2025-09-21
430
+
431
+ ### Added
432
+ - `cdp_specific` support for per-URL CDP without global `cdp: true`
433
+
434
+ ## [2.0.8] - 2025-09-20
435
+
436
+ ### Added
437
+ - User agents for Linux and macOS
438
+
439
+ ## [2.0.7] - 2025-09-20
440
+
441
+ ### Added
442
+ - `clear_sitedata.js` for CDP fixes
443
+
444
+ ### Improved
445
+ - Bump Cloudflare version
446
+
447
+ ## [2.0.6] - 2025-09-19
448
+
449
+ ### Improved
450
+ - CDP.js reliability with retry support
451
+
452
+ ## [2.0.5] - 2025-09-19
453
+
454
+ ### Fixed
455
+ - Race condition with `window_cleanup=realtime` and Cloudflare
456
+
457
+ ## [2.0.4] - 2025-09-17
458
+
459
+ ### Improved
460
+ - Cloudflare.js v2.6.1
461
+
462
+ ## [2.0.3] - 2025-09-17
463
+
464
+ ### Fixed
465
+ - Frame detach errors — [#38](https://github.com/ryanbr/network-scanner/issues/38)
466
+
467
+ ### Improved
468
+ - Cloudflare.js v2.6.0
469
+
470
+ ## [2.0.2] - 2025-09-15
471
+
472
+ ### Improved
473
+ - Cloudflare.js v2.5.0
474
+
475
+ ## [2.0.1] - 2025-09-15
476
+
477
+ ### Fixed
478
+ - Pi-hole regex slash handling
479
+ - Allow latest Puppeteer version
480
+
481
+ ## [2.0.0] - 2025-09-15
482
+
483
+ ### Changed
484
+ - Major version bump — Puppeteer compatibility and architecture updates
485
+
486
+ ## [1.0.99] - 2025-09-13
487
+
488
+ ### Improved
489
+ - Bump user agents
490
+ - Increase browser health thresholds
491
+
492
+ ## [1.0.98] - 2025-09-09
493
+
494
+ ### Added
495
+ - Realtime `window_cleanup` for larger URL lists
496
+
497
+ ## [1.0.97] - 2025-09-06
498
+
499
+ ### Added
500
+ - `window_cleanup` to close old tabs, releasing memory on larger URL lists
501
+
502
+ ## [1.0.96] - 2025-09-05
503
+
504
+ ### Improved
505
+ - CDP timeout improvements
506
+
507
+ ## [1.0.95] - 2025-09-05
508
+
509
+ ### Fixed
510
+ - Persistent failure recovery — move to next URL instead of error
511
+
512
+ ## [1.0.94] - 2025-09-04
513
+
514
+ ### Fixed
515
+ - ForceReload fallback for Puppeteer v23.x compatibility
516
+
517
+ ## [1.0.93] - 2025-09-03
518
+
519
+ ### Improved
520
+ - Health checks and fallback on `evaluateOnNewDocument` failure
521
+
522
+ ## [1.0.92] - 2025-09-03
523
+
524
+ ### Improved
525
+ - Interaction.js tweaks
526
+
527
+ ## [1.0.91] - 2025-09-03
528
+
529
+ ### Improved
530
+ - Minor version bumps
531
+
532
+ ## [1.0.88] - 2025-09-01
533
+
534
+ ### Added
535
+ - Split curl functions from `grep.js` — [#33](https://github.com/ryanbr/network-scanner/issues/33)
536
+
537
+ ### Improved
538
+ - Cloudflare.js v2.4.1
539
+
540
+ ## [1.0.86] - 2025-08-31
541
+
542
+ ### Fixed
543
+ - Puppeteer 24.x compatibility and browser health issues — [#28](https://github.com/ryanbr/network-scanner/issues/28)
544
+
545
+ ## [1.0.85] - 2025-08-31
546
+
547
+ ### Improved
548
+ - Post-processing first-party item checks
549
+
550
+ ## [1.0.83] - 2025-08-30
551
+
552
+ ### Fixed
553
+ - ForceReload logic to apply after each reload
554
+
555
+ ## [1.0.82] - 2025-08-29
556
+
557
+ ### Improved
558
+ - Cloudflare.js v2.4
559
+
560
+ ## [1.0.81] - 2025-08-28
561
+
562
+ ### Fixed
563
+ - Endless loops caused by some sites
564
+
565
+ ## [1.0.80] - 2025-08-27
566
+
567
+ ### Fixed
568
+ - Performance issues with interact and resource cleanup
569
+
570
+ ## [1.0.78] - 2025-08-27
571
+
572
+ ### Added
573
+ - INSTALL suggestions
574
+
575
+ ## [1.0.77] - 2025-08-26
576
+
577
+ ### Improved
578
+ - Cloudflare.js v2.3
579
+
580
+ ## [1.0.76] - 2025-08-21
581
+
582
+ ### Added
583
+ - Cached network requests for duplicate URLs in same JSON
584
+
585
+ ### Fixed
586
+ - Duplicate function removal
587
+
588
+ ## [1.0.75] - 2025-08-19
589
+
590
+ ### Fixed
591
+ - Nettools not firing
592
+
593
+ ## [1.0.74] - 2025-08-19
594
+
595
+ ### Fixed
596
+ - Nettools being ignored
597
+
598
+ ## [1.0.73] - 2025-08-19
599
+
600
+ ### Added
601
+ - `regex_and` to apply AND logic on filterRegex
602
+
603
+ ## [1.0.72] - 2025-08-18
604
+
605
+ ### Improved
606
+ - Cloudflare.js v2.2
607
+
608
+ ## [1.0.70] - 2025-08-17
609
+
610
+ ### Improved
611
+ - Regex tool GitHub compatibility
612
+
613
+ ## [1.0.69] - 2025-08-17
614
+
615
+ ### Improved
616
+ - Convert magic numbers to constants in nwss.js
617
+
618
+ ## [1.0.68] - 2025-08-15
619
+
620
+ ### Fixed
621
+ - URL popup protection — don't treat main URL changes as third-party
622
+
623
+ ## [1.0.67] - 2025-08-14
624
+
625
+ ### Fixed
626
+ - `third-party: true` never matches root URL
627
+
628
+ ## [1.0.66] - 2025-08-12
629
+
630
+ ### Improved
631
+ - Interaction.js performance
632
+ - Fingerprint.js refactor
633
+ - Puppeteer 23 compatibility
634
+
635
+ ## [1.0.63] - 2025-08-11
636
+
637
+ ### Fixed
638
+ - Occasional interaction.js delays
639
+ - Security vulnerabilities in `tar-fs` and `ws`
640
+
641
+ ### Improved
642
+ - Puppeteer 23.x support
643
+
644
+ ## [1.0.60] - 2025-08-10
645
+
646
+ ### Changed
647
+ - Pin to Puppeteer 20.x for stability
648
+
649
+ ## [1.0.59] - 2025-08-08
650
+
651
+ ### Improved
652
+ - Searchstring improvements
653
+ - Update dependencies for Node.js 20+
654
+
655
+ ## [1.0.58] - 2025-08-08
656
+
657
+ ### Added
658
+ - `--clear-cache` / `--ignore-cache` options
659
+
660
+ ### Improved
661
+ - Smart cache memory management
662
+
663
+ ## [1.0.57] - 2025-08-06
664
+
665
+ ### Added
666
+ - Smart caching system (`smart-cache.js`)
667
+
668
+ ## [1.0.53] - 2025-08-06
669
+
670
+ ### Added
671
+ - Automated npm publishing workflow
672
+
673
+ ## [1.0.49] - 2025-08-04
674
+
675
+ ### Added
676
+ - ESLint configuration
677
+
678
+ ### Improved
679
+ - CDP functionality separated into own module
680
+
681
+ ## [1.0.47] - 2025-08-03
682
+
683
+ ### Improved
684
+ - Mouse simulator made more modular
685
+ - Cloudflare and FlowProxy skip non-HTTP URLs
686
+
687
+ ### Fixed
688
+ - Regression on `subDomains=1`
689
+
690
+ ## [1.0.46] - 2025-08-01
691
+
692
+ ### Improved
693
+ - Skip previously detected domains
694
+ - Magic numbers converted to constants
695
+ - Cloudflare.js documentation
696
+
697
+ ## [1.0.45] - 2025-07-31
698
+
699
+ ### Added
700
+ - Whois and dig result caching
701
+
702
+ ### Improved
703
+ - Dig/nettools with multiple URLs
704
+
705
+ ## [1.0.44] - 2025-07-30
706
+
707
+ ### Added
708
+ - User-configurable `maxConcurrentSites` and `cleanup-interval`
709
+
710
+ ## [1.0.43] - 2025-07-20
711
+
712
+ ### Fixed
713
+ - Browser restart on `protocolTimeout`
714
+
715
+ ### Improved
716
+ - Cloudflare wait times and timeouts
717
+
718
+ ## [1.0.42] - 2025-07-16
719
+
720
+ ### Added
721
+ - Referrer options support
722
+
723
+ ### Improved
724
+ - Redirecting domains compatibility
725
+ - Fingerprint.js improvements
726
+
727
+ ## [1.0.41] - 2025-07-14
728
+
729
+ ### Added
730
+ - `ignore_similar` domains feature
731
+
732
+ ## [1.0.40] - 2025-07-02
733
+
734
+ ### Added
735
+ - `even_blocked` option
736
+
737
+ ### Fixed
738
+ - Puppeteer old headless deprecation warnings
739
+
740
+ ### Improved
741
+ - Domain validation — [#27](https://github.com/ryanbr/network-scanner/issues/27)
742
+ - `--append` output support
743
+
744
+ ## [1.0.39] - 2025-06-24
745
+
746
+ ### Added
747
+ - `--dry-run` option with file output
748
+
749
+ ### Improved
750
+ - `ignoreDomains` fallback removal
751
+
752
+ ## [1.0.38] - 2025-06-21
753
+
754
+ ### Fixed
755
+ - First-party/third-party and ignoreDomains prioritisation
756
+
757
+ ## [1.0.37] - 2025-06-17
758
+
759
+ ### Added
760
+ - `--remove-tempfiles` option
761
+
762
+ ## [1.0.36] - 2025-06-16
763
+
764
+ ### Added
765
+ - FlowProxy DDoS protection support — [#24](https://github.com/ryanbr/network-scanner/issues/24)
766
+
767
+ ### Improved
768
+ - Browser health checks and restart on degradation
769
+ - Chrome process killing
770
+ - Insecure site loading support
771
+
772
+ ## [1.0.35] - 2025-06-15
773
+
774
+ ### Added
775
+ - Whois and dig debug file output with ANSI stripping
776
+
777
+ ### Improved
778
+ - Whois reliability
779
+
780
+ ## [1.0.34] - 2025-06-13
781
+
782
+ ### Fixed
783
+ - Out-of-space issues from `puppeteer_dev_chrome_profile` temp files
784
+ - Error handling crash
785
+ - `about:srcdoc`, `data:`, `about:`, `chrome:`, `blob:` URL handling — [#21](https://github.com/ryanbr/network-scanner/issues/21)
786
+
787
+ ## [1.0.33] - 2025-06-11
788
+
789
+ ### Added
790
+ - Pi-hole output format (`--pihole`)
791
+ - Privoxy output format
792
+ - Comments value in JSON config
793
+
794
+ ## [1.0.32] - 2025-06-10
795
+
796
+ ### Added
797
+ - `whois_server_mode` (random/cycle)
798
+ - Configurable whois delay
799
+ - Whois error logging to `logs/debug`
800
+ - Coloured console output
801
+
802
+ ### Improved
803
+ - Browser detection with custom userAgent
804
+
805
+ ## [1.0.31] - 2025-06-09
806
+
807
+ ### Changed
808
+ - Rename `scanner-script.js` to `nwss.js`
809
+
810
+ ## [1.0.30] - 2025-06-08
811
+
812
+ ### Added
813
+ - Searchstring AND logic
814
+ - Unbound, DNSMasq output formats
815
+
816
+ ### Fixed
817
+ - Iframe debug errors
818
+
819
+ ## [1.0.29] - 2025-06-06
820
+
821
+ ### Added
822
+ - Custom whois servers with retry/fallback
823
+
824
+ ### Improved
825
+ - WSL compatibility
826
+
827
+ ## [1.0.28] - 2025-06-05
828
+
829
+ ### Added
830
+ - Global blocked domains support
831
+ - `goto_options` config
832
+
833
+ ### Improved
834
+ - Scanning method improvements
835
+
836
+ ## [1.0.27] - 2025-06-04
837
+
838
+ ### Added
839
+ - `--compare` with `--titles` support — [#1](https://github.com/ryanbr/network-scanner/issues/1)
840
+ - `--remove-dupes` alias
841
+
842
+ ### Improved
843
+ - Resource management with service restarts
844
+
845
+ ## [1.0.26] - 2025-06-02
846
+
847
+ ### Added
848
+ - Whois/dig support — [#18](https://github.com/ryanbr/network-scanner/issues/18)
849
+ - `--debug` and `--dumpurls` file output
850
+
851
+ ## [1.0.25] - 2025-05-31
852
+
853
+ ### Added
854
+ - Curl and grep alternative scan method
855
+ - Adblock rules output format
856
+
857
+ ### Improved
858
+ - Cloudflare bypass split to own module
859
+ - Output split to `output.js`
860
+ - Fingerprinting split to own module
861
+
862
+ ## [1.0.24] - 2025-05-30
863
+
864
+ ### Added
865
+ - Searchstring support (search within regex-matched content)
866
+ - `--remove-dupes` on output
867
+ - Wildcard support in ignored domains
868
+
869
+ ## [1.0.23] - 2025-05-27
870
+
871
+ ### Added
872
+ - `--debug` logging improvements
873
+
874
+ ### Improved
875
+ - Graceful exit handling
876
+ - Module split: CDP, interact, evaluateOnNewDocument, Cloudflare, CSS blocking, fingerprint
877
+
878
+ ## [1.0.22] - 2025-05-26
879
+
880
+ ### Added
881
+ - Cloudflare phishing warning bypass
882
+ - CSS blocking support — [#2](https://github.com/ryanbr/network-scanner/issues/2)
883
+
884
+ ### Improved
885
+ - Concurrent site scanning resource management
886
+
887
+ ## [1.0.21] - 2025-05-23
888
+
889
+ ### Added
890
+ - Multithread/concurrent support
891
+ - CDP logging improvements
892
+ - Address [#14](https://github.com/ryanbr/network-scanner/issues/14), [#15](https://github.com/ryanbr/network-scanner/issues/15)
893
+
894
+ ## [1.0.20] - 2025-05-21
895
+
896
+ ### Added
897
+ - Per-site verbose output with matching regex
898
+ - Scan timer
899
+ - Scan counter
900
+
901
+ ### Improved
902
+ - First-party/third-party detection
903
+
904
+ ## [1.0.19] - 2025-05-19
905
+
906
+ ### Added
907
+ - `package.json` for npm
908
+
909
+ ### Fixed
910
+ - Sandboxing issue on Linux
911
+
912
+ ## [1.0.18] - 2025-05-03
913
+
914
+ ### Added
915
+ - JSON manual (`JSONMANUAL.md`)
916
+
917
+ ### Improved
918
+ - Scanner methods — [#3](https://github.com/ryanbr/network-scanner/issues/3)
919
+ - `--plain` unformatted domain output
920
+ - Global blocked items
921
+
922
+ ## [1.0.17] - 2025-05-01
923
+
924
+ ### Added
925
+ - Headful browser mode
926
+ - Screenshot option for debugging
927
+ - Custom JSON file support
928
+
929
+ ### Fixed
930
+ - Regex crash on undefined `.replace()`
931
+
932
+ ## [1.0.16] - 2025-04-29
933
+
934
+ ### Added
935
+ - Fingerprinting support — [#7](https://github.com/ryanbr/network-scanner/issues/7)
936
+ - Multiple URL support and `--no-interact`
937
+ - HTML source output — [#12](https://github.com/ryanbr/network-scanner/issues/12)
938
+
939
+ ### Fixed
940
+ - Execution context destroyed crash in Puppeteer
941
+
942
+ ## [1.0.15] - 2025-04-28
943
+
944
+ ### Added
945
+ - Localhost JSON configs — [#11](https://github.com/ryanbr/network-scanner/issues/11)
946
+
947
+ ## [1.0.14] - 2025-04-27
948
+
949
+ ### Added
950
+ - SubDomains support
951
+ - Delay option — [#8](https://github.com/ryanbr/network-scanner/issues/8)
952
+ - UserAgent support — [#6](https://github.com/ryanbr/network-scanner/issues/6)
953
+ - Mouse interaction — [#5](https://github.com/ryanbr/network-scanner/issues/5)
954
+
955
+ ### Fixed
956
+ - Blocked JSON requests — [#4](https://github.com/ryanbr/network-scanner/issues/4)
957
+ - Subdomain and localhost output
958
+
959
+ ## [1.0.0] - 2025-04-27
960
+
961
+ ### Added
962
+ - Initial release of network scanner
963
+ - Puppeteer-based browser automation for network request analysis
964
+ - JSON configuration for site-specific scanning rules
965
+ - Regex-based URL matching with domain extraction
966
+ - First-party/third-party request classification
967
+ - Multiple output formats (hosts, adblock)
968
+ - `--dumpurls` matched URL logging
969
+ - `--debug` mode
970
+ - `--localhost` format output