@isdk/web-searcher 0.1.5 → 0.1.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. package/README.cn.md +40 -0
  2. package/README.md +40 -0
  3. package/dist/index.d.mts +14 -0
  4. package/dist/index.d.ts +14 -0
  5. package/dist/index.js +1 -1
  6. package/dist/index.mjs +1 -1
  7. package/docs/README.md +40 -0
  8. package/docs/classes/GoogleSearcher.md +150 -48
  9. package/docs/classes/WebSearcher.md +138 -48
  10. package/docs/functions/extractDate.md +1 -1
  11. package/docs/functions/extractMetadataFrom.md +1 -1
  12. package/docs/functions/fetchHeaders.md +1 -1
  13. package/docs/functions/fetchPartial.md +1 -1
  14. package/docs/functions/normalizeDate.md +1 -1
  15. package/docs/functions/parseHeaders.md +1 -1
  16. package/docs/functions/parseHtml.md +1 -1
  17. package/docs/functions/testUrlsByLatency.md +5 -1
  18. package/docs/interfaces/CustomTimeRange.md +3 -3
  19. package/docs/interfaces/ExtractOptions.md +4 -4
  20. package/docs/interfaces/FetchExtractorOptions.md +3 -3
  21. package/docs/interfaces/FetcherOptions.md +41 -29
  22. package/docs/interfaces/HtmlData.md +4 -4
  23. package/docs/interfaces/MetadataResult.md +2 -2
  24. package/docs/interfaces/PaginationConfig.md +7 -7
  25. package/docs/interfaces/SearchContext.md +6 -6
  26. package/docs/interfaces/SearchOptions.md +13 -13
  27. package/docs/interfaces/StandardSearchResult.md +10 -10
  28. package/docs/interfaces/VerifiedUrl.md +3 -3
  29. package/docs/type-aliases/MetadataType.md +1 -1
  30. package/docs/type-aliases/SafeSearchLevel.md +1 -1
  31. package/docs/type-aliases/SearchCategory.md +1 -1
  32. package/docs/type-aliases/SearchTimeRange.md +1 -1
  33. package/docs/type-aliases/SearchTimeRangePreset.md +1 -1
  34. package/docs/type-aliases/SearcherConstructor.md +1 -1
  35. package/package.json +2 -2
@@ -6,7 +6,7 @@
6
6
 
7
7
  # Abstract Class: WebSearcher
8
8
 
9
- Defined in: [web-searcher/src/searcher.ts:32](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L32)
9
+ Defined in: [web-searcher/src/searcher.ts:32](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L32)
10
10
 
11
11
  The abstract base class for all search engines.
12
12
 
@@ -41,7 +41,7 @@ WebSearcher.register(MySearcher);
41
41
 
42
42
  > **new WebSearcher**(`options?`): `WebSearcher`
43
43
 
44
- Defined in: web-fetcher/dist/index.d.ts:1171
44
+ Defined in: web-fetcher/dist/index.d.ts:1172
45
45
 
46
46
  Creates a new FetchSession.
47
47
 
@@ -67,7 +67,7 @@ Configuration options for the fetcher.
67
67
 
68
68
  > `protected` **closed**: `boolean`
69
69
 
70
- Defined in: web-fetcher/dist/index.d.ts:1165
70
+ Defined in: web-fetcher/dist/index.d.ts:1166
71
71
 
72
72
  #### Inherited from
73
73
 
@@ -79,7 +79,7 @@ Defined in: web-fetcher/dist/index.d.ts:1165
79
79
 
80
80
  > `readonly` **context**: `FetchContext`
81
81
 
82
- Defined in: web-fetcher/dist/index.d.ts:1164
82
+ Defined in: web-fetcher/dist/index.d.ts:1165
83
83
 
84
84
  The execution context for this session, containing configurations, event bus, and shared state.
85
85
 
@@ -93,7 +93,7 @@ The execution context for this session, containing configurations, event bus, an
93
93
 
94
94
  > `readonly` **id**: `string`
95
95
 
96
- Defined in: web-fetcher/dist/index.d.ts:1160
96
+ Defined in: web-fetcher/dist/index.d.ts:1161
97
97
 
98
98
  Unique identifier for the session.
99
99
 
@@ -107,7 +107,7 @@ Unique identifier for the session.
107
107
 
108
108
  > `protected` **options**: [`FetcherOptions`](../interfaces/FetcherOptions.md)
109
109
 
110
- Defined in: web-fetcher/dist/index.d.ts:1156
110
+ Defined in: web-fetcher/dist/index.d.ts:1157
111
111
 
112
112
  #### Inherited from
113
113
 
@@ -115,11 +115,21 @@ Defined in: web-fetcher/dist/index.d.ts:1156
115
115
 
116
116
  ***
117
117
 
118
+ ### \_defaultOptions?
119
+
120
+ > `static` `optional` **\_defaultOptions**: [`SearchOptions`](../interfaces/SearchOptions.md)
121
+
122
+ Defined in: [web-searcher/src/searcher.ts:55](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L55)
123
+
124
+ **`Internal`**
125
+
126
+ ***
127
+
118
128
  ### \_isFactory
119
129
 
120
130
  > `static` **\_isFactory**: `boolean` = `false`
121
131
 
122
- Defined in: [web-searcher/src/searcher.ts:34](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L34)
132
+ Defined in: [web-searcher/src/searcher.ts:34](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L34)
123
133
 
124
134
  ***
125
135
 
@@ -127,7 +137,7 @@ Defined in: [web-searcher/src/searcher.ts:34](https://github.com/isdk/web-search
127
137
 
128
138
  > `static` `optional` **alias**: `string` \| `string`[]
129
139
 
130
- Defined in: [web-searcher/src/searcher.ts:46](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L46)
140
+ Defined in: [web-searcher/src/searcher.ts:46](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L46)
131
141
 
132
142
  Engine alias(es). Can be a single string or an array of strings.
133
143
  Useful for registering shorthand names (e.g., 'g' for 'Google').
@@ -138,7 +148,7 @@ Useful for registering shorthand names (e.g., 'g' for 'Google').
138
148
 
139
149
  > `static` **createObject**: (`name`, ...`args`) => `WebSearcher`
140
150
 
141
- Defined in: [web-searcher/src/searcher.ts:85](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L85)
151
+ Defined in: [web-searcher/src/searcher.ts:122](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L122)
142
152
 
143
153
  Creates an instance of the registered search engine.
144
154
 
@@ -168,7 +178,7 @@ An instance of the search engine.
168
178
 
169
179
  > `static` `optional` **currentInstanceIndex**: `number`
170
180
 
171
- Defined in: [web-searcher/src/searcher.ts:52](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L52)
181
+ Defined in: [web-searcher/src/searcher.ts:52](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L52)
172
182
 
173
183
  Globally shared index for tracking the currently active instance (node) across sessions.
174
184
 
@@ -178,7 +188,7 @@ Globally shared index for tracking the currently active instance (node) across s
178
188
 
179
189
  > `static` `optional` **defaultBaseUrls**: `string`[]
180
190
 
181
- Defined in: [web-searcher/src/searcher.ts:49](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L49)
191
+ Defined in: [web-searcher/src/searcher.ts:49](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L49)
182
192
 
183
193
  Default base URLs for engines that support multiple instances.
184
194
 
@@ -188,7 +198,7 @@ Default base URLs for engines that support multiple instances.
188
198
 
189
199
  > `static` **forEach**: (`cb`) => `void`
190
200
 
191
- Defined in: [web-searcher/src/searcher.ts:92](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L92)
201
+ Defined in: [web-searcher/src/searcher.ts:129](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L129)
192
202
 
193
203
  Iterates over all registered engines.
194
204
 
@@ -210,7 +220,7 @@ Callback function to invoke for each registered engine.
210
220
 
211
221
  > `static` **get**: (`name`) => *typeof* `WebSearcher`
212
222
 
213
- Defined in: [web-searcher/src/searcher.ts:76](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L76)
223
+ Defined in: [web-searcher/src/searcher.ts:113](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L113)
214
224
 
215
225
  Retrieves a registered search engine class by name.
216
226
 
@@ -234,7 +244,7 @@ The search engine class constructor.
234
244
 
235
245
  > `static` `optional` **name**: `string`
236
246
 
237
- Defined in: [web-searcher/src/searcher.ts:41](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L41)
247
+ Defined in: [web-searcher/src/searcher.ts:41](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L41)
238
248
 
239
249
  Custom engine name. If not provided, it is derived from the class name.
240
250
  For example, `GoogleSearcher` becomes `Google`.
@@ -245,7 +255,7 @@ For example, `GoogleSearcher` becomes `Google`.
245
255
 
246
256
  > `static` **register**: (`ctor`, `options?`) => `boolean`
247
257
 
248
- Defined in: [web-searcher/src/searcher.ts:61](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L61)
258
+ Defined in: [web-searcher/src/searcher.ts:98](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L98)
249
259
 
250
260
  Registers a search engine class.
251
261
 
@@ -275,7 +285,7 @@ Registration options. If a string is provided, it is used as the registered name
275
285
 
276
286
  > `static` **setAliases**: (`ctor`, ...`aliases`) => `void`
277
287
 
278
- Defined in: [web-searcher/src/searcher.ts:100](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L100)
288
+ Defined in: [web-searcher/src/searcher.ts:137](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L137)
279
289
 
280
290
  Sets aliases for a registered engine.
281
291
 
@@ -303,7 +313,7 @@ Aliases to add.
303
313
 
304
314
  > `static` **unregister**: (`name?`) => `void`
305
315
 
306
- Defined in: [web-searcher/src/searcher.ts:68](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L68)
316
+ Defined in: [web-searcher/src/searcher.ts:105](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L105)
307
317
 
308
318
  Unregisters a search engine.
309
319
 
@@ -327,7 +337,7 @@ The name or class to unregister.
327
337
 
328
338
  > **get** **pagination**(): [`PaginationConfig`](../interfaces/PaginationConfig.md) \| `undefined`
329
339
 
330
- Defined in: [web-searcher/src/searcher.ts:198](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L198)
340
+ Defined in: [web-searcher/src/searcher.ts:236](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L236)
331
341
 
332
342
  Optional pagination configuration.
333
343
  Defines how the searcher navigates to subsequent pages.
@@ -346,7 +356,7 @@ If undefined, the searcher will only fetch the first page.
346
356
 
347
357
  > **get** **template**(): [`FetcherOptions`](../interfaces/FetcherOptions.md)
348
358
 
349
- Defined in: [web-searcher/src/searcher.ts:188](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L188)
359
+ Defined in: [web-searcher/src/searcher.ts:226](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L226)
350
360
 
351
361
  The declarative template for the fetch options.
352
362
 
@@ -372,13 +382,94 @@ get template() {
372
382
 
373
383
  [`FetcherOptions`](../interfaces/FetcherOptions.md)
374
384
 
385
+ ***
386
+
387
+ ### defaultOptions
388
+
389
+ #### Get Signature
390
+
391
+ > **get** `static` **defaultOptions**(): [`SearchOptions`](../interfaces/SearchOptions.md)
392
+
393
+ Defined in: [web-searcher/src/searcher.ts:61](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L61)
394
+
395
+ Gets or sets the default search parameters for this specific engine class.
396
+ This does not include settings from parent classes.
397
+
398
+ ##### Returns
399
+
400
+ [`SearchOptions`](../interfaces/SearchOptions.md)
401
+
402
+ #### Set Signature
403
+
404
+ > **set** `static` **defaultOptions**(`options`): `void`
405
+
406
+ Defined in: [web-searcher/src/searcher.ts:68](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L68)
407
+
408
+ ##### Parameters
409
+
410
+ ###### options
411
+
412
+ [`SearchOptions`](../interfaces/SearchOptions.md)
413
+
414
+ ##### Returns
415
+
416
+ `void`
417
+
375
418
  ## Methods
376
419
 
420
+ ### \_execute()
421
+
422
+ > `protected` **\_execute**\<`R`\>(`actionOptions`, `context?`): `Promise`\<`FetchActionResult`\<`R`\>\>
423
+
424
+ Defined in: web-fetcher/dist/index.d.ts:1187
425
+
426
+ Executes a single action within the session.
427
+
428
+ #### Type Parameters
429
+
430
+ ##### R
431
+
432
+ `R` *extends* `FetchReturnType` = `"response"`
433
+
434
+ The expected return type of the action.
435
+
436
+ #### Parameters
437
+
438
+ ##### actionOptions
439
+
440
+ `_RequireAtLeastOne`
441
+
442
+ Configuration for the action to be executed.
443
+
444
+ ##### context?
445
+
446
+ `FetchContext`
447
+
448
+ Optional context override for this specific execution. Defaults to the session context.
449
+
450
+ #### Returns
451
+
452
+ `Promise`\<`FetchActionResult`\<`R`\>\>
453
+
454
+ A promise that resolves to the result of the action.
455
+
456
+ #### Example
457
+
458
+ ```ts
459
+ await session.execute({ name: 'goto', params: { url: 'https://example.com' } });
460
+ ```
461
+
462
+ #### Inherited from
463
+
464
+ `FetchSession._execute`
465
+
466
+ ***
467
+
377
468
  ### \_logDebug()
378
469
 
379
470
  > `protected` **\_logDebug**(`category`, ...`args`): `void`
380
471
 
381
- Defined in: web-fetcher/dist/index.d.ts:1172
472
+ Defined in: web-fetcher/dist/index.d.ts:1173
382
473
 
383
474
  #### Parameters
384
475
 
@@ -404,7 +495,7 @@ Defined in: web-fetcher/dist/index.d.ts:1172
404
495
 
405
496
  > `protected` **createContext**(`options`): `FetchContext`
406
497
 
407
- Defined in: [web-searcher/src/searcher.ts:216](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L216)
498
+ Defined in: [web-searcher/src/searcher.ts:254](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L254)
408
499
 
409
500
  #### Parameters
410
501
 
@@ -426,7 +517,7 @@ Defined in: [web-searcher/src/searcher.ts:216](https://github.com/isdk/web-searc
426
517
 
427
518
  > **dispose**(): `Promise`\<`void`\>
428
519
 
429
- Defined in: web-fetcher/dist/index.d.ts:1231
520
+ Defined in: web-fetcher/dist/index.d.ts:1233
430
521
 
431
522
  Disposes of the session and its associated engine.
432
523
 
@@ -449,9 +540,7 @@ This method should be called when the session is no longer needed to free up res
449
540
 
450
541
  > **execute**\<`R`\>(`actionOptions`, `context?`): `Promise`\<`FetchActionResult`\<`R`\>\>
451
542
 
452
- Defined in: web-fetcher/dist/index.d.ts:1186
453
-
454
- Executes a single action within the session.
543
+ Defined in: web-fetcher/dist/index.d.ts:1188
455
544
 
456
545
  #### Type Parameters
457
546
 
@@ -459,34 +548,20 @@ Executes a single action within the session.
459
548
 
460
549
  `R` *extends* `FetchReturnType` = `"response"`
461
550
 
462
- The expected return type of the action.
463
-
464
551
  #### Parameters
465
552
 
466
553
  ##### actionOptions
467
554
 
468
555
  `_RequireAtLeastOne`
469
556
 
470
- Configuration for the action to be executed.
471
-
472
557
  ##### context?
473
558
 
474
559
  `FetchContext`
475
560
 
476
- Optional context override for this specific execution. Defaults to the session context.
477
-
478
561
  #### Returns
479
562
 
480
563
  `Promise`\<`FetchActionResult`\<`R`\>\>
481
564
 
482
- A promise that resolves to the result of the action.
483
-
484
- #### Example
485
-
486
- ```ts
487
- await session.execute({ name: 'goto', params: { url: 'https://example.com' } });
488
- ```
489
-
490
565
  #### Inherited from
491
566
 
492
567
  `FetchSession.execute`
@@ -497,7 +572,7 @@ await session.execute({ name: 'goto', params: { url: 'https://example.com' } });
497
572
 
498
573
  > **executeAll**(`actions`, `options?`): `Promise`\<\{ `outputs`: `Record`\<`string`, `any`\>; `result`: `FetchResponse` \| `undefined`; \}\>
499
574
 
500
- Defined in: web-fetcher/dist/index.d.ts:1203
575
+ Defined in: web-fetcher/dist/index.d.ts:1205
501
576
 
502
577
  Executes a sequence of actions.
503
578
 
@@ -541,7 +616,7 @@ const { result, outputs } = await session.executeAll([
541
616
 
542
617
  > `protected` **formatOptions**(`options`): `Record`\<`string`, `any`\>
543
618
 
544
- Defined in: [web-searcher/src/searcher.ts:457](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L457)
619
+ Defined in: [web-searcher/src/searcher.ts:498](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L498)
545
620
 
546
621
  Transforms standard options into engine-specific template variables.
547
622
 
@@ -569,7 +644,7 @@ A dictionary of variables to be injected into the template.
569
644
 
570
645
  > **getOutputs**(): `Record`\<`string`, `any`\>
571
646
 
572
- Defined in: web-fetcher/dist/index.d.ts:1214
647
+ Defined in: web-fetcher/dist/index.d.ts:1216
573
648
 
574
649
  Retrieves all outputs accumulated during the session.
575
650
 
@@ -589,7 +664,7 @@ A record of stored output data.
589
664
 
590
665
  > **getState**(): `Promise`\<\{ `cookies`: `Cookie`[]; `sessionState?`: `any`; \} \| `undefined`\>
591
666
 
592
- Defined in: web-fetcher/dist/index.d.ts:1220
667
+ Defined in: web-fetcher/dist/index.d.ts:1222
593
668
 
594
669
  Gets the current state of the session, including cookies and engine-specific state.
595
670
 
@@ -609,7 +684,7 @@ A promise resolving to the session state, or undefined if no engine is initializ
609
684
 
610
685
  > `protected` **getTemplate**(`variables`, `options`): [`FetcherOptions`](../interfaces/FetcherOptions.md)
611
686
 
612
- Defined in: [web-searcher/src/searcher.ts:212](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L212)
687
+ Defined in: [web-searcher/src/searcher.ts:250](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L250)
613
688
 
614
689
  Dynamically retrieves the fetch template based on current variables and search options.
615
690
 
@@ -642,7 +717,7 @@ The fetcher configuration to be used for the current request.
642
717
 
643
718
  > **search**(`query`, `options`): `Promise`\<[`StandardSearchResult`](../interfaces/StandardSearchResult.md)[]\>
644
719
 
645
- Defined in: [web-searcher/src/searcher.ts:246](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L246)
720
+ Defined in: [web-searcher/src/searcher.ts:284](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L284)
646
721
 
647
722
  Executes a search query.
648
723
 
@@ -675,7 +750,7 @@ A promise resolving to an array of standardized search results.
675
750
 
676
751
  > `protected` **transform**(`outputs`, `context`): `Promise`\<[`StandardSearchResult`](../interfaces/StandardSearchResult.md)[]\>
677
752
 
678
- Defined in: [web-searcher/src/searcher.ts:439](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L439)
753
+ Defined in: [web-searcher/src/searcher.ts:480](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L480)
679
754
 
680
755
  Transform and clean the raw extracted results.
681
756
 
@@ -708,7 +783,7 @@ A promise resolving to an array of standardized search results.
708
783
 
709
784
  > `protected` **validateFetchResult**(`results`, `context`): `Promise`\<`boolean`\>
710
785
 
711
- Defined in: [web-searcher/src/searcher.ts:421](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L421)
786
+ Defined in: [web-searcher/src/searcher.ts:462](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L462)
712
787
 
713
788
  Hook for subclasses to validate fetched results before they are accepted.
714
789
  If this returns false, the instance manager will consider the fetch a failure
@@ -736,11 +811,26 @@ A promise resolving to true if valid, false otherwise.
736
811
 
737
812
  ***
738
813
 
814
+ ### getDefaultOptions()
815
+
816
+ > `static` **getDefaultOptions**(): [`SearchOptions`](../interfaces/SearchOptions.md)
817
+
818
+ Defined in: [web-searcher/src/searcher.ts:76](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L76)
819
+
820
+ Retrieves the combined default search options by traversing the prototype chain.
821
+ Priority: Current class > Parent class > WebSearcher base class.
822
+
823
+ #### Returns
824
+
825
+ [`SearchOptions`](../interfaces/SearchOptions.md)
826
+
827
+ ***
828
+
739
829
  ### search()
740
830
 
741
831
  > `static` **search**(`engineNames`, `query`, `options`): `Promise`\<[`StandardSearchResult`](../interfaces/StandardSearchResult.md)[]\>
742
832
 
743
- Defined in: [web-searcher/src/searcher.ts:113](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/searcher.ts#L113)
833
+ Defined in: [web-searcher/src/searcher.ts:150](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/searcher.ts#L150)
744
834
 
745
835
  Static helper to execute a one-off search or a fallback chain.
746
836
 
@@ -8,7 +8,7 @@
8
8
 
9
9
  > **extractDate**(`url`, `options`): `Promise`\<`string` \| `null`\>
10
10
 
11
- Defined in: [web-searcher/src/utils/extractor/date-extractor.ts:30](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/date-extractor.ts#L30)
11
+ Defined in: [web-searcher/src/utils/extractor/date-extractor.ts:30](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/date-extractor.ts#L30)
12
12
 
13
13
  High-level convenience function to extract the publication or modification date from a URL.
14
14
  It performs a partial fetch of the content and applies multiple extraction rules
@@ -8,7 +8,7 @@
8
8
 
9
9
  > **extractMetadataFrom**(`result`, `type`): `string` \| `null`
10
10
 
11
- Defined in: [web-searcher/src/utils/extractor/extractor.ts:27](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/extractor.ts#L27)
11
+ Defined in: [web-searcher/src/utils/extractor/extractor.ts:27](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/extractor.ts#L27)
12
12
 
13
13
  Extracts specific metadata from parsed HTML and headers based on a requested type.
14
14
  Currently supports 'date' extraction with a prioritized fallback mechanism.
@@ -8,7 +8,7 @@
8
8
 
9
9
  > **fetchHeaders**(`url`, `options`): `Promise`\<`Headers` \| `null`\>
10
10
 
11
- Defined in: [web-searcher/src/utils/extractor/fetcher.ts:19](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/fetcher.ts#L19)
11
+ Defined in: [web-searcher/src/utils/extractor/fetcher.ts:19](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/fetcher.ts#L19)
12
12
 
13
13
  Fetches only the HTTP headers for a given URL using a HEAD request.
14
14
  Useful for checking 'last-modified' without downloading the body.
@@ -8,7 +8,7 @@
8
8
 
9
9
  > **fetchPartial**(`url`, `maxBytes`, `options`): `Promise`\<\{ `content`: `string`; `headers`: `Headers`; \} \| `null`\>
10
10
 
11
- Defined in: [web-searcher/src/utils/extractor/fetcher.ts:55](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/fetcher.ts#L55)
11
+ Defined in: [web-searcher/src/utils/extractor/fetcher.ts:55](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/fetcher.ts#L55)
12
12
 
13
13
  Fetches a partial amount of content from a URL.
14
14
  Automatically handles character set detection from the Content-Type header.
@@ -8,7 +8,7 @@
8
8
 
9
9
  > **normalizeDate**(`dateStr`): `string` \| `null`
10
10
 
11
- Defined in: [web-searcher/src/utils/extractor/date-normalizer.ts:9](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/date-normalizer.ts#L9)
11
+ Defined in: [web-searcher/src/utils/extractor/date-normalizer.ts:9](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/date-normalizer.ts#L9)
12
12
 
13
13
  Normalizes a date string into a standard ISO 8601 format (UTC).
14
14
  It handles various formats (YYYY-MM-DD, RFC2822, etc.) and performs
@@ -8,7 +8,7 @@
8
8
 
9
9
  > **parseHeaders**(`headers`): `Record`\<`string`, `string`\>
10
10
 
11
- Defined in: [web-searcher/src/utils/extractor/parser.ts:25](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/parser.ts#L25)
11
+ Defined in: [web-searcher/src/utils/extractor/parser.ts:25](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/parser.ts#L25)
12
12
 
13
13
  Converts a Web API Headers object into a plain JavaScript record.
14
14
  All header names are converted to lowercase for consistent access.
@@ -8,7 +8,7 @@
8
8
 
9
9
  > **parseHtml**(`html`): [`HtmlData`](../interfaces/HtmlData.md)
10
10
 
11
- Defined in: [web-searcher/src/utils/extractor/parser.ts:49](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/parser.ts#L49)
11
+ Defined in: [web-searcher/src/utils/extractor/parser.ts:49](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/parser.ts#L49)
12
12
 
13
13
  Parses an HTML string to extract generic metadata structures (Meta tags, JSON-LD, Time tags).
14
14
 
@@ -8,7 +8,7 @@
8
8
 
9
9
  > **testUrlsByLatency**(`urls`, `options`): `Promise`\<[`VerifiedUrl`](../interfaces/VerifiedUrl.md)[]\>
10
10
 
11
- Defined in: [web-searcher/src/utils/latency.ts:12](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/latency.ts#L12)
11
+ Defined in: [web-searcher/src/utils/latency.ts:12](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/latency.ts#L12)
12
12
 
13
13
  A general utility to test a list of URLs for availability and latency.
14
14
  Returns a list of verified URLs sorted by response time.
@@ -25,6 +25,10 @@ Returns a list of verified URLs sorted by response time.
25
25
 
26
26
  `number`
27
27
 
28
+ #### proxy?
29
+
30
+ `string`
31
+
28
32
  #### testPath?
29
33
 
30
34
  `string`
@@ -6,7 +6,7 @@
6
6
 
7
7
  # Interface: CustomTimeRange
8
8
 
9
- Defined in: [web-searcher/src/types.ts:113](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/types.ts#L113)
9
+ Defined in: [web-searcher/src/types.ts:113](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/types.ts#L113)
10
10
 
11
11
  ## Properties
12
12
 
@@ -14,7 +14,7 @@ Defined in: [web-searcher/src/types.ts:113](https://github.com/isdk/web-searcher
14
14
 
15
15
  > **from**: `string` \| `Date`
16
16
 
17
- Defined in: [web-searcher/src/types.ts:115](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/types.ts#L115)
17
+ Defined in: [web-searcher/src/types.ts:115](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/types.ts#L115)
18
18
 
19
19
  Start date (Date object or string like 'YYYY-MM-DD').
20
20
 
@@ -24,6 +24,6 @@ Start date (Date object or string like 'YYYY-MM-DD').
24
24
 
25
25
  > `optional` **to**: `string` \| `Date`
26
26
 
27
- Defined in: [web-searcher/src/types.ts:117](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/types.ts#L117)
27
+ Defined in: [web-searcher/src/types.ts:117](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/types.ts#L117)
28
28
 
29
29
  End date (Date object or string like 'YYYY-MM-DD'). Defaults to current date if omitted.
@@ -6,7 +6,7 @@
6
6
 
7
7
  # Interface: ExtractOptions
8
8
 
9
- Defined in: [web-searcher/src/utils/extractor/date-extractor.ts:7](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/date-extractor.ts#L7)
9
+ Defined in: [web-searcher/src/utils/extractor/date-extractor.ts:7](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/date-extractor.ts#L7)
10
10
 
11
11
  Options for the extractDate function.
12
12
 
@@ -20,7 +20,7 @@ Options for the extractDate function.
20
20
 
21
21
  > `optional` **headers**: `Record`\<`string`, `string`\>
22
22
 
23
- Defined in: [web-searcher/src/utils/extractor/fetcher.ts:8](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/fetcher.ts#L8)
23
+ Defined in: [web-searcher/src/utils/extractor/fetcher.ts:8](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/fetcher.ts#L8)
24
24
 
25
25
  Custom HTTP headers to include in the request.
26
26
 
@@ -34,7 +34,7 @@ Custom HTTP headers to include in the request.
34
34
 
35
35
  > `optional` **maxBytes**: `number`
36
36
 
37
- Defined in: [web-searcher/src/utils/extractor/date-extractor.ts:12](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/date-extractor.ts#L12)
37
+ Defined in: [web-searcher/src/utils/extractor/date-extractor.ts:12](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/date-extractor.ts#L12)
38
38
 
39
39
  Maximum number of bytes to download from the URL.
40
40
  Defaults to 32768 (32KB), which is usually enough for the HTML <head>.
@@ -45,7 +45,7 @@ Defaults to 32768 (32KB), which is usually enough for the HTML <head>.
45
45
 
46
46
  > `optional` **timeout**: `number`
47
47
 
48
- Defined in: [web-searcher/src/utils/extractor/fetcher.ts:6](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/fetcher.ts#L6)
48
+ Defined in: [web-searcher/src/utils/extractor/fetcher.ts:6](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/fetcher.ts#L6)
49
49
 
50
50
  Timeout in milliseconds. Defaults vary by function (5s to 10s).
51
51
 
@@ -6,7 +6,7 @@
6
6
 
7
7
  # Interface: FetchExtractorOptions
8
8
 
9
- Defined in: [web-searcher/src/utils/extractor/fetcher.ts:4](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/fetcher.ts#L4)
9
+ Defined in: [web-searcher/src/utils/extractor/fetcher.ts:4](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/fetcher.ts#L4)
10
10
 
11
11
  Options for network requests.
12
12
 
@@ -20,7 +20,7 @@ Options for network requests.
20
20
 
21
21
  > `optional` **headers**: `Record`\<`string`, `string`\>
22
22
 
23
- Defined in: [web-searcher/src/utils/extractor/fetcher.ts:8](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/fetcher.ts#L8)
23
+ Defined in: [web-searcher/src/utils/extractor/fetcher.ts:8](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/fetcher.ts#L8)
24
24
 
25
25
  Custom HTTP headers to include in the request.
26
26
 
@@ -30,6 +30,6 @@ Custom HTTP headers to include in the request.
30
30
 
31
31
  > `optional` **timeout**: `number`
32
32
 
33
- Defined in: [web-searcher/src/utils/extractor/fetcher.ts:6](https://github.com/isdk/web-searcher.js/blob/955bc509edda39926bd12c6c2b8c28da7eb13ff5/src/utils/extractor/fetcher.ts#L6)
33
+ Defined in: [web-searcher/src/utils/extractor/fetcher.ts:6](https://github.com/isdk/web-searcher.js/blob/0c4757eb75b3b7c5af0231806f11e7b3c3166736/src/utils/extractor/fetcher.ts#L6)
34
34
 
35
35
  Timeout in milliseconds. Defaults vary by function (5s to 10s).