rezo 1.0.4 → 1.0.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (47) hide show
  1. package/README.md +352 -9
  2. package/dist/adapters/curl.cjs +796 -0
  3. package/dist/adapters/curl.js +796 -0
  4. package/dist/adapters/entries/curl.d.ts +2332 -20
  5. package/dist/adapters/entries/fetch.d.ts +289 -20
  6. package/dist/adapters/entries/http.d.ts +289 -20
  7. package/dist/adapters/entries/http2.d.ts +289 -20
  8. package/dist/adapters/entries/react-native.d.ts +289 -20
  9. package/dist/adapters/entries/xhr.d.ts +289 -20
  10. package/dist/adapters/index.cjs +6 -6
  11. package/dist/adapters/picker.cjs +2 -2
  12. package/dist/adapters/picker.js +2 -2
  13. package/dist/cache/index.cjs +13 -13
  14. package/dist/core/rezo.cjs +2 -2
  15. package/dist/core/rezo.js +2 -2
  16. package/dist/crawler.d.ts +291 -22
  17. package/dist/entries/crawler.cjs +5 -5
  18. package/dist/index.cjs +23 -18
  19. package/dist/index.d.ts +556 -20
  20. package/dist/index.js +1 -0
  21. package/dist/platform/browser.d.ts +289 -20
  22. package/dist/platform/bun.d.ts +289 -20
  23. package/dist/platform/deno.d.ts +289 -20
  24. package/dist/platform/node.d.ts +289 -20
  25. package/dist/platform/react-native.d.ts +289 -20
  26. package/dist/platform/worker.d.ts +289 -20
  27. package/dist/plugin/crawler-options.cjs +1 -1
  28. package/dist/plugin/crawler-options.js +1 -1
  29. package/dist/plugin/crawler.cjs +2 -2
  30. package/dist/plugin/crawler.js +2 -2
  31. package/dist/plugin/index.cjs +36 -36
  32. package/dist/proxy/index.cjs +2 -2
  33. package/dist/proxy/manager.cjs +14 -1
  34. package/dist/proxy/manager.js +14 -1
  35. package/dist/queue/http-queue.cjs +313 -0
  36. package/dist/queue/http-queue.js +312 -0
  37. package/dist/queue/index.cjs +8 -0
  38. package/dist/queue/index.js +6 -0
  39. package/dist/queue/queue.cjs +346 -0
  40. package/dist/queue/queue.js +344 -0
  41. package/dist/queue/types.cjs +17 -0
  42. package/dist/queue/types.js +17 -0
  43. package/dist/types/curl-options.cjs +25 -0
  44. package/dist/types/curl-options.js +25 -0
  45. package/dist/utils/http-config.cjs +0 -15
  46. package/dist/utils/http-config.js +0 -15
  47. package/package.json +1 -2
package/README.md CHANGED
@@ -64,6 +64,7 @@ Rezo is a production-ready HTTP client library engineered for Node.js 22+ and un
64
64
  - [Cookie Management](#cookie-management)
65
65
  - [Proxy Configuration](#proxy-configuration)
66
66
  - [Proxy Manager](#proxy-manager)
67
+ - [Request Queue](#request-queue)
67
68
  - [Streaming](#streaming)
68
69
  - [File Downloads](#file-downloads)
69
70
  - [File Uploads](#file-uploads)
@@ -217,7 +218,7 @@ const client = rezo.create({ baseURL: 'https://api.example.com' });
217
218
  |---------|-------------|
218
219
  | **Retry Logic** | Configurable retry with exponential backoff |
219
220
  | **Timeout Control** | Connection, request, and total timeout options |
220
- | **Rate Limiting** | Request queue with priority and concurrency control |
221
+ | **Rate Limiting** | Built-in request queue with priority, concurrency, and domain-based rate limiting |
221
222
  | **Hooks System** | Lifecycle hooks for request/response interception |
222
223
  | **Error Handling** | Structured errors with actionable suggestions |
223
224
  | **Performance Metrics** | Detailed timing data for monitoring |
@@ -306,25 +307,270 @@ const response = await rezo.get('https://api.example.com/data');
306
307
 
307
308
  ### cURL Adapter
308
309
 
309
- Advanced adapter wrapping the cURL command-line tool for maximum compatibility and debugging.
310
+ Advanced adapter wrapping the cURL command-line tool with **120+ configuration options** for maximum compatibility, debugging, and fine-grained control.
310
311
 
311
312
  ```typescript
312
313
  import rezo from 'rezo/adapters/curl';
313
314
 
314
315
  const response = await rezo.get('https://api.example.com/data', {
315
316
  curl: {
316
- verbose: true,
317
- insecure: false
317
+ connectTimeout: 10,
318
+ limitRate: '500K',
319
+ retry: { attempts: 3, allErrors: true },
320
+ verbose: true
318
321
  }
319
322
  });
320
323
  ```
321
324
 
322
325
  **Features:**
323
- - 200+ cURL options available
324
- - Advanced authentication (Basic, Digest, NTLM, Negotiate)
325
- - Connection pooling
326
- - Detailed timing information
327
- - Perfect for debugging and testing
326
+ - 120+ cURL command-line options via `curl` property
327
+ - HTTP/1.0, 1.1, 2.0, 3.0 protocol support
328
+ - Advanced authentication (Basic, Digest, NTLM, Negotiate, AWS SigV4, OAuth2)
329
+ - Comprehensive TLS/SSL configuration
330
+ - FTP, SSH, SMTP, and TFTP protocol support
331
+ - Connection pooling and reuse
332
+ - Detailed timing and debugging information
333
+ - Proxy authentication and chaining
334
+
335
+ **Important**: The `curl` property is **only available** when importing from `rezo/adapters/curl`. It does not appear in the base `RezoRequestConfig` type.
336
+
337
+ #### cURL Options Categories
338
+
339
+ The cURL adapter provides options across the following categories:
340
+
341
+ | Category | Example Options | Description |
342
+ |----------|----------------|-------------|
343
+ | **Connection** | `connectTimeout`, `maxTime`, `tcpFastOpen`, `tcpNodelay` | Connection timing and TCP settings |
344
+ | **Rate Limiting** | `limitRate`, `speedLimit`, `maxFilesize` | Bandwidth and transfer limits |
345
+ | **Retry** | `retry.attempts`, `retry.delay`, `retry.allErrors` | Automatic retry configuration |
346
+ | **Network** | `interface`, `localPort`, `ipVersion`, `resolve` | Network interface and routing |
347
+ | **HTTP** | `httpVersion`, `pathAsIs`, `maxRedirs`, `referer` | HTTP protocol settings |
348
+ | **TLS/SSL** | `tls.min`, `tls.ciphers`, `tls.cert`, `insecure` | Comprehensive TLS configuration |
349
+ | **Proxy** | `proxyHeaders`, `proxyTls`, `proxyTunnel` | Proxy-specific settings |
350
+ | **DNS** | `dns.servers`, `dns.dohUrl`, `dns.dohInsecure` | DNS resolution and DoH |
351
+ | **Authentication** | `negotiate`, `awsSigv4`, `oauth2Bearer`, `kerberos` | Advanced auth methods |
352
+ | **FTP** | `ftp.pasv`, `ftp.createDirs`, `ftp.method` | FTP protocol options |
353
+ | **SSH** | `ssh.privateKey`, `ssh.knownHosts`, `ssh.compression` | SSH/SCP/SFTP settings |
354
+ | **SMTP** | `smtp.mailFrom`, `smtp.mailRcpt` | Email sending configuration |
355
+ | **Debug** | `verbose`, `trace`, `traceTime`, `dumpHeader` | Debugging and tracing |
356
+
357
+ #### cURL Options Examples
358
+
359
+ ##### Connection & Timeout
360
+
361
+ ```typescript
362
+ import rezo from 'rezo/adapters/curl';
363
+
364
+ await rezo.get('https://api.example.com/data', {
365
+ curl: {
366
+ connectTimeout: 10, // 10 seconds for TCP connection
367
+ maxTime: 300, // 5 minutes max for entire request
368
+ keepaliveTime: 60, // Send keepalive after 60s idle
369
+ tcpFastOpen: true, // Enable TCP Fast Open
370
+ tcpNodelay: true, // Disable Nagle algorithm
371
+ happyEyeballsTimeout: 200 // IPv6/IPv4 fallback timeout
372
+ }
373
+ });
374
+ ```
375
+
376
+ ##### Rate Limiting & Bandwidth
377
+
378
+ ```typescript
379
+ import rezo from 'rezo/adapters/curl';
380
+
381
+ await rezo.get('https://example.com/large-file.zip', {
382
+ curl: {
383
+ limitRate: '1M', // Limit to 1 MB/s
384
+ speedLimit: { limit: 1000, time: 30 }, // Abort if <1KB/s for 30s
385
+ maxFilesize: 100 * 1024 * 1024 // Max 100MB response
386
+ }
387
+ });
388
+ ```
389
+
390
+ ##### Retry Configuration
391
+
392
+ ```typescript
393
+ import rezo from 'rezo/adapters/curl';
394
+
395
+ await rezo.get('https://unreliable-api.com/data', {
396
+ curl: {
397
+ retry: {
398
+ attempts: 5, // Retry up to 5 times
399
+ delay: 2, // 2 seconds between retries
400
+ maxTime: 60, // Max 60 seconds total for retries
401
+ allErrors: true, // Retry on all errors (not just transient)
402
+ connRefused: true // Retry on connection refused
403
+ }
404
+ }
405
+ });
406
+ ```
407
+
408
+ ##### Custom DNS Resolution
409
+
410
+ ```typescript
411
+ import rezo from 'rezo/adapters/curl';
412
+
413
+ await rezo.get('https://api.example.com/data', {
414
+ curl: {
415
+ resolve: [
416
+ { host: 'api.example.com', port: 443, address: '10.0.0.1' }
417
+ ],
418
+ dns: {
419
+ servers: '8.8.8.8,8.8.4.4',
420
+ dohUrl: 'https://dns.google/dns-query'
421
+ }
422
+ }
423
+ });
424
+ ```
425
+
426
+ ##### Comprehensive TLS Configuration
427
+
428
+ ```typescript
429
+ import rezo from 'rezo/adapters/curl';
430
+
431
+ await rezo.get('https://secure.example.com/api', {
432
+ curl: {
433
+ tls: {
434
+ min: 'tlsv1.2', // Minimum TLS 1.2
435
+ tls13Ciphers: 'TLS_AES_256_GCM_SHA384', // TLS 1.3 ciphers
436
+ ciphers: 'ECDHE-RSA-AES256-GCM-SHA384', // TLS 1.2 ciphers
437
+ certStatus: true, // OCSP stapling
438
+ pinnedPubKey: 'sha256//base64hash=', // Certificate pinning
439
+ cert: '/path/to/client.crt', // Client certificate
440
+ key: '/path/to/client.key', // Client key
441
+ cacert: '/path/to/ca-bundle.crt' // CA certificate
442
+ }
443
+ }
444
+ });
445
+ ```
446
+
447
+ ##### Advanced Authentication
448
+
449
+ ```typescript
450
+ import rezo from 'rezo/adapters/curl';
451
+
452
+ // AWS Signature Version 4
453
+ await rezo.get('https://s3.amazonaws.com/bucket/object', {
454
+ curl: {
455
+ awsSigv4: {
456
+ provider: 'aws:amz',
457
+ region: 'us-east-1',
458
+ service: 's3'
459
+ }
460
+ }
461
+ });
462
+
463
+ // Kerberos/SPNEGO
464
+ await rezo.get('https://internal.company.com/api', {
465
+ curl: {
466
+ negotiate: true,
467
+ delegation: 'policy'
468
+ }
469
+ });
470
+
471
+ // OAuth 2.0 Bearer Token
472
+ await rezo.get('https://api.example.com/protected', {
473
+ curl: {
474
+ oauth2Bearer: 'your-access-token'
475
+ }
476
+ });
477
+ ```
478
+
479
+ ##### FTP Operations
480
+
481
+ ```typescript
482
+ import rezo from 'rezo/adapters/curl';
483
+
484
+ await rezo.get('ftp://ftp.example.com/file.txt', {
485
+ curl: {
486
+ ftp: {
487
+ pasv: true,
488
+ createDirs: true,
489
+ method: 'singlecwd',
490
+ sslControl: true
491
+ }
492
+ }
493
+ });
494
+ ```
495
+
496
+ ##### SSH/SFTP Configuration
497
+
498
+ ```typescript
499
+ import rezo from 'rezo/adapters/curl';
500
+
501
+ await rezo.get('sftp://server.example.com/path/file.txt', {
502
+ curl: {
503
+ ssh: {
504
+ privateKey: '/home/user/.ssh/id_rsa',
505
+ publicKey: '/home/user/.ssh/id_rsa.pub',
506
+ knownHosts: '/home/user/.ssh/known_hosts',
507
+ compression: true
508
+ }
509
+ }
510
+ });
511
+ ```
512
+
513
+ ##### Debugging & Tracing
514
+
515
+ ```typescript
516
+ import rezo from 'rezo/adapters/curl';
517
+
518
+ await rezo.get('https://api.example.com/debug', {
519
+ curl: {
520
+ verbose: true, // Show detailed output
521
+ trace: '/tmp/curl-trace.log', // Write full trace to file
522
+ traceTime: true, // Include timestamps
523
+ dumpHeader: '/tmp/headers.txt' // Dump headers to file
524
+ }
525
+ });
526
+ ```
527
+
528
+ #### CurlRequestConfig Type
529
+
530
+ When using the cURL adapter, the request configuration type extends the base with cURL-specific options:
531
+
532
+ ```typescript
533
+ import rezo, { CurlRequestConfig } from 'rezo/adapters/curl';
534
+
535
+ const config: CurlRequestConfig = {
536
+ url: 'https://api.example.com/data',
537
+ method: 'GET',
538
+ headers: { 'Accept': 'application/json' },
539
+ curl: {
540
+ connectTimeout: 10,
541
+ retry: { attempts: 3 }
542
+ }
543
+ };
544
+
545
+ const response = await rezo.request(config);
546
+ ```
547
+
548
+ #### Available cURL Option Types
549
+
550
+ The cURL adapter exports all option types for TypeScript users:
551
+
552
+ ```typescript
553
+ import type {
554
+ CurlOptions,
555
+ CurlRequestConfig,
556
+ CurlTlsOptions,
557
+ CurlFtpOptions,
558
+ CurlSshOptions,
559
+ CurlSmtpOptions,
560
+ CurlDnsOptions,
561
+ CurlRetryOptions,
562
+ CurlProxyTlsOptions,
563
+ CurlResolveEntry,
564
+ CurlConnectToEntry,
565
+ CurlSpeedLimit,
566
+ CurlParallelOptions,
567
+ CurlHttpVersion,
568
+ CurlSslVersion,
569
+ CurlIpVersion,
570
+ CurlAuthMethod,
571
+ CurlDelegation
572
+ } from 'rezo/adapters/curl';
573
+ ```
328
574
 
329
575
  ### XHR Adapter
330
576
 
@@ -1037,6 +1283,103 @@ interface ProxyManagerConfig {
1037
1283
  | Fetch | Not supported (browser security) |
1038
1284
  | React Native | Not supported (platform limitations) |
1039
1285
 
1286
+ ### Request Queue
1287
+
1288
+ Rezo includes a built-in zero-dependency request queue system for rate limiting, priority management, and concurrency control.
1289
+
1290
+ #### RezoQueue - General Purpose Queue
1291
+
1292
+ ```typescript
1293
+ import { RezoQueue, Priority } from 'rezo';
1294
+
1295
+ const queue = new RezoQueue({
1296
+ concurrency: 5, // Max concurrent tasks
1297
+ interval: 1000, // Rate limit interval (ms)
1298
+ intervalCap: 10, // Max tasks per interval
1299
+ timeout: 30000, // Task timeout (ms)
1300
+ autoStart: true // Start processing immediately
1301
+ });
1302
+
1303
+ // Add tasks with priority
1304
+ queue.add(async () => {
1305
+ return await fetch('https://api.example.com/data');
1306
+ }, { priority: Priority.HIGH });
1307
+
1308
+ // Priority constants: LOWEST (0), LOW (25), NORMAL (50), HIGH (75), HIGHEST (100), CRITICAL (1000)
1309
+
1310
+ // Event handling
1311
+ queue.on('completed', ({ id, result, duration }) => {
1312
+ console.log(`Task ${id} completed in ${duration}ms`);
1313
+ });
1314
+
1315
+ queue.on('error', ({ id, error }) => {
1316
+ console.error(`Task ${id} failed:`, error);
1317
+ });
1318
+
1319
+ // Queue control
1320
+ queue.pause();
1321
+ queue.resume();
1322
+ queue.cancel(taskId);
1323
+ queue.clear();
1324
+
1325
+ // Statistics
1326
+ const stats = queue.stats();
1327
+ console.log(`Completed: ${stats.completed}, Failed: ${stats.failed}`);
1328
+ ```
1329
+
1330
+ #### HttpQueue - HTTP-Aware Queue
1331
+
1332
+ ```typescript
1333
+ import { HttpQueue, HttpMethodPriority } from 'rezo';
1334
+
1335
+ const httpQueue = new HttpQueue({
1336
+ concurrency: 10,
1337
+ perDomainConcurrency: 2, // Limit concurrent requests per domain
1338
+ retryOnRateLimit: true, // Auto-retry on 429 responses
1339
+ maxRetries: 3,
1340
+ retryDelay: 1000
1341
+ });
1342
+
1343
+ // Add HTTP requests
1344
+ const result = await httpQueue.addHttp(
1345
+ 'https://api.example.com/users',
1346
+ async () => fetch('https://api.example.com/users'),
1347
+ {
1348
+ priority: HttpMethodPriority.GET, // GET=75, POST=50, DELETE=25
1349
+ timeout: 10000
1350
+ }
1351
+ );
1352
+
1353
+ // Domain-specific control
1354
+ httpQueue.pauseDomain('api.example.com');
1355
+ httpQueue.resumeDomain('api.example.com');
1356
+
1357
+ // Rate limit handling
1358
+ httpQueue.onHttp('rateLimited', ({ domain, retryAfter }) => {
1359
+ console.log(`Rate limited on ${domain}, retry in ${retryAfter}s`);
1360
+ });
1361
+
1362
+ // Per-domain statistics
1363
+ const domainStats = httpQueue.httpStats();
1364
+ console.log(domainStats.byDomain['api.example.com']);
1365
+ ```
1366
+
1367
+ #### Queue Events
1368
+
1369
+ | Event | Description |
1370
+ |-------|-------------|
1371
+ | `add` | Task added to queue |
1372
+ | `start` | Task started processing |
1373
+ | `completed` | Task completed successfully |
1374
+ | `error` | Task failed with error |
1375
+ | `timeout` | Task timed out |
1376
+ | `cancelled` | Task was cancelled |
1377
+ | `active` | Queue became active |
1378
+ | `idle` | Queue became idle |
1379
+ | `empty` | Queue is empty |
1380
+ | `rateLimited` | Rate limit detected (HttpQueue) |
1381
+ | `retry` | Task being retried (HttpQueue) |
1382
+
1040
1383
  ### Streaming
1041
1384
 
1042
1385
  Rezo provides powerful streaming capabilities for handling large data efficiently.