@grabbit-labs/dynafetch 0.1.0 → 0.1.1

package/README.md

@@ -0,0 +1,108 @@
+# dynafetch
+
+Fetch any website like a real browser. One function call.
+
+dynafetch provides Chrome-level TLS fingerprinting, JavaScript execution, and full network interception. The response includes fully rendered HTML and all captured requests.
+
+```ts
+import { dynafetch } from "@grabbit-labs/dynafetch";
+
+const page = await dynafetch("https://example.com");
+console.log(page.html);       // fully rendered HTML
+console.log(page.framework);  // "nextjs", "inertia", "nuxt", ...
+console.log(page.status);     // 200
+```
+
+## Features
+
+- **Chrome TLS fingerprint**: indistinguishable from a real browser at the network layer
+- **Full JS execution**: SPAs, client-rendered content, and lazy-loaded data
+- **Request interception**: captures every `fetch()`, XHR, and WebSocket call
+- **Framework detection**: identifies Next.js, Nuxt, Inertia, Remix, Astro, SvelteKit, and more
+- **Performance**: parallel module resolution and batch network calls; 700+ module Vite apps render in under 5 seconds
+
+## Hyper-Proxying
+
+Route specific request types through your proxy while letting others connect directly. This reduces proxy bandwidth and latency for requests that don't need it.
+
+```ts
+// Proxy all requests
+const page = await dynafetch({
+  url: "https://example.com",
+  proxy: "http://user:pass@ip:port",
+});
+
+// Only proxy the page fetch and API calls, not static assets
+const page = await dynafetch({
+  url: "https://example.com",
+  proxy: {
+    url: "http://user:pass@ip:port",
+    only: ["page", "api"],
+  },
+});
+```
+
+| Scope | Covers |
+|-|-|
+| `"page"` | Initial HTML document fetch |
+| `"api"` | `fetch()` and XHR calls from page scripts |
+| `"assets"` | JS scripts, ES modules, static resources |
+
+## Headers and cookies
+
+Chrome 146 headers are included by default. Custom headers merge on top; your values override the defaults, everything else is preserved.
+
+```ts
+const page = await dynafetch({
+  url: "https://example.com",
+  headers: { "Accept-Language": "fr-FR" },
+  cookies: { session: "abc123" },
+});
+```
+
+## Quiescence tuning
+
+dynafetch waits for async network activity to complete before returning. These options control that behavior:
+
+```ts
+// Return quickly
+const page = await dynafetch({
+  url: "https://example.com",
+  maxWaitMs: 1000,
+  idleWaitMs: 50,
+});
+
+// Wait longer for slow endpoints
+const page = await dynafetch({
+  url: "https://example.com",
+  maxWaitMs: 5000,
+  idleWaitMs: 200,
+});
+```
+
+| Option | Default | Description |
+|-|-|-|
+| `minWaitMs` | `75` | Minimum ms before checking idle state |
+| `idleWaitMs` | `100` | Ms of zero pending requests to consider settled |
+| `maxWaitMs` | `2000` | Hard cap on wait time |
+| `moduleWaitMs` | `6000` | Max wait for ES module bundling |
+| `timeoutMs` | none | Overall operation timeout |
+
+## Architecture
+
+1. **Harvest**: fetches the HTML document through a Go-based TLS client matching Chrome's handshake. Parses scripts, modulepreloads, and SSR state.
+
+2. **Module graph resolution**: recursively discovers and batch-fetches the full JS dependency tree in parallel. 700+ modules resolve in approximately 5 batch rounds.
+
+3. **Execute**: runs scripts in a sandboxed environment with browser API shims. All network calls are intercepted and routed through the TLS proxy.
+
+4. **Settle**: waits for async activity to complete, then returns rendered HTML, framework metadata, and timing breakdown.
+
+## Requirements
+
+- Node.js 18+
+- Go 1.21+ (for the TLS proxy)
+
+## License
+
+MIT

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@grabbit-labs/dynafetch",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Fetch any website like a real browser. Chrome TLS fingerprinting, JS execution, and request interception in one function call.",
   "type": "module",
   "main": "./src/index.ts",