@outcrawl/sdk 0.1.0

package/README.md

@@ -0,0 +1,198 @@
+# @outcrawl/sdk
+
+The typed TypeScript client for [Outcrawl](https://outcrawl.ai): scrape, crawl, search, agent runs,
+and real stealth browsers driven over CDP. One credit balance — no separate proxy, browser or
+stealth credits. Zero runtime dependencies; `playwright` and `agenthands` are optional peers,
+reached only through `await import(...)`, so a caller who never opens a browser never installs them.
+
+## Usage
+
+```ts
+import { Outcrawl } from '@outcrawl/sdk';
+
+const oc = new Outcrawl(); // reads OUTCRAWL_API_KEY; pass { apiKey } to override
+
+const doc = await oc.scrape('https://example.com', { formats: ['markdown'] });
+console.log(doc.usage); // every call returns what it cost, on the result that caused it
+
+const browser = await oc.browser({ profile: 'acme-user-42' });
+const page = await browser.newPage();
+await page.click('#login');     // human input, the default
+await page.raw.click('#login'); // Playwright's own, and recorded on the session
+await browser.close();
+
+// No schema: `data` is the agent's own summary of what it found.
+const run = oc.agent({ task: 'Report the title of the top story on Hacker News.', caps: { steps: 20, budget: '2.00', duration: '5m' } });
+const result = await run;          // await for the result
+console.log(result.data);          // "The top story is …"
+for await (const step of run) {}   // or iterate the log — both on one job is fine, it is two
+                                   // requests against one durable row
+
+// With a schema, `data` is schema-shaped instead, and satisfying it is the success signal.
+const typed = await oc.agent({
+  task: 'Report the title of the top story on Hacker News.',
+  caps: { steps: 20, budget: '2.00', duration: '5m' },
+  schema: { type: 'object', properties: { title: { type: 'string' } }, required: ['title'] },
+});
+console.log(typed.data); // { title: '…' }
+```
+
+## Configuration
+
+| Variable | Meaning | Default |
+| --- | --- | --- |
+| `OUTCRAWL_API_KEY` | your key | required unless you pass `apiKey` or a `transport` |
+| `OUTCRAWL_API_URL` | base url | `https://outcrawl.ai` |
+
+The same two names `@outcrawl/cli`, the Outcrawl MCP server and the Python SDK read, exported here
+as `API_KEY_ENV` and `API_URL_ENV` so a caller telling somebody what to set reads the name off the
+client rather than retyping it. The default host is measured, not assumed: on 2026-09-08
+`https://outcrawl.ai/v1/credits` answered `200` and `https://api.outcrawl.ai/v1/credits` did not
+resolve at all.
+
+## Surface
+
+- `Outcrawl` — one entry point. The wire types (`ScrapeResult`, `AgentResult`, `SessionSummary`,
+  `Profile`, `CreditBalance`, `Usage`, …) are re-exported here, because `@outcrawl/core` is not
+  published and a type only nameable from there is a type no caller can name.
+- `oc.scrape` / `oc.crawl` / `oc.search` — `crawl` is async-iterable.
+- `oc.agent(...)` — `caps` (`steps`, `budget`, `duration`) is required at the type level and at
+  runtime; `steps` and `duration` are hard stops. `result.durationMs` is what agent minutes bill
+  from. It returns an `AgentJob` synchronously: `submitted` is the `{ id, status }` round trip,
+  `await` polls to settled, `for await` tails the log, and everything an `AgentHandle` does is on
+  it too — `status()`, `results()`, `events()`, `cancel()`, and the three human-in-the-loop
+  methods: `control('take' | 'release', who?)`, `answer(answer, answeredBy?)` and
+  `addFile({ name, bytes, contentType? })`, which takes a `Uint8Array` and base64s it for the
+  wire. `settled()` does not treat a PARKED run as settled: a run waiting on a person is about to
+  take more steps.
+- `oc.browser(...)` — allocates a session on the fleet and connects over CDP; nothing launches
+  locally. `Browser.rawCalls` collects a `RawInputMark` per `page.raw.*` call, posted to the session.
+- `oc.sessions.get` / `list` / `export`; `session.live()` streams replay events and
+  `session.takeControl()` hands the wheel to a person.
+- `oc.profiles` and `oc.monitors` — `create` / `list` / `delete` each.
+- `oc.secrets` — `create` / `list` / `delete`, and deliberately **no `get`**: there is no
+  `secrets.get` route to widen. `list` answers the handle, the kind and a card's last four
+  (`hint`), which is everything a human needs to pick one. A run reaches a value only by naming
+  the HANDLE in `AgentRequest.secrets`, and the substitution happens below the model.
+- `oc.rules` — `get` / `set(rules)`. `set` REPLACES the set and answers the classification of
+  what it just stored, so a caller learns on that same call which sentences became `enforced`
+  gates and which are only `guidance`. Read `class`, and read `unenforceable`: it is present
+  exactly when we suspect a customer believes a rule is enforced and it is not.
+- `oc.integrations` — `connect(input)` / `list()` / `delete(id)`, the management half of Outcrawl
+  as an MCP **client**. `connect` is not granting: reach is granted per run through
+  `AgentRequest.integrations`, and a submit that names nothing reaches nothing however many
+  connectors the workspace has. `auth.secret` is a secrets-store handle, never a token. `list`
+  takes no query, because this listing is the blast radius.
+- `oc.credits()` — `{ granted, expired, spent, balance, exhausted, malformed }`, every figure a
+  decimal **string**; never parse to `Number`.
+- `BrowserOptions` — `profile`, `proxy`, `record`, and nothing else.
+- Every error `errorFromWire` can return is exported, because `instanceof` against a class you
+  cannot import is not a contract: `OutcrawlError`, `BadRequestError`, `CapExceededError`,
+  `CapabilityUnavailableError`, `ConcurrencyLimitError`, `ExitUnavailableError`, `InternalError`,
+  `MethodNotAllowedError`, `NoCapacityError`, `NotFoundError`, `PlanRequiredError`,
+  `ProfileInUseError`, `ProfileNotFoundError`, `QuotaExceededError`, `TenantScopeError`,
+  `UnauthorizedError`, plus `isOutcrawlError`, `isKnownErrorCode` and `OUTCRAWL_ERROR_CODES`.
+  Branch on `error.code`, never on `message`. The Python SDK raises the same classes with the same
+  fields and the same messages; `packages/sdk-python/tests/error_parity.json` is the shared
+  contract and both suites read it.
+- `CAPABILITY_AVAILABILITY` — `satisfies Record<CapabilityName, Unavailable | null>`, so a new
+  registry capability does not compile until somebody states whether anything serves it; a
+  capability the table does not declare is refused by name.
+
+## Rules
+
+- **Identity is minted, not chosen.** `timezoneId`, `timezone`, `locale`, `languages`, `userAgent`,
+  `viewport`, `screen`, `deviceScaleFactor`, `geolocation`, `webrtcIp`, `seed`, `identity` and
+  `fingerprint` each throw `IdentityOverrideError` naming what pins them — the `PINNED` map is the list.
+- `proxy` takes `outcrawl://<country>[/<region>[/<city>]]` or the equivalent object, never a
+  provider URL.
+- **Recording is automatic; declining is explicit.** Only `record: false` declines (on `/connect`,
+  only `?record=false` or `?record=0`); a non-boolean `record` on an agent run is a `400`. A declined
+  session never has a recorder attached — no replay, no `replay_sessions` row, no blob,
+  `sessions.export` answers `404`, no `replayUrl`, and `recordingDeclined` is `true`. Declining
+  changes what is kept, never what is charged.
+- `scrape`, `crawl`, `search` and monitor checks record nothing and have no `record` field.
+- A profile is leased while open; a second concurrent open throws `ProfileInUseError`.
+- `oc.sessions.list()` throws `UnsupportedSessionFilterError` for a filter the API does not
+  implement, rather than returning an unfiltered page.
+- **Bandwidth is charged where you chose the bytes.** Egress lands on `oc.browser()` sessions and
+  agent runs; it is recorded and not charged on `scrape`, `crawl`, `search` and monitor checks,
+  where our pipeline decides the byte count. `result.billed.bandwidthBytes` reports it either way.
+- Pricing: [outcrawl.ai/pricing](https://outcrawl.ai/pricing) is authoritative, and
+  `OUTCRAWL_RATE_CARD` in `@outcrawl/persistence` is what production computes against — ask it
+  rather than trusting a transcribed table.
+- The SDK is optional. An unmodified Playwright script pointed at our CDP endpoint keeps working;
+  this package is ergonomics, never the only door.
+
+## Testing
+
+```sh
+cd packages/sdk
+bun test ./test
+bun run build   # the publish path; `prepack` runs the same script
+```
+
+`bun run build` asserts the bundle evaluates and constructs, and that the shipped declarations
+typecheck as a customer resolves them through the `exports` map, under `moduleResolution: nodenext`
+— twice, because "the way a customer does" is not one configuration: once DOM-flavoured with
+`skipLibCheck: false`, and once as a Node backend with `lib: ["ES2024"]` and `@types/node`. The
+second shape was added after a scratch install found three `TS2749` errors that the first could not
+see; see the `Gaps` section.
+
+## Gaps
+
+- **Not on npm, and the code is not the blocker.** No `LICENSE` file in this repository and no
+  `license` field in this package; the build, the tarball and the version are ready. Proven from a
+  scratch install of the tarball outside the repo: scrape, an agent run to `completed` and an
+  events-cursor iteration, all three against production.
+- **One customer tsconfig still reports three errors, and they are in a carried declaration.**
+  Measured 2026-09-08 from a scratch install, five configurations: the default scaffold, strict with
+  `lib: ["ES2024"]` + `@types/node`, strict with DOM and `skipLibCheck: false`, and
+  `moduleResolution: bundler` are all clean; strict with `lib: ["ES2024"]` (no DOM) AND
+  `skipLibCheck: false` reports three `TS2749` on `CryptoKey` in `_deps/core/secrets.d.ts`. That
+  file is carried because `Secret`, `SecretKind` and `SecretCreateRequest` live in it; none of its
+  crypto half is on this package's surface. The fix belongs in `@outcrawl/core`, whose
+  `importSecretKey` names the ambient DOM `CryptoKey` —
+  `Awaited<ReturnType<typeof globalThis.crypto.subtle.importKey>>` resolves under all three lib
+  configurations, verified. Not changed here because this is the packaging layer and that is a
+  shared contract.
+- **Map is priced and not implemented.** It is on the pricing page with no route, no handler and no
+  meter — no method to call and nothing to bill.
+- **Extract has no REST route.** `page.extract()` works only over an open CDP connection, and is
+  charged as that session's browser time and bandwidth, never the published per-page rate.
+- **Bring-your-own-model is not available, and `model` is gone as a field.** `AgentRequest` has no
+  `model`: a request names a HARNESS — `agent: 'scout-1'` (the default) or `'voyager-1'` — and
+  which models a harness runs is ours. A `model` key is refused by the same accept-list check that
+  refuses any unknown field, and no route exists for a customer credential to reach a worker.
+- **`caps.budget` binds the INVOICE, and it does not bind model tokens.** It is a ceiling on what
+  the run is billed — pages at 1 credit, plus the published +5/minute agent and 1/minute browser
+  time — and it is enforced continuously against that figure, so a run that reaches it stops with
+  `status: 'capped'` and `stopReason: 'max-budget'`. It may overshoot by at most one step and it
+  says so when it does: a run billed over its ceiling never reports `task-complete`. Model tokens
+  are still outside it, because the worker deliberately binds no `creditsPerUsd` and sets
+  `budgetPolicy: 'warn'` — it does not know the caller's plan tier at inference time — and model
+  spend is recorded in USD in the worker's cost ledger. That is not a gap in the ceiling: the rate
+  card has no token line at all, and an agent run's model spend is exactly what its billed minutes
+  buy. Until 2026-09-08 this entry said `caps.budget` bound nothing but model spend and told you to
+  rely on `duration`; that was true, and it meant the ceiling was a no-op on every production run.
+- **`credits().exhausted` is reported, not enforced.** Nothing refuses work when the granted pool
+  empties — admission is decided against the plan's monthly allowance. Treat it as a signal to top up.
+- **The availability gate is dormant, and that is a measurement rather than a hope.** Every one of
+  the 31 rows in `CAPABILITY_AVAILABILITY` is an explicit `null` as of 2026-09-08 — including the
+  eleven that landed that day — so nothing is refused locally; the table is kept for the
+  compile-time requirement that a new registry row cannot compile until somebody states whether
+  anything serves it. `assertAvailable` still fails closed on a name the table does not declare,
+  which is the case `satisfies` cannot cover.
+- **Unrelated server-side refusals share one code.** `capability_unavailable` from the API is not
+  this SDK's gate; it means the deployment cannot serve the call. A worker with no model client
+  configured answers it for `agent`, one with no exit provisioner answers it for a request naming
+  a geography, and a control plane with no `SECRETS_KEY` answers it for `secrets.create` and for
+  an agent submit that names `secrets` — the key is optional precisely so a card is never stored
+  under a key that dies with the isolate. An agent submit naming `integrations` on a deployment
+  with no control plane is refused the same way. Branch on `code`, then read `capability` and
+  `missing` — those two fields are what separate these.
+- **`AgentFile` is not nameable from this package.** `AgentHandle.addFile` and `AgentApi.addFile`
+  answer `Result<AgentFile>`, the interface is declared and exported in `src/agent.ts`, and
+  `src/index.ts` does not re-export it — so a customer resolving through the `exports` map cannot
+  name the type their own upload call returns. Same defect the Surface note above describes for
+  `@outcrawl/core` types, one hop closer.