contextractor 0.4.1__tar.gz

contextractor-0.4.1/.gitignore

@@ -0,0 +1,18 @@
+# Build-time staged assets (flattened JS tree + current-platform .node).
+# Produced by scripts/stage_vendor.py; never committed.
+/src/contextractor/_vendor/cli/
+
+# Python build/test artifacts
+/dist/
+/build/
+*.egg-info/
+__pycache__/
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+
+# Local virtualenvs + uv lockfile (this is a published library; the lock is a
+# dev-only convenience, kept out of the tree per the repo's lockfiles-gitignored policy)
+/.venv/
+/venv/
+uv.lock

contextractor-0.4.1/PKG-INFO

@@ -0,0 +1,231 @@
+Metadata-Version: 2.4
+Name: contextractor
+Version: 0.4.1
+Summary: Drive the Contextractor Node crawler/extractor from Python — clean main-content text in txt, markdown, json, or html.
+Project-URL: Homepage, https://apify.com/glueo/contextractor
+Project-URL: Repository, https://github.com/contextractor/contextractor
+Author-email: glueo <company@glueo.com>
+License-Expression: Apache-2.0
+Keywords: content-extraction,crawlee,crawler,markdown,playwright,trafilatura,web-scraping
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
+Classifier: Topic :: Text Processing :: Markup :: HTML
+Classifier: Typing :: Typed
+Requires-Python: >=3.12
+Requires-Dist: nodejs-wheel-binaries<25,>=24.16
+Provides-Extra: test
+Requires-Dist: pytest-asyncio>=1.4; extra == 'test'
+Requires-Dist: pytest>=9; extra == 'test'
+Requires-Dist: pyyaml>=6; extra == 'test'
+Description-Content-Type: text/markdown
+
+# contextractor
+
+<p align="center"><img width="220" src="https://www.contextractor.com/media/cover-mini.svg" alt="Contextractor"></p>
+
+Crawl web pages and extract clean main-content text — `txt`, `markdown`, `json`,
+or `html` — from Python. Built on [`rs-trafilatura`](https://github.com/Murrough-Foley/rs-trafilatura)
+(extraction) and [Crawlee](https://crawlee.dev/) + [Playwright](https://playwright.dev/)
+(crawling).
+
+This package is a thin, typed wrapper that **drives the bundled Node engine** — it
+does not reimplement the crawler. A self-contained Node runtime ships with the
+wheel (via [`nodejs-wheel-binaries`](https://pypi.org/project/nodejs-wheel-binaries/)),
+so **no Node.js install is required**.
+
+## Install
+
+```bash
+pip install contextractor
+python -m contextractor install   # one-time: download the Chromium browser
+```
+
+Platform wheels are published for macOS (arm64, x86_64), Linux (x86_64, aarch64;
+glibc ≥ 2.28), and Windows (x64). Requires Python 3.12+.
+
+## Quick start
+
+```python
+import contextractor
+
+summary = contextractor.extract(
+    ["https://example.com"],
+    save=["markdown-kvs"],
+    output_dir="./out",
+    max_requests_per_crawl=10,
+)
+print(summary)
+# ExtractSummary(total=1, succeeded=1, failed=0, skipped=0,
+#                output_dir='/abs/out', manifest_path='/abs/out/manifest.json')
+```
+
+Extracted files and a `manifest.json` index are written to `output_dir`
+(default: `./contextractor-output`). The manifest is a JSON array of records, each
+tagged `status: "success" | "failed" | "skipped"`.
+
+### Async
+
+```python
+import asyncio
+import contextractor
+
+async def main():
+    summary = await contextractor.aextract(
+        ["https://example.com", "https://example.org"],
+        save=["markdown-dataset", "original-kvs"],
+        output_dir="./out",
+        max_concurrency=5,
+    )
+    print(summary.succeeded, "of", summary.total)
+
+asyncio.run(main())
+```
+
+## Single-page extraction
+
+`extract_one()` crawls exactly one URL (no link-following) and returns the
+extracted content directly — no output directory, nothing written to disk:
+
+```python
+import contextractor
+
+markdown = contextractor.extract_one("https://example.com")
+print(markdown)  # str — markdown is the default format
+```
+
+Request several formats to get a `dict` keyed by format:
+
+```python
+contents = contextractor.extract_one(
+    "https://example.com",
+    formats=["markdown", "json", "original"],
+)
+print(contents["markdown"])  # extracted markdown
+print(contents["original"])  # raw page HTML
+```
+
+Formats: `txt`, `markdown` (default), `json`, `html`, `original` (the raw page
+HTML). With one requested format the return value is a `str`; with several it is
+a `dict[str, str]`. `extract_one` accepts the single-page subset of the crawl
+options (`ExtractOneOptions`) — e.g. `proxy`, `mode`, `user_agent`, `cookies`,
+`headers`, `headless` — but not crawl-frontier options like `globs` or
+`max_crawl_depth`. A page that cannot be fetched or extracted raises
+`ContextractorError`. If the page yields no content for one of several requested
+formats, that format's key is simply absent from the returned `dict`; when the
+single requested format yields no content, `ContextractorError` is raised.
+
+### Async single page
+
+```python
+import asyncio
+import contextractor
+
+async def main():
+    markdown = await contextractor.aextract_one("https://example.com")
+    print(markdown)
+
+asyncio.run(main())
+```
+
+## Return value
+
+`extract()` / `aextract()` return an `ExtractSummary`:
+
+| Field | Meaning |
+| --- | --- |
+| `total` | Number of records in the manifest |
+| `succeeded` / `failed` / `skipped` | Counts by record `status` |
+| `output_dir` | Absolute path where files + manifest were written |
+| `manifest_path` | Absolute path to `manifest.json` |
+
+Partial failures (some URLs failed) **do not raise** — they are reflected in
+`summary.failed`. Validation/config errors and real crawl failures raise
+`ContextractorError`; a missing browser raises `MissingBrowserError` pointing you
+at `python -m contextractor install`.
+
+## Options
+
+All crawl options are typed keyword arguments (`ExtractOptions`). A selection:
+
+| Option | Type | Notes |
+| --- | --- | --- |
+| `save` | `list[str]` | `format-destination` tokens: `{txt,markdown,json,html,original}-{dataset,kvs}` (e.g. `markdown-kvs`, `original-dataset`). Default `markdown-kvs`; list a format twice to save to both. Saving `original`/`html` to the dataset risks OOM on large pages |
+| `mode` | `str` | `precision`, `balanced` (default), `recall` |
+| `max_requests_per_crawl` | `int` | `0` = unlimited |
+| `max_crawl_depth` | `int` | `0` = unlimited |
+| `globs` / `exclude` | `list[str]` | enqueue / skip URL patterns |
+| `headless` | `bool` | `False` runs a headed browser |
+| `block_media` / `images` | `bool` | toggle media loading / image extraction |
+| `links` / `comments` / `tables` | `bool` | `False` excludes that content |
+| `proxy` | `list[str]` | `http`, `https`, `socks4`, `socks5` URLs |
+| `cookies` | `list[dict]` | initial cookies (JSON) |
+| `headers` | `dict[str, str]` | custom HTTP headers (JSON) |
+| `selector` | `str` | restrict extraction to a CSS selector |
+| `deduplication` | `str` | `minimal`, `standard` (default), `aggressive` |
+| `output_dir` | `str` | where files + manifest are written |
+
+Boolean options that have a CLI default emit a flag only when you set them.
+Editor autocomplete and type-checkers see every option via the `ExtractOptions`
+`TypedDict`.
+
+### Config file
+
+Share configuration across runs with a JSON config file:
+
+```json
+{
+  "mode": "precision",
+  "save": ["markdown-kvs", "json-dataset"],
+  "maxRequestsPerCrawl": 25,
+  "maxCrawlDepth": 2
+}
+```
+
+```python
+contextractor.extract(
+    ["https://example.com"],
+    config_file="config.json",
+    output_dir="./out",
+)
+```
+
+Keyword arguments override values from the config file.
+
+### Proxies
+
+Only `http`, `https`, `socks4`, and `socks5` proxy URLs are accepted; an
+unsupported scheme raises `ProxySchemeError` before anything runs. Proxy
+credentials are never echoed in errors or logs.
+
+```python
+contextractor.extract(
+    ["https://example.com"],
+    proxy=["http://user:pass@proxy.example.com:3128"],
+    output_dir="./out",
+)
+```
+
+## Browser provisioning
+
+Browsers are not bundled in the wheel. Run `python -m contextractor install` once
+to download Chromium for the bundled engine. The standard
+`PLAYWRIGHT_BROWSERS_PATH` environment variable is honored.
+
+## Advanced
+
+- `CONTEXTRACTOR_NODE_PATH` — point at a host Node binary to use instead of the
+  bundled runtime.
+- `storage_dir` — reuse a Crawlee storage directory across runs (defaults to a
+  private temporary directory cleaned up after each call).
+- `timeout` — per-process wall-clock limit (seconds).
+
+## License
+
+Apache-2.0

contextractor-0.4.1/README.md

@@ -0,0 +1,203 @@
+# contextractor
+
+<p align="center"><img width="220" src="https://www.contextractor.com/media/cover-mini.svg" alt="Contextractor"></p>
+
+Crawl web pages and extract clean main-content text — `txt`, `markdown`, `json`,
+or `html` — from Python. Built on [`rs-trafilatura`](https://github.com/Murrough-Foley/rs-trafilatura)
+(extraction) and [Crawlee](https://crawlee.dev/) + [Playwright](https://playwright.dev/)
+(crawling).
+
+This package is a thin, typed wrapper that **drives the bundled Node engine** — it
+does not reimplement the crawler. A self-contained Node runtime ships with the
+wheel (via [`nodejs-wheel-binaries`](https://pypi.org/project/nodejs-wheel-binaries/)),
+so **no Node.js install is required**.
+
+## Install
+
+```bash
+pip install contextractor
+python -m contextractor install   # one-time: download the Chromium browser
+```
+
+Platform wheels are published for macOS (arm64, x86_64), Linux (x86_64, aarch64;
+glibc ≥ 2.28), and Windows (x64). Requires Python 3.12+.
+
+## Quick start
+
+```python
+import contextractor
+
+summary = contextractor.extract(
+    ["https://example.com"],
+    save=["markdown-kvs"],
+    output_dir="./out",
+    max_requests_per_crawl=10,
+)
+print(summary)
+# ExtractSummary(total=1, succeeded=1, failed=0, skipped=0,
+#                output_dir='/abs/out', manifest_path='/abs/out/manifest.json')
+```
+
+Extracted files and a `manifest.json` index are written to `output_dir`
+(default: `./contextractor-output`). The manifest is a JSON array of records, each
+tagged `status: "success" | "failed" | "skipped"`.
+
+### Async
+
+```python
+import asyncio
+import contextractor
+
+async def main():
+    summary = await contextractor.aextract(
+        ["https://example.com", "https://example.org"],
+        save=["markdown-dataset", "original-kvs"],
+        output_dir="./out",
+        max_concurrency=5,
+    )
+    print(summary.succeeded, "of", summary.total)
+
+asyncio.run(main())
+```
+
+## Single-page extraction
+
+`extract_one()` crawls exactly one URL (no link-following) and returns the
+extracted content directly — no output directory, nothing written to disk:
+
+```python
+import contextractor
+
+markdown = contextractor.extract_one("https://example.com")
+print(markdown)  # str — markdown is the default format
+```
+
+Request several formats to get a `dict` keyed by format:
+
+```python
+contents = contextractor.extract_one(
+    "https://example.com",
+    formats=["markdown", "json", "original"],
+)
+print(contents["markdown"])  # extracted markdown
+print(contents["original"])  # raw page HTML
+```
+
+Formats: `txt`, `markdown` (default), `json`, `html`, `original` (the raw page
+HTML). With one requested format the return value is a `str`; with several it is
+a `dict[str, str]`. `extract_one` accepts the single-page subset of the crawl
+options (`ExtractOneOptions`) — e.g. `proxy`, `mode`, `user_agent`, `cookies`,
+`headers`, `headless` — but not crawl-frontier options like `globs` or
+`max_crawl_depth`. A page that cannot be fetched or extracted raises
+`ContextractorError`. If the page yields no content for one of several requested
+formats, that format's key is simply absent from the returned `dict`; when the
+single requested format yields no content, `ContextractorError` is raised.
+
+### Async single page
+
+```python
+import asyncio
+import contextractor
+
+async def main():
+    markdown = await contextractor.aextract_one("https://example.com")
+    print(markdown)
+
+asyncio.run(main())
+```
+
+## Return value
+
+`extract()` / `aextract()` return an `ExtractSummary`:
+
+| Field | Meaning |
+| --- | --- |
+| `total` | Number of records in the manifest |
+| `succeeded` / `failed` / `skipped` | Counts by record `status` |
+| `output_dir` | Absolute path where files + manifest were written |
+| `manifest_path` | Absolute path to `manifest.json` |
+
+Partial failures (some URLs failed) **do not raise** — they are reflected in
+`summary.failed`. Validation/config errors and real crawl failures raise
+`ContextractorError`; a missing browser raises `MissingBrowserError` pointing you
+at `python -m contextractor install`.
+
+## Options
+
+All crawl options are typed keyword arguments (`ExtractOptions`). A selection:
+
+| Option | Type | Notes |
+| --- | --- | --- |
+| `save` | `list[str]` | `format-destination` tokens: `{txt,markdown,json,html,original}-{dataset,kvs}` (e.g. `markdown-kvs`, `original-dataset`). Default `markdown-kvs`; list a format twice to save to both. Saving `original`/`html` to the dataset risks OOM on large pages |
+| `mode` | `str` | `precision`, `balanced` (default), `recall` |
+| `max_requests_per_crawl` | `int` | `0` = unlimited |
+| `max_crawl_depth` | `int` | `0` = unlimited |
+| `globs` / `exclude` | `list[str]` | enqueue / skip URL patterns |
+| `headless` | `bool` | `False` runs a headed browser |
+| `block_media` / `images` | `bool` | toggle media loading / image extraction |
+| `links` / `comments` / `tables` | `bool` | `False` excludes that content |
+| `proxy` | `list[str]` | `http`, `https`, `socks4`, `socks5` URLs |
+| `cookies` | `list[dict]` | initial cookies (JSON) |
+| `headers` | `dict[str, str]` | custom HTTP headers (JSON) |
+| `selector` | `str` | restrict extraction to a CSS selector |
+| `deduplication` | `str` | `minimal`, `standard` (default), `aggressive` |
+| `output_dir` | `str` | where files + manifest are written |
+
+Boolean options that have a CLI default emit a flag only when you set them.
+Editor autocomplete and type-checkers see every option via the `ExtractOptions`
+`TypedDict`.
+
+### Config file
+
+Share configuration across runs with a JSON config file:
+
+```json
+{
+  "mode": "precision",
+  "save": ["markdown-kvs", "json-dataset"],
+  "maxRequestsPerCrawl": 25,
+  "maxCrawlDepth": 2
+}
+```
+
+```python
+contextractor.extract(
+    ["https://example.com"],
+    config_file="config.json",
+    output_dir="./out",
+)
+```
+
+Keyword arguments override values from the config file.
+
+### Proxies
+
+Only `http`, `https`, `socks4`, and `socks5` proxy URLs are accepted; an
+unsupported scheme raises `ProxySchemeError` before anything runs. Proxy
+credentials are never echoed in errors or logs.
+
+```python
+contextractor.extract(
+    ["https://example.com"],
+    proxy=["http://user:pass@proxy.example.com:3128"],
+    output_dir="./out",
+)
+```
+
+## Browser provisioning
+
+Browsers are not bundled in the wheel. Run `python -m contextractor install` once
+to download Chromium for the bundled engine. The standard
+`PLAYWRIGHT_BROWSERS_PATH` environment variable is honored.
+
+## Advanced
+
+- `CONTEXTRACTOR_NODE_PATH` — point at a host Node binary to use instead of the
+  bundled runtime.
+- `storage_dir` — reuse a Crawlee storage directory across runs (defaults to a
+  private temporary directory cleaned up after each call).
+- `timeout` — per-process wall-clock limit (seconds).
+
+## License
+
+Apache-2.0

contextractor-0.4.1/SPEC.md

@@ -0,0 +1,140 @@
+# contextractor (Python) — Specification
+
+A library-only PyPI package that drives the Contextractor Node CLI from Python. It
+**reimplements nothing**: `extract`/`aextract` spawn the bundled `dist/cli.js`,
+translate snake_case options to CLI flags, let the CLI write to disk, then read the
+export `manifest.json` back; `extract_one`/`aextract_one` spawn a single
+`extract-one` child and return the content as values. Python loads no JavaScript
+and no napi `.node` — Node does, when it runs `cli.js`. Standalone hatchling
+package; **not** a pnpm/turbo workspace member.
+
+## Public API
+
+`src/contextractor/__init__.py` exports:
+
+- `extract(urls, *, output_dir=None, storage_dir=None, timeout=None, **opts) -> ExtractSummary` — sync, primary.
+- `aextract(urls, *, output_dir=None, storage_dir=None, timeout=None, **opts) -> ExtractSummary` — async (one crawl per child process via `asyncio.create_subprocess_exec`).
+- `extract_one(url, *, formats=None, timeout=None, **opts) -> str | dict[str, str]` — sync single-page extraction: crawls exactly one URL (no link-following) and returns the content as values. `formats` defaults to `"markdown"`; one requested format returns a `str`, several return a `dict[str, str]` keyed by format. Nothing is persisted. Raises `ContextractorError` when the page cannot be extracted.
+- `aextract_one(url, *, formats=None, timeout=None, **opts) -> str | dict[str, str]` — async counterpart of `extract_one`.
+- `install(browser="chromium") -> None` — provision a Playwright browser via the bundled engine. Also reachable as `python -m contextractor install [browser]`.
+- `ExtractOptions` — `TypedDict(total=False)` of all crawl options (the typed surface for `extract`'s `**opts`).
+- `ExtractOneOptions` — `TypedDict(total=False)` of the single-page option subset (the typed surface for `extract_one`'s `**opts`).
+- `ExtractSummary` — frozen dataclass: `total`, `succeeded`, `failed`, `skipped`, `output_dir`, `manifest_path`.
+- Errors: `ContextractorError` (base), `ProxySchemeError`, `NodeRuntimeError`, `MissingBrowserError`.
+- `__version__` — read via `importlib.metadata.version("contextractor")`.
+
+`urls` accepts a single string or a list. `output_dir` defaults to
+`./contextractor-output` (resolved against the CWD). `storage_dir` defaults to a
+private temp dir that is removed after the call so the manifest reflects only the
+current run; an explicitly-passed `storage_dir` is preserved.
+
+## Orchestration (two invocations)
+
+The CLI `extract` subcommand writes only to Crawlee storage and exits `2` on
+partial failure; the separate `export` subcommand writes `<output_dir>/manifest.json`.
+So each `extract()`/`aextract()` runs **two** child processes:
+
+1. `node cli.js extract <urls> --storage <STORAGE> <mapped flags>`
+2. `node cli.js export --storage <STORAGE> --output-dir <OUTPUT_DIR>`
+
+then `read_summary(<OUTPUT_DIR>/manifest.json)`. One `--storage` path fully
+identifies a run's storage (the CLI always uses the `default` buckets), so export
+reads exactly what extract wrote — no bucket names are threaded.
+
+### Exit-code semantics (`_run.py`)
+
+- extract `0` → continue; `2` → partial success, continue (do **not** raise); `1`/other → raise `ContextractorError`.
+- extract-one `0` → success; `2` → partial (a requested format yielded no content — see the single-page section); `1`/other → raise.
+- export `0` → read manifest; non-zero → raise.
+- Both runners (`_run_sync` / `_run_async`) capture raw bytes and decode stdout/stderr as UTF-8 with `errors="replace"` — never the locale codec (Windows cp1252/cp932 mojibake) and never universal-newline translation (which would corrupt `original` raw HTML).
+- Playwright "Executable doesn't exist" in stderr → `MissingBrowserError` pointing at `python -m contextractor install`.
+- Child stderr is redacted (proxy credentials masked) before being surfaced; argv is never echoed when it carries a proxy.
+- A `timeout` (sync or async) raises `ContextractorError("contextractor timed out")` — never the raw `subprocess.TimeoutExpired`, whose `cmd` would leak the `--proxy` argv.
+
+## Single-page orchestration (`extract_one`)
+
+`extract_one()`/`aextract_one()` run **one** child process and return the content
+as values — nothing is persisted, and no save/output/file/stdout options are
+exposed; the wrapper drives the CLI `extract-one` subcommand internally:
+
+- One requested format → `node cli.js extract-one <url> <mapped flags> --save <fmt>-stdout`; the child's stdout is the raw content (diagnostics go to stderr) and is returned as a `str`.
+- Several formats → one `--save <fmt>-file` per format plus `--output <tempdir>/page`; the wrapper reads the files back into a `dict[str, str]` keyed by format, then removes the temp dir.
+- Read-back names follow the CLI's multi-format `--output` prefix: `page.txt`, `page.md`, `page.json`, `page.html`; `original` lands at `page.original.html` only when `html` is also requested (the CLI's collision tag), else at `page.html`.
+- `formats` accepts a string or a sequence, defaults to `"markdown"`, and deduplicates preserving order; an unknown format raises before spawn.
+- Exit `0` → success. Exit `2` → partial: a requested format yielded no content (the CLI warns on stderr and skips that output). The multi-format dict simply omits that format's key — the npm library's `Partial<Record<…>>` semantics; the single-format route raises `ContextractorError("extract-one produced no <fmt> output")`, since a `str` cannot represent absence. Exit `1`/other → raise (hard failure).
+
+## Option mapping (`_options.py`)
+
+A single data-driven table, `OPTION_SPECS`, applied immediately before spawn (per
+`.claude/rules/python-option-mapping.md`). `ExtractOptions` keys must equal
+`OPTION_SPECS` keys (enforced by `tests/test_options.py`). Categories:
+
+- **scalar** → `--flag <value>` (e.g. `max_crawl_depth`, `mode`, `start_urls_file`, …).
+- **bool-pair** → `--flag` / `--no-flag`: `headless`, `block_media`, `images`.
+- **negation-only** (default include; `False` emits the `--no-` flag): `links`, `comments`, `tables`.
+- **bare-switch** (`True` emits the flag): `purge`, `ignore_cors_and_csp`, `close_cookie_modals`, `ignore_https_errors`, `keep_url_fragment`, `use_sitemaps`, `respect_robots_txt`, `store_skipped_urls`, `verbose` (`-v`).
+- **repeatable** (one flag per item): `proxy`, `globs`, `exclude`, `save` (`format-destination` tokens, e.g. `markdown-kvs`).
+- **json** (`--flag <json.dumps>`): `cookies`, `headers`.
+
+`storage_dir`, `output_dir`, `timeout` are explicit parameters, not in the table.
+`dataset` / `key_value_store` / `request_queue` no longer exist — the CLI always
+uses the `default` buckets under `--storage`.
+`apify_proxy` / `groups` / `use_apify_proxy` are intentionally absent — the CLI
+accepts only `http`/`https`/`socks4`/`socks5` proxies; unknown keys raise, and bad
+proxy schemes raise `ProxySchemeError` before spawn.
+
+`ExtractOneOptions` is the single-page subset of `ExtractOptions` (the proxy,
+session, rendering, network, content, and verbosity knobs only).
+`EXTRACT_ONE_OPTION_KEYS` (frozen from its annotations; enforced by
+`tests/test_options.py` to stay a subset of `OPTION_SPECS`) gates
+`build_extract_one_args`, which raises `ContextractorError` for every
+`extract`-only key (crawl-frontier/storage/output options such as `globs`,
+`selector`, `max_crawl_depth`, `save`, `purge`, plus `session_pool_name` —
+cross-run session sharing needs persisted session-pool state under
+`--storage`, which `extract-one` never touches) before delegating to
+`build_extract_args`.
+
+## Runtime resolution (`_runtime.py`)
+
+- `resolve_node()` — `CONTEXTRACTOR_NODE_PATH` override, else the `nodejs-wheel-binaries` binary at `nodejs_wheel.executable.ROOT_DIR` (`bin/node` on POSIX, `node.exe` on Windows). Restores the exec bit (POSIX) if a wheel ZIP dropped it.
+- `vendor_cli_dir()` — context manager that materializes the staged `_vendor/cli` tree as a real directory via `importlib.resources.as_file()` (a no-op yielding the on-disk path for a normal wheel; extracts to a temp dir, removed on exit, when imported from a zip/pex/shiv). Stays open across the whole subprocess run so the tree outlives the child.
+- `cli_js(cli_dir)` / `playwright_cli_js(cli_dir)` — resolve `dist/cli.js` and `node_modules/playwright/cli.js` inside that materialized tree; raise `NodeRuntimeError` if assets were not staged.
+
+## Asset bundling
+
+The Node CLI ships **un-bundled** (plain `tsc` output; Crawlee/Playwright/commander
+resolve assets via `__dirname`, so esbuild/ncc/SEA are forbidden). At wheel-build
+time `scripts/stage_vendor.py` copies a `pnpm deploy --prod --config.node-linker=hoisted`
+tree (npm-style real files — wheels can't carry pnpm's symlink store) into
+`src/contextractor/_vendor/cli/`, restores `"type": "module"` (pnpm deploy strips
+it), prunes every non-build-platform `.node` — both the bundled
+`dist/native/contextractor-extraction-native.*.node` prebuilds and any legacy
+`node_modules/@contextractor/extraction-native-*` packages (defensive only:
+`@contextractor/*` are devDependencies now, so `pnpm deploy --prod` no longer
+carries them) — and seeds an `__init__.py` in every subdir (for
+`importlib.resources`).
+`_vendor/cli` is gitignored and force-included via the wheel `artifacts` glob. The
+Node runtime itself is **not** bundled (it comes from `nodejs-wheel-binaries`);
+browsers are never bundled (`python -m contextractor install`).
+
+## Packaging & distribution
+
+- Backend: hatchling + `hatch_build.py` (`pure_python=False`, `infer_tag=True`) → `py3-none-{platform}` wheels. Forbid maturin / scikit-build-core / uv_build.
+- `version` is static in `pyproject.toml` (the `/git:release` and `/publish:all` bump target); `__version__` is read from installed metadata.
+- `readme = "README.md"` → the PyPI project page (per `.claude/rules/user-facing-docs.md`); included in the sdist.
+- Wheel matrix: `macosx_*_arm64`, `macosx_*_x86_64`, `manylinux_2_28_x86_64`, `manylinux_2_28_aarch64`, `win_amd64`, plus an sdist. **musl is unsupported** — the napi loader throws a clear import error rather than ship a broken `.node`.
+- CI: `.github/workflows/release-pypi.yml` (cibuildwheel; `CIBW_BEFORE_ALL` stages `_vendor`; auditwheel/delocate repair disabled — there is no ELF Python extension; publish via PyPI Trusted Publishing / OIDC). It is sequenced **after** the napi-refresh PR opened by `build-napi.yml` for a `v*` tag, so wheels bundle current `.node` files — the gate is encoded in `/publish:all`.
+
+## Tests
+
+`pytest` + `pytest-asyncio` (subprocess boundary mocked; no network): argv mapping
+per category, manifest tally, exit-2-is-partial (for both `extract` and
+`extract-one`, incl. the partial multi-format dict and the single-format raise),
+exit-1/other raise, UTF-8/CRLF byte fidelity through the sync runner, async path,
+proxy redaction (incl. the sync-timeout path), node/CLI resolution + exec-bit
+restore, and the Windows-napi consistency check. Two non-mocked layers:
+`tests/test_integration_real_cli.py` drives the repo-built
+`packages/standalone/dist/cli.js` with the system `node` through a multi-format
+`extract_one` against a local `http.server` page (guards the file-naming
+contract the fake CLI re-implements; auto-skips when `node` or the built CLI is
+absent), and one env-gated real e2e (`CONTEXTRACTOR_E2E`).

contextractor-0.4.1/hatch_build.py

@@ -0,0 +1,40 @@
+"""Hatchling build hook forcing a platform-tagged wheel.
+
+The wheel bundles a flattened Node CLI tree plus the build platform's native
+`.node` addon, so it must be tagged `py3-none-{platform}` rather than
+`py3-none-any`. The assets themselves are staged into
+``src/contextractor/_vendor/`` before the build by ``scripts/stage_vendor.py``
+(driven by ``CIBW_BEFORE_ALL`` in CI); this hook only sets the wheel tag.
+
+``CONTEXTRACTOR_WHEEL_PLATFORM`` pins the platform tag explicitly (e.g.
+``manylinux_2_28_x86_64``). CI sets it per matrix row because auditwheel/delocate
+repair is disabled — there is no ELF Python extension to relabel a bare
+``linux_x86_64`` wheel into a PyPI-acceptable ``manylinux`` tag. When unset (e.g. a
+local ``python -m build`` on the native platform), the tag is inferred.
+"""
+
+from __future__ import annotations
+
+import os
+from typing import Any
+
+from hatchling.builders.hooks.plugin.interface import BuildHookInterface
+
+
+class CustomBuildHook(BuildHookInterface):
+    def initialize(self, version: str, build_data: dict[str, Any]) -> None:
+        # Platform-specific (bundles a .node) but ABI-agnostic: the package is pure
+        # Python with no CPython extension, so it runs on any Python 3.x →
+        # py3-none-{platform}, not cp312-cp312-{platform}.
+        build_data["pure_python"] = False
+        build_data["tag"] = f"py3-none-{self._platform()}"
+
+    def _platform(self) -> str:
+        # CI pins the platform tag per matrix row; locally infer it (auditwheel
+        # repair is disabled, so the tag must be correct at build time).
+        pinned = os.environ.get("CONTEXTRACTOR_WHEEL_PLATFORM")
+        if pinned:
+            return pinned
+        from packaging.tags import sys_tags
+
+        return next(iter(sys_tags())).platform