periplus-python-sdk 0.11.0__py3-none-any.whl → 0.13.0__py3-none-any.whl

{periplus_python_sdk-0.11.0.dist-info → periplus_python_sdk-0.13.0.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: periplus-python-sdk
-Version: 0.11.0
+Version: 0.13.0
 Summary: Typed Periplus platform client with SQL and notebook integration
 License-Expression: AGPL-3.0-only
 Project-URL: Repository, https://github.com/elei-io/periplus
@@ -21,8 +21,9 @@ Dynamic: license-file
 # Periplus Python SDK
 
 Typed organization operations and SQL access through the Periplus HTTP API.
-The 0.11.0 platform interface requires the matching API release. It includes
-the source-snapshot metadata introduced in 0.10.0 alongside customer operations.
+The 0.12.0 platform interface requires the matching API release. It replaces the
+discovery, retention and monitoring namespaces with captures, crawls, pins and
+schedules, and includes the source-snapshot metadata introduced in 0.10.0.
 Install from PyPI:
 
 ```sh
@@ -46,7 +47,7 @@ Use HTTPS outside loopback development. Connect directly to the API origin,
 not the public marketing site.
 
 Version 0.9.0 requires personal API keys instead of database credentials. Create
-a key in the app's Settings → My API keys. It inherits your current access in
+a Developer key in the app's Access → SDK & API. It inherits your current access in
 that organization; SQL requires your membership to have `organization:sql:exec`.
 For local development, install `./clients/periplus-python-sdk` from the repository
 root and connect to `http://localhost:8000`.
@@ -55,9 +56,12 @@ root and connect to `http://localhost:8000`.
 returns typed columns/rows for read-only queries. `schema()` returns
 visible tables, column types/descriptions and helper documentation. All SQL uses
 `POST /api/v1/sql`; schema discovery uses `GET /api/v1/schema`. ClickHouse enforces
-permissions. The SDK never retries automatically, including failed queries.
+permissions. Buffered SQL reads retry HTTP 502, 503 and 504 up to twice with bounded
+backoff starting in 0.12.1. Set `sql_retries=0` to disable this, or choose up to three retries. Each
+attempt may observe a newer corpus snapshot. Mutations, transport failures and
+streamed queries are not retried automatically.
 
-Public HTML joins use `parse_id` and `node_index`; `document_id` identifies exact
+Public HTML joins use `capture_id` and `node_index`; `document_id` identifies exact
 raw bytes. Public shorthand uses the `public_v1` schema.
 
 In 0.10.0, `result.source_snapshot` is a typed `SourceSnapshot` containing
@@ -96,15 +100,16 @@ See [the public schema](../../docs/SCHEMA.md) and [query boundary](../../docs/QU
 
 ## Organization resources
 
-The same key selects your organization and inherits your live membership access.
+A Developer key selects your organization and uses your live membership access. Agent keys authenticate only to the hosted MCP endpoint, not this SDK.
 `Client` and `AsyncClient` expose the same namespaces:
 
 | Namespace | Operations |
 | --- | --- |
 | `identity`, `availability` | `get` |
-| `discovery` | `create`, `get`, `list`, `iter`, `arrivals`, `cancel` |
-| `retention`, `monitoring` | `create`, `preview`, `activate`, `get`, `list`, `iter`, `members`, `update`, `pause`, `resume`, `delete` |
-| `saved_queries` | `create`, `get`, `list`, `iter`, `rename`, `delete` |
+| `captures`, `crawls` | `create`, `get`, `list`, `iter`, `pages`, `cancel` |
+| `pins` | `create`, `get`, `list`, `iter`, `captures`, `update`, `delete` |
+| `schedules` | `create`, `get`, `list`, `iter`, `pages`, `captures`, `update`, `pause`, `resume`, `delete` |
+| `saved_queries` | `create`, `get`, `list`, `iter`, `update`, `rename`, `delete` |
 | `members` | `list`, `update_role`, `remove` |
 | `invitations` | `list`, `create`, `cancel` |
 | `api_keys` | `create`, `list`, `iter`, `revoke` |
@@ -112,30 +117,61 @@ The same key selects your organization and inherits your live membership access.
 | `query_history` | `list`, `iter`, `get`, `summary` |
 | `audit` | `list`, `iter` |
 
+Choose the resource by what you know and how long you need it:
+
+| Resource | Use it when |
+| --- | --- |
+| `captures` | You know the URLs (1–100) and need them in SQL now |
+| `crawls` | You know a starting point, not the URLs |
+| `pins` | You need exact captures for longer than seven days |
+| `schedules` | You need the same pages again later |
+
 ```python
 from periplus_sdk import Client
 
 with Client() as client:
-    preview = client.retention.preview(
+    capture = client.captures.create(urls=["https://example.com/pricing"])
+    capture = client.captures.get(capture, wait_seconds=30)
+    if capture.finished and capture.queryable:
+        for page in client.captures.pages(capture).items:
+            print(page.url, page.status, page.capture_id)
+
+    crawl = client.crawls.create(seeds=["https://example.com/docs/"], page_limit=200,
+                                 allowed_paths=["/docs/*"])
+
+    pin = client.pins.create(
         name="Research sources", days=90,
         sql="SELECT capture_id FROM captures WHERE domain(url) = ?",
         parameters=["example.com"],
     )
-    print(preview.id, preview.capture_count, preview.sample)
-    # Review the selection before calling:
-    # active = client.retention.activate(preview)
+    schedule = client.schedules.create(name="Pricing", every_days=7,
+                                       urls=["https://example.com/pricing"])
+    print(pin.captures, schedule.projected_pages_per_month)
 ```
 
-SQL previews persist inactive, frozen selections; activation never reruns SQL.
-Explicit IDs/URLs passed to `create` activate immediately. Monitoring URLs must
-already be known in the corpus. Use `members` and its `next_cursor` for complete
-membership: policy detail contains a sample. Updates/deletion accept a fetched
-`Policy` or an ID with `expected_version`; conflicts are never retried.
+Every capture is kept for seven days after collection unless a pin keeps it. A
+capture always fetches again and never returns an existing capture, so check
+coverage with SQL first. `request_id` (a capture or crawl `id`) is what you
+ordered; each page's `capture_id` is what the crawler produced and is the SQL key.
+`get(..., wait_seconds=N)` returns as soon as the request is finished and queryable,
+or after at most 30 seconds; the SDK never polls on its own.
+
+Pins and schedules accept explicit IDs/URLs or one read-only SQL query with
+parameters, and are active immediately. SQL runs once at creation and its exact
+result is locked in; running the same query beforehand is only an estimate.
+Schedule URLs need not be in the corpus. `pin_days` on a capture, crawl or schedule
+pins each successful capture it produces and requires `organization:pins:write`.
+`pins.update` changes `days` (moving every capture's expiry by the difference) or
+the name; `schedules.update` changes `every_days` or the name. Both accept a fetched
+object or an ID with `expected_version`; conflicts are never retried. Deletion needs
+no version: deleting a pin removes future protection, not captures.
 
 Paginated lists return `Page[T]` (`items`, `next_cursor`). Pass cursors unchanged
 or use `iter()`; membership/invitation lists are bounded snapshots instead.
-Offset-based lists can shift during concurrent changes. Query history and arrivals
-use keyset cursors. UTC usage ranges have an exclusive end date, at most 93 days.
+Offset-based lists can shift during concurrent changes. Query history, request
+page feeds and scheduled capture feeds use opaque keyset cursors. UTC usage ranges have an exclusive end date, at most 93 days. Usage reports pages
+captured (broken down by capture, crawl or schedule), challenge-resolution pages and
+pinned capture-days.
 Query history is best-effort and expires after 30 days; it is not a billing ledger.
 
 `ApiError` includes HTTP status, code, optional request ID, validation fields and

periplus_python_sdk-0.13.0.dist-info/RECORD

@@ -0,0 +1,22 @@
+periplus_python_sdk-0.13.0.dist-info/licenses/licensing/LICENSE,sha256=DZak_2itbUtvHzD3E7GNUYSRK6jdOJ-GqncQ2weavLA,34523
+periplus_python_sdk-0.13.0.dist-info/licenses/licensing/NOTICE,sha256=W4W3pzZe-a0ebw4MxgWVmFGj7NSGL1qj8hFx4YG9iF0,105
+periplus_python_sdk-0.13.0.dist-info/licenses/licensing/README.md,sha256=NABTC2wm2Y9iRNMnlMeyVMA2dsAw7XRmCrsV1B2TQdk,1996
+periplus_python_sdk-0.13.0.dist-info/licenses/licensing/THIRD_PARTY_NOTICES.md,sha256=-egdv-dVsKhyRE2pPaHZgg4XYdpQfpH0VwM-sHYjUTg,9643
+periplus_sdk/__init__.py,sha256=QzgQOvgN2YBTr6Vf65uCwKzAPjjnxsq4kdXy2XWxRAs,1020
+periplus_sdk/async_resources.py,sha256=APJ2S-5OsjLwkOzdl3N76yZXtCq8F49EiNlW4kZrRaw,20583
+periplus_sdk/async_stream.py,sha256=O5hO_w_-ooQuDV1fOZRX4Qei1-w7Wri8eIlsn3yCpGQ,4592
+periplus_sdk/client.py,sha256=zVt1uso8LXQuwjVjOz3Nm9PH8bCAv1ZwLbuAvSj5BAI,15592
+periplus_sdk/dbapi.py,sha256=dkJA-DkrMtcgMGjLQ5Vv8-pb6riRrAaG4Yqrn22uRZY,12064
+periplus_sdk/errors.py,sha256=nnbmRzEu1GEFGeKNsLqEo6pHCFu0oFy__5t-yBLO2hw,965
+periplus_sdk/py.typed,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
+periplus_sdk/resources.py,sha256=Ui08saEs6cUgDttGU7R_VBLDQmFUmk5xJ07uNS3Db2E,20178
+periplus_sdk/resources_types.py,sha256=_UBaTWaut8xq9jDcrJKZdswTDFjSi179P7V6LFjOhQU,8949
+periplus_sdk/sql_api.py,sha256=uUWdwMRSZFMNXXy3-yLozcuT6WluvT6BTjYDB0lNrAU,1917
+periplus_sdk/sqlalchemy.py,sha256=XE_OzdDh9OZ8goWWRoKlBzp7Q0FM-eS9-hjH7Ofzy-E,6292
+periplus_sdk/stream.py,sha256=6Ytb6FkP3Y5C4sDA_SDdU0MJHBQdgHde1lNM6YxHdIQ,5560
+periplus_sdk/types.py,sha256=TSaQRjN5vng_yV4hjZV03s5sU_RHL874qJe40-Hb1KQ,3644
+periplus_python_sdk-0.13.0.dist-info/METADATA,sha256=j0tHG6yuYZ5l-Wnn1ZZIzH3x7CMFT7r4Lb40pB4EQgs,9208
+periplus_python_sdk-0.13.0.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
+periplus_python_sdk-0.13.0.dist-info/entry_points.txt,sha256=Pr14L_7AhLinq-4qDxB4awFVubrvR1BEfkaFVRpdCbU,73
+periplus_python_sdk-0.13.0.dist-info/top_level.txt,sha256=o41t5TzwgoxzSmbKoP6olWW1FyEAGWVYTjeoKadBK40,13
+periplus_python_sdk-0.13.0.dist-info/RECORD,,

{periplus_python_sdk-0.11.0.dist-info → periplus_python_sdk-0.13.0.dist-info}/licenses/licensing/THIRD_PARTY_NOTICES.md

@@ -7,7 +7,7 @@ The Periplus license does not replace these terms.
 
 ## Selectolax and Lexbor
 
-The backend pins Selectolax 0.4.11 (MIT) and its bundled Lexbor 3.1.0
+The backend pins Selectolax 0.4.12 (MIT) and its bundled Lexbor 3.1.0
 (Apache-2.0). The DOM adapter's read-only native layouts follow Lexbor's DOM and
 HTML interface headers. Selectolax retains its packaged MIT license; Periplus
 includes Lexbor's license and notice under `licensing/third-party/lexbor/`

periplus_sdk/__init__.py

@@ -3,9 +3,11 @@ from .dbapi import connect
 from .client import AsyncClient, Client
 from .errors import ApiError, ConfigurationError, PeriplusError, ResponseError, TransportError
 from .types import Diagnostic, PreparedQuery, QueryHelper, QueryHelpers, QueryResult, SourceSnapshot
-from .resources_types import Page, Policy, Discovery, Member, Invitation, ApiKey, CreatedApiKey, Identity, Usage
+from .resources_types import (FollowRule, QueryCondition, Page, Request, RequestPage, Pin, PinCapture, Schedule, SchedulePage, Member,
+                              Invitation, ApiKey, CreatedApiKey, Identity, Usage)
 
 __all__ = ["connect", "AsyncClient", "Client", "ApiError", "ConfigurationError", "PeriplusError",
            "ResponseError", "TransportError", "Diagnostic", "PreparedQuery", "QueryHelper",
-           "QueryHelpers", "QueryResult", "Page", "Policy", "Discovery", "Member", "Invitation",
+           "QueryHelpers", "QueryResult", "FollowRule", "QueryCondition", "Page", "Request", "RequestPage", "Pin", "PinCapture",
+           "Schedule", "SchedulePage", "Member", "Invitation",
            "ApiKey", "CreatedApiKey", "Identity", "Usage", "SourceSnapshot"]

periplus_sdk/async_resources.py

@@ -1,7 +1,7 @@
 """Async customer operations; the same contract as resources.py."""
 from __future__ import annotations
 from collections.abc import AsyncIterator, Sequence
-from datetime import date
+from datetime import date, datetime
 from typing import Generic, Literal, TypeVar
 from urllib.parse import quote
 from uuid import UUID, uuid4
@@ -9,9 +9,9 @@ from uuid import UUID, uuid4
 from pydantic import JsonValue
 
 from .resources_types import (
-    ApiKey, ArrivalsPage, AuditEvent, Availability, CreatedApiKey, Discovery,
-    Identity, Invitation, Member, Members, Page, Policy, PolicyMember,
-    QueryExecution, QueryUsage, Role, SavedQuery, Usage,
+    ApiKey, AuditEvent, Availability, CreatedApiKey, Identity, Invitation, Member, Members,
+    FollowRule, Page, Pin, PinCapture, QueryExecution, QueryUsage, Request, RequestPagesPage, Role,
+    SavedQuery, Schedule, SchedulePage, Usage,
 )
 
 
@@ -33,14 +33,28 @@ def page_size(limit: int) -> int:
     return limit
 
 
-def policy_ref(policy: Policy | str | UUID, expected_version: int | None, kind: str):
-    if isinstance(policy, Policy):
-        if policy.kind != kind:
-            raise ValueError("Policy belongs to a different resource")
-        return policy.id, policy.version if expected_version is None else expected_version
+def versioned_ref(value: Pin | Schedule | str | UUID, expected_version: int | None, model: type):
+    if isinstance(value, (Pin, Schedule)):
+        if not isinstance(value, model):
+            raise ValueError(f"Expected a {model.__name__}")
+        return value.id, value.version if expected_version is None else expected_version
     if expected_version is None:
-        raise ValueError("An ID requires expected_version; fetch the policy before changing it")
-    return policy, expected_version
+        raise ValueError("An ID requires expected_version; fetch the resource before changing it")
+    return value, expected_version
+
+
+def object_id(value) -> str:
+    return path_id(value.id if isinstance(value, (Request, Pin, Schedule)) else value)
+
+
+def selection(ids_name: str, ids: Sequence | None, sql: str | None, parameters: Sequence[JsonValue]) -> dict:
+    if (ids is None) == (sql is None):
+        raise ValueError(f"Choose either {ids_name} or sql")
+    if sql is None:
+        if parameters:
+            raise ValueError("parameters require sql")
+        return {ids_name: [str(value) for value in ids]}
+    return {"sql": sql, "parameters": list(parameters)}
 
 
 def member_ref(member: Member | str, expected_role: Role | None):
@@ -83,133 +97,186 @@ class AvailabilityResource(Resource):
         return await self._client._resource_request("GET", "access", Availability)
 
 
-class DiscoveryResource(PagedResource[Discovery]):
-    async def select_urls(self, sql: str, parameters: Sequence[JsonValue] = ()) -> list[str]:
-        """Resolve a complete starting-page selection without creating discovery."""
-        from .selection import starting_urls
-        return starting_urls(await self._client.execute(sql, parameters))
-
-    async def create(self, *, urls: Sequence[str], link_depth: int = 2, page_limit: int = 10_000,
-               follow_external_links: bool = False, allowed_hosts: Sequence[str] = (),
-               excluded_hosts: Sequence[str] = (), allowed_paths: Sequence[str] = (),
-               excluded_paths: Sequence[str] = (), retain_days: int | None = None,
-               id: UUID | None = None) -> Discovery:
-        return await self._client._resource_request("POST", "discoveries", Discovery, json={
-            "id": str(id or uuid4()), "specification": {
-                "seed_urls": list(urls), "max_depth": link_depth, "page_limit": page_limit,
-                "follow_scope": "linked_sites" if follow_external_links else "starting_sites",
-                "allowed_hosts": list(allowed_hosts), "excluded_hosts": list(excluded_hosts),
-                "allowed_paths": list(allowed_paths), "excluded_paths": list(excluded_paths),
-                "retain_days": retain_days}})
-
-    async def list(self, *, cursor: str | None = None, limit: int = 20,
-             status: Literal["active", "paused", "settled"] | None = None) -> Page[Discovery]:
-        return await self._client._resource_request("GET", "discoveries", Page[Discovery],
-            params={"offset": page_offset(cursor), "limit": page_size(limit), "status": status})
-
-    async def get(self, id: str | UUID) -> Discovery:
-        return await self._client._resource_request("GET", f"discoveries/{path_id(id)}", Discovery)
-
-    async def cancel(self, id: str | UUID) -> Discovery:
-        return await self._client._resource_request("POST", f"discoveries/{path_id(id)}/actions",
-            Discovery, json={"action": "cancel"})
-
-    async def arrivals(self, id: str | UUID, *, cursor: str | None = None, limit: int = 20) -> ArrivalsPage:
-        return await self._client._resource_request("GET", f"discoveries/{path_id(id)}/arrivals",
-            ArrivalsPage, params={"cursor": cursor, "limit": page_size(limit)})
-
-
-class PolicyResource(PagedResource[Policy]):
-    kind: Literal["retention", "refresh"]
-
-    async def list(self, *, cursor: str | None = None, limit: int = 20,
-             include_drafts: bool = True) -> Page[Policy]:
-        return await self._client._resource_request("GET", "page-policies", Page[Policy],
-            params={"kind": self.kind, "offset": page_offset(cursor), "limit": page_size(limit),
-                    "include_drafts": include_drafts})
-
-    async def get(self, id: str | UUID) -> Policy:
-        return await self._client._resource_request("GET", f"page-policies/{path_id(id)}", Policy,
-            params={"kind": self.kind, "members_limit": 20})
-
-    async def members(self, id: str | UUID, *, cursor: str | None = None, limit: int = 100) -> Page[PolicyMember]:
-        return await self._client._resource_request("GET", f"page-policies/{path_id(id)}/members",
-            Page[PolicyMember], params={"kind": self.kind, "offset": page_offset(cursor), "limit": page_size(limit)})
-
-    async def _create(self, name, days, selection, retain_days, id) -> Policy:
-        return await self._client._resource_request("POST", "page-policies", Policy, json={
-            "id": str(id or uuid4()), "kind": self.kind, "name": name, "days": days,
-            "selection": selection, "retain_days": retain_days})
-
-    async def _action(self, policy, action, expected_version) -> Policy:
-        id, version = policy_ref(policy, expected_version, self.kind)
-        return await self._client._resource_request("POST", f"page-policies/{path_id(id)}/actions", Policy,
-            params={"kind": self.kind}, json={"action": action, "expected_version": version})
-
-    async def activate(self, policy: Policy | str | UUID, *, expected_version: int | None = None) -> Policy:
-        """Activate the persisted SQL selection without executing SQL again."""
-        return await self._action(policy, "activate", expected_version)
-
-    async def pause(self, policy: Policy | str | UUID, *, expected_version: int | None = None) -> Policy:
-        return await self._action(policy, "pause", expected_version)
-
-    async def resume(self, policy: Policy | str | UUID, *, expected_version: int | None = None) -> Policy:
-        """Resume without restarting the original retention period."""
-        return await self._action(policy, "resume", expected_version)
-
-    async def delete(self, policy: Policy | str | UUID, *, expected_version: int | None = None) -> None:
-        id, version = policy_ref(policy, expected_version, self.kind)
-        return await self._client._resource_request("DELETE", f"page-policies/{path_id(id)}", None,
-            params={"kind": self.kind, "expected_version": version})
-
-    async def _update(self, policy, expected_version, fields) -> Policy:
-        id, version = policy_ref(policy, expected_version, self.kind)
-        return await self._client._resource_request("PATCH", f"page-policies/{path_id(id)}", Policy,
-            params={"kind": self.kind}, json={"expected_version": version, **fields})
-
-
-class RetentionResource(PolicyResource):
-    kind = "retention"
-
-    async def create(self, *, name: str, capture_ids: Sequence[str | UUID], days: int,
-               id: UUID | None = None) -> Policy:
-        """Protect explicit capture IDs immediately."""
-        return await self._create(name, days, {"capture_ids": [str(id) for id in capture_ids]}, None, id)
-
-    async def preview(self, *, name: str, sql: str, days: int,
-                parameters: Sequence[JsonValue] = (), id: UUID | None = None) -> Policy:
-        """Save an inactive, frozen SQL selection. Call activate separately."""
-        return await self._create(name, days, {"sql": sql, "parameters": list(parameters)}, None, id)
-
-    async def update(self, policy: Policy | str | UUID, *, name: str | None = None,
-               days: int | None = None, expected_version: int | None = None) -> Policy:
-        return await self._update(policy, expected_version,
-            {key: value for key, value in {"name": name, "days": days}.items() if value is not None})
-
-
-class MonitoringResource(PolicyResource):
-    kind = "refresh"
-
-    async def create(self, *, name: str, urls: Sequence[str], interval_days: int,
-               retain_days: int | None = None, id: UUID | None = None) -> Policy:
-        """Monitor exact known corpus URLs immediately; does not follow links."""
-        return await self._create(name, interval_days, {"urls": list(urls)}, retain_days, id)
-
-    async def preview(self, *, name: str, sql: str, interval_days: int,
-                parameters: Sequence[JsonValue] = (), retain_days: int | None = None,
-                id: UUID | None = None) -> Policy:
-        return await self._create(name, interval_days, {"sql": sql, "parameters": list(parameters)}, retain_days, id)
+class RequestResource(PagedResource[Request]):
+    """Shared reads for captures and crawls: one request view and one page feed."""
+    noun: Literal["captures", "crawls"]
+
+    async def list(self, *, finished: bool | None = None, cursor: str | None = None,
+             limit: int = 20) -> Page[Request]:
+        return await self._client._resource_request("GET", self.noun, Page[Request],
+            params={"finished": finished, "offset": page_offset(cursor), "limit": page_size(limit)})
+
+    async def get(self, id: Request | str | UUID, *, wait_seconds: int = 0) -> Request:
+        """Return as soon as the request is finished and queryable, or after `wait_seconds` (0-30).
+
+        Waiting never delays capture. The SDK does not poll; call again to keep waiting."""
+        if not 0 <= wait_seconds <= 30:
+            raise ValueError("wait_seconds must be between 0 and 30")
+        return await self._client._resource_request("GET", f"{self.noun}/{object_id(id)}", Request,
+            params={"wait_seconds": wait_seconds or None})
+
+    async def pages(self, id: Request | str | UUID, *,
+              status: Literal["captured", "failed", "pending"] | None = None,
+              search: str | None = None, host: str | None = None, depth: int | None = None,
+              failure_code: str | None = None, render_incomplete: bool | None = None,
+              cursor: str | None = None, limit: int = 20) -> RequestPagesPage:
+        """Requested pages, newest first; `capture_id` is the SQL key. Pass cursors unchanged.
+
+        Failure-code filtering scans a bounded window, so an empty page can still have a cursor.
+        `render_incomplete=True` lists captured pages whose HTML was only a loading placeholder."""
+        return await self._client._resource_request("GET", f"{self.noun}/{object_id(id)}/pages", RequestPagesPage,
+            params={"status": status, "search": search, "host": host, "depth": depth,
+                    "failure_code": failure_code, "render_incomplete": render_incomplete,
+                    "cursor": cursor, "limit": page_size(limit)})
+
+    async def cancel(self, id: Request | str | UUID) -> Request:
+        """Stop remaining work. Captures already made are kept."""
+        return await self._client._resource_request("POST", f"{self.noun}/{object_id(id)}/cancel", Request)
+
+
+class CapturesResource(RequestResource):
+    noun = "captures"
+
+    async def create(self, *, urls: Sequence[str], pin_days: int | None = None,
+               id: UUID | None = None) -> Request:
+        """Fetch 1-100 known URLs once, now.
+
+        Always fetches again and never returns an existing capture: check coverage with SQL
+        first. `pin_days` pins each successful capture (requires organization:pins:write).
+        Keep `id` to reconcile an uncertain write; the same ID and input return the same request."""
+        return await self._client._resource_request("POST", "captures", Request, json={
+            "id": str(id or uuid4()), "urls": list(urls),
+            **({"pin_days": pin_days} if pin_days is not None else {})})
+
+
+class CrawlsResource(RequestResource):
+    noun = "crawls"
+
+    async def admission_preview(self, *, seeds: Sequence[str], accepted_queue_wait_seconds: int | None = None) -> Availability:
+        return await self._client._resource_request("POST", "access/crawl-preview", Availability, json={
+            "seed_urls": list(seeds), "accepted_queue_wait_seconds": accepted_queue_wait_seconds})
+
+    async def create(self, *, seeds: Sequence[str], page_limit: int, max_depth: int = 2,
+               follow_scope: Literal["starting_sites", "linked_sites"] = "starting_sites",
+               allowed_hosts: Sequence[str] = (), excluded_hosts: Sequence[str] = (),
+               allowed_paths: Sequence[str] = (), excluded_paths: Sequence[str] = (),
+               follow_rules: Sequence[FollowRule | dict] = (),
+               accepted_queue_wait_seconds: int | None = None,
+               pin_days: int | None = None, id: UUID | None = None) -> Request:
+        """Follow links from seeds when the URLs are not known yet.
+
+        `page_limit` means up to N pages including seeds, not guaranteed coverage. `max_depth`
+        is at least 1; use `captures.create` for known URLs. Hosts include subdomains; path
+        patterns match the whole URL-encoded path and only `*` is special."""
+        return await self._client._resource_request("POST", "crawls", Request, json={
+            "id": str(id or uuid4()), "seeds": list(seeds), "page_limit": page_limit,
+            "max_depth": max_depth, "follow_scope": follow_scope,
+            **({"accepted_queue_wait_seconds": accepted_queue_wait_seconds} if accepted_queue_wait_seconds is not None else {}),
+            "allowed_hosts": list(allowed_hosts), "excluded_hosts": list(excluded_hosts),
+            "allowed_paths": list(allowed_paths), "excluded_paths": list(excluded_paths),
+            "follow_rules": [rule.model_dump(exclude_none=True) if isinstance(rule, FollowRule) else rule
+                             for rule in follow_rules],
+            **({"pin_days": pin_days} if pin_days is not None else {})})
+
+
+class PinsResource(PagedResource[Pin]):
+    async def create(self, *, name: str, days: int, capture_ids: Sequence[str | UUID] | None = None,
+               sql: str | None = None, parameters: Sequence[JsonValue] = (),
+               saved_query_id: str | UUID | None = None, id: UUID | None = None) -> Pin:
+        """Keep exact captures for `days`, active immediately.
+
+        Pass `capture_ids`, one read-only `sql` query returning `capture_id`, or the
+        `saved_query_id` of such a query in your organization. SQL runs once at creation and its
+        exact result is locked in; running it beforehand is only an estimate. Membership never
+        changes. SQL selection also requires organization:sql:exec; a saved query also
+        requires organization:sql:read."""
+        if saved_query_id is not None:
+            if capture_ids is not None or sql is not None:
+                raise ValueError("Choose one of capture_ids, sql or saved_query_id")
+            source = {"saved_query_id": str(saved_query_id), "parameters": list(parameters)}
+        else:
+            source = selection("capture_ids", capture_ids, sql, parameters)
+        return await self._client._resource_request("POST", "pins", Pin, json={
+            "id": str(id or uuid4()), "name": name, "days": days, **source})
+
+    async def list(self, *, cursor: str | None = None, limit: int = 20) -> Page[Pin]:
+        return await self._client._resource_request("GET", "pins", Page[Pin],
+            params={"offset": page_offset(cursor), "limit": page_size(limit)})
 
-    async def update(self, policy: Policy | str | UUID, *, name: str | None = None,
-               interval_days: int | None = None, retain_days: int | None = None,
-               clear_retention: bool = False, expected_version: int | None = None) -> Policy:
-        if clear_retention and retain_days is not None:
-            raise ValueError("Choose retain_days or clear_retention, not both")
-        fields = {key: value for key, value in {"name": name, "days": interval_days,
-                  "retain_days": retain_days}.items() if value is not None}
-        if clear_retention:
-            fields["retain_days"] = None
-        return await self._update(policy, expected_version, fields)
+    async def get(self, id: Pin | str | UUID) -> Pin:
+        return await self._client._resource_request("GET", f"pins/{object_id(id)}", Pin)
+
+    async def captures(self, id: Pin | str | UUID, *, cursor: str | None = None,
+                 limit: int = 100) -> Page[PinCapture]:
+        """Pinned captures with each one's expiry."""
+        return await self._client._resource_request("GET", f"pins/{object_id(id)}/captures", Page[PinCapture],
+            params={"offset": page_offset(cursor), "limit": page_size(limit)})
+
+    async def update(self, pin: Pin | str | UUID, *, name: str | None = None, days: int | None = None,
+               expected_version: int | None = None) -> Pin:
+        """Rename, or change `days`: every capture's expiry moves by the difference."""
+        id, version = versioned_ref(pin, expected_version, Pin)
+        return await self._client._resource_request("PATCH", f"pins/{path_id(id)}", Pin, json={
+            "expected_version": version,
+            **{key: value for key, value in {"name": name, "days": days}.items() if value is not None}})
+
+    async def delete(self, id: Pin | str | UUID) -> None:
+        """Remove future protection. Captures and past usage are not deleted."""
+        return await self._client._resource_request("DELETE", f"pins/{object_id(id)}", None)
+
+
+class SchedulesResource(PagedResource[Schedule]):
+    async def create(self, *, name: str, every_days: int, urls: Sequence[str] | None = None,
+               sql: str | None = None, parameters: Sequence[JsonValue] = (),
+               pin_days: int | None = None, id: UUID | None = None) -> Schedule:
+        """Capture a fixed URL list every `every_days`, active immediately.
+
+        Pass `urls` (they need not be in the corpus) or one read-only `sql` query returning a
+        `url` column; SQL runs once and its URLs are locked in. Scheduled runs follow no links.
+        `pin_days` pins every successful scheduled capture (requires organization:pins:write)."""
+        return await self._client._resource_request("POST", "schedules", Schedule, json={
+            "id": str(id or uuid4()), "name": name, "every_days": every_days,
+            **selection("urls", urls, sql, parameters),
+            **({"pin_days": pin_days} if pin_days is not None else {})})
+
+    async def list(self, *, cursor: str | None = None, limit: int = 20) -> Page[Schedule]:
+        return await self._client._resource_request("GET", "schedules", Page[Schedule],
+            params={"offset": page_offset(cursor), "limit": page_size(limit)})
+
+    async def get(self, id: Schedule | str | UUID) -> Schedule:
+        return await self._client._resource_request("GET", f"schedules/{object_id(id)}", Schedule)
+
+    async def pages(self, id: Schedule | str | UUID, *, search: str | None = None,
+              cursor: str | None = None, limit: int = 100) -> Page[SchedulePage]:
+        """Scheduled URLs with their latest success, failure and next due time."""
+        return await self._client._resource_request("GET", f"schedules/{object_id(id)}/pages", Page[SchedulePage],
+            params={"search": search, "offset": page_offset(cursor), "limit": page_size(limit)})
+
+    async def captures(self, id: Schedule | str | UUID, *, status: Literal["captured", "failed"] | None = None,
+                 search: str | None = None, render_incomplete: bool | None = None,
+                 cursor: str | None = None, limit: int = 20) -> RequestPagesPage:
+        """Results of scheduled runs, newest first, in the request page-feed shape."""
+        return await self._client._resource_request("GET", f"schedules/{object_id(id)}/captures", RequestPagesPage,
+            params={"status": status, "search": search, "render_incomplete": render_incomplete,
+                    "cursor": cursor, "limit": page_size(limit)})
+
+    async def update(self, schedule: Schedule | str | UUID, *, name: str | None = None,
+               every_days: int | None = None, expected_version: int | None = None) -> Schedule:
+        id, version = versioned_ref(schedule, expected_version, Schedule)
+        return await self._client._resource_request("PATCH", f"schedules/{path_id(id)}", Schedule, json={
+            "expected_version": version, **{key: value for key, value in
+                {"name": name, "every_days": every_days}.items() if value is not None}})
+
+    async def pause(self, id: Schedule | str | UUID) -> Schedule:
+        """Stop new runs; an unstarted run is cancelled, started captures finish."""
+        return await self._client._resource_request("POST", f"schedules/{object_id(id)}/pause", Schedule)
+
+    async def resume(self, id: Schedule | str | UUID) -> Schedule:
+        return await self._client._resource_request("POST", f"schedules/{object_id(id)}/resume", Schedule)
+
+    async def delete(self, id: Schedule | str | UUID) -> None:
+        """Remove the schedule. Captures it produced are kept."""
+        return await self._client._resource_request("DELETE", f"schedules/{object_id(id)}", None)
 
 
 class SavedQueriesResource(PagedResource[SavedQuery]):
@@ -224,6 +291,13 @@ class SavedQueriesResource(PagedResource[SavedQuery]):
         return await self._client._resource_request("POST", "saved-queries", SavedQuery,
             json={"id": str(id or uuid4()), "name": name, "sql": sql})
 
+    async def update(self, id: str | UUID, *, name: str | None = None, sql: str | None = None) -> SavedQuery:
+        """Revise the shared definition without executing SQL or changing pinned results."""
+        payload = {key: value for key, value in {"name": name, "sql": sql}.items() if value is not None}
+        if not payload:
+            raise ValueError("Supply name, sql, or both")
+        return await self._client._resource_request("PATCH", f"saved-queries/{path_id(id)}", SavedQuery, json=payload)
+
     async def rename(self, id: str | UUID, *, name: str) -> SavedQuery:
         return await self._client._resource_request("PATCH", f"saved-queries/{path_id(id)}", SavedQuery, json={"name": name})
 
@@ -251,7 +325,7 @@ class InvitationsResource(Resource):
     async def list(self) -> list[Invitation]:
         return (await self._client._resource_request("GET", "members", Members)).invitations
 
-    async def create(self, *, email: str, role: Role = "member") -> None:
+    async def create(self, *, email: str, role: Role = "developer") -> None:
         return await self._client._resource_request("POST", "members/invitations", None,
             json={"email": email, "role": role})
 
@@ -264,10 +338,11 @@ class ApiKeysResource(PagedResource[ApiKey]):
         return await self._client._resource_request("GET", "api-keys", Page[ApiKey],
             params={"offset": page_offset(cursor), "limit": page_size(limit)})
 
-    async def create(self, *, name: str, id: UUID | None = None) -> CreatedApiKey:
+    async def create(self, *, name: str, id: UUID | None = None, profile: Literal["developer", "agent"] = "developer", expires_at: datetime | None = None) -> CreatedApiKey:
         """Secret is returned once; access via secret.get_secret_value()."""
         return await self._client._resource_request("POST", "api-keys", CreatedApiKey,
-            json={"id": str(id or uuid4()), "name": name})
+            json={"id": str(id or uuid4()), "name": name, **({"profile": profile} if profile != "developer" else {}),
+                  **({"expires_at": expires_at.isoformat()} if expires_at is not None else {})})
 
     async def revoke(self, id: str | UUID) -> None:
         return await self._client._resource_request("DELETE", f"api-keys/{path_id(id)}", None)