keble-scraper-api 0.2.0__tar.gz

keble_scraper_api-0.2.0/.gitignore

@@ -0,0 +1,16 @@
+.venv/
+__pycache__/
+*.py[cod]
+.pytest_cache/
+.mypy_cache/
+.pyright/
+dist/
+build/
+*.egg-info/
+.env
+.env.*
+!.env.example
+frontend/node_modules/
+frontend/dist/
+frontend/playwright-report/
+frontend/test-results/

keble_scraper_api-0.2.0/AiAgentChangesLogs.md

@@ -0,0 +1,173 @@
+# AI Agent Change Log
+
+## 2026-07-19 20:12 CST
+State: `fix/review2-scraper-recovery`; API `0.2.0` and contract `0.3.0` release
+candidates after the verified local clean break.
+
+zma: Continue WP8 through immutable dependency-first releases and final main
+integration.
+
+Ai: Advanced API metadata to consume the mandatory retention-revision contract
+and Data Infra `0.9.0`; the runtime includes typed failures, bounded history,
+namespace ownership fencing, and audited repair tooling. Tag/publication remain
+behind full portable, real-Mongo, Pyright, build, clean-install, and image gates.
+
+## 2026-07-19 19:35 CST
+State: `fix/review2-scraper-recovery`; WP8 guarded namespace-reset tooling is
+verified on the topic branch; local execution and coordinated 0.2.0 release remain open.
+
+zma:
+- Continue the finalized REVIEW2 plan through audited clean break, complete
+  releases, main integration, and runtime proof.
+
+Ai:
+- Promoted one canonical artifact-store selector for API/workers/maintenance and
+  added a DRY_RUN-first local/test clean-break service plus CLI. APPLY requires
+  exact writer, database, Redis, artifact owner, backup/inventory, campaign, and
+  audit evidence; it deletes streamed binary keys before fixed Mongo metadata
+  and then only the configured Redis namespace.
+- Added five focused tests for authorization, count-only inspection, stable key
+  streaming, dependency ordering, replay idempotency, binary-failure metadata
+  preservation, adjacent namespace survival, and audit promotion. Ruff, strict
+  Pyright, and changed-symbol docstring/prohibited-pattern audits pass.
+
+## 2026-07-17 21:59 CST
+State: `fix/review2-scraper-recovery`; API `0.1.2` consumes contract `0.2.0`
+while its coordinated `0.2.0` release work remains open.
+
+zma: Implement all finalized REVIEW2 scraper/provider recovery work without
+raw error strings, false payment inference, unbounded paid retry, or incomplete
+test/release claims.
+
+Ai: Moved the API-local error enum to the shared contract, added one total
+fetch/domain-to-failure mapper, and changed append-only attempts, terminal job
+snapshots, Mongo transitions, and callbacks to store typed evidence. Retry
+selection now consumes the failure disposition and dead-letter snapshots retain
+the upstream cause while marking bounded-attempt exhaustion. HTTP envelope,
+lease recovery, Sentry, retention, Shopify adaptation, and the API version/tag
+remain explicitly pending.
+
+## 2026-07-15 19:07 CST
+State: `fix/artifact-access-evidence`; scraper API 0.1.2 remains unchanged.
+
+zma: Continue the Shopify rollout audit by proving private object presence,
+retention, and exact reconciliation between object-store reads and Mongo audit
+events without enabling provider traffic.
+
+Ai: Added a server-owned audit UUID to every signed OSS/local artifact URL and
+matching immutable audit event, persisted SHA-256 user metadata on new OSS
+objects, and added offline coverage for metadata, signature, local URL, and
+end-to-end service-download correlation. This creates the source contract used
+by typed rollout object-storage evidence; no credential or provider call is
+introduced.
+
+## 2026-07-15 13:18 CST
+State: `feature/local-shared-artifacts`; scraper API 0.1.2 remains unchanged.
+
+zma: Support the isolated local ten-store Shopify run with maintainable shared
+artifacts, safe Mac network egress, and correct worker rate behavior.
+
+Ai: Added typed development-only local artifact and proxy/DoH settings/adapters,
+kept SSRF validation and TLS/IP pinning intact, and changed Redis route
+admission to reserve asynchronously awaited future timestamps. Unit coverage
+now proves configuration fail-closed behavior, content integrity, DoH parsing,
+direct-proxy trust, and burst pacing without route changes.
+
+## 2026-07-15 10:49 CST
+State: `fix/node24-image-workflows`; API `0.1.2` source is unchanged.
+
+zma: Continue the release audit and remove avoidable workflow warnings.
+
+Ai: Upgraded the API image publication actions to reviewed Node 24 commits and
+made portable workflow tests reject any action-pin drift.
+
+## 2026-07-15 10:11 CST
+State: `feature/immutable-image-publishing`; API `0.1.2` remains unchanged while
+its release image receives an immutable GHCR publication path.
+
+zma: Continue the full rollout audit and close the missing registry/digest gate.
+
+Ai: Added annotated-tag/version workflow guards and portable tests proving the
+API image has source-SHA identity, provenance, SBOM, and retained digest evidence.
+
+## 2026-07-15 02:44
+State: v0.1.0 source on `feature/shopify-scraping-stack`; release pending workspace verification.
+
+zma: Implement the generalized asynchronous scraper backend, workers, proxy/cost/artifact controls, authentication, and complete test layers required by the Shopify infrastructure plan.
+
+Ai: Added typed application ports, secure transport adapters, ACID Mongo lifecycle/outbox and retention workflows, Redis budgets, Celery queue/beat composition, private OSS, signed callbacks, Truth/Google sessions, Sentry, API routes, and deterministic unit/integration/live coverage.
+
+## 2026-07-15 03:39
+State: v0.1.0 source on `feature/shopify-scraping-stack`; Shopify service artifact consumption is implemented and release remains pending the full-stack audit.
+
+zma: Continue the complete plan with a usable Shopify-to-scraper artifact contract, additive retention, and full tests.
+
+Ai: Added audited service download/pin routes, pin-strengthening ACID behavior, complete public Storefront detail/variant/media query fields, typed validation errors, and unit/integration regressions.
+
+## 2026-07-15 05:50
+State: v0.1.0 source on `feature/shopify-scraping-stack`; safe artifact metadata lookup now completes the Shopify consumer contract.
+
+zma: Close the remaining scraper-to-Shopify artifact contract before continuing data-infra integration.
+
+Ai: Added a service-authenticated metadata endpoint and application read that returns `ScrapeArtifactRef` only, without granting object bytes; expanded API documentation and offline/integration regressions.
+
+## 2026-07-15 07:42 CST
+State: API `0.1.1` patch on `fix/conservative-bright-cost`; contract remains
+`0.1.0` and the immutable initial tags are retained.
+
+zma: Complete the cost-effective Shopify infrastructure without allowing
+temporary vendor promotions to understate proxy spend.
+
+Ai: Changed the default Bright reservation from the temporary USD 4/GB coupon
+rate to the official undiscounted USD 8/GB pay-as-you-go baseline; synchronized
+package/OpenAPI/import/Sentry versions, kept Cloudflare at its documented USD
+0.09/browser-hour rate, rejected zero unit prices, and added price/version tests.
+
+## 2026-07-15 07:58 CST
+State: API `0.1.2` on `fix/docker-build-timeouts`; contract remains `0.1.0`.
+
+zma: Finish the infrastructure with exact image proof and no concealed build
+failure.
+
+Ai: Added reusable BuildKit `uv` download caches and a 120-second bounded
+timeout to both frozen install stages after two different 30-second PyPI
+timeouts; kept every dependency locked and synchronized release identity.
+
+## 2026-07-16 15:03 CST
+State: `fix/blank-optional-artifact-env`; API version remains unchanged.
+
+zma: Make the local scalable Shopify preload runtime use mandatory private OSS
+without retaining obsolete local-artifact settings from an env file.
+
+Ai: Added a narrow typed validator that treats blank optional local-artifact
+directory/base-URL values as absent and a unit regression proving a complete
+private OSS backend remains the sole configured artifact store.
+
+Ai follow-up: Bound each Mongo database to one persisted object-key namespace
+at startup and reject pre-existing or later prefix drift before job execution,
+closing the content-ID/object-key collision found by the staged Shopify run.
+
+## 2026-07-19 02:36 CST
+State: `fix/review2-scraper-recovery`; API version remains unchanged while
+R2-B-M1 awaits the coordinated REVIEW2 release train.
+
+zma: Continue the finalized REVIEW2 plan and replace the unsafe prefix-only
+artifact fence with a source-proven, typed, ACID-safe ownership contract.
+
+Ai: Added complete backend/location/prefix identity persistence and a stable
+owner canary in OSS/local storage, rejected legacy rows and all identity drift
+before readiness, and covered cross-database ownership plus backend/bucket
+swaps with offline and isolated real-Mongo regressions.
+
+## 2026-07-19 05:56 CST
+State: `fix/review2-scraper-recovery`; API version remains unchanged while
+R2-B-M11 awaits coordinated Shopify delivery and WP8 release.
+
+zma: Continue the finalized REVIEW2 plan with ACID artifact-retention recovery
+and comprehensive edge coverage.
+
+Ai: Replaced expiry-strengthening-only pins with monotonic revision-fenced
+policy synchronization. The transaction increments references once, applies
+newer finite or indefinite expiry, ignores stale delivery, accepts exact replay,
+and rejects same-revision policy drift; service audit now records the accepted
+retention revision.

keble_scraper_api-0.2.0/PKG-INFO

@@ -0,0 +1,160 @@
+Metadata-Version: 2.4
+Name: keble-scraper-api
+Version: 0.2.0
+Summary: Asynchronous policy-bounded scraping API, workers, proxy router, and artifact service.
+Author-email: zma <bob0103779@gmail.com>
+Requires-Python: <3.14,>=3.13
+Requires-Dist: aiohttp<4,>=3.11
+Requires-Dist: celery<6,>=5.6
+Requires-Dist: cryptography<47,>=45
+Requires-Dist: fastapi<1,>=0.116
+Requires-Dist: google-auth<3,>=2.40
+Requires-Dist: httpx<1,>=0.28
+Requires-Dist: keble-data-infra-contract<1,>=0.9.0
+Requires-Dist: keble-helpers<2,>=1.54.0
+Requires-Dist: keble-scraper-contract==0.3.0
+Requires-Dist: nh3<1,>=0.3
+Requires-Dist: oss2<3,>=2.19
+Requires-Dist: pydantic-settings<3,>=2.10
+Requires-Dist: pymongo<5,>=4.12
+Requires-Dist: redis<7,>=5
+Requires-Dist: sentry-sdk[celery,fastapi]<3,>=2.32
+Requires-Dist: uvicorn[standard]<1,>=0.35
+Description-Content-Type: text/markdown
+
+# Keble Scraper API
+
+The API package owns FastAPI composition, Mongo lifecycle/index persistence,
+Redis-backed egress budgets and sessions, Celery workers/beat, secure HTTP and
+Cloudflare fetching, private OSS artifacts, HMAC callbacks, authentication,
+auditing, and Sentry instrumentation. It imports `keble-scraper-contract`; the
+contract never imports this package.
+
+Version `0.2.0` retains the undiscounted Bright pay-as-you-go baseline for
+pre-dispatch cost reservation so temporary coupon pricing cannot understate
+production spend and releases typed provider recovery, crash-safe callback and
+job budgets, bounded history, artifact-namespace ownership, ordered retention,
+and the guarded first-release reset.
+
+The workspace consumes `keble-scraper-contract 0.3.0` as the foundation for
+the API `0.2.0` recovery release. Attempt/job/callback
+persistence stores discriminated typed failure snapshots instead of terminal
+strings, and retry classification uses their finite dispositions. Public HTTP
+errors use the same envelope; mandatory retention revisions prevent delayed pin
+delivery from reversing a newer paid-storage policy.
+
+Each job and callback row owns its immutable attempt budget;
+jobs also own a fixed deadline. Lease acquisition and recovery enforce those
+stored values, temporary capacity retains its future horizon, and unexpected
+runtime faults remain leased for expiry recovery rather than becoming a false
+network failure. Cache pointers cannot outlive referenced artifacts, missing
+cache metadata invalidates the pointer before refetch, and expired callback
+leases are recovered or dead-lettered by maintenance.
+
+The WP8 clean-break command lives at `scripts/reset_pre_release_state.py` and
+delegates to the package-owned typed maintenance module. DRY_RUN is the default.
+Local/test-only APPLY deletes referenced binaries before their Mongo metadata,
+then the fixed Scraper collection allowlist and exact Redis namespace. It
+requires explicit writer-stop, database, Redis, artifact-owner, snapshot,
+artifact-inventory, campaign, and immutable-audit evidence; failures propagate
+with metadata or RUNNING evidence retained for replay. See `../docs/RUNBOOK.md`.
+
+HTTP, Cloudflare, and registered callback transports own reusable connection
+pools closed by `ScraperContainer`. The shared Data Infra Retry-After parser is
+used at every response boundary, with durable due times rounded upward to Mongo
+millisecond precision. Maintenance projects due job ID plus transport policy in
+one indexed read and publishes with configured bounded concurrency.
+
+Maintenance also owns bounded historical retirement. Terminal Jobs without a
+callback receive an absolute retention clock immediately; callback Jobs receive
+one only in the same transaction that marks their outbox event delivered.
+Pending, leased, and dead-letter outbox rows therefore cannot be purged. One
+Mongo singleton lease serializes each minute's bounded purge, and the majority
+transaction deletes each expired Job with all immutable Attempt children before
+reporting success. Delivered callbacks and audit facts use independent absolute
+clocks. Defaults are 90 days for Job/Attempt evidence, 30 days for delivered
+callback replay, 400 days for security audit, and 1,000 rows per collection per
+minute; every value is a typed `SCRAPER_*` environment setting.
+
+Sentry is initialized explicitly for the API, Celery parent, every worker child,
+and beat. Runtime tags distinguish those processes while credentials, URLs,
+request bodies, and provider payloads stay out of telemetry configuration.
+
+Side effects if changes:
+- Shopify must repin and exhaustively map local versus upstream failures;
+- frontend error presentation and OpenAPI generation ship with the API release;
+- pre-production `0.1.x` scraper job rows require the guarded reset/repair step.
+
+Runtime commands:
+
+```bash
+uv run --package keble-scraper-api keble-scraper-api
+uv run --package keble-scraper-api keble-scraper-worker fetch
+uv run --package keble-scraper-api keble-scraper-worker render
+uv run --package keble-scraper-api keble-scraper-worker proxy-health
+uv run --package keble-scraper-api keble-scraper-worker maintenance
+uv run --package keble-scraper-api keble-scraper-beat
+```
+
+Service-authenticated consumers may download a verified artifact through the
+audited signed-URL endpoint and synchronize a downstream retention pin with a
+required positive `retentionRevision`. The Mongo transaction increments the
+artifact reference only on first owner-pin creation, applies only newer policy
+revisions, ignores stale deliveries, and rejects a divergent replay at the same
+revision. Both finite archive expiry and indefinite latest retention are valid
+ordered policies.
+The operator and service paths share the same public download route but retain
+separate authorization and audit identities.
+
+The download service reserves a server UUID, places it in the immutable audit
+event and the signed URL's `x-keble-audit-id` query, and returns the URL only
+after the audit append succeeds. OSS `GetObject` logs can therefore reconcile
+reads exactly. New OSS objects also carry their canonical SHA-256 as private
+user metadata for bounded integrity checks.
+
+For a multi-process local canary without OSS, configure both
+`SCRAPER_LOCAL_ARTIFACT_DIRECTORY` and `SCRAPER_LOCAL_ARTIFACT_BASE_URL`, then
+serve only that isolated directory on localhost. The development adapter keeps
+content-addressed bytes shared between API and Celery workers; staging and
+production reject it and continue to require private OSS.
+
+Empty or whitespace-only values for those two optional local-artifact settings
+are treated as absent. This supports explicit Compose overrides when private OSS
+replaces a legacy local-artifact `env_file` configuration; all backend-completeness
+and environment restrictions still apply.
+
+The optional OSS endpoint, bucket, access-key ID, and access-key secret are also
+normalized from blank to absent as one typed tuple. Partial or empty production
+configuration fails startup before any external request or artifact write.
+
+Startup now applies two independent artifact fences before readiness. Mongo
+persists and compares the complete non-secret backend identity: backend kind,
+deterministic prefix, OSS endpoint/bucket or resolved local directory, and the
+configured owner. The binary store also contains one deterministic owner canary
+inside that prefix. Existing rows without a complete marker, backend/location
+changes, and a second owner all fail closed before workers accept jobs.
+
+Set `SCRAPER_ARTIFACT_NAMESPACE_OWNER` in staging and production to a stable,
+non-secret value unique to the Mongo deployment/database that owns the prefix,
+for example `cn-hangzhou/mongo-primary/keble_scraper`. Do not reuse it for a
+second database. Development/test may derive `environment/database` locally.
+The pre-production prefix-only marker is intentionally incompatible and must be
+cleared or explicitly migrated before this release line starts.
+
+When a local VPN returns fake-IP DNS answers, configure both
+`SCRAPER_DEVELOPMENT_HTTP_PROXY_URL` and
+`SCRAPER_DEVELOPMENT_DNS_OVER_HTTPS_URL`. The worker validates real DoH A/AAAA
+answers through the explicit proxy, preserves IP pinning, and applies the proxy
+only to the verified `DIRECT` route. The pair is rejected outside
+development/test.
+
+Every route admission is atomically assigned a future Redis timestamp derived
+from its requests-per-second policy. A fetch worker asynchronously waits for
+that timestamp, preserving the requested route and job attempt; internal burst
+pressure therefore does not become terminal `ROUTE_UNAVAILABLE` evidence or a
+paid fallback.
+
+The repository release workflow builds this package's runtime image only from
+an annotated `keble-scraper-api-vX.Y.Z` tag whose version matches this package.
+GHCR version/source tags are discovery aids; production consumes the digest
+evidence artifact and pins `ghcr.io/keble-ai/keble-scraper-api@sha256:...`.

keble_scraper_api-0.2.0/README.md

@@ -0,0 +1,136 @@
+# Keble Scraper API
+
+The API package owns FastAPI composition, Mongo lifecycle/index persistence,
+Redis-backed egress budgets and sessions, Celery workers/beat, secure HTTP and
+Cloudflare fetching, private OSS artifacts, HMAC callbacks, authentication,
+auditing, and Sentry instrumentation. It imports `keble-scraper-contract`; the
+contract never imports this package.
+
+Version `0.2.0` retains the undiscounted Bright pay-as-you-go baseline for
+pre-dispatch cost reservation so temporary coupon pricing cannot understate
+production spend and releases typed provider recovery, crash-safe callback and
+job budgets, bounded history, artifact-namespace ownership, ordered retention,
+and the guarded first-release reset.
+
+The workspace consumes `keble-scraper-contract 0.3.0` as the foundation for
+the API `0.2.0` recovery release. Attempt/job/callback
+persistence stores discriminated typed failure snapshots instead of terminal
+strings, and retry classification uses their finite dispositions. Public HTTP
+errors use the same envelope; mandatory retention revisions prevent delayed pin
+delivery from reversing a newer paid-storage policy.
+
+Each job and callback row owns its immutable attempt budget;
+jobs also own a fixed deadline. Lease acquisition and recovery enforce those
+stored values, temporary capacity retains its future horizon, and unexpected
+runtime faults remain leased for expiry recovery rather than becoming a false
+network failure. Cache pointers cannot outlive referenced artifacts, missing
+cache metadata invalidates the pointer before refetch, and expired callback
+leases are recovered or dead-lettered by maintenance.
+
+The WP8 clean-break command lives at `scripts/reset_pre_release_state.py` and
+delegates to the package-owned typed maintenance module. DRY_RUN is the default.
+Local/test-only APPLY deletes referenced binaries before their Mongo metadata,
+then the fixed Scraper collection allowlist and exact Redis namespace. It
+requires explicit writer-stop, database, Redis, artifact-owner, snapshot,
+artifact-inventory, campaign, and immutable-audit evidence; failures propagate
+with metadata or RUNNING evidence retained for replay. See `../docs/RUNBOOK.md`.
+
+HTTP, Cloudflare, and registered callback transports own reusable connection
+pools closed by `ScraperContainer`. The shared Data Infra Retry-After parser is
+used at every response boundary, with durable due times rounded upward to Mongo
+millisecond precision. Maintenance projects due job ID plus transport policy in
+one indexed read and publishes with configured bounded concurrency.
+
+Maintenance also owns bounded historical retirement. Terminal Jobs without a
+callback receive an absolute retention clock immediately; callback Jobs receive
+one only in the same transaction that marks their outbox event delivered.
+Pending, leased, and dead-letter outbox rows therefore cannot be purged. One
+Mongo singleton lease serializes each minute's bounded purge, and the majority
+transaction deletes each expired Job with all immutable Attempt children before
+reporting success. Delivered callbacks and audit facts use independent absolute
+clocks. Defaults are 90 days for Job/Attempt evidence, 30 days for delivered
+callback replay, 400 days for security audit, and 1,000 rows per collection per
+minute; every value is a typed `SCRAPER_*` environment setting.
+
+Sentry is initialized explicitly for the API, Celery parent, every worker child,
+and beat. Runtime tags distinguish those processes while credentials, URLs,
+request bodies, and provider payloads stay out of telemetry configuration.
+
+Side effects if changes:
+- Shopify must repin and exhaustively map local versus upstream failures;
+- frontend error presentation and OpenAPI generation ship with the API release;
+- pre-production `0.1.x` scraper job rows require the guarded reset/repair step.
+
+Runtime commands:
+
+```bash
+uv run --package keble-scraper-api keble-scraper-api
+uv run --package keble-scraper-api keble-scraper-worker fetch
+uv run --package keble-scraper-api keble-scraper-worker render
+uv run --package keble-scraper-api keble-scraper-worker proxy-health
+uv run --package keble-scraper-api keble-scraper-worker maintenance
+uv run --package keble-scraper-api keble-scraper-beat
+```
+
+Service-authenticated consumers may download a verified artifact through the
+audited signed-URL endpoint and synchronize a downstream retention pin with a
+required positive `retentionRevision`. The Mongo transaction increments the
+artifact reference only on first owner-pin creation, applies only newer policy
+revisions, ignores stale deliveries, and rejects a divergent replay at the same
+revision. Both finite archive expiry and indefinite latest retention are valid
+ordered policies.
+The operator and service paths share the same public download route but retain
+separate authorization and audit identities.
+
+The download service reserves a server UUID, places it in the immutable audit
+event and the signed URL's `x-keble-audit-id` query, and returns the URL only
+after the audit append succeeds. OSS `GetObject` logs can therefore reconcile
+reads exactly. New OSS objects also carry their canonical SHA-256 as private
+user metadata for bounded integrity checks.
+
+For a multi-process local canary without OSS, configure both
+`SCRAPER_LOCAL_ARTIFACT_DIRECTORY` and `SCRAPER_LOCAL_ARTIFACT_BASE_URL`, then
+serve only that isolated directory on localhost. The development adapter keeps
+content-addressed bytes shared between API and Celery workers; staging and
+production reject it and continue to require private OSS.
+
+Empty or whitespace-only values for those two optional local-artifact settings
+are treated as absent. This supports explicit Compose overrides when private OSS
+replaces a legacy local-artifact `env_file` configuration; all backend-completeness
+and environment restrictions still apply.
+
+The optional OSS endpoint, bucket, access-key ID, and access-key secret are also
+normalized from blank to absent as one typed tuple. Partial or empty production
+configuration fails startup before any external request or artifact write.
+
+Startup now applies two independent artifact fences before readiness. Mongo
+persists and compares the complete non-secret backend identity: backend kind,
+deterministic prefix, OSS endpoint/bucket or resolved local directory, and the
+configured owner. The binary store also contains one deterministic owner canary
+inside that prefix. Existing rows without a complete marker, backend/location
+changes, and a second owner all fail closed before workers accept jobs.
+
+Set `SCRAPER_ARTIFACT_NAMESPACE_OWNER` in staging and production to a stable,
+non-secret value unique to the Mongo deployment/database that owns the prefix,
+for example `cn-hangzhou/mongo-primary/keble_scraper`. Do not reuse it for a
+second database. Development/test may derive `environment/database` locally.
+The pre-production prefix-only marker is intentionally incompatible and must be
+cleared or explicitly migrated before this release line starts.
+
+When a local VPN returns fake-IP DNS answers, configure both
+`SCRAPER_DEVELOPMENT_HTTP_PROXY_URL` and
+`SCRAPER_DEVELOPMENT_DNS_OVER_HTTPS_URL`. The worker validates real DoH A/AAAA
+answers through the explicit proxy, preserves IP pinning, and applies the proxy
+only to the verified `DIRECT` route. The pair is rejected outside
+development/test.
+
+Every route admission is atomically assigned a future Redis timestamp derived
+from its requests-per-second policy. A fetch worker asynchronously waits for
+that timestamp, preserving the requested route and job attempt; internal burst
+pressure therefore does not become terminal `ROUTE_UNAVAILABLE` evidence or a
+paid fallback.
+
+The repository release workflow builds this package's runtime image only from
+an annotated `keble-scraper-api-vX.Y.Z` tag whose version matches this package.
+GHCR version/source tags are discovery aids; production consumes the digest
+evidence artifact and pins `ghcr.io/keble-ai/keble-scraper-api@sha256:...`.

keble_scraper_api-0.2.0/pyproject.toml

@@ -0,0 +1,48 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "keble-scraper-api"
+version = "0.2.0"
+description = "Asynchronous policy-bounded scraping API, workers, proxy router, and artifact service."
+readme = "README.md"
+requires-python = ">=3.13,<3.14"
+dependencies = [
+    "aiohttp>=3.11,<4",
+    "celery>=5.6,<6",
+    "cryptography>=45,<47",
+    "fastapi>=0.116,<1",
+    "google-auth>=2.40,<3",
+    "httpx>=0.28,<1",
+    "keble-data-infra-contract>=0.9.0,<1",
+    "keble-helpers>=1.54.0,<2",
+    "keble-scraper-contract==0.3.0",
+    "nh3>=0.3,<1",
+    "oss2>=2.19,<3",
+    "pydantic-settings>=2.10,<3",
+    "pymongo>=4.12,<5",
+    "redis>=5,<7",
+    "sentry-sdk[fastapi,celery]>=2.32,<3",
+    "uvicorn[standard]>=0.35,<1",
+]
+
+[[project.authors]]
+name = "zma"
+email = "bob0103779@gmail.com"
+
+[project.scripts]
+keble-scraper-api = "keble_scraper_api.main:run"
+keble-scraper-worker = "keble_scraper_api.workers.cli:run_worker"
+keble-scraper-beat = "keble_scraper_api.workers.cli:run_beat"
+
+[dependency-groups]
+test = [
+    "fakeredis[lua]>=2.30,<3",
+    "pytest>=8.2,<9",
+    "pytest-asyncio>=0.23,<1",
+    "respx>=0.22,<1",
+]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/keble_scraper_api"]

keble_scraper_api-0.2.0/scripts/__init__.py

@@ -0,0 +1 @@
+"""Operator-only Scraper maintenance command modules."""

keble_scraper_api-0.2.0/scripts/reset_pre_release_state.py

@@ -0,0 +1,9 @@
+"""Run the canonical guarded Scraper first-release reset command."""
+
+from asyncio import run
+
+from keble_scraper_api.maintenance.reset_pre_release_state import amain
+
+
+if __name__ == "__main__":
+    run(amain())

keble_scraper_api-0.2.0/src/keble_scraper_api/__init__.py

@@ -0,0 +1,3 @@
+"""Keble scraper API, workers, persistence, and operations composition."""
+
+__version__ = "0.2.0"

keble_scraper_api-0.2.0/src/keble_scraper_api/api/dependencies.py

@@ -0,0 +1,100 @@
+"""Runtime container binding and shared request authorization dependencies."""
+
+from __future__ import annotations
+
+from uuid import uuid4
+
+from fastapi import Request
+
+from keble_scraper_contract import OperatorSession
+
+from keble_scraper_api.container import ScraperContainer
+
+
+class ContainerProvider:
+    """Bind one lifespan-created service graph to static route definitions.
+
+    Side effects if changes:
+    - every FastAPI dependency resolves process-owned services here.
+    """
+
+    def __init__(self) -> None:
+        """Initialize an intentionally unbound provider for OpenAPI creation.
+
+        Side effects if changes:
+        - pre-start request behavior remains fail-closed.
+        """
+
+        self._container: ScraperContainer | None = None
+
+    def bind(self, *, container: ScraperContainer) -> None:
+        """Install exactly one initialized process container.
+
+        Side effects if changes:
+        - route execution becomes available after lifespan startup.
+        """
+
+        if self._container is not None:
+            raise RuntimeError("scraper container is already bound")
+        self._container = container
+
+    def clear(self) -> None:
+        """Remove the binding before process resources close.
+
+        Side effects if changes:
+        - shutdown requests fail closed.
+        """
+
+        self._container = None
+
+    def require(self) -> ScraperContainer:
+        """Return initialized services or reject pre-start/post-shutdown access.
+
+        Side effects if changes:
+        - every API endpoint dependency.
+        """
+
+        if self._container is None:
+            raise RuntimeError("scraper runtime is not initialized")
+        return self._container
+
+
+def request_id(*, request: Request) -> str:
+    """Use caller correlation metadata or generate a server request identity.
+
+    Side effects if changes:
+    - audit correlation; never job/idempotency identity.
+    """
+
+    return request.headers.get("x-request-id") or uuid4().hex
+
+
+def authorize_service(*, request: Request, provider: ContainerProvider) -> str:
+    """Authenticate the single Shopify service audience with current/next keys.
+
+    Side effects if changes:
+    - job submit/read/cancel service access.
+    """
+
+    container = provider.require()
+    container.service_keys.authorize(
+        supplied=request.headers.get("x-service-key")
+    )
+    return container.settings.service_caller_identity
+
+
+async def authorize_operator(
+    *,
+    request: Request,
+    provider: ContainerProvider,
+) -> OperatorSession:
+    """Resolve a backend-managed encrypted HttpOnly operator session.
+
+    Side effects if changes:
+    - every admin read/mutation authorization decision.
+    """
+
+    container = provider.require()
+    return await container.auth.session_from_cookie(
+        token=request.cookies.get(container.settings.session_cookie_name)
+    )