point-topic-access 0.2.0__tar.gz → 0.2.1__tar.gz

{point_topic_access-0.2.0 → point_topic_access-0.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: point-topic-access
-Version: 0.2.0
+Version: 0.2.1
 Summary: Shared permission -> ClickHouse access-control logic for Point Topic apps (claims, JWT verification, geo filter composer, per-org provisioning)
 License: MIT
 Requires-Python: >=3.12
@@ -137,8 +137,28 @@ repos, so the production box's `uv sync` couldn't clone it. Public PyPI matches
 | `point-topic-mcp` | ✅ **live** (deployed 2026-08-07, E2E-verified) | dependency `point-topic-access>=0.1.0`; local modules deleted |
 | `onto-app-new` | next | issue **#30** (supersedes #25); needs v0.2.0 — `provision_onto_app_role` / `provision_onto_public_role` (§3.4 contract); `docs/ONTOLOGY_APP_HOOKUP.md` is the reference |
 | `upc_query_agent` | follow-up | already reads `pt_org_id`/`products[]` in `api/auth_handler.py` — adopt `claims.py` |
+| `pt-dashboards` | ✅ **live** (dashboards hub) | jwt-only consumer — `Auth0TokenVerifier(auth0_domain=…, audience=[<client_id>, "http://ss-api"])` with explicit args; imports only `pt_access.jwt`, so it uses none of the package's env keys (the `CLICKHOUSE_*` / `SUB_SITE_MONGODB_URI` set) → **no secret injection needed**; API env is public config only, no `.env` on the box since 2026-09-05, deploys via GH Actions |
 | `local-pricing-dashboard`, `european-fttp-forecasts` | don't break | read `products[].permissions.role` — the claim shape is pinned by tests |
 
+## Secrets injection for consumers — no `.env`
+
+`pt_access` reads its credentials from env vars only; it never touches AWS itself. Consumers
+must **not** persist those values in a `.env` file — inject them at process startup with
+`tools/exec-env` (installed once per box, e.g. `/opt/bin/exec-env`):
+
+```bash
+exec-env pt-access-credentials -- <app launch command>
+```
+
+This fetches the `pt-access-credentials` JSON secret from AWS Secrets Manager and exports its
+keys (`CLICKHOUSE_*`, `CLICKHOUSE_PROVISIONING_*`, `SUB_SITE_MONGODB_URI`, …) into the process
+env — never written to disk. Full design, the box's Roles Anywhere identity, and the staged
+per-consumer rollout: [`docs/secrets-injection-plan.md`](docs/secrets-injection-plan.md).
+Behaviour tests: `bash tests/test_exec_env.sh`.
+
+Exception: **jwt-only consumers** (e.g. `pt-dashboards` — imports `pt_access.jwt` only) need
+no injection — they use none of the package's env keys (deep dive: `Point-Topic/pt-dashboards#2`).
+
 ## Development
 
 ```bash

{point_topic_access-0.2.0 → point_topic_access-0.2.1}/README.md

@@ -119,8 +119,28 @@ repos, so the production box's `uv sync` couldn't clone it. Public PyPI matches
 | `point-topic-mcp` | ✅ **live** (deployed 2026-08-07, E2E-verified) | dependency `point-topic-access>=0.1.0`; local modules deleted |
 | `onto-app-new` | next | issue **#30** (supersedes #25); needs v0.2.0 — `provision_onto_app_role` / `provision_onto_public_role` (§3.4 contract); `docs/ONTOLOGY_APP_HOOKUP.md` is the reference |
 | `upc_query_agent` | follow-up | already reads `pt_org_id`/`products[]` in `api/auth_handler.py` — adopt `claims.py` |
+| `pt-dashboards` | ✅ **live** (dashboards hub) | jwt-only consumer — `Auth0TokenVerifier(auth0_domain=…, audience=[<client_id>, "http://ss-api"])` with explicit args; imports only `pt_access.jwt`, so it uses none of the package's env keys (the `CLICKHOUSE_*` / `SUB_SITE_MONGODB_URI` set) → **no secret injection needed**; API env is public config only, no `.env` on the box since 2026-09-05, deploys via GH Actions |
 | `local-pricing-dashboard`, `european-fttp-forecasts` | don't break | read `products[].permissions.role` — the claim shape is pinned by tests |
 
+## Secrets injection for consumers — no `.env`
+
+`pt_access` reads its credentials from env vars only; it never touches AWS itself. Consumers
+must **not** persist those values in a `.env` file — inject them at process startup with
+`tools/exec-env` (installed once per box, e.g. `/opt/bin/exec-env`):
+
+```bash
+exec-env pt-access-credentials -- <app launch command>
+```
+
+This fetches the `pt-access-credentials` JSON secret from AWS Secrets Manager and exports its
+keys (`CLICKHOUSE_*`, `CLICKHOUSE_PROVISIONING_*`, `SUB_SITE_MONGODB_URI`, …) into the process
+env — never written to disk. Full design, the box's Roles Anywhere identity, and the staged
+per-consumer rollout: [`docs/secrets-injection-plan.md`](docs/secrets-injection-plan.md).
+Behaviour tests: `bash tests/test_exec_env.sh`.
+
+Exception: **jwt-only consumers** (e.g. `pt-dashboards` — imports `pt_access.jwt` only) need
+no injection — they use none of the package's env keys (deep dive: `Point-Topic/pt-dashboards#2`).
+
 ## Development
 
 ```bash

point_topic_access-0.2.1/docs/secrets-injection-plan.md

@@ -0,0 +1,190 @@
+# Plan — `pt_access` env vars from Secrets Manager (point-topic-access#2)
+
+**Status:** plan agreed; this repo's part **implemented** (`tools/exec-env`, tested). Consumer
+rollout happens per-repo **after** this is verified working, as its own ticket per app.
+
+## Goal
+
+`pt_access` keeps reading its credentials from env vars. We stop the consumer-side
+**persistent plaintext `.env`** (written today by each app's `deploy.sh`) and instead inject
+the values at process startup, in memory only, via **one shared tool**. Adding a future app
+then becomes a one-line change, not a new bespoke mechanism.
+
+The package itself needs **no code change** — injection happens *outside* the app, before it
+starts. That removes any need for boto3 in the package, per-app secrets code, or per-app AWS
+roles.
+
+## Design in one picture
+
+```
+                            AWS Secrets Manager
+                                pt-access-credentials
+                                       ▲
+                                       │ GetSecretValue (box role)
+                 ┌─────────────────────┴───────────────┐
+                 │        one box identity              │
+                 │   Roles Anywhere role/profile/cert   │
+                 │   (IP-restricted; reads this box's   │
+                 │    secrets only)                     │
+                 └─────────────────────┬───────────────┘
+                                       │ ambient creds
+                 ┌─────────────────────┴───────────────┐
+                 │        one injector: exec-env       │
+                 │   fetch → export → exec (never disk)│
+                 └───────────────┬─────────┬──────────┘
+                                 │         │
+                    exec-env <secret> -- app   exec-env <secret> -- docker compose up
+                                 │         │
+                                 ▼         ▼
+                           each app reads os.environ → pt_access → ClickHouse / sub-site Mongo
+```
+
+**Trust model:** the **box is the trust boundary** for *internal* apps — one identity and one
+injector per box. Be honest about the trade: per-app roles protect against **individual app
+compromise** (a web-exploitable app can reach only *its* secrets), while a shared box role lets
+an exploited app reach **all the box's secrets**. The shared role widens the blast radius
+**per-app-compromise**, not per-box-compromise. For internal, trusted apps that is a deliberate,
+reasonable trade that costs far less to maintain.
+
+**Co-existence rule — do NOT rip out what's built:** keep the **per-app roles for MCP and
+pt-mail** (isolation-correct, and MCP is public-facing with a real data boundary) and for any
+client-facing app. Use the **box-shared model for new internal apps** (onto-app, pt-dashboards,
+upc_query_agent, etc.). Both models coexist on the same box.
+
+## The one new secret
+
+`pt-access-credentials` in Secrets Manager, holding exactly the keys this package reads:
+
+```
+CLICKHOUSE_HOST, CLICKHOUSE_PORT, CLICKHOUSE_DATABASE,
+CLICKHOUSE_PROVISIONING_USER, CLICKHOUSE_PROVISIONING_PASSWORD,
+SUB_SITE_MONGODB_URI, POINT_TOPIC_ORG_ID, MCP_CLICKHOUSE_GRANT_TO_USER
+```
+
+- Least privilege: a consumer reads **this** secret, not the whole `ptserver-secret` mega-bundle.
+- **Decision (weighed):** a dedicated bundle chosen for the cleanest, least-privileged IAM. The
+  acknowledged cost is that the shared CH/Mongo values are duplicated from wherever they are
+  canonical, so **rotation must update all copies** — add it to the secrets catalogue. The
+  alternative (point consumers at the canonical `ptserver-secret` keys, avoiding the mirror)
+  was considered and rejected because it regrants them read access to the whole box bundle.
+- The `POINT_TOPIC_ORG_ID` / `MCP_CLICKHOUSE_GRANT_TO_USER` defaults (`provisioning.py:117,120`)
+  stop being baked via env absence — they're explicit in the secret so an app can't silently
+  diverge.
+
+## The one injector: `exec-env`
+
+A small, language-agnostic tool installed once per box (`/opt/bin/exec-env`). sops `exec-env`
+style: fetch the secret, export its keys into the child process's env, run it — **nothing written
+to disk**. **Implemented at `tools/exec-env`** (bash 3.2+; behaviour-tested via
+tests/test_exec_env.sh with the `aws` CLI stubbed — no AWS needed to run the tests):
+
+```bash
+# install once per box
+install -m 0755 tools/exec-env /opt/bin/exec-env
+
+# usage — prefix any app's launch command
+/opt/bin/exec-env <secret-id> [--region <region>] -- <command [args...]>
+/opt/bin/exec-env pt-access-credentials -- uvicorn backend.main:app
+```
+
+- Uses the **ambient credential chain** (`aws` CLI → Roles Anywhere profile) — no AWS code in any app.
+- Depends on the `aws` CLI + `python3` (JSON parsing; python3 is present on every consumer box).
+- Exports **all** keys in the secret (the secret *is* the allowlist, so this stays simple and safe).
+- **Fail closed & never echo:** on any fetch/parse failure it exits non-zero and **never prints a
+  secret value** — not in an error message, not in the raw response; `aws`/`python3` stderr is
+  suppressed and replaced with a generic message (secret id only). Keys are validated
+  (`[A-Za-z0-9_]+`); values must be single-line strings. Memory-only + don't-echo is what makes
+  this safe; keep it that way.
+
+## The one box identity (Roles Anywhere)
+
+One shared identity for **internal apps**; coexists with (and does not replace) the existing
+per-app MCP/pt-mail roles. Per box, not per app:
+
+1. **Reuse the shared trust anchor**
+   `arn:aws:rolesanywhere:eu-west-1:543072339765:trust-anchor/3efa399c-648d-482e-a4ea-29d20eadc604`.
+   Never create a new CA.
+2. **One role** (e.g. `<box>-roles-anywhere`): trust policy restricted by `aws:SourceArn` = the
+   trust anchor **and** `aws:PrincipalTag/x509Subject/CN` = the box's cert CN. Identity policy:
+   `secretsmanager:GetSecretValue` on **the secrets that box's apps need** (e.g.
+   `pt-access-credentials`), IP-restricted to the box's egress IP. Nothing else.
+3. **One profile** — remember it is **created disabled; call `enable_profile`**.
+4. **One cert** (CN = the box) — signed by the shared CA (CA key at
+   `~/.aws/roles-anywhere-ca/ca-key.pem` / Secret `pt-roles-anywhere-ca`). Key on box `chmod 600`;
+   chown the cert dir to the user that runs the apps.
+5. **Wire** `aws_signing_helper` + `credential_process` + `region = eu-west-1` in `~/.aws/config`.
+   Set `AWS_PROFILE=<box>` (or default) for the `aws` CLI used by `exec-env`.
+
+## Wire each app — one line
+
+Every app's launch command gets prefixed with `exec-env <secret> --`:
+
+| App | Before | After |
+|---|---|---|
+| onto-app (compose) | `deploy.sh`: write `.env` → `docker compose up -d` | `exec-env pt-access-credentials -- docker compose up -d` (compose interpolates `${CLICKHOUSE_PASSWORD}` from the exported env — `docker-compose.yml:11,41` keep working, zero compose change) |
+| MCP backend (compose `command`) | `command: python -m point_topic_mcp.server_http` | `exec-env pt-access-credentials -- python -m point_topic_mcp.server_http` |
+| pt-dashboards / other | its existing start command | `exec-env <its-secret> -- <start command>` |
+
+Notes:
+- onto-app's ClickHouse container needs `CLICKHOUSE_PASSWORD` at **compose-parse time**; wrapping
+  the `docker compose up` call with `exec-env` puts the value in compose's shell env, which is
+  exactly when it's interpolated. (If `exec-env` isn't desired for compose, do the equivalent in
+  `deploy.sh`: `export` from AWS, then `docker compose up`.)
+- MCP: also drop the `env_file: ${MCP_ENV_FILE:-../.env}` entries (`docker-compose.yml:37,75`) —
+  compose fails if a referenced file is gone. Use the `exec-env`-exported env instead.
+- Ordering safety: this injects before the app starts, so `pt_access`'s module-level env reads
+  (`POINT_TOPIC_ORG_ID`/`GRANT_TO_USER`, `provisioning.py:117,120`) always see the injected values.
+
+## Remove the `.env`
+
+For each consumer: delete the `.env` write from `deploy.sh` / the deploy workflow, delete the
+file on the box, and any compose `env_file:` references to it. No static key on the box.
+
+## Security posture (still secure)
+
+- **Never on disk:** values live in the child process env only; nothing persisted, nothing logged.
+- **Box-least-privilege:** the box role reads only the secrets *its* apps need, IP-restricted.
+- **No fallback:** `exec-env` is the deliberate single source — the app never "reads env, else
+  fetch." (Avoids the MCP's existing `_resolve_credentials` env-else-fetch fallback pattern.)
+- **Memory-only env is the intended safe holder:** dies with the process; just never log values,
+  and don't echo secrets in fail-closed errors.
+- **Reuses, never recreates** the shared trust anchor and CA.
+
+## Adding a future app (runbook)
+
+1. If it needs a secret that the box role can't already read: create the secret and add its ARN
+   to the box role's `GetSecretValue` (IP-restricted to the box).
+2. Prefix its start command with `exec-env <secret> -- …`.
+3. Delete its old `.env` (if any).
+
+That's it. No new role, cert, code, or per-app mechanism.
+
+## Consumer rollout order
+
+1. **This repo (DONE):** this plan + `tools/exec-env` (implemented, tested). Each consumer gets
+   its own ticket **after** this is verified working; every app's rollout needs its own deep dive.
+2. **point-topic-mcp:** `exec-env` prefix + drop `../.env` `env_file` entries + remove `.env`
+   write from `deploy-http-prod.yml`. (Note: MCP's `.env` holds its whole secret set, sourced
+   from GH Actions — treat as a per-app secret of its own; separate, larger ticket.)
+3. **onto-app-new:** `exec-env pt-access-credentials -- docker compose up -d` in `deploy.sh`;
+   remove `.env` write/read.
+4. **pt-dashboards:** `exec-env` + its own secret(s); add to the package's consumer table.
+5. **upc_query_agent:** `exec-env` in a follow-up. (Adopting `pt_access.claims` in its auth path
+   is a separate, unrelated change — not part of this secrets plan.)
+
+Each rollout is staged: create secret → grant box role → install + test `exec-env` → flip the
+launch line at a normal deploy → verify → delete `.env`. No app should experience downtime
+beyond its normal deploy restart window.
+
+## Files in this repo
+
+- `docs/secrets-injection-plan.md` — this plan (design + rollout).
+- `tools/exec-env` — the injector (implemented, tested); install to `/opt/bin/exec-env` per box.
+- `tests/test_exec_env.sh` — behaviour tests for the injector (stubbed `aws`, no network).
+- `README.md` — “Secrets injection” section pointing consumers at this plan.
+
+## Open for confirmation
+
+- ~~Location of `exec-env` source~~ → **landed in this repo** (`tools/exec-env`) per the plan.
+- ~~Shared vs per-app box role~~ → **resolved by review:** keep per-app roles for MCP/pt-mail and
+  any client-facing app; shared `<box>-roles-anywhere` role for new internal apps.

point_topic_access-0.2.1/docs/tickets/point-topic-mcp-lock-bump-open-data.md

@@ -0,0 +1,39 @@
+# point-topic-mcp: bump point-topic-access to 0.2.1 (open_data data source)
+
+**Repo:** Point-Topic/point-topic-mcp
+**Type:** task
+**Blocked by:** point-topic-access 0.2.1 published to PyPI
+
+## Why
+
+The MCP image builds with `uv sync --frozen` against the committed `uv.lock`,
+which currently resolves `point-topic-access` 0.1.1. pt-access 0.2.1 swaps the
+`edgap_ekg` data source for `open_data` (`ALL_DATA_SOURCES` + contract fixture).
+
+Without this bump, a rebuild/redeploy keeps 0.1.1 and org row policies never
+include `(DATA_SOURCE='open_data')` — agents on the MCP stay blind to the new
+source.
+
+## Change
+
+```bash
+uv lock --upgrade-package point-topic-access   # 0.1.1 -> 0.2.1
+uv sync                                        # local check
+uv run pytest -q                               # MCP suite
+```
+
+Commit the updated `uv.lock`. `pyproject.toml` currently pins
+`point-topic-access>=0.1.0`, which already covers 0.2.1 — optional to raise the
+floor to `>=0.2.1` for clarity.
+
+## Deploy & verify
+
+- Rebuild + redeploy via `infrastructure/docker-deploy.sh` on the box.
+- Fresh login for a test org; confirm the org's row policy includes
+  `(DATA_SOURCE='open_data')` and an open_data query returns rows.
+
+## Context
+
+- pt-access change: Point-Topic/point-topic-access (0.2.1 — swap on `main`).
+- Sub-site: the `open_data` product must exist and sit on the org's licence for
+  the JWT `products[]` to carry it.

{point_topic_access-0.2.0 → point_topic_access-0.2.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "point-topic-access"
-version = "0.2.0"
+version = "0.2.1"
 description = "Shared permission -> ClickHouse access-control logic for Point Topic apps (claims, JWT verification, geo filter composer, per-org provisioning)"
 readme = "README.md"
 requires-python = ">=3.12"

{point_topic_access-0.2.0 → point_topic_access-0.2.1}/src/pt_access/composer.py

@@ -55,41 +55,26 @@ CONTAINMENT_TABLE = "ontology.cto_containment_assertion"
 
 # Dataset ↔ dimension compatibility, warn level. UK-geography fields (country,
 # home_nation, la, postcode) on these datasets return 0 rows or are
-# misleading: pcd_checker / edgap_ekg are European (no UK postcode
+# misleading: pcd_checker / open_data are European (no UK postcode
 # containment), ebm is NUTS3-scoped, gbs and global_tariffs are mostly
 # non-UK. Any dataset not listed is compatible with all geography fields.
 # Only "warn" exists today; kept as a table so stricter levels can be added
 # without code change.
 DATASET_FIELD_COMPATIBILITY: dict[str, dict[str, str]] = {
     "pcd_checker": {
-        FIELD_COUNTRY: "warn",
-        FIELD_HOME_NATION: "warn",
-        FIELD_LA: "warn",
-        FIELD_POSTCODE: "warn",
+        FIELD_COUNTRY: "warn", FIELD_HOME_NATION: "warn", FIELD_LA: "warn", FIELD_POSTCODE: "warn"
     },
-    "edgap_ekg": {
-        FIELD_COUNTRY: "warn",
-        FIELD_HOME_NATION: "warn",
-        FIELD_LA: "warn",
-        FIELD_POSTCODE: "warn",
+    "open_data": {
+        FIELD_COUNTRY: "warn", FIELD_HOME_NATION: "warn", FIELD_LA: "warn", FIELD_POSTCODE: "warn"
     },
     "ebm": {
-        FIELD_COUNTRY: "warn",
-        FIELD_HOME_NATION: "warn",
-        FIELD_LA: "warn",
-        FIELD_POSTCODE: "warn",
+        FIELD_COUNTRY: "warn", FIELD_HOME_NATION: "warn", FIELD_LA: "warn", FIELD_POSTCODE: "warn"
     },
     "gbs": {
-        FIELD_COUNTRY: "warn",
-        FIELD_HOME_NATION: "warn",
-        FIELD_LA: "warn",
-        FIELD_POSTCODE: "warn",
+        FIELD_COUNTRY: "warn", FIELD_HOME_NATION: "warn", FIELD_LA: "warn", FIELD_POSTCODE: "warn"
     },
     "global_tariffs": {
-        FIELD_COUNTRY: "warn",
-        FIELD_HOME_NATION: "warn",
-        FIELD_LA: "warn",
-        FIELD_POSTCODE: "warn",
+        FIELD_COUNTRY: "warn", FIELD_HOME_NATION: "warn", FIELD_LA: "warn", FIELD_POSTCODE: "warn"
     },
 }
 
@@ -127,9 +112,7 @@ def _in_list(values: list[str]) -> str:
     return ", ".join(escape_sql_string(v) for v in values)
 
 
-def compose_geo_predicate(
-    field: str, values: list[str | None] | None
-) -> tuple[str, list[str]]:
+def compose_geo_predicate(field: str, values: list[str | None] | None) -> tuple[str, list[str]]:
     """Build the SQL predicate for one filter row.
 
     Args:
@@ -213,9 +196,7 @@ def compose_sql_filter(datasets: dict[str, dict] | None) -> tuple[str, list[str]
         for f in filters:
             field = (f or {}).get("field")
             if field not in GEO_FIELDS:
-                warnings.append(
-                    f"Dataset '{ds}': unknown filter field '{field}' — ignored"
-                )
+                warnings.append(f"Dataset '{ds}': unknown filter field '{field}' — ignored")
                 continue
             compat = DATASET_FIELD_COMPATIBILITY.get(ds, {}).get(field)
             if compat == "warn":
@@ -232,9 +213,7 @@ def compose_sql_filter(datasets: dict[str, dict] | None) -> tuple[str, list[str]
             # A broad row (whole country / home nation) subsumes narrower LA
             # and postcode rows when OR-combined — warn, don't drop silently.
             filter_fields = {
-                (f or {}).get("field")
-                for f in filters
-                if (f or {}).get("field") in GEO_FIELDS
+                (f or {}).get("field") for f in filters if (f or {}).get("field") in GEO_FIELDS
             }
             if len(preds) > 1 and {FIELD_COUNTRY, FIELD_HOME_NATION} & filter_fields:
                 warnings.append(

{point_topic_access-0.2.0 → point_topic_access-0.2.1}/src/pt_access/jwt.py

@@ -36,15 +36,15 @@ class Auth0TokenVerifier:
         jwks_url: str | None = None,
     ):
         """Args:
-        auth0_domain: Auth0 tenant domain (e.g. 'point-topic.eu.auth0.com').
-        audience: Accepted audience(s). May be a single string, a
-            comma-separated string, or a list. Sub-site tokens carry
-            'http://ss-api'; tokens minted for the MCP resource (RFC 9728)
-            carry the resource URL (e.g. 'https://mcp.point-topic.com/mcp');
-            ID tokens from express-openid-connect carry the app's
-            client_id.
-        jwks_url: Optional JWKS URL. Defaults to
-            https://{domain}/.well-known/jwks.json
+            auth0_domain: Auth0 tenant domain (e.g. 'point-topic.eu.auth0.com').
+            audience: Accepted audience(s). May be a single string, a
+                comma-separated string, or a list. Sub-site tokens carry
+                'http://ss-api'; tokens minted for the MCP resource (RFC 9728)
+                carry the resource URL (e.g. 'https://mcp.point-topic.com/mcp');
+                ID tokens from express-openid-connect carry the app's
+                client_id.
+            jwks_url: Optional JWKS URL. Defaults to
+                https://{domain}/.well-known/jwks.json
         """
         self.auth0_domain = auth0_domain
         if isinstance(audience, str):
@@ -61,11 +61,7 @@ class Auth0TokenVerifier:
     async def _get_jwks(self) -> dict | None:
         """Fetch JWKS from Auth0, cached with a TTL for key rotation."""
         now = time.time()
-        if (
-            self._jwks is None
-            or self._jwks_fetched_at is None
-            or now - self._jwks_fetched_at > self._jwks_ttl
-        ):
+        if self._jwks is None or self._jwks_fetched_at is None or now - self._jwks_fetched_at > self._jwks_ttl:
             async with httpx.AsyncClient() as client:
                 response = await client.get(self.jwks_url)
                 response.raise_for_status()

{point_topic_access-0.2.0 → point_topic_access-0.2.1}/src/pt_access/ontology-permission-contract.json

@@ -11,7 +11,7 @@
     "pcd_checker",
     "roadworks",
     "ofcom_tar",
-    "edgap_ekg"
+    "open_data"
   ],
   "fields": ["country", "home_nation", "la", "postcode"],
   "fieldProducts": {

{point_topic_access-0.2.0 → point_topic_access-0.2.1}/src/pt_access/provisioning.py

@@ -94,9 +94,10 @@ MEASUREMENT_TABLES = [
     "ontology.cto_measurement_record_staging",
 ]
 
-# The 12 live DATA_SOURCEs (verified on prod 2026-08-03). The Point Topic
-# org's full-access role covers exactly these — add new sources here so the
-# admin role keeps seeing them.
+# The 12 live DATA_SOURCEs (verified on prod 2026-08-03; open_data replaced
+# edgap_ekg as the EDGAP data source, 2026-10). The Point Topic org's
+# full-access role covers exactly these — add new sources here so the admin
+# role keeps seeing them.
 ALL_DATA_SOURCES = [
     "upc",
     "upc_takeup",
@@ -109,7 +110,7 @@ ALL_DATA_SOURCES = [
     "pcd_checker",
     "roadworks",
     "ofcom_tar",
-    "edgap_ekg",
+    "open_data",
 ]
 
 # Point Topic org — internal admin, full access (env-overridable; mirrors

{point_topic_access-0.2.0 → point_topic_access-0.2.1}/tests/test_claims.py

@@ -4,6 +4,7 @@ Ported from point-topic-mcp tests/test_middleware.py — but only the pure
 functions; the FastMCP middleware itself stays in the MCP.
 """
 
+
 from pt_access import claims
 from pt_access.provisioning import ALL_DATA_SOURCES, POINT_TOPIC_ORG_ID
 
@@ -16,7 +17,6 @@ def _claims(**overrides) -> dict:
 
 # ── get_org_id / is_pt_admin ───────────────────────────────────────────────────
 
-
 def test_get_org_id():
     assert claims.get_org_id(_claims()) == "abc123"
     assert claims.get_org_id({}) is None
@@ -31,14 +31,8 @@ def test_is_pt_admin():
 
 # ── get_held_products (both token shapes) ──────────────────────────────────────
 
-
 def test_held_products_current_shape():
-    c = _claims(
-        products=[
-            {"name": "upc", "permissions": {}},
-            {"name": "gbs", "permissions": {}},
-        ]
-    )
+    c = _claims(products=[{"name": "upc", "permissions": {}}, {"name": "gbs", "permissions": {}}])
     assert claims.get_held_products(c) == ["upc", "gbs"]
 
 
@@ -55,24 +49,13 @@ def test_held_products_missing_or_malformed():
 
 # ── has_data_source_access ─────────────────────────────────────────────────────
 
-
 def test_any_data_source_product_unlocks_ontology_family():
     for ds in ALL_DATA_SOURCES:
-        assert (
-            claims.has_data_source_access(
-                _claims(products=[{"name": ds, "permissions": {}}])
-            )
-            is True
-        )
+        assert claims.has_data_source_access(_claims(products=[{"name": ds, "permissions": {}}])) is True
 
 
 def test_legacy_ontology_product_still_accepted():
-    assert (
-        claims.has_data_source_access(
-            _claims(products=[{"name": "ontology", "permissions": {}}])
-        )
-        is True
-    )
+    assert claims.has_data_source_access(_claims(products=[{"name": "ontology", "permissions": {}}])) is True
 
 
 def test_non_data_source_products_do_not_unlock():
@@ -86,7 +69,6 @@ def test_no_products_denies():
 
 # ── get_product_permission / get_product_role ─────────────────────────────────
 
-
 def test_product_permission_reads_scalar_value():
     c = _claims(products=[{"name": "ontology_app", "permissions": {"role": "admin"}}])
     assert claims.get_product_permission(c, "ontology_app", "role") == "admin"

{point_topic_access-0.2.0 → point_topic_access-0.2.1}/tests/test_composer.py

@@ -18,7 +18,6 @@ from pt_access.composer import (
 
 # ── escaping ───────────────────────────────────────────────────────────────────
 
-
 def test_escape_sql_string_doubles_quotes():
     assert escape_sql_string("King's Lynn") == "'King''s Lynn'"
 
@@ -40,7 +39,6 @@ def test_injection_in_dataset_name_escaped():
 
 # ── country ────────────────────────────────────────────────────────────────────
 
-
 def test_country_shape():
     sql, warnings = compose_sql_filter(
         {"upc": {"filters": [{"field": "country", "values": ["United Kingdom"]}]}}
@@ -56,7 +54,6 @@ def test_country_multiple_values():
 
 # ── postcode ───────────────────────────────────────────────────────────────────
 
-
 def test_postcode_shape():
     sql, warnings = compose_sql_filter(
         {"upc": {"filters": [{"field": "postcode", "values": ["AB1 2CD", "DD1 4HN"]}]}}
@@ -67,7 +64,6 @@ def test_postcode_shape():
 
 # ── LA branch ──────────────────────────────────────────────────────────────────
 
-
 def test_la_names_shape():
     """la_name branch — live-verified containment subquery shape."""
     sql, warnings = compose_sql_filter(
@@ -119,7 +115,6 @@ def test_unknown_field_raises():
 
 # ── home_nation branch ─────────────────────────────────────────────────────────
 
-
 def test_home_nation_shape():
     """home_nation containment branch — same shape as la, level='home_nation'.
 
@@ -197,15 +192,12 @@ def test_home_nation_alone_no_subsume_warning():
 
 # ── per-dataset disjunction ────────────────────────────────────────────────────
 
-
 def test_disjunction_guards_each_dataset():
     """Each disjunct is guarded by its own DATA_SOURCE literal (no IN blob)."""
     sql, warnings = compose_sql_filter(
         {
             "upc": {"filters": [{"field": "country", "values": ["United Kingdom"]}]},
-            "upc_takeup": {
-                "filters": [{"field": "country", "values": ["United Kingdom"]}]
-            },
+            "upc_takeup": {"filters": [{"field": "country", "values": ["United Kingdom"]}]},
             "global_tariffs": {},
         }
     )
@@ -311,7 +303,6 @@ def test_none_datasets_yield_no_sql():
 
 # ── value cleaning ─────────────────────────────────────────────────────────────
 
-
 def test_values_are_stripped_deduped_and_empties_dropped():
     sql, _ = compose_geo_predicate(
         "postcode", ["  AB1 2CD ", "", "DD1 4HN", "  ", "AB1 2CD", None]
@@ -347,7 +338,6 @@ def test_empty_filter_row_plus_other_datasets_excludes_only_that_dataset():
 
 # ── compatibility warnings ─────────────────────────────────────────────────────
 
-
 def test_compatible_dataset_no_warning():
     _, warnings = compose_sql_filter(
         {"upc": {"filters": [{"field": "country", "values": ["United Kingdom"]}]}}
@@ -356,7 +346,7 @@ def test_compatible_dataset_no_warning():
 
 
 @pytest.mark.parametrize(
-    "ds", ["pcd_checker", "edgap_ekg", "ebm", "gbs", "global_tariffs"]
+    "ds", ["pcd_checker", "open_data", "ebm", "gbs", "global_tariffs"]
 )
 def test_incompatible_dataset_warns(ds):
     """UK-geography fields on these datasets warn but still compose."""
@@ -368,7 +358,7 @@ def test_incompatible_dataset_warns(ds):
 
 
 @pytest.mark.parametrize(
-    "ds", ["pcd_checker", "edgap_ekg", "ebm", "gbs", "global_tariffs"]
+    "ds", ["pcd_checker", "open_data", "ebm", "gbs", "global_tariffs"]
 )
 def test_home_nation_warns_on_incompatible_datasets(ds):
     """home_nation is UK geography — same warn-on-non-UK-datasets rule."""
@@ -380,7 +370,7 @@ def test_home_nation_warns_on_incompatible_datasets(ds):
 
 
 def test_compatibility_table_covers_all_warned_datasets():
-    for ds in ("pcd_checker", "edgap_ekg", "ebm", "gbs", "global_tariffs"):
+    for ds in ("pcd_checker", "open_data", "ebm", "gbs", "global_tariffs"):
         assert DATASET_FIELD_COMPATIBILITY[ds] == {
             "country": "warn",
             "home_nation": "warn",

{point_topic_access-0.2.0 → point_topic_access-0.2.1}/tests/test_contract.py

@@ -20,15 +20,15 @@ from pt_access.composer import GEO_FIELDS
 
 
 def _contract() -> dict:
-    fixture = resources.files("pt_access").joinpath("ontology-permission-contract.json")
+    fixture = resources.files("pt_access").joinpath(
+        "ontology-permission-contract.json"
+    )
     return json.loads(fixture.read_text(encoding="utf-8"))
 
 
 def test_data_sources_match_contract():
     contract = _contract()
-    assert len(op.ALL_DATA_SOURCES) == len(contract["dataSources"]), (
-        "duplicate data sources?"
-    )
+    assert len(op.ALL_DATA_SOURCES) == len(contract["dataSources"]), "duplicate data sources?"
     assert sorted(op.ALL_DATA_SOURCES) == sorted(contract["dataSources"])
 
 

point_topic_access-0.2.1/tests/test_exec_env.sh

@@ -0,0 +1,85 @@
+#!/usr/bin/env bash
+# Behaviour tests for tools/exec-env — no AWS needed, the aws CLI is stubbed.
+# Run: bash tests/test_exec_env.sh
+set -u
+
+HERE="$(cd "$(dirname "$0")/.." && pwd)"
+EXEC_ENV="$HERE/tools/exec-env"
+TMP="$(mktemp -d)"
+trap 'rm -rf "$TMP"' EXIT
+
+fail() { echo "FAIL: $1" >&2; exit 1; }
+pass() { echo "ok: $1"; }
+
+cat > "$TMP/aws" <<'STUB'
+#!/usr/bin/env bash
+# Stub `aws secretsmanager get-secret-value`: canned output per --secret-id.
+id=""
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --secret-id) id="$2"; shift 2 ;;
+    *) shift ;;
+  esac
+done
+case "$id" in
+  good)
+    printf '%s' '{"CLICKHOUSE_HOST":"db.example.com","CLICKHOUSE_PORT":"443","CLICKHOUSE_PROVISIONING_USER":"provisioner","CLICKHOUSE_PROVISIONING_PASSWORD":"p@ss word$=x","SUB_SITE_MONGODB_URI":"mongodb://user:pw@host:27017/db","MCP_CLICKHOUSE_GRANT_TO_USER":"mcp_service","POINT_TOPIC_ORG_ID":"65dc868c4f33e6b7d256fab4"}'
+    ;;
+  fails) echo '{"CLICKHOUSE_PASSWORD":"SUPERSECRET"}'; exit 1 ;;
+  badjson) printf '%s' '{not json' ;;
+  badkey) printf '%s' '{"BAD KEY":"v","OK_KEY":"fine"}' ;;
+  nonstring) printf '%s' '{"CLICKHOUSE_PORT":443}' ;;
+  multiline) printf '%s' '{"CLICKHOUSE_PASSWORD":"line1\nline2"}' ;;
+  empty) printf '%s' '{}' ;;
+esac
+exit 0
+STUB
+chmod +x "$TMP/aws"
+
+run() { PATH="$TMP:$PATH" "$EXEC_ENV" "$@"; }
+
+# 1. Happy path: exports every key verbatim and execs the command.
+out="$(run good -- sh -c 'printf "%s|%s|%s|%s" "$CLICKHOUSE_HOST" "$CLICKHOUSE_PROVISIONING_PASSWORD" "$SUB_SITE_MONGODB_URI" "$POINT_TOPIC_ORG_ID"')"
+[ "$out" = 'db.example.com|p@ss word$=x|mongodb://user:pw@host:27017/db|65dc868c4f33e6b7d256fab4' ] \
+  || fail "happy path (got: $out)"
+pass "happy path: keys exported verbatim, command exec'd"
+
+# 2. exec semantics: child's exit code propagates.
+run good -- sh -c 'exit 7' 2>/dev/null
+[ $? -eq 7 ] || fail "exit code passthrough"
+pass "exec semantics: child exit code propagates"
+
+# 3. --region is accepted.
+out="$(run good --region eu-west-1 -- sh -c 'printf "%s" "$CLICKHOUSE_HOST"')"
+[ "$out" = "db.example.com" ] || fail "--region parsing (got: $out)"
+pass "--region accepted"
+
+# 4. Fetch failure: exits non-zero, generic message, NEVER echoes the secret.
+out="$(run fails -- true 2>&1)"; code=$?
+[ $code -ne 0 ] || fail "fetch failure should exit non-zero"
+case "$out" in
+  *SUPERSECRET*) fail "fetch failure echoed the secret: $out" ;;
+esac
+case "$out" in
+  *"failed to fetch secret 'fails'"*) : ;;
+  *) fail "fetch failure missing generic message: $out" ;;
+esac
+pass "fetch failure: fail closed, secret never echoed"
+
+# 5. Malformed payloads all fail closed with no value echo.
+for id in badjson badkey nonstring multiline empty; do
+  out="$(run "$id" -- true 2>&1)"; code=$?
+  [ $code -ne 0 ] || fail "$id should exit non-zero"
+  case "$out" in
+    *line1*) fail "$id echoed a value" ;;
+  esac
+done
+pass "malformed payloads (badjson/badkey/nonstring/multiline/empty) fail closed"
+
+# 6. Usage errors exit 2.
+run 2>/dev/null;            [ $? -eq 2 ] || fail "no args should exit 2"
+run good true 2>/dev/null;  [ $? -eq 2 ] || fail "missing '--' should exit 2"
+run good -- 2>/dev/null;    [ $? -eq 2 ] || fail "no command after '--' should exit 2"
+pass "usage errors exit 2"
+
+echo "all exec-env tests passed"

{point_topic_access-0.2.0 → point_topic_access-0.2.1}/tests/test_jwt.py

@@ -91,9 +91,7 @@ def jwks_fixture():
 
 
 def _patch_jwks(jwks):
-    return patch(
-        "httpx.AsyncClient.get", new=AsyncMock(return_value=_FakeResponse(jwks))
-    )
+    return patch("httpx.AsyncClient.get", new=AsyncMock(return_value=_FakeResponse(jwks)))
 
 
 async def test_verify_valid_token_returns_claims(verifier, jwks_fixture):

point_topic_access-0.2.1/tools/exec-env

@@ -0,0 +1,92 @@
+#!/usr/bin/env bash
+# exec-env — fetch a JSON secret from AWS Secrets Manager and exec a command
+# with every key exported into its environment (sops exec-env style).
+#
+#   exec-env <secret-id> [--region <region>] -- <command [args...]>
+#
+# Design rules (docs/secrets-injection-plan.md):
+#   - Uses the ambient AWS credential chain (Roles Anywhere / profile);
+#     the wrapped app itself never needs AWS access.
+#   - Never writes anything to disk — values live only in the child env.
+#   - Fail closed: exits non-zero on any fetch/parse failure and never
+#     prints a secret value (aws/python3 stderr is suppressed).
+#   - Env keys must match [A-Za-z0-9_]+; values must be single-line strings.
+#   - bash 3.2+ (macOS default) and any box Linux bash.
+
+set -euo pipefail
+
+REGION="${AWS_DEFAULT_REGION:-eu-west-1}"
+
+usage() {
+  echo "usage: exec-env <secret-id> [--region <region>] -- <command [args...]>" >&2
+}
+
+[[ $# -gt 0 ]] || { usage; exit 2; }
+case "$1" in
+  -h|--help) usage; exit 0 ;;
+esac
+SECRET_ID="$1"
+shift
+
+while [[ $# -gt 0 && "$1" != "--" ]]; do
+  case "$1" in
+    --region)
+      [[ $# -ge 2 ]] || { echo "exec-env: --region needs a value" >&2; exit 2; }
+      REGION="$2"
+      shift 2
+      ;;
+    *)
+      echo "exec-env: unknown option '$1' — expected '--' before the command" >&2
+      exit 2
+      ;;
+  esac
+done
+[[ ${1:-} == "--" ]] || { echo "exec-env: missing '--' before the command" >&2; usage; exit 2; }
+shift
+[[ $# -gt 0 ]] || { echo "exec-env: no command given after '--'" >&2; exit 2; }
+
+# Fetch. The secret never leaves this process and is never printed.
+json="$(aws secretsmanager get-secret-value \
+    --secret-id "$SECRET_ID" \
+    --region "$REGION" \
+    --query SecretString \
+    --output text 2>/dev/null)" || {
+  echo "exec-env: failed to fetch secret '$SECRET_ID' (region $REGION) — check the role's GetSecretValue grant and the ambient AWS profile" >&2
+  exit 1
+}
+
+# Parse into key=value lines. Values must be single-line strings (no NUL,
+# no newline); keys are re-validated below before export.
+pairs="$(printf '%s' "$json" | python3 -c "
+import json, sys
+data = json.load(sys.stdin)
+if not isinstance(data, dict):
+    sys.exit('payload is not a JSON object')
+for key, value in data.items():
+    if not key or not all('a' <= c <= 'z' or 'A' <= c <= 'Z' or '0' <= c <= '9' or c == '_' for c in key):
+        sys.exit('invalid env key')
+    if not isinstance(value, str) or chr(0) in value or chr(10) in value or chr(13) in value:
+        sys.exit('unsupported value for key ' + key)
+    sys.stdout.write(key + '=' + value + chr(10))
+" 2>/dev/null)" || {
+  echo "exec-env: failed to parse secret '$SECRET_ID' (expected a flat JSON object of single-line strings)" >&2
+  exit 1
+}
+unset json
+
+exported=0
+while IFS='=' read -r key value; do
+  case "$key" in
+    ""|*[!A-Za-z0-9_]*)
+      echo "exec-env: secret '$SECRET_ID' contains an invalid env key — refusing to export" >&2
+      exit 1
+      ;;
+  esac
+  export "$key=$value"
+  exported=$((exported + 1))
+done <<< "$pairs"
+unset pairs
+
+[[ "$exported" -gt 0 ]] || { echo "exec-env: secret '$SECRET_ID' has no keys to export" >&2; exit 1; }
+
+exec "$@"

{point_topic_access-0.2.0 → point_topic_access-0.2.1}/uv.lock

@@ -415,7 +415,7 @@ wheels = [
 
 [[package]]
 name = "point-topic-access"
-version = "0.2.0"
+version = "0.2.1"
 source = { editable = "." }
 dependencies = [
     { name = "clickhouse-connect" },