sum-engine 0.5.0__tar.gz → 0.7.0__tar.gz

{sum_engine-0.5.0/sum_engine.egg-info → sum_engine-0.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sum-engine
-Version: 0.5.0
+Version: 0.7.0
 Summary: SUM — bidirectional knowledge distillation with optional cryptographic attestation. Pipe prose, get a CanonicalBundle (HMAC / Ed25519 / W3C VC 2.0), verify anywhere.
 Author: ototao
 License: Apache-2.0
@@ -25,9 +25,11 @@ Requires-Dist: cryptography>=41.0.0
 Requires-Dist: sympy>=1.12
 Provides-Extra: sieve
 Requires-Dist: spacy>=3.7.0; extra == "sieve"
+Provides-Extra: openai
+Requires-Dist: openai<3.0.0,>=1.40.0; extra == "openai"
+Requires-Dist: pydantic>=2.0.0; extra == "openai"
 Provides-Extra: llm
-Requires-Dist: openai<3.0.0,>=1.40.0; extra == "llm"
-Requires-Dist: pydantic>=2.0.0; extra == "llm"
+Requires-Dist: sum-engine[openai]; extra == "llm"
 Provides-Extra: anthropic
 Requires-Dist: anthropic>=0.97.0; extra == "anthropic"
 Requires-Dist: pydantic>=2.0.0; extra == "anthropic"
@@ -51,7 +53,7 @@ Requires-Dist: build>=1.0.0; extra == "dev"
 Requires-Dist: hypothesis>=6.0.0; extra == "dev"
 Provides-Extra: all
 Requires-Dist: sum-engine[sieve]; extra == "all"
-Requires-Dist: sum-engine[llm]; extra == "all"
+Requires-Dist: sum-engine[openai]; extra == "all"
 Requires-Dist: sum-engine[anthropic]; extra == "all"
 Requires-Dist: sum-engine[receipt-verify]; extra == "all"
 Requires-Dist: sum-engine[mcp]; extra == "all"
@@ -66,9 +68,15 @@ Dynamic: license-file
 [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
 [![Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
 
-> **A cross-runtime trust surface for LLM-rendered text.** Three runtimes (Python, Node, modern browsers) produce byte-identical Ed25519 signatures over the same JCS-canonical bytes. Every render through the hosted Worker carries a detached-JWS receipt (`sum.render_receipt.v1`) that any third party can verify offline against `/.well-known/jwks.json`. Live at https://sum-demo.ototao.workers.dev.
+> **SUM lets people and agents transform knowledge without losing the ability to verify what changed, what stayed the same, who signed it, and what remains unproven.**
 
-That is the load-bearing claim and what makes SUM different from a generic summarisation tool. The cryptographic side is **mechanically proven** — three independent verifier implementations agreeing byte-for-byte on every signed bundle, locked in CI on every PR. The semantic side (extraction quality, slider fact preservation) is **empirically measured** with explicit per-corpus numbers and explicit per-corpus boundaries; SUM does not blur the line between the two. [`docs/PROOF_BOUNDARY.md`](docs/PROOF_BOUNDARY.md) is the arbiter.
+Every transformation — extract triples from prose, render a tome at a controlled slider position, compose bundles across documents, share a render — emits a cryptographically-signed receipt that any third party can verify offline. The receipt attests *that the transformation happened and what its inputs were*. Separate per-axis benchmarks attest *how much the transformation preserved meaning*. Both are kept honest by separate proof discipline — and the project never blurs the line between them.
+
+*Live trust loop:* https://sum-demo.ototao.workers.dev — three runtimes (Python, Node, modern browsers) produce byte-identical Ed25519 signatures over the same JCS-canonical bytes; verify offline against `/.well-known/jwks.json`. Mechanically proven; locked in CI on every PR.
+
+**Built for:** journalists working under deepfake-era citation requirements, academic survey writers who need provenance back to source PDFs, agentic-AI builders who need their agents to pass verifiable evidence and not just messages, and regulated-domain content (EU AI Act Article 12, FTC AI disclosure, HIPAA, SOC 2, PCI DSS) where "we say it's true" isn't enough.
+
+The cryptographic side is **mechanically proven** — three independent verifier implementations agreeing byte-for-byte on every signed bundle, locked in CI on every PR. The semantic side (extraction quality, slider fact preservation) is **empirically measured** with explicit per-corpus numbers and explicit per-corpus boundaries. [`docs/PROOF_BOUNDARY.md`](docs/PROOF_BOUNDARY.md) is the arbiter.
 
 Headline supporting numbers (each links to its source of truth):
 
@@ -110,14 +118,29 @@ A minimal Node verifier using `jose` + `canonicalize` is in [`docs/RENDER_RECEIP
 | Surface | Status | Verifies |
 |---|---|---|
 | `pip install 'sum-engine[sieve]'` — `sum attest` / `sum verify` / `sum render` / `sum resolve` / `sum ledger` / `sum inspect` / `sum schema` | shipped on PyPI ≥ 0.4.1 | structural reconstruction; HMAC-SHA256 + Ed25519 signatures (W3C VC 2.0 `eddsa-jcs-2022`); bidirectional `sum attest` ↔ `sum render` symmetry from the shell |
-| Cloudflare Worker at `sum-demo.ototao.workers.dev` | shipped | `/api/render` → tome + `render_receipt`; `/.well-known/jwks.json` → JWKS; `/api/qid` → Wikidata resolver |
+| Cloudflare Worker at `sum-demo.ototao.workers.dev` | shipped | `/api/render` → tome + `render_receipt`; `/api/transform` → generic transform-registry dispatch + `sum.transform_receipt.v1`; `/api/complete` → LLM proxy; `/api/qid` → Wikidata resolver; `/.well-known/jwks.json` + `/.well-known/revoked-kids.json` → trust-loop endpoints. Public LLM-axis routes are rate-limited per IP — see [`docs/PUBLIC_API_RATE_LIMITS.md`](docs/PUBLIC_API_RATE_LIMITS.md) (5/day operator-keyed demo; 100/hr with BYO key via `X-Render-LLM-Key-Anthropic` / `-OpenAI`). |
 | Single-file browser demo (`single_file_demo/index.html`) | shipped | paste prose → in-browser attest → CanonicalBundle JSON; same bytes verify under `node standalone_verifier/verify.js` (Chrome / Firefox / Safari with WebCrypto Ed25519 support) |
 | Cross-runtime trust triangle | locked by CI (`make xruntime`) | K1 / K1-mw / K2 / K3 / K4 — Python ↔ Node ↔ Browser agree byte-for-byte on valid bundles. `make xruntime-adversarial` adds A1–A6 rejection-class equivalence. |
-| 5-axis slider rendering surface | density actioned deterministically; length / formality / audience / perspective LLM-conditioned via the Worker (Anthropic, Cloudflare AI Gateway optional) | bench: median LLM-axis fact preservation 1.000, p10 0.769 (long, n=16) / 0.818 (short, n=8), order preservation 1.000 wherever measurable |
+| 5-axis slider rendering surface | density actioned deterministically; length / formality / audience / perspective LLM-conditioned. Two dispatch paths: Worker `/api/render` (Anthropic + Cloudflare AI Gateway optional) producing `sum.render_receipt.v1`, OR Python `sum transform apply slider` (OpenAI via `OPENAI_API_KEY`) producing `sum.transform_receipt.v1` | bench: median LLM-axis fact preservation 1.000, p10 0.769 (long, n=16) / 0.818 (short, n=8), order preservation 1.000 wherever measurable. Tightening worktrail at [`docs/BENCH_HARDENING_FROM_QCVV.md`](docs/BENCH_HARDENING_FROM_QCVV.md) adds iteration-stability + DKW worst-case bounds + capability-region headlines |
 | MCP server (`sum-mcp` console script) | shipped | five tools (`extract` / `attest` / `verify` / `inspect` / `schema`) exposed over stdio; bundles attested via MCP verify byte-identically through the CLI / Node / browser verifiers |
+| Transform substrate (`sum.transform_receipt.v1` + registry) | shipped (CLI in repo HEAD; PyPI catch-up tag pending) | `sum transform list` / `sum transform apply <name>` — three registered transforms (`slider` / `extract` / `compose`); receipts via Ed25519 / JCS / detached JWS just like render-receipts; 20-fixture cross-runtime K-matrix locks accept + reject across Python ↔ Node ↔ browser; T4 `source_chain_hash` binds receipts to source byte ranges; T5 `ShareableRender` round-trips signed renders for offline verification; T6 multi-school extract runs two extractors in tandem for adversarial-divergence detection. Wire spec at [`docs/TRANSFORM_RECEIPT_FORMAT.md`](docs/TRANSFORM_RECEIPT_FORMAT.md); design at [`docs/TRANSFORM_REGISTRY.md`](docs/TRANSFORM_REGISTRY.md). |
+| Replay-defense window (`signed_at_out_of_window`) | shipped | opt-in `max_age_seconds` parameter across all four verifier surfaces (Python render / Python transform / JS render / JS transform). Default-off preserves archival use; receivers opt in per use-case (agent-swarm 60s, real-time 600s, newsletter 1d, legal-discovery no window). |
+| `sum verify --explain` layered output | shipped | Per-dimension report (`sum.verify_explained.v1`): cryptographic integrity / canonical reconstruction / axiom consistency / extraction provenance / source evidence coverage / semantic preservation / truth of content. Each carries `epistemic_status` (`provable` / `certified` / `empirical-benchmark` / `not-asserted`). Truth of content is ALWAYS `not_asserted` — locked by test. |
+| Negative-control corpus (T5 of bench-hardening) | shipped | 20 hand-authored documents across 5 failure modes (ambiguous coref / predicate-alias / contradictions / entity-resolution-adversarial / non-extractable). Runner exits 1 if observed failures don't match annotations. Baseline at [`fixtures/bench_receipts/negative_control_2026-05-17.json`](fixtures/bench_receipts/negative_control_2026-05-17.json). |
+| Compliance validators (six regimes) | shipped | `sum compliance check --regime <id> --audit-log <path>` — EU AI Act Article 12, GDPR Article 30, HIPAA § 164.312(b), ISO/IEC 27001 A.8.15, SOC 2 CC 7.2, PCI DSS v4.0 Req 10. All six produce the same `sum.compliance_report.v1` schema; per-regime docs at `docs/COMPLIANCE_*.md`. |
 
 The slider's product claim — *axis changes do not lose facts* — is the load-bearing empirical result. It is verified by NLI audit on every embedding-flagged "loss" cell; full attribution in [`docs/SLIDER_CONTRACT.md`](docs/SLIDER_CONTRACT.md).
 
+## Strategic context
+
+The operational compass — read in this order if you want the project's intent + how it operates + where it's going:
+
+- [`docs/CHARTER_2026-05-17.md`](docs/CHARTER_2026-05-17.md) — intent, the Why, strategy, objectives, success criteria, constraints, and the operational loop. The compass every other doc resolves to.
+- [`docs/PRODUCT_DELIBERATION_2026-05-14.md`](docs/PRODUCT_DELIBERATION_2026-05-14.md) — three-option strategic analysis + grant-outcome decision tree.
+- [`docs/ZENITH_FRAMING_2026-05-16.md`](docs/ZENITH_FRAMING_2026-05-16.md) — destination framing (SUM as chain-of-custody for AI-transformed knowledge) plus three new concepts (Perspective Receipts, Trust Profiles, Epistemic Nutrition Label) on the design queue.
+- [`docs/BENCH_HARDENING_FROM_QCVV.md`](docs/BENCH_HARDENING_FROM_QCVV.md) — five-task empirical-benchmark hardening plan (T1–T5; T5 shipped, T1–T4 queued).
+- [`docs/DOGFOOD_QUICKSTART.md`](docs/DOGFOOD_QUICKSTART.md) — five-minute guide to running SUM on your own writing.
+
 ### LLM narrative round-trip — closed across measured corpora (2026-04-28)
 
 The hardest measurement in `PROOF_BOUNDARY.md` is the full LLM narrative round-trip (`text → LLM-extract → axioms → LLM-generate → prose' → LLM-extract → axioms'`). The unprompted-pipeline baseline on `seed_v1` was **drift = 107.75% / exact-match recall = 0.12** — facts preserved, keys not.
@@ -165,7 +188,7 @@ The reverse direction also runs under explicit slider control. The local path ac
 sum render --density 0.5 < bundle.json
 # → keeps the lex-prefix half of the axioms; @sliders header records what was requested
 
-sum render --length 0.9 --use-worker https://sum.ototao.com --json < bundle.json
+sum render --length 0.9 --use-worker https://sum-demo.ototao.workers.dev --json < bundle.json
 # → LLM-conditioned tome + signed render_receipt (sum.render_receipt.v1) on stdout
 ```
 
@@ -192,7 +215,7 @@ pip install 'sum-engine[mcp,sieve]'
 
 ### Calling SUM over HTTP
 
-The hosted Worker at `https://sum.ototao.com` exposes `/api/render`, `/api/complete`, `/api/qid`, and the `/.well-known/{jwks,revoked-kids}.json` verification surfaces. [`docs/API_REFERENCE.md`](docs/API_REFERENCE.md) is the wire spec — request/response shapes, error codes, the six-step receipt-verification flow, working Node + Python examples. Use this when the caller is a web app, mobile app, or server-side service; use the MCP server when the caller is a local LLM client.
+The hosted Worker at `https://sum-demo.ototao.workers.dev` exposes `/api/render`, `/api/complete`, `/api/qid`, and the `/.well-known/{jwks,revoked-kids}.json` verification surfaces. [`docs/API_REFERENCE.md`](docs/API_REFERENCE.md) is the wire spec — request/response shapes, error codes, the six-step receipt-verification flow, working Node + Python examples. Use this when the caller is a web app, mobile app, or server-side service; use the MCP server when the caller is a local LLM client.
 
 ---
 
@@ -228,7 +251,29 @@ Below the slider sits the substrate that earlier phases shipped and verified. Po
 - **Bundle public-key attestation (provable).** Ed25519-signed CanonicalBundles are tamper-detectable by any third party in any of the three runtimes. [`docs/PROOF_BOUNDARY.md`](docs/PROOF_BOUNDARY.md) §1.3.1.
 - **Merkle hash-chain integrity (provable, including under concurrent writers).** [`docs/PROOF_BOUNDARY.md`](docs/PROOF_BOUNDARY.md) §1.7.
 - **Extraction F1 (empirical-benchmark).** 1.000 on `seed_v1` (50 simple-SVO docs); 0.762 with precision 1.000 on `seed_v2` (20-doc difficulty corpus). Every remaining `seed_v2` failure is a recall miss, not a truth inversion. [`docs/PROOF_BOUNDARY.md`](docs/PROOF_BOUNDARY.md) §2.1.
-- **103 numbered features**, each with a reproducible verification command, in [`docs/FEATURE_CATALOG.md`](docs/FEATURE_CATALOG.md).
+- **168 numbered features**, each with a reproducible verification command, in [`docs/FEATURE_CATALOG.md`](docs/FEATURE_CATALOG.md).
+
+### Research substrate (under `sum_engine_internal/research/`)
+
+Less-surfaced but shipped:
+
+- **MinHash-LSH bundle similarity index** (`research/lsh/`) — near-duplicate bundle detection at scale.
+- **Robust PCA corruption score** (`research/robust_pca/`) — `corruption_score` field in bundle metadata; flags adversarially-perturbed bundles.
+- **Sequential & conformal-prediction** (`research/sequential/`, `research/conformal/`) — bench-side confidence bounds with documented coverage guarantees.
+- **MMD distribution distance** (`research/mmd/`) — `axiom_distribution_mmd` field on bundles; surfaces when an attested bundle is structurally unlike its baseline corpus.
+- **Spectral entropy** (`research/spectral_entropy/`) — axiom-graph entropy on every bundle, with confidence interval.
+- **Bootstrap multiplier spike detection** (`research/bootstrap/`) — see [`docs/MULTIPLIER_BOOTSTRAP_SPIKE_FINDINGS.md`](docs/MULTIPLIER_BOOTSTRAP_SPIKE_FINDINGS.md).
+- **SMT consistency checking** (`research/smt_consistency/`) — z3-backed `axiom_consistency_check` on every bundle.
+- **Sheaf-Laplacian hallucination detection** — see [`docs/SHEAF_HALLUCINATION_DETECTOR.md`](docs/SHEAF_HALLUCINATION_DETECTOR.md) (research direction).
+
+### Other substrate-adjacent surfaces
+
+- **Trust-root manifest** (`sum_engine_internal/trust_root/`) — operator-issued signed manifest binding kid lifecycle, revocation policy, and verifier expectations.
+- **Merkle sidecar format** (`sum_engine_internal/merkle_sidecar/`) — see [`docs/MERKLE_SIDECAR_FORMAT.md`](docs/MERKLE_SIDECAR_FORMAT.md).
+- **Evidence-chain layer** (`sum_engine_internal/evidence/`) — substrate behind `source_chain_hash` (T4).
+- **Algorithm registry** — see [`docs/ALGORITHM_REGISTRY.md`](docs/ALGORITHM_REGISTRY.md) (the in-tree list of permitted signing algs; crypto-agility gate).
+- **Audit log format** — every CLI operation can emit `sum.audit_log.v1` events; see [`docs/AUDIT_LOG_FORMAT.md`](docs/AUDIT_LOG_FORMAT.md).
+- **Agent surface** (`sum_engine_internal/agent_surface/`) — see [`docs/AGENT_SURFACE_FINDINGS.md`](docs/AGENT_SURFACE_FINDINGS.md).
 
 ---
 
@@ -275,7 +320,7 @@ Source anchoring in the bundle schema, bundle explorer / viewer, `sum verify --e
 
 ```bash
 make install              # editable install with sieve + dev extras
-make test                 # full pytest run (1000+ tests)
+make test                 # full pytest run (2000+ tests)
 make xruntime             # cross-runtime K1/K1-mw/K2/K3/K4 (Python ↔ Node)
 make xruntime-adversarial # rejection-matrix A1–A6
 make fortress             # 21-check pure-math invariants

{sum_engine-0.5.0 → sum_engine-0.7.0}/README.md

@@ -5,9 +5,15 @@
 [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
 [![Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
 
-> **A cross-runtime trust surface for LLM-rendered text.** Three runtimes (Python, Node, modern browsers) produce byte-identical Ed25519 signatures over the same JCS-canonical bytes. Every render through the hosted Worker carries a detached-JWS receipt (`sum.render_receipt.v1`) that any third party can verify offline against `/.well-known/jwks.json`. Live at https://sum-demo.ototao.workers.dev.
+> **SUM lets people and agents transform knowledge without losing the ability to verify what changed, what stayed the same, who signed it, and what remains unproven.**
 
-That is the load-bearing claim and what makes SUM different from a generic summarisation tool. The cryptographic side is **mechanically proven** — three independent verifier implementations agreeing byte-for-byte on every signed bundle, locked in CI on every PR. The semantic side (extraction quality, slider fact preservation) is **empirically measured** with explicit per-corpus numbers and explicit per-corpus boundaries; SUM does not blur the line between the two. [`docs/PROOF_BOUNDARY.md`](docs/PROOF_BOUNDARY.md) is the arbiter.
+Every transformation — extract triples from prose, render a tome at a controlled slider position, compose bundles across documents, share a render — emits a cryptographically-signed receipt that any third party can verify offline. The receipt attests *that the transformation happened and what its inputs were*. Separate per-axis benchmarks attest *how much the transformation preserved meaning*. Both are kept honest by separate proof discipline — and the project never blurs the line between them.
+
+*Live trust loop:* https://sum-demo.ototao.workers.dev — three runtimes (Python, Node, modern browsers) produce byte-identical Ed25519 signatures over the same JCS-canonical bytes; verify offline against `/.well-known/jwks.json`. Mechanically proven; locked in CI on every PR.
+
+**Built for:** journalists working under deepfake-era citation requirements, academic survey writers who need provenance back to source PDFs, agentic-AI builders who need their agents to pass verifiable evidence and not just messages, and regulated-domain content (EU AI Act Article 12, FTC AI disclosure, HIPAA, SOC 2, PCI DSS) where "we say it's true" isn't enough.
+
+The cryptographic side is **mechanically proven** — three independent verifier implementations agreeing byte-for-byte on every signed bundle, locked in CI on every PR. The semantic side (extraction quality, slider fact preservation) is **empirically measured** with explicit per-corpus numbers and explicit per-corpus boundaries. [`docs/PROOF_BOUNDARY.md`](docs/PROOF_BOUNDARY.md) is the arbiter.
 
 Headline supporting numbers (each links to its source of truth):
 
@@ -49,14 +55,29 @@ A minimal Node verifier using `jose` + `canonicalize` is in [`docs/RENDER_RECEIP
 | Surface | Status | Verifies |
 |---|---|---|
 | `pip install 'sum-engine[sieve]'` — `sum attest` / `sum verify` / `sum render` / `sum resolve` / `sum ledger` / `sum inspect` / `sum schema` | shipped on PyPI ≥ 0.4.1 | structural reconstruction; HMAC-SHA256 + Ed25519 signatures (W3C VC 2.0 `eddsa-jcs-2022`); bidirectional `sum attest` ↔ `sum render` symmetry from the shell |
-| Cloudflare Worker at `sum-demo.ototao.workers.dev` | shipped | `/api/render` → tome + `render_receipt`; `/.well-known/jwks.json` → JWKS; `/api/qid` → Wikidata resolver |
+| Cloudflare Worker at `sum-demo.ototao.workers.dev` | shipped | `/api/render` → tome + `render_receipt`; `/api/transform` → generic transform-registry dispatch + `sum.transform_receipt.v1`; `/api/complete` → LLM proxy; `/api/qid` → Wikidata resolver; `/.well-known/jwks.json` + `/.well-known/revoked-kids.json` → trust-loop endpoints. Public LLM-axis routes are rate-limited per IP — see [`docs/PUBLIC_API_RATE_LIMITS.md`](docs/PUBLIC_API_RATE_LIMITS.md) (5/day operator-keyed demo; 100/hr with BYO key via `X-Render-LLM-Key-Anthropic` / `-OpenAI`). |
 | Single-file browser demo (`single_file_demo/index.html`) | shipped | paste prose → in-browser attest → CanonicalBundle JSON; same bytes verify under `node standalone_verifier/verify.js` (Chrome / Firefox / Safari with WebCrypto Ed25519 support) |
 | Cross-runtime trust triangle | locked by CI (`make xruntime`) | K1 / K1-mw / K2 / K3 / K4 — Python ↔ Node ↔ Browser agree byte-for-byte on valid bundles. `make xruntime-adversarial` adds A1–A6 rejection-class equivalence. |
-| 5-axis slider rendering surface | density actioned deterministically; length / formality / audience / perspective LLM-conditioned via the Worker (Anthropic, Cloudflare AI Gateway optional) | bench: median LLM-axis fact preservation 1.000, p10 0.769 (long, n=16) / 0.818 (short, n=8), order preservation 1.000 wherever measurable |
+| 5-axis slider rendering surface | density actioned deterministically; length / formality / audience / perspective LLM-conditioned. Two dispatch paths: Worker `/api/render` (Anthropic + Cloudflare AI Gateway optional) producing `sum.render_receipt.v1`, OR Python `sum transform apply slider` (OpenAI via `OPENAI_API_KEY`) producing `sum.transform_receipt.v1` | bench: median LLM-axis fact preservation 1.000, p10 0.769 (long, n=16) / 0.818 (short, n=8), order preservation 1.000 wherever measurable. Tightening worktrail at [`docs/BENCH_HARDENING_FROM_QCVV.md`](docs/BENCH_HARDENING_FROM_QCVV.md) adds iteration-stability + DKW worst-case bounds + capability-region headlines |
 | MCP server (`sum-mcp` console script) | shipped | five tools (`extract` / `attest` / `verify` / `inspect` / `schema`) exposed over stdio; bundles attested via MCP verify byte-identically through the CLI / Node / browser verifiers |
+| Transform substrate (`sum.transform_receipt.v1` + registry) | shipped (CLI in repo HEAD; PyPI catch-up tag pending) | `sum transform list` / `sum transform apply <name>` — three registered transforms (`slider` / `extract` / `compose`); receipts via Ed25519 / JCS / detached JWS just like render-receipts; 20-fixture cross-runtime K-matrix locks accept + reject across Python ↔ Node ↔ browser; T4 `source_chain_hash` binds receipts to source byte ranges; T5 `ShareableRender` round-trips signed renders for offline verification; T6 multi-school extract runs two extractors in tandem for adversarial-divergence detection. Wire spec at [`docs/TRANSFORM_RECEIPT_FORMAT.md`](docs/TRANSFORM_RECEIPT_FORMAT.md); design at [`docs/TRANSFORM_REGISTRY.md`](docs/TRANSFORM_REGISTRY.md). |
+| Replay-defense window (`signed_at_out_of_window`) | shipped | opt-in `max_age_seconds` parameter across all four verifier surfaces (Python render / Python transform / JS render / JS transform). Default-off preserves archival use; receivers opt in per use-case (agent-swarm 60s, real-time 600s, newsletter 1d, legal-discovery no window). |
+| `sum verify --explain` layered output | shipped | Per-dimension report (`sum.verify_explained.v1`): cryptographic integrity / canonical reconstruction / axiom consistency / extraction provenance / source evidence coverage / semantic preservation / truth of content. Each carries `epistemic_status` (`provable` / `certified` / `empirical-benchmark` / `not-asserted`). Truth of content is ALWAYS `not_asserted` — locked by test. |
+| Negative-control corpus (T5 of bench-hardening) | shipped | 20 hand-authored documents across 5 failure modes (ambiguous coref / predicate-alias / contradictions / entity-resolution-adversarial / non-extractable). Runner exits 1 if observed failures don't match annotations. Baseline at [`fixtures/bench_receipts/negative_control_2026-05-17.json`](fixtures/bench_receipts/negative_control_2026-05-17.json). |
+| Compliance validators (six regimes) | shipped | `sum compliance check --regime <id> --audit-log <path>` — EU AI Act Article 12, GDPR Article 30, HIPAA § 164.312(b), ISO/IEC 27001 A.8.15, SOC 2 CC 7.2, PCI DSS v4.0 Req 10. All six produce the same `sum.compliance_report.v1` schema; per-regime docs at `docs/COMPLIANCE_*.md`. |
 
 The slider's product claim — *axis changes do not lose facts* — is the load-bearing empirical result. It is verified by NLI audit on every embedding-flagged "loss" cell; full attribution in [`docs/SLIDER_CONTRACT.md`](docs/SLIDER_CONTRACT.md).
 
+## Strategic context
+
+The operational compass — read in this order if you want the project's intent + how it operates + where it's going:
+
+- [`docs/CHARTER_2026-05-17.md`](docs/CHARTER_2026-05-17.md) — intent, the Why, strategy, objectives, success criteria, constraints, and the operational loop. The compass every other doc resolves to.
+- [`docs/PRODUCT_DELIBERATION_2026-05-14.md`](docs/PRODUCT_DELIBERATION_2026-05-14.md) — three-option strategic analysis + grant-outcome decision tree.
+- [`docs/ZENITH_FRAMING_2026-05-16.md`](docs/ZENITH_FRAMING_2026-05-16.md) — destination framing (SUM as chain-of-custody for AI-transformed knowledge) plus three new concepts (Perspective Receipts, Trust Profiles, Epistemic Nutrition Label) on the design queue.
+- [`docs/BENCH_HARDENING_FROM_QCVV.md`](docs/BENCH_HARDENING_FROM_QCVV.md) — five-task empirical-benchmark hardening plan (T1–T5; T5 shipped, T1–T4 queued).
+- [`docs/DOGFOOD_QUICKSTART.md`](docs/DOGFOOD_QUICKSTART.md) — five-minute guide to running SUM on your own writing.
+
 ### LLM narrative round-trip — closed across measured corpora (2026-04-28)
 
 The hardest measurement in `PROOF_BOUNDARY.md` is the full LLM narrative round-trip (`text → LLM-extract → axioms → LLM-generate → prose' → LLM-extract → axioms'`). The unprompted-pipeline baseline on `seed_v1` was **drift = 107.75% / exact-match recall = 0.12** — facts preserved, keys not.
@@ -104,7 +125,7 @@ The reverse direction also runs under explicit slider control. The local path ac
 sum render --density 0.5 < bundle.json
 # → keeps the lex-prefix half of the axioms; @sliders header records what was requested
 
-sum render --length 0.9 --use-worker https://sum.ototao.com --json < bundle.json
+sum render --length 0.9 --use-worker https://sum-demo.ototao.workers.dev --json < bundle.json
 # → LLM-conditioned tome + signed render_receipt (sum.render_receipt.v1) on stdout
 ```
 
@@ -131,7 +152,7 @@ pip install 'sum-engine[mcp,sieve]'
 
 ### Calling SUM over HTTP
 
-The hosted Worker at `https://sum.ototao.com` exposes `/api/render`, `/api/complete`, `/api/qid`, and the `/.well-known/{jwks,revoked-kids}.json` verification surfaces. [`docs/API_REFERENCE.md`](docs/API_REFERENCE.md) is the wire spec — request/response shapes, error codes, the six-step receipt-verification flow, working Node + Python examples. Use this when the caller is a web app, mobile app, or server-side service; use the MCP server when the caller is a local LLM client.
+The hosted Worker at `https://sum-demo.ototao.workers.dev` exposes `/api/render`, `/api/complete`, `/api/qid`, and the `/.well-known/{jwks,revoked-kids}.json` verification surfaces. [`docs/API_REFERENCE.md`](docs/API_REFERENCE.md) is the wire spec — request/response shapes, error codes, the six-step receipt-verification flow, working Node + Python examples. Use this when the caller is a web app, mobile app, or server-side service; use the MCP server when the caller is a local LLM client.
 
 ---
 
@@ -167,7 +188,29 @@ Below the slider sits the substrate that earlier phases shipped and verified. Po
 - **Bundle public-key attestation (provable).** Ed25519-signed CanonicalBundles are tamper-detectable by any third party in any of the three runtimes. [`docs/PROOF_BOUNDARY.md`](docs/PROOF_BOUNDARY.md) §1.3.1.
 - **Merkle hash-chain integrity (provable, including under concurrent writers).** [`docs/PROOF_BOUNDARY.md`](docs/PROOF_BOUNDARY.md) §1.7.
 - **Extraction F1 (empirical-benchmark).** 1.000 on `seed_v1` (50 simple-SVO docs); 0.762 with precision 1.000 on `seed_v2` (20-doc difficulty corpus). Every remaining `seed_v2` failure is a recall miss, not a truth inversion. [`docs/PROOF_BOUNDARY.md`](docs/PROOF_BOUNDARY.md) §2.1.
-- **103 numbered features**, each with a reproducible verification command, in [`docs/FEATURE_CATALOG.md`](docs/FEATURE_CATALOG.md).
+- **168 numbered features**, each with a reproducible verification command, in [`docs/FEATURE_CATALOG.md`](docs/FEATURE_CATALOG.md).
+
+### Research substrate (under `sum_engine_internal/research/`)
+
+Less-surfaced but shipped:
+
+- **MinHash-LSH bundle similarity index** (`research/lsh/`) — near-duplicate bundle detection at scale.
+- **Robust PCA corruption score** (`research/robust_pca/`) — `corruption_score` field in bundle metadata; flags adversarially-perturbed bundles.
+- **Sequential & conformal-prediction** (`research/sequential/`, `research/conformal/`) — bench-side confidence bounds with documented coverage guarantees.
+- **MMD distribution distance** (`research/mmd/`) — `axiom_distribution_mmd` field on bundles; surfaces when an attested bundle is structurally unlike its baseline corpus.
+- **Spectral entropy** (`research/spectral_entropy/`) — axiom-graph entropy on every bundle, with confidence interval.
+- **Bootstrap multiplier spike detection** (`research/bootstrap/`) — see [`docs/MULTIPLIER_BOOTSTRAP_SPIKE_FINDINGS.md`](docs/MULTIPLIER_BOOTSTRAP_SPIKE_FINDINGS.md).
+- **SMT consistency checking** (`research/smt_consistency/`) — z3-backed `axiom_consistency_check` on every bundle.
+- **Sheaf-Laplacian hallucination detection** — see [`docs/SHEAF_HALLUCINATION_DETECTOR.md`](docs/SHEAF_HALLUCINATION_DETECTOR.md) (research direction).
+
+### Other substrate-adjacent surfaces
+
+- **Trust-root manifest** (`sum_engine_internal/trust_root/`) — operator-issued signed manifest binding kid lifecycle, revocation policy, and verifier expectations.
+- **Merkle sidecar format** (`sum_engine_internal/merkle_sidecar/`) — see [`docs/MERKLE_SIDECAR_FORMAT.md`](docs/MERKLE_SIDECAR_FORMAT.md).
+- **Evidence-chain layer** (`sum_engine_internal/evidence/`) — substrate behind `source_chain_hash` (T4).
+- **Algorithm registry** — see [`docs/ALGORITHM_REGISTRY.md`](docs/ALGORITHM_REGISTRY.md) (the in-tree list of permitted signing algs; crypto-agility gate).
+- **Audit log format** — every CLI operation can emit `sum.audit_log.v1` events; see [`docs/AUDIT_LOG_FORMAT.md`](docs/AUDIT_LOG_FORMAT.md).
+- **Agent surface** (`sum_engine_internal/agent_surface/`) — see [`docs/AGENT_SURFACE_FINDINGS.md`](docs/AGENT_SURFACE_FINDINGS.md).
 
 ---
 
@@ -214,7 +257,7 @@ Source anchoring in the bundle schema, bundle explorer / viewer, `sum verify --e
 
 ```bash
 make install              # editable install with sieve + dev extras
-make test                 # full pytest run (1000+ tests)
+make test                 # full pytest run (2000+ tests)
 make xruntime             # cross-runtime K1/K1-mw/K2/K3/K4 (Python ↔ Node)
 make xruntime-adversarial # rejection-matrix A1–A6
 make fortress             # 21-check pure-math invariants

{sum_engine-0.5.0 → sum_engine-0.7.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "sum-engine"
-version = "0.5.0"
+version = "0.7.0"
 description = "SUM — bidirectional knowledge distillation with optional cryptographic attestation. Pipe prose, get a CanonicalBundle (HMAC / Ed25519 / W3C VC 2.0), verify anywhere."
 readme = "README.md"
 license = { text = "Apache-2.0" }
@@ -39,10 +39,19 @@ dependencies = [
 [project.optional-dependencies]
 # Dependency-injection extras. Users pick the extractor they want:
 #   pip install sum-engine[sieve]   # deterministic, offline spaCy path
-#   pip install sum-engine[llm]     # OpenAI structured-output path
-#   pip install sum-engine[all]     # both, plus dev tooling
+#   pip install sum-engine[openai]  # OpenAI structured-output path
+#   pip install sum-engine[llm]     # alias for [openai] (legacy name)
+#   pip install sum-engine[all]     # everything, plus dev tooling
 sieve = ["spacy>=3.7.0"]
-llm = ["openai>=1.40.0,<3.0.0", "pydantic>=2.0.0"]
+# `[openai]` is the canonical, vendor-named extra; `[llm]` is kept as a
+# back-compat alias because it predates the multi-provider dispatcher
+# (Anthropic and OpenAI now have their own named extras). Both install
+# identical dependencies — the openai SDK plus pydantic for structured
+# outputs. The vendor adapter is at
+# sum_engine_internal/ensemble/llm_dispatch.py::OpenAIAdapter; the
+# render-path TS companion is worker/src/routes/render.ts::callOpenAI.
+openai = ["openai>=1.40.0,<3.0.0", "pydantic>=2.0.0"]
+llm = ["sum-engine[openai]"]
 # Anthropic Messages-API adapter (Claude family). Installed alongside
 # [llm] for the §2.5 frontier-LLM benchmark runs that compare
 # generator-side interventions across providers. The runner picks
@@ -107,7 +116,7 @@ dev = [
 ]
 all = [
     "sum-engine[sieve]",
-    "sum-engine[llm]",
+    "sum-engine[openai]",
     "sum-engine[anthropic]",
     "sum-engine[receipt-verify]",
     "sum-engine[mcp]",

sum_engine-0.7.0/sum_cli/audit_log.py

@@ -0,0 +1,171 @@
+"""sum_cli.audit_log — universal audit-log streaming for compliance.
+
+When the ``SUM_AUDIT_LOG`` environment variable is set to a file path,
+every ``sum attest`` / ``sum verify`` / ``sum render`` operation
+appends a single JSONL row to that file describing what was done.
+
+This is the regime-agnostic foundation of the compliance primitives
+direction (Path 3). A specific regime (GDPR-Art-30, HIPAA-164.514,
+EU AI Act Annex IV, etc.) can be implemented as a downstream
+consumer of this audit log: tail the file, validate per-regime
+required fields, raise on policy violation. The audit log itself
+makes no regime-specific assumptions — it records *what happened*
+verbatim so any auditor can reconstruct the operation.
+
+Schema: ``sum.audit_log.v1`` — additive; new optional fields may
+appear in future minor versions; consumers should ignore unknown
+keys.
+
+Required fields per row:
+  - ``schema`` = ``"sum.audit_log.v1"``
+  - ``timestamp`` (ISO 8601 UTC, e.g. ``"2026-05-01T18:35:14.123Z"``)
+  - ``operation`` ∈ ``{"attest", "verify", "render"}``
+  - ``cli_version`` (the ``sum-cli`` version that produced the row)
+
+Operation-specific optional fields:
+  - attest:  ``source_uri``, ``state_integer_digits``, ``axiom_count``,
+             ``extractor``, ``signed`` (bool — Ed25519 attached?),
+             ``hmac`` (bool), ``branch``
+  - verify:  ``state_integer_digits``, ``axiom_count``, ``signatures``
+             ``{ed25519, hmac}``, ``branch``, ``ok`` (bool)
+  - render:  ``axiom_count_input``, ``mode`` (``"local-deterministic"``
+             or ``"worker"``), ``sliders``, ``render_receipt_kid`` (if
+             worker), ``worker_url`` (if worker)
+
+Failures (exit code != 0) emit a row with ``error`` field describing
+the failure class. Audit-log writes themselves never fail loudly —
+if ``SUM_AUDIT_LOG`` points at a non-writable path, the operation
+proceeds and the failure is reported on stderr only when ``--verbose``
+is set; the audit semantics fail-open by design (a non-functional
+audit destination should not break the trust loop).
+
+Env var precedence:
+  1. ``SUM_AUDIT_LOG`` env var (path to JSONL file; appended to)
+  2. unset → no audit logging
+
+The path may be ``-`` for stdout (rare; mostly useful for piping
+into another tool); otherwise treated as a file path with append-mode
+open.
+
+Concurrency: writes are O_APPEND on POSIX, so multiple sum
+processes writing to the same audit log produce a serialised
+ordering (atomic per write() up to PIPE_BUF on most systems —
+single-line JSONL records well under that bound). We write each
+row in one ``f.write()`` call to maximise atomicity.
+"""
+from __future__ import annotations
+
+import json
+import os
+import sys
+from datetime import datetime, timezone
+from typing import Any
+
+
+_AUDIT_LOG_SCHEMA = "sum.audit_log.v1"
+
+
+def _audit_log_path() -> str | None:
+    """Return the configured audit-log destination, or None if unset."""
+    p = os.environ.get("SUM_AUDIT_LOG")
+    if p is None or p == "":
+        return None
+    return p
+
+
+def _identity_fields() -> dict[str, Any]:
+    """Optional identity fields populated from env vars.
+
+    Sprint 4 of the intensification path to arXiv (PR #140). Closes
+    the PCI DSS Req 10.2.2 user-identification gap named in
+    ``docs/COMPLIANCE_PCI_DSS_4_REQ_10.md``. Three env vars are
+    consulted; each populated value becomes an optional field on
+    the audit-log row:
+
+      - ``SUM_AUDIT_USER_ID`` → ``user_id`` field. Per PCI Req 10.2.2,
+        "user identification" is the FIRST required field for each
+        audit-log event. Operators running SUM behind an
+        authenticating proxy populate this from the proxy's session
+        identity at process start.
+      - ``SUM_AUDIT_HOST_ID`` → ``host_id`` field. Multi-host
+        deployments (clusters, k8s pods, container fleets) use this
+        to attribute events to specific compute units.
+      - ``SUM_AUDIT_IP_ADDRESS`` → ``ip_address`` field. Network-
+        layer origination, useful for incident-response / forensic
+        analysis under PCI Req 10.2.2's "origination of event"
+        requirement.
+
+    All three are *optional*. An unset env var produces an absent
+    field (not a null value). The audit-log schema stays at
+    ``sum.audit_log.v1``; these are additive optional fields under
+    the existing schema's "consumers should ignore unknown keys"
+    convention. Backward compat: rows without these fields still
+    pass every existing validator.
+
+    PCI DSS validator R7 (``pci-dss-4-req-10.user-identification-
+    recommended``, added in the same PR as this function) treats
+    a missing ``user_id`` as a Req 10.2.2 violation in compliance-
+    mode runs.
+    """
+    out: dict[str, Any] = {}
+    for env_var, field in (
+        ("SUM_AUDIT_USER_ID", "user_id"),
+        ("SUM_AUDIT_HOST_ID", "host_id"),
+        ("SUM_AUDIT_IP_ADDRESS", "ip_address"),
+    ):
+        value = os.environ.get(env_var)
+        if value:  # non-empty (skips both None and empty string)
+            out[field] = value
+    return out
+
+
+def emit_audit_event(operation: str, payload: dict[str, Any]) -> None:
+    """Append a single JSONL row to the audit log if configured.
+
+    Fail-open: if the audit destination is unwritable, the operation
+    proceeds normally; the failure is silent unless verbose stderr
+    is enabled by the caller. Audit logging is *advisory* — the
+    canonical bundle / receipt still carries the load-bearing trust
+    properties; the audit log is for downstream compliance tools
+    to ingest.
+
+    Identity fields (``user_id`` / ``host_id`` / ``ip_address``) are
+    populated from env vars by :func:`_identity_fields`. They appear
+    on the row only when the corresponding env var is set; absent
+    env vars produce absent fields (not null values). The
+    ``payload`` argument takes precedence over identity fields if
+    a caller passes overlapping keys — useful for tests that want
+    to pin specific identity values without touching the
+    environment.
+    """
+    path = _audit_log_path()
+    if path is None:
+        return
+
+    from sum_cli import __version__
+
+    row = {
+        "schema": _AUDIT_LOG_SCHEMA,
+        "timestamp": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.")
+                     + f"{datetime.now(timezone.utc).microsecond // 1000:03d}Z",
+        "operation": operation,
+        "cli_version": __version__,
+        # Identity fields from env, then payload last so the caller
+        # wins on overlap (intentional for test seams).
+        **_identity_fields(),
+        **payload,
+    }
+    line = json.dumps(row, separators=(",", ":")) + "\n"
+
+    try:
+        if path == "-":
+            sys.stdout.write(line)
+            sys.stdout.flush()
+        else:
+            with open(path, "a", encoding="utf-8") as f:
+                f.write(line)
+    except OSError:
+        # Fail-open: audit destination unavailable; do not block
+        # the operation. The canonical bundle / receipt remains the
+        # load-bearing trust artifact.
+        pass