crprotocol 2.0.0__tar.gz → 2.3.0__tar.gz

crprotocol-2.3.0/.github/workflows/docs.yml

@@ -0,0 +1,32 @@
+# Copyright © 2025 Constantinos Vidiniotis. All rights reserved.
+# Licensed under Elastic License 2.0 — see LICENSE.md for details.
+
+name: Deploy Docs
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - 'site-docs/**'
+      - 'mkdocs.yml'
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.13'
+
+      - name: Install docs dependencies
+        run: pip install mkdocs-material pymdownx-extensions
+
+      - name: Build and deploy
+        run: mkdocs gh-deploy --force --strict

{crprotocol-2.0.0 → crprotocol-2.3.0}/CHANGELOG.md

@@ -12,6 +12,11 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 ## [Unreleased]
 
 ### Added
+- (none yet)
+
+## [2.2.0] - 2026-04-23
+
+### Added — Miscellaneous (from 2.1 Unreleased pool)
 - Trademark notice: "Context Relay Protocol" (application pending, Class 9)
 - Contact emails: info@crprotocol.io (general), contact@crprotocol.io (enterprise)
 - GitHub Discussions enabled
@@ -27,6 +32,133 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 - `TypeAlias` annotations and extended type aliases in `crp/_typing.py`
 - All error types and config classes exported from `crp.__init__`
 
+### Changed — Miscellaneous (from 2.1 Unreleased pool)
+- `FactGraph` edge lookups now O(1) via indexed dicts (was O(n) list scan)
+- Provider error responses sanitized — no internal details leak to clients
+- Envelope builder truncates when exceeding token budget, clamps saturation to 1.0
+- Test fixtures use `tmp_path` pytest fixture instead of `tempfile.TemporaryDirectory()`
+- Correlation IDs propagated through ingest path
+
+### Fixed — Miscellaneous (from 2.1 Unreleased pool)
+- Unknown config kwargs now produce a warning log instead of silent ignore
+
+### Added — Enforcement Pipeline, Ledger & Injection Detection (§7.14.4–§7.14.5)
+
+CRP 2.1 defined the **vocabulary** for context-source provenance
+(`ContextSource`, `ContextManifest`). CRP 2.2 defines the **enforcement
+point** — the single wire-side choke-point every envelope assembly flows
+through — plus cross-turn persistence, key rotation, and content-side
+verification of declared trust labels. This closes the gap between
+"context is labelled" and "context is actually enforced on the wire".
+
+- New module `crp.core.context_enforcer` (§7.14.4):
+  - `EnforcementPolicy` (`OBSERVE` / `WARN` / `REJECT`) — application-
+    level policy for what happens on a violation.
+  - `ContextEnforcer` — composes manifest signature-verify, expiry
+    check, attestation mismatch scan (`check_attestation`), and
+    injection-signal scan into a single `check(manifest, observed)`
+    call. Under `REJECT`, raises `CRPError(CONTEXT_ATTESTATION_MISMATCH)`
+    or `CRPError(CONTEXT_MANIFEST_INVALID)`. Under `OBSERVE` / `WARN`,
+    emits audit events and continues.
+  - `detect_injection_signals()` — six high-precision regex patterns
+    (instruction override, role jailbreak, secret exfil, delimiter
+    forgery, dangerous payload URLs, embedded tool calls) applied to
+    sources declared `TrustLevel.TRUSTED`. Addresses the gap between
+    *declared* and *actual* trust — developers marking a system prompt
+    as TRUSTED doesn't make templated user input safe.
+  - `AuditSink` protocol + `LoggingAuditSink` + thread-safe
+    `InMemoryAuditSink` (bounded ring buffer). Every mismatch and
+    injection signal is emitted as a structured event for SIEM / audit
+    pipelines.
+  - `default_enforcer()` / `set_default_enforcer()` — process-wide
+    opt-in installer so libraries (dispatch router, CKF, warm store)
+    can find a configured enforcer without explicit plumbing.
+  - `observed_content(source, text)` helper for presenting source +
+    payload pairs to the enforcer for content scanning.
+
+- New module `crp.core.manifest_ledger` (§7.14.5):
+  - `ManifestLedger` — append-only JSONL store
+    (`crp_sessions/<session_id>.manifest.jsonl`) of every manifest ever
+    attached to a session. Supports `record()`, `load()`, `history()`,
+    `latest()`, `find_by_source_id()`, `find_by_kind()`, and
+    `verify_signatures()` for periodic integrity audits.
+  - `KeyProvider` abstraction with two implementations:
+    - `EnvVarKeyProvider` — reads HMAC secret from an environment
+      variable, auto-detects hex encoding, enforces ≥32-byte minimum.
+    - `RotatingKeyProvider` — current + retired key ring with
+      configurable grace window, allowing in-flight manifests signed
+      under the previous key to continue verifying after rotation.
+
+### Added — Consumer-Side Integration (§7.14.4 wire point)
+
+- `crp.core.dispatch_router.assemble_messages()` now accepts optional
+  `manifest`, `observed_sources`, and `enforcer` keyword arguments.
+  When supplied (or a process-wide default enforcer is installed),
+  every envelope assembly runs the full enforcement pipeline
+  **before** messages are constructed. Zero-cost when no enforcer is
+  configured — fully backward compatible with CRP 2.1 call sites.
+
+- `crp.state.warm_store.get_active_facts_as_extraction()` now stamps
+  every un-sourced fact with
+  `ContextSource(kind=WARM_STORE, origin=OBSERVED, trust_level=TRUSTED)`.
+
+- `crp.ckf.fabric.ContextKnowledgeFabric.retrieve()` now stamps every
+  un-sourced merged fact with `ContextSource(kind=CKF_RETRIEVAL, ...)`
+  carrying the retrieval modes and score in `metadata`.
+
+### Changed — Source-Kind Detection
+
+- `crp.core.context_source.detect_source_kind()` strengthened:
+  - New structural hint parser: attempts `json.loads(content)` and
+    inspects object keys (`function_name` + `arguments` → `FUNCTION_CALL`,
+    `mcp` / `mcp_server` → `MCP_TOOL`, `url` + `snippet` →
+    `WEB_SEARCH`, `embedding` / `vector` → `VECTOR_DB`).
+  - Six new heuristic regex patterns for JSON/XML/YAML code blocks,
+    SQL DML, Cypher / graph query keywords, SERPAPI / Google / Bing
+    signatures, common vector-DB provider names (Pinecone, Weaviate,
+    Qdrant, Chroma, pgvector, Milvus), and `function_name` / `tool_name`
+    tokens.
+  - Structural hints are preferred over regex when parsing succeeds.
+
+### Security
+
+- Manifest verification is now always attempted when both a signature
+  and a secret are present, even if `require_signed_manifest=False`.
+  Tampered signatures are detected and emitted as
+  `CONTEXT_MANIFEST_INVALID` audit events.
+- `ManifestLedger` rejects session IDs that sanitize to empty
+  (prevents directory traversal via crafted session identifiers).
+- HMAC verification continues to use `hmac.compare_digest` (constant
+  time). `RotatingKeyProvider` iterates candidate keys for verification
+  so key rotation does not create a signature-oracle side channel.
+
+### Tests
+
+- 38 new tests in `tests/test_context_enforcer.py` (injection patterns,
+  audit sinks, all three policies, default-enforcer install, integration
+  with `assemble_messages`).
+- 34 new tests in `tests/test_manifest_ledger.py` (JSONL round-trip,
+  cross-instance rehydration, lineage queries, key rotation, hex/utf-8
+  env secret decode, grace-window verification).
+- Total suite: **182 passing** (110 pre-2.2 baseline + 72 new).
+
+## [2.1.0] - 2026-04-23
+
+### Added — Context-Source Provenance (§7.14.3)
+- Contact emails: info@crprotocol.io (general), contact@crprotocol.io (enterprise)
+- GitHub Discussions enabled
+- Operational runbook (`docs/OPERATIONS_RUNBOOK.md`) — deployment, monitoring, incident response
+- Integrity chain external verification (`export_for_verification()`, `verify_external()`)
+- `/metrics` endpoint on HTTP sidecar (Prometheus format)
+- Session file schema versioning with forward-compatible migration path
+- Bounded ingest queue (`maxsize=1000`) prevents unbounded memory growth
+- Pipeline short-circuit — skips late extraction stages when early stages produce enough facts
+- Config file layer — YAML/JSON config files (`~/.crp/config.yaml`, `.crp.yaml`)
+- Structured logging wired into orchestrator initialization
+- Metadata size limits on `Fact` (64 keys, 128-char key length, 4096-char value size)
+- `TypeAlias` annotations and extended type aliases in `crp/_typing.py`
+- All error types and config classes exported from `crp.__init__`
+
 ### Changed
 - `FactGraph` edge lookups now O(1) via indexed dicts (was O(n) list scan)
 - Provider error responses sanitized — no internal details leak to clients
@@ -37,6 +169,84 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 ### Fixed
 - Unknown config kwargs now produce a warning log instead of silent ignore
 
+## [2.1.0] - 2026-04-23
+
+### Added — Context-Source Provenance (§7.14.3)
+
+CRP's Decision Provenance Engine already classifies every *output* claim as
+`CONTEXT_GROUNDED | PARAMETRIC | MIXED | UNCERTAIN`. This release adds the
+symmetric **input-side** primitive: every fact that enters the envelope
+can now carry a record of *where it came from* (RAG chunk, vector DB,
+database read, MCP tool, function call, web search, user turn, file upload,
+agent memory, or parametric). This is foundational for ISO/IEC 42001 §4
+(Context of the organisation), EU AI Act Art. 10 (Data governance),
+GDPR Art. 30 (Records of Processing), and NIST AI RMF MAP-4.
+
+- New module `crp.core.context_source` with:
+  - `SourceKind` — closed enumeration (14 values) of upstream source
+    categories. Additions require an RFC.
+  - `ContextSource` — immutable (`frozen=True`) record: `kind`, `source_id`,
+    `origin` (declared / observed / heuristic), `trust_level`,
+    `contains_pii`, `sensitivity`, `region`, `retrieval_query`,
+    `retrieved_at`, `upstream_uri`, `declared_by_manifest_id`, `metadata`.
+  - `ContextManifest` — customer-authored declarative attestation of
+    intended upstream sources. HMAC-SHA256 signed over canonical JSON,
+    with `sign()` / `verify()` using constant-time comparison and
+    `is_expired()` helpers.
+  - `detect_source_kind(content, role=…)` — detective-mode heuristic
+    parser that classifies message content by OpenAI-style role plus a
+    conservative pattern library (`<RAG>`, `[retrieved]`, `<mcp:>`,
+    `SELECT … FROM …`, web-search markers, etc.).
+  - `check_attestation(observed, manifest)` — returns a list of
+    `AttestationMismatch` rows (reasons: `no_manifest`,
+    `manifest_expired`, `unattested_kind`, `unattested_source_id`).
+    `to_audit_event()` produces the §7.14.2 audit-event envelope shape.
+- `Fact.source: ContextSource | None` — optional field; defaults to
+  `None` so v2.0 callers see zero behavioural change.
+- Envelope section `[CONTEXT_SOURCES]` registered in the Tier 3 priority
+  list (`crp/envelope/formatter.py::TIER_3_SECTIONS`).
+- Error codes `CONTEXT_ATTESTATION_MISMATCH = 1040` and
+  `CONTEXT_MANIFEST_INVALID = 1041`.
+- `ManifestValidationError` raised on malformed JSON / empty signing key.
+- All primitives exported from `crp` (`SourceKind`, `SourceOrigin`,
+  `TrustLevel`, `ContextSource`, `ContextManifest`, `AttestationMismatch`,
+  `ManifestValidationError`, `detect_source_kind`, `check_attestation`).
+- 41-test suite (`tests/test_context_source.py`) covering frozen
+  invariant, size limits, JSON round-trip, HMAC sign / verify, expiry,
+  detective-mode classification, attestation-mismatch edge cases, audit
+  event shape, and public API surface.
+
+### Changed
+
+- `docs/CRP_CAPABILITIES.md` now lists context-source provenance as a
+  first-class capability.
+- MkDocs site gains a dedicated *Protocol → Context Sources* page.
+
+### Migration notes
+
+This release is **fully backward-compatible**. `Fact.source` defaults to
+`None`; consumers that ignore the field continue to work unchanged. To
+adopt, wrap retrieved chunks with `ContextSource` at ingestion time:
+
+```python
+from crp import ContextSource, SourceKind, SourceOrigin, Fact
+
+fact = Fact(
+    text=chunk.text,
+    source=ContextSource(
+        kind=SourceKind.VECTOR_DB,
+        source_id="acme-hr-policies-vdb",
+        origin=SourceOrigin.OBSERVED,
+        contains_pii=True,
+        region="eu-west-1",
+        retrieval_query=user_query,
+    ),
+)
+```
+
+See `docs/protocol/context-sources.md` (published on crprotocol.io) for
+the full integration guide.
+
 ## [2.0.0] - 2026-04-06
 
 ### Added

crprotocol-2.3.0/HOSTING_POSITIONING.md

@@ -0,0 +1,140 @@
+# CRP Hosting & Positioning Strategy
+
+## Positioning: "Local-First, Cloud-Ready"
+
+CRP products are available as **hosted SaaS applications** and as **locally deployable software**. The SaaS offering provides instant onboarding via product subdomains, while the local deployment option targets enterprises and regulated environments that require on-premises control — this is AutoCyber AI's go-to for enterprise deployments.
+
+### Deployment Models
+
+| Model | Description | Target |
+|---|---|---|
+| **SaaS (Hosted)** | Sign up at product subdomains, Stripe billing | Startups, SMBs, developers |
+| **Local (Downloadable)** | Self-hosted on-premises or private cloud | Enterprises, regulated industries, government |
+
+The local deployment uses the same codebase — packaged as a downloadable application (future: Tauri desktop app) that runs entirely on the customer's infrastructure. Data never leaves their environment.
+
+### Why SaaS, Not Packages
+
+| Consideration | Package (pip install) | SaaS (hosted) |
+|---|---|---|
+| **Revenue** | No billing control | Stripe subscription billing |
+| **Updates** | User must upgrade manually | Instant, zero-downtime deploys |
+| **Security patches** | User may lag behind | Applied immediately |
+| **Compliance audit trail** | Stored locally (risky) | HMAC-chained, server-side |
+| **PII scanning** | User's responsibility | CRP-managed, always current |
+| **Onboarding** | pip install + config | Sign up → start |
+| **Support** | Debug user's environment | Controlled environment |
+| **Telemetry** | Opt-in only | Built-in usage metrics |
+
+### Product URLs
+
+| Product | URL | Status |
+|---|---|---|
+| CRP Scribe | `https://scribe.crprotocol.io` | Planned |
+| CRP Comply | `https://comply.crprotocol.io` | Planned |
+| Marketing site | `https://crprotocol.io` | **LIVE** |
+| Documentation | `https://crprotocol.io/docs/` | **LIVE** |
+
+---
+
+## Hosting: Railway
+
+### Why Railway
+
+| Factor | Railway | Vercel | AWS/GCP |
+|---|---|---|---|
+| **FastAPI support** | Native (Docker) | Serverless only | Manual setup |
+| **WebSocket support** | ✓ | Limited | ✓ but complex |
+| **Background workers** | ✓ | ✗ | ✓ |
+| **Persistent storage** | PostgreSQL, Redis | External only | Managed services |
+| **Custom domains** | ✓ (free SSL) | ✓ | Route53 + ACM |
+| **Pricing model** | Usage-based ($5/mo hobby) | Per-seat | Pay-as-you-go |
+| **Deploy from GitHub** | ✓ (auto-deploy) | ✓ | CodePipeline |
+| **Startup friendly** | ✓ | ✓ | ✗ (complex) |
+
+### Railway Service Architecture
+
+```
+railway.app
+├── crp-scribe-api       (FastAPI, port 8500)
+│   ├── Dockerfile
+│   ├── PostgreSQL (scribe-db)
+│   └── Redis (scribe-cache)
+│
+├── crp-scribe-web       (Vite static build)
+│   └── served via nginx or Railway static hosting
+│
+├── crp-comply-api       (FastAPI, port 8400)
+│   ├── Dockerfile
+│   ├── PostgreSQL (comply-db)
+│   └── Redis (comply-cache)
+│
+└── crp-comply-web       (Vite static build)
+    └── served via nginx or Railway static hosting
+```
+
+### Deployment Tiers
+
+| Environment | Railway Plan | Cost | Purpose |
+|---|---|---|---|
+| Development | Hobby ($5/mo) | ~$5-10/mo | Dev/test |
+| Staging | Pro ($20/mo) | ~$20-40/mo | Pre-production |
+| Production | Pro ($20/mo) | ~$50-200/mo | Live customers |
+
+---
+
+## Security: "The Security Is In The Protocol"
+
+CRP products don't need to run in a locked-down VPC because security is embedded at the protocol layer:
+
+| CRP Security Module | What It Does | Where It Runs |
+|---|---|---|
+| `PIIScanner` | 7-category PII detection | In every proxy request |
+| `InjectionDetector` | 21 patterns + ML scoring | In every proxy request |
+| `ComplianceAuditTrail` | HMAC-chained audit records | Server-side, append-only |
+| `ProcessingRecordKeeper` | GDPR Art. 30 records | Server-side |
+| `ErasureManager` | GDPR Art. 17 right to erasure | Server-side |
+| `DataClassification` | PUBLIC → CRITICAL classification | Per-request |
+| `RiskClassifier` | EU AI Act Art. 6 risk levels | Per-request |
+| `ConsentManager` | Purpose-based consent tracking | Server-side |
+| `ContextValidator` | Schema + semantic validation | In every request |
+| `EncryptionManager` | AES-256-GCM field encryption | At rest + in transit |
+| `AccessControl` | RBAC + ABAC policies | Per-request |
+| `EnvelopeProtocol` | Signed, versioned message envelopes | Every message |
+
+This means Railway's standard infrastructure (HTTPS, network isolation, managed PostgreSQL with encryption at rest) is **sufficient** — the CRP layer handles application-level security.
+
+---
+
+## Positioning Matrix
+
+### For Investors
+
+> "CRP is a protocol-first company. Our SaaS products (Scribe, Comply) are hosted applications with Stripe subscription billing. Security is built into the protocol itself — we don't need expensive cloud infrastructure to be enterprise-grade."
+
+### For Enterprise Customers
+
+> "CRP Comply runs as a managed service or a locally deployed application — your choice. In SaaS mode, your AI traffic flows through our compliance proxy with PII scanning and audit trails. In local mode, the same software runs entirely on your infrastructure — data never leaves your environment."
+
+### For Regulators
+
+> "All processing records, audit trails, and consent records are maintained with HMAC-chain verification. GDPR Article 30 records and right-to-erasure requests are handled through documented API endpoints with full audit history. For regulated environments, CRP products can be deployed on-premises."
+
+### For Developers
+
+> "Sign up at comply.crprotocol.io or scribe.crprotocol.io. Get an API key. Point your OpenAI calls at our proxy endpoint. Done. The CRP protocol library (`pip install crprotocol`) is available separately for developers who want to build their own CRP-compatible tools."
+
+---
+
+## Key Distinction
+
+| | CRP Protocol (`crprotocol`) | CRP Products (SaaS) | CRP Products (Local) |
+|---|---|---|---|
+| **Distribution** | PyPI (`pip install crprotocol`) | Hosted SaaS | Downloadable app / Docker |
+| **License** | ELv2 | ELv2 | ELv2 (commercial license) |
+| **Pricing** | Free (open-source core) | Freemium + Stripe billing | Enterprise license |
+| **Target** | Developers building CRP tools | Startups / SMBs | Enterprises / regulated |
+| **Auth** | N/A | Clerk | Clerk or customer SSO |
+| **Billing** | N/A | Stripe | Invoice / PO |
+| **Data residency** | N/A | Railway (AU/US/EU) | Customer's infrastructure |
+| **URL** | pypi.org/project/crprotocol | scribe/comply.crprotocol.io | On-premises |

{crprotocol-2.0.0 → crprotocol-2.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: crprotocol
-Version: 2.0.0
+Version: 2.3.0
 Summary: Context Relay Protocol — unbounded context, unbounded generation, amplified reasoning for LLMs
 Project-URL: Homepage, https://crprotocol.io
 Project-URL: Documentation, https://crprotocol.io
@@ -73,7 +73,7 @@ Description-Content-Type: text/markdown
 <p align="center">
   <a href="LICENSE.md"><img src="https://img.shields.io/badge/Spec-CC_BY--SA_4.0-blue.svg" alt="Spec: CC BY-SA 4.0"></a>
   <a href="LICENSE.md"><img src="https://img.shields.io/badge/Code-ELv2-orange.svg" alt="Code: Elastic License 2.0"></a>
-  <img src="https://img.shields.io/badge/Spec_Version-2.0.0-brightgreen.svg" alt="Spec Version: 2.0.0">
+  <img src="https://img.shields.io/badge/Spec_Version-2.2.0-brightgreen.svg" alt="Spec Version: 2.2.0">
   <img src="https://img.shields.io/badge/RFC_2119-Conformant-orange.svg" alt="RFC 2119">
   <img src="https://img.shields.io/badge/Language_Neutral-JSON_Schema-yellow.svg" alt="Language Neutral">
   <img src="https://img.shields.io/badge/Status-Specification_Complete-green.svg" alt="Status: Specification Complete">
@@ -190,6 +190,7 @@ For every LLM call you already make, CRP:
 - **Works with any LLM provider** — auto-detected, 3 fields to configure. Built-in adapters for OpenAI, Anthropic, Ollama, and llama.cpp — plus `CustomProvider` to wrap any LLM in 3 lines
 - **Structured knowledge extraction** — 6-stage graduated pipeline (regex → statistical NLP → GLiNER NER → UIE relations → RST discourse → LLM-assisted relational). Not just text chunking
 - **Contextual Knowledge Fabric (CKF)** — graph-structured knowledge with 4-mode retrieval (graph walk + pattern query + semantic fallback + community summaries), event-sourced history, and cross-session persistence
+- **Two-sided provenance** — CRP classifies every model *output* as `CONTEXT_GROUNDED | PARAMETRIC | MIXED | UNCERTAIN` **and** records every *input* fact's upstream source (RAG chunk, vector DB, MCP tool, function call, web search, user turn, file upload, agent memory, or parametric). `ContextManifest` lets you sign a declaration of intended sources; anything observed outside the declaration is flagged as `CONTEXT_ATTESTATION_MISMATCH` in the audit log. Foundational for **ISO/IEC 42001 §4**, **EU AI Act Art. 10**, and **GDPR Art. 30**
 - **Unbounded input** — automatically ingests documents larger than any model's context window through structure-aware chunking with protected spans
 - **Unbounded output** — automatic continuation with voice profile preservation, document maps, degradation-triggered re-grounding, and content-type-aware stitching
 - **Honest quality guarantees** — a degradation model, not magic claims. Quality tiers S through D, reported with every dispatch. Extraction recall percentages published per stage