crprotocol 4.3.2__tar.gz → 5.0.0__tar.gz

{crprotocol-4.3.2 → crprotocol-5.0.0}/AGENTS.md

@@ -8,7 +8,7 @@
 
 ## Project Overview
 
-**Context Relay Protocol (CRP)** is an open Python SDK and protocol specification for structured context management across LLM invocations. It acts as a middleware layer that wraps existing LLM calls to provide:
+**Context Relay Protocol (CRP) for Agentic AI** is an open Python SDK and protocol specification for agentic context and tool orchestration across LLM invocations. It acts as a middleware layer that wraps existing LLM calls and structures the surrounding agentic ecosystem to provide:
 
 - **Unbounded context** — auto-ingests documents larger than any model's context window
 - **Unbounded generation** — automatic continuation when output hits physical limits
@@ -18,7 +18,7 @@
 **Package name on PyPI:** `crprotocol`  
 **License:** Elastic License 2.0 (code) / CC BY-SA 4.0 (specification documents)
 
-The core value proposition: instead of one shared context window where everything competes for token budget, CRP gives each LLM call its own dedicated window with a curated "envelope" of relevant historical facts extracted from prior windows.
+The core value proposition: instead of one shared context window where everything competes for token budget, CRP gives each LLM call its own dedicated window with a curated "envelope" of relevant historical facts, selected tools, and operation goals extracted from prior windows.
 
 **v4 adds:** the quality layer (CDR/CDGR/CSO), the positioning layer (STL), and the product revenue loop (Gateway visual console, Comply upgrade, Scan remediation + payments).
 
@@ -245,7 +245,7 @@ The core value proposition: instead of one shared context window where everythin
 
 ---
 
-## Complete Spec Index (v4 — 48 Specifications)
+## Complete Spec Index (v4 — 49 Specifications)
 
 All v4 specs are in `SPECS_5_06_2026_CRP_v4/specs/` (main repo). v3 specs are in `crprotocol-specs/` (separate repo).
 
@@ -299,9 +299,11 @@ All v4 specs are in `SPECS_5_06_2026_CRP_v4/specs/` (main repo). v3 specs are in
 | SPEC-046 | User-Defined Cognition | Frontier | **NEW v4** | Deferred (R5) |
 | SPEC-047 | Monetisation & Payments | Revenue — Tier 3 | **NEW v4** | B (R3) |
 | SPEC-048 | Comply Low-Code + GitHub App | Product — Tier 3 | **NEW v4** | B (R3) |
+| SPEC-049 | SLM Agent Execution Profile | Cross-cutting — Agentic Execution | **NEW v4.1** | B (R3+) |
+| SPEC-050 | Tool Capability Fabric & Operation Orchestration | Cross-cutting — Agentic Execution | **NEW v4.1** | B (R3+) |
 
 **Additional v4 documents in `SPECS_5_06_2026_CRP_v4/`:**
-n- `CRP-BUILD-PROMPT.md` — Build instructions
+- `CRP-BUILD-PROMPT.md` — Build instructions
 - `CRP-CODER-BRIEF.md` — Coder brief with gap analysis
 - `CRP-ECOSYSTEM-BOUNDARIES.md` — Ecosystem boundaries
 - `CRP-GATEWAY-BLUEPRINT.md` — 22-step Gateway lifecycle blueprint
@@ -351,6 +353,8 @@ crp/
 │   ├── depth_model.py
 │   ├── frame_builder.py
 │   ├── goal_compass.py
+│   ├── operation_state.py  ← NEW (SPEC-050) — operation state machine
+│   ├── tool_positioner.py  ← NEW (SPEC-050) — build Tool Positioning Frame
 │   └── orchestrator.py
 ├── security/
 │   ├── control_plane.py    ← NEW (SPEC-033) [Agent-B]
@@ -360,7 +364,13 @@ crp/
 ├── gateway/
 │   ├── api.py              ← EXTEND (22-step lifecycle) [Agent-A]
 │   ├── router.py           ← NEW (SPEC-016) [Agent-A]
-│   └── key_vault.py        ← NEW (SPEC-016) [Agent-A]
+│   ├── key_vault.py        ← NEW (SPEC-016) [Agent-A]
+│   └── capability_router.py ← NEW (SPEC-049/050) — capability-profile mapper & structured-output translation [Agent-B]
+├── tools/                  ← NEW (SPEC-050) — Tool Capability Fabric
+│   ├── __init__.py
+│   ├── capability_fabric.py # TCF registry and retrieval
+│   ├── descriptor.py        # CapabilityDescriptor + validation
+│   └── executor.py          # tool-call execution and observation storage
 ├── scan/
 │   ├── semantic_ingestion.py ← NEW (SPEC-039) [Agent-A]
 │   ├── remediation.py      ← NEW (SPEC-036) [Agent-A]

{crprotocol-4.3.2 → crprotocol-5.0.0}/CHANGELOG.md

@@ -11,6 +11,59 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ## [Unreleased]
 
+## [5.0.0] - 2026-06-30
+
+CRP v5 turns the protocol from an inference-governance layer into a full
+**agentic execution** layer for small/local language models. The doctrine is
+*positioning, not injection*: the model is placed on one isolated operation at a
+time with only the 1–3 tools that operation needs — it never sees the tool
+catalogue — and a durable Cognitive State Object carries findings forward so
+hundreds of tool calls compose into one coherent answer without flooding the
+context window. Validated end-to-end on `meta-llama-3.1-8b-instruct` (LM Studio).
+
+### Added
+- **Tool Capability Fabric (TCF)** — `crp/tools/` (CRP-SPEC-050). Capability
+  descriptors, registry + semantic/policy-filtered retrieval, executor, and a
+  `ToolResultExtractor` that turns raw tool output into typed observations.
+  `fabric_from_callables()` auto-registers `@client.tool` functions.
+- **Semantic Task Layer positioned loop** — `crp/stl/positioned.py`
+  (`run_positioned`). The live STL spine that classifies a request into the
+  canonical 10-operation taxonomy (`RETRIEVE, ANALYSE, COMPARE, SYNTHESISE,
+  GENERATE, VERIFY, TRANSFORM, PLAN, CLARIFY, REVISE`), positions the model on
+  one operation at a time, and assembles a coherent final response.
+- **Operation State Machine** — `crp/stl/operation_state.py` + Tool Positioner
+  (`crp/stl/tool_positioner.py`). Emits an HMAC-chained, streamable visibility
+  event stream (`intent_classified → operation_positioned → tool_selected →
+  tool_executed → observation_stored → operation_verified → integrated →
+  complete`) — the same mechanism powers live UI and audit.
+- **CSO observation store** — `crp/state/cso.py` stores each tool result as a
+  typed `EstablishedFact(provenance=TOOL)`, decoupled from the prompt window, so
+  the working set stays bounded (O(N), not O(N²)) independent of session length.
+- **Resource Governor** — `crp/resources/governor.py`. Proactive,
+  device-tier-aware capping of frame size, tool concurrency, continuation and
+  depth. Prime directive: *slow and steady over fast and failing* — it throttles
+  and serialises rather than risking OOM; it never refuses a call.
+- **CLARIFY + Checkpoint consolidation** — `crp/security/clarify.py`. CLARIFY is
+  a first-class operation that raises a checkpoint when the model lacks
+  information or authority, gating destructive/mutating tools (Invariant 10
+  graceful fallback retained).
+- **SDK surface** — `CRPClient.dispatch_positioned()` runs the positioned loop
+  with the client's provider; returns a `PositionedResult` exposing `.text`,
+  `.cso`, `.event_stream`, `.observation_count`, `.headers`, `.halted`.
+- **Agent execution headers** — `CRP-Agent-Execution-Mode: positioned-tool-loop`,
+  `CRP-Agent-Operation-State/Plan/Type`, `CRP-Tool-Observation-Count`,
+  `CRP-STL-Frame-Tokens`, `CRP-Agent-Capability-Profile` (CRP-SPEC-049).
+- **Specifications** — CRP-SPEC-049 (SLM Agent Execution Profile) and
+  CRP-SPEC-050 (Tool Capability Fabric) promoted from Draft to Stable.
+- **Demo** — `examples/crp_demos/positioned_8b_validation.py` validates the live
+  loop on a local SLM and prints the event stream, CSO observations, bounded
+  window metrics and header surface.
+
+### Notes
+- No `CRP-*` header ever reaches the LLM provider (Axiom 4 preserved).
+- Frontier models remain explicit providers; there is no silent auto-routing
+  (D4).
+
 ## [4.3.2] - 2026-06-26
 
 ### Added

crprotocol-5.0.0/CRP_SLM_PROTOCOL_PROPOSAL.md

@@ -0,0 +1,292 @@
+# CRP Agentic AI Protocol — SLM Execution Reframe
+
+> **Status:** revised proposal — aligned with CRP-SPEC-049 (SLM Agent Execution Profile) and CRP-SPEC-050 (Tool Capability Fabric)  
+> **Target:** CRP v4.1+ and `crp-comply` v0.2.x  
+> **Companion research:** [`crp-comply/SLM_INTEGRATION_RESEARCH.md`](https://github.com/AutoCyber-AI/crp-comply/blob/master/SLM_INTEGRATION_RESEARCH.md) and [`crp-comply/LOCAL_8B_MODEL_ANALYSIS.md`](https://github.com/AutoCyber-AI/crp-comply/blob/master/LOCAL_8B_MODEL_ANALYSIS.md)
+
+---
+
+## 1. TL;DR
+
+The current CRP Comply agent loop fails on 8 B local models because it was designed as if the model should be the planner, tool selector, and executor at the same time. The fix is **not** to limit the model or to fall back to a frontier model. The fix is to make CRP the **agentic AI protocol** — the protocol layer that reasons *around* the model so the model only has to generate the next focused output.
+
+The concrete additions are:
+
+1. **Tool Capability Fabric (TCF)** — a protocol-level registry and retrieval system that selects the 1–3 capabilities the current operation actually needs (CRP-SPEC-050).
+2. **Operation State Machine** — explicit tracking of where the agent is (`OPERATION_POSITIONED`, `TOOL_SELECTED`, `TOOL_EXECUTED`, `OPERATION_VERIFIED`, etc.) so the protocol, not the model, manages progress (CRP-SPEC-050).
+3. **Tool Positioning Frame** — the minimal set of tools and schemas presented to the model for one STL operation, keeping tool selection out of the prompt (CRP-SPEC-050).
+4. **Capability advertisement** — `CRP-Agent-Capability-Profile` (`frontier` / `capable-local` / `small-local`) and `CRP-Agent-Model-Capabilities` tell the protocol how much scaffolding to provide, not what the model is forbidden from doing (CRP-SPEC-049).
+5. **Structured-decoding directives** — CRP carries JSON-schema / grammar hints that the gateway translates into provider-specific constrained decoding (CRP-SPEC-049).
+6. **Prompt-cache lifecycle hints** — CRP tells the inference server which prefix must stay byte-stable so llama.cpp / vLLM / SGLang can reuse KV state across turns (CRP-SPEC-049).
+7. **Preventive safety** — the safety gate inspects the operation frame and tool parameters *before* generation/execution, and returns the problematic frame if it must halt (CRP-SPEC-050).
+8. **No hybrid auto-routing** — the assigned model does the work. Frontier models are explicit providers, not silent fallbacks (CRP-SPEC-049 §4.9).
+
+These changes are formalised as **CRP-SPEC-049** and **CRP-SPEC-050**.
+
+---
+
+## 2. Reframing: CRP as the agentic AI protocol
+
+CRP's founding insight is **positioning, not injection**: instead of dumping everything into one context window, the protocol places the LLM on a focused task with a curated envelope. The SLM problem is the extreme version of the same insight: a small model needs *more* positioning, not less.
+
+> **The smaller the model, the more the surrounding protocol must do.**
+
+CRP is therefore repositioned as:
+
+> **CRP — Context Relay Protocol for Agentic AI**  
+> *The protocol that places every LLM on exactly the right task, with exactly the right context and tools, at exactly the right time — so even small local models operate as part of an extremely capable agent ecosystem.*
+
+The competitive map becomes clean:
+
+- **MCP** exposes tools.
+- **A2A** connects agents.
+- **CRP** positions every agent on the right task, context, and tools — the missing layer that makes MCP and A2A work at scale, especially for SLMs.
+
+This is a reframe, not a full rebrand. The CRP acronym, package name (`crprotocol`), and code base are preserved.
+
+---
+
+## 3. Why SLMs can outperform their weight class
+
+Recent work consistently shows that ecosystem factors let small models punch above their parameter count:
+
+- **TinyAgent** (Erdogan et al., EMNLP 2024): a 1.1 B / 7 B fine-tuned model with tool retrieval and constrained decoding surpasses GPT-4-Turbo on a narrow tool space.
+- **Gorilla** (Patil et al., NeurIPS 2024): a 7 B retrieval-aware model reduces API hallucinations from 78 % to 11 % by retrieving the right API docs at inference time.
+- **ToolLLM / ToolChain\*** (Qin et al., 2023; Zhu et al.): depth-first search decision trees and A\* planning over tool-action spaces let smaller models solve multi-step tool-use problems.
+- **Tool-to-Agent Retrieval** (arXiv 2511.01854): joint retrieval over tools and parent agents in a shared vector space outperforms both tool-only and agent-only retrieval.
+- **RATS / Dynamic Tool Allocation**: separating "find the capability" from "execute the task" is now an established production pattern.
+- **XGrammar-2**: raises Llama-3.2-3B correct-call rates from ~33 % to ~78 % and schema validity to 100 % when structured decoding is enforced.
+
+The unifying pattern: **tool selection and task sequencing belong outside the model**. CRP already has the primitives to do this (STL, CSO, CKF, CDR, DPE, safety budget, audit chain). What is missing is the **tool orchestration layer** that ties them together.
+
+---
+
+## 4. The tool-injection problem
+
+Current agent frameworks (OpenAI function calling, LangChain tools, MCP) inject every tool schema into the system prompt and ask the model to choose. This is the injection pattern CRP was designed to avoid:
+
+- Tool schemas consume 10 K–50 K tokens.
+- The model must reason about tool semantics *and* task semantics simultaneously.
+- Tool outputs are dumped back into the same context window, contaminating later reasoning.
+- There is no protocol-level record of *why* a tool was selected at that point.
+
+CRP's alternative is **protocol-level tool orchestration**:
+
+1. The STL classifies the current cognitive operation (`RETRIEVE`, `ANALYSE`, `COMPARE`, ...).
+2. The TCF returns capabilities that serve that operation type.
+3. A policy filter removes disallowed capabilities.
+4. A dependency resolver builds the minimal execution DAG.
+5. The protocol composes a **Tool Positioning Frame** with 1–3 capabilities.
+6. The model emits the tool call with constrained decoding.
+7. The orchestrator executes the call, validates the result, and stores the observation in the CSO as a typed fact.
+
+The model never sees the full tool catalogue. It sees only the tools the protocol has determined are relevant to *this* operation.
+
+---
+
+## 5. The Tool Capability Fabric (TCF)
+
+The TCF is a protocol-level registry of every tool, MCP server capability, sub-agent, and callable primitive available to a session. Each entry is a **Capability Descriptor** with:
+
+- `capability_id`, `kind`, `version`
+- `operation_types` — which STL operations it serves
+- `input_schema` / `output_schema`
+- `produces_facts`, `consumes_facts`
+- `dependencies`, `mutually_exclusive_with`
+- `cost_profile` — tokens, latency, safety class
+- `data_residency`, `allowed_policy_domains`
+- `embedding_vector`, `embedding_model`
+- `metadata`
+
+The canonical JSON Schema is in `schemas/capability-descriptor.json`.
+
+For each operation, the TCF runs a selection algorithm:
+
+1. Filter by operation type.
+2. Apply the policy pre-filter (allowlist/blocklist/residency/safety class).
+3. Score by semantic similarity, schema compatibility, intent overlap, and inverse cost.
+4. Resolve dependencies and mutual exclusions.
+5. Select the top-K capabilities, where K is determined by the model's capability profile.
+
+The result is a **Tool Positioning Frame** — a compact prompt fragment that tells the model exactly which tools it may use for this operation and what each returns.
+
+---
+
+## 6. Operation State Machine
+
+CRP explicitly tracks the lifecycle of an agentic task in the session token / CSO:
+
+| State | Meaning |
+|-------|---------|
+| `INTENT_CLASSIFIED` | STL mapped the request to an operation plan |
+| `OPERATION_POSITIONED` | Operation Frame + Tool Positioning Frame built |
+| `TOOL_SELECTED` | Model emitted a valid tool-selection output |
+| `TOOL_EXECUTED` | Capability ran; observation stored in CSO |
+| `OPERATION_VERIFIED` | DPE / safety verifier checked output |
+| `INTEGRATED` | Result merged into CSO; next operation prepared |
+| `COMPLETE` | Final response assembled |
+| `HALTED` | Preventive safety or failure halt |
+
+This state machine is exposed via `CRP-Agent-Operation-State`, `CRP-Agent-Operation-Type`, and `CRP-Agent-Operation-Plan`. It lets observability, the UI, and safety know exactly where the agent is at any moment.
+
+---
+
+## 7. Preventive safety
+
+Current `halt-on` returns HTTP 451 after detecting a problem. For agentic tool use, safety must be **preventive**: the frame is inspected *before* the model generates output or the tool executes.
+
+The safety gate checks:
+
+- Is the capability allowed by policy / allowlist / data residency?
+- Do the tool arguments violate parameter-level rules?
+- Does the operation type require an oversight token?
+- Does the operation-state transition violate the plan?
+
+If a gate fires, the response is HTTP 451 with the **actual problematic frame**:
+
+```json
+{
+  "crp_halt_reason": "PREVENTIVE_SAFETY_VIOLATION",
+  "halt_point": "TOOL_SELECTED",
+  "problematic_frame": {
+    "operation_type": "RETRIEVE",
+    "capability_id": "web_search",
+    "violation": "data_residency_policy_EU_only",
+    "blocked_parameter": "region=us-east-1"
+  },
+  "session_id": "crp_sess_...",
+  "audit_trail_uri": "https://comply.crprotocol.io/t/..."
+}
+```
+
+The LLM never receives the problematic frame, so it cannot generate an AI-unsafe response from it.
+
+---
+
+## 8. No hybrid auto-routing
+
+The first draft of this proposal included `CRP-Routing-Target: hybrid` and automatic escalation to frontier models. That approach is rejected because:
+
+- It violates the SLM-first design principle.
+- It is unacceptable for air-gapped, privacy-first, and cost-constrained deployments.
+- It hides the real problem: the protocol should make the local model capable, not give up on it.
+
+CRP-SPEC-049 therefore **removes** `CRP-Routing-Target`, `CRP-Routing-Decision`, and the `CRP-Agent-Model-Tier` limiting label. Instead:
+
+- The assigned model is the model that does the work.
+- Frontier models are configured explicitly as providers.
+- The protocol amplifies the local model through STL decomposition, TCF retrieval, CSO state relay, verifier operations, and constrained decoding.
+
+---
+
+## 9. SLM Amplification Rule
+
+The protocol follows a simple rule:
+
+> **For every halving of effective model capability, double the use of positioning, tool retrieval, operation decomposition, and verification.**
+
+| Model class | Profile | Ecosystem scaffolding |
+|-------------|---------|----------------------|
+| Frontier (70B+) | `frontier` | Full tool frame; model may plan and reflect |
+| Capable local (8B–14B) | `capable-local` | 3–4 tools per frame; STL plan; verifier on ANALYSE/EVALUATE |
+| Small local (≤8B) | `small-local` | 1–2 tools per frame; strict structured output; explicit state machine; CSO carries all progress |
+
+This is **amplification**, not limitation. The protocol adds structure so the model can focus on the one cognitive job it is positioned to do.
+
+---
+
+## 10. Protocol surface (summary)
+
+### Added / reshaped headers
+
+- `CRP-Agent-Capability-Profile` (`frontier` / `capable-local` / `small-local`)
+- `CRP-Agent-Model-Capabilities` (`tool-call`, `grammar`, `prefix-cache`, ...)
+- `CRP-Agent-Tool-Subset` (tools enabled for this window)
+- `CRP-Agent-Execution-Mode` (`single-shot`, `planner-actor`, `react`, `verifier`, `retrieval-then-answer`, `positioned-tool-loop`)
+- `CRP-Agent-Loop-Guard` (safety bounds, not capability limits)
+- `CRP-LLM-Structured-Output-Mode` / `CRP-LLM-Structured-Output-Schema`
+- `CRP-LLM-Prompt-Cache-Hint`
+- `CRP-Agent-Operation-State` / `CRP-Agent-Operation-Type` / `CRP-Agent-Operation-Plan`
+- `CRP-Tool-Positioning-Frame` / `CRP-Tool-Observation-Count`
+- `CRP-Tool-Allowlist` / `CRP-Tool-Blocklist`
+- `CRP-Structured-Output-Contract`
+- `CRP-Safety-Preventive-Halt`
+
+### Removed headers
+
+- `CRP-Agent-Model-Tier` — replaced by capability profile.
+- `CRP-Routing-Target` / `CRP-Routing-Decision` — hybrid routing removed.
+
+### Session token / CSO additions
+
+- `capability_profile`, `operation_state`, `operation_type`, `operation_plan_hash`
+- `tcf_selection` — selected capability IDs and why
+- `tool_observations` — typed facts from executed capabilities
+- `preventive_halt_history` — record of halted frames
+- `structured_output_contract`
+
+Full normative definitions are in CRP-SPEC-049 and CRP-SPEC-050.
+
+---
+
+## 11. Implementation roadmap
+
+### Phase 1 — Reframe and specify (now)
+
+- Update README and AGENTS.md tagline to "Context Relay Protocol for Agentic AI."
+- Publish CRP-SPEC-049 (revised) and CRP-SPEC-050 (new).
+- Publish `schemas/capability-descriptor.json`.
+- Add new header constants to `crp/headers/names.py`.
+
+### Phase 2 — Tool Capability Fabric and operation state machine
+
+- Implement `crp/tools/capability_fabric.py`, `descriptor.py`, `executor.py`.
+- Implement `crp/stl/operation_state.py` and `tool_positioner.py`.
+- Extend `crp/state/cso.py` with `tool_observations` and `preventive_halt_history`.
+
+### Phase 3 — Tool positioning frame and constrained execution
+
+- Build Tool Positioning Frame composer.
+- Translate `CRP-Structured-Output-Contract` to OpenAI/Ollama/vLLM/SGLang/llama.cpp.
+- Enforce byte-stable prefixes for KV-cache reuse.
+
+### Phase 4 — Preventive safety
+
+- Extend `CRP-Safety-Policy` grammar with `allow-tools`, `block-tools`, `require-oversight-for`, `block-parameter`.
+- Implement safety gate that inspects operation frame and tool parameters.
+- Add preventive halt response format and audit logging.
+
+### Phase 5 — CRP Comply agent rewrite
+
+- Replace the single ReAct loop with STL-driven operation state machine.
+- Register all Comply tools in the TCF with operation-type tags.
+- Use TCF retrieval to select 1–3 tools per operation for the SLM path.
+- Add verifier operation and constrained decoding.
+- Remove frontier fallback; local model is the assigned provider.
+
+### Phase 6 — Benchmark and prove the claim
+
+- Build benchmark comparing:
+  - Baseline: 8B model with all 13 tools injected.
+  - CRP agentic: 8B model with TCF + STL + CSO + verifier.
+- Measure: task success rate, token consumption, latency, schema validity, safety halt rate.
+- Publish results in BENCHMARKS.md.
+
+---
+
+## 12. Why this is a defensible strategic move
+
+- **No existing protocol owns "agentic positioning."** MCP owns tool access; A2A/ACP own agent-agent messaging; CRP can own the layer that decides *what the model should do with what context and tools*.
+- **SLM-first is a differentiated moat.** Most protocols are designed for frontier models and then retrofitted. Building for the smallest model first forces token efficiency and clean separation of concerns.
+- **The codebase is already 80 % there.** CKF, CDR, CDGR, CSO, STL, safety budget, and audit chain are the hard parts. The TCF and operation state machine are the missing glue.
+- **The market timing is right.** Local/edge SLM deployment is accelerating; enterprises need a protocol that makes small models trustworthy and capable without cloud dependency.
+
+---
+
+## 13. References
+
+- `SPECS_5_06_2026_CRP_v4/specs/CRP-SPEC-049-slm-agent-execution.md`
+- `SPECS_5_06_2026_CRP_v4/specs/CRP-SPEC-050-tool-capability-fabric.md`
+- `schemas/capability-descriptor.json`
+- `crp-comply/SLM_INTEGRATION_RESEARCH.md`
+- `crp-comply/LOCAL_8B_MODEL_ANALYSIS.md`

crprotocol-5.0.0/CRPv5_FINALISATION_TRACKING.md

@@ -0,0 +1,165 @@
+# CRP v5 — Finalisation Tracking Report
+
+> **Status:** ACTIVE — approved plan, gated on WASA AI critical gap audit
+> **Owner:** Agent B (Safety Surface, SDK, Config, Products) + Core Protocol
+> **Base version:** `crprotocol` 4.3.2 (PyPI live) → target **5.0.0**
+> **Doctrine (non-negotiable):** *"The assigned model does the work. Make the local
+> model capable — do not give up on it."* Positioning over injection. No hybrid
+> auto-routing. No `CRP-*` header reaches the provider (Axiom 4).
+> **Companion docs:** `WASA_AI_CRITICAL_GAP_AUDIT.md`, `CRP_SLM_PROTOCOL_PROPOSAL.md`,
+> `SPECS_5_06_2026_CRP_v4/specs/CRP-SPEC-049-slm-agent-execution.md`,
+> `SPECS_5_06_2026_CRP_v4/specs/CRP-SPEC-050-tool-capability-fabric.md`
+
+---
+
+## 0. The one-sentence goal
+
+Make any **< 15 B model that runs on a laptop** behave as a capable agent by
+**positioning** it on one isolated, specialised operation at a time — with full
+awareness of where it is in the plan, the right 1–3 tools for that operation, and a
+durable state object that carries findings forward — so that hundreds of tool calls
+and frames compose into one coherent, complete response **without flooding context or
+exceeding a bounded resource budget**.
+
+---
+
+## 1. Approved decisions (locked)
+
+| # | Decision | Status |
+|---|----------|--------|
+| D1 | **Positioning is the only mode.** Zero tool-schema injection. The model never sees the catalogue. | ✅ Locked |
+| D2 | **Canonical 10-operation taxonomy:** `RETRIEVE, ANALYSE, COMPARE, SYNTHESISE, GENERATE, VERIFY, TRANSFORM, PLAN, CLARIFY, REVISE` (single casing across enum + schema + headers). | ✅ Locked |
+| D3 | **CLARIFY is first-class.** When the model lacks information/authority, it raises a CLARIFY operation that prompts the user for input or an action (wired to a checkpoint). | ✅ Locked |
+| D4 | **No hybrid auto-routing.** Frontier models are explicit providers, never silent fallbacks. | ✅ Locked |
+| D5 | **Findings are decoupled from context.** Observations → typed CSO facts + raw → Scratch Buffer pointers. The window never accumulates tool history. | ✅ Locked |
+| D6 | **Bounded working set guarantee.** Per-operation context is bounded by the capability profile, independent of session length (O(N), not O(N²)). | ✅ Locked |
+
+---
+
+## 2. Build sequence & status
+
+Legend: ☐ not started · ◐ in progress · ☑ done
+
+| Step | Deliverable | Files | Status |
+|------|-------------|-------|--------|
+| **0** | **Taxonomy reconciliation** — unify enum ↔ schema ↔ headers on the canonical 10 ops | `crp/stl/classifier.py`, `schemas/capability-descriptor.json`, `crp/headers/names.py`, SPEC-049/050 | ☑ **done** — canonical 10 ops; `operation_from_token` normaliser (EVALUATE→VERIFY); spec ABNF aligned at step 8 |
+| **1** | **Tool Capability Fabric** — descriptor + registry/retrieval + executor + `ToolResultExtractor` | `crp/tools/{descriptor,capability_fabric,executor,__init__}.py` | ☑ **done** — 29 TCF + 20 STL + 6 smoke tests green; ruff clean |
+| **2** | **Operation State Machine** + CSO extensions (`tool_observations`, `preventive_halt_history`) | `crp/stl/operation_state.py`, `crp/state/cso.py` | ☑ **done** — 10 state-machine tests; CSO observations stored as typed facts |
+| **3** | **Tool Positioner** — Operation Frame → Tool Positioning Frame (1–3 tools by profile) | `crp/stl/tool_positioner.py` | ☑ **done** — frame composer + robust tool-call parser (15 tests) |
+| **4** | **Make STL live** — real positioned loop (`run_positioned`); preventive-safety gate; INTEGRATE→CSO; structured-decoding contract at the `model_call` boundary | `crp/stl/positioned.py` | ☑ **done** — live loop + 8B validation; full 13-stage DPE verify is a follow-up |
+| **5** | **Observation Store + Resource Governor + Visibility event stream** | `crp/state/`, `crp/observability/`, `crp/resources/` | ☑ **done** — `ResourceGovernor` (slow-and-steady, device-tier capping); observations→CSO; OSM event stream |
+| **6** | **CLARIFY + Checkpoint consolidation** — make checkpoints faultless, test them, wire CLARIFY | `crp/security/clarify.py`, `crp/security/checkpoint.py` | ☑ **done** — sync `clarify` bridge (Invariant 10 fallback); CLARIFY op + oversight approval wired; async `Checkpoint` retained for webhooks |
+| **7** | **Expose** — `client.dispatch_positioned()` / execution-mode `positioned-tool-loop`; preventive safety gate; SDK + Gateway wiring | `crp/sdk/client.py`, `crp/core/orchestrator.py`, `crp-gateway/` | ☐ |
+| **8** | **Version bump 5.0.0**, promote SPEC-049/050 from draft, conformance, CHANGELOG | `crp/_version.py`, specs, `CHANGELOG.md` | ☐ |
+
+---
+
+## 3. New v5 components (beyond the original TCF plan)
+
+### 3.1 Observation Store (formalises the wasa-ai archetype)
+- Each tool result → typed `EstablishedFact(provenance=TOOL)` in the CSO + raw output → `ScratchBuffer` pointer (pinned until the report operation runs).
+- **Fixes wasa-ai's M1 bug by design:** the report operation reads the full typed observation set from the CSO; raw evidence is never destroyed before reporting.
+
+### 3.2 Resource Governor (CRP's protocol-layer slice of resource management)
+- Detects device tier → adapts **capability profile** (frame size: small-local 1–2, capable-local 3–4, frontier 5–7), **tool concurrency**, **continuation / loop-guard limits**, **depth**, and **prompt-cache pinning**.
+- **Proactive** (acts at ~70–80% of budget), not OOM-reactive.
+- **Prime directive — "slow and steady over fast and failing."** On a constrained device the Governor *throttles and serialises* (smaller frames, single-flight tool execution, lower concurrency, capped continuation) to keep utilisation under the target — deliberately accepting slower runs rather than risking OOM/overheat. Speed is the trade; stability is the guarantee.
+- Explicitly **does NOT** do quantization / KV compression / offloading — that is the inference engine's job (llama.cpp / vLLM / llama-swap). Component **offload is application-specific**: for WASA AI the candidates are **WASA's own heavy services** (its RAG service, knowledge/embedding components) hosted on Railway — **not** the CRP Gateway — while the positioned **LLM stays local** (see §5 and the WASA audit).
+
+### 3.3 Visibility event stream ("not a blind machine")
+- Every Operation-State-Machine transition emits a streamable, HMAC-chained event: `intent_classified → operation_positioned(type, tools) → tool_selected → tool_executed → observation_stored(fact) → operation_verified → integrated(completion%, remaining) → … → complete`.
+- One mechanism delivers both **live UI** (user sees plan, current op, each tool call, each finding, % complete) and **audit**.
+
+### 3.4 Operation-as-role (the six SLM roles)
+- The report's six roles (router, guardrail, tool-caller, edge-executor, doc-processor, workflow-step) are realised as **operation/role frames** — one local model wearing different hats per operation — with the *option* to register distinct fine-tuned local SLMs as TCF capability providers. Always local; never silent frontier escalation.
+
+### 3.5 CLARIFY + Checkpoint consolidation
+- **Known issue:** checkpoints exist (`crp/security/checkpoint.py`) but are **fragmented and untested**. v5 consolidates them into one tested mechanism and wires the CLARIFY operation to raise a checkpoint that prompts the user for information or an action (and gates `destructive`/`mutating` tools — essential for a pen-test agent).
+
+---
+
+## 4. The bounded-resource guarantee (the "40%" question)
+
+CRP makes **two distinct guarantees**, and it is important not to conflate them:
+
+1. **Deterministic CRP guarantee (always holds):** the per-operation working set
+   (prompt + KV cache) is **bounded by the capability profile's frame budget** and is
+   **independent of how many tool calls have already happened**. A session of 3 calls
+   and a session of 300 calls use the **same-size window**. This is the O(N) (not O(N²))
+   property — and it is exactly what prevents the context/memory growth that caused
+   WASA's OOM and overheating during long runs.
+
+2. **System baseline guarantee (conditional):** total local CPU/RAM staying under a
+   target (e.g. 40%) also depends on **co-resident components** CRP does not control —
+   model weights, embedding models, and the Kali QEMU VM. To hold the baseline on
+   constrained hardware the **Resource Governor** keeps the positioned LLM **local** but
+   runs it **slow and steady** (throttled, serialised) to stay under budget, and the
+   application offloads its **own** heavy services — for WASA AI, its RAG service and
+   knowledge/embedding components — to Railway (**not** the CRP Gateway).
+
+> **Net:** CRP v5 guarantees the **growth** is bounded (no OOM from long runs, steady
+> CPU from small prompts), with an accepted modest speed trade-off — **slow and steady
+> over fast and failing**. The **baseline** under 40% on a laptop is held by the Governor
+> throttling the local model plus offloading the application's own heavy services (e.g.
+> WASA's RAG/knowledge) to Railway. See `WASA_AI_CRITICAL_GAP_AUDIT.md` §4 for the math.
+
+---
+
+## 5. Cross-repo impact & sequencing
+
+| Repo | Role in v5 | Action |
+|------|-----------|--------|
+| **context-relay-protocol** | The protocol + library (this repo) | Build steps 0–8 here first; validate on local 8B |
+| **crp-gateway** (`C:\Users\User\Desktop\crp-gateway`) | CRP's own managed runtime (an independent product, **not** a WASA dependency) | Wire Operation State Machine + TCF into the hosted Gateway |
+| **crp-comply** | First CRP-native consumer (revenue) | Retire bespoke ReAct loop → `dispatch_positioned()` |
+| **crp-scan** | Static scanner / GitHub Action | Later; consumes capability descriptors |
+| **wasa_ai-master** | Proof case + design pressure-test | Becomes CRP-native; offloads **its own** RAG/knowledge services to Railway |
+
+---
+
+## 6. Gates (must pass, in order)
+
+1. **G0 — WASA critical gap audit** (this turn) → defines what CRP must solve vs. infra/consolidation. → see companion doc.
+2. **G1 — Taxonomy reconciliation** lands cleanly; existing tests stay green.
+3. **G2 — TCF + Operation State Machine** unit-tested; descriptor validates against `schemas/capability-descriptor.json`.
+4. **G3 — Live positioned loop** runs end-to-end on **meta-llama-3.1-8b-instruct** (LM Studio `192.168.0.6:1234`) for a multi-operation task, bounded window confirmed.
+5. **G4 — SQB re-run** (kimi-k2.6 judge): positioned ≥ baseline on factual F1 and usefulness.
+6. **G5 — Scale test:** ≥ 100 tool calls in one session with **flat** working-set size (resource telemetry recorded).
+7. **G6 — Demo:** a visual demo that conveys the positioned tool loop, the operation state machine, CSO growth, and flat resource usage.
+
+---
+
+## 7. Testing & demo plan (kimi + local SLM + visuals)
+
+**Test bench (confirmed reachable):** LM Studio `http://192.168.0.6:1234/v1`
+- `meta-llama-3.1-8b-instruct` — capable-local target
+- `qwen2.5-7b-instruct`, `qwen3-4b`, `mergebench-gemma-2-2b-it` — profile spread
+- `gemma-3-270m-it-qat` — small-local extreme
+- `text-embedding-nomic-embed-text-v1.5` — TCF semantic retrieval
+- **Judge:** kimi-k2.6 (`api.moonshot.ai/v1`, `temperature=0.6`, `thinking:{type:disabled}`) — key read from local file, **never printed**.
+
+**Tests to build:**
+- `tests/test_tcf.py` — descriptor validation, retrieval, policy pre-filter, top-K by profile.
+- `tests/test_operation_state.py` — state machine transitions + CSO observation storage.
+- `tests/test_positioned_loop.py` — end-to-end positioned loop (mock provider).
+- `examples/crp_demos/` live runs against the local 8B; SQB via `sqb_benchmark.py`.
+- **Scale harness:** synthetic 100-tool session asserting flat working-set + bounded RSS.
+
+**Demos to finalise/build (some exist but are broken):**
+- `examples/crp_demos/v4/server.py` (+ `static/`) — assess & repair.
+- **Flagship:** a "positioned tool loop" visual demo — live operation state machine, the 1–3-tool frames, CSO findings accumulating, and a resource graph staying flat across 100s of calls.
+
+---
+
+## 8. Risks & open items
+
+- **R1** — Checkpoints fragmented/untested (D3 dependency). Must be consolidated before CLARIFY ships.
+- **R2** — Two operation taxonomies in the tree today; step 0 must not break the live classifier/frame_builder.
+- **R3** — `stl_execute` currently simulates the model call; making it live touches the provider manager + DPE + CSO — highest-integration-risk step.
+- **R4** — Resource Governor must never block a call; it adapts, it does not refuse.
+- **R5** — Railway offload introduces a network dependency; the Governor must degrade gracefully to local-only (accepting the speed downside) when offline/air-gapped.
+
+---
+
+*Tracking report generated 2026-06-30. Update the status columns in §2 and the gate
+checklist in §6 as work lands.*