neotoma 0.3.0 → 0.3.2

package/docs/proposals/iterative_chat_store_mcp_instructions.md

@@ -1,155 +0,0 @@
----
-title: "Iterative Chat Store via MCP Instructions"
-status: "in_progress"
-source_plan: "chat_review_neotoma_store_command_7681eed3.plan.md (ateles)"
-migrated_date: "2026-02-17"
-priority: "p2"
-estimated_effort: "MCP instruction and tool-description updates; optional conversation/turn entity schema and store flow."
----
-
-# Iterative Chat Store via MCP Instructions
-
-## Implementation status (Option A)
-
-Option A implemented 2026-02-17: iterative chat store instructions added to `docs/developer/mcp/instructions.md`, `store` and `store_structured` descriptions updated in `docs/developer/mcp/tool_descriptions.yaml`, and fallback instructions synced in `src/server.ts`. Option B (conversation/turn contract) and Option C (unstructured turn envelope) remain for future work.
-
-## Proposal Context
-
-**Source:** Akin to the ateles Cursor command plan `chat_review_neotoma_store_command_7681eed3.plan.md` (review chat, preview, confirm, then store conversation + messages + attachments in Neotoma), but **iterative**: store the chat and each turn as the agent makes each turn, so the user does not have to manually run an "end of chat" command.
-
-**Relevance:** Users and agents benefit from conversation and turn data in Neotoma for recall, audit, and downstream workflows. An end-of-session command is easy to forget and creates a single point of failure; iterative storage removes that dependency.
-
-**Architecture Alignment:** Complements existing proactive-storage MCP instructions and structured/unstructured store paths. Does not require raw-first ingestion; can work with current store API.
-
----
-
-## Additive Behavior: Structured and Unstructured Data Per Turn
-
-Implementing iterative chat store **adds** conversation and turn structure; it does **not** replace existing storage behavior. The agent continues to store all of the following in the same turn:
-
-| What | How | Unchanged by this proposal? |
-|------|-----|-----------------------------|
-| **Raw chat history** | Conversation entity + agent_message (or equivalent) per turn, with relationships using Neotoma’s supported types only (e.g. message PART_OF conversation; see relationship types in MCP spec). | **New:** this is what we add. |
-| **Structured data from the turn** | Entities extracted from the message (people, tasks, events, contacts, locations, etc.) via `store` / `store_structured`. Existing instructions already say: "Extract and store all relevant entities... Store first, then respond." | **Unchanged.** Agent still stores structured entities from each turn. |
-| **Unstructured data from the turn** | (1) Attachments: files the user or agent references; stored via unstructured path (`file_content` + `mime_type` or `file_path`), then linked (e.g. EMBEDS) to the message or conversation. (2) Option C only: the turn itself as an unstructured envelope (`application/x-neotoma-conversation-turn`) for full provenance. | **Unchanged** for attachments; **optional addition** for turn envelope. |
-
-So after implementation, in each turn the agent will:
-
-1. **Create or update the conversation** and **store the current turn** as an agent_message linked to it (the new iterative-chat-store behavior).
-2. **Continue to extract and store structured entities** from the turn (contacts, tasks, events, etc.) and create relationships as today.
-3. **Continue to store any attachments** via the unstructured path and link them (e.g. EMBEDS) to the message or conversation.
-4. **Optionally** store the turn as an unstructured envelope for provenance (if Option C is adopted).
-
-No existing instruction that tells the agent to "store entities from the conversation" or "store attachments" is removed or weakened.
-
----
-
-## Comparison: Raw-First Ingestion vs Iterative Chat Store
-
-| Aspect | Raw-First Ingestion Architecture | Iterative Chat Store (this proposal) |
-|--------|----------------------------------|-------------------------------------|
-| **Primary goal** | Decouple ingestion from interpretation: store raw bytes first, interpret and curate later. Reduce schema friction and failure modes at ingest. | Persist the **conversation and each turn** in Neotoma as the chat progresses, without requiring a manual "store chat" command at the end. |
-| **Trigger** | User/agent uploads a file or submits content; ingest runs at submission time. | Each agent turn (or each user+agent exchange): agent stores conversation + current turn (and optionally prior turns not yet stored). |
-| **What gets stored** | Raw bytes (any type), then interpretations, then curated truth. Focus on **content** and **provenance chain**. | **Conversation** entity, **agent_message** (or equivalent) per turn, attachments, relationships using supported types only (PART_OF for message→conversation, EMBEDS for attachments). Focus on **session structure** and **turn-level artifacts**. |
-| **User interaction** | None required for raw ingest; optional for curation. | None required; storage is proactive and iterative. Contrast with ateles command: preview → user confirms → then store once. |
-| **MCP surface** | New or revised tools: `ingest`, `interpret`, `curate`, `ingest_structured`. | No new tools required for MVP: **MCP instructions** (and optionally tool descriptions) updated to tell agents to create/update a **conversation** and store **each turn** (and attachments) as they go. |
-| **Complements** | Iterative chat store can use raw-first later: e.g. store each turn as raw, then interpret. | Raw-first can absorb "turn as unstructured blob" (see all-storage-unstructured plan) so iterative store could write turn envelopes into raw layer. |
-| **Determinism** | Ingestion is deterministic (hash-based); interpretation and curation are explicit. | Store calls are deterministic per idempotency; conversation and message entities are explicit. |
-
-**Summary:** Raw-first is about **how** ingestion works (raw → interpret → curate). Iterative chat store is about **when** and **what** to store (conversation + turns, as you go). They are orthogonal and can be combined.
-
----
-
-## Relationship to Ateles Chat Review Store Command
-
-The ateles plan defines a **command** `/review_chat_store_neotoma` that:
-
-1. Parses the full chat in context and builds a preview (conversation, messages, attachments, Parquet migration).
-2. Displays the preview and **waits for user confirmation** (or revision).
-3. On confirmation, stores conversation, messages, attachments, and runs migration.
-
-**This proposal** keeps the same **target state** (conversation + messages + attachments in Neotoma) but changes the **flow**:
-
-- **No single "end of chat" command.** The agent is instructed to store the conversation and each turn **as it makes each turn** (or at a defined cadence, e.g. after each user message).
-- **No mandatory preview/confirm** for the whole chat; storage is incremental. Optional: allow a "preview/confirm for this turn" or "do not store this turn" via user preference or instruction.
-- **Parquet migration** can remain a separate, explicit step (e.g. same command or a different one) when the user wants it; it is not required for iterative conversation storage.
-
----
-
-## Proposed MCP Instruction Additions (Iterative Chat Store)
-
-Add or extend instructions so that agents:
-
-1. **Create or update a conversation entity** at the start of a session (or when first storing in that chat), and link it to the session/thread identifier used by the client.
-2. **After each assistant turn**, store that turn as an **agent_message** (or equivalent) entity linked to the conversation (relationship: **message PART_OF conversation**; use only Neotoma-supported relationship types), and store any new attachments referenced in the turn.
-3. **Use a stable conversation_id** (e.g. from client context or generated once per chat) so all turns in the same chat attach to the same conversation.
-4. **Do not wait for the user to say "save" or run a command;** perform this storage in the same turn as the response, when the agent has the conversation and latest message in context.
-
-Optional refinements:
-
-- **Batching:** If the client does not expose full history each turn, instruct the agent to store "since last stored" (e.g. delta) when history is available.
-- **Unstructured turn artifact:** If the "conversation-turn envelope" (see `.cursor/plans/all-storage-unstructured_provenance_df0b7a8f.plan.md`) is implemented, instruct agents to store each turn as that envelope (unstructured) for full provenance, in addition to or instead of structured message entities.
-- **User opt-out:** Document a way for the user to disable iterative chat storage (e.g. "do not store this conversation" or a client setting) so agents can skip conversation/turn store when requested.
-
----
-
-## Implementation Options
-
-### Option A: Instructions and tool descriptions only (minimal)
-
-- **Update** `docs/developer/mcp/instructions.md`: add a short block (e.g. 2–4 bullets) for iterative conversation and turn storage as above.
-- **Update** `docs/developer/mcp/tool_descriptions.yaml` (and any in-code descriptions): mention that `store` / `store_structured` should be used to persist conversation and agent_message entities each turn when in a chat context.
-- **No schema or server changes.** Relies on existing entity types (e.g. conversation, agent_message) and relationships if they exist; otherwise agents store with descriptive types and the system infers schema.
-
-### Option B: Instructions + conversation/turn contract
-
-- Same as Option A, plus:
-- **Define** a small contract (in MCP_SPEC or vocabulary): conversation entity (minimal fields: e.g. conversation_id, title or placeholder, turn_count), agent_message (turn_number, role, content truncation, optional attachment refs), and relationship types using only Neotoma’s supported set (PART_OF for message→conversation, REFERS_TO for conversation/message→topic, EMBEDS for attachments, SUPERSEDES for reverted turns). No custom types (e.g. "contains_message"); use PART_OF and optionally a mapping layer for display labels.
-- **Optional:** Add a dedicated MCP action (e.g. `store_conversation_turn`) that accepts conversation_id, turn payload, and attachment refs and creates conversation + message + relationships in one call. Reduces instruction complexity and keeps behavior consistent.
-
-### Option C: Iterative + unstructured turn envelope
-
-- Same as Option B, plus:
-- **Adopt** the conversation-turn envelope from the all-storage-unstructured plan (`application/x-neotoma-conversation-turn`): agent stores each turn as that unstructured blob (with turn_id, conversation_id, content, attachment_source_ids) so Neotoma keeps the raw artifact and runs interpretation server-side.
-- Instructions tell agents to store **both** (a) the turn as unstructured envelope (for provenance) and (b) structured conversation + agent_message (for query and UX), or only one path depending on product choice.
-
----
-
-## Reverts and chat forks (follow-on proposal)
-
-Option A does not define turn identity, idempotency for messages, or branch/fork semantics. Reverts and forks are therefore best-effort: duplicate messages, mixed branches, or inconsistent dedupe can occur. A separate proposal covers the fix: **`docs/proposals/conversation_turn_identity_reverts_forks.md`** (conversation turn identity, idempotency_key convention, and handling reverts/forks when the MCP host exposes branch_id/turn_id).
-
----
-
-## What's In Scope
-
-- MCP instruction and tool-description changes to encourage iterative storage of conversation and turns.
-- Optional: lightweight contract for conversation and agent_message entities and relationships.
-- Optional: dedicated `store_conversation_turn`-style action or use of unstructured turn envelope.
-
-## What's Out of Scope
-
-- Changing raw-first ingestion architecture (separate proposal).
-- Changing interpretation or curation flow (separate proposals).
-- Parquet migration (can stay as a separate command or step).
-- Mandatory preview/confirm UI for each turn (can be a later opt-in).
-
----
-
-## Success Criteria
-
-- Agents using Neotoma MCP store conversation and each turn as the chat progresses, without the user running an end-of-chat store command.
-- A single conversation entity is reused for the whole chat; turns are linked to it.
-- Attachments referenced in turns are stored and linked using supported relationship types (EMBEDS). Message→conversation uses PART_OF; no custom types (see `conversation_turn_identity_reverts_forks.md` for relationship-type alignment).
-- **Structured and unstructured storage per turn is unchanged:** agents still store extracted entities (people, tasks, events, etc.) and attachments from each turn in addition to the conversation and message structure.
-- User can opt out of iterative chat storage when desired (documented or configurable).
-
----
-
-## References
-
-- Raw-first ingestion: `docs/proposals/raw-first-ingestion-architecture.md`
-- Ateles chat review store command: `../ateles` / `~/.cursor/plans/chat_review_neotoma_store_command_7681eed3.plan.md`
-- All-storage-unstructured (turn envelope): `.cursor/plans/all-storage-unstructured_provenance_df0b7a8f.plan.md`
-- MCP instructions: `docs/developer/mcp/instructions.md`
-- MCP spec: `docs/specs/MCP_SPEC.md`

package/docs/proposals/llm_sampling_parameters_interpretation_config.md

@@ -1,196 +0,0 @@
----
-title: "LLM Sampling Parameters in Interpretation Config"
-status: "proposal"
-source_plan: null
-created_date: "2026-02-09"
-priority: "p2"
-estimated_effort: "1-2 days"
----
-
-# LLM Sampling Parameters in Interpretation Config
-
-## Scope
-
-This document covers:
-- Adding top-K and top-P (nucleus sampling) to Neotoma's interpretation configuration
-- Task-specific sampling presets for factual vs creative LLM use cases
-- Documentation of sampling parameters for auditability and reproducibility
-
-This document does NOT cover:
-- Changes to reducers, entity resolution, or other deterministic components
-- Strategy Layer or Execution Layer LLM usage (future work)
-
-## Proposal Context
-
-**Source:** Analysis of [Understanding Top-K and Top-P in Prompt Engineering](https://medium.com/@8926581/understanding-top-k-and-top-p-in-prompt-engineering-00a3b93dcd40) (Nikita Zaharov) applied to Neotoma's interpretation architecture.
-
-**Relevance:** Neotoma uses LLMs for AI interpretation of unstructured files (PDFs, images). Current implementation uses `temperature` and `top_p` but does not log `top_k`, and interpretation config lacks task-specific presets. Adding full sampling parameter support improves control over extraction quality and supports auditability.
-
-**Architecture Alignment:** Verified. Aligns with interpretation auditability in `docs/architecture/determinism.md`. Parameters are part of interpretation config, not Truth Layer core logic.
-
-## Purpose
-
-Extend Neotoma's interpretation configuration to include full LLM sampling parameters (top-K, top-P) alongside temperature, enabling:
-
-1. Complete audit trail of sampling configuration used for each interpretation run
-2. Task-specific presets (factual extraction vs schema recommendation) with documented rationale
-3. Mitigation of repetition loop bugs when they occur in structured extraction output
-4. Consistent parameter handling across all LLM-calling services
-
-## Overview
-
-### Current State
-
-| Location | Temperature | Top-P | Top-K | Logged in config? |
-|----------|-------------|-------|-------|-------------------|
-| `llm_extraction.ts` | 0 | 1 | Not set | Partial |
-| `server.ts` interpretation | 0 (default) | Not set | Not set | Yes (temp only) |
-| `schema_icon_service.ts` | 0.1, 0.3 | Not set | Not set | No |
-| `schema_recommendation.ts` | 0.3 | Not set | Not set | No |
-
-`InterpretationConfig` in `src/services/interpretation.ts` includes:
-- `provider`, `model_id`, `temperature`, `prompt_hash`, `code_version`, `feature_flags`
-- Does NOT include `top_k` or `top_p`
-
-### Proposed State
-
-1. **Extend InterpretationConfig** with `top_k` and `top_p` (optional, with defaults)
-2. **Define task-specific presets** in configuration
-3. **Apply presets** to interpretation, LLM extraction, and schema services
-4. **Document** sampling parameter rationale in architecture docs
-
-## Technical Details
-
-### 1. InterpretationConfig Schema Extension
-
-```typescript
-// src/services/interpretation.ts
-export interface InterpretationConfig {
-  provider: string;
-  model_id: string;
-  temperature: number;
-  top_k?: number;      // NEW: Limit to K most likely tokens (e.g., 20-40)
-  top_p?: number;      // NEW: Nucleus sampling threshold (e.g., 0.9-0.99)
-  prompt_hash: string;
-  code_version: string;
-  feature_flags?: Record<string, boolean>;
-}
-```
-
-### 2. Task-Specific Presets
-
-Based on prompt engineering best practices (factual extraction vs creative tasks):
-
-| Preset | Use Case | Temperature | Top-K | Top-P | Rationale |
-|--------|----------|-------------|-------|-------|-----------|
-| `factual_extraction` | Document field extraction, interpretation | 0 | 20 | 1 | Maximum determinism; temp=0 picks most likely token; top-K=20 narrows vocabulary |
-| `balanced` | Schema recommendation, icon matching | 0.2 | 30 | 0.95 | Slight flexibility; avoids repetition loop |
-| `creative` | Future: summaries, descriptions | 0.7 | 40 | 0.99 | More variety; not for Truth Layer extraction |
-
-Default for interpretation: `factual_extraction`.
-
-### 3. Implementation Locations
-
-| File | Change |
-|------|--------|
-| `src/services/interpretation.ts` | Add `top_k`, `top_p` to InterpretationConfig; pass to LLM calls |
-| `src/services/llm_extraction.ts` | Add `top_k: 20`; read from config if provided |
-| `src/server.ts` | Parse `top_k`, `top_p` from `interpretation_config`; apply defaults |
-| `src/services/schema_icon_service.ts` | Use `balanced` preset |
-| `src/services/schema_recommendation.ts` | Use `balanced` preset |
-| `docs/architecture/determinism.md` | Document sampling parameters in interpretation auditability section |
-
-### 4. Repetition Loop Mitigation
-
-If interpretation output exhibits repetition (e.g., "AI is useful. AI is useful. AI is useful."):
-
-- **Cause:** Temperature 0 with restrictive sampling can cause deterministic loops
-- **Fix:** Use `balanced` preset (temp 0.2, top-P 0.95) or increase top-K to 30
-- **Trade-off:** Slightly more variance; idempotence preserved via canonicalization
-
-## Data Models
-
-### InterpretationConfig (Extended)
-
-```yaml
-InterpretationConfig:
-  provider: string
-  model_id: string
-  temperature: number
-  top_k: number | null   # Optional; null = provider default
-  top_p: number | null   # Optional; null = provider default
-  prompt_hash: string
-  code_version: string
-  feature_flags: Record<string, boolean> | null
-```
-
-### interpretations Table
-
-Ensure `interpretation_config` JSONB column can store `top_k` and `top_p` (no migration needed; JSONB is schemaless for additional keys).
-
-## Implementation Steps
-
-1. Extend `InterpretationConfig` interface in `interpretation.ts`
-2. Add preset constants (factual_extraction, balanced, creative) to a shared config module
-3. Update `llm_extraction.ts` to accept and use `top_k`, `top_p` from config
-4. Update `server.ts` interpretation handler to parse and pass `top_k`, `top_p`
-5. Update `schema_icon_service.ts` and `schema_recommendation.ts` to use `balanced` preset
-6. Add documentation to `docs/architecture/determinism.md` Section 1.3 or 1.4
-7. Add unit tests for preset application and config serialization
-
-## Testing Requirements
-
-- Unit test: InterpretationConfig with `top_k` and `top_p` serializes and deserializes correctly
-- Unit test: Preset application produces expected parameter values
-- Integration test: Interpretation run stores full config (including top_k, top_p) in interpretations
-- Regression: Existing interpretation flows work with null/undefined top_k and top_p (backward compatible)
-
-## Invariants
-
-- Interpretation config MUST remain auditable (all sampling params logged)
-- Default behavior MUST NOT change for existing callers (backward compatible)
-- Idempotence via canonicalization MUST remain the primary guarantee (sampling params are an aid, not a replacement)
-
-## Definitions
-
-- **Top-K:** LLM sampling method that limits token selection to the K most probable tokens at each step
-- **Top-P (nucleus sampling):** LLM sampling method that selects the smallest set of tokens whose cumulative probability reaches P
-- **Factual extraction:** Structured field extraction from documents (invoices, receipts); requires low variance
-
-## References
-
-- Source article: https://medium.com/@8926581/understanding-top-k-and-top-p-in-prompt-engineering-00a3b93dcd40
-- `docs/architecture/determinism.md` — Idempotence vs determinism, interpretation auditability
-- `src/services/interpretation.ts` — InterpretationConfig interface
-- `src/services/llm_extraction.ts` — Current temperature/top_p usage
-
-## Agent Instructions
-
-### When to Load This Document
-
-Load when implementing or modifying interpretation configuration, LLM sampling parameters, or interpretation auditability.
-
-### Required Co-Loaded Documents
-
-- `docs/architecture/determinism.md`
-- `docs/subsystems/ingestion/ingestion.md`
-- `src/services/interpretation.ts`
-
-### Constraints Agents Must Enforce
-
-1. All LLM sampling parameters used MUST be logged in interpretation config
-2. Defaults MUST preserve backward compatibility (null/undefined = provider default)
-3. Factual extraction MUST use factual_extraction preset (or equivalent low-variance params)
-
-### Forbidden Patterns
-
-- Adding sampling parameters without updating InterpretationConfig
-- Using creative preset for Truth Layer interpretation (extraction must use factual or balanced)
-
-### Validation Checklist
-
-- [ ] InterpretationConfig includes top_k and top_p
-- [ ] Presets are documented with rationale
-- [ ] All LLM-calling services use presets or explicit config
-- [ ] documentation/determinism.md updated
-- [ ] Tests cover config serialization and preset application

package/docs/proposals/mcp-cli-action-items.md

@@ -1,207 +0,0 @@
----
-title: "MCP and CLI action items proposal"
-status: "proposal"
-source_plan: "chat_action_items_analysis_8a1cabc6.plan.md"
-migrated_date: "2026-02-02"
-priority: "p1"
-estimated_effort: "3 days (assumes Cursor agent execution)"
----
-
-# MCP and CLI action items proposal
-## Scope
-This proposal captures key action items derived from the MCP vs CLI chat transcript. It documents required codebase changes and documentation updates. It does not implement changes or set release timelines.
-
-## Purpose
-Provide a single, actionable reference that links each action item to current codebase state and required changes. This ensures alignment between the contract surface, MCP tooling, and CLI behavior.
-
-## Documentation doctrine
-This proposal follows the Neotoma Documentation Doctrine in `docs/conventions/documentation_standards_rules.mdc`.
-
-```
-Neotoma is a Truth Layer - not an app, agent, or strategy or execution system.
-Core invariants:
-- Deterministic: Same input -> same output, always
-- Immutable: Truth never changes after storage
-- Provenance: Every output traces to source
-- Schema-first: All extraction derives from schemas
-- Explicit control: User approves all ingestion
-- Privacy-maximal: No background data collection
-- Graph integrity: No orphans, no cycles, no inferred edges
-- Truth-Layer bounded: No strategy, execution, or agent logic
-- Event-sourced: All state updates via Domain Events -> Reducers
-- Pure Strategy: Strategy Layer has no side effects (State in -> Decisions out)
-- Pure Execution: Execution Layer emits Domain Events (Commands in -> Events out)
-Layered architecture (example: financial system):
-┌───────────────────────────────────────────────┐
-│ Execution Layer                               │
-│ (Agentic Wallet + Domain Agents)              │
-│ Commands -> Side Effects -> Domain Events     │
-└────────────▲─────────────────────────────────┘
-             │ Reads Only, Receives Commands
-┌────────────▼─────────────────────────────────┐
-│ Strategy Layer                                │
-│ (Agentic Portfolio is example instance)       │
-│ State -> Evaluates -> Decisions + Commands    │
-└────────────▲─────────────────────────────────┘
-             │ Reads Only
-┌────────────▼─────────────────────────────────┐
-│ Neotoma (Truth Layer)                         │
-│ Event-sourced, Reducer-driven                 │
-│ Domain Events -> Reducers -> State            │
-└───────────────────────────────────────────────┘
-Note: Agentic Portfolio is an example instance of Strategy Layer. Agentic Wallet is part of Execution Layer alongside domain agents. Many other agent-driven layers are possible.
-This document enforces Truth Layer purity.
-```
-
-## Invariants
-1. The contract surface remains the single source of truth for tool and CLI behavior.
-2. MCP tools MUST be schema bound and deterministic, with no hidden side effects.
-3. CLI behavior MUST remain auditable and machine stable when using `--json`.
-4. Truth Layer boundaries MUST remain intact. No strategy or execution logic is introduced.
-5. Documentation MUST avoid AI generated writing patterns and em dashes.
-
-## Definitions
-- **Contract surface**: OpenAPI specification and domain schemas that define operations, inputs, outputs, and error models.
-- **MCP tool**: A typed action exposed via Model Context Protocol, defined in the MCP server tool catalog.
-- **CLI**: A command line interface that executes contract operations and supports machine stable output.
-- **Adapter**: A transport layer implementation that maps the contract surface to MCP or CLI.
-- **Action item**: A required change derived from the chat transcript, documented with current state and needed updates.
-
-## Data models and schemas
-The proposal references current schema sources and contract definitions:
-- **OpenAPI**: `openapi.yaml`
-- **MCP tool catalog**: `src/server.ts`
-- **MCP specification**: `docs/specs/MCP_SPEC.md`
-- **REST schemas**: `src/actions.ts`
-- **Domain schemas**: `docs/subsystems/schema_snapshots/`
-
-## Flow and sequence
-This proposal focuses on alignment between contract definitions, MCP tooling, and CLI behavior.
-
-```mermaid
-flowchart TD
-    Transcript[ChatTranscript] --> Proposal[ActionItemsProposal]
-    Proposal --> ContractSurface[OpenAPIAndDomainSchemas]
-    Proposal --> MCPTools[MCPToolCatalog]
-    Proposal --> CLIWrapper[CLIWrapper]
-    ContractSurface --> Alignment[ContractAlignment]
-    MCPTools --> Alignment
-    CLIWrapper --> Alignment
-    Alignment --> Logging[CLIEquivalentLogging]
-    Alignment --> Fixtures[ReplayableFixtures]
-```
-
-## Action items and codebase context
-
-### Contract and architecture
-1. Treat OpenAPI and domain schemas as the single core. MCP and CLI are generated or derived adapters.
-   - Current state: OpenAPI exists in `openapi.yaml`. Contract mappings are tracked in `src/shared/contract_mappings.ts`. MCP tool definitions live in `src/server.ts` and `docs/specs/MCP_SPEC.md`.
-   - Needed change: Define a single contract authority and align MCP and CLI surfaces to it. Use OpenAPI as the canonical source and ensure MCP tool schemas are derived or mirrored.
-
-2. Enforce 1:1 mapping between OpenAPI operations and MCP tools or CLI commands. Allow only a small set of agent native composites.
-   - Current state: Mapping registry exists in `src/shared/contract_mappings.ts`. Parity tests validate OpenAPI operation coverage in `tests/contract/contract_mapping.test.ts`.
-   - Needed change: Add a mapping policy and a validation checklist to ensure every OpenAPI operation is represented by a tool and CLI command, except approved composites.
-
-3. Log the CLI equivalent invocation for every MCP tool call.
-   - Current state: MCP request handling logs CLI equivalents using `buildCliEquivalentInvocation()` in `src/server.ts`.
-   - Needed change: Introduce a CLI wrapper or command registry and emit a redaction aware CLI equivalent log at the MCP handler layer.
-
-4. Define a CLI output contract with `--json` and `--pretty`. MCP must never parse human output.
-   - Current state: CLI supports `--json` and `--pretty` output modes in `src/cli/index.ts`.
-   - Needed change: Ensure CLI only emits machine stable output in `--json` mode and keep `--pretty` for human usage.
-
-5. Require idempotency keys for ingestion and corrections.
-   - Current state: Ingestion flows are documented in `docs/subsystems/ingestion/ingestion.md`. MCP actions for ingest and correct are implemented in `src/server.ts`. The schemas in `src/actions.ts` do not show idempotency keys.
-   - Needed change: Add idempotency key fields to ingest and correct schemas, then enforce them in service handlers.
-
-6. Enforce a strict, machine checkable error taxonomy.
-   - Current state: `handleApiError` in `src/actions.ts` returns `{"error": "message"}`. This is a string error model.
-   - Needed change: Introduce structured error envelopes and align them with `docs/subsystems/errors.md`.
-
-7. Maintain replayable fixtures with expected extracted graphs.
-   - Current state: Fixtures are consolidated under `tests/fixtures/` with broad coverage documented in `tests/fixtures/README.md`.
-   - Needed change: Add expected extracted graphs or snapshots alongside fixtures to support deterministic replay checks.
-
-### CLI and MCP execution rules
-1. Treat the CLI as the reference executor. If a change cannot be done via CLI, it is not a real operation.
-   - Current state: Execution entrypoints are MCP stdio in `src/index.ts` and HTTP transport in `src/actions.ts`. No dedicated CLI exists.
-   - Needed change: Define a CLI module that calls the same service layer as MCP and REST. Use it as the reference executor.
-
-2. Keep MCP tools boring and atomic.
-   - Current state: MCP tool definitions are in `src/server.ts`. Test failures in `docs/reports/FAILING_TESTS_SUMMARY.md` show mismatched method names, which indicates some MCP tooling instability.
-   - Needed change: Stabilize method names and enforce atomic tool boundaries before adding composite actions.
-
-### Replace web app effort
-1. Publish a canonical walkthrough: ingest, normalize, extract, query, replay.
-   - Current state: `docs/subsystems/ingestion/ingestion.md` documents the pipeline. `docs/releases/v0.2.0/integration_tests.md` includes step sequences.
-   - Needed change: Add a focused walkthrough doc or section that links to ingestion and testing steps.
-
-2. Maintain a fixtures repository with sample documents and expected outputs.
-   - Current state: `tests/fixtures/` already provides a consolidated repository of JSON, CSV, and PDF fixtures.
-   - Needed change: Document expected outputs in a single place to align with fixtures.
-
-3. Build a failure gallery for determinism breaks and system responses.
-   - Current state: Failure analyses are scattered under `docs/reports/`.
-   - Needed change: Create a curated index or new report in `docs/reports/` that summarizes determinism failure cases.
-
-## Examples
-**CLI equivalent logging**
-```
-neotoma store --json --source-id src_abc123 --entity-type invoice
-```
-
-**Structured error envelope shape**
-```
-{
-  "error_code": "INGESTION_INVALID_SOURCE",
-  "message": "Source format is invalid.",
-  "trace_id": "trace_abc123"
-}
-```
-
-## Testing requirements
-1. Add unit tests for idempotency key validation on ingest and correct actions.
-2. Add MCP tool parity tests that validate 1:1 mapping between OpenAPI operations and MCP tools.
-3. Add CLI output contract tests for `--json` and `--pretty`.
-4. Add fixture replay tests that compare expected snapshots to actual reducer outputs.
-5. Run documentation dependency validation after adding this proposal.
-
-## Open issues and prerequisites
-No open issues at this time.
-
-## Agent Instructions
-### When to Load This Document
-Load this document when planning or evaluating changes to MCP tooling, CLI behavior, or contract alignment.
-
-### Required Co-Loaded Documents
-- `docs/NEOTOMA_MANIFEST.md` if present
-- `docs/foundation/core_identity.md`
-- `docs/architecture/determinism.md`
-- `docs/specs/MCP_SPEC.md`
-- `docs/subsystems/ingestion/ingestion.md`
-- `docs/subsystems/errors.md`
-- `docs/testing/fixtures_standard.md`
-- `docs/conventions/documentation_standards_rules.mdc`
-- `docs/conventions/writing_style_guide.md`
-
-### Constraints Agents Must Enforce
-1. Contract surface remains canonical and consistent across MCP and CLI.
-2. MCP tools remain deterministic and schema bound.
-3. CLI outputs are machine stable when using `--json`.
-4. No Truth Layer boundary violations.
-5. Documentation uses active voice, short sentences, and no em dashes.
-
-### Forbidden Patterns
-- Introducing strategy or execution logic into Truth Layer code.
-- Parsing non JSON CLI output in MCP tools.
-- Free form string error responses in MCP or REST.
-- Adding composite MCP tools without explicit contract justification.
-- Omitting provenance or idempotency validation for ingestion.
-
-### Validation Checklist
-- [ ] Proposal includes required sections and Agent Instructions.
-- [ ] No em dashes or en dashes used.
-- [ ] Contract and tool mapping references are accurate.
-- [ ] OpenAPI and MCP tool definitions are linked to current code.
-- [ ] Action items include current state and needed changes.
-- [ ] Documentation dependencies are updated.