neotoma 0.3.0 → 0.3.2

package/docs/feature_units/standards/worker_agent_template.md

@@ -1,191 +0,0 @@
-# Worker Agent Instructions Template
-## Instructions for {{FU_ID}} in Batch {{BATCH_ID}}
-You are a worker agent executing Feature Unit **{{FU_ID}}** in Batch **{{BATCH_ID}}** for Release **{{RELEASE_ID}}**.
-## Your Task
-### 1. Load FU Specification
-- **Primary**: `docs/feature_units/completed/{{FU_ID}}/FU-{{FU_ID}}_spec.md`
-- **Fallback**: `docs/specs/MVP_FEATURE_UNITS.md` (search for `{{FU_ID}}`)
-### 2. Execute Feature Unit Workflow
-Follow the standard Feature Unit execution workflow:
-1. **Check if FU spec exists**
-   - If not, run `Create New Feature Unit` workflow (Checkpoint 0)
-   - Load `.cursor/commands/create_feature_unit.md`
-2. **If UI FU and no prototype exists**
-   - Run `Create Prototype` workflow
-   - Load `.cursor/commands/create_prototype.md`
-   - Get prototype approval (Checkpoint 1)
-3. **Run implementation workflow**
-   - Load `.cursor/commands/run_feature_workflow.md`
-   - Follow implementation steps from FU spec
-   - Write code, tests, documentation
-4. **Run tests**
-   - Unit tests: `npm run test:unit -- --grep "{{FU_ID}}"`
-   - Integration tests: `npm run test:integration -- --grep "{{FU_ID}}"`
-   - E2E tests: `npm run test:e2e -- --grep "{{FU_ID}}"`
-5. **Final review**
-   - Load `.cursor/commands/final_review.md`
-   - Run Checkpoint 2 (Final Review)
-### 3. Update Status File
-**Location**: `{{STATUS_FILE}}`
-**Update Frequency**: Every 5-10 minutes during execution
-**Status Update Format**:
-```json
-{
-  "fu_id": "{{FU_ID}}",
-  "status": "running" | "completed" | "failed",
-  "progress": 0.0-1.0,
-  "last_update": "ISO_TIMESTAMP",
-  "message": "Optional status message",
-  "tests": {
-    "unit": {
-      "passed": true | false,
-      "coverage": 0-100,
-      "command": "npm run test:unit -- --grep {{FU_ID}}"
-    },
-    "integration": {
-      "passed": true | false,
-      "command": "npm run test:integration -- --grep {{FU_ID}}"
-    },
-    "e2e": {
-      "passed": true | false | null,
-      "command": "npm run test:e2e -- --grep {{FU_ID}}"
-    }
-  }
-}
-```
-**Completion Signal**:
-When FU is complete, update status with test results:
-```json
-{
-  "fu_id": "{{FU_ID}}",
-  "status": "completed",
-  "progress": 1.0,
-  "completed_at": "ISO_TIMESTAMP",
-  "tests": {
-    "unit": {
-      "passed": true,
-      "coverage": 85,
-      "command": "npm run test:unit -- --grep FU-100"
-    },
-    "integration": {
-      "passed": true,
-      "command": "npm run test:integration -- --grep FU-100"
-    },
-    "e2e": {
-      "passed": true,
-      "command": "npm run test:e2e -- --grep FU-100"
-    }
-  }
-}
-```
-**Test Execution Requirements:**
-1. **Unit Tests** (REQUIRED):
-   - Run: `npm run test:unit -- --grep "{{FU_ID}}"`
-   - Must pass before marking complete
-   - Report coverage percentage if available
-2. **Integration Tests** (REQUIRED):
-   - Run: `npm run test:integration -- --grep "{{FU_ID}}"`
-   - Must pass before marking complete
-3. **E2E Tests** (REQUIRED for Medium/High risk FUs):
-   - Run: `npm run test:e2e -- --grep "{{FU_ID}}"`
-   - Check FU spec for risk level
-   - Must pass if required
-**Test Failure Handling:**
-If any required test fails:
-- Set `status: "failed"`
-- Include error details in `error` field
-- Do NOT mark FU as complete
-- Report failure immediately
-**Error Reporting**:
-If FU fails, update status:
-```json
-{
-  "fu_id": "{{FU_ID}}",
-  "status": "failed",
-  "error": "Error message",
-  "failed_at": "ISO_TIMESTAMP"
-}
-```
-## Constraints
-### Repository-Wide Constraints
-Follow all constraints from `docs/foundation/agent_instructions.md`:
-1. **Never violate Truth Layer boundaries** (no strategy/execution logic)
-2. **Never introduce nondeterminism** (no random IDs, no LLM extraction in MVP)
-3. **Never generate features outside MVP scope** (no semantic search, no agents)
-4. **Always follow schema-first principles** (type-driven, not inference-driven)
-5. **Enforce immutability** (raw_text, schema_type, extracted_fields)
-6. **Enforce safety and explicit control** (user approves all ingestion)
-7. **Always output deterministic, validated artifacts** (code, specs, docs)
-### Multi-Agent Constraints
-1. **Do not modify FUs assigned to other agents**
-   - Only work on `{{FU_ID}}`
-   - Do not modify files related to other FUs in the batch
-2. **Update status file atomically**
-   - Use file locking if available
-   - Read-modify-write pattern
-   - Handle concurrent updates gracefully
-3. **Report failures immediately**
-   - Don't retry indefinitely
-   - Set `status: "failed"` if FU cannot be completed
-   - Include error details in status update
-4. **Respect execution limits**
-   - Do not spawn additional agents
-   - Do not exceed resource limits
-   - Report if FU requires more resources than allocated
-## Status File Location
-**File**: `{{STATUS_FILE}}`
-**Structure**: See `docs/feature_units/standards/multi_agent_orchestration.md` for complete status file schema.
-**Update Protocol**:
-1. Read current status file
-2. Find your FU entry in `batches[].feature_units[]`
-3. Update your FU's status fields
-4. Write status file atomically
-**Example Update**:
-```javascript
-// Pseudo-code
-const status = JSON.parse(await fs.readFile(STATUS_FILE));
-const batch = status.batches.find(b => b.batch_id === {{BATCH_ID}});
-const fu = batch.feature_units.find(f => f.fu_id === "{{FU_ID}}");
-fu.status = "running";
-fu.progress = 0.5;
-fu.last_update = new Date().toISOString();
-fu.tests.unit.passed = true;
-fu.tests.unit.coverage = 85;
-await fs.writeFile(STATUS_FILE, JSON.stringify(status, null, 2));
-```
-## Error Handling
-### If FU Spec Not Found
-1. Check if FU exists in `docs/specs/MVP_FEATURE_UNITS.md`
-2. If not found, create FU spec using `Create New Feature Unit` workflow
-3. Update status: `{"status": "spec_created", "message": "Created FU spec"}`
-### If Tests Fail
-1. Fix implementation issues
-2. Re-run tests
-3. If tests still fail after 3 attempts, report failure:
-   ```json
-   {
-     "status": "failed",
-     "error": "Tests failed after 3 attempts",
-     "test_output": "..."
-   }
-   ```
-### If Dependencies Not Met
-1. Check FU dependencies in spec
-2. Verify dependency FUs are complete in status file
-3. If dependencies incomplete, wait and check again in 5 minutes
-4. If dependencies still incomplete after 30 minutes, report:
-   ```json
-   {
-     "status": "blocked",
-     "error": "Dependencies not met: FU-XXX, FU-YYY"
-   }
-   ```
-## Completion Checklist
-Before marking FU as completed:
-- [ ] All code implemented per spec
-- [ ] All unit tests passing
-- [ ] All integration tests passing
-- [ ] All E2E tests passing (if applicable)
-- [ ] Documentation updated (if applicable)
-- [ ] Status file updated with `status: "completed"` and test results
-- [ ] Final review completed (Checkpoint 2)

package/docs/foundation/ai_safety.md

@@ -1,10 +0,0 @@
-# AI Safety Doctrine
-## AI Tool Requirements
-AI tools (ChatGPT, Claude, Cursor) MUST:
-- Access truth **only via MCP** (no direct DB access)
-- Reference `record_id` in answers (provenance)
-- Never invent entities (only use existing)
-- Never invent fields (only use extracted)
-- Never reference nonexistent truth
-- Default to conservative outputs ("I don't see that field")
-**Any spec allowing LLM guessing without MCP grounding is invalid.**

package/docs/foundation/composability_analysis.md

@@ -1,308 +0,0 @@
-# Composability as a Core Principle: Analysis and Recommendation
-## 1. How Composability Manifests in Neotoma
-### 1.1 Architectural Composability
-Neotoma is designed as a **composable substrate** that enables multiple layers to build on top:
-**Layered Architecture Model:**
-- **Truth Layer (Neotoma):** Event-sourced, reducer-driven deterministic world model
-- **Strategy Layer:** Pure cognition (e.g., Agentic Portfolio) — reads truth, outputs Decisions + Commands
-- **Execution Layer:** Pure effect (e.g., Agentic Wallet) — takes Commands, performs side effects, emits Domain Events
-**Key Composability Mechanisms:**
-1. **Read-Only Boundaries:** Upper layers can read truth but cannot mutate it directly
-2. **Domain Event → Reducer Pattern:** All truth updates flow through reducers processing Domain Events
-3. **Protocol-Based Interfaces:** MCP exposes structured, validated access points
-4. **Clear Layer Separation:** Strategy and Execution layers are architecturally distinct from Truth Layer
-### 1.2 Integration Composability
-**MCP Protocol Integration:**
-- Standardized interface enables any MCP-compatible agent to integrate
-- Works across ChatGPT, Claude, Cursor without platform lock-in
-- Structured action catalog (13+ actions) provides predictable integration surface
-**External System Composition:**
-- External adapters (Gmail API, Plaid, file storage) are swappable
-- Infrastructure layer abstracts external dependencies
-- Domain layer remains independent of external implementations
-### 1.3 Ecosystem Composability
-**Multiple Upper Layers Possible:**
-- Strategy Layer examples: Agentic Portfolio (financial), General Strategy Engine (general)
-- Execution Layer examples: Agentic Wallet (financial), Domain Agents (various domains)
-- Future layers: Other agent-driven systems can build on Neotoma
-**Protocol Evolution:**
-- MCP-based interfaces support future protocol-first, decentralized evolution
-- Event-sourced architecture enables replication and tokenized permission layers
-- No architectural drag from internal chat or conversational state
-## 2. Composability vs. Interoperability
-### 2.1 Interoperability (Cross-Platform Access)
-**Definition:** Ability to work across multiple platforms/tools without lock-in.
-**Neotoma's Interoperability:**
-- MCP enables ChatGPT, Claude, Cursor to access the same memory
-- Memory persists across all AI tools, not platform-specific
-- **Current Status:** Explicitly articulated as differentiator #3 ("Cross-Platform Access")
-**Why Defensible:**
-- Providers cannot pursue due to platform lock-in business models
-- Startups cannot pursue due to separate consumer app positioning
-### 2.2 Composability (Layered Building)
-**Definition:** Ability for external systems to build layers on top of Neotoma, creating new capabilities through composition.
-**Neotoma's Composability:**
-- Strategy Layer and Execution Layer can build on Truth Layer
-- Clear boundaries enable independent layer development
-- Domain Event → Reducer pattern enables predictable composition
-- **Current Status:** Architecturally fundamental but not explicitly articulated as differentiator
-**Key Distinction:**
-- **Interoperability** = horizontal (works across platforms)
-- **Composability** = vertical (layers build on top)
-### 2.3 Relationship
-Composability and interoperability are **complementary but distinct**:
-- **Interoperability** enables Neotoma to work with multiple AI platforms (horizontal)
-- **Composability** enables Neotoma to serve as foundation for Strategy/Execution layers (vertical)
-Both are architecturally fundamental, but they address different dimensions:
-- Interoperability = "works with all your tools"
-- Composability = "foundation for building new capabilities"
-## 3. Should Composability Be Articulated as a Principle?
-### 3.1 Arguments FOR Articulating Composability
-**1. Architectural Foundation:**
-- Composability is architecturally fundamental (enables Strategy/Execution layers)
-- Not just a feature—it's a core design principle
-- Distinguishes Neotoma from monolithic systems
-**2. Ecosystem Enablement:**
-- Enables others to build on Neotoma (Agentic Portfolio, Agentic Wallet examples)
-- Creates network effects and ecosystem value
-- Positions Neotoma as platform, not just product
-**3. Defensibility:**
-- Competitors may not design for composability (monolithic architectures)
-- Clear layer boundaries are difficult to retrofit
-- Protocol-first design is structural advantage
-**4. Strategic Positioning:**
-- "Substrate" language implies composability but doesn't explicitly state it
-- Explicit articulation clarifies value proposition
-- Distinguishes from "just another memory app"
-**5. Separation from Interoperability:**
-- Interoperability = cross-platform access (horizontal)
-- Composability = layered building (vertical)
-- Both are important but address different dimensions
-### 3.2 Arguments AGAINST Articulating Composability
-**1. Current Articulation Sufficient:**
-- "Substrate" and "foundation layer" language already implies composability
-- Layered architecture document explains mechanism
-- May be redundant with existing positioning
-**2. Not User-Facing:**
-- Composability benefits developers/ecosystem builders, not end users
-- End users care about "works with all my tools" (interoperability)
-- May confuse positioning if over-emphasized
-**3. Not Yet Proven:**
-- Strategy/Execution layers are examples, not production systems
-- Ecosystem value is potential, not realized
-- Premature to claim as differentiator
-**4. Focus Dilution:**
-- Three defensible differentiators are already clear
-- Adding fourth may dilute focus
-- Interoperability already covers "works everywhere"
-### 3.3 Recommendation: Articulate as Architectural Principle, Not Differentiator
-**Composability should be articulated as a core architectural principle** (like determinism, immutability) but **not as a separate defensible differentiator** at this stage.
-**Rationale:**
-1. **Architectural Principle:** Composability is fundamental to Neotoma's design (enables layered architecture). It belongs alongside determinism, immutability, and privacy-first in core principles.
-2. **Not Yet Differentiator:** While composability is architecturally defensible, it's not yet proven as market differentiator. Strategy/Execution layers are examples, not production systems. Ecosystem value is potential.
-3. **Complementary to Interoperability:** Interoperability (cross-platform access) is the user-facing differentiator. Composability (layered building) is the developer/ecosystem enabler. Both are important but serve different audiences.
-4. **Future Differentiator Potential:** Once Strategy/Execution layers are production systems and ecosystem is proven, composability could become explicit differentiator #4.
-## 4. How to Articulate Composability
-### 4.1 As Architectural Principle
-**Add to `docs/foundation/philosophy.md`:**
-```markdown
-## 5.8 Composability (Layered Architecture)
-Neotoma is designed as a **composable substrate** that enables multiple layers to build on top:
-- **Read-only boundaries:** Upper layers can read truth but cannot mutate it directly
-- **Domain Event → Reducer pattern:** All truth updates flow through reducers processing Domain Events
-- **Protocol-based interfaces:** MCP exposes structured, validated access points
-- **Clear layer separation:** Strategy and Execution layers are architecturally distinct from Truth Layer
-**Layer Composition:**
-- Strategy Layer (pure cognition): Reads world state, outputs Decisions + Commands
-- Execution Layer (pure effect): Takes Commands, performs side effects, emits Domain Events
-- Truth Layer (Neotoma): Processes Domain Events through reducers to update state
-**Why Composability:**
-- Enables ecosystem building (others can build on Neotoma)
-- Creates network effects and platform value
-- Distinguishes Neotoma from monolithic systems
-- Supports future protocol-first, decentralized evolution
-```
-### 4.2 Update Core Identity
-**Update `docs/foundation/core_identity.md`:**
-Add to "Core Architectural Choices (Defensible Differentiators)" section:
-```markdown
-**Architectural Principles (Enabling Differentiators):**
-4. **Composability (Layered Architecture)**
-   - Read-only boundaries enable Strategy/Execution layers to build on Truth Layer
-   - Domain Event → Reducer pattern enables predictable composition
-   - Protocol-based interfaces (MCP) enable ecosystem integration
-   - **Why Enabling:** Enables ecosystem building and platform value, distinguishes from monolithic systems
-```
-### 4.3 Update Product Positioning
-**Update `docs/foundation/product_positioning.md`:**
-Add to positioning section:
-```markdown
-**Architectural Enablers (Supporting Differentiators):**
-- **Composability:** Layered architecture enables Strategy/Execution layers to build on Truth Layer. Protocol-based interfaces enable ecosystem integration. Distinguishes Neotoma from monolithic systems.
-```
-**Distinguish from Interoperability:**
-```markdown
-**Composability vs. Interoperability:**
-- **Interoperability (Differentiator #3):** Works across multiple AI platforms (ChatGPT, Claude, Cursor) via MCP. Horizontal integration.
-- **Composability (Architectural Principle):** Enables Strategy/Execution layers to build on Truth Layer. Vertical composition. Enables ecosystem building.
-```
-### 4.4 Update README
-**Add to "Neotoma's Structured Personal Data Memory" section:**
-```markdown
-**4. Composability (Layered Architecture)**
-- Read-only boundaries enable Strategy/Execution layers to build on Truth Layer
-- Domain Event → Reducer pattern enables predictable composition
-- Protocol-based interfaces enable ecosystem integration
-- Foundation for agent-native personal computing
-```
-## 5. Implementation Recommendations
-### 5.1 Immediate Actions
-1. **Add composability to `docs/foundation/philosophy.md`** as architectural principle 5.8
-2. **Update `docs/foundation/core_identity.md`** to include composability in architectural principles
-3. **Clarify distinction** between composability and interoperability in positioning docs
-### 5.2 Future Considerations
-**When to Elevate to Differentiator:**
-- Strategy/Execution layers are production systems (not just examples)
-- Ecosystem has proven value (multiple systems building on Neotoma)
-- Market recognizes composability as competitive advantage
-- Clear evidence that competitors cannot pursue composability
-**Metrics to Track:**
-- Number of external systems building on Neotoma
-- Strategy/Execution layer production deployments
-- Ecosystem developer adoption
-- Market recognition of composability value
-## 6. Bitcoin-Like Minimalism: Architectural Evolution Analysis
-### 6.1 Bitcoin's Minimalism Principle
-Bitcoin's whitepaper solved double-spending with minimal primitives:
-1. **Proof-of-work** — computational cost to create blocks
-2. **Merkle trees** — efficient transaction verification
-3. **Timestamp server** — ordering via chain structure
-4. **Peer-to-peer network** — propagation without central authority
-5. **Digital signatures** — ownership/authorization
-**Key Insight:** Bitcoin's core protocol remained minimal. Complexity emerged in the ecosystem (exchanges, wallets, layer-2s), not the protocol itself.
-### 6.2 Neotoma's Architectural Evolution (Release History)
-**v0.1.0 (Internal MCP Release):**
-- Core primitives: Payload → Observation → Entity → Snapshot
-- Event-sourcing foundation
-- 4 merge strategies (last_write, highest_priority, most_specific, merge_array)
-- Typed relationships (7 types: PART_OF, CORRECTS, REFERS_TO, SETTLES, DUPLICATE_OF, DEPENDS_ON, SUPERSEDES)
-- Schema registry (config-driven evolution)
-- **Status:** Relatively minimal core, but already includes multiple merge strategies and typed relationships
-**v0.2.0 (Minimal Ingestion + Correction Loop):**
-- Sources-first ingestion architecture
-- Raw fragments for unknown fields
-- Interpretation runs (versioned)
-- **Status:** Focused on core loop, deferred complexity (async retry, quota tiers, schema promotion)
-**v0.3.0 (Operational Hardening):**
-- Async upload retry (upload_queue table)
-- Quota enforcement (storage_usage table)
-- Stale interpretation cleanup
-- **Status:** Adds operational complexity after core loop validated
-**v0.4.0 (Intelligence + Housekeeping):**
-- Duplicate detection worker
-- Schema promotion pipeline (analytics on raw_fragments)
-- Archival job
-- **Status:** Intelligence features deferred until operational stability proven
-**v1.0.0 (MVP):**
-- UI layer (design system)
-- Auth + RLS (user isolation)
-- **Status:** User-facing complexity added for MVP validation
-**v2.0.0 (End-to-End Encryption):**
-- Crypto library (X25519/Ed25519)
-- Local-first datastore (Browser SQLite WASM)
-- Encrypted MCP bridge
-- Migration tooling
-- Dual-mode operation (plaintext + encrypted)
-- **Status:** Massive complexity addition (13 Feature Units, 20 weeks development)
-### 6.3 Complexity Comparison: Bitcoin vs. Neotoma
-**Bitcoin's Core Protocol (Whitepaper):**
-- 5 primitives (proof-of-work, merkle trees, timestamp server, p2p network, signatures)
-- Fixed transaction format (no schema evolution)
-- Single ordering rule (chain order)
-- No merge strategies (just ordering)
-- Untyped relationships (transaction inputs/outputs)
-- No capability system (just signatures)
-**Neotoma's Current Architecture:**
-- 4-layer truth model (Payload → Observation → Entity → Snapshot)
-- 4 merge strategies (last_write, highest_priority, most_specific, merge_array)
-- 7 relationship types (PART_OF, CORRECTS, REFERS_TO, SETTLES, DUPLICATE_OF, DEPENDS_ON, SUPERSEDES)
-- Schema registry (config-driven evolution)
-- Schema promotion pipeline (automated discovery)
-- Capability tokens (versioned intents)
-- Explicit provenance tracking (field → observation → document → file)
-- E2EE architecture (v2.0.0: crypto, local-first, migration)
-### 6.4 Simplification Opportunities (Bitcoin-Like Minimalism)
-**1. Single Merge Strategy: Time-Ordered**
-- **Current:** 4 merge strategies with configurable policies
-- **Bitcoin approach:** Single ordering rule (chain order)
-- **Simplification:** Use only `last_write` (time-ordered). Remove `source_priority` and `specificity_score`. Observations ordered by `observed_at`; latest wins.
-- **Rationale:** Bitcoin doesn't merge; it orders. Time ordering is sufficient for most cases. Priority/specificity can be handled by ingestion order.
-**2. Defer Schema Promotion Pipeline**
-- **Current:** Automated schema promotion with analytics, suggestions, approvals
-- **Bitcoin approach:** Fixed transaction format (no schema evolution)
-- **Simplification:** Remove automated promotion. Keep `raw_fragments` for unknown fields. Manual schema creation only. Defer automation until needed.
-- **Rationale:** Bitcoin's format is fixed. Manual schemas are simpler and more predictable.
-**3. Untyped Relationships**
-- **Current:** 7 relationship types with typed semantics
-- **Bitcoin approach:** Single relationship type (transaction inputs/outputs)
-- **Simplification:** Use untyped edges with optional `metadata` JSONB. Semantics emerge from usage, not types.
-- **Rationale:** Bitcoin's graph is untyped. Types add complexity without clear benefit initially.
-**4. Remove Capability Tokens**
-- **Current:** `capability_id` in payloads for versioned intents
-- **Bitcoin approach:** No capability system (just signatures)
-- **Simplification:** Remove `capability_id`. Authorization handled at API layer, not in payloads.
-- **Rationale:** Bitcoin doesn't have capabilities. Keep authorization separate from data.
-**5. Implicit Provenance**
-- **Current:** Explicit provenance tracking at every level (field → observation → document → file)
-- **Bitcoin approach:** Provenance is implicit (chain structure)
-- **Simplification:** Track `source_payload_id` on observations. Provenance queries traverse payload → observation → snapshot. Remove per-field provenance maps.
-- **Rationale:** Bitcoin's provenance is implicit. Explicit tracking adds complexity; implicit is sufficient.
-**6. Hardcode Schemas Initially**
-- **Current:** Schema registry with runtime evolution
-- **Bitcoin approach:** Fixed transaction format
-- **Simplification:** Hardcode core schemas (invoice, transaction, person, etc.) in code. Add registry later if needed.
-- **Rationale:** Bitcoin's format is fixed. Hardcoded schemas are simpler and more predictable.
-### 6.5 Minimal Core Primitives (Bitcoin-Inspired)
-**Bitcoin's Primitives:**
-1. Transaction (inputs/outputs)
-2. Block (merkle tree)
-3. Chain (ordering)
-4. Signature (authorization)
-**Neotoma's Minimal Primitives (Mirroring Bitcoin):**
-1. **Payload** (unified ingestion)
-2. **Observation** (time-ordered facts)
-3. **Entity** (stable ID)
-4. **Snapshot** (latest observation wins)
-5. **Edge** (untyped relationships)
-**Remove:**
-- Multiple merge strategies → single time-ordered merge
-- Schema promotion → manual schemas
-- Typed relationships → untyped edges
-- Capability tokens → API-level auth
-- Explicit provenance → implicit via payload chain
-- Schema registry → hardcoded schemas
-**Result:** Simpler, more predictable, easier to reason about—like Bitcoin's whitepaper.
-### 6.6 Release History Insights
-**Pattern Observed:**
-- v0.1.0-v0.2.0: Relatively minimal core (focused on ingestion loop)
-- v0.3.0-v0.4.0: Operational complexity added after core validated
-- v1.0.0: User-facing complexity (UI, auth)
-- v2.0.0: Massive complexity addition (E2EE, local-first, migration)
-**Bitcoin's Pattern:**
-- Core protocol remained minimal
-- Complexity emerged in ecosystem (exchanges, wallets, layer-2s)
-- Protocol itself stayed simple
-**Recommendation:**
-- Keep core Truth Layer minimal (like Bitcoin's protocol)
-- Push complexity to upper layers (Strategy/Execution layers, like Bitcoin's ecosystem)
-- Defer features until core is validated (like v0.2.0 deferred async retry until v0.3.0)
-## 7. Summary
-**Composability is architecturally fundamental** to Neotoma's design and should be articulated as a core architectural principle alongside determinism, immutability, and privacy-first.
-**Composability is distinct from interoperability:**
-- **Interoperability** = horizontal (works across platforms) — explicit differentiator #3
-- **Composability** = vertical (layers build on top) — architectural principle
-**Bitcoin-Like Minimalism:**
-- Core Truth Layer should remain minimal (like Bitcoin's protocol)
-- Complexity belongs in upper layers (Strategy/Execution, like Bitcoin's ecosystem)
-- Defer features until core is validated (release history shows this pattern)
-**Recommendation:** Articulate composability as architectural principle now, consider elevating to differentiator once ecosystem value is proven. Apply Bitcoin-like minimalism to core Truth Layer: single merge strategy, untyped relationships, implicit provenance, hardcoded schemas initially.

package/docs/foundation/core_identity.md

@@ -1,50 +0,0 @@
-# Neotoma Core Identity
-
-## What Neotoma Is
-Neotoma is a **Truth Layer** — a privacy-first, idempotent, cross-platform personal-data substrate designed for AI-native workflows.
-It is:
-- The **lowest-level, canonical source of truth** for a user's personal and professional data (documents + agent-created data)
-- A **substrate** for AI-native personal computing
-- A **privacy-first structured memory system** that transforms fragmented personal data into structured, queryable truth via dual-path ingestion
-- The **foundation layer** beneath agent-driven layers (e.g., Strategy Layer with Agentic Portfolio as example instance, Execution Layer with Agentic Wallet as part)
-**Core Architectural Choices (Defensible Differentiators):**
-1. **Privacy-First:** User-controlled memory, no provider access, never used for training
-2. **Idempotent/Verifiable Domain (Creates Verifiable Domain for Personal Data):** Same operation → same final state, always (reproducible, explainable, no hallucinations, no duplicates). Hash-based entity IDs ensure deterministic, tamper-evident records.
-
-   **For LLM interpretation**: Model outputs vary (stochastic), but system enforces idempotence through:
-   - Canonicalization (normalize, sort, round)
-   - Hashing (create identity)
-   - Deduplication (prevent duplicates)
-
-   By making operations idempotent and verifiable, Neotoma creates objective (non-gameable) results—a verifiable domain for personal data that enables reliable, consistent results LLMs can depend on.
-3. **Cross-Platform:** Works with all AI tools via MCP (ChatGPT, Claude, Cursor), not platform-locked
-4. **Immutable Audit Trail:** Every change permanently recorded with full provenance; event-sourced architecture enables historical replay and time-travel queries
-5. **Cryptographic Integrity:** Hash-based entity IDs and event chaining ensure deterministic, tamper-evident records
-These architectural choices are defensible because competitors (model providers, OS providers, startups) cannot pursue them due to structural constraints (business model conflicts, architectural constraints, platform lock-in revenue models). See [`docs/private/competitive/defensible_differentiation_framework.md`](../private/competitive/defensible_differentiation_framework.md).
-
-### Core Responsibilities
-Neotoma focuses exclusively on:
-1. **Ingestion** — Dual-path ingestion: user-provided file uploads (explicit, never automatic) and agent interactions where users provide contextual data via MCP `ingest` action, enabling incremental memory growth as agent usage scales
-2. **Normalization** — Format conversion, text extraction, OCR (file uploads only)
-3. **Extraction** — Deterministic field extraction via rule-based parsing (file uploads) or direct property assignment (agent interactions)
-4. **Schema Assignment** — Type detection (FinancialRecord, IdentityDocument, etc.)
-5. **Observation Creation** — Granular, source-specific facts extracted from documents or provided via agent interactions
-6. **Reducer Execution** — Deterministic computation of entity snapshots from observations
-7. **Schema Registry Management** — Config-driven schema evolution and versioning
-8. **Entity Resolution** — Canonical ID generation for people, companies, locations across all personal data
-9. **Event Creation** — Timeline events from date fields across all personal data
-10. **Memory Graph Construction** — Records → Entities → Events with typed edges, relationships
-11. **Deterministic Retrieval** — Structured search and queries; optional semantic similarity for entity retrieval when text query provided
-12. **AI-Safe Access** — Truth exposure via MCP tools, enabling agents to both read and write structured memory (cross-platform: ChatGPT, Claude, Cursor)
-
-## What Neotoma Is NOT
-Neotoma is **not**:
-- An LLM agent or autonomous system
-- A productivity tool, task manager, or workflow engine
-- A note-taking system or PKM app
-- A writing assistant or browser integration
-- A calendar client or financial planner
-- A crypto wallet or general semantic search platform over arbitrary unstructured content (structured retrieval is primary; entity semantic search is an optional retrieval mode over the structured store)
-- A strategy layer (e.g., [Agentic Portfolio](../architecture/agentic_portfolio_overview.md) as an example instance for financial strategy)
-- An execution layer (e.g., [Agentic Wallet](../architecture/agentic_wallet_overview.md) as part of the execution layer for financial execution)
-**Any attempt to generate features outside this scope MUST be rejected.**