neotoma 0.3.0 → 0.3.2

package/docs/developer/mcp/instructions.md

@@ -1,71 +0,0 @@
-# MCP interaction instructions
-
-## Scope
-
-Instruction text the Neotoma MCP server sends to clients so agents know how to use MCP actions. The server loads this file at runtime; edits here take effect without code changes in `src/server.ts`. For user-facing docs see `docs/specs/MCP_SPEC.md` and `docs/developer/mcp_overview.md`.
-
-## Instructions (content sent to clients)
-
-The block below is loaded and sent to MCP clients as interaction instructions. One instruction per line.
-
-```
-Do not respond or use any other tool before completing the store call for this turn. Then follow the recipes below.
-MANDATORY: Every turn—including when the user asks for a draft, advice, or a non-storage task—perform one store call (conversation + this user message) first; responding or calling other tools before that store is forbidden.
-Before storing: run a bounded retrieval for entities implied by the message (retrieve_entity_by_identifier for names/emails/identifiers, retrieve_entities for related types e.g. task, invoice, contact). Use results when storing to link to or reuse existing records; skipping this retrieval before storing is forbidden.
-Storage (chat, attachment, entity-extraction): Recipes below are self-contained. Do not list_dir, glob, or read_file MCP tool descriptor/schema files for these flows, all parameter names and response paths are here.
-FIRST ACTIONS EVERY TURN: (1) Bounded retrieval for entities implied by the message (retrieve_entity_by_identifier for identifiers, retrieve_entities for related types). (2) One store call for the conversation and current user message (and any entities implied by the message, see unified recipe). If the user attached a file, store it in the same request when possible and link with EMBEDS (see Attachment turn recipe below). After (1) and (2), then respond or use other tools.
-Chat, attachment, and entity-extraction: do NOT list, glob, or read MCP tool descriptor/schema files. Use only the recipe below, no exploration. Tool parameters: store(entities, idempotency_key, relationships, file_path|file_content+mime_type, file_idempotency_key?); create_relationship(relationship_type, source_entity_id, target_entity_id). Response IDs: combined store → structured.entities[].entity_id and unstructured.interpretation.entities[0].entityId (camelCase). Do not open schemas to verify.
-Unified store (one call per turn): store_structured with entities = [ { entity_type: "conversation", title? }, { entity_type: "agent_message", role: "user", content: "<exact message>", turn_key: "{conversation_id}:{turn_id}" }, …optional extracted entities… ]. Always include relationship PART_OF from message (index 1) to conversation (index 0). If the message implies any entity (purchase, task, event, person, place, etc.), append one or more entities with a descriptive entity_type and whatever properties the message implies—no fixed schema; server accepts arbitrary fields. For each extracted entity (index 2, 3, …), add REFERS_TO from message (index 1) to that entity. Indices: 0=conversation, 1=message, 2+=extracted. Response: entities[0]=conversation id, [1]=message id, [2]=first extracted id, etc. idempotency_key: "conversation-{conversation_id}-{turn_id}-{suffix}" or "conversation-chat-<turn>-{suffix}". Do not call list_entity_types before storing; do not reason about schema—use descriptive entity_type and fields from the message.
-Attachment turn (exact recipe; use response paths inline, do not open tool schemas): (1) one store call with entities [ { entity_type: "conversation", title? }, { entity_type: "agent_message", role: "user", content: "<exact message or 'Attached: <filename>'", turn_key: "{conversation_id}:{turn_id}" } ], idempotency_key "conversation-{conversation_id}-{turn_id}-{timestamp_ms}", relationships [ { relationship_type: "PART_OF", source_index: 1, target_index: 0 } ], and file_path: "<user attachment path>" (resolve to absolute when possible) or file_content+mime_type. Optional file_idempotency_key: "file-<short-slug>" (e.g. file-sample-receipt-pdf). (2) create_relationship with relationship_type: "EMBEDS", source_entity_id: step1_response.structured.entities[1].entity_id, target_entity_id: step1_response.unstructured.interpretation.entities[0].entityId. Do not read or interpret file contents, server interprets when interpret=true.
-Screenshot/image: When the user provides a screenshot or image, extract structured data from the visible content (people, messages, dates, criteria, tasks, events, offers, transactions, etc.) and store it. Use the attachment turn recipe: store with entities [ conversation, message, …extracted entities ], file_path or file_content+mime_type for the image, relationships PART_OF (message→conversation), REFERS_TO (message→each extracted entity), EMBEDS (message→file entity). Extract every distinct entity visible in the image before responding.
-If host provides no conversation_id/turn_id: use idempotency_key "conversation-chat-<turn>-<timestamp_ms>" and turn_key "chat:<turn>" (e.g. chat:attach-1).
-Checklist each turn: (1) store per unified recipe (conversation + message + any extracted entities in one call) (2) if attachment: include file in the same store call, then create_relationship(EMBEDS, step1.structured.entities[1].entity_id, step1.unstructured.interpretation.entities[0].entityId) (3) then other actions and reply.
-MANDATORY in chat: every turn persist conversation and user message (and any implied entities) in one store call per unified recipe. Do not skip for greetings or minimal messages. Same turn as your response. Responding to the user before storing this turn is forbidden.
-Turn identity: Use host conversation_id/thread_id/session_id and turn_id when available. If the host does not provide them, use synthetic id (e.g. "cursor-chat") and turn index or timestamp; turn_key = "{conversation_id}:{turn_id}" or "chat:<turn>"; idempotency_key unique per turn (e.g. conversation-cursor-chat-1-<timestamp_ms> or conversation-cursor-chat-1-coffee).
-Fallback if relationships not supported: (1) store_structured with entities and idempotency_key; (2) create_relationship(PART_OF, response.entities[1].entity_id, response.entities[0].entity_id). For extracted entities, create_relationship(REFERS_TO, response.entities[1].entity_id, response.entities[N].entity_id) for each.
-create_relationship (only if not using one-call store): create_relationship(relationship_type, source_entity_id, target_entity_id). Use response.entities[N].entity_id from a prior store_structured; no tool schema read.
-Chat details: Overwriting between branches is acceptable; users can view historical turns or branches via observation history (e.g. list_observations). Reverted turns: optionally create_relationship(SUPERSEDES, new_message_id, previous_message_id). Relationship types: PART_OF, REFERS_TO, EMBEDS, SUPERSEDES, etc. (see MCP spec).
-Do not mention storage, memory, or linking unless the user asked. Do not describe internal persistence in thought or reply (e.g. "Persisting this turn, then replying", "Storing the conversation first"). When confirming something was stored, use memory-related language ("remember", "recall", "stored in memory") and include one of those phrases.
-Use MCP actions proactively. Store when the user states relevant information; store first, then respond. Do not skip store because the user did not ask to save.
-Other MCPs: When you pull data (email, calendar, search), extract and store people, companies, locations, events, tasks, notifications, device status, relationships. Create tasks for action items. Link events/tasks to locations. Do not respond with that data until storage is complete (same turn).
-Result lists from other MCPs (for example Gmail search/list endpoints): before replying with the list, store the retrieved records in Neotoma in the same turn (store each record or a complete batch snapshot with all key fields). Replying with fetched list data before this storage is forbidden.
-For list responses, do a completeness check before replying: ensure stored_count matches fetched_count for the requested result set (or explain truncation). If counts do not match, keep storing and do not return list contents yet.
-User identity: When the user provides or implies their identity (name, email, "me"/"myself"), store as contact or person in the same turn when you have enough to identify.
-Extract and store all relevant entities from the user: people, tasks, events, commitments, preferences, possessions, relationships, places. Store every distinct entity. Places: store as location/property/place and link to tasks or other entities. Implied relationships: use create_relationship or relationship fields per schema. Container+asset: EMBEDS with source=container, target=asset; if file in Neotoma then store file, create image/media entity, create_relationship(EMBEDS, container_id, asset_id); if asset elsewhere store only a reference on the container.
-Tasks: Create a task when the user expresses intent, obligation, or future action ("I need to", "remind me", deadlines). Unless they say no reminders/tasks, create task with due date if available and relate to person or entity.
-Entity types (schema-agnostic): For storage from chat, use a descriptive entity_type and whatever properties the message implies; server accepts arbitrary fields and infers schema. Do not call list_entity_types before storing. Examples of entity_type (not fixed shapes): transaction, task, event, person, contact, company, receipt, note, location, place. For other flows, list_entity_types with a keyword or store with a descriptive type.
-Conventions: Avoid get_authenticated_user unless the next action needs it. Check for existing records before storing. Include all fields from source when storing. Structured path (entities) for conversation- or tool-sourced data; omit original_filename. Unstructured path (file_content+mime_type or file_path) for user attachments or files to preserve; pass raw file, do not interpret. Omit user_id (inferred). After MCP actions, summarize entities and relationships with snapshot fields. Do not repeat the same phrase as both thought and section heading. At session start, optionally call npm_check_update with packageName "neotoma" and the client version; if updateAvailable, prompt user to upgrade.
-```
-
-## One-call chat persistence
-
-Optional `relationships` on `store_structured`: array of `{ relationship_type, source_index, target_index }` (indices into the request’s `entities` array). Server creates entities then relationships in one request. No separate `store_chat_turn` action; the generic store covers chat.
-
-## Reducing agent exploration on storage turns
-
-The instruction block is tuned so agents can do “conversation + message + store file + EMBEDS” without opening tool schemas or exploring the MCP tool set:
-
-1. **No exploration** — A single, prominent line: "do NOT list, glob, or read MCP tool descriptor/schema files" for chat and attachment flow. Use only the recipe and quick reference.
-2. **Inline response paths** — Step (3) and the checklist use literal paths (`step1.entities[1].entity_id`, `step2.interpretation.entities[0].entityId`) so the agent does not need to open schemas to find where IDs live.
-3. **Unified store shape** — One recipe: entities = [conversation, message, …optional extracted entities]; always PART_OF(message→conversation), plus REFERS_TO(message→each extracted entity) when present. Reduces branching between chat-only and entity-extraction.
-4. **Fallback ids** — One line gives a concrete fallback when the host provides no conversation_id/turn_id (`conversation-chat-<turn>-<timestamp_ms>`, `turn_key "chat:<turn>"`), so the agent does not re-derive the pattern.
-5. **One recipe, one checklist** — Attachment steps, response paths, and file_path guidance are in one recipe; the checklist references it. Duplicate "user only attached" and separate "response IDs" lines were removed to cut parsing and branching.
-
-Keeping the recipe in sync with server response shapes (e.g. `unstructured.interpretation.entities[0].entityId` camelCase) reduces overhead. Attachment turns now use one store request for structured and unstructured data, then one EMBEDS relationship call.
-
-## Reducing agent work on entity-extraction turns
-
-For turns where the user shares a fact that implies an entity (e.g. “I bought a $5 coffee at Starbucks today”), agents can otherwise do extra work: exploring tool schemas (glob/list/read of MCP descriptor files), multiple tool calls (store conversation+message, then store transaction, then create_relationship), and extra reasoning. The instruction block is tuned to avoid that:
-
-1. **No exploration** — The “do NOT list, glob, or read” line explicitly covers “entity-extraction flow” so the agent uses only the recipe and quick reference for storing extracted entities and linking them to the message, without opening tool schemas.
-2. **Unified store (one call)** — The “Entity-extraction turn (one call preferred)” recipe specifies a single `store_structured` with three entities (conversation, agent_message, extracted entity) and two relationships (PART_OF message→conversation, REFERS_TO message→extracted entity). That reduces three tool calls to one. Response paths are explicit: `Response.entities[0]`, `[1]`, `[2]`.
-3. **Schema-agnostic extracted entities** — The recipe tells the agent to use a descriptive entity_type and whatever properties the message implies (no fixed schema) and "Do not call list_entity_types before storing; do not reason about schema," removing schema lookup and canonical-shape reasoning.
-4. **create_relationship quick reference** — When a separate link is needed (e.g. multi-entity turn or fallback), one line gives the exact parameters and tells the agent to use `response.entities[N].entity_id` from a prior store; no schema read required.
-
-## Related documents
-
-- `docs/specs/MCP_SPEC.md` — Action catalog and entity type rules
-- `docs/developer/mcp_overview.md` — Overview and setup
-- `docs/developer/mcp/unauthenticated.md` — Unauthenticated instructions
-- `docs/developer/mcp/tool_descriptions.yaml` — Per-tool descriptions
-- `src/server.ts` — Loads this file via `getMcpInteractionInstructions()`

package/docs/developer/mcp/invalid_connection_id_handling.md

@@ -1,181 +0,0 @@
-# Invalid Connection ID Handling
-
-## Purpose
-
-Ensures users see a reconnection option when their configured X-Connection-Id is invalid or expired, instead of just showing "No tools, prompts, or resources".
-
-## Problem
-
-When a user has an invalid or expired X-Connection-Id in their Cursor MCP configuration:
-
-1. The MCP server initialization would catch the "connection not found" error
-2. Return an unauthenticated response (with Connect button capabilities)
-3. However, `listTools()` and `listResources()` would silently fail and return empty arrays
-4. Cursor would show "No tools, prompts, or resources" with green status
-5. User would not know they need to reconnect
-
-## Solution
-
-### Changes Made
-
-#### 1. Enhanced Error Detection in Initialize Handler
-
-**File:** `src/server.ts`
-
-**Initialize handler:**
-- Added explicit detection of "connection not found" errors
-- When invalid connection ID is detected during initialization, **throws `McpError`** to prevent successful connection
-- This causes Cursor to show an error state, allowing user to fix configuration
-- Clear error message instructs user to remove invalid X-Connection-Id from mcp.json
-
-**`listTools()` and `listResources()` handlers:**
-- Enhanced error detection and logging for invalid connection IDs
-- Do NOT throw errors (to avoid "Error - Show Output" state)
-- Rely on initialize handler to prevent connection with invalid credentials
-
-**Code pattern (initialize handler):**
-```typescript
-} catch (error: any) {
-  const isConnectionNotFound =
-    error.message?.includes("Connection not found") ||
-    error.message?.includes("connection_id") ||
-    error.code === "OAUTH_CONNECTION_NOT_FOUND";
-
-  if (isConnectionNotFound) {
-    logger.error(
-      `[MCP Server] Invalid or expired X-Connection-Id: ${connectionId}. Throwing error to prevent initialization and trigger reconnection.`
-    );
-    // Throw the error to prevent successful initialization
-    // This will cause Cursor to show error state and allow reconnection
-    throw new McpError(
-      ErrorCode.InvalidRequest,
-      "Invalid or expired connection ID. Please remove the X-Connection-Id header from your mcp.json and click Connect to authenticate.",
-      { reason: "connection_expired", connectionId }
-    );
-  }
-  // For other auth failures, return unauthenticated response
-  return this.getUnauthenticatedResponse();
-}
-```
-
-#### 2. Improved Error Messages and Logging
-
-**File:** `src/server.ts`
-
-**All handlers:**
-- Added specific detection of connection not found errors
-- Logs clear messages distinguishing invalid connections from other auth failures
-- Initialize handler throws error to prevent connection
-- listTools/listResources log but don't throw (to avoid double error states)
-
-#### 3. Updated Unauthenticated Messages
-
-**File:** `docs/developer/mcp/unauthenticated.md`
-
-**Before:**
-- "Authentication needed. Use the Connect button to sign in."
-
-**After:**
-- "Authentication needed or expired. Use the Connect button to sign in."
-- Longer description mentions expired X-Connection-Id specifically
-
-## User Experience Flow
-
-### Before Changes
-
-1. User has invalid X-Connection-Id in mcp.json
-2. Cursor shows "neotoma" with green status
-3. Shows "No tools, prompts, or resources"
-4. No indication of what's wrong or how to fix
-
-### After Changes
-
-1. User has invalid X-Connection-Id in mcp.json
-2. Initialize detects invalid connection and throws `McpError`
-3. Cursor shows error state: "Invalid or expired connection ID. Please remove the X-Connection-Id header from your mcp.json and click Connect to authenticate."
-4. User removes invalid X-Connection-Id from `.cursor/mcp.json`
-5. User restarts Cursor or waits for auto-reconnection
-6. Cursor shows Connect button
-7. User clicks Connect to complete OAuth flow and get new valid connection ID
-
-## Testing
-
-### Manual Test
-
-1. Configure invalid X-Connection-Id in `.cursor/mcp.json`:
-   ```json
-   {
-     "mcpServers": {
-       "neotoma": {
-         "url": "http://localhost:8080/mcp",
-         "headers": {
-           "X-Connection-Id": "invalid-connection-id-12345"
-         }
-       }
-     }
-   }
-   ```
-
-2. Restart Cursor
-3. Check MCP server status - should show error with Connect button
-4. Click Connect to re-authenticate
-5. Should complete OAuth flow and get new valid connection ID
-
-### Expected Behavior
-
-- Invalid connection ID triggers initialization failure
-- Cursor shows error state with clear message instructing user to remove X-Connection-Id
-- User manually removes invalid header from `.cursor/mcp.json`
-- After removal and Cursor restart, Connect button appears
-- User completes OAuth flow to get new valid connection ID
-- Tools/resources appear normally after successful authentication
-
-## Error Codes
-
-The solution detects connection not found errors by checking:
-
-1. Error message contains "Connection not found"
-2. Error message contains "connection_id"
-3. Error code is "OAUTH_CONNECTION_NOT_FOUND"
-
-This matches the errors thrown by `getAccessTokenForConnection()` in `src/services/mcp_oauth.ts`.
-
-## Related Files
-
-- `src/server.ts` - Main MCP server implementation
-- `src/services/mcp_oauth.ts` - OAuth connection management
-- `docs/developer/mcp/unauthenticated.md` - Unauthenticated state messages
-- `.cursor/mcp.json` - User's MCP configuration
-
-## How to Fix Invalid Connection ID
-
-When you see the error "Invalid or expired connection ID" in Cursor:
-
-1. **Open your MCP configuration:**
-   - Location: `~/.cursor/mcp.json` (or `.cursor/mcp.json` in your workspace)
-
-2. **Remove or comment out the invalid X-Connection-Id:**
-   ```json
-   {
-     "mcpServers": {
-       "neotoma": {
-         "url": "http://localhost:8080/mcp",
-         "headers": {
-           // "X-Connection-Id": "invalid-id-here"  // Remove this line
-         }
-       }
-     }
-   }
-   ```
-
-3. **Restart Cursor** or wait for it to auto-reconnect
-
-4. **Click the Connect button** when it appears
-
-5. **Complete the OAuth flow** to get a new valid connection ID
-
-The new connection ID will be automatically saved and you won't need to manually configure it.
-
-## Related Issues
-
-This fixes the issue where users with expired connection IDs would see "No tools, prompts, or resources" with no clear indication of what was wrong or how to fix it.

package/docs/developer/mcp/tool_descriptions.yaml

@@ -1,41 +0,0 @@
-# Per-tool descriptions for MCP ListTools response.
-# Loaded at runtime by src/server.ts. Edit here to change tool copy without code changes.
-# Keys must match tool names in server.ts. Parameter descriptions remain in code.
-
-tools:
-  retrieve_file_url: "Retrieve a signed URL for accessing a file"
-  retrieve_entity_snapshot: "Retrieve the current snapshot of an entity with provenance information. Supports historical snapshots via 'at' parameter."
-  list_observations: "List all observations for a given entity"
-  retrieve_field_provenance: "Retrieve the provenance chain for a specific field in an entity snapshot"
-  create_relationship: |
-    Create a typed relationship between two entities. relationship_type: PART_OF, CORRECTS, REFERS_TO, SETTLES, DUPLICATE_OF, DEPENDS_ON, SUPERSEDES, or EMBEDS. Use EMBEDS when a container entity (e.g. blog post, document) embeds an asset entity (e.g. image, attachment): source_entity_id = container, target_entity_id = asset. For images/files stored in Neotoma: store the file via store (get source_id), create an image/media entity with source_id, then create_relationship(EMBEDS, post_entity_id, image_entity_id). Optional metadata: caption, order.
-  list_relationships: "List relationships for an entity"
-  get_relationship_snapshot: "Get the current snapshot of a specific relationship with provenance"
-  retrieve_entities: "Query entities with filters (type, pagination). Returns entities with their snapshots. Supports optional search param for semantic similarity when embeddings configured."
-  list_timeline_events: "Query timeline events with filters (type, date range, source). Returns chronological events derived from date fields in sources."
-  retrieve_entity_by_identifier: "Retrieve entity by identifier (name, email, etc.) across entity types or specific type. Falls back to semantic search when keyword match returns no results."
-  retrieve_related_entities: "Retrieve entities connected to a given entity via relationships. Supports n-hop traversal."
-  retrieve_graph_neighborhood: "Retrieve complete graph neighborhood around a node (entity or source): related entities, relationships, sources, and events."
-  store: |
-    Unified storing for structured, unstructured, or combined payloads in one request. Choose path by source: file- or resource-sourced (attachment/file to preserve) → use file_content+mime_type or file_path; conversation- or tool-sourced (chat or other MCP) → use entities. You may send both entities and file input in the same call. In chat contexts, use store to persist conversation + turn message (and optional extracted entities) plus attachment file when present. Overwriting between branches is OK, history via list_observations. For unstructured (files): provide EITHER file_content (base64-encoded) + mime_type OR file_path (local file path). For structured (entities): provide entities array. When both are provided, response includes structured and unstructured sections. Content-addressed storage uses SHA-256 deduplication per user. IMPORTANT FOR UNSTRUCTURED FILES: Agents MUST NOT interpret or extract data before storing; pass raw file content/path only. The server handles interpretation when interpret=true (default). Re-storing the same file (same content or same idempotency_key) with interpret=true runs interpretation again (reinterpret); agents do not need to call reinterpret separately. IMPORTANT FOR STRUCTURED DATA: Include ALL fields from source data. Schema fields go to observations; non-schema fields go to raw_fragments for future schema expansion.
-  store_structured: "Store structured entities only. Use for conversation- or tool-sourced data (e.g. from chat or another MCP). In chat: one call per turn with entities [conversation, agent_message], idempotency_key (per-store unique, e.g. conversation-{conversation_id}-{turn_id}-{timestamp_ms}), stable turn identity in agent_message (e.g. turn_key or id = conversation_id:turn_id), and relationships: [{ relationship_type: PART_OF, source_index: 1, target_index: 0 }]. Overwriting between branches is OK; history via list_observations. Use when you already have entity objects and do not need file ingestion."
-  store_unstructured: "Store raw files only. Use when data is file- or resource-sourced (user attachment or file to preserve). Provide file_content (base64) + mime_type or file_path. Re-storing the same file (same content or same idempotency_key) with interpret=true runs interpretation again (reinterpret); agents do not need to call reinterpret separately."
-  reinterpret: "Re-run AI interpretation on an existing source with new config. Creates new observations without modifying existing ones."
-  interpret_uninterpreted: "Interpret stored sources that currently have no interpretation runs. Supports limit and dry_run for safe backfills."
-  correct: "Create high-priority correction observation to override AI-extracted fields. Corrections always win in snapshot computation."
-  merge_entities: "Merge duplicate entities. Rewrites observations from source entity to target entity and marks source as merged."
-  delete_entity: "Delete an entity. Creates a deletion observation so the entity is excluded from snapshots and queries. Immutable and reversible for audit; use for user-initiated or GDPR-style removal from active use."
-  delete_relationship: "Delete a relationship. Creates a deletion observation so the relationship is excluded from snapshots and queries. Immutable and reversible for audit."
-  restore_entity: "Restore a deleted entity. Creates a restoration observation (priority 1001) that overrides the deletion. Entity becomes visible in snapshots and queries again. Immutable restoration for audit."
-  restore_relationship: "Restore a deleted relationship. Creates a restoration observation (priority 1001) that overrides the deletion. Relationship becomes visible in snapshots and queries again. Immutable restoration for audit."
-  list_entity_types: |
-    List available entity types. Without keyword: returns a short summary (entity_type, schema_version, field_count) for all types. With keyword: returns full schema (field_names, field_summary) for matching types; pass summary: true to get summary only. Uses hybrid search when keyword is provided: keyword matching first, then vector semantic search. Use before storing structured data to determine the correct entity_type.
-  analyze_schema_candidates: |
-    Analyze raw_fragments to identify fields that should be promoted to schema fields. Returns recommendations with confidence scores based on frequency and type consistency.
-  get_schema_recommendations: "Get schema update recommendations for an entity type from raw_fragments analysis, agent suggestions, or inference."
-  update_schema_incremental: |
-    Incrementally update a schema by adding new fields from raw_fragments or agent recommendations. Creates new schema version and activates it immediately, so all new data stored after this call will use the updated schema. Optionally migrates existing raw_fragments to observations for historical data backfill.
-  register_schema: "Register a new schema or schema version. Supports both global and user-specific schemas."
-  get_authenticated_user: "Get the authenticated user ID for the current MCP session. Returns the user_id that is automatically used for all authenticated actions."
-  health_check_snapshots: "Check for stale entity snapshots (snapshots with observation_count=0 but observations exist). Returns health status and count of stale snapshots."
-  npm_check_update: "Check if a newer npm version is available. Returns updateAvailable, message, and suggestedCommand. Call at session start to encourage user to upgrade."

package/docs/developer/mcp/unauthenticated.md

@@ -1,30 +0,0 @@
-# Unauthenticated-state instructions
-
-## Scope
-
-Text the MCP server returns when the client is not authenticated. The server loads this file at runtime and uses it in the Initialize response (instructions and serverInfo.description).
-
-## Purpose
-
-Single source for unauthenticated messaging so copy can be changed without editing `src/server.ts`.
-
-## Instructions (short line shown to user)
-
-The following block is returned as `instructions` in the Initialize response when auth is required.
-
-```
-Authentication needed or expired. Use the Connect button to sign in.
-```
-
-## Server info description (longer text for serverInfo.description)
-
-The following block is returned as `serverInfo.description` when auth is required.
-
-```
-Authentication needed or expired. If you have an X-Connection-Id configured, it may be invalid or expired. Use the Connect button to sign in again and access tools.
-```
-
-## Related documents
-
-- `docs/developer/mcp/instructions.md` — Authenticated-state instructions
-- `src/server.ts` — Loads this file in unauthenticated Initialize branch

package/docs/developer/mcp_chatgpt_setup.md

@@ -1,159 +0,0 @@
-# ChatGPT Custom GPT Setup Guide
-
-This guide explains how to configure Neotoma as a Custom GPT Action in ChatGPT using the HTTP Actions API (OpenAPI schema).
-
-For other integrations, see:
-- [`mcp_cursor_setup.md`](mcp_cursor_setup.md) - Cursor integration
-- [`mcp_claude_code_setup.md`](mcp_claude_code_setup.md) - Claude Code localhost agent setup
-## Prerequisites
-1. **ChatGPT Plus/Pro/Team/Enterprise account** (required for Custom GPT creation)
-2. **Deployed Neotoma HTTP server** (local development or production)
-3. **OpenAPI schema** (`openapi.yaml`) available via URL or file
-## Step 1: Start the HTTP Actions Server
-The Neotoma HTTP server exposes REST endpoints that ChatGPT can call as Actions.
-### Option A: Local Development (using ngrok or similar tunnel)
-1. **Build the project:**
-   ```bash
-   npm run build:server
-   ```
-2. **Start the HTTP server:**
-   ```bash
-   npm run dev:server
-   ```
-   Should see: `HTTP Actions listening on :8080`
-3. **Create a tunnel** to expose localhost (required for ChatGPT to access):
-   **Using ngrok:**
-   ```bash
-   ngrok http 8080
-   ```
-   Copy the HTTPS URL (e.g., `https://abc123.ngrok.io`)
-### Option B: Production Deployment
-Deploy the HTTP server to a cloud provider (see `docs/infrastructure/deployment.md`). Ensure:
-- Server is accessible via HTTPS
-- Environment variables are configured (see [Getting started](getting_started.md))
-- Server is running on the configured port
-## Step 2: Configure Authentication
-Neotoma uses bearer token authentication. You have two options:
-### Option A: Simple Bearer Token (Development)
-1. **Set a bearer token** in your environment:
-   ```bash
-   export ACTIONS_BEARER_TOKEN=your-secret-token-here
-   ```
-2. **Use this token** as the API key in Custom GPT configuration
-### Option B: Ed25519 Public Key (Production)
-Neotoma supports encrypted responses using Ed25519 key pairs:
-1. **Generate keys** (if not already present, server generates on startup)
-2. **Get the public key** from server logs or `/keys/public` endpoint
-3. **Base64url-encode the public key** for use as bearer token
-## Step 3: Create Custom GPT
-1. **Go to ChatGPT** and click "Explore GPTs" → "Create"
-2. **Configure Basic Settings:**
-   - **Name:** "Neotoma Memory Assistant" (or your preference)
-   - **Description:** "Interact with your personal Neotoma memory graph"
-   - **Instructions:** Add guidance like:
-     ```
-     You are a Neotoma memory assistant. Help users store, retrieve, and query their personal data using Neotoma actions.
-     - Use store_record to save structured data from conversations
-     - Use retrieve_records to search and filter records
-     - Use retrieve_entities to find entities (companies, people, etc.)
-     - Use get_graph_neighborhood to explore relationships
-     ```
-3. **Add Actions:**
-   - Click "Create new action"
-   - Click "Import from URL" or "Upload file"
-   - **Option 1:** Import from URL (Recommended):
-     - URL: `https://your-server-url/openapi.yaml`
-     - The server serves the OpenAPI schema at `/openapi.yaml` endpoint
-     - Example: If using ngrok with URL `https://abc123.ngrok.io`, use `https://abc123.ngrok.io/openapi.yaml`
-     - Must be publicly accessible via HTTPS
-   - **Option 2:** Upload file:
-     - Upload `openapi.yaml` from the project root (`/path/to/neotoma/openapi.yaml`)
-4. **Configure Authentication:**
-   - **Type:** API Key
-   - **Name:** `Authorization`
-   - **Location:** Header
-   - **Prefix:** `Bearer `
-   - **Value:** Your bearer token (from Step 2)
-5. **Save the Custom GPT**
-## Step 4: Test the Integration
-1. **Open your Custom GPT** in ChatGPT
-2. **Test storing a record:**
-   ```
-   Store a note about our meeting today: "Reviewed Q4 budget with team"
-   ```
-3. **Test retrieving records:**
-   ```
-   Show me all notes from the past week
-   ```
-4. **Test entity queries:**
-   ```
-   Find all companies mentioned in my records
-   ```
-## Available Actions for Custom GPT
-The OpenAPI schema includes these actions:
-- **Record Operations:** `store_record`, `retrieve_records`, `update_record`, `delete_record`
-- **File Operations:** `upload_file`, `get_file_url`
-- **Entity Operations:** `retrieve_entities`, `get_entity_by_identifier`, `get_entity_snapshot`, `list_observations`
-- **Relationship Operations:** `create_relationship`, `list_relationships`, `get_related_entities`
-- **Timeline Operations:** `list_timeline_events`
-- **Graph Operations:** `get_graph_neighborhood`
-## Troubleshooting
-### Issue: "Failed to import schema" or "Invalid OpenAPI schema"
-**Solutions:**
-- Ensure `openapi.yaml` is valid OpenAPI 3.1.0
-- Check that the schema URL is publicly accessible
-- Verify HTTPS is enabled (ChatGPT requires HTTPS)
-- Try uploading the file directly instead of using URL import
-### Issue: "Authentication failed" or "401 Unauthorized"
-**Solutions:**
-- Verify bearer token matches server configuration
-- Check that `ACTIONS_BEARER_TOKEN` is set on the server (or use Ed25519 keys)
-- Ensure token is prefixed with "Bearer " in Custom GPT settings
-- Verify server is running and accessible
-### Issue: "Connection refused" or "Cannot reach server"
-**Solutions:**
-- Verify server is running: `curl http://localhost:8080/health`
-- Check firewall rules allow incoming connections
-- For local development, ensure tunnel (ngrok) is running
-- Verify HTTPS endpoint is accessible from the internet
-### Issue: Actions appear but don't work
-**Solutions:**
-- Check server logs for errors
-- Verify environment and storage are configured
-- Test endpoints directly: `curl -H "Authorization: Bearer YOUR_TOKEN" https://your-server/store_record`
-- Review OpenAPI schema matches server implementation
-## Differences: Cursor vs ChatGPT
-| Feature        | Cursor (MCP)                 | ChatGPT (HTTP Actions) |
-| -------------- | ---------------------------- | ---------------------- |
-| Protocol       | stdio (JSON-RPC)             | HTTP REST (OpenAPI)    |
-| Transport      | stdin/stdout                 | HTTPS                  |
-| Server mode    | `npm run dev` or `npm run start:mcp` | `npm run dev:server` |
-| Schema         | MCP tool definitions         | OpenAPI 3.1.0          |
-| Authentication | N/A (local process)          | Bearer token required  |
-| Deployment     | Local only                   | Local or production    |
-For other MCP integrations, see:
-- [`mcp_cursor_setup.md`](mcp_cursor_setup.md) - Cursor integration
-- [`mcp_claude_code_setup.md`](mcp_claude_code_setup.md) - Claude Code localhost agent setup
-## Quick Reference
-```bash
-# Build server
-npm run build:server
-# Start HTTP server
-npm run dev:server
-# Set environment variables
-export ACTIONS_BEARER_TOKEN=your-token-here
-# Create tunnel (local development)
-ngrok http 8080
-# Test health endpoint
-curl http://localhost:8080/health
-# Test with authentication
-curl -H "Authorization: Bearer your-token" http://localhost:8080/health
-# Access OpenAPI schema (for Custom GPT import)
-curl http://localhost:8080/openapi.yaml
-```
-## Additional Resources
-- **MCP Specification:** `docs/specs/MCP_SPEC.md`
-- **OpenAPI Schema:** `openapi.yaml` in project root
-- **HTTP Actions Server:** `src/actions.ts`
-- **Getting Started Guide:** `docs/developer/getting_started.md`
-- **Cursor MCP Setup:** `docs/developer/mcp_cursor_setup.md`