neotoma 0.11.0 → 0.12.0

package/README.md

@@ -46,13 +46,14 @@ graph LR
 - **Replayable.** Inspect any entity at any point in time. Diff versions. Reconstruct history from the observation log.
 - **Structure-first.** Schema-first extraction with deterministic retrieval. Optional similarity search when embeddings are configured.
 
-### Three foundations
+### Four foundations
 
-| Foundation         | What it means                                                                                                                           |
-| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------- |
-| **Privacy-first**  | Your data stays local. Never used for training. User-controlled storage, optional encryption at rest. Full export and deletion control. |
-| **Deterministic**  | Same input always produces same output. Schema-first extraction, hash-based entity IDs, full provenance. No silent mutation.            |
-| **Cross-platform** | One memory graph across Claude, ChatGPT, Cursor, OpenClaw, IronClaw, Codex, and CLI. MCP-based access. No platform lock-in. Works alongside native memory. |
+| Foundation                  | What it means                                                                                                                                              |
+| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Privacy-first**           | Your data stays local. Never used for training. Nothing is stored unless you approve it; no background scanning or implicit captures. Full export and deletion control. |
+| **Deterministic**           | Same input always produces same output. Schema-first extraction, hash-based entity IDs, full provenance. No silent mutation.                                |
+| **Immutable and verifiable** | Append-only observations; history cannot be rewritten. Hash-based entity IDs are tamper-evident. Full provenance chain from any state to its source.       |
+| **Cross-platform**          | One memory graph across Claude, ChatGPT, Cursor, OpenClaw, IronClaw, Codex, and CLI. MCP-based access. No platform lock-in. Works alongside native memory. |
 
 ## State guarantees
 
@@ -72,6 +73,8 @@ Most AI memory systems optimize storage or retrieval. Neotoma enforces state int
 | Zero-setup onboarding                | ✓         | ✗               | ✗          | ✗             | ✗             |
 | Semantic similarity search           | ✗         | ✓               | ✗          | ✗             | ✓             |
 | Direct human editability             | ✗         | ✗               | ✓          | ✗             | ✗             |
+| Strong consistency                   | ✗         | ✗               | ✗          | ✓             | ✓             |
+| Transactional writes                 | ✗         | ✗               | ✗          | ✓             | ✓             |
 
 **Platform:** Claude, ChatGPT, Gemini, Copilot. **Retrieval:** Mem0, Zep, LangChain Memory. **Files:** Markdown files, JSON stores, CRDT docs. **Database:** SQLite, Postgres, MySQL. **Neotoma:** Deterministic state layer (reference implementation).
 
@@ -116,6 +119,26 @@ neotoma upload ./invoice.pdf
 
 Results reflect versioned entity state with full provenance. Agents perform the same operations through MCP tool calls (`store`, `retrieve_entities`, `retrieve_entity_by_identifier`).
 
+## Available skills
+
+Skills are guided workflows that teach your AI agent to import, extract, and persist data into Neotoma memory. They ship with the npm package and are installed by `neotoma setup`.
+
+| Skill | Description |
+|-------|-------------|
+| **ensure-neotoma** | Install Neotoma, configure MCP, verify connectivity. Prerequisite for all other skills. |
+| **remember-email** | Configure email MCP, import emails, extract contacts, tasks, events, transactions. |
+| **remember-conversations** | Import ChatGPT/Claude/Slack exports, reconstruct decision timeline. |
+| **remember-meetings** | Ingest meeting transcripts, extract decisions and action items. |
+| **remember-finances** | Import bank statements, receipts, invoices. Extract structured transactions. |
+| **remember-contacts** | Consolidate contacts from email, calendar, chat, vCards, LinkedIn. |
+| **remember-calendar** | Configure calendar MCP, import events and commitments. |
+| **remember-codebase** | Developer repo integration — inventory, architecture decisions, MCP wiring. |
+| **store-data** | Generic: persist any structured data or file with provenance. |
+| **query-memory** | Generic: retrieve what your agent knows about anything. |
+| **recover-sqlite-database** | Troubleshooting: check integrity and recover corrupted Neotoma database. |
+
+[Full skill documentation →](https://neotoma.io/skills) | [Skill strategy →](docs/skills/skill_strategy.md)
+
 ## Interfaces
 
 Three interfaces. One state invariant. Every interface provides the same deterministic behavior regardless of how you access the state layer.

package/dist/actions.d.ts

@@ -1,18 +1,14 @@
 import express from "express";
+import { type GuestIdentity } from "./services/access_policy.js";
+import { type AgentIdentity } from "./crypto/agent_identity.js";
 import { type StoreInterpretationInput } from "./shared/action_schemas.js";
 export declare const app: import("express-serve-static-core").Express;
 /**
- * True when the request arrived over a loopback socket.
+ * True when the request is genuinely local to this process.
  *
- * SECURITY: derived from the TCP socket's remote address, NOT the `Host`
- * header. `req.headers.host` is attacker-controlled; using it to gate
- * authentication / auto-approval produces a trivial bypass when the server is
- * bound to a non-loopback interface. We check `req.socket.remoteAddress`
- * directly so spoofed `Host: localhost` headers do not promote a remote
- * caller into the local-dev trust zone.
- *
- * Express's `req.ip` is also unsafe here because `trust proxy` honours the
- * X-Forwarded-For header — any caller can claim to be loopback.
+ * SECURITY: a same-host reverse proxy (Caddy, nginx, Cloudflare tunnel, etc.)
+ * connects to Node over loopback even for public internet callers. In
+ * production, loopback alone is therefore not enough to grant local-dev auth.
  */
 export declare function isLocalRequest(req: express.Request): boolean;
 /**
@@ -23,6 +19,41 @@ export declare function isLocalRequest(req: express.Request): boolean;
  * a different authority. See src/middleware/aauth_verify.ts.
  */
 export declare function canonicalAauthAuthority(): string;
+type UserPrincipal = {
+    kind: "user";
+    userId: string;
+};
+type GuestPrincipal = {
+    kind: "guest";
+    guestId: GuestIdentity;
+    agentIdentity?: AgentIdentity | null;
+    accessToken?: string;
+};
+type RoutePrincipal = UserPrincipal | GuestPrincipal;
+/** Exported for unit tests: routes where guest (AAuth / guest token) may be stamped before handlers run. */
+export declare function routeAcceptsGuestPrincipal(req: Pick<express.Request, "method" | "path">): boolean;
+/** Exported for unit tests: guest-capable routes that mutate state and should consume a guest write-rate budget. */
+export declare function routeConsumesGuestWriteBudget(req: Pick<express.Request, "method" | "path">): boolean;
+/** Exported for unit tests: guest-rate-limit key precedence is thumbprint -> guest token -> IP. */
+export declare function guestWriteRateLimitKey(req: express.Request): string;
+/**
+ * Derive a userId for a guest principal from the validated token grant.
+ *
+ * Resolution order:
+ *   1. `(req as any).authenticatedUserId` — set by earlier auth middleware
+ *      (covers the rare case where user-level auth was stamped before the
+ *      guest branch).
+ *   2. The `user_id` stored on the `guest_access_token` entity that matches
+ *      the guest's access token. This is the owner who generated the token.
+ *   3. Local-only fallback: when the request originates from localhost and
+ *      does not carry a Bearer token, use the local dev user. This keeps
+ *      local-only developer flows working without a real token grant.
+ *
+ * Returns `null` for non-guest principals so callers can fall through to
+ * `getAuthenticatedUserId`. Throws for remote/hosted guests that cannot
+ * be resolved — prevents silent mis-attribution.
+ */
+export declare function resolveGuestUserId(req: express.Request, principal: RoutePrincipal): Promise<string | null>;
 type StoreRelationshipRef = {
     relationship_type: string;
     source_index?: number;
@@ -36,6 +67,8 @@ export declare function storeStructuredForApi(params: {
     entities: Record<string, unknown>[];
     sourcePriority: number;
     observationSource?: import("./shared/action_schemas.js").ObservationSource;
+    /** When set, written observations carry this peer id for sync loop prevention. */
+    sourcePeerId?: string | null;
     idempotencyKey: string;
     originalFilename?: string;
     relationships?: StoreRelationshipRef[];

package/dist/actions.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"actions.d.ts","sourceRoot":"","sources":["../src/actions.ts"],"names":[],"mappings":"AAAA,OAAO,OAAO,MAAM,SAAS,CAAC;AA4G9B,OAAO,EAyBL,KAAK,wBAAwB,EAE9B,MAAM,4BAA4B,CAAC;AA+BpC,eAAO,MAAM,GAAG,6CAAY,CAAC;AAygB7B;;;;;;;;;;;;GAYG;AACH,wBAAgB,cAAc,CAAC,GAAG,EAAE,OAAO,CAAC,OAAO,GAAG,OAAO,CAU5D;AA4PD;;;;;;GAMG;AACH,wBAAgB,uBAAuB,IAAI,MAAM,CAmBhD;AAiyHD,KAAK,oBAAoB,GAAG;IAC1B,iBAAiB,EAAE,MAAM,CAAC;IAC1B,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACpC,CAAC;AAuDF,wBAAsB,qBAAqB,CAAC,MAAM,EAAE;IAClD,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,EAAE,CAAC;IACpC,cAAc,EAAE,MAAM,CAAC;IACvB,iBAAiB,CAAC,EAAE,OAAO,4BAA4B,EAAE,iBAAiB,CAAC;IAC3E,cAAc,EAAE,MAAM,CAAC;IACvB,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,aAAa,CAAC,EAAE,oBAAoB,EAAE,CAAC;IACvC,cAAc,CAAC,EAAE,wBAAwB,CAAC;IAC1C,sBAAsB,CAAC,EAAE,MAAM,CAAC;IAChC,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,MAAM,CAAC,EAAE,OAAO,CAAC;CAClB;;;;;;;;;;;;;cA+gBS,MAAM;gBACJ,MAAM;2BACK,MAAM;qBACZ,MAAM;mBACR,MAAM;wBACD,MAAM;uBACP,MAAM;;;;;mBA3JV,MAAM;qBACJ,MAAM;wBACH,MAAM,GAAG,IAAI;2BACV,MAAM;;wBAET,MAAM;uBACP,MAAM,EAAE;wBACP,MAAM;uBACP,MAAM;;kBA9NX,MAAM;oBACJ,MAAM;yBACD,MAAM;4BACH,MAAM;2BACP,MAAM;;+BA4NF,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAAG,IAAI;;;2BAkFlC,MAAM;0BACP,MAAM;0BACN,MAAM;;;;;;;;GAsG3B;AAmhED,wBAAsB,eAAe;;;eA8FpC"}
+{"version":3,"file":"actions.d.ts","sourceRoot":"","sources":["../src/actions.ts"],"names":[],"mappings":"AAAA,OAAO,OAAO,MAAM,SAAS,CAAC;AAqC9B,OAAO,EAGL,KAAK,aAAa,EACnB,MAAM,6BAA6B,CAAC;AAarC,OAAO,EAGL,KAAK,aAAa,EAGnB,MAAM,4BAA4B,CAAC;AA8DpC,OAAO,EAgCL,KAAK,wBAAwB,EAE9B,MAAM,4BAA4B,CAAC;AAgCpC,eAAO,MAAM,GAAG,6CAAY,CAAC;AAkjB7B;;;;;;GAMG;AACH,wBAAgB,cAAc,CAAC,GAAG,EAAE,OAAO,CAAC,OAAO,GAAG,OAAO,CAa5D;AA4PD;;;;;;GAMG;AACH,wBAAgB,uBAAuB,IAAI,MAAM,CA2BhD;AAo4CD,KAAK,aAAa,GAAG;IACnB,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,MAAM,CAAC;CAChB,CAAC;AAEF,KAAK,cAAc,GAAG;IACpB,IAAI,EAAE,OAAO,CAAC;IACd,OAAO,EAAE,aAAa,CAAC;IACvB,aAAa,CAAC,EAAE,aAAa,GAAG,IAAI,CAAC;IACrC,WAAW,CAAC,EAAE,MAAM,CAAC;CACtB,CAAC;AAEF,KAAK,cAAc,GAAG,aAAa,GAAG,cAAc,CAAC;AA2CrD,4GAA4G;AAC5G,wBAAgB,0BAA0B,CAAC,GAAG,EAAE,IAAI,CAAC,OAAO,CAAC,OAAO,EAAE,QAAQ,GAAG,MAAM,CAAC,GAAG,OAAO,CAqBjG;AAED,oHAAoH;AACpH,wBAAgB,6BAA6B,CAC3C,GAAG,EAAE,IAAI,CAAC,OAAO,CAAC,OAAO,EAAE,QAAQ,GAAG,MAAM,CAAC,GAC5C,OAAO,CAWT;AAED,mGAAmG;AACnG,wBAAgB,sBAAsB,CAAC,GAAG,EAAE,OAAO,CAAC,OAAO,GAAG,MAAM,CAanE;AA2FD;;;;;;;;;;;;;;;;GAgBG;AACH,wBAAsB,kBAAkB,CACtC,GAAG,EAAE,OAAO,CAAC,OAAO,EACpB,SAAS,EAAE,cAAc,GACxB,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC,CA4BxB;AAuvFD,KAAK,oBAAoB,GAAG;IAC1B,iBAAiB,EAAE,MAAM,CAAC;IAC1B,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACpC,CAAC;AAuDF,wBAAsB,qBAAqB,CAAC,MAAM,EAAE;IAClD,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,EAAE,CAAC;IACpC,cAAc,EAAE,MAAM,CAAC;IACvB,iBAAiB,CAAC,EAAE,OAAO,4BAA4B,EAAE,iBAAiB,CAAC;IAC3E,kFAAkF;IAClF,YAAY,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC7B,cAAc,EAAE,MAAM,CAAC;IACvB,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,aAAa,CAAC,EAAE,oBAAoB,EAAE,CAAC;IACvC,cAAc,CAAC,EAAE,wBAAwB,CAAC;IAC1C,sBAAsB,CAAC,EAAE,MAAM,CAAC;IAChC,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,MAAM,CAAC,EAAE,OAAO,CAAC;CAClB;;;;;;;;;;;;;cAksBS,MAAM;gBACJ,MAAM;2BACK,MAAM;qBACZ,MAAM;mBACR,MAAM;wBACD,MAAM;uBACP,MAAM;;;;;mBArNV,MAAM;qBACJ,MAAM;wBACH,MAAM,GAAG,IAAI;2BACV,MAAM;;wBAET,MAAM;uBACP,MAAM,EAAE;wBACP,MAAM;uBACP,MAAM;;kBAhUX,MAAM;oBACJ,MAAM;yBACD,MAAM;4BACH,MAAM;2BACP,MAAM;;+BA8TF,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAAG,IAAI;;;2BA8HlC,MAAM;0BACP,MAAM;0BACN,MAAM;;;;;;;;GAoH3B;AAg7FD,wBAAsB,eAAe;;;eA+HpC"}