dotmem 0.0.5 → 0.0.8

package/README.md

@@ -0,0 +1,417 @@
+# dotmem (.mem)
+
+![banner](https://raw.githubusercontent.com/passariello/container/refs/heads/main/dotmem/banner.svg)
+
+AI memory and continuity infrastructure for agents and applications.
+
+**Built on:** [memorio](https://www.npmjs.com/package/memorio) (state, store, session, IndexedDB, SQLite, cache)
+
+Dotmem provides structured, queryable, and policy-controlled memory for AI agents. It supports experience acquisition, bounded context projection, continuity across sessions, rule-based reasoning, and MCP integration. The memory substrate is independent of a specific model, vendor, host or conversational client.
+
+> **Retrieval model**: dotmem uses deterministic lexical/structured retrieval with applicability-based eligibility. It does not use embeddings, vector indexes, or semantic similarity search. The term "AI-oriented" refers to the memory lifecycle and API design (acquisition with evidence/provenance, applicability scoping, policy-gated writes, context projection for LLM consumption), not the underlying retrieval algorithm. For semantic/vector search, pair dotmem with a dedicated embedding store.
+
+---
+
+## Installation
+
+```bash
+npm i dotmem
+# or
+bun i dotmem
+# or
+pnpm i dotmem
+```
+
+---
+
+## Quick Start
+
+```typescript
+import { memory } from 'dotmem'
+
+// Discover relevant acquired experience for a situation
+const discovery = await memory.discover('User needs a language recommendation for a new project', {
+  depth: 'current',
+  maxCandidates: 10
+})
+
+// Project a bounded context from discovered claims
+const projection = memory.project(discovery, { maxCandidates: 5, maxCharacters: 4000 })
+console.info(projection.text)
+
+// Acquire durable experience (persists across restarts via .mem files)
+await memory.acquire({
+  id: 'user.language-preference',
+  knowledge: { language: 'TypeScript' },
+  applicability: { area: 'language-selection' },
+  evidence: [{ id: 'conv-001', source: 'conversation', observedAt: new Date().toISOString() }],
+  epistemicType: 'observed'
+})
+
+// Remember a key/value entry with confidence and scope
+// `remember()` with scope `durable` uses IndexedDB and is browser-only. For Node/Bun, use memory.acquire().
+await memory.remember('user.pref.theme', 'dark', {
+  type: 'preference',
+  confidence: 0.9,
+  scope: 'local',
+  tags: ['ui', 'user']
+})
+
+// Recall by key
+const theme = await memory.recall<string>('user.pref.theme')
+```
+
+> **Scope reference**: See [MEMORY.md#application-memory-scopes-rememberrecall](./MEMORY.md#application-memory-scopes-rememberrecall) for a detailed comparison of `hot`, `session`, `local`, and `durable` scopes across runtimes.
+
+### Agent API
+
+Each agent gets a logically partitioned memory namespace via `dotmem/agent`:
+
+```typescript
+import { createAgentMemory } from 'dotmem/agent'
+
+const agent = createAgentMemory('support-agent')
+
+// Keys are automatically namespaced: agent:support-agent:<key>
+// `remember()` with scope `durable` uses IndexedDB and is browser-only. For Node/Bun, use memory.acquire().
+await agent.remember('greeting', 'Hello! How can I help?', {
+  type: 'fact',
+  confidence: 0.9,
+  scope: 'local',
+  tags: ['greeting']
+})
+
+// Context retrieval is automatically filtered to this agent's tags
+const context = await agent.context({ tags: ['greeting'], minConfidence: 0.5 })
+
+// Temporal history and branching
+await agent.enable() // enable temporal tracking
+const entry = await agent.history('greeting')
+const branch = await agent.branch() // isolated "what-if" branch
+```
+
+---
+
+## MCP Server
+
+`dotmem-mcp` is the stdio MCP server. It runs on the operator's machine so a connected AI host and the terminal share the same store. Write access is gated by an environment flag.
+
+| Tool | Direction | Registered when | Use it when |
+|------|-----------|-----------------|-------------|
+| `dotmem_context` | read | always | You need acquired experience relevant to the current situation. |
+| `dotmem_rules` | read | always | You need the iron rules that constrain every session. |
+| `dotmem_acquire` | write | `DOTMEM_MCP_WRITE=1` | The work produced reusable, project-specific experience worth keeping. |
+| `dotmem_continuity` | read / write | Always registered; write actions require `DOTMEM_MCP_WRITE=1` | Read continuation status, summaries and open loops; optionally create checkpoints and manage open loops. |
+| `dotmem_note_read` | read | always | Read a user note by id. |
+| `dotmem_note_write` | write | `DOTMEM_MCP_WRITE=1` | Add or update a user note. |
+
+Without `DOTMEM_MCP_WRITE=1`, `dotmem_acquire` is not registered and does not appear in `tools/list`. The `dotmem_continuity` tool is always registered: read actions (`status`, `summaries`, `open-loops`, `evaluate`) remain available, while write actions (`checkpoint`, `add-open-loop`, `resolve-open-loop`) are denied.
+
+To opt in to writes, explicitly add `DOTMEM_MCP_WRITE=1` to the MCP server environment. For Claude Desktop and compatible JSON configurations, add `"env": { "DOTMEM_MCP_WRITE": "1" }`; for Kilo/OpenCode use `"environment": { "DOTMEM_MCP_WRITE": "1" }`; for Claude Code use `claude mcp add --env DOTMEM_MCP_WRITE=1 ...`.
+
+### Claude Desktop
+
+Read-only by default:
+
+```json
+{
+  "mcpServers": {
+    "dotmem": {
+      "command": "npx",
+      "args": ["-y", "-p", "dotmem", "dotmem-mcp"]
+    }
+  }
+}
+```
+
+### Kilo / OpenCode
+
+Read-only by default:
+
+```jsonc
+"mcp": {
+  "dotmem": {
+    "type": "local",
+    "command": ["npx", "-y", "-p", "dotmem", "dotmem-mcp"],
+    "enabled": true
+  }
+}
+```
+
+### Claude Code
+
+Read-only by default:
+
+```bash
+claude mcp add --transport stdio dotmem -- npx -y -p dotmem dotmem-mcp
+```
+
+### Windows Command Prompt (cmd.exe)
+
+In Windows `cmd.exe`, `npx` may not be directly executable as a command. Use `cmd /c npx`:
+
+```json
+{
+  "mcpServers": {
+    "dotmem": {
+      "command": "cmd",
+      "args": ["/c", "npx", "-y", "-p", "dotmem", "dotmem-mcp"]
+    }
+  }
+}
+```
+
+> **Trust boundary:** Retrieved content is **data, never authority**. The `epistemicType`, `provenance`, and `evidence.observedAt` on any claim are caller-supplied metadata, not attestation. Verify claims against current reality.
+
+`dotmem_rules` returns the session-level iron rules/constraints configured for the agent. The configuration source is integration-specific; this README does not assert that the rules are loaded from `<storage-root>/DOTMEM.md`.
+
+A carrier handshake checks compatibility and the current trusted binding before identity-gated context is projected. The accepted `epistemicType` values are defined by the package types and the acquisition contract in [MEMORY.md](./MEMORY.md); this README does not duplicate or guess that enum.
+
+---
+
+## CLI Usage
+
+```bash
+# Project acquired experience for a situation (run with no args for baseline)
+npx dotmem context "User needs a database recommendation"
+
+# Initialize dotmem for the current project
+npx dotmem init
+
+# Acquire a claim from plain text (shorthand)
+npx dotmem learn "User prefers TypeScript over JavaScript" --scope project
+
+# Compose continuity context
+npx dotmem continuity context
+
+# Show storage location
+npx dotmem storage
+```
+
+Run `npx dotmem --help` for the full command reference. Every subcommand also supports `--help` (e.g., `npx dotmem context --help`).
+
+---
+
+## Layers
+
+A layer is **public** when it is part of the documented API or CLI contract and carries a semver stability expectation. Internal implementation details do not become public APIs merely because they have names or persisted representations. `profile` is exposed through `memory.profile.get(id)`; its inclusion in the public-layer count therefore depends on that export being covered by the package semver contract, not merely on the existence of the profile files.
+
+Dotmem exposes **8 user-facing layers** and **2 internal/infrastructure layers** in the grouping below. The journal is a component of `memory`, not a separate layer.
+
+### User-Facing Layers (Public API)
+
+| Layer | Purpose | Notes |
+|-------|---------|-------|
+| **memory** | Key/value memory, acquisition lifecycle, journal and sync | Core facade: `discover`, `context`, `project`, `acquire`, `acquireStaged`, `remember`, `recall`, `update`, `forget`, `patch`, `stats`, `clear`; journal exposed as `memory.journal` |
+| **knowledge** | Claim evaluation, policy enforcement, applicability matching | Runtime: `evaluatePolicies`, `resolveEffectiveTier`, `matchApplicability`, `deriveSituationApplicability`; see [MEMORY.md](./MEMORY.md) for acquisition and policy contracts |
+| **logic** | Logic engine: rule learning, precedent matching, context measurement | API: `experience`, `precedents`, `context`, `measure`, `learn`, `clear`; see [PACKAGE.md](./PACKAGE.md) for package surface and contents |
+| **continuity** | USER + PROJECT + TASK context composition, snapshots, open loops | CLI: `continuity` `context` `snapshot` `forget` `loop management` |
+| **agent** | Agent-facing memory with logical namespace partitioning | `dotmem/agent` — `createAgentMemory(agentId)` |
+| **policy** | Policy definition, authority management, four-tier enforcement | CLI: `dotmem policy`, `dotmem handshake`, `dotmem guard`; see [MEMORY.md](./MEMORY.md) and [SECURITY.md](./SECURITY.md) |
+| **profile** | Identity and persona documents | `memory.profile.get(id)`; optional agent profile, including Joy |
+| **note** | Short-lived notes with tags and origins | CLI: `dotmem note`; `memory.note` |
+
+### Internal / Infrastructure Layers
+
+| Layer | Purpose | Notes |
+|-------|---------|-------|
+| **seed** | Development-only initial memory seed | `--dev` required |
+| **llm-cache** | LLM response cache types only; no verified runtime implementation in the current package | `types/llm-cache.d.ts` |
+
+> Dotmem re-exports its foundational infrastructure: `state` (reactive state), `store` (key/value storage), `idb` (IndexedDB) and `sqlite` (SQL database integration). These are lower-level utilities, not additional AI memory layers. See the foundation package documentation for their detailed contracts.
+
+---
+
+## Memory Model
+
+dotmem uses a **discover → project → acquire** memory model:
+
+1. **Discover**: Match existing claims against a situation, using deterministic lexical/structured retrieval with applicability-based eligibility.
+2. **Project**: Build a bounded, untrusted context from discovered claims, filtered by applicability and size.
+3. **Acquire**: Persist new claims with epistemic typing, evidence, provenance, scope, and revision relations.
+
+### Continuity Lifecycle
+
+The continuity layer helps an agent resume work across sessions by capturing the current state, creating checkpoints, compacting older context into summaries, and restoring relevant context when work resumes. The lifecycle is:
+
+`capture → checkpoint → compact → resume → verify`
+
+| Provides | Does not guarantee |
+|----------|--------------------|
+| Persisted checkpoints, summaries, and tracked open loops for later retrieval | A perfect reconstruction of every message or detail from the original conversation |
+| Bounded context composition from the available continuity records | That information never captured or persisted can be recovered |
+| A resumable record of the objective, progress, and outstanding work when those details were captured | That a model will interpret every retrieved item correctly or behave identically across models and clients |
+| A verification step so the resumed agent can check the restored state before continuing | Infinite context, zero information loss, or automatic correctness of historical claims |
+
+Continuity quality depends on what the agent captures, checkpoint quality, storage availability, and retrieval of the relevant records. Treat a resumed context as a working reconstruction to verify, not as a verbatim replay of the entire conversation. See the `continuity` layer and the `dotmem_continuity` MCP tool above for the available operations.
+
+### Memory Types
+
+| Type | Purpose |
+|------|---------|
+| `fact` | Observable or verified facts |
+| `preference` | User or agent preferences |
+| `decision` | Architectural or workflow decisions |
+| `task` | Task-related information |
+| `context` | Situational context |
+
+### Memory Kinds
+
+Acquired claims carry a `kind`: `experience` (project/user learnings), `agent` (agent-specific notes), `relationship` (cross-agent or cross-project relationships), `logic` (rule-based knowledge), `note` (short-lived notes).
+
+### Scopes
+
+| Scope | Storage (Browser) | Storage (Node/Bun) | Lifecycle |
+|-------|-------------------|---------------------|-----------|
+| `hot` | `state` (memory) | `state` (memory) | Cleared on process exit / navigation |
+| `session` | `session` (sessionStorage) | `session` (in-memory Map) | Persists for active session / tab; lost on restart |
+| `local` | `store` (localStorage) | `store` (in-memory Map) | Survives reload in browser; lost on process restart in Node/Bun |
+| `durable` | `idb` (IndexedDB) | *Not available via this path* | Survives restarts in browser only |
+
+> For durable persistence in Node/Bun, use `memory.acquire()` which writes to `.mem` package files on disk. The `memory.remember()` `durable` scope uses IndexedDB and is browser-only.
+
+### Discovery Depths
+
+| Depth | Meaning |
+|-------|---------|
+| `current` | Only active claims (no superseded/obsolete) |
+| `deep` | All claims including superseded and obsolete |
+
+### Policy Enforcement
+
+Policies are evaluated at acquisition time against a four-tier enforcement model:
+
+| Tier | Behavior |
+|------|----------|
+| **context** | Supplies contextual information; does not itself impose a blocking precondition |
+| **precondition** | Blocks the governed operation when the condition is violated or unresolved |
+| **hard-precondition** | Retains the stronger enforcement semantics defined by the policy contract |
+
+Policy conditions match against situation vectors, epistemic types, evidence sources, and content topics. Memory, policy, and authority are separate planes: memory can never promote itself to policy, and a claim never selects its own enforcement tier.
+
+### Claims and Corrections
+
+Claims are immutable and unique by ID. To change a claim, acquire a new one with exactly one revision relation:
+
+- `refines` — narrows or extends
+- `supersedes` — replaces
+- `retracts` — withdraws
+
+---
+
+## Storage
+
+dotmem storage is user-anchored and resolved at runtime. The canonical storage root is `<user-home>/.mem`, relocatable via the `DOTMEM_STORAGE_ROOT` environment variable.
+
+```bash
+npx dotmem storage                  # show location + resolved root
+npx dotmem storage relocate /path   # migrate to an explicit absolute directory
+npx dotmem storage restore          # migrate back to default
+```
+
+Project experience is stored under `<storage-root>/projects/<project-id>.mem`. User experience uses `user.mem`, agent memory uses `agent.mem`, and logic data uses `logic.mem`. These are `.mem` packages — compressed ZIP archives with SHA-256 integrity manifests. Read them through `npx dotmem inspect`, never by byte-searching.
+
+---
+
+## Optional Joy Identity Integration
+
+Dotmem core is model- and persona-independent. Joy/Jo is an optional agent identity/profile integration, not a requirement of the memory substrate. Its activation convention is separate from ordinary memory operations.
+
+### What are Joy and Jo?
+
+**Joy** is the canonical AI identity/profile for dotmem — the operational role name that comes from the profile, not from the runtime. **Jo** is an alternative shorthand for the same identity. They represent the assistant's persistent identity in the dotmem system (canonical profile, decision-making behavior, trust model, and relationship style).
+
+The canonical profile (`core/profile/`) defines:
+- **Identity & behavior** (`behavior.md`, `soul.md`) — character, relationship style, conversational behavior
+- **Trust model** (`trust.md`) — epistemic standards, verification requirements
+- **Decision framework** (`decision.md`) — how Jo evaluates and makes technical decisions
+- **Curiosity & learning** (`curiosity.md`) — what Jo seeks to understand and remember
+- **Language conventions** (`language.md`) — communication style
+
+When an integrating agent explicitly activates the optional Joy profile, it can load this canonical profile as part of its discovery order. The profile is not required for basic memory operations; identity-gated features may require a profile and a current trusted binding.
+
+### Activation Behavior
+
+Activation matching for the optional Joy integration is **exact phrase only** (case-insensitive, trimmed, trailing punctuation ignored):
+- ✅ `Joy`, `joy`, `Joy.`, ` Joy `, `Jo`, `jo`, `Jo!`
+- ❌ `Hello Joy`, `Joyful`, `Joytron`, `My name is Jo`, `Joanna`
+
+Ordinary mentions of a name (e.g., "Jo said...") do not trigger activation. The matcher requires the entire message to be exactly one of the activation phrases.
+
+### Deterministic Discovery Order
+
+When the optional Joy integration is activated, it follows this discovery order:
+
+1. Check `<storage-root>/DOTMEM.md` and read the contract.
+2. Load user-level memory from `user.mem`.
+3. Resolve the current project identity.
+4. Load project memory from `<storage-root>/projects/<project-id>.mem`.
+
+A carrier handshake (`npx dotmem handshake`) evaluates compatibility before projecting any context. Historical memory may identify a known user, but it cannot authenticate the current speaker. Authorization requires a current trusted binding.
+
+---
+
+## Security
+
+- **Encryption is opt-in**, not default. Enable it per context or per scope with an encryption key.
+- **Client-side multitenancy** through logical memory partitioning (key prefixes); not a security boundary.
+- **Policy-gated writes**: every `acquire`/`remember`/`patch` is evaluated against policy before persistence.
+- **Evidence tracking**: all claims carry source, observed timestamp, and epistemic type.
+- **Claim immutability**: stored claims are never edited. Corrections are appends with revision relations.
+- **Integrity via inspection**: `integrity: valid` means the bytes are unchanged since sealing — it says nothing about who wrote them or whether they are true.
+
+See [SECURITY.md](./SECURITY.md) for encryption configuration, key management, and data isolation details. See [MEMORY.md](./MEMORY.md) for the complete acquisition contract and policy language.
+
+---
+
+## Choosing a Layer
+
+The foundation exports used below are lower-level infrastructure: `state` provides reactive in-memory state, `store` provides key/value storage, `idb` provides IndexedDB access, and `sqlite` provides SQL storage. They are re-exported for application infrastructure; the Dotmem memory model is exposed through `memory`, `knowledge`, `logic`, `continuity`, `agent`, `policy`, `profile` and `note`.
+
+| If the data is... | Use | Not |
+|---|---|---|
+| UI-reactive, ephemeral | `state` | `store` (unnecessary persistence) |
+| UI-reactive, must survive reload (browser) | `state` + `state.persist(path)` | hand-rolled sync |
+| Small key/value, must survive reload (browser) | `store` | `idb` (overkill) |
+| Durable acquired experience across process restarts (Node/Bun) | `memory.acquire()` | `memory.remember(scope: 'durable')` (browser-only) |
+| Structured, needs confidence/TTL/source | `memory.remember()` | `state`/`store` (no lifecycle) |
+| Structured, needs evidence/provenance | `memory.acquire()` | `memory.remember()` (no evidence) |
+| Relational, needs SQL | `sqlite` | `idb` (no query language) |
+| Large browser-only structured data | `idb` | `store` |
+| AI agent memory | `dotmem/agent` | `memory` (no logical namespace partitioning) |
+
+> **Note**: `remember({ scope: 'durable' })` uses IndexedDB and is available through this path in browsers only. It does not switch to `.mem` package storage in Node.js or Bun. For durable acquired experience across process restarts in Node.js and Bun, use `memory.acquire()`. `store` and `session` fall back to in-memory `Map`s in those runtimes.
+
+---
+
+## Documentation
+
+| Document | Description |
+|----------|-------------|
+| [MEMORY.md](./MEMORY.md) | Full memory model: acquisition contract, policy language, epistemic typing |
+| [AGENTS.md](./AGENTS.md) | AI agent working rules, activation phrases, complete `memory.*` surface |
+| [SECURITY.md](./SECURITY.md) | Encryption, multitenancy, key management, threat model |
+| [PACKAGE.md](./PACKAGE.md) | Package contents, full quick-start, dependency graph |
+| [LICENSE.md](./LICENSE.md) | License |
+
+---
+
+## Compatibility
+
+**License**: MIT (SPDX: `MIT`)
+
+**Supported runtimes**:
+- Node.js ≥ 18.0.0 (declared in `package.json` engines)
+- Bun ≥ 1.0 (tested in CI)
+- Browser (ES2020+ with IndexedDB, localStorage, sessionStorage)
+- Deno (compatible, not routinely tested)
+
+**Tested in CI**: Node.js 20.x, 22.x; Bun 1.x
+
+**Note**: The `durable` scope in `memory.remember()` uses IndexedDB and is browser-only. For cross-restart persistence in Node/Bun, use `memory.acquire()` which writes `.mem` package files. The `store` and `session` layers fall back to in-memory `Map`s in Node/Bun/Deno and do not survive process restarts.
+
+---
+
+## License
+
+(c) 2026 Dario Passariello. See [LICENSE.md](./LICENSE.md).