stenotype 0.1.1 → 0.2.0

package/README.md

@@ -1,217 +1,43 @@
-# Stenotype — Memory Plugin for OpenClaw
+# Stenotype
 
-![version](https://img.shields.io/badge/version-0.4.0-blue)
-![tests](https://img.shields.io/badge/tests-88%20passing-brightgreen)
-![license](https://img.shields.io/badge/license-MIT-lightgrey)
-
-Automatic memory capture and recall. LLM-powered extraction with hybrid vector+keyword search.
-
-## Quick links
-
-- [Changelog](./CHANGELOG.md)
-- [Architecture](./docs/architecture.md)
-- [Config reference](./docs/config-reference.md)
-- [FAQ](./docs/faq.md)
-- [Benchmarking](./docs/benchmarking.md)
-- [Release checklist](./docs/release-checklist.md)
-
-## Architecture
-
-Three stages:
-
-1. **Recall** (`before_agent_start`) — embed user message, query SQLite with hybrid scoring (vector similarity + FTS5 keywords + temperature-weighted recency + importance + usage frequency), inject relevant memories via `prependContext`
-2. **Log** (`agent_end` + session watcher) — append raw conversation to NDJSON files + SQLite. Session watcher scans `~/.openclaw/agents/*/sessions` for cross-agent coverage.
-3. **Extract** (background service, every 15min) — LLM extracts structured memories (category, content, subject, confidence, tags) from unprocessed logs. Deduplicates and supersedes contradicted memories.
+Automatic memory capture and recall for AI agent workflows.
 
 ## Install
 
-`openclaw stenotype setup` now includes:
-- explicit extraction + embedding provider selection
-- optional provider connectivity checks
-- optional safe `MEMORY.md` import on next start
-- daily summary cron scheduling (keep/enable/disable, once-daily)
-
-It also patches existing **Daily Summary** crons to use Stenotype NDJSON logs (`~/.stenotype/logs/YYYY-MM-DD.ndjson`) as the source of truth instead of `ai-memory/session-logs`.
-
 ```bash
-# In OpenClaw config, add to plugins.load.paths:
-# "plugins": { "load": { "paths": ["/path/to/stenotype"] } }
-
-# Or symlink into extensions:
-ln -s /path/to/stenotype ~/.openclaw/extensions/stenotype
-
-# Set the memory slot:
-# "plugins": { "slots": { "memory": "stenotype" } }
+npm install -g stenotype
+stenotype setup
 ```
 
 ## Configuration
 
-```json5
-{
-  plugins: {
-    slots: { memory: "stenotype" },
-    entries: {
-      stenotype: {
-        enabled: true,
-        config: {
-          embeddingApiKey: "${OPENAI_API_KEY}",
-          extractionApiKey: "${OPENAI_API_KEY}",
-          extractionModel: "gpt-4o-mini",
-          agentId: "main",          // defaults to OPENCLAW_AGENT_ID or "main"
-          recallScope: "agent",      // "agent" (isolated) or "global"
-          autoRecall: true,
-          autoCapture: true,
-          fileSync: true
-        }
-      }
-    }
-  }
-}
-```
-
-## CLI
-
-```bash
-openclaw stenotype status   # Health check
-openclaw stenotype stats    # Detailed statistics
-openclaw stenotype search "query"  # Search memories
-openclaw stenotype sync     # Force MEMORY.md sync
-openclaw stenotype setup    # Interactive setup wizard
-openclaw stenotype setup --headless --summary-schedule enable --summary-time 02:15
-openclaw stenotype temps    # Update temperature tiers
-openclaw stenotype baseline # 7-day telemetry/token baseline
-openclaw stenotype baseline --out ./artifacts/baseline # writes JSON+MD report
-
-# local model management
-openclaw stenotype model status
-openclaw stenotype model download
-```
-
-Benchmark harness (issue #14 support):
-
-```bash
-cp bench/benchmark-dataset.template.json bench/benchmark-dataset.json
-node scripts/benchmark-local-vs-cloud.mjs --dataset ./bench/benchmark-dataset.json --local-model ~/.cache/stenotype/models/qwen2.5-3b-instruct-q4_k_m.gguf
-```
-
-## Memory reliability features (v0.4 workstream)
-
-- **Deterministic preflight gate** (`preflightMode: "warn" | "strict"`): checks pinned directives + open commitments before recall injection.
-- **Pinned durable memory tier**: use `memory_pin` tool to mark must-load context.
-- **Commitment ledger**: use `commitment_add` tool (plus auto-capture from assistant "I will…" statements).
-- **Two-stage recall packing**: deterministic (pinned/commitments) first, semantic/FTS second, then dedupe + char-budget packing.
-- **Token-aware routing**: `~/.stenotype/job-policy.json` controls max input chars/token caps for extraction/summary/recall jobs.
-- **Visible token thresholds + alerts**: warn-level telemetry (`kind: "alert"`) and hard-limit throttling (`kind: "throttle"`) are emitted when budgets are crossed.
-- **Baseline telemetry**: NDJSON events under `~/.stenotype/telemetry/` power before/after token and recall comparisons.
-
-## Token routing policy (`~/.stenotype/job-policy.json`)
+Add to your agent config:
 
 ```json
 {
-  "extraction": { "model": "low-cost", "maxInputChars": 15000, "warnTokens": 50000, "hardMaxTokens": 120000 },
-  "summary": { "model": "low-cost", "maxInputChars": 10000, "warnTokens": 25000, "hardMaxTokens": 70000 },
-  "recall": { "model": "balanced", "maxInputChars": 6000, "warnTokens": 8000, "hardMaxTokens": 16000 }
+  "memory": {
+    "provider": "stenotype",
+    "agentId": "main",
+    "embeddingApiKey": "sk-...",
+    "extractionApiKey": "sk-...",
+    "extractionModel": "gpt-4o-mini"
+  }
 }
 ```
 
-- `warnTokens`: emits `alert` telemetry and warning logs (job continues).
-- `hardMaxTokens`: emits `throttle` telemetry and enforces a hard stop/fallback for that job.
-
-## Agent isolation
-
-- Memories/logs are tagged with `agent_id` (default `main`).
-- Recall defaults to agent-local scope (`recallScope: "agent"`) and can be widened with `"global"`.
-- Existing databases are auto-migrated with backfill to `agent_id = "main"` to preserve single-agent behavior.
-
-## Storage
-
-- **Database:** `~/.stenotype/memories.db` (SQLite + FTS5)
-- **Logs:** `~/.stenotype/logs/YYYY-MM-DD.ndjson`
-- **Embeddings:** Stored as binary blobs, cosine similarity computed in-process
-- **Temperature tiers:** Hot (<3d or accessed <7d), Warm (<14d or accessed <30d), Cold (everything else)
-
-## Dashboard (built-in + local)
-
-Stenotype currently ships two dashboard surfaces:
-
-1. **Built-in dashboard command** (`openclaw stenotype dashboard`) from `src/dashboard/server.ts`
-2. **Docker-first local dashboard MVP** in `dashboard-local/`
-
-### Built-in dashboard (issue #2 scope for current ship)
-
-Run:
-
-```bash
-openclaw stenotype dashboard --port 4789
-# open http://localhost:4789
-```
-
-Current capabilities:
-- Memory browser with search
-- Filter by category, temperature, and date range
-- Stats overview (total/active memories, temperature distribution, DB size, today's extraction count)
-- Extraction queue status + health signal (healthy/degraded/stalled)
-- Memory actions: edit + delete
-- Temperature distribution visualization + category bars
-
-Known limitations (intentionally out of current ship scope):
-- No pagination/virtualized table for very large memory sets
-- No dedicated extraction-job drilldown (worker runs, retries, error traces)
-- No multi-day trend charting/delta analytics yet
-
-### Local dashboard MVP (Docker-first)
-
-A standalone local dashboard is included at `dashboard-local/`.
-
-### What it shows
-
-- Summary cards (total/active memories, last 24h count, log events shown)
-- Recent memory events from NDJSON logs (`~/.stenotype/logs/*.ndjson`)
-- Memories list from SQLite (`~/.stenotype/memories.db`)
-- Search + category filtering
-
-### Quick start (Docker Compose)
-
-```bash
-cd /path/to/stenotype
-
-docker compose -f docker-compose.dashboard.yml up --build
-# open http://localhost:4789
-```
-
-Optional custom data path:
-
-```bash
-STENOTYPE_DATA_PATH=/absolute/path/to/.stenotype \
-  docker compose -f docker-compose.dashboard.yml up --build
-```
-
-### Local run (no Docker)
+## CLI
 
 ```bash
-cd /path/to/stenotype
-node dashboard-local/server.mjs
-# open http://localhost:4789
+stenotype setup          # Interactive setup
+stenotype status         # Health check
+stenotype stats          # Memory statistics
+stenotype search "query" # Search memories
+stenotype sync           # Sync memory file
+stenotype temps          # Update memory tiers
+stenotype baseline       # Usage baseline report
+stenotype dashboard      # Launch local dashboard
 ```
 
-### Data behavior / fallback
-
-- If `memories.db` exists, dashboard reads memory summary + memory list from SQLite.
-- If logs exist, dashboard reads recent event feed from NDJSON logs.
-- If neither exists, dashboard serves mock fallback data with an explicit note in UI.
-
-## Recall harness CI gate
-
-- Workflow: `.github/workflows/recall-harness-gate.yml`
-- Gate test file: `tests/recall-harness.test.ts`
-- Threshold behavior: build fails when deterministic recall assertions or context-budget assertions fail.
-
-### TODO (post-MVP)
-
-- Add pagination + virtualized event rendering for very large logs.
-- Add memory management actions (edit/delete) in standalone dashboard.
-- Add richer analytics (trend chart by day/week, category deltas).
-
-## Memory Categories
+## License
 
-decision, task, person, project, preference, fact, emotion, instruction, other
+MIT

package/package.json

@@ -1,9 +1,9 @@
 {
   "name": "stenotype",
-  "version": "0.1.1",
-  "description": "Automatic memory capture and recall for AI agent systems",
+  "version": "0.2.0",
+  "description": "Automatic memory capture and recall for AI agent workflows",
   "type": "module",
-  "main": "dist/index.js",
+  "main": "dist/index.cjs",
   "bin": {
     "stenotype": "dist/index.cjs"
   },
@@ -11,52 +11,26 @@
     "dist/index.cjs",
     "dist/stenotype.jsc"
   ],
-  "engines": {
-    "node": ">=20.0.0"
-  },
   "scripts": {
     "build": "tsc",
     "build:protected": "node scripts/build-protected.mjs",
-    "start": "node dist/index.js",
-    "status": "node dist/index.js status",
-    "recall": "node dist/index.js recall",
-    "test": "vitest run",
-    "test:watch": "vitest",
-    "test:recall-harness": "vitest run tests/recall-harness.test.ts --reporter=verbose",
-    "typecheck": "tsc --noEmit",
-    "baseline:report": "node scripts/generate-baseline-report.mjs main ./artifacts/baseline",
-    "bench:local-vs-cloud": "node scripts/benchmark-local-vs-cloud.mjs --dataset ./bench/benchmark-dataset.json"
-  },
-  "openclaw": {
-    "extensions": [
-      "./index.ts"
-    ]
+    "dev": "tsx src/index.ts",
+    "prepublishOnly": "node scripts/preflight-publish.mjs"
   },
   "dependencies": {
-    "@modelcontextprotocol/sdk": "^1.29.0",
-    "@types/express": "^5.0.6",
-    "@types/pg": "^8.18.0",
-    "better-sqlite3": "^11.9.1",
-    "express": "^5.2.1",
-    "jose": "^6.2.3",
-    "keytar": "^7.9.0",
-    "node-llama-cpp": "^3.17.1",
-    "openai": "^4.80.0",
-    "pg": "^8.20.0",
-    "pgvector": "^0.2.1",
-    "prompts": "^2.4.2",
-    "bytenode": "^1.5.7"
+    "@anthropic-ai/sdk": "^0.90.0",
+    "bytenode": "^1.5.7",
+    "openai": "^4.78.1",
+    "sqlite-vec": "^0.1.7",
+    "sqlite-vec-linux-x64": "^0.1.7"
   },
   "devDependencies": {
-    "@sinclair/typebox": "^0.34.0",
-    "@types/better-sqlite3": "^7.6.13",
-    "@types/keytar": "^4.4.0",
-    "@types/prompts": "^2.4.9",
-    "@types/supertest": "^7.2.0",
-    "bytenode": "^1.5.7",
-    "esbuild": "^0.28.0",
-    "supertest": "^7.2.2",
-    "typescript": "^5.9.3",
-    "vitest": "^4.0.18"
+    "@types/node": "^22.0.0",
+    "esbuild": "^0.24.0",
+    "tsx": "^4.19.0",
+    "typescript": "^5.7.0"
+  },
+  "engines": {
+    "node": ">=20.0.0"
   }
-}
+}

package/LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2026 Stenotype contributors
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

package/dist/index.js

@@ -1,10 +0,0 @@
-#!/usr/bin/env node
-/**
- * Stenotype standalone entrypoint (post-OpenClaw port).
- *
- * The original OpenClaw plugin registration lives in index.openclaw.ts.archive
- * for reference. That file depended on `openclaw/plugin-sdk` and assumed an
- * in-process runtime. Ductor runs Stenotype out-of-process as a CLI, so this
- * file simply delegates to the subcommand dispatcher in src/cli.ts.
- */
-import "./src/cli.js";