suemo 0.0.1

package/README.md

@@ -0,0 +1,248 @@
+# 🐘 suemo
+
+> Persistent semantic memory for AI agents — backed by SurrealDB.
+
+> [!CAUTION]
+> **Bun-only.** This project will not run on Node.js and there are no plans to support it.
+>
+> suemo is experimental software built for the author's personal agent infrastructure. APIs may change without notice. Use at your own risk.
+
+suemo gives AI agents a memory that survives across sessions, models, and runtimes. Write observations from a Telegram bot, query them from OpenCode, consolidate overnight — all agents share one source of truth in SurrealDB.
+
+---
+
+## Features
+
+- **Hybrid retrieval** — vector (HNSW), BM25 full-text, and graph spreading activation, fused with reciprocal rank fusion server-side
+- **Bi-temporal nodes** — every fact has `valid_from` / `valid_until`; nothing is hard-deleted
+- **Contradiction detection** — calling `believe()` with a conflicting belief automatically invalidates the old one and links them with a `contradicts` edge
+- **Two-phase consolidation** — NREM clusters and compresses redundant observations via LLM; REM integrates new summaries into the broader graph with auto-scored relations
+- **SurrealKV time-travel** — `VERSION d'...'` queries let you inspect any node's state at any past datetime (requires `SURREAL_DATASTORE_RETENTION=90d`)
+- **Namespace isolation** — one SurrealDB instance, multiple agents, zero collision (`namespace` = agent group, `database` = agent identity)
+- **MCP + CLI** — both interfaces are thin shells over the same domain functions; no business logic duplication
+- **Lightweight** — no bundled vector store, no embedded database process, no OpenAI SDK; embedding via `fn::embed()` runs server-side in SurrealDB
+
+---
+
+## Requirements
+
+| Dependency                         | Version                   |
+| ---------------------------------- | ------------------------- |
+| [Bun](https://bun.sh)              | ≥ 1.3                     |
+| [SurrealDB](https://surrealdb.com) | ≥ 3.0 (SurrealKV storage) |
+
+SurrealDB must be started with SurrealKV and a retention window:
+
+```sh
+SURREAL_DATASTORE_RETENTION=90d surreal start \
+  --bind 0.0.0.0:8000 \
+  surrealkv:///path/to/data
+```
+
+An embedding model must be configured in SurrealDB for `fn::embed()` to work.
+
+---
+
+## Installation
+
+```sh
+git clone https://github.com/UA-sensei/suemo
+cd suemo
+bun install
+```
+
+---
+
+## Quick Start
+
+**1. Create config**
+
+```sh
+bun run src/cli/index.ts init
+# or, once linked: suemo init
+```
+
+This writes `~/.suemo/suemo.ts`. Edit it with your SurrealDB URL, credentials, and LLM endpoint.
+
+**2. Apply schema**
+
+`suemo init` applies the schema automatically if `SURREAL_*` env vars are set. Otherwise:
+
+```sh
+SURREAL_URL=ws://localhost:8000 \
+SURREAL_USER=root \
+SURREAL_PASS=root \
+suemo init
+```
+
+**3. Store a memory**
+
+```sh
+suemo observe "Bun is significantly faster than Node for CLI tools" \
+  --scope coding --tags bun,runtime
+```
+
+**4. Query it**
+
+```sh
+suemo query "which JS runtime is faster"
+```
+
+**5. Start the MCP server**
+
+```sh
+suemo serve
+# Listening on http://127.0.0.1:4242
+```
+
+---
+
+## CLI Reference
+
+```
+suemo <command> [options]
+
+Global flags (inherited by all commands):
+  -c, --config <path>   Path to config file
+  -d, --debug           Verbose debug logging
+
+Commands:
+  init                  Create config template and apply DB schema
+  serve                 Start the MCP server
+  observe <content>     Store an observation
+  believe <content>     Store a belief (triggers contradiction detection)
+  query <input>         Hybrid semantic search
+  recall <nodeId>       Fetch a node + its neighbours (ticks FSRS)
+  wander                Spreading-activation walk through the memory graph
+  timeline              Chronological view of memories
+  goal set <content>    Create a goal node
+  goal list             List active goals
+  goal resolve <id>     Mark a goal achieved
+  consolidate           Run NREM + REM consolidation pipeline
+  health                Memory health report
+  health vitals         Last 10 consolidation runs + node counts
+  sync                  Push memories to a remote SurrealDB instance
+  export                Stream memories to JSONL on stdout
+  import <file>         Import memories from a JSONL file
+```
+
+---
+
+## Config
+
+Config lives at `~/.suemo/suemo.ts` (user-level) or `suemo.config.ts` in the project root. Project-local takes precedence. Override with `--config <path>`.
+
+```ts
+// ~/.suemo/suemo.ts
+import { defineConfig } from 'suemo'
+
+export default defineConfig({
+	surreal: {
+		url: process.env.SURREAL_URL!,
+		namespace: 'bearcu',
+		database: 'bearcu-telegram',
+		auth: {
+			user: process.env.SURREAL_USER!,
+			pass: process.env.SURREAL_PASS!,
+		},
+	},
+	embedding: {
+		provider: 'surreal', // fn::embed() — configured in SurrealDB
+		dimension: 1536,
+	},
+	consolidation: {
+		trigger: 'timer',
+		intervalMinutes: 30,
+		llm: {
+			url: process.env.LLM_URL!, // any OpenAI-compatible endpoint
+			model: process.env.LLM_MODEL!,
+			apiKey: process.env.LLM_API_KEY!,
+		},
+	},
+	retrieval: {
+		weights: { vector: 0.5, bm25: 0.25, graph: 0.15, temporal: 0.1 },
+	},
+	mcp: { port: 4242, host: '127.0.0.1' },
+})
+```
+
+Multiple agents on the same machine use separate config files that extend a shared base:
+
+```ts
+// ~/.suemo/opencode.ts
+import base from './suemo.ts'
+export default {
+	...base,
+	surreal: { ...base.surreal, database: 'bearcu-opencode' },
+}
+```
+
+```sh
+suemo serve --config ~/.suemo/opencode.ts
+```
+
+---
+
+## MCP Tools
+
+| Tool            | Description                                                         |
+| --------------- | ------------------------------------------------------------------- |
+| `observe`       | Store an observation, belief, question, or hypothesis               |
+| `believe`       | Store a belief — auto-detects and invalidates contradicting beliefs |
+| `invalidate`    | Soft-delete a node by ID (sets `valid_until`)                       |
+| `query`         | Hybrid semantic search (vector + BM25 + graph)                      |
+| `recall`        | Fetch a node and its 1-hop neighbourhood; ticks FSRS                |
+| `wander`        | Spreading-activation walk through the graph                         |
+| `timeline`      | Chronological memory slice with optional date range                 |
+| `episode_start` | Begin a bounded session window                                      |
+| `episode_end`   | Close a session, optionally with a summary                          |
+| `goal_set`      | Create a goal node                                                  |
+| `goal_resolve`  | Mark a goal achieved                                                |
+| `goal_list`     | List active (or all) goals                                          |
+| `consolidate`   | Manually trigger NREM + REM consolidation                           |
+| `health`        | Graph health report                                                 |
+
+Agents never supply temporal fields (`valid_from`, `valid_until`). These are system-managed.
+
+---
+
+## Architecture
+
+```
+Agent / User
+     │
+     ▼
+┌─────────┐     ┌───────────┐
+│   CLI   │     │  MCP      │
+│ (crust) │     │ (Elysia)  │
+└────┬────┘     └─────┬─────┘
+     └───────┬─────────┘
+             ▼
+     ┌───────────────┐
+     │  Domain layer │  src/memory/, src/cognitive/, src/goal.ts
+     └───────┬───────┘
+             ▼
+     ┌───────────────┐
+     │  SurrealDB    │  SurrealKV · HNSW · BM25 · graph · VERSION
+     └───────────────┘
+```
+
+The CLI and MCP server are both thin shells. All logic lives in the domain layer — one implementation, two interfaces.
+
+---
+
+## Stack
+
+- **Runtime** — [Bun](https://bun.sh)
+- **Language** — TypeScript (strict, `exactOptionalPropertyTypes`)
+- **Database** — [SurrealDB](https://surrealdb.com) v3.0+ (SurrealKV)
+- **SDK** — [surrealdb.js](https://github.com/surrealdb/surrealdb.js) v2 + `@surrealdb/node`
+- **CLI** — [crust](https://github.com/chenxin-yan/crust) (`@crustjs/core`)
+- **HTTP / MCP** — [Elysia](https://elysiajs.com) v1.4
+- **Validation** — [Zod](https://zod.dev) v4
+- **Logging** — [LogTape](https://github.com/dahlia/logtape)
+- **Linting** — [Biome](https://biomejs.dev)
+
+---
+
+<p align="center"><sub><strong>© 2025 MEDRIVIA ／ Umar Alfarouk</strong></sub></p>

package/package.json

@@ -0,0 +1,58 @@
+{
+	"name": "suemo",
+	"version": "0.0.1",
+	"description": "Persistent semantic memory for AI agents — backed by SurrealDB.",
+	"author": {
+		"name": "Umar Alfarouk",
+		"email": "medrivia@gmail.com",
+		"url": "https://alfarouk.id"
+	},
+	"license": "GPL-3.0-only",
+	"homepage": "https://github.com/mdrv/suemo",
+	"repository": {
+		"type": "git",
+		"url": "git+https://github.com/mdrv/suemo.git"
+	},
+	"bugs": {
+		"url": "https://github.com/mdrv/suemo/issues"
+	},
+	"keywords": ["surrealdb", "memory", "ai", "agent", "mcp", "vector-search", "semantic-memory"],
+	"publishConfig": {
+		"access": "public"
+	},
+	"type": "module",
+	"engines": {
+		"bun": ">=1.3"
+	},
+	"bin": {
+		"suemo": "./src/cli/index.ts"
+	},
+	"files": ["src", "LICENSE", "README.md"],
+	"exports": {
+		".": {
+			"bun": "./src/index.ts",
+			"types": "./src/index.ts"
+		}
+	},
+	"scripts": {
+		"dev": "bun run src/cli/index.ts",
+		"start": "bun run src/cli/index.ts",
+		"check": "bun tsc --noEmit"
+	},
+	"dependencies": {
+		"@crustjs/core": "^0.0.15",
+		"@crustjs/plugins": "^0.0.19",
+		"@crustjs/prompts": "^0.0.9",
+		"@crustjs/style": "^0.0.5",
+		"@logtape/file": "^2.0.4",
+		"@logtape/logtape": "^2.0.4",
+		"@surrealdb/node": "^3.0.3",
+		"elysia": "^1.4.28",
+		"surrealdb": "^2.0.3",
+		"zod": "^4.3.6"
+	},
+	"devDependencies": {
+		"@types/bun": "^1.3.11",
+		"typescript": "^5.9.3"
+	}
+}

package/src/cli/commands/believe.ts

@@ -0,0 +1,27 @@
+import { loadConfig } from '../../config.ts'
+import { connect, disconnect } from '../../db/client.ts'
+import { initLogger } from '../../logger.ts'
+import { believe } from '../../memory/write.ts'
+import { app } from '../shared.ts'
+
+export const believeCmd = app.sub('believe')
+	.meta({ description: 'Store a belief (triggers contradiction detection)' })
+	.args([{ name: 'content', type: 'string', required: true }])
+	.flags({
+		scope: { type: 'string', short: 's', description: 'Scope label' },
+		confidence: { type: 'number', description: 'Confidence 0.0–1.0', default: 1.0 },
+	})
+	.run(async ({ args, flags }) => {
+		await initLogger({ level: flags.debug ? 'debug' : 'info' })
+		const config = await loadConfig(process.cwd(), flags.config)
+		const db = await connect(config.surreal)
+		const { node, contradicted } = await believe(db, {
+			content: args.content,
+			scope: flags.scope,
+			confidence: flags.confidence,
+		})
+		await disconnect()
+		const out: Record<string, unknown> = { id: node.id, valid_from: node.valid_from }
+		if (contradicted) out.contradicted = contradicted.id
+		console.log(JSON.stringify(out, null, 2))
+	})

package/src/cli/commands/consolidate.ts

@@ -0,0 +1,24 @@
+import { consolidate } from '../../cognitive/consolidate.ts'
+import { loadConfig } from '../../config.ts'
+import { connect, disconnect } from '../../db/client.ts'
+import { initLogger } from '../../logger.ts'
+import { app } from '../shared.ts'
+
+export const consolidateCmd = app.sub('consolidate')
+	.meta({ description: 'Manually trigger memory consolidation (NREM + REM)' })
+	.flags({
+		'nrem-only': { type: 'boolean', description: 'Run only NREM (compression) phase', default: false },
+	})
+	.run(async ({ flags }) => {
+		await initLogger({ level: flags.debug ? 'debug' : 'info' })
+		const config = await loadConfig(process.cwd(), flags.config)
+		const db = await connect(config.surreal)
+		const run = await consolidate(db, {
+			nremOnly: flags['nrem-only'],
+			nremSimilarityThreshold: config.consolidation.nremSimilarityThreshold,
+			remRelationThreshold: config.consolidation.remRelationThreshold,
+			llm: config.consolidation.llm,
+		})
+		await disconnect()
+		console.log(JSON.stringify(run, null, 2))
+	})

package/src/cli/commands/export-import.ts

@@ -0,0 +1,91 @@
+import { createReadStream } from 'node:fs'
+import { createInterface } from 'node:readline'
+import { loadConfig } from '../../config.ts'
+import { connect, disconnect } from '../../db/client.ts'
+import { initLogger } from '../../logger.ts'
+import type { MemoryNode, Relation } from '../../types.ts'
+import { app } from '../shared.ts'
+
+// ── export ─────────────────────────────────────────────────────────────────
+export const exportCmd = app.sub('export')
+	.meta({ description: 'Export memories to JSONL on stdout' })
+	.flags({
+		scope: { type: 'string', short: 's', description: 'Filter by scope' },
+		all: { type: 'boolean', description: 'Include invalidated nodes' },
+	})
+	.run(async ({ flags }) => {
+		await initLogger({ level: flags.debug ? 'debug' : 'info' })
+		const config = await loadConfig(process.cwd(), flags.config)
+		const db = await connect(config.surreal)
+
+		const activeFilter = flags.all ? 'true' : '(valid_until = NONE OR valid_until > time::now())'
+		const scopeFilter = '($scope = NONE OR scope = $scope)'
+
+		const [nodesResult, relationsResult] = await Promise.all([
+			db.query<[MemoryNode[]]>(
+				`
+        SELECT * FROM memory WHERE ${activeFilter} AND ${scopeFilter} ORDER BY created_at ASC
+      `,
+				{ scope: flags.scope ?? null },
+			),
+			db.query<[Relation[]]>('SELECT * FROM relates_to ORDER BY created_at ASC'),
+		])
+
+		for (const node of nodesResult[0] ?? []) {
+			process.stdout.write(JSON.stringify({ _type: 'memory', ...node }) + '\n')
+		}
+		for (const rel of relationsResult[0] ?? []) {
+			process.stdout.write(JSON.stringify({ _type: 'relation', ...rel }) + '\n')
+		}
+
+		await disconnect()
+	})
+
+// ── import ──────────────────────────────────────────────────────────────────
+export const importCmd = app.sub('import')
+	.meta({ description: 'Import memories from a JSONL file' })
+	.args([{ name: 'file', type: 'string', required: true }])
+	.run(async ({ args, flags }) => {
+		await initLogger({ level: flags.debug ? 'debug' : 'info' })
+		const config = await loadConfig(process.cwd(), flags.config)
+		const db = await connect(config.surreal)
+
+		const rl = createInterface({ input: createReadStream(args.file) })
+		let lineNum = 0
+		let imported = 0
+		let skipped = 0
+		let errors = 0
+
+		for await (const line of rl) {
+			lineNum++
+			if (!line.trim()) continue
+			let row: Record<string, unknown>
+			try {
+				row = JSON.parse(line)
+			} catch {
+				console.error(`Line ${lineNum}: invalid JSON — stopping`)
+				break
+			}
+
+			const type = row['_type']
+			delete row['_type']
+
+			try {
+				if (type === 'memory') {
+					await db.query('INSERT IGNORE INTO memory $row', { row })
+					imported++
+				} else if (type === 'relation') {
+					await db.query('INSERT IGNORE INTO relates_to $row', { row })
+					imported++
+				} else {
+					skipped++
+				}
+			} catch (e) {
+				console.error(`Line ${lineNum}: insert failed — ${String(e)}`)
+				errors++
+			}
+		}
+
+		await disconnect()
+		console.log(JSON.stringify({ imported, skipped, errors, lines: lineNum }, null, 2))
+	})

package/src/cli/commands/goal.ts

@@ -0,0 +1,71 @@
+import { loadConfig } from '../../config.ts'
+import { connect, disconnect } from '../../db/client.ts'
+import { goalList, goalResolve, goalSet } from '../../goal.ts'
+import { initLogger } from '../../logger.ts'
+import { app } from '../shared.ts'
+
+const goal = app.sub('goal')
+	.meta({ description: 'Manage goal nodes' })
+
+// suemo goal set "finish suemo v1"
+const setCmd = goal.sub('set')
+	.meta({ description: 'Create a new goal' })
+	.args([{ name: 'content', type: 'string', required: true }])
+	.flags({
+		scope: { type: 'string', short: 's', description: 'Scope label' },
+		tags: { type: 'string', short: 't', description: 'Comma-separated tags' },
+	})
+	.run(async ({ args, flags }) => {
+		await initLogger({ level: flags.debug ? 'debug' : 'info' })
+		const config = await loadConfig(process.cwd(), flags.config)
+		const db = await connect(config.surreal)
+		const node = await goalSet(db, args.content, {
+			...(flags.scope ? { scope: flags.scope } : {}),
+			tags: flags.tags ? flags.tags.split(',').map((t) => t.trim()) : [],
+		})
+		await disconnect()
+		console.log(JSON.stringify({ id: node.id, content: node.content }, null, 2))
+	})
+
+// suemo goal list
+const listCmd = goal.sub('list')
+	.meta({ description: 'List active goals' })
+	.flags({
+		scope: { type: 'string', short: 's', description: 'Filter by scope' },
+		resolved: { type: 'boolean', description: 'Include resolved goals', default: false },
+	})
+	.run(async ({ flags }) => {
+		await initLogger({ level: flags.debug ? 'debug' : 'info' })
+		const config = await loadConfig(process.cwd(), flags.config)
+		const db = await connect(config.surreal)
+		const goals = await goalList(db, {
+			...(flags.scope ? { scope: flags.scope } : {}),
+			includeResolved: flags.resolved,
+		})
+		await disconnect()
+		for (const g of goals) {
+			const status = g.valid_until ? `resolved ${g.valid_until}` : 'active'
+			console.log(`[${status}] ${g.id}`)
+			console.log(`  ${g.content}`)
+			console.log()
+		}
+	})
+
+// suemo goal resolve <id>
+const resolveCmd = goal.sub('resolve')
+	.meta({ description: 'Mark a goal as resolved' })
+	.args([{ name: 'goalId', type: 'string', required: true }])
+	.run(async ({ args, flags }) => {
+		await initLogger({ level: flags.debug ? 'debug' : 'info' })
+		const config = await loadConfig(process.cwd(), flags.config)
+		const db = await connect(config.surreal)
+		await goalResolve(db, args.goalId)
+		await disconnect()
+		console.log(`✓ Goal ${args.goalId} resolved`)
+	})
+
+// Export the container with its subcommands registered
+export const goalCmd = goal
+	.command(setCmd)
+	.command(listCmd)
+	.command(resolveCmd)

package/src/cli/commands/health.ts

@@ -0,0 +1,43 @@
+import { healthReport, vitals } from '../../cognitive/health.ts'
+import { loadConfig } from '../../config.ts'
+import { connect, disconnect } from '../../db/client.ts'
+import { initLogger } from '../../logger.ts'
+import { app } from '../shared.ts'
+
+const health = app.sub('health')
+	.meta({ description: 'Memory health report and vitals' })
+
+const reportCmd = health.sub('report')
+	.meta({ description: 'Full health report (default)' })
+	.run(async ({ flags }) => {
+		await initLogger({ level: flags.debug ? 'debug' : 'info' })
+		const config = await loadConfig(process.cwd(), flags.config)
+		const db = await connect(config.surreal)
+		const report = await healthReport(db)
+		await disconnect()
+		console.log(JSON.stringify(report, null, 2))
+	})
+
+const vitalsCmd = health.sub('vitals')
+	.meta({ description: 'Last 10 consolidation runs + node counts' })
+	.run(async ({ flags }) => {
+		await initLogger({ level: flags.debug ? 'debug' : 'info' })
+		const config = await loadConfig(process.cwd(), flags.config)
+		const db = await connect(config.surreal)
+		const v = await vitals(db)
+		await disconnect()
+		console.log(JSON.stringify(v, null, 2))
+	})
+
+export const healthCmd = health
+	.command(reportCmd)
+	.command(vitalsCmd)
+	// Default: run the report when just `suemo health` is called
+	.run(async ({ flags }) => {
+		await initLogger({ level: flags.debug ? 'debug' : 'info' })
+		const config = await loadConfig(process.cwd(), flags.config)
+		const db = await connect(config.surreal)
+		const report = await healthReport(db)
+		await disconnect()
+		console.log(JSON.stringify(report, null, 2))
+	})

package/src/cli/commands/init.ts

@@ -0,0 +1,90 @@
+import { existsSync, mkdirSync, writeFileSync } from 'node:fs'
+import { join } from 'node:path'
+import { loadConfig } from '../../config.ts'
+import { connect, disconnect } from '../../db/client.ts'
+import { checkCompatibility } from '../../db/preflight.ts'
+import { runSchema } from '../../db/schema.ts'
+import { initLogger } from '../../logger.ts'
+import { app } from '../shared.ts'
+
+export const initCmd = app.sub('init')
+	.meta({ description: 'Initialize suemo: create config template and apply DB schema' })
+	.flags({
+		force: { type: 'boolean', description: 'Overwrite existing config without prompting' },
+	})
+	.run(async ({ flags }) => {
+		await initLogger({ level: flags.debug ? 'debug' : 'info' })
+
+		const homeConfig = join(process.env.HOME ?? '~', '.suemo', 'suemo.ts')
+		const configExists = existsSync(homeConfig)
+
+		if (configExists && !flags.force) {
+			console.log(`Config already exists at ${homeConfig}`)
+			console.log('Pass --force to overwrite.')
+			return
+		}
+
+		const template = `import { defineConfig } from "suemo";
+
+export default defineConfig({
+  surreal: {
+    url:       process.env.SURREAL_URL  ?? "ws://localhost:8000",
+    namespace: process.env.SURREAL_NS   ?? "myagents",
+    database:  process.env.SURREAL_DB   ?? "default",
+    auth: {
+      user: process.env.SURREAL_USER!,
+      pass: process.env.SURREAL_PASS!,
+    },
+  },
+  embedding: {
+    provider:  "surreal",
+    dimension: 1536,
+  },
+  consolidation: {
+    trigger:                 "timer",
+    intervalMinutes:         30,
+    reactiveThreshold:       50,
+    nremSimilarityThreshold: 0.85,
+    remRelationThreshold:    0.4,
+    llm: {
+      url:    process.env.LLM_URL!,
+      model:  process.env.LLM_MODEL ?? "gpt-4o",
+      apiKey: process.env.LLM_API_KEY!,
+    },
+  },
+  retrieval: {
+    weights: { vector: 0.5, bm25: 0.25, graph: 0.15, temporal: 0.1 },
+  },
+  mcp: {
+    port: Number(process.env.SUEMO_PORT) || 4242,
+    host: "127.0.0.1",
+  },
+});
+`
+
+		mkdirSync(join(process.env.HOME ?? '~', '.suemo'), { recursive: true })
+		writeFileSync(homeConfig, template, 'utf-8')
+		console.log(`✓ Config written to ${homeConfig}`)
+
+		// Apply schema if SURREAL_* env vars are present
+		if (process.env.SURREAL_URL && process.env.SURREAL_USER) {
+			try {
+				const config = await loadConfig(process.cwd(), flags.config)
+				const db = await connect(config.surreal)
+				const compat = await checkCompatibility(db)
+				if (!compat.ok) {
+					console.error('\n✗ Compatibility check failed:')
+					for (const e of compat.errors) console.error(`  • ${e}`)
+					await disconnect()
+					return
+				}
+				await runSchema(db)
+				await disconnect()
+				console.log('✓ Schema applied to SurrealDB')
+			} catch (e) {
+				console.warn(`  (Schema not applied — configure SURREAL_* env vars first: ${String(e)})`)
+			}
+		} else {
+			console.log('  Set SURREAL_URL, SURREAL_USER, SURREAL_PASS, then run `suemo init` again to apply schema.')
+		}
+	})