@nguyentamdat/mempalace 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,127 @@
1
+ # mempalace
2
+
3
+ Give your AI a memory. No API key required.
4
+
5
+ A structured, persistent memory system for AI agents — featuring a palace metaphor (wings → rooms → drawers), a temporal knowledge graph, the AAAK compression dialect, and a 4-layer memory stack.
6
+
7
+ This is a **Bun/TypeScript** port of the original [mempalace](https://github.com/milla-jovovich/mempalace) by [milla-jovovich](https://github.com/milla-jovovich).
8
+
9
+ ## Requirements
10
+
11
+ - [Bun](https://bun.sh/) ≥ 1.0
12
+ - [ChromaDB](https://www.trychroma.com/) server running (`chroma run`)
13
+
14
+ ## Install
15
+
16
+ ```bash
17
+ git clone https://github.com/nguyentamdat/mempalace-js.git
18
+ cd mempalace-js
19
+ bun install
20
+ ```
21
+
22
+ ## Usage
23
+
24
+ ### CLI
25
+
26
+ ```bash
27
+ # Initialize a new palace
28
+ bun run src/index.ts init
29
+
30
+ # Mine project files into the palace
31
+ bun run src/index.ts mine --mode projects --path ./my-project
32
+
33
+ # Mine conversation transcripts
34
+ bun run src/index.ts mine --mode convos --path ./transcripts
35
+
36
+ # Search memories
37
+ bun run src/index.ts search "some query"
38
+
39
+ # Compress memories with AAAK dialect
40
+ bun run src/index.ts compress
41
+
42
+ # Load memory layers (wake-up)
43
+ bun run src/index.ts wake-up
44
+
45
+ # Split mega transcript files
46
+ bun run src/index.ts split --path ./mega-file.txt
47
+
48
+ # Palace status
49
+ bun run src/index.ts status
50
+ ```
51
+
52
+ ### MCP Server
53
+
54
+ Run as a JSON-RPC MCP server over stdin/stdout (19 tools):
55
+
56
+ ```bash
57
+ bun run src/mcp-server.ts
58
+ ```
59
+
60
+ Tools include: `mempalace_search`, `mempalace_kg_query`, `mempalace_kg_timeline`, `mempalace_kg_add`, `mempalace_kg_invalidate`, `mempalace_diary_write`, `mempalace_diary_read`, `mempalace_status`, `mempalace_list_wings`, `mempalace_list_rooms`, `mempalace_add_drawer`, `mempalace_delete_drawer`, `mempalace_traverse`, `mempalace_find_tunnels`, and more.
61
+
62
+ ## Architecture
63
+
64
+ ### Palace Structure
65
+
66
+ ```
67
+ Palace (~/.mempalace/)
68
+ ├── Wings (top-level categories: work, personal, health, etc.)
69
+ │ └── Rooms (specific topics within a wing)
70
+ │ └── Drawers (individual memories stored in ChromaDB)
71
+ ├── Knowledge Graph (SQLite — entities + temporal triples)
72
+ ├── Diary (daily entries)
73
+ └── Config (config.json, entity_registry.json, identity.txt)
74
+ ```
75
+
76
+ ### Memory Layers
77
+
78
+ | Layer | Content | Size |
79
+ |-------|---------|------|
80
+ | Layer 0 | `identity.txt` — who am I | ~100 tokens |
81
+ | Layer 1 | Auto-generated from top drawers | ~500-800 tokens |
82
+ | Layer 2 | On-demand wing/room filtered | Variable |
83
+ | Layer 3 | Deep semantic search | Variable |
84
+
85
+ ### AAAK Compression Dialect
86
+
87
+ A compact encoding format for memories that reduces token usage while preserving meaning. Includes emotion codes, flag signals, and stop-word removal.
88
+
89
+ ### Knowledge Graph
90
+
91
+ SQLite-based temporal entity-relationship graph with validity windows and confidence scores. Supports entities (people, projects, concepts) and triples (subject → predicate → object).
92
+
93
+ ## Modules
94
+
95
+ | Module | Description |
96
+ |--------|-------------|
97
+ | `config.ts` | Palace configuration management |
98
+ | `knowledge-graph.ts` | Temporal entity-relationship graph (bun:sqlite) |
99
+ | `dialect.ts` | AAAK compression dialect |
100
+ | `layers.ts` | 4-layer memory stack |
101
+ | `searcher.ts` | Semantic search via ChromaDB |
102
+ | `palace-graph.ts` | Graph traversal and tunnel detection |
103
+ | `miner.ts` | Project file mining |
104
+ | `convo-miner.ts` | Conversation mining |
105
+ | `normalize.ts` | Chat export normalization (Claude, ChatGPT, Slack) |
106
+ | `general-extractor.ts` | Extract decisions, preferences, milestones |
107
+ | `entity-registry.ts` | Persistent entity registry with Wikipedia lookup |
108
+ | `entity-detector.ts` | Auto-detect people/projects from text |
109
+ | `onboarding.ts` | Interactive setup wizard |
110
+ | `room-detector-local.ts` | Room detection from folder structure |
111
+ | `spellcheck.ts` | Optional spell correction |
112
+ | `split-mega-files.ts` | Split concatenated transcripts |
113
+ | `mcp-server.ts` | JSON-RPC MCP server (19 tools) |
114
+
115
+ ## Tests
116
+
117
+ ```bash
118
+ bun test
119
+ ```
120
+
121
+ ## Acknowledgements
122
+
123
+ This project is a Bun/TypeScript port of the original [mempalace](https://github.com/milla-jovovich/mempalace) by [milla-jovovich](https://github.com/milla-jovovich). All credit for the palace architecture, AAAK compression dialect, knowledge graph design, memory layer system, and MCP tool definitions goes to their work.
124
+
125
+ ## License
126
+
127
+ MIT
@@ -0,0 +1,133 @@
1
+ # MemPalace Hooks — Auto-Save for Terminal AI Tools
2
+
3
+ These hook scripts make MemPalace save automatically. No manual "save" commands needed.
4
+
5
+ ## What They Do
6
+
7
+ | Hook | When It Fires | What Happens |
8
+ |------|--------------|-------------|
9
+ | **Save Hook** | Every 15 human messages | Blocks the AI, tells it to save key topics/decisions/quotes to the palace |
10
+ | **PreCompact Hook** | Right before context compaction | Emergency save — forces the AI to save EVERYTHING before losing context |
11
+
12
+ The AI does the actual filing — it knows the conversation context, so it classifies memories into the right wings/halls/closets. The hooks just tell it WHEN to save.
13
+
14
+ ## Install — Bun repo setup
15
+
16
+ From the repo root:
17
+
18
+ ```bash
19
+ bun install
20
+ chmod +x hooks/mempal_save_hook.sh hooks/mempal_precompact_hook.sh
21
+ ```
22
+
23
+ ## Install — Claude Code
24
+
25
+ Add to `.claude/settings.local.json`:
26
+
27
+ ```json
28
+ {
29
+ "hooks": {
30
+ "Stop": [{
31
+ "matcher": "*",
32
+ "hooks": [{
33
+ "type": "command",
34
+ "command": "/absolute/path/to/hooks/mempal_save_hook.sh",
35
+ "timeout": 30
36
+ }]
37
+ }],
38
+ "PreCompact": [{
39
+ "hooks": [{
40
+ "type": "command",
41
+ "command": "/absolute/path/to/hooks/mempal_precompact_hook.sh",
42
+ "timeout": 30
43
+ }]
44
+ }]
45
+ }
46
+ }
47
+ ```
48
+
49
+ ## Install — OpenCode
50
+
51
+ Configure the same shell scripts as your `Stop` and `PreCompact` command hooks in OpenCode, pointing to:
52
+
53
+ - `/absolute/path/to/hooks/mempal_save_hook.sh`
54
+ - `/absolute/path/to/hooks/mempal_precompact_hook.sh`
55
+
56
+ The scripts are standalone bash hooks and auto-resolve the Bun repo root from their own location.
57
+
58
+ ## Configuration
59
+
60
+ Edit `mempal_save_hook.sh` to change:
61
+
62
+ - **`SAVE_INTERVAL=15`** — How many human messages between saves. Lower = more frequent saves, higher = less interruption.
63
+ - **`STATE_DIR`** — Where hook state is stored (defaults to `~/.mempalace/hook_state/`)
64
+ - **`MEMPAL_DIR`** — Optional. Set to a conversations directory to auto-run `bun run src/index.ts mine <dir>` on each save trigger. Leave blank (default) to let the AI handle saving via the block reason message.
65
+
66
+ ### mempalace CLI
67
+
68
+ The relevant commands are:
69
+
70
+ ```bash
71
+ bun run src/index.ts mine <dir> # Mine all files in a directory
72
+ bun run src/index.ts mine <dir> --mode convos # Mine conversation transcripts only
73
+ ```
74
+
75
+ The hooks resolve the repo root automatically from their own path, so they work regardless of where you install the repo.
76
+
77
+ ## How It Works (Technical)
78
+
79
+ ### Save Hook (Stop event)
80
+
81
+ ```
82
+ User sends message → AI responds → Claude Code / OpenCode fires Stop hook
83
+
84
+ Hook counts human messages in JSONL transcript
85
+
86
+ ┌─── < 15 since last save ──→ echo "{}" (let AI stop)
87
+
88
+ └─── ≥ 15 since last save ──→ {"decision": "block", "reason": "save..."}
89
+
90
+ AI saves to palace
91
+
92
+ AI tries to stop again
93
+
94
+ stop_hook_active = true
95
+
96
+ Hook sees flag → echo "{}" (let it through)
97
+ ```
98
+
99
+ The `stop_hook_active` flag prevents infinite loops: block once → AI saves → tries to stop → flag is true → we let it through.
100
+
101
+ ### PreCompact Hook
102
+
103
+ ```
104
+ Context window getting full → Claude Code / OpenCode fires PreCompact
105
+
106
+ Hook ALWAYS blocks
107
+
108
+ AI saves everything
109
+
110
+ Compaction proceeds
111
+ ```
112
+
113
+ No counting needed — compaction always warrants a save.
114
+
115
+ ## Debugging
116
+
117
+ Check the hook log:
118
+
119
+ ```bash
120
+ cat ~/.mempalace/hook_state/hook.log
121
+ ```
122
+
123
+ Example output:
124
+ ```
125
+ [14:30:15] Session abc123: 12 exchanges, 12 since last save
126
+ [14:35:22] Session abc123: 15 exchanges, 15 since last save
127
+ [14:35:22] TRIGGERING SAVE at exchange 15
128
+ [14:40:01] Session abc123: 18 exchanges, 3 since last save
129
+ ```
130
+
131
+ ## Cost
132
+
133
+ **Zero extra tokens.** The hooks are bash scripts that run locally. They don't call any API. The only "cost" is the AI spending a few seconds organizing memories at each checkpoint — and it's doing that with context it already has loaded.
@@ -0,0 +1,35 @@
1
+ #!/bin/bash
2
+ STATE_DIR="$HOME/.mempalace/hook_state"
3
+ mkdir -p "$STATE_DIR"
4
+
5
+ MEMPAL_DIR=""
6
+
7
+ INPUT=$(cat)
8
+
9
+ json_get() {
10
+ local key="$1"
11
+
12
+ if command -v jq >/dev/null 2>&1; then
13
+ printf '%s' "$INPUT" | jq -r --arg key "$key" '.[$key] // empty' 2>/dev/null
14
+ elif command -v node >/dev/null 2>&1; then
15
+ printf '%s' "$INPUT" | node -e 'const fs = require("fs"); const key = process.argv[1]; try { const data = JSON.parse(fs.readFileSync(0, "utf8")); const value = data?.[key]; if (value !== undefined && value !== null) process.stdout.write(String(value)); } catch {}' "$key" 2>/dev/null
16
+ fi
17
+ }
18
+
19
+ SESSION_ID=$(json_get "session_id")
20
+ SESSION_ID=${SESSION_ID:-unknown}
21
+
22
+ echo "[$(date '+%H:%M:%S')] PRE-COMPACT triggered for session $SESSION_ID" >> "$STATE_DIR/hook.log"
23
+
24
+ if [ -n "$MEMPAL_DIR" ] && [ -d "$MEMPAL_DIR" ]; then
25
+ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
26
+ REPO_DIR="$(dirname "$SCRIPT_DIR")"
27
+ bun run "$REPO_DIR/src/index.ts" mine "$MEMPAL_DIR" >> "$STATE_DIR/hook.log" 2>&1
28
+ fi
29
+
30
+ cat << 'HOOKJSON'
31
+ {
32
+ "decision": "block",
33
+ "reason": "COMPACTION IMMINENT. Save ALL topics, decisions, quotes, code, and important context from this session to your memory system. Be thorough — after compaction, detailed context will be lost. Organize into appropriate categories. Use verbatim quotes where possible. Save everything, then allow compaction to proceed."
34
+ }
35
+ HOOKJSON
@@ -0,0 +1,80 @@
1
+ #!/bin/bash
2
+ SAVE_INTERVAL=15
3
+ STATE_DIR="$HOME/.mempalace/hook_state"
4
+ mkdir -p "$STATE_DIR"
5
+
6
+ MEMPAL_DIR=""
7
+
8
+ INPUT=$(cat)
9
+
10
+ json_get() {
11
+ local key="$1"
12
+
13
+ if command -v jq >/dev/null 2>&1; then
14
+ printf '%s' "$INPUT" | jq -r --arg key "$key" '.[$key] // empty' 2>/dev/null
15
+ elif command -v node >/dev/null 2>&1; then
16
+ printf '%s' "$INPUT" | node -e 'const fs = require("fs"); const key = process.argv[1]; try { const data = JSON.parse(fs.readFileSync(0, "utf8")); const value = data?.[key]; if (value !== undefined && value !== null) process.stdout.write(String(value)); } catch {}' "$key" 2>/dev/null
17
+ fi
18
+ }
19
+
20
+ count_user_messages() {
21
+ if command -v jq >/dev/null 2>&1; then
22
+ jq -Rnc '[inputs | (try fromjson catch empty) | .message | select(type == "object" and .role == "user" and ((.content | type) != "string" or (.content | contains("<command-message>") | not)))] | length' "$TRANSCRIPT_PATH" 2>/dev/null
23
+ elif command -v node >/dev/null 2>&1; then
24
+ node -e 'const fs = require("fs"); const path = process.argv[1]; let count = 0; try { for (const line of fs.readFileSync(path, "utf8").split(/\r?\n/)) { if (!line.trim()) continue; try { const entry = JSON.parse(line); const msg = entry?.message; if (msg && typeof msg === "object" && msg.role === "user") { const content = msg.content; if (typeof content === "string" && content.includes("<command-message>")) continue; count += 1; } } catch {} } } catch {} process.stdout.write(String(count));' "$TRANSCRIPT_PATH" 2>/dev/null
25
+ fi
26
+ }
27
+
28
+ SESSION_ID=$(json_get "session_id")
29
+ SESSION_ID=${SESSION_ID:-unknown}
30
+
31
+ STOP_HOOK_ACTIVE=$(json_get "stop_hook_active")
32
+ STOP_HOOK_ACTIVE=${STOP_HOOK_ACTIVE:-false}
33
+
34
+ TRANSCRIPT_PATH=$(json_get "transcript_path")
35
+ TRANSCRIPT_PATH=${TRANSCRIPT_PATH:-}
36
+
37
+ TRANSCRIPT_PATH="${TRANSCRIPT_PATH/#\~/$HOME}"
38
+
39
+ if [ "$STOP_HOOK_ACTIVE" = "True" ] || [ "$STOP_HOOK_ACTIVE" = "true" ]; then
40
+ echo "{}"
41
+ exit 0
42
+ fi
43
+
44
+ if [ -f "$TRANSCRIPT_PATH" ]; then
45
+ EXCHANGE_COUNT=$(count_user_messages)
46
+ EXCHANGE_COUNT=${EXCHANGE_COUNT:-0}
47
+ else
48
+ EXCHANGE_COUNT=0
49
+ fi
50
+
51
+ LAST_SAVE_FILE="$STATE_DIR/${SESSION_ID}_last_save"
52
+ LAST_SAVE=0
53
+ if [ -f "$LAST_SAVE_FILE" ]; then
54
+ LAST_SAVE=$(cat "$LAST_SAVE_FILE")
55
+ fi
56
+
57
+ SINCE_LAST=$((EXCHANGE_COUNT - LAST_SAVE))
58
+
59
+ echo "[$(date '+%H:%M:%S')] Session $SESSION_ID: $EXCHANGE_COUNT exchanges, $SINCE_LAST since last save" >> "$STATE_DIR/hook.log"
60
+
61
+ if [ "$SINCE_LAST" -ge "$SAVE_INTERVAL" ] && [ "$EXCHANGE_COUNT" -gt 0 ]; then
62
+ echo "$EXCHANGE_COUNT" > "$LAST_SAVE_FILE"
63
+
64
+ echo "[$(date '+%H:%M:%S')] TRIGGERING SAVE at exchange $EXCHANGE_COUNT" >> "$STATE_DIR/hook.log"
65
+
66
+ if [ -n "$MEMPAL_DIR" ] && [ -d "$MEMPAL_DIR" ]; then
67
+ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
68
+ REPO_DIR="$(dirname "$SCRIPT_DIR")"
69
+ bun run "$REPO_DIR/src/index.ts" mine "$MEMPAL_DIR" >> "$STATE_DIR/hook.log" 2>&1 &
70
+ fi
71
+
72
+ cat << 'HOOKJSON'
73
+ {
74
+ "decision": "block",
75
+ "reason": "AUTO-SAVE checkpoint. Save key topics, decisions, quotes, and code from this session to your memory system. Organize into appropriate categories. Use verbatim quotes where possible. Continue conversation after saving."
76
+ }
77
+ HOOKJSON
78
+ else
79
+ echo "{}"
80
+ fi
package/package.json ADDED
@@ -0,0 +1,36 @@
1
+ {
2
+ "name": "@nguyentamdat/mempalace",
3
+ "version": "1.0.0",
4
+ "description": "Give your AI a memory. No API key required.",
5
+ "type": "module",
6
+ "bin": {
7
+ "mempalace": "./src/index.ts"
8
+ },
9
+ "scripts": {
10
+ "start": "bun run src/index.ts",
11
+ "check": "bunx tsc --noEmit",
12
+ "test": "bun test",
13
+ "mcp": "bun run src/mcp-server.ts"
14
+ },
15
+ "dependencies": {
16
+ "chromadb": "^1.9.2",
17
+ "citty": "^0.1.6",
18
+ "js-yaml": "^4.1.0",
19
+ "@clack/prompts": "^0.9.1"
20
+ },
21
+ "devDependencies": {
22
+ "@types/js-yaml": "^4.0.9",
23
+ "@types/bun": "latest"
24
+ },
25
+ "license": "MIT",
26
+ "author": "nguyentamdat",
27
+ "repository": {
28
+ "type": "git",
29
+ "url": "https://github.com/nguyentamdat/mempalace-js"
30
+ },
31
+ "files": [
32
+ "src",
33
+ "hooks",
34
+ "README.md"
35
+ ]
36
+ }
package/src/cli.ts ADDED
@@ -0,0 +1,50 @@
1
+ #!/usr/bin/env bun
2
+ /**
3
+ * MemPalace — Give your AI a memory. No API key required.
4
+ *
5
+ * Two ways to ingest:
6
+ * Projects: mempalace mine ~/projects/my_app (code, docs, notes)
7
+ * Conversations: mempalace mine ~/chats/ --mode convos (Claude, ChatGPT, Slack)
8
+ *
9
+ * Commands:
10
+ * mempalace init <dir> Detect rooms from folder structure
11
+ * mempalace split <dir> Split concatenated mega-files
12
+ * mempalace mine <dir> Mine project files (default)
13
+ * mempalace mine <dir> --mode convos Mine conversation exports
14
+ * mempalace search "query" Find anything, exact words
15
+ * mempalace wake-up Show L0 + L1 wake-up context
16
+ * mempalace status Show what's been filed
17
+ */
18
+
19
+ import { defineCommand, runMain } from "citty";
20
+ import { MempalaceConfig } from "./config";
21
+
22
+ const mainCommand = defineCommand({
23
+ meta: {
24
+ name: "mempalace",
25
+ version: "1.0.0",
26
+ description: "Give your AI a memory. No API key required.",
27
+ },
28
+ args: {
29
+ palace: {
30
+ type: "string",
31
+ description: "Where the palace lives (default: from config or ~/.mempalace/palace)",
32
+ },
33
+ },
34
+ subCommands: {
35
+ init: () => import("./commands/init").then((m) => m.default),
36
+ mine: () => import("./commands/mine").then((m) => m.default),
37
+ search: () => import("./commands/search").then((m) => m.default),
38
+ compress: () => import("./commands/compress").then((m) => m.default),
39
+ "wake-up": () => import("./commands/wake-up").then((m) => m.default),
40
+ split: () => import("./commands/split").then((m) => m.default),
41
+ status: () => import("./commands/status").then((m) => m.default),
42
+ },
43
+ });
44
+
45
+ export function resolvePalacePath(palaceArg?: string): string {
46
+ if (palaceArg) return palaceArg.replace(/^~/, process.env.HOME ?? "~");
47
+ return new MempalaceConfig().palacePath;
48
+ }
49
+
50
+ export default mainCommand;
@@ -0,0 +1,161 @@
1
+ import { defineCommand } from "citty";
2
+ import { resolvePalacePath } from "../cli";
3
+
4
+ export default defineCommand({
5
+ meta: { description: "Compress drawers using AAAK Dialect (~30x reduction)" },
6
+ args: {
7
+ wing: { type: "string", description: "Wing to compress (default: all wings)" },
8
+ "dry-run": { type: "boolean", description: "Preview compression without storing", default: false },
9
+ config: { type: "string", description: "Entity config JSON (e.g. entities.json)" },
10
+ },
11
+ async run({ args }) {
12
+ const { existsSync } = await import("node:fs");
13
+ const { join, basename } = await import("node:path");
14
+ const { ChromaClient, DefaultEmbeddingFunction, IncludeEnum } = await import("chromadb");
15
+ const { Dialect } = await import("../dialect");
16
+
17
+ const palacePath = resolvePalacePath(args.palace as string | undefined);
18
+ const dryRun = args["dry-run"];
19
+
20
+ // Load dialect (with optional entity config)
21
+ let configPath = args.config;
22
+ if (!configPath) {
23
+ for (const candidate of ["entities.json", join(palacePath, "entities.json")]) {
24
+ if (existsSync(candidate)) {
25
+ configPath = candidate;
26
+ break;
27
+ }
28
+ }
29
+ }
30
+
31
+ let dialect: InstanceType<typeof Dialect>;
32
+ if (configPath && existsSync(configPath)) {
33
+ dialect = Dialect.fromConfig(configPath);
34
+ console.log(` Loaded entity config: ${configPath}`);
35
+ } else {
36
+ dialect = new Dialect();
37
+ }
38
+
39
+ // Connect to palace
40
+ let client: InstanceType<typeof ChromaClient>;
41
+ let col: Awaited<ReturnType<InstanceType<typeof ChromaClient>["getCollection"]>>;
42
+ try {
43
+ client = new ChromaClient({ path: palacePath });
44
+ col = await client.getCollection({
45
+ name: "mempalace_drawers",
46
+ embeddingFunction: new DefaultEmbeddingFunction(),
47
+ });
48
+ } catch {
49
+ console.log(`\n No palace found at ${palacePath}`);
50
+ console.log(" Run: mempalace init <dir> then mempalace mine <dir>");
51
+ process.exit(1);
52
+ }
53
+
54
+ // Query drawers in the wing
55
+ const where = args.wing ? { wing: args.wing } : undefined;
56
+ let results: Awaited<ReturnType<typeof col.get>>;
57
+ try {
58
+ results = await col.get({
59
+ where,
60
+ include: [IncludeEnum.Documents, IncludeEnum.Metadatas],
61
+ });
62
+ } catch (e) {
63
+ console.log(`\n Error reading drawers: ${e}`);
64
+ process.exit(1);
65
+ }
66
+
67
+ const docs = results.documents ?? [];
68
+ const metas = results.metadatas ?? [];
69
+ const ids = results.ids;
70
+
71
+ if (docs.length === 0) {
72
+ const wingLabel = args.wing ? ` in wing '${args.wing}'` : "";
73
+ console.log(`\n No drawers found${wingLabel}.`);
74
+ return;
75
+ }
76
+
77
+ console.log(
78
+ `\n Compressing ${docs.length} drawers` +
79
+ (args.wing ? ` in wing '${args.wing}'` : "") +
80
+ "..."
81
+ );
82
+ console.log();
83
+
84
+ let totalOriginal = 0;
85
+ let totalCompressed = 0;
86
+ const compressedEntries: Array<{
87
+ id: string;
88
+ compressed: string;
89
+ meta: Record<string, unknown>;
90
+ stats: ReturnType<typeof Dialect.prototype.compressionStats>;
91
+ }> = [];
92
+
93
+ for (let i = 0; i < docs.length; i++) {
94
+ const doc = docs[i];
95
+ if (doc === null || doc === undefined) {
96
+ continue;
97
+ }
98
+
99
+ const meta = (metas[i] ?? {}) as Record<string, unknown>;
100
+ const docId = ids[i];
101
+
102
+ const compressed = dialect.compress(doc, meta);
103
+ const stats = dialect.compressionStats(doc, compressed);
104
+
105
+ totalOriginal += stats.originalChars;
106
+ totalCompressed += stats.compressedChars;
107
+ compressedEntries.push({ id: docId, compressed, meta, stats });
108
+
109
+ if (dryRun) {
110
+ const wingName = (meta.wing as string) ?? "?";
111
+ const roomName = (meta.room as string) ?? "?";
112
+ const source = basename((meta.source_file as string) ?? "?");
113
+ console.log(` [${wingName}/${roomName}] ${source}`);
114
+ console.log(
115
+ ` ${stats.originalTokens}t -> ${stats.compressedTokens}t (${stats.ratio.toFixed(1)}x)`
116
+ );
117
+ console.log(` ${compressed}`);
118
+ console.log();
119
+ }
120
+ }
121
+
122
+ // Store compressed versions (unless dry-run)
123
+ if (!dryRun) {
124
+ try {
125
+ const compCol = await client.getOrCreateCollection({
126
+ name: "mempalace_compressed",
127
+ embeddingFunction: new DefaultEmbeddingFunction(),
128
+ });
129
+ for (const { id, compressed, meta, stats } of compressedEntries) {
130
+ const compMeta = {
131
+ ...meta,
132
+ compression_ratio: Math.round(stats.ratio * 10) / 10,
133
+ original_tokens: stats.originalTokens,
134
+ };
135
+ await compCol.upsert({
136
+ ids: [id],
137
+ documents: [compressed],
138
+ metadatas: [compMeta as Record<string, string | number | boolean>],
139
+ });
140
+ }
141
+ console.log(
142
+ ` Stored ${compressedEntries.length} compressed drawers in 'mempalace_compressed' collection.`
143
+ );
144
+ } catch (e) {
145
+ console.log(` Error storing compressed drawers: ${e}`);
146
+ process.exit(1);
147
+ }
148
+ }
149
+
150
+ // Summary
151
+ const ratio = totalOriginal / Math.max(totalCompressed, 1);
152
+ const origTokens = Dialect.countTokens("x".repeat(totalOriginal));
153
+ const compTokens = Dialect.countTokens("x".repeat(totalCompressed));
154
+ console.log(
155
+ ` Total: ${origTokens.toLocaleString()}t -> ${compTokens.toLocaleString()}t (${ratio.toFixed(1)}x compression)`
156
+ );
157
+ if (dryRun) {
158
+ console.log(" (dry run -- nothing stored)");
159
+ }
160
+ },
161
+ });
@@ -0,0 +1,40 @@
1
+ import { defineCommand } from "citty";
2
+ import { MempalaceConfig } from "../config";
3
+
4
+ export default defineCommand({
5
+ meta: { description: "Detect rooms from your folder structure" },
6
+ args: {
7
+ dir: { type: "positional", description: "Project directory to set up", required: true },
8
+ yes: { type: "boolean", description: "Auto-accept all detected entities", default: false },
9
+ },
10
+ async run({ args }) {
11
+ const { scanForDetection, detectEntities, confirmEntities } = await import("../entity-detector");
12
+ const { detectRoomsLocal } = await import("../room-detector");
13
+ const { writeFileSync } = await import("fs");
14
+ const { resolve } = await import("path");
15
+
16
+ // Pass 1: auto-detect people and projects from file content
17
+ console.log(`\n Scanning for entities in: ${args.dir}`);
18
+ const files = scanForDetection(args.dir);
19
+ if (files.length > 0) {
20
+ console.log(` Reading ${files.length} files...`);
21
+ const detected = detectEntities(files);
22
+ const total =
23
+ detected.people.length + detected.projects.length + detected.uncertain.length;
24
+ if (total > 0) {
25
+ const confirmed = await confirmEntities(detected, args.yes);
26
+ if (confirmed.people.length > 0 || confirmed.projects.length > 0) {
27
+ const entitiesPath = resolve(args.dir, "entities.json");
28
+ writeFileSync(entitiesPath, JSON.stringify(confirmed, null, 2));
29
+ console.log(` Entities saved: ${entitiesPath}`);
30
+ }
31
+ } else {
32
+ console.log(" No entities detected — proceeding with directory-based rooms.");
33
+ }
34
+ }
35
+
36
+ // Pass 2: detect rooms from folder structure
37
+ await detectRoomsLocal(args.dir);
38
+ new MempalaceConfig().init();
39
+ },
40
+ });