memorix 1.6.3 → 1.7.1

package/docs/ARCHITECTURE.md

@@ -497,8 +497,11 @@ It provides an operational view for:
 - identity health
 - sessions
 - retention state
+- preview-first cleanup, consolidation, intelligent deduplication, and retention archival
 - read-only orchestration coordination state in the standalone dashboard and live state in HTTP service mode
 
+Dashboard maintenance is not a separate business-logic implementation. CLI, MCP, background jobs, and Dashboard routes share the same cleanup, consolidation, deduplication, and retention functions. Mutation routes issue a project-bound signed preview token and reject execution when the candidate set changed after preview.
+
 This is part of Memorix's shift from a single MCP server to a broader local memory platform.
 
 ---

package/docs/DASHBOARD.md

@@ -0,0 +1,54 @@
+# Dashboard
+
+The Memorix Dashboard is the local control surface for inspecting project memory and running explicit maintenance. It is available through the shared background service or as a standalone local process.
+
+## Start
+
+```bash
+memorix background start
+# dashboard: http://127.0.0.1:3211
+
+memorix dashboard
+# standalone dashboard: http://127.0.0.1:3210
+```
+
+Both modes read the same project-scoped stores. The project switcher changes the active project filter; it does not merge projects or grant access to personal memories.
+
+## Maintenance Workbench
+
+The Observations page exposes three explicit maintenance actions:
+
+| Action | Effect | Shared implementation |
+| --- | --- | --- |
+| Cleanup | Deletes low-quality activity records and exact duplicates; optionally archives test/demo noise | `analyzeCleanupObservations` and `applyCleanupMutations` |
+| Consolidate | Merges strongly similar project-visible observations while preserving facts and references | `findConsolidationCandidates` and `executeConsolidation` |
+| Deduplicate | Uses the configured memory LLM to identify redundant or superseded records and marks them resolved | `planMemoryDeduplication` and `applyDeduplicationPlan` |
+
+The Retention page exposes Archive for records already classified as archive candidates by the canonical retention projection.
+
+Every action follows the same contract:
+
+1. The Dashboard requests a project-scoped preview.
+2. The server returns the candidate records, counts, and a signed preview token.
+3. The user confirms the displayed plan.
+4. Execution verifies the token and current candidate set before mutating data.
+5. If memory changed after preview, execution fails with `409` and requires a new preview.
+
+Cleanup noise archival is opt-in. Consolidation never touches personal or team-scoped observations from an unbound Dashboard. Deduplication marks records resolved rather than deleting them. Retention changes status to archived. Manual selection and hard deletion remain separate from the maintenance workbench.
+
+## HTTP Contract
+
+Maintenance endpoints are local Dashboard APIs and are not a replacement for MCP or CLI integrations:
+
+```text
+POST /api/maintenance/cleanup/preview
+POST /api/maintenance/cleanup/execute
+POST /api/maintenance/consolidate/preview
+POST /api/maintenance/consolidate/execute
+POST /api/maintenance/deduplicate/preview
+POST /api/maintenance/deduplicate/execute
+POST /api/maintenance/retention/preview
+POST /api/maintenance/retention/execute
+```
+
+Use the CLI for scripts and remote terminals. Use MCP for agent-driven maintenance. The Dashboard is the reviewable local operator surface; all three call the same maintenance logic.

package/docs/README.md

@@ -33,6 +33,7 @@ The public docs are organized by user intent:
 | CLI commands and MCP tools | [API_REFERENCE.md](API_REFERENCE.md) |
 | Memory Autopilot, Code State, and context packs | [API_REFERENCE.md § Memory Autopilot, Code State, and Context Packs](API_REFERENCE.md#memory-autopilot-code-state-and-context-packs), [1.2 Code State](1.2.0-CODE-STATE.md), and [1.2 Workset Retrieval](1.2.0-WORKSET-RETRIEVAL.md) |
 | Reviewed episodic, semantic, and procedural long-term memory | [1.3 Memory Architecture](1.3-MEMORY-ARCHITECTURE.md) |
+| Browse memory and run preview-first local maintenance | [Dashboard](DASHBOARD.md) |
 | Reviewable knowledge pages and workflow inheritance | [API_REFERENCE.md § Knowledge Workspace and Workflows](API_REFERENCE.md#knowledge-workspace-and-workflows), [1.2 Knowledge Workspace](1.2.0-KNOWLEDGE-WORKSPACE.md), and [1.2 Workflow Inheritance](1.2.0-WORKFLOW-INHERITANCE.md) |
 | Git-derived engineering memory | [GIT_MEMORY.md](GIT_MEMORY.md) |
 | Memory formation and quality pipeline | [MEMORY_FORMATION_PIPELINE.md](MEMORY_FORMATION_PIPELINE.md) |
@@ -51,6 +52,7 @@ The public docs are organized by user intent:
 | TOML-first configuration | [CONFIGURATION.md](CONFIGURATION.md) |
 | Docker/compose deployment | [DOCKER.md](DOCKER.md) |
 | Resource and timeout tuning | [PERFORMANCE.md](PERFORMANCE.md) |
+| Dashboard modes, maintenance actions, and safety contract | [DASHBOARD.md](DASHBOARD.md) |
 | AI-facing install and troubleshooting playbook | [Agent Playbook](AGENT_OPERATOR_PLAYBOOK.md) |
 
 ---

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "memorix",
   "mcpName": "io.github.AVIDS2/memorix",
-  "version": "1.6.3",
+  "version": "1.7.1",
   "description": "Local-first shared memory layer for AI coding agents across MCP clients, Git history, reasoning context, and project sessions.",
   "type": "module",
   "sideEffects": false,

package/plugins/claude/memorix/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "$schema": "https://anthropic.com/claude-code/plugin.schema.json",
   "name": "memorix",
-  "version": "1.6.3",
+  "version": "1.7.1",
   "description": "Shared workspace memory for Claude Code and other AI coding agents.",
   "author": {
     "name": "AVIDS2",

package/plugins/codebuddy/memorix/.codebuddy-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "memorix",
-  "version": "1.6.3",
+  "version": "1.7.1",
   "description": "Local-first project memory for CodeBuddy Code and other coding agents.",
   "author": {
     "name": "AVIDS2",

package/plugins/codex/memorix/.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "memorix",
-  "version": "1.6.3",
+  "version": "1.7.1",
   "description": "Shared workspace memory for Codex and other AI coding agents.",
   "author": {
     "name": "AVIDS2",

package/plugins/copilot/memorix/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "memorix",
-  "version": "1.6.3",
+  "version": "1.7.1",
   "description": "Shared local workspace memory for GitHub Copilot CLI and other AI coding agents.",
   "author": {
     "name": "AVIDS2",

package/plugins/gemini/memorix/gemini-extension.json

@@ -1,6 +1,6 @@
 {
   "name": "memorix",
-  "version": "1.6.3",
+  "version": "1.7.1",
   "mcpServers": {
     "memorix": {
       "command": "memorix",

package/plugins/omp/memorix/package.json

@@ -1,6 +1,6 @@
 {
   "name": "memorix-omp-package",
-  "version": "1.6.3",
+  "version": "1.7.1",
   "description": "Memorix shared workspace memory package for Oh-my-Pi.",
   "license": "Apache-2.0",
   "keywords": [

package/plugins/openclaw/memorix/.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "memorix",
-  "version": "1.6.3",
+  "version": "1.7.1",
   "description": "Shared workspace memory for OpenClaw and other AI coding agents.",
   "author": {
     "name": "AVIDS2",

package/plugins/pi/memorix/package.json

@@ -1,6 +1,6 @@
 {
   "name": "memorix-pi-package",
-  "version": "1.6.3",
+  "version": "1.7.1",
   "description": "Memorix shared workspace memory package for Pi coding agent.",
   "license": "Apache-2.0",
   "keywords": [

package/server.json

@@ -7,7 +7,7 @@
     "url": "https://github.com/AVIDS2/memorix",
     "source": "github"
   },
-  "version": "1.6.3",
+  "version": "1.7.1",
   "websiteUrl": "https://github.com/AVIDS2/memorix#readme",
   "icons": [
     {
@@ -22,7 +22,7 @@
     {
       "registryType": "npm",
       "identifier": "memorix",
-      "version": "1.6.3",
+      "version": "1.7.1",
       "runtimeHint": "npx",
       "packageArguments": [
         {

package/src/cli/commands/cleanup.ts

@@ -17,88 +17,15 @@ import { defineCommand } from 'citty';
 import type { Observation } from '../../types.js';
 import { detectProject } from '../../project/detector.js';
 import { getProjectDataDir } from '../../store/persistence.js';
-import type { ObservationStore } from '../../store/obs-store.js';
 import { getObservationStore, initObservationStore } from '../../store/obs-store.js';
 import { filterReadableObservations } from '../../memory/visibility.js';
+import {
+    analyzeCleanupObservations,
+    applyCleanupMutations,
+    isLowQualityObservation,
+} from '../../memory/cleanup.js';
 
-/** Patterns that indicate auto-generated, low-value observations */
-const LOW_QUALITY_PATTERNS = [
-    /^Session activity/i,
-    /^Updated \S+\.\w+$/i,
-    /^Created \S+\.\w+$/i,
-    /^Deleted \S+\.\w+$/i,
-    /^Modified \S+\.\w+$/i,
-    /^Ran command:/i,
-    /^Read file:/i,
-];
-
-/** Patterns for demo/test noise — matches session.ts NOISE_PATTERNS */
-const NOISE_PATTERNS = [
-    /\[测试\]/i, /\[test\]/i, /验证/i, /兼容/i, /\bcompat(?:ibility)?\b/i,
-    /\bdemo\b/i, /展示/i, /全能力/i, /handoff/i, /交接/i,
-    /for_memmcp_test/i, /\bbenchmark\b/i, /\bsandbox\b/i, /\bplayground\b/i,
-];
-
-/** Patterns for Memorix system self-reference — should not pollute unrelated projects */
-const SYSTEM_SELF_PATTERNS = [
-    /memorix.demo/i, /memorix.*全能力/i, /memorix.*工具.*能力/i,
-    /memorix.*runtime.*mode/i, /memorix.*运行模式/i, /memorix.*control.plane/i,
-    /session.*inject(?:ion)?/i, /注入.*逻辑/i,
-    /\b22\s*(?:个|tools?).*(?:工具|能力|capabilit)/i,
-    /memorix.*(?:v\d|版本|version)/i, /memorix.*(?:兼容|compat)/i,
-    /memorix.*(?:测试|test)/i, /memmcp/i,
-];
-
-/** Check if an observation title matches low-quality patterns */
-function isLowQuality(title: string): boolean {
-    return LOW_QUALITY_PATTERNS.some(p => p.test(title.trim()));
-}
-
-/** Check if observation text matches noise/demo/test/system-self patterns */
-function isNoisePollution(obs: { title?: string; narrative?: string; entityName?: string; facts?: string[]; concepts?: string[] }): { isNoise: boolean; reason: string } {
-    const text = [obs.title, obs.narrative, obs.entityName, ...(obs.facts ?? []), ...(obs.concepts ?? [])]
-        .filter(Boolean).join('\n');
-    for (const p of SYSTEM_SELF_PATTERNS) {
-        if (p.test(text)) return { isNoise: true, reason: 'system-self' };
-    }
-    for (const p of NOISE_PATTERNS) {
-        if (p.test(text)) return { isNoise: true, reason: 'demo/test/noise' };
-    }
-    return { isNoise: false, reason: '' };
-}
-
-function requireObservationIds(observations: Observation[], action: string): number[] {
-    const ids = observations.map((observation) => observation.id);
-    if (ids.some((id) => typeof id !== 'number')) {
-        throw new Error(`Cannot ${action}: an observation has no persisted ID.`);
-    }
-    return ids as number[];
-}
-
-/**
- * Apply cleanup mutations without replacing the shared observation table.
- * Keeping lifecycle updates targeted prevents a cleanup in one project from
- * overwriting observations written concurrently by another project.
- */
-export async function applyCleanupMutations(
-    store: ObservationStore,
-    toArchive: Observation[],
-    toRemove: Observation[],
-): Promise<{ archived: number; removed: number }> {
-    const archiveIds = requireObservationIds(toArchive, 'archive');
-    const removeIds = requireObservationIds(toRemove, 'delete');
-    const removals = new Set(removeIds);
-    if (archiveIds.some((id) => removals.has(id))) {
-        throw new Error('Cleanup cannot archive and delete the same observation.');
-    }
-
-    await store.atomic(async (tx) => {
-        await Promise.all(archiveIds.map((id) => tx.setStatus(id, 'archived')));
-        await Promise.all(removeIds.map((id) => tx.remove(id)));
-    });
-
-    return { archived: archiveIds.length, removed: removeIds.length };
-}
+export { applyCleanupMutations } from '../../memory/cleanup.js';
 
 export default defineCommand({
     meta: {
@@ -170,40 +97,12 @@ export default defineCommand({
             return;
         }
 
-        // Categorize: low-quality
-        const lowQuality = projectObs.filter(o => isLowQuality(o.title ?? ''));
-        const highQuality = projectObs.filter(o => !isLowQuality(o.title ?? ''));
-
-        // Find duplicates (same title + type + entity)
-        const seen = new Set<string>();
-        const duplicates: typeof projectObs = [];
-        const unique: typeof projectObs = [];
-        for (const obs of highQuality) {
-            const key = `${obs.type}|${obs.title}|${obs.entityName}`;
-            if (seen.has(key)) {
-                duplicates.push(obs);
-            } else {
-                seen.add(key);
-                unique.push(obs);
-            }
-        }
-
-        // Find noise pollution (demo/test/system-self)
-        const noiseHits: Array<{ obs: typeof projectObs[0]; reason: string }> = [];
-        if (args.noise) {
-            for (const obs of projectObs) {
-                if (lowQuality.includes(obs) || duplicates.includes(obs)) continue;
-                const { isNoise, reason } = isNoisePollution(obs);
-                if (isNoise) noiseHits.push({ obs, reason });
-            }
-        }
-
-        const toRemove = [...lowQuality, ...duplicates];
-        const toArchive = noiseHits.map(h => h.obs);
+        const analysis = analyzeCleanupObservations(projectObs as Observation[], { includeNoise: args.noise });
+        const { lowQuality, duplicates, noise: noiseHits, toRemove, toArchive } = analysis;
 
         console.log(`Analysis (active observations for ${projectId}):`);
-        console.log(`   Total active:       ${projectObs.length}`);
-        console.log(`   High quality:       ${unique.length - toArchive.length}`);
+        console.log(`   Total active:       ${analysis.totalActive}`);
+        console.log(`   High quality:       ${analysis.highQuality}`);
         console.log(`   Low quality:        ${lowQuality.length}`);
         console.log(`   Duplicates:         ${duplicates.length}`);
         if (args.noise) {
@@ -224,7 +123,7 @@ export default defineCommand({
         if (toRemove.length > 0) {
             console.log('Items to DELETE:');
             toRemove.slice(0, 10).forEach(o => {
-                const tag = isLowQuality(o.title ?? '') ? '(low-quality)' : '(duplicate)';
+                const tag = isLowQualityObservation(o.title ?? '') ? '(low-quality)' : '(duplicate)';
                 console.log(`   ${tag} #${o.id ?? '?'} "${o.title}" [${o.type}]`);
             });
             if (toRemove.length > 10) {
@@ -236,8 +135,8 @@ export default defineCommand({
         // Preview noise archival
         if (toArchive.length > 0) {
             console.log('Items to ARCHIVE (noise pollution):');
-            noiseHits.slice(0, 15).forEach(({ obs, reason }) => {
-                console.log(`   (${reason}) #${obs.id ?? '?'} "${obs.title}" [${obs.type}] entity=${obs.entityName}`);
+            noiseHits.slice(0, 15).forEach(({ observation, reason }) => {
+                console.log(`   (${reason}) #${observation.id ?? '?'} "${observation.title}" [${observation.type}] entity=${observation.entityName}`);
             });
             if (toArchive.length > 15) {
                 console.log(`   ... and ${toArchive.length - 15} more`);

package/src/cli/commands/memory.ts

@@ -3,7 +3,9 @@ import { compactDetail, compactSearch, compactTimeline } from '../../compact/eng
 import { withFreshIndex } from '../../memory/freshness.js';
 import { getAllObservations, getObservation, getProjectObservations, resolveObservations, storeObservation, suggestTopicKey } from '../../memory/observations.js';
 import { buildGraphContextPacket, formatGraphContextPrompt } from '../../memory/graph-context.js';
+import { applyDeduplicationPlan, planMemoryDeduplication } from '../../memory/deduplication.js';
 import { canManageObservation, filterReadableObservations, resolveObservationVisibility } from '../../memory/visibility.js';
+import { getObservationStore } from '../../store/obs-store.js';
 import {
   asStringArg,
   coerceObservationStatus,
@@ -288,12 +290,11 @@ export default defineCommand({
             return;
           }
 
-          const { deduplicateMemory } = await import('../../llm/memory-manager.js');
           const allObs = await withFreshIndex(() =>
             filterReadableObservations(
               getAllObservations().filter((obs) => (obs.status ?? 'active') === 'active' && obs.projectId === project.id),
               reader,
-            ),
+            ).filter((observation) => canManageObservation(observation, reader)),
           );
 
           if (allObs.length < 2) {
@@ -310,55 +311,23 @@ export default defineCommand({
             candidates = allObs.slice(-20);
           }
 
-          const byEntity = new Map<string, typeof candidates>();
-          for (const obs of candidates) {
-            const bucket = byEntity.get(obs.entityName) ?? [];
-            bucket.push(obs);
-            byEntity.set(obs.entityName, bucket);
-          }
-
-          const actions: string[] = [];
-          const toResolve: number[] = [];
-          for (const [, group] of byEntity) {
-            if (group.length < 2) continue;
-            for (let index = 0; index < group.length; index += 1) {
-              for (let compareIndex = index + 1; compareIndex < group.length; compareIndex += 1) {
-                const newer = group[compareIndex];
-                const older = group[index];
-                try {
-                  const decision = await deduplicateMemory(
-                    { title: newer.title, narrative: newer.narrative, facts: newer.facts },
-                    [{ id: older.id, title: older.title, narrative: older.narrative, facts: older.facts.join('\n') }],
-                  );
-                  if (decision && (decision.action === 'DELETE' || decision.action === 'UPDATE' || decision.action === 'NONE')) {
-                    actions.push(`Resolve #${older.id} because it duplicates newer #${newer.id}`);
-                    toResolve.push(older.id);
-                  }
-                } catch {
-                  // Ignore failed pair analysis so one bad comparison doesn't abort the batch.
-                }
-              }
-            }
-          }
+          const plan = await planMemoryDeduplication(candidates);
+          const actionLines = plan.actions.map((item) =>
+            `Resolve #${item.resolveId} "${item.resolveTitle}"; keep #${item.keepId} "${item.keepTitle}" (${item.reason})`,
+          );
 
-          if (dryRun || toResolve.length === 0) {
+          if (dryRun || plan.resolveIds.length === 0) {
             emitResult(
-              { project, actions, resolved: [], dryRun: true },
-              actions.length === 0 ? 'No duplicate candidates found.' : actions.join('\n'),
+              { project, plan, resolved: [], dryRun: true },
+              actionLines.length === 0 ? 'No duplicate candidates found.' : actionLines.join('\n'),
               asJson,
             );
             return;
           }
 
-          const result = await resolveObservations(
-            [...new Set(toResolve)].filter((id) => {
-              const observation = getObservation(id, project.id);
-              return observation ? canManageObservation(observation, reader) : false;
-            }),
-            'resolved',
-          );
+          const result = await applyDeduplicationPlan(getObservationStore(), project.id, plan.resolveIds, reader);
           emitResult(
-            { project, actions, resolved: result.resolved, notFound: result.notFound, dryRun: false },
+            { project, plan, resolved: result.resolved, skipped: result.skipped, dryRun: false },
             `Resolved ${result.resolved.length} duplicate observation(s).`,
             asJson,
           );