claude_memory 0.9.1 → 0.11.0

data/docs/reflection_memory_as_accumulating_judgment.md

@@ -0,0 +1,67 @@
+# Memory as Accumulating Judgment
+
+*A reflection on what ClaudeMemory is really doing, and a simple formula for applying AI to engineering teams at any scale.*
+
+---
+
+## The one thing
+
+**What accumulates, wins.** Intelligence is now cheap; context is the expensive part. Every AI interaction without memory pays full cost to re-derive what was already known. Memory converts ephemeral intelligence into accumulating judgment — the same thing that makes senior engineers valuable.
+
+## Thoughts on this project
+
+ClaudeMemory is betting on the right axis. Not "make the model smarter" (commodified, Anthropic's job), but "make what's already known stop disappearing." The dual-database split (global vs project), provenance, supersession, and predicate vocabulary are all machinery in service of one goal: **judgment that persists past the session boundary**. The open conflicts and distiller-hallucination churn visible in this repo are the honest signal — accumulation is the hard problem, not retrieval.
+
+## 10 theories (in priority order)
+
+1. **Context-rebuild dominates cost.** Token count matters less than relevance ratio.
+2. **Decisions have half-lives.** Memory's job is slowing decay, not freezing truth.
+3. **Why > what.** Remembering the reason behind a decision outvalues remembering the decision.
+4. **Scope layering beats flat memory.** global → team → project → role → session.
+5. **Corrections compound.** Each remembered correction reduces N future corrections.
+6. **Onboarding = context-rebuild.** Same problem, different substrate.
+7. **Small teams need memory more.** No tribal knowledge to fall back on.
+8. **Conflict is signal, not noise.** Unresolved conflicts mark where judgment is actually forming.
+9. **Provenance is trust.** Facts without sources are guesses at scale.
+10. **Surface area of trust is the real moat.** The more an AI remembers correctly, the more humans delegate.
+
+## Where examples break
+
+- **Solo dev, greenfield project:** memory underperforms — nothing to remember yet. Theory 7 bends: memory needs *inputs* before it pays off.
+- **Huge monorepo, 200 engineers:** global memory collides constantly. Theory 4 becomes load-bearing — without scope, memory becomes noise.
+- **Short-lived prototypes:** accumulation cost > payoff. Theories 1 and 5 invert.
+- **Rapidly-evolving codebase:** half-life is short (theory 2), memory goes stale faster than it's written. Supersession machinery has to outpace change.
+
+**Pattern:** memory's value is a function of churn rate and team size, not raw code volume.
+
+## The formula
+
+Strip it to one relation, Ohm's-Law style:
+
+> **V = R / C**
+>
+> - `V` = value per interaction
+> - `R` = judgment retained from prior interactions (corrections, decisions, why-reasons)
+> - `C` = context that must be rebuilt from scratch each time
+
+When `C → 0`, `V → ∞`. When `R → 0`, `V → 0`. Everything ClaudeMemory does — FTS5, semantic recall, provenance, supersession — is in service of raising `R` and lowering `C`.
+
+## At scale
+
+Let `C = Σ Cᵢ` across scope layers (personal, project, team, org):
+
+- **Solo:** `C ≈ C_personal`. Memory wins by remembering your preferences and past decisions. Small surface, high per-unit payoff.
+- **Small team (2–10):** `C ≈ C_project + C_personal`. Memory wins by codifying conventions so the team doesn't re-litigate them weekly.
+- **Org (50+):** `C ≈ C_org + C_team + C_project`. Memory must be scoped or it becomes noise. Value shifts from *remembering* to *routing* — the right fact to the right person at the right moment.
+
+**Rule of thumb:** apply AI where `R/C` is highest. That's wherever the same context is re-established most often — code review, architecture decisions, onboarding, incident postmortems. Avoid where `R` can't accumulate (one-off scripts, throwaway prototypes).
+
+## The delivered value, simply
+
+> **To whom:** engineers re-deciding things they already decided.
+>
+> **For whom:** the future version of themselves and their teammates.
+>
+> **What:** the elimination of re-derivation.
+
+Memory is not a feature. It's the thing that makes AI-for-engineering a compounding asset instead of a rentable tool.

data/lib/claude_memory/activity_log.rb

@@ -0,0 +1,86 @@
+# frozen_string_literal: true
+
+require "json"
+
+module ClaudeMemory
+  # Records activity events for debugging and observability.
+  # Events are stored in the activity_events table and surfaced
+  # via the dashboard and `memory.activity` MCP tool.
+  module ActivityLog
+    module_function
+
+    # Record an activity event in the given store.
+    #
+    # @param store [Store::SQLiteStore] database to write to
+    # @param event_type [String] e.g. "hook_ingest", "hook_context", "recall"
+    # @param status [String] "success", "skipped", or "error"
+    # @param session_id [String, nil] Claude session ID
+    # @param duration_ms [Integer, nil] operation duration in milliseconds
+    # @param details [Hash, nil] event-specific metadata
+    def record(store, event_type:, status:, session_id: nil, duration_ms: nil, details: nil)
+      store.activity_events.insert(
+        event_type: event_type,
+        session_id: session_id,
+        status: status,
+        duration_ms: duration_ms,
+        detail_json: details&.to_json,
+        occurred_at: Time.now.utc.iso8601
+      )
+    rescue => e
+      ClaudeMemory.logger.warn("activity_log",
+        message: "Failed to record event",
+        event_type: event_type,
+        status: status,
+        error_class: e.class.name,
+        error: e.message,
+        backtrace: e.backtrace&.first(3))
+      nil
+    end
+
+    # Query recent activity events.
+    #
+    # @param store [Store::SQLiteStore] database to read from
+    # @param limit [Integer] max events to return
+    # @param event_type [String, nil] filter by type
+    # @param since [String, nil] ISO 8601 lower bound
+    # @return [Array<Hash>] event records with parsed details
+    def recent(store, limit: 50, event_type: nil, since: nil)
+      dataset = store.activity_events.order(Sequel.desc(:occurred_at)).limit(limit)
+      dataset = dataset.where(event_type: event_type) if event_type
+      dataset = dataset.where { occurred_at >= since } if since
+
+      dataset.all.map do |row|
+        row[:details] = row[:detail_json] ? JSON.parse(row[:detail_json], symbolize_names: true) : nil
+        row.delete(:detail_json)
+        row
+      end
+    rescue => e
+      ClaudeMemory.logger.warn("activity_log", message: "Failed to query events", error: e.message)
+      []
+    end
+
+    # Summarize activity counts grouped by event_type.
+    #
+    # @param store [Store::SQLiteStore]
+    # @param since [String, nil] ISO 8601 lower bound
+    # @return [Hash] e.g. {"hook_ingest" => {success: 5, error: 1}, ...}
+    def summary(store, since: nil)
+      dataset = store.activity_events
+      dataset = dataset.where { occurred_at >= since } if since
+
+      rows = dataset
+        .group_and_count(:event_type, :status)
+        .all
+
+      result = {}
+      rows.each do |row|
+        result[row[:event_type]] ||= {}
+        result[row[:event_type]][row[:status].to_sym] = row[:count]
+      end
+      result
+    rescue => e
+      ClaudeMemory.logger.warn("activity_log", message: "Failed to summarize events", error: e.message)
+      {}
+    end
+  end
+end

data/lib/claude_memory/commands/census_command.rb

@@ -0,0 +1,210 @@
+# frozen_string_literal: true
+
+require "json"
+require "digest"
+
+module ClaudeMemory
+  module Commands
+    # Aggregates predicate/entity/schema usage across many ClaudeMemory databases
+    # into a privacy-safe JSON report. Used for informed vocabulary curation
+    # across machines without exposing content, names, or paths.
+    #
+    # What's emitted: schema versions, fact counts by predicate × status,
+    # entity type counts, novel predicates (outside the curated vocabulary),
+    # and synonym candidates (novel predicates overlapping known ones).
+    #
+    # What's *never* emitted: object_literal, entity names, project paths,
+    # provenance quotes, raw session IDs.
+    class CensusCommand < BaseCommand
+      DB_FILENAME = "memory.sqlite3"
+      TOP_PREDICATES_PER_DB = 5
+      SYNONYM_OVERLAP_THRESHOLD = 0.4
+      DEFAULT_ROOT = "~/src"
+
+      def call(args)
+        opts = parse_options(args, {root: DEFAULT_ROOT, output: nil, pretty: false, include_global: true}) do |o|
+          OptionParser.new do |parser|
+            parser.banner = "Usage: claude-memory census [options]"
+            parser.on("--root DIR", "Directory to scan (default: #{DEFAULT_ROOT})") { |v| o[:root] = v }
+            parser.on("--output FILE", "Write JSON to file instead of stdout") { |v| o[:output] = v }
+            parser.on("--pretty", "Pretty-print JSON output") { o[:pretty] = true }
+            parser.on("--no-global", "Skip the global database (~/.claude/memory.sqlite3)") { o[:include_global] = false }
+          end
+        end
+        return 1 if opts.nil?
+
+        root = File.expand_path(opts[:root])
+        paths = discover_databases(root)
+        paths << global_db_path if opts[:include_global] && File.exist?(global_db_path) && !paths.include?(global_db_path)
+
+        if paths.empty?
+          stderr.puts "No ClaudeMemory databases found under #{root}"
+          return 0
+        end
+
+        report = build_report(paths)
+        json = opts[:pretty] ? JSON.pretty_generate(report) : JSON.generate(report)
+
+        if opts[:output]
+          File.write(opts[:output], json)
+          stderr.puts "Census: scanned #{paths.size} database(s); wrote #{opts[:output]}"
+        else
+          stdout.puts json
+        end
+
+        0
+      end
+
+      private
+
+      def global_db_path
+        ClaudeMemory::Configuration.new.global_db_path
+      end
+
+      def discover_databases(root)
+        return [] unless Dir.exist?(root)
+        Dir.glob(File.join(root, "**", ".claude", DB_FILENAME)).sort
+      end
+
+      def build_report(paths)
+        known = ClaudeMemory::Resolve::PredicatePolicy.known_predicates.to_set
+
+        report = {
+          version: ClaudeMemory::VERSION,
+          generated_at: Time.now.utc.iso8601,
+          database_count: paths.size,
+          schema_versions: Hash.new(0),
+          totals: {facts: Hash.new(0), entities: 0, content_items: 0},
+          predicates: {},
+          entity_types: Hash.new(0),
+          novel_predicates: [],
+          synonym_candidates: [],
+          databases: []
+        }
+
+        predicates = Hash.new { |h, k| h[k] = {total: 0, by_status: Hash.new(0), db_count: 0} }
+
+        paths.each do |path|
+          summary = scan_database(path)
+          next unless summary
+
+          report[:schema_versions][summary[:schema_version].to_s] += 1 if summary[:schema_version]
+          report[:totals][:entities] += summary[:entity_count]
+          report[:totals][:content_items] += summary[:content_count]
+
+          summary[:facts_by_status].each { |status, count| report[:totals][:facts][status] += count }
+
+          summary[:entity_types].each { |type, count| report[:entity_types][type.to_s] += count }
+
+          summary[:predicates].each do |predicate, statuses|
+            entry = predicates[predicate]
+            entry[:db_count] += 1
+            statuses.each do |status, count|
+              entry[:total] += count
+              entry[:by_status][status] += count
+            end
+          end
+
+          report[:databases] << anonymize_db(path, summary)
+        end
+
+        report[:predicates] = predicates.each_with_object({}) do |(predicate, entry), acc|
+          acc[predicate] = {
+            total: entry[:total],
+            by_status: entry[:by_status],
+            db_count: entry[:db_count],
+            known: known.include?(predicate)
+          }
+        end
+
+        report[:novel_predicates] = predicates.keys.reject { |p| known.include?(p) }.sort
+        report[:synonym_candidates] = synonym_candidates(report[:novel_predicates], known)
+
+        report
+      end
+
+      def scan_database(path)
+        db = Sequel.connect("extralite://#{path}")
+
+        schema_version = begin
+          db[:meta].where(key: "schema_version").get(:value)&.to_i
+        rescue Sequel::DatabaseError
+          nil
+        end
+
+        facts_by_status = db[:facts].group_and_count(:status).all.each_with_object(Hash.new(0)) do |row, acc|
+          acc[row[:status].to_s] += row[:count].to_i
+        end
+
+        predicates = db[:facts].select(:predicate, :status).group_and_count(:predicate, :status).all
+          .each_with_object(Hash.new { |h, k| h[k] = Hash.new(0) }) do |row, acc|
+            acc[row[:predicate].to_s][row[:status].to_s] += row[:count].to_i
+        end
+
+        entity_types = db[:entities].group_and_count(:type).all.each_with_object(Hash.new(0)) do |row, acc|
+          acc[row[:type].to_s] += row[:count].to_i
+        end
+
+        entity_count = db[:entities].count
+        content_count = db[:content_items].count
+
+        {
+          schema_version: schema_version,
+          facts_by_status: facts_by_status,
+          predicates: predicates,
+          entity_types: entity_types,
+          entity_count: entity_count,
+          content_count: content_count
+        }
+      rescue Sequel::DatabaseError, Extralite::Error => e
+        stderr.puts "Skipping #{path}: #{e.message}"
+        nil
+      ensure
+        db&.disconnect
+      end
+
+      def anonymize_db(path, summary)
+        top = summary[:predicates]
+          .map { |predicate, statuses| [predicate, statuses.values.sum] }
+          .sort_by { |(_, count)| -count }
+          .first(TOP_PREDICATES_PER_DB)
+          .to_h
+
+        {
+          id: Digest::SHA256.hexdigest(path)[0, 12],
+          schema_version: summary[:schema_version],
+          facts: summary[:facts_by_status],
+          entities: summary[:entity_count],
+          content_items: summary[:content_count],
+          top_predicates: top
+        }
+      end
+
+      def synonym_candidates(novels, known)
+        novels.each_with_object([]) do |novel, acc|
+          novel_tokens = tokenize(novel)
+          next if novel_tokens.empty?
+
+          best = known.map do |canonical|
+            {canonical: canonical, overlap: jaccard(novel_tokens, tokenize(canonical))}
+          end.max_by { |candidate| candidate[:overlap] }
+
+          next unless best && best[:overlap] >= SYNONYM_OVERLAP_THRESHOLD
+
+          acc << {novel: novel, closest_known: best[:canonical], overlap: best[:overlap].round(2)}
+        end
+      end
+
+      def tokenize(predicate)
+        predicate.to_s.downcase.split(/[_\s-]+/).reject(&:empty?).to_set
+      end
+
+      def jaccard(a, b)
+        return 0.0 if a.empty? || b.empty?
+        intersection = (a & b).size
+        union = (a | b).size
+        union.zero? ? 0.0 : intersection.to_f / union
+      end
+    end
+  end
+end

data/lib/claude_memory/commands/completion_command.rb

@@ -80,6 +80,9 @@ module ClaudeMemory
                   completion)
                     _arguments '--shell[Shell type]:shell:(bash zsh)'
                     ;;
+                  dashboard)
+                    _arguments '--port[Server port]:port:' '--no-open[Skip browser open]'
+                    ;;
                   install-skill)
                     local -a skills
                     skills=(#{skill_names.map { |s| "'#{s}'" }.join(" ")})

data/lib/claude_memory/commands/dashboard_command.rb

@@ -0,0 +1,54 @@
+# frozen_string_literal: true
+
+require "optparse"
+
+module ClaudeMemory
+  module Commands
+    # Starts a local web dashboard for inspection and trust signals. Default
+    # port 3377; reads global + project databases, no writes from page loads.
+    # Panels: feed (Moments), Trust sidebar (utilization, fingerprint,
+    # feedback ratio, needs-review), Knowledge (predicate-grouped facts),
+    # Conflicts (deduped at display layer), Reuse, Health, Timeline,
+    # Activity drill-down, Query tester. Each panel is backed by a dedicated
+    # `Dashboard::*` collaborator class.
+    #
+    # See `docs/dashboard.md` for the user-facing guide.
+    class DashboardCommand < BaseCommand
+      def call(args)
+        opts = parse_options(args, {port: Dashboard::Server::DEFAULT_PORT, no_open: false}) do |o|
+          OptionParser.new do |parser|
+            parser.banner = "Usage: claude-memory dashboard [options]"
+            parser.on("--port PORT", Integer, "Server port (default: #{Dashboard::Server::DEFAULT_PORT})") { |v| o[:port] = v }
+            parser.on("--no-open", "Don't auto-open browser") { o[:no_open] = true }
+          end
+        end
+        return 1 if opts.nil?
+
+        manager = Store::StoreManager.new
+
+        unless manager.global_exists? || manager.project_exists?
+          stderr.puts "No memory databases found. Run 'claude-memory init' first."
+          manager.close
+          return 1
+        end
+
+        manager.ensure_both! if manager.global_exists? && manager.project_exists?
+        manager.ensure_global! if manager.global_exists?
+        manager.ensure_project! if manager.project_exists?
+
+        stdout.puts "Starting ClaudeMemory dashboard on http://localhost:#{opts[:port]}"
+        stdout.puts "Press Ctrl+C to stop."
+
+        server = Dashboard::Server.new(
+          manager: manager,
+          port: opts[:port],
+          open_browser: !opts[:no_open]
+        )
+
+        server.start
+        manager.close
+        0
+      end
+    end
+  end
+end

data/lib/claude_memory/commands/dedupe_conflicts_command.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+require "optparse"
+
+module ClaudeMemory
+  module Commands
+    # One-time cleanup for historical conflict-row duplication caused by a
+    # resolver bug that created a new disputed fact + conflict row each time
+    # the same contradicting value was re-extracted (see
+    # Resolver#apply_conflict dedupe fix landed 2026-04-24). New conflicts
+    # can no longer duplicate this way; this command cleans the tail.
+    class DedupeConflictsCommand < BaseCommand
+      def call(args)
+        opts = parse_options(args, {scope: "project", dry_run: false}) do |o|
+          OptionParser.new do |parser|
+            parser.banner = "Usage: claude-memory dedupe-conflicts [options]"
+            parser.on("--scope SCOPE", %w[project global], "Database scope (default: project)") { |v| o[:scope] = v }
+            parser.on("--dry-run", "Show what would be resolved without writing") { o[:dry_run] = true }
+          end
+        end
+        return 1 if opts.nil?
+
+        manager = ClaudeMemory::Store::StoreManager.new
+        store = manager.store_for_scope(opts[:scope])
+
+        begin
+          result = Sweep::Maintenance.new(store).dedupe_open_conflicts(dry_run: opts[:dry_run])
+        ensure
+          manager.close
+        end
+
+        print_result(opts, result)
+        0
+      end
+
+      private
+
+      def print_result(opts, result)
+        mode = opts[:dry_run] ? "DRY RUN" : "DEDUPE"
+        stdout.puts "#{mode}: scope=#{opts[:scope]}"
+        stdout.puts "=" * 50
+        stdout.puts "Conflicts inspected: #{result[:inspected]}"
+        stdout.puts "Duplicates resolved: #{result[:resolved]}"
+
+        return if result[:decisions].empty?
+
+        stdout.puts
+        stdout.puts "Decisions:"
+        result[:decisions].each do |d|
+          stdout.puts "  conflict ##{d[:conflict_id]} -> merged into ##{d[:keeper_id]} (rejects fact ##{d[:duplicate_fact_id]})"
+        end
+      end
+    end
+  end
+end