@orquentra/codebuddy 0.0.0-stage → 2.4.0

package/ARCHITECTURE.md

@@ -0,0 +1,269 @@
+# Architecture
+
+## Storage Model
+
+CodeBuddy uses a dual-write transition model:
+
+- facts and summaries are written to PostgreSQL and Markdown
+- files live under `.codebuddy/memory/facts/` and
+  `.codebuddy/memory/summaries/`
+- PostgreSQL remains the active query/index layer
+- `codebuddy reindex --full` can recreate fact, summary, and pending embedding
+  records from those files
+- reindex is a reconciliation, not an append: rows are upserted by file ID with
+  the file's content, confidence, `sourceDeleted`, `supersededBy` and
+  `createdAt`; changed content gets a fresh pending embedding; a deleted file
+  deletes its row. Malformed or conflicting files are reported per file
+  (`failures[]`) and never abort the run or count as deletions. A running MCP
+  server re-imports on startup, after `context_after_turn`, and whenever a
+  managed file under `.codebuddy/memory/{facts,summaries}/` changes
+- ownership rule: only rows a reindex has imported from Markdown
+  (`file_backed = true`, set per namespace) are deleted when their file is
+  gone. Rows that predate migration `0003` or were never mirrored to a file
+  (e.g. snapshot shares) are left alone; once their file is seen by a reindex
+  they become file-backed. `source_interaction_id` is historical provenance,
+  not a foreign key, so facts re-import after the interactions are lost
+
+The files use versioned YAML front matter, atomic temporary-file writes,
+`fsync`, and rename. Repository-root validation rejects traversal and
+symlinked memory paths.
+
+The following data remains database-only:
+
+- raw interactions
+- share tombstones (live reference grants are mirrored in the source fact's `sharedWith`)
+- audit logs
+- model diagnostics
+- generated embedding vectors
+
+`codebuddy migrate to-files` is a non-writing preview. `--write` exports
+missing files but never truncates database tables or overwrites conflicts.
+Same-ID/different-content conflicts are reported with SHA-256 checksums.
+
+## Concurrency Model
+
+`remember` uses Postgres advisory locks keyed by `(namespace, session_id)` so concurrent writers for the same session do not race.
+
+Write idempotency rules:
+
+- callers may pass `idempotencyKey`
+- when omitted, CodeBuddy computes `sha256(namespace + session_id + content)`
+- `content_hash` exists on both `interactions` and `facts`
+- both tables enforce unique `(namespace_id, content_hash)`
+
+Transaction rules:
+
+- `remember` stores the interaction and the matching `embeddings` row with `status=pending` in one transaction
+- embedding generation happens outside the transaction
+- `share` is a single transaction across share records and audit inserts
+- `forget` performs cascade behavior inside one transaction
+
+## Embedding Pipeline
+
+Embeddings are asynchronous and run in an in-process worker.
+
+Embedding record lifecycle:
+
+- `pending`
+- `ready`
+- `failed`
+
+Operational fields:
+
+- `attempts`
+- `last_error`
+
+Embedding retrieval partitions rows by a canonical `(model, revision, schema,
+dimensions)` identity. Query embeddings retain this identity; both SQL candidate
+selection (before its limit) and the planner enforce an exact match before cosine
+scoring. A provider fallback writes its own partition and cannot score against the
+primary model's vectors. Missing compatible rows use the recency/summary fallback.
+Providers must return finite vectors with consistent dimensions and batch size.
+
+Provider adapters may supply `revision` and `schema` on embedding responses. These
+identify the actual deployed revision and preprocessing contract; custom adapters
+should use an immutable revision for pinned deployments. Omitted metadata uses
+`unversioned` / `provider-v1`. The hosted Hugging Face adapter identifies its schema
+as `hf-feature-extraction-v1` and uses `unversioned` unless deployment metadata is
+configured; the HF SDK does not expose revision pinning. A remote update behind an
+unversioned deployment cannot be detected locally, so changing a deployment requires
+changing its revision/schema identity and rebuilding the derived index.
+
+Migration `0003_embedding_space` requeues all legacy vectors with unknown space
+metadata, including failed jobs and jobs with stale leases. It preserves owners and
+model IDs, resets attempts/leases, and lets workers regenerate the appropriate
+partition. Recency/summary fallback remains available while this rebuild completes.
+No model space is inferred from vector dimensions.
+
+Recall candidate selection:
+
+- pgvector ranks every ready vector in the namespace that matches the query's
+  exact embedding space (source-deleted and superseded facts excluded), then the candidate
+  limit (default 256) applies; only IDs and scores are returned for hydration
+- `stats.candidateCoverage` reports compatible ready vectors, candidates, and
+  hydrated rows; `stats.fallbackReason` says why vector search was skipped
+
+Recall behavior when embeddings are not ready or the provider fails:
+
+- use recency and summary-based recall, plus lexical matches over durable
+  facts from every session in the namespace
+- skip pending vectors instead of blocking the caller
+
+Rationale:
+
+- better to return slightly stale ranking than block on Hugging Face cold starts or rate limits
+
+## Sharing Model
+
+Shares are references by default.
+
+Modes:
+
+- `reference`
+- `snapshot`
+
+Ownership: only the configured namespace can share its own facts (`from` must be
+the configured namespace). Repeating a share is idempotent.
+
+Reference mode:
+
+- target namespace points to the live source fact; `recall` in the target resolves
+  live references alongside its own embeddings (even when it has none of its own)
+- source updates propagate
+- deleting the source fact, or revoking with `unshare`, removes it from target recall
+- durable form: the source fact's Markdown front matter lists `sharedWith: [namespace]`;
+  `reindex` of the source project restores the share rows
+
+Snapshot mode:
+
+- target namespace receives a copy, owned by the target
+- the copy is written as a fact file (`predicate: snapshot_of`) into the target
+  project's own store, given explicitly as `targetProjectRoot` (MCP `to_project_root`),
+  whose configured namespace must match the target; it is never written under the
+  source project
+- later source changes do not propagate
+- source deletion does not delete the snapshot; the target's `reindex` rebuilds it after
+  database loss
+
+ASCII model:
+
+```text
+reference:
+source fact A -----> shared reference in target
+     update A -----> visible in target
+     delete A -----> target reference tombstoned (also on unshare)
+
+snapshot:
+source fact A -----> copied fact B in target
+     update A -----> no change to B
+     delete A -----> B remains
+```
+
+## Conflicts
+
+Facts are compared by an explicit `propositionKey` (fact front matter, mirrored
+to the `facts.proposition_key` column), never by rendered text. Facts sharing a
+key are rival assertions of one single-valued claim (for example
+`database.primary.engine`); facts without a key are multi-valued and are never
+grouped, so identical surface text cannot collapse distinct claims and
+different wording cannot hide a contradiction.
+
+Supported recall modes:
+
+- `all` (default): keep every rival assertion
+- `latest`: keep the newest assertion per key (then confidence, then id)
+- `highest_confidence`: keep the most confident assertion per key (then recency, then id)
+
+Superseded (`supersededBy`) and source-deleted facts are excluded before
+resolution. Each recalled fact message carries `fact` provenance (subject,
+predicate, object, key, confidence, createdAt, createdByAgent). When a key had
+rivals, `fact.conflict` lists the candidate ids and sets `unresolved: true` if
+differing objects remain undecided (any disagreement in `all`, an exact
+time/confidence tie otherwise); the system prompt then names those keys. The
+downstream agent reconciles unresolved contradictions. Reindex carries the
+file's confidence, timestamp, `sourceDeleted`, `supersededBy` and key into the
+database.
+
+Explicit decision capture can now include `supersedes fact_<id>`. The older
+fact remains on disk for audit history but is marked `supersededBy` and omitted
+from current automatic context. Incident-resolution capture similarly links to
+an unresolved incident on the same changed paths and sets `resolvedBy` instead
+of creating a duplicate incident.
+
+## MCP Surface
+
+The current MCP server intentionally ships tools only.
+
+Deferred:
+
+- resources such as `codebuddy://namespace/<name>/facts`
+- prompts
+
+Rationale:
+
+- tools are enough for first release workflows
+- resources and prompts add another compatibility surface and are deferred deliberately, not forgotten
+
+## Caching
+
+The background index maintains `.codebuddy/cache/architecture.json` as a
+versioned, rebuildable import-graph cache. Content hashes from `index.json`
+identify changed modules. When the source-file set is stable, only changed
+modules are reparsed and unchanged nodes/edges are reused; additions, removals,
+and renames trigger a correctness-first graph rebuild. Context retrieval shares
+one index snapshot across graph and symbol stages. Missing, corrupt,
+or incompatible caches fall back to rebuilding and never become source of
+truth.
+
+`CodeBuddyNode` supports recall caching with:
+
+```ts
+new CodeBuddyNode({ memory, mode: "recall", cacheTtlSeconds: 30 })
+```
+
+Default cache:
+
+- 30 seconds
+- keyed by `(namespace, sessionId, query)`
+
+This is especially useful in cyclic LangGraph flows that would otherwise repeat identical recall calls.
+
+## Logging
+
+Use `pino` for structured logs.
+
+Rules:
+
+- MCP stdio server logs JSON to stderr
+- CLI uses `pino-pretty` when attached to a TTY
+- CLI falls back to JSON logs when not attached to a TTY
+- logs must redact secrets such as `HF_TOKEN`
+- LangSmith should capture the same structured fields where tracing is enabled
+
+## Privacy
+
+- facts and summaries are plaintext Markdown
+- conversations, shares, telemetry, and indexes are plaintext in PostgreSQL
+- project owners decide whether `.codebuddy/memory/` is committed or ignored
+- encryption at rest is a deployment responsibility, not an app-layer feature
+- automatic PII detection is planned but not shipped
+- later releases may add sensitivity metadata and redaction workflows
+
+## Forget Semantics
+
+`forget(interactionId)`:
+
+- hard-deletes the interaction
+- hard-deletes its embedding
+- tombstones extracted facts with `source_deleted: true`
+
+`forget(factId)`:
+
+- hard-deletes the fact
+- hard-deletes its embedding
+- tombstones referenced facts in other namespaces unless they were shared as snapshots
+
+Audit requirements:
+
+- `model_calls` records provider calls only
+- persist forget, share, and write audit events in `audit_log`

package/CLAUDE_CODEX.md

@@ -0,0 +1,433 @@
+# Claude Code And Codex Setup
+
+This guide shows how to use CodeBuddy with Claude Code and Codex using the
+project's local PostgreSQL database and optional local Graphify architecture
+context.
+
+## Requirements
+
+- Node.js 20+
+- PostgreSQL 16 + pgvector
+- npm
+- Optional: Docker, if you explicitly choose the bundled database fallback
+
+## Install CodeBuddy
+
+Install the package globally:
+
+```bash
+npm install -g @orquentra/codebuddy
+```
+
+Check the CLI:
+
+```bash
+codebuddy --help
+```
+
+## Start PostgreSQL With Docker
+
+CodeBuddy ships with a Docker Compose file that runs PostgreSQL 16 with `pgvector`.
+
+Start the bundled database:
+
+```bash
+codebuddy postgres up
+```
+
+Check status:
+
+```bash
+codebuddy postgres status
+```
+
+The default database URL is:
+
+```text
+postgres://codebuddy:codebuddy@localhost:5432/codebuddy
+```
+
+You can also use raw Docker Compose from a local clone of this repository:
+
+```bash
+docker compose up -d
+docker compose ps
+```
+
+View logs:
+
+```bash
+docker compose logs postgres
+docker compose logs -f postgres
+```
+
+Open a PostgreSQL shell:
+
+```bash
+docker exec -it codebuddy-postgres psql -U codebuddy -d codebuddy
+```
+
+Inside `psql`, verify `pgvector`:
+
+```sql
+SELECT extname FROM pg_extension WHERE extname = 'vector';
+```
+
+Exit:
+
+```sql
+\q
+```
+
+Stop the database while preserving data:
+
+```bash
+codebuddy postgres down
+```
+
+or:
+
+```bash
+docker compose down
+```
+
+Reset the local database by deleting the Docker volume:
+
+```bash
+docker compose down
+docker volume ls | grep codebuddy
+docker volume rm <codebuddy-volume-name>
+docker compose up -d
+```
+
+Before resetting the volume, export any older database-only facts and
+summaries:
+
+```bash
+codebuddy migrate to-files
+codebuddy migrate to-files --write
+```
+
+After resetting the volume:
+
+```bash
+codebuddy migrate
+codebuddy reindex --full
+```
+
+Facts and summaries can be restored from `.codebuddy/memory/`. Raw
+interactions, shares, audit logs, and model diagnostics remain database-only
+and are lost with the volume.
+
+## Memory Privacy
+
+Markdown memory is plaintext. Inspect it before committing:
+
+```bash
+find .codebuddy/memory -type f -maxdepth 3
+```
+
+To keep all project memory private, add this to the project `.gitignore`:
+
+```gitignore
+.codebuddy/memory/
+```
+
+Do not place API tokens, passwords, private keys, or unredacted personal data
+in committed memory files.
+
+## Initialize CodeBuddy
+
+The shortest path is to open a project folder and run:
+
+```bash
+codebuddy
+```
+
+The setup asks whether to enable Graphify. Choose **yes** to install/detect
+Graphify, register both local MCP servers in `.mcp.json`, and enable Graphify
+architecture input. Choose **no** to register only CodeBuddy; its own local
+index and MCP context tools work the same way.
+
+Both choices also create `.codebuddy/config.json`, validate the local Postgres
+connection, apply migrations, install `CLAUDE.md` and `AGENTS.md` workflow
+instructions, and register Claude/Codex MCP entries. Credentials stay in the
+private CodeBuddy config, not in `.mcp.json`.
+
+When Graphify is enabled, setup builds the graph automatically. To refresh it
+after repository changes, run:
+
+```bash
+codebuddy graphify index
+```
+
+That creates or updates `graphify-out/graph.json`; CodeBuddy consumes it
+automatically for architecture context. `codebuddy serve` also prepares a
+missing or invalid graph before starting.
+
+Run the setup wizard:
+
+```bash
+codebuddy init
+```
+
+The legacy wizard asks for:
+
+- Optional Hugging Face token
+- PostgreSQL URL
+- default namespace
+- whether to register with Claude Desktop
+
+For local Docker PostgreSQL, use:
+
+```text
+postgres://codebuddy:codebuddy@localhost:5432/codebuddy
+```
+
+Run migrations manually if needed:
+
+```bash
+codebuddy migrate
+```
+
+Check health:
+
+```bash
+codebuddy doctor
+```
+
+Skip live model checks:
+
+```bash
+codebuddy doctor --skip-model-check
+```
+
+## One-Command Project Setup
+
+For each project, run:
+
+```bash
+cd /path/to/your-project
+codebuddy use
+```
+
+This command:
+
+- derives the namespace from the project folder
+- reuses saved Hugging Face and database settings
+- validates local PostgreSQL; Docker is only used with `codebuddy use --docker`
+- applies migrations
+- registers Claude Desktop
+- registers Codex
+- asks whether to enable Graphify and writes the project MCP config
+- installs managed `CLAUDE.md` and `AGENTS.md` instructions
+
+Use a custom namespace:
+
+```bash
+codebuddy use my-project
+```
+
+Use a custom PostgreSQL URL:
+
+```bash
+codebuddy use --postgres-url "postgres://user:password@host:5432/database"
+```
+
+Skip Claude Desktop registration:
+
+```bash
+codebuddy use --skip-claude
+```
+
+Skip Codex registration:
+
+```bash
+codebuddy use --skip-codex
+```
+
+## Claude Code Setup
+
+Claude Code reads MCP servers from a project `.mcp.json` file.
+
+In your project:
+
+```bash
+cd /path/to/your-project
+```
+
+Create `.mcp.json`:
+
+```json
+{
+  "mcpServers": {
+    "codebuddy": {
+      "command": "codebuddy",
+      "args": ["serve"],
+      "env": {
+        "DATABASE_URL": "postgres://codebuddy:codebuddy@localhost:5432/codebuddy",
+        "HF_TOKEN": "hf_your_token",
+        "CODEBUDDY_NAMESPACE": "your-project"
+      }
+    }
+  }
+}
+```
+
+Replace:
+
+- `hf_your_token` with your Hugging Face token
+- `your-project` with your namespace
+
+Restart Claude Code after creating or editing `.mcp.json`.
+
+Test from Claude Code by asking:
+
+```text
+Remember that this project uses CodeBuddy for MCP memory.
+```
+
+Then ask:
+
+```text
+Recall what you know about this project.
+```
+
+## Codex Setup
+
+The easiest Codex setup is:
+
+```bash
+cd /path/to/your-project
+codebuddy use
+```
+
+Then restart Codex.
+
+Check Codex entries:
+
+```bash
+codebuddy codex list
+```
+
+Install a Codex entry manually:
+
+```bash
+codebuddy codex install --namespace your-project
+```
+
+Remove a Codex entry:
+
+```bash
+codebuddy codex remove codebuddy-your-project
+```
+
+CodeBuddy writes Codex MCP configuration to:
+
+```text
+~/.codex/config.toml
+```
+
+The generated entry looks like:
+
+```toml
+[mcp_servers."codebuddy-your-project"]
+command = "codebuddy"
+args = ["serve"]
+
+[mcp_servers."codebuddy-your-project".env]
+CODEBUDDY_NAMESPACE = "your-project"
+DATABASE_URL = "postgres://codebuddy:codebuddy@localhost:5432/codebuddy"
+HF_TOKEN = "hf_your_token"
+```
+
+Restart Codex after installing or changing an MCP entry.
+
+## Useful CLI Commands
+
+List namespaces:
+
+```bash
+codebuddy namespaces
+```
+
+Inspect a namespace:
+
+```bash
+codebuddy inspect your-project
+```
+
+List facts:
+
+```bash
+codebuddy list-facts
+```
+
+Export a namespace:
+
+```bash
+codebuddy export your-project > codebuddy-memory.json
+```
+
+Show storage and usage stats:
+
+```bash
+codebuddy stats
+```
+
+Delete old memories:
+
+```bash
+codebuddy prune --older-than 30d
+```
+
+## Troubleshooting
+
+If PostgreSQL is not reachable:
+
+```bash
+codebuddy postgres up
+codebuddy doctor --skip-model-check
+```
+
+If Docker is not running, start Docker first, then run:
+
+```bash
+codebuddy postgres up
+```
+
+If port `5432` is already in use:
+
+```bash
+lsof -i :5432
+```
+
+Stop the conflicting service or use another PostgreSQL URL.
+
+If migrations are missing:
+
+```bash
+codebuddy migrate
+```
+
+If Claude Code does not show CodeBuddy tools:
+
+- confirm `.mcp.json` is in the project root
+- confirm `HF_TOKEN`, `DATABASE_URL`, and `CODEBUDDY_NAMESPACE`
+- restart Claude Code
+
+If Codex does not show CodeBuddy tools:
+
+```bash
+codebuddy codex list
+```
+
+Then restart Codex.
+
+If model checks fail because a Hugging Face model is cold, gated, or rate-limited, run:
+
+```bash
+codebuddy doctor --skip-model-check
+```
+
+Database and MCP setup can still be valid even when live Hugging Face diagnostics are temporarily unavailable.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Orquentra Inc
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.