@amg555/frameforge-mcp 2.7.6 → 2.7.8

package/.agent/skills/gstack-setup-gbrain/memory.md

@@ -1,426 +0,0 @@
-# gstack memory ingest — what it does, what stays local, what you can do with it
-
-This is the user-facing reference for the transcript + memory ingest
-feature in `/setup-gbrain` and `/sync-gbrain`. If either asked whether to
-ingest your coding-agent sessions into gbrain, this doc explains what each
-answer does.
-
-## What gets ingested
-
-| Source | Type | Where | Sensitivity |
-|---|---|---|---|
-| Claude Code session JSONL | `transcript` | `~/.claude/projects/*/` | High — full conversations including tool I/O |
-| Codex CLI session JSONL | `transcript` | `~/.codex/sessions/YYYY/MM/DD/` | High |
-| Cursor session SQLite (V1.0.1) | `transcript` | `~/Library/Application Support/Cursor/` | Same — deferred V1.0.1 |
-| Eureka log | `eureka` | `~/.gstack/analytics/eureka.jsonl` | Medium — your insights, often non-secret |
-| Project learnings | `learning` | `~/.gstack/projects/<slug>/learnings.jsonl` | Medium |
-| Project timeline | `timeline` | `~/.gstack/projects/<slug>/timeline.jsonl` | Low |
-| CEO plans | `ceo-plan` | `~/.gstack/projects/<slug>/ceo-plans/*.md` | Medium |
-| Design docs | `design-doc` | `~/.gstack/projects/<slug>/*-design-*.md` | Medium |
-| Retros | `retro` | `~/.gstack/projects/<slug>/retros/*.md` | Medium |
-| Builder profile | `builder-profile-entry` | `~/.gstack/builder-profile.jsonl` | Low |
-
-## Transcripts
-
-Curated memory (learnings, timeline, plans, designs, retros, eureka, builder
-profile) syncs on every `/sync-gbrain`. Session transcripts sync only after
-you choose to share them. The choice is the `transcript_ingest_mode` config
-key, optionally narrowed by the `transcript_repos` allowlist:
-
-| Value | What `/sync-gbrain` ingests |
-|---|---|
-| `recent` | Sessions from the last 90 days |
-| `all` | All history (`--full` walks every session file) |
-| `new@2026-10-03T17:00:00Z` | Only sessions whose first record is at or after that UTC time. A session that started earlier stays out even when it is appended to later. A session with no start timestamp stays out. |
-| `off` | No transcripts; other memory still syncs |
-| `recent+repos`, `all+repos`, `new@…+repos` | The same window, only for the repos in `transcript_repos`. gstack adds and removes `+repos` itself. |
-
-Both sources are covered: Claude Code sessions (`~/.claude/projects/`) and
-Codex sessions (`~/.codex/sessions/`), from every project on this machine
-that your per-remote trust policy allows, not only the current repo. Pages go
-to your brain: the local engine (PGLite or Supabase) in local-stdio mode, or
-the artifacts repo the remote brain pulls from in remote-http mode.
-
-Set or change it:
-
-```bash
-gstack-config set transcript_ingest_mode recent   # or all, or off
-gstack-config set transcript_ingest_mode "new@$(date -u +%Y-%m-%dT%H:%M:%SZ)"   # only sessions from now on
-gstack-config set transcript_repos github.com/acme/app,github.com/acme/api       # only these repos
-gstack-config unset transcript_repos                                             # every repo again
-gstack-config has transcript_ingest_mode && gstack-config get transcript_ingest_mode
-bun run lib/transcript-consent.ts --describe                                     # the effective consent in words
-```
-
-`gstack-config set` rejects any other value and keeps the stored one. The
-cutoff takes only the `Z` (UTC) form, to the second. `transcript_repos`
-accepts any git remote spelling (`https://…`, `git@…:…`, with or without
-`.git`) and stores its canonical form; an empty list is rejected. Setting
-or clearing the allowlist updates the `+repos` marker in the same write.
-`gstack-config set transcript_ingest_mode <base>` keeps the marker while an
-allowlist exists, and `off` never carries it.
-
-**Several config roots.** When gstack reads more than one state root (for
-example `GSTACK_HOME` plus `~/.gstack`), the most restrictive answer wins:
-`off` in any root means off, the latest cutoff and any 90-day window both
-apply, and only repos in every root's allowlist count. The resolved root
-must hold the consent itself.
-
-**Not set.** Until you choose, transcripts are skipped. Each sync prints one
-line saying so, even with `--quiet`, and the next interactive `/sync-gbrain`
-asks once. Older values from previous gate versions (`A`-`E`, `incremental`)
-and values this version does not recognize also count as not chosen and get
-the question again. A stored `off` prints `transcripts off (your choice)`
-without `--quiet` and is never asked again.
-
-**One-run override.** A `--sources` list (or `GSTACK_MEMORY_INGEST_SOURCES`)
-that names `transcript` ingests transcripts for that run whatever the mode;
-the sync still prints the mode notice. An override never widens a scoped
-consent: a `new@` cutoff and the `transcript_repos` allowlist still apply.
-`--sources all`, an empty list, or a list with no valid types does not
-override.
-
-```bash
-bun run bin/gstack-gbrain-sync.ts --incremental --sources transcript
-```
-
-**What each sync tells you.** Without `--quiet` the sync prints the consent
-in words (window or cutoff, up to three repos, the config roots it came
-from). The ingest summary counts sessions left out by reason: before the
-cutoff, no start time, not in the allowlist, and repo policy deny. A sync
-under `new@` that ingests nothing says so and says new sessions will appear
-on the next sync.
-
-**Interrupted imports.** If a sync was interrupted mid-import and your
-transcript choice changed before the next run (the mode, the cutoff or the
-allowlist), the next sync restages memory from scratch once instead of
-resuming, and prints a line saying so.
-
-**Staged pages.** Transcript pages already staged under
-`~/.gstack/transcripts/` (remote-http mode) wait while the mode is not
-`recent`, `all` or `new@…`: they stay on disk and are not pushed by
-`gstack-brain-sync`. Other artifacts keep pushing. When you narrow a scoped
-consent (a later cutoff, a smaller allowlist), the next sync removes the
-staged pages that are now out of scope and prints `removed N staged
-transcript pages outside the new scope`; widening again re-stages them from
-the original sessions. Before every push, `gstack-brain-sync` also checks
-its own unpushed commits and rewrites them without any page outside your
-consent (`removed N excluded transcript pages from unpublished sync
-commits`). The removal covers staged and unpushed pages only.
-
-**Already-ingested transcripts stay.** Switching to `off` or narrowing the
-scope stops new ingests; it does not remove pages already in the brain or
-already pushed. Delete them there (see "Delete a page" below).
-
-**Downgrading gstack.** A gstack that predates `new@` and `transcript_repos`
-reads those values as unrecognized and ingests no transcripts
-automatically. It does honour an explicit transcript override without any
-scope, so before downgrading: remove `--sources transcript` and
-`GSTACK_MEMORY_INGEST_SOURCES=transcript` from scripts, and let pending sync
-publish, or run `gstack-config set transcript_ingest_mode off`.
-
-## What stays local
-
-- **State files** (`~/.gstack/.gbrain-sync-state.json`,
-  `~/.gstack/.transcript-ingest-state.json`,
-  `~/.gstack/.gbrain-engine-cache.json`,
-  `~/.gstack/.gbrain-errors.jsonl`) are local-only per ED1 (state file
-  sync semantics decision). They are not synced via the brain remote.
-
-- **Sessions with no resolvable git remote** (running in `/tmp/`, scratch
-  dirs, etc.) are skipped by default. Pass `--include-unattributed` to
-  the ingest helper to opt them in.
-
-- **Repos under a `deny` trust policy** (set in `/setup-gbrain` Step 6)
-  are skipped — neither code nor transcripts from those repos ingest.
-
-## Per-remote trust policy (deny / read-only)
-
-Transcript ingest respects the same per-remote trust store as code import
-(`~/.gstack/gbrain-repo-policy.json`, managed by
-`gstack-gbrain-repo-policy`). Each transcript's git remote is checked
-against the store before anything is written:
-
-- **deny** — the transcript is skipped (reported as `skipped (policy deny)`).
-- **read-only** — skipped too: read-only means "search allowed, page
-  writes never", and transcript ingest writes pages (reported as
-  `skipped (policy read-only)`).
-- **read-write, or no entry** — ingests normally.
-- **Corrupted or unreadable store** — ingestion aborts before any writes
-  rather than bypassing a set policy. Inspect the store with
-  `gstack-gbrain-repo-policy list`; re-run `/setup-gbrain` if it's corrupt.
-
-Artifacts (learnings, plans, retros, etc.) are never policy-filtered — the
-policy is keyed by git remote, which artifacts don't have.
-
-## What gets scanned for secrets
-
-The cross-machine secret boundary is `gstack-brain-sync` (the git push
-to your private artifacts repo), which runs its own scanner before any
-content leaves this Mac. Local PGLite ingest doesn't change the exposure
-surface for content that already lives on disk in plaintext.
-
-Per-file **gitleaks** scanning during memory ingest is **opt-in** as of
-v1.33.0.0 — off by default. To re-enable it (adds ~4-8 min to cold runs
-on a large transcript corpus), use either:
-
-```bash
-bun run bin/gstack-memory-ingest.ts --bulk --scan-secrets
-# or
-GSTACK_MEMORY_INGEST_SCAN_SECRETS=1 bun run bin/gstack-memory-ingest.ts --bulk
-```
-
-When enabled, gitleaks scans each rendered page, the exact markdown that
-gets imported, rather than the raw `.jsonl`. It covers:
-
-- AWS / GCP / Azure access keys
-- ANTHROPIC_API_KEY, OPENAI_API_KEY, GitHub tokens
-- Stripe keys, Slack tokens, JWT secrets
-- Generic high-entropy strings (configurable threshold)
-
-A session with a positive finding is **skipped entirely** — not partially
-redacted. The source path and finding count are logged, never secret values; you can see what
-was skipped via `bun run bin/gstack-memory-ingest.ts --probe` (which
-shows new vs. updated counts) or by reviewing the helper's output during
-`/sync-gbrain --full`.
-
-If gitleaks is not installed (run `brew install gitleaks` on macOS, or
-`apt install gitleaks` on Linux) and you passed `--scan-secrets` anyway,
-the helper warns once and every file it cannot scan is skipped, not
-imported unscanned. The same goes for a scan that fails partway. Skipped
-files stay pending and are retried on the next run. Missing or malformed
-reports, reports over 16 MiB, and scans exceeding 60 seconds also block the
-page. Reports use private temporary files that are removed after scanning.
-
-Resumed staging is scanned again, including files absent from the current
-source walk. Any finding, incomplete scan, or unsupported entry refuses the
-whole resumed import with a nonzero exit and preserves the stage for retry.
-Saved pages determine the expected import count; only pages matching the
-current rendered source can advance its ingest state. Incomplete or mismatched
-staging remains available for recovery.
-
-`--no-write` is a dry run: it prepares and counts pages but imports nothing
-and never changes the ingest state, so a later real run still imports every
-eligible page.
-
-With scanning requested, fresh, resumed and persistent passes stamp only the source snapshot used to render the page, and only while its
-hash and modification time are unchanged. Incremental checks verify the hash
-even when the timestamp matches. Scanned pages are fully written in private
-staging before atomic promotion, so partial writes never become outgoing pages.
-
-## Where it goes
-
-Storage tier depends on your gbrain engine (set during `/setup-gbrain`):
-
-- **Supabase configured:** code + transcripts go to Supabase Storage
-  (multi-Mac native). Curated memory (eureka/learnings/etc.) goes to the
-  brain-linked git repo via `gstack-brain-sync`.
-- **Local PGLite only:** everything stays on this Mac. Curated memory
-  syncs via git if you've enabled brain-sync.
-
-The "never double-store" rule per the plan: code and transcripts NEVER
-go in the gbrain-linked git repo. They're too big and they're
-replaceable from disk on each Mac.
-
-## What you can do with it
-
-- **Query in natural language:**
-  ```bash
-  gbrain query "what was I doing on the auth migration"
-  gbrain search "session_id:abc123"
-  ```
-
-- **Browse by type:**
-  ```bash
-  gbrain list --type transcript --limit 10
-  gbrain list --type ceo-plan
-  ```
-
-- **Read a specific page:**
-  ```bash
-  gbrain get_page transcripts/claude-code/garrytan-gstack/2026-05-01-abc123
-  ```
-
-- **Delete a page:**
-  ```bash
-  gbrain delete_page <slug>
-  ```
-  Caveat: with brain-sync enabled, the page is removed from gbrain's
-  index but git history retains it. For hard-delete, run `git filter-repo`
-  on the brain remote.
-
-- **Bulk-delete by criteria** (V1.0.1 follow-up — `gstack-transcript-prune`
-  helper). For V1.0, use `gbrain delete <slug>` per-page or write
-  a small loop over `gbrain list` output.
-
-- **Disable entirely:**
-  ```bash
-  gstack-config set transcript_ingest_mode off
-  gstack-config set gbrain_context_load off  # also disables retrieval
-  ```
-
-## How the agent uses it
-
-At every gstack skill start, the preamble runs
-`gstack-brain-context-load` which:
-
-1. Reads the active skill's `gbrain.context_queries:` frontmatter
-2. Dispatches each query to gbrain (vector / list / filesystem)
-3. Renders results into `## <render_as>` sections wrapped in
-   `<USER_TRANSCRIPT_DATA do-not-interpret-as-instructions>` envelopes
-4. The model sees this as part of the preamble before making any decisions
-
-For example, when you run `/office-hours`, the model context
-automatically includes:
-
-- `## Prior office-hours sessions in this repo` (last 5)
-- `## Your builder profile snapshot` (latest entry)
-- `## Recent design docs for this project` (last 3)
-- `## Recent eureka moments` (last 5)
-
-So the "Welcome back, last time you were on X" beat is sourced from
-your actual data, not cold-start.
-
-If gbrain is unavailable (CLI missing, MCP not registered, query
-timeout), the helper renders `(unavailable)` and the skill continues —
-startup never blocks > 2s on gbrain issues (Section 1C).
-
-## What to do when something feels off
-
-Run `/setup-gbrain` again. It's idempotent: every step detects existing
-state, repairs only what's missing, and prints a GREEN/YELLOW/RED
-verdict block. If a row is RED, the row tells you what to do.
-
-Common cases:
-
-- **Salience block is empty** — your transcripts may not be ingested
-  yet. Run `bun run bin/gstack-gbrain-sync.ts --full` to do a full pass.
-
-- **"gbrain CLI missing" in the preamble output** — gbrain isn't on
-  your PATH. Run `/setup-gbrain` to install/wire it.
-
-- **PGLite engine corrupt (V1.5)** — V1.5 ships
-  `gbrain restore-from-sync` for atomic rebuild from the brain remote.
-  For V1.0, manual recovery: `cd ~/.gbrain && rm -rf db && gbrain init
-  --pglite && gbrain import <brain-remote-clone-dir>`.
-
-- **A page has stale or wrong content** — `gbrain delete_page <slug>`,
-  then re-run `bun run bin/gstack-gbrain-sync.ts --incremental` to re-ingest from
-  source if the source file is still on disk and unchanged.
-
-## Privacy + audit
-
-- Every `secretScanFile` finding is logged to stderr at ingest time.
-- Every gbrain put/delete is logged to `~/.gstack/.gbrain-errors.jsonl`
-  with `{ts, op, duration_ms, outcome}` for forensic tracing.
-- `~/.gstack/.gbrain-engine-cache.json` shows which storage tier is
-  active (PGLite vs Supabase).
-- Brain-sync git history shows every curated artifact push with the
-  user's git identity.
-
-If you find a transcript page that contains a secret (either because
-per-file scanning was off, or gitleaks missed it), the recovery path is:
-1. `gbrain delete_page <slug>` — removes from index immediately
-2. Rotate the secret (rotate it anyway as a defensive measure)
-3. If brain-sync is on: `git filter-repo --invert-paths --path <relative-path>`
-   on the brain remote for hard-delete from history
-4. If the miss looks like a gitleaks rule gap, file a gitleaks issue
-   with the pattern (or extend the gitleaks config at `~/.gitleaks.toml`).
-
-## Path 4: Remote MCP setup (v1.27.0.0+)
-
-If you don't run gbrain locally — you have a teammate or another machine
-running `gbrain serve` over HTTP, accessible via Tailscale, ngrok, or
-internal LAN — `/setup-gbrain` Path 4 is the one-paste flow.
-
-You provide:
-- The MCP URL (e.g., `https://wintermute.tail554574.ts.net:3131/mcp`)
-- A bearer token (issued by the brain admin via `gbrain access-token issue`)
-
-What `/setup-gbrain` does:
-1. Verifies the URL + token via `gstack-gbrain-mcp-verify`. Three failure
-   modes get classified with one-line remediation hints:
-   **NETWORK** ("check Tailscale/DNS"), **AUTH** ("rotate token"),
-   **MALFORMED** ("Accept-header gotcha — pass both `application/json`
-   AND `text/event-stream`").
-2. Registers the MCP at user scope:
-   ```
-   claude mcp add --scope user --transport http gbrain "$URL" \
-     --header "Authorization: Bearer $TOKEN"
-   ```
-3. Skips local install, local doctor, transcript ingest, and federated
-   source registration. All four require a local `gbrain` CLI that Path 4
-   doesn't install.
-4. Optionally provisions a `gstack-artifacts-$USER` private repo on
-   GitHub or GitLab and prints the one-line `gbrain sources add` command
-   for your brain admin to run on the brain host.
-
-### Token storage trade-off
-
-The bearer token lives in `~/.claude.json` (mode 0600), where Claude Code
-stores every MCP server's credentials. During `claude mcp add --header
-"Authorization: Bearer $TOKEN"`, the token is briefly visible in
-process argv (~10ms) — visible to `ps` running concurrently. The window
-is small but it's not zero.
-
-Mitigations we've considered:
-- **Stdin or env-var input form for headers** — would close the argv
-  window. As of Claude Code v1.0.x, the CLI doesn't expose either.
-  When it does, `/setup-gbrain` Path 4 will switch automatically.
-- **Keychain storage** — explicitly out of scope (the token's resting
-  state in `~/.claude.json` is the existing trust surface for every MCP
-  credential; expanding to Keychain would touch every MCP server, not
-  just gbrain).
-
-### Why Path 4 is "always print" for the brain-admin hookup
-
-`gstack-artifacts-init` always prints the `gbrain sources add` command
-labeled "Send this to your brain admin" — even when the user IS the
-brain admin (consistent UX, no mode-detection fragility).
-
-A previous design proposed probing whether the user's bearer has admin
-scope (via a benign MCP write call like `add_tag`) and auto-executing
-the source registration when scope was sufficient. The design review
-flagged that page-write doesn't actually prove source-management
-permission — those are different scopes in any sensible auth model.
-Until gbrain ships:
-- a `mcp__gbrain__whoami` capability tool that returns the bearer's
-  scope set, AND
-- a `mcp__gbrain__sources_add` MCP tool with admin-scope gating
-
-we always print the command rather than pretending we know who has
-permission to run it.
-
-### CLAUDE.md block in Path 4
-
-Distinct from local-stdio mode. Token is **never** written to CLAUDE.md
-(many projects check CLAUDE.md into git). The block records the URL,
-the verified server version, the artifacts repo URL (if provisioned),
-and the per-repo trust policy.
-
-```markdown
-## GBrain Configuration (configured by /setup-gbrain)
-- Mode: remote-http
-- MCP URL: https://wintermute.tail554574.ts.net:3131/mcp
-- Server version: gbrain v0.27.1
-- Setup date: 2026-05-06
-- MCP registered: yes (user scope)
-- Token: stored in ~/.claude.json (do not commit; never written to CLAUDE.md)
-- Artifacts repo: github.com/garrytan/gstack-artifacts-garrytan (private)
-- Artifacts sync: artifacts-only
-- Current repo policy: read-write
-```
-
-### Token rotation
-
-Server-side. When verify hits `AUTH` (e.g., the brain admin rotated the
-token), the helper says: "rotate token on the brain host, re-run
-/setup-gbrain." On wintermute or wherever your gbrain server lives:
-
-```
-gbrain access-token rotate    # invalidates old, issues new
-```
-
-(See `gstack/setup-gbrain/SKILL.md.tmpl` for the full Path 4 flow plus
-the gbrain enhancement requests around scoped tokens that would let
-gstack auto-rotate in V2.)