brainclaw 1.8.0 → 1.9.1

package/docs/review.md

@@ -1,45 +1,45 @@
-# Review and Reflection
-
-brainclaw includes a reflective layer so not every observation has to become canonical memory immediately.
-
-## Why this exists
-
-Some facts are:
-
-- durable and worth keeping
-- useful but still uncertain
-- too local or too temporary to promote immediately
-
-A review layer helps filter and promote the right items.
-
-## Typical flow
-
-```
-runtime-note → reflect-runtime-note → review candidate → accept / reject
-```
-
-1. capture a runtime note (`brainclaw runtime-note`)
-2. reflect it into a candidate when useful (`brainclaw reflect-runtime-note`)
-3. review the candidate (`brainclaw review`)
-4. accept or reject it (`brainclaw accept` / `brainclaw reject`)
-
-For MCP-capable agents, `brainclaw runtime-note` maps to `bclaw_write_note`.
-
-## Star and use signals
-
-Before formal review, candidates can accumulate signals:
-
-```bash
-brainclaw star-candidate cnd_001 --by copilot    # passive interest
-brainclaw use-candidate cnd_001 --by copilot --context "auth rollout"  # active reuse
-```
-
-`review --prioritized` uses these signals (along with freshness and semantic relevance) to surface the most important candidates.
-
-## curator role
-
-The `curator` trust level has direct promote access. Other roles create candidates for review.
-
-## Why this is valuable
-
-This keeps canonical memory cleaner while still preserving useful observations long enough to evaluate them.
+# Review and Reflection
+
+brainclaw includes a reflective layer so not every observation has to become canonical memory immediately.
+
+## Why this exists
+
+Some facts are:
+
+- durable and worth keeping
+- useful but still uncertain
+- too local or too temporary to promote immediately
+
+A review layer helps filter and promote the right items.
+
+## Typical flow
+
+```
+runtime-note → reflect-runtime-note → review candidate → accept / reject
+```
+
+1. capture a runtime note (`brainclaw runtime-note`)
+2. reflect it into a candidate when useful (`brainclaw reflect-runtime-note`)
+3. review the candidate (`brainclaw review`)
+4. accept or reject it (`brainclaw accept` / `brainclaw reject`)
+
+For MCP-capable agents, `brainclaw runtime-note` maps to `bclaw_write_note`.
+
+## Star and use signals
+
+Before formal review, candidates can accumulate signals:
+
+```bash
+brainclaw star-candidate cnd_001 --by copilot    # passive interest
+brainclaw use-candidate cnd_001 --by copilot --context "auth rollout"  # active reuse
+```
+
+`review --prioritized` uses these signals (along with freshness and semantic relevance) to surface the most important candidates.
+
+## curator role
+
+The `curator` trust level has direct promote access. Other roles create candidates for review.
+
+## Why this is valuable
+
+This keeps canonical memory cleaner while still preserving useful observations long enough to evaluate them.

package/docs/security.md

@@ -1,53 +1,212 @@
-# Security
-
-brainclaw is designed to be safe by default.
-
-## Security model
-
-### No network access
-The CLI does not need to call external services to function.
-
-### No telemetry
-brainclaw does not collect or send usage data.
-
-### No secret management
-brainclaw is not a vault and should not be treated like one.
-
-### Plain-text visibility
-The storage model is intentionally inspectable.
-That makes review easier, but it also means users must be careful about what they write and commit.
-
-## Built-in safety behaviors
-
-brainclaw warns when content appears sensitive, for example when text includes patterns such as:
-
-- `api_key`
-- `secret`
-- `token`
-- `password`
-
-It can also warn about sensitive paths such as:
-
-- `.env`
-- `secrets/`
-
-Redaction behavior is configurable in `config.yaml`:
-
-```yaml
-security:
-  mode: warn               # 'warn' or 'strict'
-  strict_redaction: false  # if true, blocks entries with sensitive content
-  block_sensitive_paths: true
-```
-
-## Recommended stance
-
-- do not store secrets
-- review what gets committed
-- keep machine-local observations machine-local when appropriate
-- use stricter redaction settings in sensitive environments
-
-## Important nuance
-
-brainclaw reduces hidden behavior, but it does not remove the need for operational discipline.
-It warns; the team still decides what belongs in shared memory.
+# Security
+
+brainclaw is designed to be safe by default.
+
+## Security model
+
+### No network access by default
+The CLI does not need to call external services to function. The optional
+supply-chain gate (see below) is the one feature that talks to a remote
+service, and it is off until explicitly enabled.
+
+### No telemetry
+brainclaw does not collect or send usage data.
+
+### No secret management
+brainclaw is not a vault and should not be treated like one.
+
+### Plain-text visibility
+The storage model is intentionally inspectable.
+That makes review easier, but it also means users must be careful about what they write and commit.
+
+## Built-in safety behaviors
+
+### Content scanning (scanText)
+
+Whenever content is added to `.brainclaw/` brainclaw scans the text in three
+layers:
+
+1. **User-configured regex patterns** from `redaction.patterns`. The
+   defaults catch words like `api_key`, `secret`, `token`, `password`.
+2. **Structural detectors** for well-known token shapes — GitHub PATs
+   (`ghp_`, `gho_`, `github_pat_…`), AWS access keys (`AKIA…`/`ASIA…`),
+   Google API keys (`AIza…`), Slack tokens and webhooks, Stripe
+   `sk_live_/sk_test_` keys, JWTs, PEM-encoded private keys, and URLs
+   with embedded `user:password@host` credentials.
+3. **Entropy detector** that flags high-Shannon-entropy substrings near
+   a secret keyword (api_key/token/secret/password/auth/...). This
+   catches custom-shaped keys the structural list does not enumerate.
+
+In `security.mode: warn` matches surface as warnings; in
+`security.strict_redaction: true` (or `security.mode: strict`) the same
+matches block the write.
+
+### Sensitive paths
+
+When `security.block_sensitive_paths: true` (default), content that
+references paths like `.env`, `secrets/`, `.git/`, or `node_modules/`
+emits a warning.
+
+### Configuration
+
+```yaml
+security:
+  mode: warn               # 'warn' or 'strict' — affects redaction scanning
+  strict_redaction: false  # if true, blocks entries with sensitive content
+  block_sensitive_paths: true
+  token_detection:
+    enabled: true          # turn off all structural + entropy detection
+    entropy:
+      enabled: true
+      min_length: 32       # minimum substring length to consider
+      min_entropy: 4.0     # minimum Shannon entropy (bits/char)
+    detectors:             # per-detector override map; explicit false disables a detector
+      aws_access_key: true
+      jwt: true
+
+redaction:
+  enabled: true
+  patterns:
+    - '(?i)api[_-]?key'
+    - '(?i)secret'
+    - '(?i)token'
+    - '(?i)password'
+```
+
+## Supply-chain pre-install gate
+
+`brainclaw setup-security` enables a preinstall gate that intercepts
+`npm`/`pnpm`/`yarn`/`pip`/`pip3` install commands and asks Socket.dev's
+scoring service whether the requested packages are risky.
+
+```
+brainclaw setup-security --mode advisory    # default — warn-only
+brainclaw setup-security --mode enforced    # block risky installs
+```
+
+### Advisory vs enforced
+
+The mode determines what happens when the gate decides a package is
+risky:
+
+- **advisory** — verdicts are surfaced (printed warning, security trap
+  created) but the install is *not* aborted.
+- **enforced** — a `block` verdict aborts the install with a non-zero
+  exit code.
+
+The mode is read from `security.preinstall.mode` and can be overridden
+per call with `brainclaw check-security --mode <advisory|enforced>`.
+
+### Exit codes
+
+`brainclaw check-security` emits exit codes that already encode the mode
+decision, so wrapper scripts are mode-agnostic:
+
+| Exit | Meaning                                                    |
+|------|------------------------------------------------------------|
+| 0    | pass — no risky packages                                   |
+| 1    | warn — advisory-mode block, or warn-threshold verdict      |
+| 2    | block — enforced-mode block; the wrapper aborts the install|
+
+The wrapper scripts (`<.brainclaw/security/bin>/npm`, `pnpm`, `yarn`,
+`pip`, `pip3` and their `.ps1` counterparts) call the CLI and react
+purely to the exit code, so flipping advisory↔enforced is a config
+change — no regeneration required.
+
+### Composite scoring
+
+Each package gets five sub-scores from Socket (supply-chain,
+vulnerability, quality, maintenance, license) on a 0–100 scale. The
+gate combines them into a single composite using configurable weights:
+
+```yaml
+security:
+  preinstall:
+    weights:
+      supply_chain: 0.35
+      vulnerability: 0.30
+      quality: 0.15
+      maintenance: 0.15
+      license: 0.05
+    thresholds:
+      composite_pass: 70        # composite >= 70 → pass
+      composite_warn: 50        # composite >= 50 but < 70 → warn; below → block
+      supply_chain_block: 30    # hard block when supply_chain < 30
+      vulnerability_block: 20   # hard block when vulnerability < 20
+```
+
+Weights are normalized to sum to 1.0 automatically — a config like
+`{ supply_chain: 1, vulnerability: 1 }` is rescaled rather than
+producing composites above 100. Thresholds are clamped to `[0,100]`
+and `composite_warn` is capped at `composite_pass` so the verdict
+function stays monotonic.
+
+### Allowlist / denylist
+
+```yaml
+security:
+  preinstall:
+    allowlist:
+      - npm:internal-pkg            # ecosystem-scoped
+      - npm:axios@1.14.0            # exact version pin
+      - lodash                       # bare name — any ecosystem, any version
+    denylist:
+      - npm:axios@1.14.1            # known-compromised version
+      - pypi:bad-pkg
+```
+
+Matching is exact on each component. Bare names match by package name
+only; an `ecosystem:` prefix scopes the match to that ecosystem;
+appending `@version` (or `==version` for pip-style) requires an exact
+version. The wildcard `@*` matches any version (useful for explicit
+"any version" entries).
+
+This is a hard tightening from the MVP: previously `denylist: ['lodash']`
+matched any package whose purl contained the substring `lodash`, so
+`react-lodash` was silently blocked. The new matcher requires exact
+package-name equality.
+
+### Package extraction sources
+
+`check-security` can pull the list of packages from three places, which
+may be combined:
+
+- `--packages axios,express@1.2.3` — comma-separated specs
+- `--requirements requirements.txt` — pip-style requirements file;
+  recursive `-r` includes, env markers, and extras are handled
+- `--lockfile package-lock.json` — npm package-lock (v1, v2, v3);
+  scans top-level direct deps and devDeps
+
+The wrappers auto-translate `npm install -r foo.txt` → `--requirements`,
+skip filesystem paths and URL specs that aren't registry packages
+(`./local-pkg`, `git+https://…`, `*.tgz`, `*.whl`), and handle the
+common npm/pip flag conventions.
+
+### Offline / fetch-error fallback
+
+When the Socket call fails (network down, service unreachable), the
+gate uses `security.preinstall.fallback_on_error`:
+
+- `block` — fail closed regardless of mode
+- `warn` — surface a warning, continue with whatever cached scores
+  exist; if no cache exists the gate exits with code 1
+- `pass`  — silent fall-through (the install proceeds)
+
+Cached scores have a TTL of `cache_ttl_hours` (default 24) and live
+in `.brainclaw/security/cache.json`.
+
+## Recommended stance
+
+- do not store secrets — even with detection, the safer move is to never
+  write them
+- review what gets committed
+- keep machine-local observations machine-local when appropriate
+- use stricter redaction settings in sensitive environments
+- run `setup-security --mode advisory` first; flip to `enforced` once
+  the team has gotten used to the verdicts
+
+## Important nuance
+
+brainclaw reduces hidden behavior, but it does not remove the need for
+operational discipline. It warns; the team still decides what belongs
+in shared memory.

package/docs/server-operations.md

@@ -1,118 +1,118 @@
-# Server And Remote Operations
-
-This guide is for complex operator environments such as DGX hosts, remote Linux servers, SSH-driven workflows, and multi-project workspaces where Brainclaw is not being used from a single local IDE.
-
-## When this guide matters
-
-Use this path when one or more of these are true:
-
-- Brainclaw runs on a remote machine over SSH
-- multiple repos or child projects live under one server workspace
-- different agents resume each other over time on the same host
-- the operator sometimes uses CLI directly instead of MCP
-
-## Recommended baseline
-
-Prefer a disciplined single-environment workflow on the server:
-
-1. choose one canonical checkout per active task
-2. initialize Brainclaw in the target project before starting coordination work
-3. avoid mixing multiple shells, PATH setups, or Node installations during the same session
-4. treat shared-checkout parallel editing as risky unless each session gets a dedicated worktree
-
-## Multi-project targeting
-
-If the server workspace contains several Brainclaw projects, resolve the target project first.
-
-```bash
-brainclaw switch --list
-brainclaw switch apps/trainer
-brainclaw status
-```
-
-You can also use:
-
-- `BRAINCLAW_PROJECT` for a shell-scoped default
-- `brainclaw --cwd <path> ...` for a one-off override
-
-The goal is to avoid writing memory into the wrong store when the operator is launched from a workspace root.
-
-## Recommended session flow
-
-For MCP-capable agents:
-
-1. `bclaw_work(intent="execute", scope=...)` — opens a session, loads context, and claims the scope (with a per-claim git worktree) in a single call. Returns a compact payload by default; pass `compact: false` for the full memory dump or call `bclaw_context(kind="memory")` after.
-2. `bclaw_context(kind="execution")` — when local tooling / brainclaw version visibility is needed (already loaded implicitly by `bclaw_work`).
-3. `bclaw_write_note` / `bclaw_create(entity=...)` / `bclaw_update(entity=...)` — record observations, plans, decisions, or traps via the canonical grammar.
-4. `bclaw_session_end(narrative=..., auto_release: true)` — close cleanly; releases active claims and hands off the session record.
-
-For direct CLI operation:
-
-```bash
-brainclaw session-start --agent codex
-brainclaw plan list --all
-brainclaw claim create "Investigate training config drift" --scope configs/training/
-brainclaw runtime-note "Observed host-specific CUDA mismatch on dgx-a"
-brainclaw session-end --auto-release
-```
-
-`brainclaw runtime-note` is the CLI equivalent of `bclaw_write_note`.
-`brainclaw note create "..."` is accepted as a resource-style alias if that fits the operator workflow better.
-
-## Isolation and worktrees
-
-The safest default remains sequential collaboration: one active agent per checkout, then handoff.
-
-If you need stronger isolation:
-
-```bash
-brainclaw worktree create feat/dgx-audit
-brainclaw worktree list
-```
-
-In MCP flows, `bclaw_claim` can also create a dedicated worktree automatically when supported by the current Brainclaw version and configuration.
-
-Use dedicated worktrees when:
-
-- two sessions need to touch the same repo without sharing a checkout
-- an operator wants to preserve a clean main checkout while an agent runs elsewhere
-- a remote machine hosts long-lived or resumable agent sessions
-
-## Plan and step semantics
-
-Be explicit about the difference between plan items and plan steps:
-
-- `brainclaw plan show <id>` reads a single plan item
-- `brainclaw plan get <id>` is accepted as a read alias, but `brainclaw plan show <id>` remains the canonical form
-- `brainclaw plan update <id>` expects a `pln_*` plan ID
-- `brainclaw complete-step <planId> <stepId>` is the canonical way to complete a step
-
-If you only have a step ID, inspect the parent plan first with `plan show` or `plan list --json`.
-
-## Notes, decisions, and traps
-
-Do not use `decision` as a workaround for a session observation.
-
-Use:
-
-- `brainclaw runtime-note` for in-session observations
-- `brainclaw reflect-runtime-note` when the note may deserve promotion
-- `brainclaw decision` only for a durable decision
-- `brainclaw trap` only for a reusable pitfall
-
-This keeps canonical memory cleaner on long-lived server projects.
-
-## Common failure modes
-
-- Wrong project selected: use `brainclaw switch --list` and verify `brainclaw status`
-- Shared-checkout collisions: prefer a worktree per session when parallel activity is unavoidable
-- Confusion between plan IDs and step IDs: use `plan show` before `plan update`
-- Lost operator observations: record them with `runtime-note` instead of temporary shell notes
-
-## Recommended reading
-
-- [quickstart.md](quickstart.md)
-- [cli.md](cli.md)
-- [integrations/mcp.md](integrations/mcp.md)
-- [concepts/plans-and-claims.md](concepts/plans-and-claims.md)
-- [concepts/runtime-notes.md](concepts/runtime-notes.md)
+# Server And Remote Operations
+
+This guide is for complex operator environments such as DGX hosts, remote Linux servers, SSH-driven workflows, and multi-project workspaces where Brainclaw is not being used from a single local IDE.
+
+## When this guide matters
+
+Use this path when one or more of these are true:
+
+- Brainclaw runs on a remote machine over SSH
+- multiple repos or child projects live under one server workspace
+- different agents resume each other over time on the same host
+- the operator sometimes uses CLI directly instead of MCP
+
+## Recommended baseline
+
+Prefer a disciplined single-environment workflow on the server:
+
+1. choose one canonical checkout per active task
+2. initialize Brainclaw in the target project before starting coordination work
+3. avoid mixing multiple shells, PATH setups, or Node installations during the same session
+4. treat shared-checkout parallel editing as risky unless each session gets a dedicated worktree
+
+## Multi-project targeting
+
+If the server workspace contains several Brainclaw projects, resolve the target project first.
+
+```bash
+brainclaw switch --list
+brainclaw switch apps/trainer
+brainclaw status
+```
+
+You can also use:
+
+- `BRAINCLAW_PROJECT` for a shell-scoped default
+- `brainclaw --cwd <path> ...` for a one-off override
+
+The goal is to avoid writing memory into the wrong store when the operator is launched from a workspace root.
+
+## Recommended session flow
+
+For MCP-capable agents:
+
+1. `bclaw_work(intent="execute", scope=...)` — opens a session, loads context, and claims the scope (with a per-claim git worktree) in a single call. Returns a compact payload by default; pass `compact: false` for the full memory dump or call `bclaw_context(kind="memory")` after.
+2. `bclaw_context(kind="execution")` — when local tooling / brainclaw version visibility is needed (already loaded implicitly by `bclaw_work`).
+3. `bclaw_write_note` / `bclaw_create(entity=...)` / `bclaw_update(entity=...)` — record observations, plans, decisions, or traps via the canonical grammar.
+4. `bclaw_session_end(narrative=..., auto_release: true)` — close cleanly; releases active claims and hands off the session record.
+
+For direct CLI operation:
+
+```bash
+brainclaw session-start --agent codex
+brainclaw plan list --all
+brainclaw claim create "Investigate training config drift" --scope configs/training/
+brainclaw runtime-note "Observed host-specific CUDA mismatch on dgx-a"
+brainclaw session-end --auto-release
+```
+
+`brainclaw runtime-note` is the CLI equivalent of `bclaw_write_note`.
+`brainclaw note create "..."` is accepted as a resource-style alias if that fits the operator workflow better.
+
+## Isolation and worktrees
+
+The safest default remains sequential collaboration: one active agent per checkout, then handoff.
+
+If you need stronger isolation:
+
+```bash
+brainclaw worktree create feat/dgx-audit
+brainclaw worktree list
+```
+
+In MCP flows, `bclaw_claim` can also create a dedicated worktree automatically when supported by the current Brainclaw version and configuration.
+
+Use dedicated worktrees when:
+
+- two sessions need to touch the same repo without sharing a checkout
+- an operator wants to preserve a clean main checkout while an agent runs elsewhere
+- a remote machine hosts long-lived or resumable agent sessions
+
+## Plan and step semantics
+
+Be explicit about the difference between plan items and plan steps:
+
+- `brainclaw plan show <id>` reads a single plan item
+- `brainclaw plan get <id>` is accepted as a read alias, but `brainclaw plan show <id>` remains the canonical form
+- `brainclaw plan update <id>` expects a `pln_*` plan ID
+- `brainclaw complete-step <planId> <stepId>` is the canonical way to complete a step
+
+If you only have a step ID, inspect the parent plan first with `plan show` or `plan list --json`.
+
+## Notes, decisions, and traps
+
+Do not use `decision` as a workaround for a session observation.
+
+Use:
+
+- `brainclaw runtime-note` for in-session observations
+- `brainclaw reflect-runtime-note` when the note may deserve promotion
+- `brainclaw decision` only for a durable decision
+- `brainclaw trap` only for a reusable pitfall
+
+This keeps canonical memory cleaner on long-lived server projects.
+
+## Common failure modes
+
+- Wrong project selected: use `brainclaw switch --list` and verify `brainclaw status`
+- Shared-checkout collisions: prefer a worktree per session when parallel activity is unavoidable
+- Confusion between plan IDs and step IDs: use `plan show` before `plan update`
+- Lost operator observations: record them with `runtime-note` instead of temporary shell notes
+
+## Recommended reading
+
+- [quickstart.md](quickstart.md)
+- [cli.md](cli.md)
+- [integrations/mcp.md](integrations/mcp.md)
+- [concepts/plans-and-claims.md](concepts/plans-and-claims.md)
+- [concepts/runtime-notes.md](concepts/runtime-notes.md)