brainclaw 1.7.5 → 1.9.0

package/docs/security.md

@@ -4,8 +4,10 @@ brainclaw is designed to be safe by default.
 
 ## Security model
 
-### No network access
-The CLI does not need to call external services to function.
+### No network access by default
+The CLI does not need to call external services to function. The optional
+supply-chain gate (see below) is the one feature that talks to a remote
+service, and it is off until explicitly enabled.
 
 ### No telemetry
 brainclaw does not collect or send usage data.
@@ -19,35 +21,192 @@ That makes review easier, but it also means users must be careful about what the
 
 ## Built-in safety behaviors
 
-brainclaw warns when content appears sensitive, for example when text includes patterns such as:
+### Content scanning (scanText)
 
-- `api_key`
-- `secret`
-- `token`
-- `password`
+Whenever content is added to `.brainclaw/` brainclaw scans the text in three
+layers:
 
-It can also warn about sensitive paths such as:
+1. **User-configured regex patterns** from `redaction.patterns`. The
+   defaults catch words like `api_key`, `secret`, `token`, `password`.
+2. **Structural detectors** for well-known token shapes — GitHub PATs
+   (`ghp_`, `gho_`, `github_pat_…`), AWS access keys (`AKIA…`/`ASIA…`),
+   Google API keys (`AIza…`), Slack tokens and webhooks, Stripe
+   `sk_live_/sk_test_` keys, JWTs, PEM-encoded private keys, and URLs
+   with embedded `user:password@host` credentials.
+3. **Entropy detector** that flags high-Shannon-entropy substrings near
+   a secret keyword (api_key/token/secret/password/auth/...). This
+   catches custom-shaped keys the structural list does not enumerate.
 
-- `.env`
-- `secrets/`
+In `security.mode: warn` matches surface as warnings; in
+`security.strict_redaction: true` (or `security.mode: strict`) the same
+matches block the write.
 
-Redaction behavior is configurable in `config.yaml`:
+### Sensitive paths
+
+When `security.block_sensitive_paths: true` (default), content that
+references paths like `.env`, `secrets/`, `.git/`, or `node_modules/`
+emits a warning.
+
+### Configuration
 
 ```yaml
 security:
-  mode: warn               # 'warn' or 'strict'
+  mode: warn               # 'warn' or 'strict' — affects redaction scanning
   strict_redaction: false  # if true, blocks entries with sensitive content
   block_sensitive_paths: true
+  token_detection:
+    enabled: true          # turn off all structural + entropy detection
+    entropy:
+      enabled: true
+      min_length: 32       # minimum substring length to consider
+      min_entropy: 4.0     # minimum Shannon entropy (bits/char)
+    detectors:             # per-detector override map; explicit false disables a detector
+      aws_access_key: true
+      jwt: true
+
+redaction:
+  enabled: true
+  patterns:
+    - '(?i)api[_-]?key'
+    - '(?i)secret'
+    - '(?i)token'
+    - '(?i)password'
+```
+
+## Supply-chain pre-install gate
+
+`brainclaw setup-security` enables a preinstall gate that intercepts
+`npm`/`pnpm`/`yarn`/`pip`/`pip3` install commands and asks Socket.dev's
+scoring service whether the requested packages are risky.
+
+```
+brainclaw setup-security --mode advisory    # default — warn-only
+brainclaw setup-security --mode enforced    # block risky installs
 ```
 
+### Advisory vs enforced
+
+The mode determines what happens when the gate decides a package is
+risky:
+
+- **advisory** — verdicts are surfaced (printed warning, security trap
+  created) but the install is *not* aborted.
+- **enforced** — a `block` verdict aborts the install with a non-zero
+  exit code.
+
+The mode is read from `security.preinstall.mode` and can be overridden
+per call with `brainclaw check-security --mode <advisory|enforced>`.
+
+### Exit codes
+
+`brainclaw check-security` emits exit codes that already encode the mode
+decision, so wrapper scripts are mode-agnostic:
+
+| Exit | Meaning                                                    |
+|------|------------------------------------------------------------|
+| 0    | pass — no risky packages                                   |
+| 1    | warn — advisory-mode block, or warn-threshold verdict      |
+| 2    | block — enforced-mode block; the wrapper aborts the install|
+
+The wrapper scripts (`<.brainclaw/security/bin>/npm`, `pnpm`, `yarn`,
+`pip`, `pip3` and their `.ps1` counterparts) call the CLI and react
+purely to the exit code, so flipping advisory↔enforced is a config
+change — no regeneration required.
+
+### Composite scoring
+
+Each package gets five sub-scores from Socket (supply-chain,
+vulnerability, quality, maintenance, license) on a 0–100 scale. The
+gate combines them into a single composite using configurable weights:
+
+```yaml
+security:
+  preinstall:
+    weights:
+      supply_chain: 0.35
+      vulnerability: 0.30
+      quality: 0.15
+      maintenance: 0.15
+      license: 0.05
+    thresholds:
+      composite_pass: 70        # composite >= 70 → pass
+      composite_warn: 50        # composite >= 50 but < 70 → warn; below → block
+      supply_chain_block: 30    # hard block when supply_chain < 30
+      vulnerability_block: 20   # hard block when vulnerability < 20
+```
+
+Weights are normalized to sum to 1.0 automatically — a config like
+`{ supply_chain: 1, vulnerability: 1 }` is rescaled rather than
+producing composites above 100. Thresholds are clamped to `[0,100]`
+and `composite_warn` is capped at `composite_pass` so the verdict
+function stays monotonic.
+
+### Allowlist / denylist
+
+```yaml
+security:
+  preinstall:
+    allowlist:
+      - npm:internal-pkg            # ecosystem-scoped
+      - npm:axios@1.14.0            # exact version pin
+      - lodash                       # bare name — any ecosystem, any version
+    denylist:
+      - npm:axios@1.14.1            # known-compromised version
+      - pypi:bad-pkg
+```
+
+Matching is exact on each component. Bare names match by package name
+only; an `ecosystem:` prefix scopes the match to that ecosystem;
+appending `@version` (or `==version` for pip-style) requires an exact
+version. The wildcard `@*` matches any version (useful for explicit
+"any version" entries).
+
+This is a hard tightening from the MVP: previously `denylist: ['lodash']`
+matched any package whose purl contained the substring `lodash`, so
+`react-lodash` was silently blocked. The new matcher requires exact
+package-name equality.
+
+### Package extraction sources
+
+`check-security` can pull the list of packages from three places, which
+may be combined:
+
+- `--packages axios,express@1.2.3` — comma-separated specs
+- `--requirements requirements.txt` — pip-style requirements file;
+  recursive `-r` includes, env markers, and extras are handled
+- `--lockfile package-lock.json` — npm package-lock (v1, v2, v3);
+  scans top-level direct deps and devDeps
+
+The wrappers auto-translate `npm install -r foo.txt` → `--requirements`,
+skip filesystem paths and URL specs that aren't registry packages
+(`./local-pkg`, `git+https://…`, `*.tgz`, `*.whl`), and handle the
+common npm/pip flag conventions.
+
+### Offline / fetch-error fallback
+
+When the Socket call fails (network down, service unreachable), the
+gate uses `security.preinstall.fallback_on_error`:
+
+- `block` — fail closed regardless of mode
+- `warn` — surface a warning, continue with whatever cached scores
+  exist; if no cache exists the gate exits with code 1
+- `pass`  — silent fall-through (the install proceeds)
+
+Cached scores have a TTL of `cache_ttl_hours` (default 24) and live
+in `.brainclaw/security/cache.json`.
+
 ## Recommended stance
 
-- do not store secrets
+- do not store secrets — even with detection, the safer move is to never
+  write them
 - review what gets committed
 - keep machine-local observations machine-local when appropriate
 - use stricter redaction settings in sensitive environments
+- run `setup-security --mode advisory` first; flip to `enforced` once
+  the team has gotten used to the verdicts
 
 ## Important nuance
 
-brainclaw reduces hidden behavior, but it does not remove the need for operational discipline.
-It warns; the team still decides what belongs in shared memory.
+brainclaw reduces hidden behavior, but it does not remove the need for
+operational discipline. It warns; the team still decides what belongs
+in shared memory.

package/docs/storage.md

@@ -7,7 +7,7 @@ brainclaw is local-first and workspace-centric.
 ```text
 .brainclaw/
   config.yaml                    ← Project configuration
-  project.md                     ← Derived readable view (best-effort, regenerable)
+  project.md                     ← Legacy derived readable view (best-effort, regenerable)
   events.jsonl                   ← Append-only event log (agent notifications)
   audit.log                      ← Append-only audit trail (before/after snapshots)
   memory/
@@ -42,7 +42,9 @@ Benefits:
 - isolated corruption (one bad file does not affect others)
 
 ### Derived views are best-effort
-`project.md` is a **derived view** regenerated from canonical state via `rebuildProjectMd()`. It is not a write target — it can always be rebuilt from the JSON files using `brainclaw rebuild`.
+`.brainclaw/project.md` is a **legacy derived view** regenerated from canonical state via `rebuildProjectMd()`. It is not a write target and no longer carries live claims, plans, handoffs, or runtime state — use `brainclaw agent-board`, `brainclaw context`, or MCP board/context tools for that operational data.
+
+Root `PROJECT.md` is different: it is the durable project vision and should contain stable product, architecture, and contribution guidance that does not expire every session.
 
 Failures to regenerate `project.md` are logged but never block mutations. If it gets stale, `brainclaw doctor` detects the drift.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "brainclaw",
-  "version": "1.7.5",
+  "version": "1.9.0",
   "description": "Shared project memory for humans and coding agents.",
   "type": "module",
   "bin": {
@@ -26,23 +26,25 @@
     "clean:build": "node scripts/clean-build.mjs dist dist-test",
     "clean:dist": "node scripts/clean-build.mjs dist",
     "clean:dist-test": "node scripts/clean-build.mjs dist-test",
-    "build": "npm run clean:dist && tsc && node scripts/copy-default-profiles.mjs && node scripts/build-vsix.mjs",
-    "build:cli": "npm run clean:dist && tsc && node scripts/copy-default-profiles.mjs",
-    "emit:facts": "npm run build:cli && node scripts/emit-site-facts.mjs",
+    "build": "npm run build:cli && node scripts/build-vsix.mjs --optional",
+    "build:cli": "npm run clean:dist && tsc && node scripts/copy-default-profiles.mjs",
+    "build:release": "npm run build:cli && node scripts/emit-site-facts.mjs && node scripts/build-vsix.mjs",
+    "emit:facts": "npm run build:cli && node scripts/emit-site-facts.mjs",
     "dev": "npm run clean:dist && tsc && node scripts/copy-default-profiles.mjs && node dist/cli.js",
     "dev:mcp": "tsc && node scripts/copy-default-profiles.mjs",
     "build:mcp-schemas": "npm run clean:dist && tsc && node scripts/build-mcp-schemas.mjs",
     "dev:link": "npm link && echo 'Symlink created: global brainclaw → local repo. MCP agents now use dist/ directly.'",
-    "build:test": "npm run build && npm run clean:dist-test && tsc -p tsconfig.test.json",
+    "build:test": "npm run build:cli && npm run clean:dist-test && tsc -p tsconfig.test.json",
     "pack:check": "node scripts/check-pack.mjs",
     "test": "npm run build:test && node scripts/run-tests.mjs default",
     "test:unit": "npm run build:test && node scripts/run-tests.mjs unit",
     "test:smoke": "npm run build:test && node scripts/run-tests.mjs smoke",
     "test:e2e": "npm run build:test && node scripts/run-tests.mjs e2e",
     "test:all": "npm run build:test && node scripts/run-tests.mjs all",
+    "test:cross": "npm run build:test && node scripts/test-cross.mjs default",
     "test:coverage": "npm run build:test && c8 --all --src dist-test/src --reporter=text-summary --reporter=lcov --exclude=dist-test/tests/** --exclude=dist-test/scripts/** node scripts/run-tests.mjs default",
     "test:coverage:check": "npm run build:test && c8 --all --check-coverage --lines 55 --functions 60 --branches 65 --statements 55 --src dist-test/src --reporter=text-summary --exclude=dist-test/tests/** --exclude=dist-test/scripts/** node scripts/run-tests.mjs default",
-    "prepublishOnly": "npm run build && node scripts/emit-site-facts.mjs && npm run pack:check"
+    "prepublishOnly": "npm run build:release && npm run pack:check"
   },
   "keywords": [
     "cli",