loki-mode 9.56.0 → 9.59.0

package/docs/dev/release-checklist.md

@@ -0,0 +1,157 @@
+# Release checklist and local CI
+
+Moved out of CLAUDE.md (S-78) to keep the root file short. Updated in this
+pass to remove two items that `docs/v10/DECISIONS.md` D25 and
+`docs/LOKI-10-BUILD-PROMPT.md` section 1 have superseded:
+- `git add -A` in step 4 is replaced with staging by name (repo rule, always).
+- Per-merge pushing is replaced by the release-train model (D25): batch
+  approved merges locally, push once per train, freeze main until Tests, Bun
+  Parity and Security Audit are all green on that exact SHA, then release.
+  "Wait for user approval before commit" from the old workflow is superseded
+  by the standing authorization in `docs/LOKI-10-BUILD-PROMPT.md` section 1
+  for sessions operating under that program; outside it, ask as normal.
+
+## Local CI Before Every Push
+
+**The FAST tier is the release gate. The FULL tier is not a blocker.**
+
+Founder decision 2026-07-31: GitHub CI runs Tests in ~31s and Release in ~2min
+via 4-way sharding. The local FULL tier takes ~27min with no sharding, too
+slow for an hourly-or-faster release cadence.
+
+- **Before push/release: `bash scripts/local-ci.sh`** (fast tier, default).
+- **Do NOT block a release on `LOCAL_CI_TIER=full`.** Ship, let GitHub Actions
+  verify, and fix what it finds in the next release train.
+- Run the FULL tier when diagnosing something specific, or on a quiet cycle,
+  not as a release precondition.
+
+**Why the fast tier and not nothing.** Of seven real defects found on
+2026-07-31, four were caught by the local gate ALONE. The sharpest is dist
+freshness: CI never validates that the committed `loki-ts/dist/loki.js`
+matches src; when that slipped, three releases shipped the wrong version.
+
+**The packaged artifact is the blind spot (2026-08-01).** Everything works
+from a git checkout, so no in-repo test and no GitHub CI job can see:
+- quality-gate detector test files missing from npm's `files[]` (v8.38.0)
+- committed `dist/loki.js` hardcoded to a stale version for 27 releases (v8.40.0)
+- `npm pack` content checks that tolerate losing required artifacts (v9.11.0)
+
+Rules that generalize past packaging:
+1. A check that guards the shipped artifact must run in the FAST tier.
+2. Assert each required thing individually, never a count.
+3. Guard against vacuity (an empty capture that makes every assertion pass;
+   `npm pack`'s listing is on STDERR).
+
+`LOCAL_CI_SHARDS` (default 4) controls local sharding; `LOCAL_CI_SERIAL=1`
+forces serial for diagnosis.
+
+After a release ships, run post-release distribution validation:
+- npm: `npm pack loki-mode@<VERSION>`, untar, run `bash package/bin/loki version`
+- Docker: `docker pull asklokesh/loki-mode:<VERSION>`, `docker run --rm <img> version`,
+  `doctor --json`, `status --json`
+- Brew: WebFetch the live formula, verify version + sha256
+- Both routes (Bun + LOKI_LEGACY_BASH=1) on each channel
+
+Cleanup after every local-ci run and post-release validation uses
+`loki_run_tmp_cleanup` (see `docs/dev/tmp-cleanup.md`).
+
+## Release Workflow
+
+**Step 0 (always first): `bash scripts/local-ci.sh`** -- pre-push gate.
+
+### 1. Version Bump - ALL Files
+
+```
+VERSION                                  # Single line: X.Y.Z
+package.json                             # "version": "X.Y.Z"
+SKILL.md                                 # Header (line ~6) AND footer (last line)
+Dockerfile                               # LABEL version= AND org.opencontainers.image.version (both, or it drifts)
+Dockerfile.sandbox                       # Same two labels
+plugins/loki-mode/.claude-plugin/plugin.json  # "version": "X.Y.Z" (marketplace.json carries no version)
+server.json                              # "version" AND packages[loki-mode].version (MCP registry manifest; enforced by tests/test-server-json-current.sh)
+vscode-extension/package.json            # DEPRECATED v7.2.0, no longer published; bump only if vendoring
+CLAUDE.md                                # Version pointer, if present
+dashboard/__init__.py                    # __version__ = "X.Y.Z"
+mcp/__init__.py                          # __version__ = "X.Y.Z"
+CHANGELOG.md                             # Add new version entry at top
+docs/INSTALLATION.md                     # Version header
+wiki/Home.md, wiki/_Sidebar.md, wiki/API-Reference.md
+README.md, docker-compose.yml            # Docker image tags (MAJOR/MINOR bumps)
+```
+
+### 2. Build Dashboard Frontend
+
+```bash
+cd dashboard-ui && npm ci && npm run build:all && cd ..
+ls -la dashboard/static/index.html   # verify >100KB
+```
+`npm publish`'s `prepublishOnly` also triggers this build automatically.
+
+### 3. Run Tests
+
+```bash
+bash -n autonomy/run.sh
+bash -n autonomy/loki
+python3 -c "import ast, os; [ast.parse(open(f'dashboard/{f}').read()) for f in os.listdir('dashboard') if f.endswith('.py')]"
+python3 -c "import json; json.load(open('package.json')); print('JSON OK')"
+cd dashboard-ui && npx playwright test && cd ..   # requires dashboard on 57374
+```
+
+### 3a. Pre-Publish Validation (MANDATORY)
+
+```bash
+npm pack --dry-run 2>&1 | grep -E "web-app/dist|dashboard/static" || echo "FAIL: expected files missing"
+git ls-files web-app/dist/index.html | grep -q . || echo "FAIL: web-app/dist/ not tracked"
+git ls-files dashboard/static/index.html | grep -q . || echo "FAIL: dashboard/static/ not tracked"
+npm pack && npm install -g ./loki-mode-*.tgz
+loki --version
+loki web --no-open &
+sleep 3
+curl -s http://127.0.0.1:57374/ | grep -q "Loki" && echo "PASS: web app serves" || echo "FAIL"
+curl -s http://127.0.0.1:57374/api/status | python3 -c "import json,sys; json.load(sys.stdin)" 2>/dev/null && echo "PASS: API responds" || echo "FAIL"
+loki web stop
+npm install -g loki-mode
+rm -f loki-mode-*.tgz
+```
+If any check fails, do not release; fix the root cause first.
+
+### 4. Commit and land (release-train model, D25)
+
+Stage files by name (never `git add -A` or `git add .`), commit with the
+repo-local `asklokesh` identity, no co-author. Batch approved merges locally;
+push once per train (`git push origin main`); wait for Tests, Bun Parity and
+Security Audit to go green on that exact SHA before bumping VERSION and
+releasing. GitHub Actions then creates the tag, the GitHub Release, publishes
+npm, builds/pushes Docker, and updates the Homebrew tap. Do not manually
+create tags.
+
+### 5. Verify ALL Distribution Channels
+
+```bash
+gh run list --limit 1 && gh run watch <run-id>
+npm view loki-mode version
+npm pack loki-mode --dry-run 2>&1 | grep dashboard/static
+docker pull asklokesh/loki-mode:X.Y.Z && docker run --rm asklokesh/loki-mode:X.Y.Z loki version
+brew update && brew info loki-mode
+gh release view vX.Y.Z
+```
+
+### Distribution Channel Checklist
+
+| Channel | Dashboard API | Dashboard Frontend | Memory System | Skills/References |
+|---------|--------------------------|------------------------------|---------------|-------------------|
+| npm     | `dashboard/*.py`         | `dashboard/static/index.html`| `memory/`     | `skills/`, `references/` |
+| Docker  | `COPY dashboard/`        | Built in Dockerfile or committed | `memory/` | `skills/`, `references/` |
+| Homebrew| Full tarball             | Full tarball                 | Full tarball  | Full tarball |
+| VSCode  | DEPRECATED v7.2.0 -- no longer published | -- | -- | -- |
+| Release | Skill-only zip           | N/A                          | N/A           | `references/` |
+
+### Credentials (GitHub Secrets)
+`NPM_TOKEN`, `DOCKERHUB_USERNAME` / `DOCKERHUB_TOKEN`, `HOMEBREW_TAP_TOKEN`.
+
+## Testing
+
+```bash
+./benchmarks/run-benchmarks.sh humaneval --execute --loki
+./benchmarks/run-benchmarks.sh swebench --execute --loki
+```

package/docs/dev/research-foundation.md

@@ -0,0 +1,29 @@
+# Research foundation
+
+Moved out of CLAUDE.md (S-78) to keep the root file short. Content unchanged.
+
+Built on 2025 research from three major AI labs:
+
+**OpenAI:**
+- Agents SDK (guardrails, tripwires, handoffs, tracing)
+- AGENTS.md / Agentic AI Foundation (AAIF) standards
+
+**Google DeepMind:**
+- SIMA 2 (self-improvement, hierarchical reasoning)
+- Google DeepMind Robotics (VLA models, planning)
+- Dreamer 4 (world model training)
+- Scalable Oversight via Debate
+
+**Anthropic:**
+- Constitutional AI (principles-based self-critique)
+- Alignment Faking Detection (sleeper agent probes)
+- Claude Code Best Practices (Explore-Plan-Code)
+
+**Academic:**
+- CONSENSAGENT (anti-sycophancy)
+- GoalAct (hierarchical planning)
+- A-Mem/MIRIX (memory systems)
+- Multi-Agent Reflexion (MAR)
+- NVIDIA ToolOrchestra (efficiency metrics)
+
+See `references/openai-patterns.md`, `references/lab-research-patterns.md`, and `references/advanced-patterns.md`.

package/docs/dev/tmp-cleanup.md

@@ -0,0 +1,157 @@
+# Temp directory and process cleanup (mandatory helpers)
+
+Moved out of CLAUDE.md (S-78) to keep the root file short. Content unchanged.
+CLAUDE.md still states the invariant; this file holds the runnable helpers.
+
+## Test and Resource Cleanup (MANDATORY - NEVER SKIP)
+
+**Before reporting ANY task as done, run ALL cleanup steps below. No exceptions.**
+
+1. **Create one run-owned temp directory before the first temp write.** Keep
+   every archive, package extraction, log, PID file, and test fixture for the
+   task below this directory. Never use fixed names or shared globs under
+   `/tmp`.
+
+2. **Stop only processes started by the current task.** Retain their exact PIDs
+   below `$LOKI_RUN_TMP` and signal those PIDs individually. Do not use
+   `pkill`, a shared-port sweep, or a name-pattern kill as cleanup.
+
+3. **Remove only the validated run-owned directory.** Use the helpers below.
+   Cleanup fails closed unless the target is a direct child of the canonical
+   temp root, has the exact ownership marker created with it, is owned by the
+   current UID, still carries the private permissions the helper set (`700`
+   for the directory, `600` for the marker), is not a symlink, and is not a
+   Git worktree. Never replace this with a wildcard deletion under `/tmp` or
+   `$TMPDIR`.
+
+```bash
+loki_run_tmp_create() {
+    local temp_root marker
+
+    if [ -n "${LOKI_RUN_TMP:-}" ]; then
+        printf '%s\n' "LOKI_RUN_TMP is already set; refusing to replace it" >&2
+        return 64
+    fi
+
+    temp_root="$(cd "${TMPDIR:-/tmp}" 2>/dev/null && pwd -P)" || {
+        printf '%s\n' "Cannot resolve the temp root" >&2
+        return 64
+    }
+    LOKI_RUN_TMP="$(mktemp -d "${temp_root}/loki-run.XXXXXXXX")" || return 1
+    marker="${LOKI_RUN_TMP}/.loki-run-owned"
+
+    chmod 700 "$LOKI_RUN_TMP" || {
+        rmdir "$LOKI_RUN_TMP" 2>/dev/null || true
+        unset LOKI_RUN_TMP
+        return 1
+    }
+    if ! printf '%s\n' "$LOKI_RUN_TMP" >"$marker" || ! chmod 600 "$marker"; then
+        rm -f -- "$marker"
+        rmdir "$LOKI_RUN_TMP" 2>/dev/null || true
+        unset LOKI_RUN_TMP
+        return 1
+    fi
+    export LOKI_RUN_TMP
+}
+
+# Read one numeric stat field portably. GNU uses -c, BSD uses -f, and each
+# MISPARSES the other's flag: GNU -f means --file-system and still prints a
+# filesystem block on stdout while exiting non-zero, so a bare `a || b`
+# fallback captures that block and concatenates the real value onto it.
+# Validate the captured value instead of trusting exit status or stream, so a
+# usage string, a filesystem block, an empty result, or a concatenation of any
+# of those is rejected on every platform.
+loki_run_tmp_stat_field() {
+    local gnu_fmt="$1" bsd_fmt="$2" path="$3" value
+
+    value="$(stat -c "$gnu_fmt" -- "$path" 2>/dev/null)" || value=''
+    case "$value" in
+        '' | *[!0-9]*) value="$(stat -f "$bsd_fmt" -- "$path" 2>/dev/null)" || value='' ;;
+    esac
+    case "$value" in
+        '' | *[!0-9]*) return 1 ;;
+    esac
+    printf '%s\n' "$value"
+}
+
+loki_run_tmp_cleanup() {
+    local target temp_root marker marker_value target_real
+    local target_uid marker_uid target_mode marker_mode current_uid
+
+    target="${LOKI_RUN_TMP:-}"
+    [ -n "$target" ] || return 0
+    temp_root="$(cd "${TMPDIR:-/tmp}" 2>/dev/null && pwd -P)" || return 64
+
+    case "$target" in
+        "${temp_root}"/loki-run.*) ;;
+        *)
+            printf '%s\n' "Refusing cleanup outside the run-owned temp namespace: $target" >&2
+            return 64
+            ;;
+    esac
+    [ "$(dirname -- "$target")" = "$temp_root" ] || return 64
+    [ -d "$target" ] && [ ! -L "$target" ] || return 64
+
+    marker="${target}/.loki-run-owned"
+    [ -f "$marker" ] && [ ! -L "$marker" ] || return 64
+    IFS= read -r marker_value <"$marker" || return 64
+    [ "$marker_value" = "$target" ] || return 64
+
+    current_uid="$(id -u)" || return 64
+    target_uid="$(loki_run_tmp_stat_field '%u' '%u' "$target")" || return 64
+    marker_uid="$(loki_run_tmp_stat_field '%u' '%u' "$marker")" || return 64
+    [ "$target_uid" = "$current_uid" ] && [ "$marker_uid" = "$current_uid" ] || return 64
+
+    # Refuse anything readable, writable, or executable by group or other. Only
+    # the low three permission digits are compared: GNU '%a' includes the
+    # setuid/setgid/sticky digit and BSD '%Lp' does not, and `mktemp -d` under a
+    # setgid temp root inherits g+s that a three-digit `chmod 700` preserves.
+    # Comparing the raw value against "700" would therefore read 2700 on Linux,
+    # refuse, and leak the very directory this helper exists to remove.
+    target_mode="$(loki_run_tmp_stat_field '%a' '%Lp' "$target")" || return 64
+    marker_mode="$(loki_run_tmp_stat_field '%a' '%Lp' "$marker")" || return 64
+    # Keep the last three digits, zero-padded, then require exactly 700/600.
+    target_mode="00${target_mode}"
+    marker_mode="00${marker_mode}"
+    [ "${target_mode#"${target_mode%???}"}" = "700" ] || return 64
+    [ "${marker_mode#"${marker_mode%???}"}" = "600" ] || return 64
+
+    target_real="$(cd "$target" 2>/dev/null && pwd -P)" || return 64
+    [ "$target_real" = "$target" ] || return 64
+    [ "$(dirname -- "$target_real")" = "$temp_root" ] || return 64
+
+    # A linked worktree has a .git file; a primary worktree has a .git
+    # directory. Refuse both, including a broken .git symlink.
+    if [ -e "${target}/.git" ] || [ -L "${target}/.git" ]; then
+        printf '%s\n' "Refusing to remove a Git worktree: $target" >&2
+        return 64
+    fi
+
+    rm -rf -- "$target"
+    [ ! -e "$target" ] || return 1
+    unset LOKI_RUN_TMP
+}
+```
+
+4. **Verify exact cleanup.** Confirm each recorded PID is gone and the saved
+   `$LOKI_RUN_TMP` path no longer exists. Do not scan or delete other users'
+   or runs' temp paths.
+
+5. **Report cleanup status** to the user in the task completion message,
+   including only the explicit run-owned path and process IDs handled.
+
+## Feedback Loop Requirement (before documenting a new feature)
+
+1. **Verify it exists** - check files, run commands, test endpoints.
+2. **Run feedback loop** - Task tool with Opus to review claims for accuracy.
+3. **Be factual only** - never document features that don't work yet.
+4. **Mark planned features** - use "Coming Soon" or "Planned" labels.
+
+```bash
+# Before documenting "npm install -g loki-mode"
+npm view loki-mode  # Does package exist on registry?
+# Before documenting a CLI command
+which loki && loki --help
+# Before documenting a file path
+ls -la path/to/file
+```