@misterhuydo/sentinel 1.6.9 → 1.6.11

package/.cairn/.hint-lock

@@ -1 +1 @@
-2026-04-24T14:02:31.712Z
+2026-04-27T13:09:20.340Z

package/.cairn/memory/auto-memory/MEMORY.md

@@ -0,0 +1,21 @@
+# Memory Index
+
+- [sentinel_deployment_server](experience_sentinel_deployment_server.md) — Sentinel deploys on Oracle Ampere ARM server, SSH via ageri user `[experience]`
+- [publish_safety_check](experience_publish_safety_check.md) — Always syntax-check Python and JS files before npm publish `[experience]`
+- [cairn_session_discipline](experience_cairn_session_discipline.md) — Use cairn_resume at session start, cairn_checkpoint at end — not cairn_maintain `[experience]`
+- [secrets_in_gitignored_files](experience_secrets_in_gitignored_files.md) — All secrets go in gitignored files only — never in committed config `[experience]`
+- [auto_commit_auto_release_split](decision_auto_commit_auto_release_split.md) — AUTO_PUBLISH split into AUTO_COMMIT + AUTO_RELEASE with 3-level hierarchy `[decision]`
+- [bash_local_shadows_env_var](experience_bash_local_shadows_env_var.md) — bash `local var=""` shadows same-named env vars — use a distinct SENTINEL_*_OVERRIDE pattern `[experience]`
+- [sentinel_upgrade_requires_file_copy](knowledge_sentinel_upgrade_requires_file_copy.md) — npm install -g does NOT update the running instance — must copy files from npm package to /home/sentinel/sentinel/code/ and restart `[knowledge]`
+- [multi_repo_fix_architecture](decision_multi_repo_fix_architecture.md) — Multi-repo fix flow: parse_multi_repo_patch → atomic dry-run → per-repo apply/PR with sibling list `[decision]`
+- [per_project_claude_session](decision_per_project_claude_session.md) — Long-lived `claude --resume` session per project, persisted in claude_sessions table; per-project asyncio.Lock `[decision]`
+- [cairn_federation_for_sentinel_projects](knowledge_cairn_federation_for_sentinel_projects.md) — Sentinel project root + each sub-repo both run `cairn install`; federation auto-mounts via parent walk-up `[knowledge]`
+- [claude_resume_stale_context_risk](experience_claude_resume_stale_context_risk.md) — claude --resume carries file-content memory across turns — must use per-route session keys + force fresh Read `[experience]`
+- [no_api_key_for_heavy_coding](preference_no_api_key_for_heavy_coding.md) — Never bill heavy coding tasks against ANTHROPIC_API_KEY — must use Claude Pro OAuth `[preference]`
+- [cairn_hooks_block_oauth_read_edit](knowledge_cairn_hooks_block_oauth_read_edit.md) — Cairn PreToolUse hooks (minify, edit-guard) exit 2 to block Read/Edit; bypass via --setting-sources project,local + re-add MCP via --mcp-config `[knowledge]`
+- [git_apply_recount_for_llm_diffs](decision_git_apply_recount_for_llm_diffs.md) — Always pass `git apply --recount` for LLM-generated patches — fixes off-by-one hunk header counts `[decision]`
+- [mvn_negative_cache_blocks_cascade_retries](experience_mvn_negative_cache_blocks_cascade_retries.md) — mvn caches "artifact not found" responses for ~24h — pass `-U` to bypass when polling for a freshly-published artifact `[experience]`
+- [jenkins_wait_before_cascade](decision_jenkins_wait_before_cascade.md) — manage_release blocks on Jenkins build (wait=True) before firing cascade — eliminates Nexus race `[decision]`
+- [cicd_user_must_not_be_hardcoded](experience_cicd_user_must_not_be_hardcoded.md) — Jenkins triggers must use repo.cicd_user — hardcoded "sentinel" caused silent 401s for all elprint builds `[experience]`
+- [sentinel_repo_remotes](knowledge_sentinel_repo_remotes.md) — J:\Projects\Sentinel = source of truth; misterhuydo/Sentinel primary; exoreaction/Sentinel = downstream mirror `[knowledge]`
+- [sentinel_systemd_inactive_is_misleading](knowledge_sentinel_systemd_inactive_is_misleading.md) — systemctl status sentinel reports inactive even when workers run; check ps + per-project PID files instead `[knowledge]`

package/.cairn/memory/auto-memory/decision_auto_commit_auto_release_split.md

@@ -0,0 +1,22 @@
+---
+name: auto_commit_auto_release_split
+description: AUTO_PUBLISH split into AUTO_COMMIT + AUTO_RELEASE with 3-level hierarchy
+type: decision
+created_at: 2026-04-08T16:00:35.305Z
+updated_at: 2026-04-08T16:00:35.305Z
+---
+
+AUTO_PUBLISH was split into two independent flags (v1.5.17):
+
+- AUTO_COMMIT: push directly to main (no PR). Default: false.
+- AUTO_RELEASE: trigger Jenkins/GHA pipeline after push. Default: false.
+
+Both follow 3-level hierarchy: repo override → project default (sentinel.properties) → false.
+
+Current config:
+- sentinel-1881: AUTO_COMMIT=true, AUTO_RELEASE=false (project level)
+- sentinel-elprint: AUTO_COMMIT=true, AUTO_RELEASE=false (project level)
+- All individual repo configs: no AUTO_COMMIT/AUTO_RELEASE (inherit from project)
+
+Pending releases: when AUTO_COMMIT=true and AUTO_RELEASE=false, committed pushes are
+recorded in pending_releases table. get_status exposes them. manage_release clears them.

package/.cairn/memory/auto-memory/decision_git_apply_recount_for_llm_diffs.md

@@ -0,0 +1,17 @@
+---
+name: git_apply_recount_for_llm_diffs
+description: Always pass `git apply --recount` for LLM-generated patches — fixes off-by-one hunk header counts
+type: decision
+created_at: 2026-04-24T12:34:44.489Z
+updated_at: 2026-04-24T12:34:44.489Z
+---
+
+All `git apply` invocations in `sentinel/git_manager.py` (both `--check` dry-runs and actual applies, single-repo and multi-repo) MUST include `--recount`.
+
+**Why:** LLM-generated unified diffs frequently miscount hunk header line counts (`@@ -X,N +Y,M @@`) by ±1 — Claude can write a hunk body with 9 old / 19 new lines but emit a header `@@ -4,9 +4,18 @@`. Without `--recount`, this fails dry-run with "corrupt patch at line N". With `--recount`, git infers counts from the actual `+/-/ ` lines and the patch applies cleanly.
+
+**Origin:** v1.6.5 ended a long debugging arc where Claude in `--print` mode kept producing patches with miscounted hunks even after fixing cairn-hook permission denials and per-route session contamination. The actual content was always correct — only headers were off. `--recount` made every prior failure into a clean apply.
+
+**Where applied:** `git_manager.py:266` (apply_and_commit dry-run), `:271` (apply), `:380` (apply_and_commit_multi dry-run), `:412` (apply_and_commit_multi apply). All four sites now use `["apply", "--check", "--recount", "--ignore-whitespace", ...]` or `["apply", "--recount", "--ignore-whitespace", ...]`.
+
+**Caveat:** `--recount` only fixes header miscounts. If the actual `+/-/ ` lines are wrong (missing context, wrong content), the patch still fails — which is correct safety behavior.

package/.cairn/memory/auto-memory/decision_jenkins_wait_before_cascade.md

@@ -0,0 +1,23 @@
+---
+name: jenkins_wait_before_cascade
+description: manage_release blocks on Jenkins build (wait=True) before firing cascade — eliminates Nexus race
+type: decision
+created_at: 2026-04-24T13:57:52.942Z
+updated_at: 2026-04-24T13:57:52.942Z
+---
+
+When `Boss.manage_release(operation='release' or 'release_and_cascade', confirmed=True)` is called, sentinel_boss.py now invokes `_trigger_jenkins_release(repo, wait=True)` whenever a cascade will follow.
+
+**Why:** the cascade calls `mvn compile -DskipTests -U` per dependent repo, which queries Nexus for the new artifact. If Jenkins is still building, Nexus returns 404 and (without `-U` — fixed in v1.6.6) mvn caches that for ~24h. Even with `-U`, racing Jenkins is wasteful — better to know exactly when the build finished via Jenkins's own API.
+
+**How `_wait_for_jenkins_build` works** (`sentinel/cicd_trigger.py`):
+- Records `lastBuild` number BEFORE triggering
+- Polls `<job_url>/api/json` every 20s for up to 15 min (`JENKINS_RELEASE_TIMEOUT=900`)
+- Returns True only if a NEW build appears AND completes with `result=SUCCESS`
+- Returns False on FAILURE / ABORTED / TIMEOUT
+
+**Caveat:** Boss conversation BLOCKS for up to 15 min while waiting. Mitigation: a Slack `:hourglass_flowing_sand:` message is posted right before the wait so the user knows what's happening. Future improvement: run the wait async with periodic Slack updates.
+
+**Where applied:** `sentinel/sentinel_boss.py` `manage_release` handler, `operation in ("release", "release_and_cascade")` branch (around line 4912 in v1.6.7).
+
+Shipped in v1.6.7. Pairs with v1.6.6's `-U` fix to mvn — both are needed: -U for the case where someone runs the cascade independently, wait=True for the integrated trigger-then-cascade flow.

package/.cairn/memory/auto-memory/decision_multi_repo_fix_architecture.md

@@ -0,0 +1,25 @@
+---
+name: multi_repo_fix_architecture
+description: Multi-repo fix flow: parse_multi_repo_patch → atomic dry-run → per-repo apply/PR with sibling list
+type: decision
+created_at: 2026-04-24T10:45:04.157Z
+updated_at: 2026-04-24T10:45:04.157Z
+---
+
+Sentinel now supports cross-repo fixes (one task → multiple repos).
+
+**Patch format claude must produce:**
+- Paths prefixed `repos/<repo-name>/...` (relative to project root)
+- Optional `# Affected repos: a, b, c` header line — order = merge order (library first, consumer after)
+
+**Flow (in main._generate_apply_publish):**
+1. `generate_fix(..., all_repos=cfg_loader.repos.values())` — Claude runs from project root with cairn federation visibility, gets `--resume <session_id>` + `--output-format json`
+2. `apply_and_commit_multi(event, patch_path, all_repos, cfg)` — splits combined patch by `repos/<name>/` prefix, atomic dry-run all repos first; if ANY fails, ALL marked "aborted" with no commits. Then per-repo: apply + test + commit (per-repo failures don't block others)
+3. `publish_multi(event, results, cfg)` — pushes branches, opens PRs with `extra_body` listing sibling repo NAMES (URL cross-refs deferred to v2 — needs two-pass GitHub PATCH)
+4. Per-repo state goes to `fix_repos` table; primary repo also lands in legacy `fixes` for back-compat with reporter/Boss
+
+**Fallback:** If patch has no `repos/<name>/` prefix (legacy/manual), `apply_and_commit_multi` returns `[]` and main falls back to single-repo `apply_and_commit + publish`.
+
+**Files:** sentinel/git_manager.py (parse_multi_repo_patch + apply_and_commit_multi + publish_multi), sentinel/fix_engine.py (multi-repo prompt + JSON parser + session resume), sentinel/main.py (_generate_apply_publish + _apply_publish_single fallback).
+
+**Removed:** MAX_FILES_IN_PATCH=5 / MAX_LINES_IN_PATCH=200 limits. Claude is trusted to size patches appropriately.

package/.cairn/memory/auto-memory/decision_per_project_claude_session.md

@@ -0,0 +1,23 @@
+---
+name: per_project_claude_session
+description: Long-lived `claude --resume` session per project, persisted in claude_sessions table; per-project asyncio.Lock
+type: decision
+created_at: 2026-04-24T10:45:21.608Z
+updated_at: 2026-04-24T10:45:21.608Z
+---
+
+Each Sentinel project keeps ONE long-lived Claude conversation across fix tasks (NOT one subprocess — claude is still spawned per task, but with `--resume <session_id>` so it continues the previous conversation).
+
+**Why this design (not long-lived subprocess):**
+- `claude` CLI isn't built for stdin/stdout piping — no clean response framing, hangs on confirmations, ANSI escapes leak. Subprocess startup (~1s) is negligible vs the 30s-2min fix runtime.
+- `--resume` gives prompt-cache reuse + cross-task context without process-management complexity.
+
+**Mechanism:**
+- `state_store.claude_sessions(project_name PK, session_id, last_used, total_cost_usd, turn_count)` table
+- `fix_engine.generate_fix` reads `store.get_claude_session(cfg.project_name)` and passes the id via `--resume <id>`
+- After the call, parses `{"session_id": "...", "total_cost_usd": ...}` from the JSON output and `store.set_claude_session(project_name, id, cost_delta=cost)` — accumulates cost + turn count
+- `--output-format json` is required to extract session_id deterministically. Live tool-use progress streaming (`⏺ Bash(...)`, etc) is sacrificed — `_progress_from_line` is now dead code in the JSON path. stream-json migration deferred.
+
+**Concurrency safety:**
+- `main._project_locks: dict[str, asyncio.Lock]` keyed by `cfg.project_name` (or `"_default"` if empty)
+- `_handle_error` wraps `_generate_apply_publish` in `async with _project_lock(...)` — guarantees one Claude session active per project at a time. Different projects can run in parallel.

package/.cairn/memory/auto-memory/experience_bash_local_shadows_env_var.md

@@ -0,0 +1,11 @@
+---
+name: bash_local_shadows_env_var
+description: bash `local var=""` shadows same-named env vars — use a distinct SENTINEL_*_OVERRIDE pattern
+type: experience
+created_at: 2026-04-21T07:22:54.615Z
+updated_at: 2026-04-21T07:22:54.615Z
+---
+
+In fetch_log.sh, `local OUTPUT_DIR=""` inside fetch_from_properties() shadows any OUTPUT_DIR env var set by the Python caller. Python was setting env["OUTPUT_DIR"] = temp_path, but bash's local declaration wiped it, causing filtered logs to always write to $SCRIPT_DIR instead of the temp dir — so monitor results were always empty.
+
+Fix: use a dedicated env var (SENTINEL_OUTPUT_DIR_OVERRIDE) that doesn't conflict with a local variable name, and apply it after the local declarations. This is the same pattern already used for SENTINEL_GREP_FILTER_OVERRIDE.

package/.cairn/memory/auto-memory/experience_cairn_session_discipline.md

@@ -0,0 +1,9 @@
+---
+name: cairn_session_discipline
+description: Use cairn_resume at session start, cairn_checkpoint at end — not cairn_maintain
+type: experience
+created_at: 2026-03-31T03:26:44.345Z
+updated_at: 2026-03-31T03:26:44.345Z
+---
+
+At the start of a session, always call `cairn_resume` (not `cairn_maintain`) to get incremental re-index + memory surface from the last checkpoint. `cairn_maintain` does a full re-index and loses prior session context. At the end of every session, call `cairn_checkpoint` with a message, active_files, and any notes worth carrying forward. Skipping checkpoint means the next session starts cold.

package/.cairn/memory/auto-memory/experience_cicd_user_must_not_be_hardcoded.md

@@ -0,0 +1,17 @@
+---
+name: cicd_user_must_not_be_hardcoded
+description: Jenkins triggers must use repo.cicd_user — hardcoded "sentinel" caused silent 401s for all elprint builds
+type: experience
+created_at: 2026-04-24T14:06:02.248Z
+updated_at: 2026-04-24T14:06:02.248Z
+---
+
+In `sentinel/cicd_trigger.py`, `_trigger_jenkins(repo)` was passing `auth=("sentinel", repo.cicd_token)` — hardcoding "sentinel" as the username. This silently broke every Jenkins API call when the repo's API token was issued under a different user (e.g. CICD_USER=misterhuydo).
+
+**Detection** (2026-04-24): Boss tried to trigger 14 elprint builds; all returned HTTP 401. Direct curl from EC2 with `-u misterhuydo:<token>` returned HTTP 200, proving creds were valid. Bug was the hardcoded "sentinel" username.
+
+**Fix** (v1.6.8): use `auth=(repo.cicd_user or "sentinel", repo.cicd_token)` — same pattern already in use by `_trigger_jenkins_release` (which was unaffected). One-line fix.
+
+**Lesson:** when adding new auth code, always thread `repo.cicd_user` through. Never hardcode usernames. The fallback to "sentinel" is fine for backward compat, but the configured user must take priority.
+
+Affected functions: `_trigger_jenkins` (fixed in v1.6.8). Others (`_trigger_jenkins_release`, `_trigger_github_actions`) already correct.

package/.cairn/memory/auto-memory/experience_claude_resume_stale_context_risk.md

@@ -0,0 +1,23 @@
+---
+name: claude_resume_stale_context_risk
+description: claude --resume carries file-content memory across turns — must use per-route session keys + force fresh Read
+type: experience
+created_at: 2026-04-24T11:26:21.418Z
+updated_at: 2026-04-24T11:26:21.418Z
+---
+
+When using `claude --print --resume <session_id>` for sentinel fix engine, the resumed Claude carries memory of file contents from earlier turns. If the next task targets a DIFFERENT repo (or the same file has changed), Claude may patch from stale memory instead of re-reading the actual file → diff context mismatch → `git apply --check` fails.
+
+**Real incident (2026-04-24):**
+- Turn 1: fix attempt routed to `1881-SSOLoginWebApp` (local override of FirstName.java). Claude got session `d1c18a02`. Failed.
+- Turn 2: new fix routed to `Whydah-TypeLib` (the upstream FirstName.java which differs from the local override). With per-project resume, Claude resumed `d1c18a02` and patched TypeLib using the SSOLWA file's content from memory. Patch context didn't match TypeLib's actual file. Dry-run aborted.
+
+**Three layered defenses (all shipped in v1.6.2):**
+
+1. **Per-route session key**: in fix_engine.generate_fix, key claude_sessions by `f"{cfg.project_name}/{repo.repo_name}"` instead of just `cfg.project_name`. Different routes get fresh sessions.
+
+2. **Pre-fix `git pull` of every project repo** via `git_manager.pull_all_repos(all_repos)`. Closes the window where the on-disk file lags behind a recent sentinel commit or human commit.
+
+3. **Prompt instruction**: "CRITICAL — fresh reads only. Before you write ANY diff line, use the Read tool to view the CURRENT content of every file you intend to modify. Do NOT rely on prior memory ..."
+
+**General rule when using --resume:** if the task topic / file scope changes between turns, either start a fresh session OR explicitly tell Claude to re-read everything. Never trust memory across context boundaries.

package/.cairn/memory/auto-memory/experience_envelope_first_auth_detection.md

@@ -0,0 +1,11 @@
+---
+name: envelope_first_auth_detection
+description: Substring-based auth-error detection on LLM output triggers false positives when patches contain words like "UNAUTHORIZED" — check the JSON envelope first
+type: experience
+originSessionId: d78671ef-fb75-4934-949b-11dda1645051
+---
+When a Claude `--print --output-format json` invocation succeeds, its `result` field can contain arbitrary text from the codebase being patched — including unchanged-context diff lines like `return new ResponseEntity<>(err, HttpStatus.UNAUTHORIZED)`. Substring matching on the raw output (e.g. checking for `"unauthorized"`, `"unauthenticated"`, `"login required"`) will fire even when the request authenticated cleanly, producing false "auth failure" Slack alerts and wasting both attempts in a fallback loop.
+
+**Why:** On 2026-04-27 fp `6cb7a875` produced a valid HttpRequestMethodNotSupportedException handler patch, but the diff included an unchanged `HttpStatus.UNAUTHORIZED` context line for a sibling exception handler. Both OAuth and API-key attempts were marked auth-failed, the patch was discarded, and the user got a misleading "out of session?" alert. Cost ~$0.62 for the wasted retries; fix would have shipped to prod 3 hours earlier without the false positive.
+
+**How to apply:** Whenever you have both a structured envelope (`is_error` flag, JSON status) and a free-text scan, *consult the envelope first*. The substring scan is only valid as a fallback for early-exit failures that never produced a parseable envelope (binary missing, immediate stderr crash). Fix in `sentinel/fix_engine.py` shipped in v1.6.10: parse JSON before running `_is_auth_error()`; if `is_error:false` and `result` is non-empty, treat it as success regardless of body content.

package/.cairn/memory/auto-memory/experience_mvn_negative_cache_blocks_cascade_retries.md

@@ -0,0 +1,17 @@
+---
+name: mvn_negative_cache_blocks_cascade_retries
+description: mvn caches "artifact not found" responses for ~24h — pass `-U` to bypass when polling for a freshly-published artifact
+type: experience
+created_at: 2026-04-24T13:27:45.148Z
+updated_at: 2026-04-24T13:27:45.148Z
+---
+
+When a sentinel cascade fires immediately after triggering a Jenkins release, mvn often hits Nexus before Jenkins finishes publishing the new artifact. The `404 not found` response is **cached locally** in `~/.m2/repository/<group>/<artifact>/<version>/_remote.repositories` (or via the `--no-transfer-progress` machinery), and mvn refuses to re-check that artifact for ~24h (the default `updatePolicy` interval).
+
+Symptom: cascade reports "Artifact X:Y not yet in Nexus", user retries 5 minutes later, same failure even though Nexus actually has the artifact. The user can manually confirm via their local IDE (which has its OWN m2 cache and saw the artifact at a different time).
+
+**Fix:** add `-U` (`--update-snapshots`) to every `mvn` invocation in the cascade path. `-U` forces mvn to re-check remote repos on every call, bypassing the negative cache. Cost: slightly more network traffic. Benefit: cascade retries actually work.
+
+Shipped in v1.6.6 in `sentinel/git_manager.py:maven_compile_check` — added `-U` to the `mvn compile -DskipTests -q --batch-mode` command.
+
+**Also worth knowing:** Sentinel detects the Nexus URL from the project's `pom.xml` (`<repositories>` block), NOT from `~/.m2/settings.xml`. settings.xml only carries credentials, not URLs.

package/.cairn/memory/auto-memory/experience_publish_safety_check.md

@@ -0,0 +1,13 @@
+---
+name: publish_safety_check
+description: Always syntax-check Python and JS files before npm publish
+type: experience
+created_at: 2026-03-31T03:26:41.450Z
+updated_at: 2026-03-31T03:26:41.450Z
+---
+
+Before running `npm publish`, always validate modified files:
+- Python: `python -c "import ast; ast.parse(open('file.py').read())"`
+- JS/Node: `node --check file.js`
+
+The bundle step does NOT catch Python syntax errors. Backtick characters inside JS template literals will also break when embedding Python code blocks — avoid them or escape carefully.

package/.cairn/memory/auto-memory/experience_secrets_in_gitignored_files.md

@@ -0,0 +1,9 @@
+---
+name: secrets_in_gitignored_files
+description: All secrets go in gitignored files only — never in committed config
+type: experience
+created_at: 2026-03-31T03:26:47.431Z
+updated_at: 2026-03-31T03:26:47.431Z
+---
+
+Never put tokens, API keys, passwords, or any secrets in committed config files. Keep them in a gitignored file such as `private_*.properties`, `.env`, or equivalent that lives only on the server. If a project has a committed config template (e.g. `sentinel.properties`), use placeholder values like `<token>` and document that the real values go in the private file.

package/.cairn/memory/auto-memory/experience_sentinel_deployment_server.md

@@ -0,0 +1,13 @@
+---
+name: sentinel_deployment_server
+description: Sentinel deploys on EC2 server, SSH via ec2-user with sentinel.pem key
+type: experience
+created_at: 2026-03-31T03:23:46.605Z
+updated_at: 2026-04-21T00:00:00.000Z
+originSessionId: bbd5aef7-18ed-4068-be3f-60d68de4936f
+---
+Sentinel is deployed on EC2 at `13.50.101.130`. SSH: `ssh -i ~/.ssh/sentinel.pem ec2-user@13.50.101.130` (key at `C:\Users\huy\.ssh\sentinel.pem`).
+
+Single service: `sentinel.service` (not sentinel-1881 or sentinel-elprint — just `sentinel`).
+
+Deploy: `sudo npm install -g @misterhuydo/sentinel@<version> && sudo systemctl restart sentinel`

package/.cairn/memory/auto-memory/feedback_ageri_rag_architecture.md

@@ -0,0 +1,19 @@
+---
+name: Ageri RAG and CoreSkill architecture decisions
+description: CoreSkill is passive observer only; personal skill uses layered context pipeline for RAG
+type: feedback
+---
+
+CoreSkill must remain a passive observer — no URL fetching, no active retrieval. It only listens, extracts explicit facts, and writes to memory silently. The moment it fetches URLs it changes nature.
+
+Personal skill RAG uses a layered context pipeline (pre-LLM step):
+1. Memory search — what we know about this user/topic
+2. URL fetch — if message contains a URL, fetch it
+3. prior_results — what other skills returned this turn
+4. User-uploaded docs — files stored from previous uploads
+
+One context build → one LLM call. Not scattered retrieval inside each skill.
+
+**Why:** User confirmed this pushback explicitly. Separation of concerns — CoreSkill listens, research skill searches broadly, personal skill retrieves personal context.
+
+**How to apply:** Never add active retrieval to CoreSkill. When building personal skill RAG, use the pipeline approach above, not ad-hoc fetching in individual handlers.

package/.cairn/memory/auto-memory/feedback_design_principle.md

@@ -0,0 +1,16 @@
+---
+name: Design principle — simple but sophisticated
+description: User's core design philosophy for Sentinel and all work in this project
+type: feedback
+---
+
+"Simple but sophisticated" — the guiding principle for all design decisions.
+
+**Why:** User explicitly stated this after I added a redundant SLACK_WORKSPACE_ID workspace verification check that duplicated isolation Slack's own token architecture already provides.
+
+**How to apply:**
+- Solve the real problem, not hypothetical edge cases the platform already handles
+- Don't add config options, fields, or checks that guard against things that can't happen
+- When the underlying system (Slack, GitHub, SQLite, etc.) already enforces a constraint, trust it — don't re-enforce it in code
+- Prefer fewer moving parts. A sophisticated outcome should feel simple to operate.
+- Before adding any new mechanism, ask: "does something already handle this?"

package/.cairn/memory/auto-memory/feedback_publish_workflow.md

@@ -0,0 +1,14 @@
+---
+name: Sentinel publish workflow
+description: How to correctly publish Sentinel npm package
+type: feedback
+
+---
+
+Always run syntax checks before publishing: `python -c "import ast; ast.parse(open(f, encoding='utf-8').read())"` on modified Python files, and `node --check` on modified JS files.
+
+**Why:** Multiple bugs shipped due to literal newlines in JS strings and Python indentation errors. These are caught instantly by syntax checks.
+
+**How to apply:** Before every `npm publish`, check all modified files. The bundle step will succeed even with broken Python (it just copies files), so Python errors only surface at runtime on the server.
+
+Also: JS template literals containing backtick characters (Python f-strings with backtick code formatting) will cause syntax errors. Use string concatenation instead of template literals for multi-line Python code blocks in patch scripts.

package/.cairn/memory/auto-memory/feedback_secrets_handling.md

@@ -0,0 +1,15 @@
+---
+name: Secrets handling — never commit tokens
+description: All secrets (GITHUB_TOKEN, SLACK tokens, API keys) go in private_sentinel.properties only, never in committed config files
+type: feedback
+---
+
+Never put secrets in config repo files (sentinel.properties, repo-configs/*.properties). They get committed to GitHub.
+
+**Why:** GITHUB_TOKEN, SLACK_BOT_TOKEN, ANTHROPIC_API_KEY etc. were accidentally added to committed sentinel.properties files. These end up in GitHub history even if later removed.
+
+**How to apply:**
+- All secrets go in `private_sentinel.properties` (gitignored, lives next to the project dir on the server)
+- Committed `sentinel.properties` files contain only non-sensitive config (PROJECT_NAME, MAILS, POLL_INTERVAL, etc.)
+- The `.gitignore` in every sentinel config repo must include `private_sentinel.properties`
+- On the server: `~/sentinel/private_sentinel.properties` for workspace-level secrets

package/.cairn/memory/auto-memory/feedback_slack_admin_allowlist.md

@@ -0,0 +1,11 @@
+---
+name: SLACK_ADMIN_USERS implicitly allowed
+description: Admins are always allowed to talk to Boss — no need to also add them to SLACK_ALLOWED_USERS
+type: feedback
+---
+
+`SLACK_ADMIN_USERS` implies access. Do not require admins to also appear in `SLACK_ALLOWED_USERS`.
+
+**Why:** User pointed out the redundancy — if you're an admin you should obviously be able to talk to the bot. Fixed in slack_bot.py: allowlist check now skips users who are in `slack_admin_users`.
+
+**How to apply:** When configuring a new Sentinel instance, only set `SLACK_ADMIN_USERS` in `sentinel.properties`. `SLACK_ALLOWED_USERS` is only needed to grant access to non-admin users.

package/.cairn/memory/auto-memory/feedback_slack_thinking_status.md

@@ -0,0 +1,14 @@
+---
+name: Slack thinking status messages
+description: User confirmed they like the random thinking status messages in Slack Boss
+type: feedback
+---
+
+User explicitly approved the random thinking status approach (`_THINKING_STATUS` list with `random.choice()`).
+
+Messages like "poking around...", "on it...", "cooking..." are preferred over the static "thinking...".
+
+The "(still on it...)" suffix appended on long responses was also well received — it reassures the user the bot hasn't stalled.
+
+**Why:** More personality, less robotic. Matches the casual tone the user wants from Boss.
+**How to apply:** Keep the `_THINKING_STATUS` list and the "(still on it...)" update pattern. Do not revert to static "thinking...".

package/.cairn/memory/auto-memory/feedback_start_sh_patching.md

@@ -0,0 +1,16 @@
+---
+name: start.sh patching approach
+description: How to safely patch start.sh on the server — SSH double-quote expansion pitfall, $HOME vs hardcoded paths
+type: feedback
+---
+
+Always use `$HOME` (not hardcoded user paths like `/home/sentinel/`) in start.sh templates and patches. The script may run as any Linux user, not just `sentinel`.
+
+**Why:** The server user might not be `sentinel` — admin may choose any username. Using `$HOME` makes the script portable.
+
+**How to apply:** The generate.js template already uses `$HOME`. When patching start.sh on the server via SSH, never write Python patch scripts that include `$HOME` inside an SSH double-quoted string — bash will expand `$HOME` to the *local* machine's home before sending. Instead:
+- Use `sed -i` with single-quoted replacement strings (prevents local expansion)
+- Or write the Python patch to `/tmp/fix.py` via heredoc `<< 'PYEOF'` (single-quoted prevents expansion), then run separately
+- Or use SCP to upload the script file
+
+Also: `sentinel upgrade` regenerates start.sh from generate.js template. The template (line ~65 in generate.js) now includes auto-detection of JAVA_HOME via a for-loop over `$HOME/jdk-*` patterns, using `$HOME` throughout. This is permanent fix — no need to re-patch after upgrades.

package/.cairn/memory/auto-memory/knowledge_cairn_federation_for_sentinel_projects.md

@@ -0,0 +1,20 @@
+---
+name: cairn_federation_for_sentinel_projects
+description: Sentinel project root + each sub-repo both run `cairn install`; federation auto-mounts via parent walk-up
+type: knowledge
+created_at: 2026-04-24T10:45:36.058Z
+updated_at: 2026-04-24T10:45:36.058Z
+---
+
+Cairn supports nested federation natively. For Sentinel:
+- Project root (e.g. `/home/sentinel/sentinel/sentinel-1881/`) gets `.cairn/` (its own `.cairn-project` marker + `index.db`)
+- Each sub-repo under `repos/<name>/` ALSO gets its own `.cairn/`
+- When Claude runs from the project root, cairn's `mountParentSubIndexes` (Cairn `db.js:97`) auto-discovers sibling sub-indexes — no explicit registration needed (works "even if the parent project has never run cairn_maintain")
+
+**Sentinel wiring:**
+- `sentinel/cairn_client.py:_install_cairn_at(path)` — idempotent (skip if `.cairn/.cairn-project` marker exists), takes any path
+- `init_project_root(project_dir)` — wrapper invoked at sentinel startup
+- `index_repo(repo)` — same wrapper, called for each sub-repo (existing behaviour)
+- `main._startup_checks` calls `init_project_root(Path(cfg.workspace_dir).parent)` BEFORE iterating sub-repos with `index_repo`
+
+**Cairn CLI surface is limited** (`install`, `install-hooks`, `minify`, `edit-guard`, `validate-map`, `checkpoint --auto`, `resume-hint`). All federation/index queries are MCP-only — Claude calls them via MCP at runtime, Sentinel doesn't shell out for them.

package/.cairn/memory/auto-memory/knowledge_cairn_hooks_block_oauth_read_edit.md

@@ -0,0 +1,24 @@
+---
+name: cairn_hooks_block_oauth_read_edit
+description: Cairn PreToolUse hooks (minify, edit-guard) exit 2 to block Read/Edit; bypass via --setting-sources project,local + re-add MCP via --mcp-config
+type: knowledge
+created_at: 2026-04-24T12:00:46.946Z
+updated_at: 2026-04-24T12:00:46.946Z
+---
+
+Cairn ships two `PreToolUse` hooks installed in `~/.claude/settings.json`:
+- `cairn minify` on `Read` — exits with code 2 (blocking) and writes minified content to stderr; advances per-file state machine to `compressed`
+- `cairn edit-guard` on `Edit`/`Write` — exits 2 if file is in `compressed` state, shows full content, advances to `edit-ready`. Next Edit attempt then succeeds.
+
+This works fine for INTERACTIVE Claude Code (model retries naturally). In headless `claude --print` sessions doing complex multi-step work (26 turns of fix engine), the model often does NOT retry the blocked Edit — it falls back to hand-crafting a unified diff in markdown. That hand-crafted diff has off-by-one hunk-line counting bugs (`@@ -X,10 +X,11 @@` with a body of 9/10 lines) → `git apply --check` fails with "corrupt patch at line N".
+
+`--bare` would skip ALL hooks (per `claude --help`: "skip hooks, LSP, plugin sync...") but ALSO forces ANTHROPIC_API_KEY auth — incompatible with OAuth.
+
+**Surgical fix** for headless OAuth sessions:
+- `--setting-sources project,local` — skip user-scope settings.json (where cairn hooks live), keep project/local settings working
+- `--mcp-config '{"mcpServers":{"cairn":{"command":"cairn-mcp"}}}'` — re-add cairn MCP (which was also in user settings) standalone, so cairn_search/outline/checkpoint stay available
+- `--dangerously-skip-permissions` — keep, redundant given Read(**) allow but harmless
+
+This combo is in `sentinel/fix_engine.py:_claude_cmd` since v1.6.3.
+
+NOTE: When invoking `claude --mcp-config <json>` from a shell, the JSON string can be misparsed if a positional prompt follows. Use `--mcp-config=<json>` (= form) OR pass via Python subprocess as a list element (no shell). MCP config flag accepts space-separated values — that's why a positional prompt gets pulled in as a "second config".

package/.cairn/memory/auto-memory/knowledge_sentinel_repo_remotes.md

@@ -0,0 +1,28 @@
+---
+name: sentinel_repo_remotes
+description: J:\Projects\Sentinel = source of truth; misterhuydo/Sentinel primary; exoreaction/Sentinel = downstream mirror
+type: knowledge
+created_at: 2026-04-24T15:03:35.440Z
+updated_at: 2026-04-24T15:03:35.440Z
+---
+
+**Sentinel repo topology (set 2026-04-24):**
+
+- **J:\Projects\Sentinel** — single source of truth, primary working directory. ALL development happens here.
+- **misterhuydo/Sentinel** on GitHub — the personal/canonical remote, pushed to from J:\
+- **exoreaction/Sentinel** on GitHub — company-visible MIRROR. Always kept in sync with main; NOT the place to develop in.
+- **H:\Projects\exoreaction\Sentinel** — local clone of exoreaction/Sentinel; redundant when J:\ is the working dir, but harmless to keep around.
+
+**Workflow:**
+1. Edit, commit, and push from J:\Projects\Sentinel as usual.
+2. After pushing to misterhuydo, also push the same refs to exoreaction so the mirror stays current. Easiest: add exoreaction as a second remote in J:\ and push to both.
+
+```
+git -C /j/Projects/Sentinel remote add exoreaction git@github.com:exoreaction/Sentinel.git
+git -C /j/Projects/Sentinel push origin main
+git -C /j/Projects/Sentinel push exoreaction main
+```
+
+Or chain into one command via an alias / shell function.
+
+**Do NOT** do new work in H:\Projects\exoreaction\Sentinel — that workspace is just a local checkout of the mirror.

package/.cairn/memory/auto-memory/knowledge_sentinel_systemd_inactive_is_misleading.md

@@ -0,0 +1,18 @@
+---
+name: sentinel_systemd_inactive_is_misleading
+description: systemctl status sentinel shows inactive even when worker processes are healthy — check ps for python -m sentinel.main, not systemd
+type: knowledge
+originSessionId: 98f6382b-f7e7-45dd-8f41-8f5b249e0853
+---
+The `sentinel.service` systemd unit is `Type=forking` but its `ExecStart=/home/sentinel/sentinel/startAll.sh` backgrounds per-project children (`python -m sentinel.main`) without writing a `PIDFile=` that systemd can track. As a result `systemctl status sentinel` reports `inactive (dead)` immediately after start, even when all workers are running normally.
+
+**Why:** This is a unit-file design quirk, not an outage. The wrapper script + `Type=forking` mismatch means systemd loses track of the children. Restarting the unit will start a NEW set of workers without killing the existing ones — risk of duplicates.
+
+**How to apply:** When asked "is sentinel down?":
+1. Don't trust `systemctl is-active sentinel`.
+2. Check `ps -ef | grep "python3 -m sentinel.main"` and `ls /proc/<pid>/cwd` to map PIDs → projects.
+3. Cross-check against `sentinel.pid` in each `/home/sentinel/sentinel/<project>/`.
+4. Check `tail logs/sentinel.log` for recent activity.
+5. If duplicate processes for the same project exist (multiple PIDs with the same cwd), the older one is an orphan — `stop.sh` only kills the one in `sentinel.pid`. Kill orphans manually with `kill <pid>`.
+
+Never run `systemctl restart sentinel` to "fix" this without first stopping all running children, or you'll create duplicates.

package/.cairn/memory/auto-memory/knowledge_sentinel_upgrade_requires_file_copy.md

@@ -0,0 +1,20 @@
+---
+name: sentinel_upgrade_requires_file_copy
+description: npm install -g does NOT update the running instance — must copy files from npm package to /home/sentinel/sentinel/code/ and restart
+type: knowledge
+created_at: 2026-04-21T08:02:15.597Z
+updated_at: 2026-04-21T08:02:15.597Z
+---
+
+The running Sentinel instance uses PYTHONPATH=/home/sentinel/sentinel/code/ (not the npm package directory at /usr/lib/node_modules/@misterhuydo/sentinel/python/).
+
+`npm install -g @misterhuydo/sentinel@X.Y.Z` only updates the global npm package. To actually apply code changes to the running instance:
+
+1. Copy changed Python/shell files:
+   sudo cp /usr/lib/node_modules/@misterhuydo/sentinel/python/sentinel/<file>.py /home/sentinel/sentinel/code/sentinel/<file>.py
+   sudo cp /usr/lib/node_modules/@misterhuydo/sentinel/python/scripts/<file>.sh /home/sentinel/sentinel/code/scripts/<file>.sh
+
+2. Restart the instance:
+   sudo -u sentinel bash /home/sentinel/sentinel/sentinel-1881/stop.sh && sleep 2 && sudo -u sentinel bash /home/sentinel/sentinel/sentinel-1881/start.sh
+
+The `sentinel upgrade` CLI command (via Slack Boss) likely does this automatically, but manual deploys via SSH must do both steps.

package/.cairn/memory/auto-memory/preference_no_api_key_for_heavy_coding.md

@@ -0,0 +1,17 @@
+---
+name: no_api_key_for_heavy_coding
+description: Never bill heavy coding tasks against ANTHROPIC_API_KEY — must use Claude Pro OAuth
+type: preference
+created_at: 2026-04-24T11:39:14.740Z
+updated_at: 2026-04-24T11:39:14.740Z
+---
+
+User explicitly stated: "I never want to use API key for heavy coding tasks."
+
+**How to apply:**
+- For Sentinel fix_engine, ask_codebase, repo_task_engine: MUST use Claude Pro (OAuth) via the `claude` CLI's cached login, not `--bare`/API-key
+- API key remains acceptable for the lightweight Boss conversation loop (structured tool-use), but heavy code work is OAuth-only
+- If OAuth fails, do NOT silently fall back to API key — surface the failure and fix the OAuth issue instead
+- Configuration default: `CLAUDE_PRO_FOR_TASKS=true` is the only acceptable mode; do not propose API-key fallback as a "workaround"
+
+**Why:** Claude Pro covers heavy/agentic work via subscription. API credit is finite and gets exhausted (we hit the "Credit balance too low" wall earlier in 2026-04-24). Pro is the durable answer; falling back to API key is a regression in cost discipline.

package/.cairn/memory/auto-memory/project_ageri.md

@@ -0,0 +1,45 @@
+---
+name: Ageri project
+description: Personal AI platform — live on Oracle Ampere server, Slack connected, personal app responding
+type: project
+---
+
+Ageri is a personal AI platform, separate from Sentinel.
+
+**Domain:** ageri.ai (registered on Cloudflare)
+**GitHub:** git@github.com:misterhuydo/Ageri.git (private)
+**Local path:** J:\Projects\Ageri
+**Deploy key:** ~/.ssh/ageri_deploy on Oracle server
+
+**Server:** Oracle Ampere ARM 138.2.17.152, user: `ageri`
+**SSH from local:** `ssh -i /c/Users/huy/.ssh/oracle/devtest-arm-ampere.key ageri@138.2.17.152`
+**Code on server:** ~/ageri/code
+**Venv:** ~/ageri/venv
+**Config dir:** ~/ageri (ageri.properties + private_ageri.properties)
+**Log file:** ~/ageri/ageri.log
+**Start command:** `cd ~/ageri/code && AGERI_CONFIG=~/ageri ~/ageri/venv/bin/python -m ageri.main >> ~/ageri/ageri.log 2>&1 &`
+
+**Current status (2026-03-31):** LIVE — Ageri running on Oracle server, connected to Slack workspace via Socket Mode. Personal app active. DMs to @Ageri in Slack are working.
+
+**Architecture:**
+- Orchestrator → AppRegistry → Apps (personal, research, sentinel stubs)
+- Three-tier SQLite memory (session/working/long_term)
+- Slack Socket Mode interface
+- ageri-sdk: separate PyPI package at sdk/ — install with `pip install -e sdk/`
+- CLI at cli/ — `ageri init`, `ageri slack`, `ageri slack --update`
+
+**CLI (npm):** cli/ — not yet published. Run locally with `node bin/ageri.js`
+- `ageri init` — full setup (venv, pip, Slack prompts, config files)
+- `ageri slack` — prints one-click Slack manifest URL
+
+**Slack app:** messages_tab_enabled=true, interactivity=true, socket_mode=true
+**Scopes:** chat:write, im:*, channels:*, groups:*, reactions:write, users:read
+
+**Key fixes applied:**
+- Ubuntu 22.04 externally-managed Python → venv required
+- ageri-sdk not in requirements → `pip install -e sdk/` needed
+- setuptools.backends not available → changed to setuptools.build_meta
+- anthropic package missing from requirements.txt → added >=0.25
+- Slack messages tab disabled → added app_home.messages_tab_enabled=true to manifest
+
+**Why:** Platform play — Sentinel becomes an app module on top. BYOK + license business model.