nubos-pilot 0.2.2 → 0.4.0

package/workflows/scan-codebase.md

@@ -0,0 +1,155 @@
+---
+command: np:scan-codebase
+description: Initial deep codebase scan — inventories files, groups them into coherent modules, writes skill-style docs under .nubos-pilot/codebase/ that dev-agents must read before touching code.
+---
+
+# np:scan-codebase
+
+Perform the initial codebase scan for a project. Produces a full set of
+module docs under `.nubos-pilot/codebase/modules/`, an `INDEX.md` pointer
+file, a `.hashes.json` manifest for staleness tracking, and a
+`.doc-index.json` mapping of doc → source paths.
+
+Dev-agents (executor, code-fixer, planner, researcher, documenter, custom)
+MUST read `INDEX.md` + any relevant module docs before modifying code, and
+MUST call `np:update-docs` after changes.
+
+## Philosophy
+
+<philosophy>
+The codebase doc layer is the shared memory of every agent working in the
+project. If it is empty or stale, each agent re-derives context from raw
+source, burns tokens, and can drift. This workflow pays the one-time cost
+of a deep inventory, hybrid-parser-plus-documenter-agent pass so that every
+subsequent agent starts informed.
+
+Nothing in this workflow is Claude-specific. The subcommand returns
+module-facts JSON, the documenter agent is a prompt that any orchestrator
+(Claude Code, OpenAI, Codex) can dispatch. Stay runtime-agnostic.
+</philosophy>
+
+## Scope Guardrail
+
+<scope_guardrail>
+This workflow ONLY writes inside `.nubos-pilot/codebase/`. It NEVER:
+
+- modifies application source code
+- touches PROJECT.md, REQUIREMENTS.md, or roadmap.yaml
+- runs git commands beyond read-only `git log` via the scanner
+- spawns network calls outside the documenter agent's own tool surface
+
+Refuse and report if any of these boundaries would be crossed.
+</scope_guardrail>
+
+## Downstream Awareness
+
+<downstream_awareness>
+- `np:update-docs` reads `.doc-index.json` + `.hashes.json` to diff.
+- Every `np:execute-*` workflow's post-step calls `np:update-docs`.
+- Every np-agent is instructed to read `.nubos-pilot/codebase/INDEX.md` and
+  relevant module docs before editing source.
+
+If this workflow produces malformed INDEX.md or unreadable frontmatter,
+every downstream agent degrades silently. Validate the written files at the
+end.
+</downstream_awareness>
+
+## Single-Call Init
+
+Scan, group, write manifest + stubs, emit module-facts in one call:
+
+```bash
+PLAN=$(node np-tools.cjs scan-codebase --project-name "$PROJECT_NAME")
+if [[ "$PLAN" == @file:* ]]; then PLAN=$(cat "${PLAN#@file:}"); fi
+```
+
+`--project-name` is optional; when provided it goes into `INDEX.md`. Other
+flags: `--batch-size N` (default 500), `--max-files N`.
+
+Parse: `mode`, `stats`, `modules[]` (each with `id`, `directory`,
+`primary_language`, `file_count`, `facts`), `index_path`, `manifest_path`.
+
+## Process
+
+### Step 1: Confirm scan scope with the user
+
+Show a summary before iterating modules:
+
+```
+Scan complete.
+- Files walked: <stats.file_count>
+- Hashed: <stats.hashed_count>
+- Languages: <top 3 by count>
+- Modules discovered: <modules.length>
+- Manifests captured: <manifests.length> (package.json, …)
+
+Generate doc prose for all <N> modules? (yes / pick subset)
+```
+
+Large codebases may warrant a subset pass. Honor the user's choice — do
+not blast through 500 agent calls without consent.
+
+### Step 2: For each selected module, dispatch the documenter
+
+Per module, build the prompt from `facts` and dispatch the documenter
+subagent. The subagent is defined in `agents/np-codebase-documenter.md` and
+is runtime-agnostic — pick whichever dispatch mechanism your host supports.
+
+```bash
+PROSE_FILE=$(mktemp -t np-prose-XXXXXX.json)
+# Host dispatches agent with buildDocumenterPrompt(facts) and writes JSON
+# to $PROSE_FILE. Validate JSON before proceeding.
+python -c 'import json,sys; json.load(open(sys.argv[1]))' "$PROSE_FILE"
+```
+
+Batch pacing: the user opted into batches during Step 1. Between batches,
+show a progress line (`[37/120 modules documented]`) and give the user a
+chance to pause with Ctrl-C. Never eat interrupt signals.
+
+### Step 3: Apply prose per module
+
+```bash
+node np-tools.cjs scan-codebase --apply-prose \
+  --module "$MODULE_ID" \
+  --prose-file "$PROSE_FILE"
+```
+
+The subcommand re-reads the module's source hashes (handles files that
+changed between scan and apply), merges the prose, and atomically writes
+`modules/<id>.md`.
+
+### Step 4: Validate written artifacts
+
+Before declaring success:
+
+```bash
+test -f .nubos-pilot/codebase/INDEX.md
+test -f .nubos-pilot/codebase/.hashes.json
+test -f .nubos-pilot/codebase/.doc-index.json
+ls .nubos-pilot/codebase/modules/ | wc -l
+```
+
+Warn if a module doc is still `_TBD_` — user may have opted out mid-run.
+
+## Output
+
+```
+np:scan-codebase complete.
+
+Indexed: .nubos-pilot/codebase/INDEX.md
+Modules: <N> (documented: <M>, stubs only: <N-M>)
+Manifest: .nubos-pilot/codebase/.hashes.json
+
+Dev-agents will now read INDEX.md + relevant module docs before editing.
+Next change → np:update-docs runs automatically from np:execute-* workflows.
+```
+
+## Errors
+
+| Code | Trigger | User action |
+|------|---------|-------------|
+| `scan-codebase-not-initialized` | `.nubos-pilot/` missing | Run `np:new-project` first |
+| `scan-codebase-missing-module` | `--apply-prose` without `--module` | Supply flag |
+| `scan-codebase-missing-prose` | `--apply-prose` without `--prose-file` | Supply flag |
+| `scan-codebase-module-not-found` | id does not exist in current scan | Re-list via default mode |
+| `scan-codebase-prose-unreadable` | prose JSON unreadable or invalid | Fix JSON; re-dispatch agent |

package/workflows/update-docs.md

@@ -0,0 +1,132 @@
+---
+command: np:update-docs
+description: Incremental codebase-doc refresh — diffs current source against .hashes.json, refreshes only affected module docs, and updates the manifest.
+---
+
+# np:update-docs
+
+Run after any code change to keep `.nubos-pilot/codebase/` in sync with the
+source. Detects added / changed / removed files, maps them to affected
+modules, dispatches the documenter agent per stale module, and writes back
+updated docs + manifest.
+
+Dev-agents must invoke this after writing/editing source. `np:execute-*`
+workflows call it automatically. Humans can call it manually.
+
+## Philosophy
+
+<philosophy>
+Docs that lag the code are worse than no docs — agents act on stale facts.
+`np:update-docs` is the backpressure that keeps the shared memory fresh.
+It is cheap when little has changed (diff is empty → no-op) and incremental
+when much has changed (only stale modules re-documented).
+
+Runtime-agnostic: the subcommand returns stale-module facts + new module
+stubs; the host dispatches the documenter agent on each; the workflow
+calls `--apply-prose` per module.
+</philosophy>
+
+## Scope Guardrail
+
+<scope_guardrail>
+This workflow ONLY writes inside `.nubos-pilot/codebase/`. It NEVER:
+
+- rewrites source code
+- renames or removes doc files for modules that still exist
+- regenerates prose for modules that did not change
+</scope_guardrail>
+
+## Downstream Awareness
+
+<downstream_awareness>
+- Called from every `np:execute-*` workflow's post-step.
+- Reads `.doc-index.json` to map touched paths → stale docs.
+- Writes `.hashes.json` on completion so the next run has a fresh baseline.
+
+If `.doc-index.json` is missing, fall back to a full rescan via
+`np:scan-codebase` instead of guessing.
+</downstream_awareness>
+
+## Single-Call Init
+
+```bash
+PLAN=$(node np-tools.cjs update-docs)
+if [[ "$PLAN" == @file:* ]]; then PLAN=$(cat "${PLAN#@file:}"); fi
+```
+
+Parse: `mode`, `diff_summary` (added/removed/changed/unchanged counts),
+`stale_modules[]`, `added_modules[]`, `removed_modules[]`.
+
+If all counts are zero, print "No doc updates needed." and exit.
+
+## Process
+
+### Step 1: Report diff to the user
+
+```
+Diff vs last manifest:
+- Added files: <n>
+- Changed files: <n>
+- Removed files: <n>
+
+Stale modules: <n>   — prose will be refreshed
+Added modules: <n>   — stubs written; prose pending
+Removed modules: <n> — marked for user review
+```
+
+For non-zero stale/added, ask: "Refresh prose now? (yes / no / pick)".
+If "no", exit with stubs-only state — user can run the workflow later.
+
+### Step 2: Dispatch documenter per stale + added module
+
+For each module in `stale_modules ++ added_modules`:
+
+```bash
+# Build prompt from facts, dispatch agent, capture JSON to $PROSE_FILE
+node np-tools.cjs update-docs --apply-prose \
+  --module "$MODULE_ID" \
+  --prose-file "$PROSE_FILE"
+```
+
+### Step 3: Handle removed modules
+
+For `removed_modules`, prompt the user:
+
+```
+Module <id> has no source files anymore. Delete its doc?
+  (delete / archive to modules/_archived/ / keep)
+```
+
+Default: archive. The workflow MUST NOT silently delete docs.
+
+### Step 4: Verify manifest is fresh
+
+```bash
+stat -f "%m" .nubos-pilot/codebase/.hashes.json
+```
+
+The `generated_at` in the JSON should be within the last few seconds.
+
+## Output
+
+```
+np:update-docs complete.
+
+Diff: +<added> ~<changed> -<removed>
+Refreshed: <n> modules
+Added:     <n> modules
+Archived:  <n> modules
+
+Manifest: .nubos-pilot/codebase/.hashes.json
+```
+
+## Errors
+
+| Code | Trigger | User action |
+|------|---------|-------------|
+| `update-docs-not-initialized` | `.nubos-pilot/` missing | Run `np:new-project` |
+| `update-docs-missing-module` | `--apply-prose` without `--module` | Supply flag |
+| `update-docs-missing-prose` | `--apply-prose` without `--prose-file` | Supply flag |
+| `update-docs-module-not-found` | id does not exist in current scan | Rescan |
+| `update-docs-prose-unreadable` | prose JSON unreadable or invalid | Fix JSON; re-dispatch agent |
+| `manifest-schema-mismatch` | old `.hashes.json` from prior schema | Delete manifest; run `np:scan-codebase` |