@sleepy_panda_srl/portulan 0.2.0 → 0.4.0

package/packs/tools/local-llm/README.md

@@ -0,0 +1,176 @@
+# `tools/local-llm` — Hybrid local-model execution under Portulan's rails
+
+A tool pack for **Hybrid AI Orchestration**, argued in Proposal [`0039`](../../../.portulan/proposals/0039-a-local-model-runs-without-the-host-floor-and-under-a-thinking-cap.md). It lets a frontier orchestrator session offload bounded code transformations, verify-gated red→green retries and a screen of a curated slot for duplicate files to a local OpenAI-compatible runtime (such as **LM Studio** on `http://127.0.0.1:1234/v1` or **Ollama** on `http://127.0.0.1:11434/v1`) with **no API tokens for the local model's own work** and a **zero-floor prompt prefix**. Each call to it is still a parent request, which re-reads the parent's context like any other (fact 1 below).
+
+Composed through a workspace's `packs` array as `tools/local-llm`.
+
+| Contributes | What |
+|---|---|
+| Skill | [`delegate-local`](skills/delegate-local/SKILL.md) — the pass a frontier session follows to hand a bounded edit or digest to the runner; its description names the precondition, a workspace composing this pack and a server answering on loopback |
+| Persona | [`local-worker`](personas/local-worker.md) — a context-firewall worker operating with an omitted host floor (`omitClaudeMd`) and a hard input-byte ceiling |
+| Verify | [`runner-rails`](verify/README.md) — offline verification of the runner's loopback-only guard, input-budget rail, rollback contract, and `dedup`'s refusals (zero network calls) |
+| Gate | `local-llm-off-loopback` (`propose`) — marks leaving loopback (`127.0.0.1` / `localhost` / `[::1]`) as a decision requiring explicit approval |
+
+The skill registers where a plugin manifest declares `skills/`, and this repository's manifests may
+declare only the skills of packs its own workspace composes, as its `.portulan/workspace.json` composes
+this one.
+
+## Why this pack exists
+
+Proposal [`0036`](../../../.portulan/proposals/0036-what-a-host-loads-into-every-context-is-budgeted.md) budgets what a host loads into every fresh context, and Proposal [`0038`](../../../.portulan/proposals/0038-what-a-change-spends-is-measured.md) prices what a change spends across turns (`m_r × (F·T + g·T²/2)`). Together they expose two structural facts about hybrid local+cloud development:
+
+1. **Parent-context turns are quadratic, and a local call is one of them.** Every tool call in a frontier session re-reads the entire session history, the one that runs the runner included, so a local call costs nothing only in the local model's own tokens. A one-span `transform` that replaces one parent Edit saves the few hundred output tokens between writing the instruction and writing the code, and a miss, a call that keeps nothing or a kept edit the parent then touches up, adds a whole request. One run measured in [#583](https://github.com/sleepy-panda-srl/portulan/issues/583) missed 20% of its calls, against a break-even of 5–12% at a 236k-token context and the price ratios the skill's [*When it pays*](skills/delegate-local/SKILL.md#when-it-pays) states, and spent more parent tokens than it saved. A 10-file comment cleanup sent as ten `transform` calls, each its own tool call, is ten parent requests; a loop over the ten inside one Bash call is one, and that batching is not measured.
+2. **Local models choke on the host floor (`F`), not on the task.** A cloud subagent inherits a 44k–85k token prefix (`F`) subsidized by provider prompt caching. On local hardware, pre-filling 85k tokens into a 30B model's KV cache takes tens of seconds before the first token appears, whereas a **surgical prompt of a few hundred tokens** (built using [`cli/symbols.mjs`](../../../cli/symbols.mjs) spans) pre-filled in 0.85–1.9 seconds in proposal 0039's measurement.
+
+## What the runner enforces in code
+
+[`runner/local-llm.mjs`](runner/local-llm.mjs) has **zero npm dependencies** (Node.js `>=20` built-in HTTP, `fetch` and `node:child_process`) and enforces these rails:
+
+1. **Loopback-only by default.** Only `127.0.0.1`, `localhost`, and `[::1]` are permitted unless `--allow-remote` is explicitly passed, and a redirect is refused rather than followed off the machine. A proxy listening on loopback that forwards elsewhere is not visible to it, which is what the `local-llm-off-loopback` gate fragment records.
+2. **Zero-floor input budget (`--max-input-bytes`, default `32768`).** Refuses oversized prompts, and for `dedup` any file over the ceiling, before hitting the local server, directing the caller to scope with [`cli/symbols.mjs`](../../../cli/symbols.mjs) and `--lines` first. A value that is not a whole number is refused rather than read as no limit.
+3. **Explicit thinking request (`--effort low`).** Passes `reasoning_effort: "low"` and `enable_thinking: false` by default. Backend/template support determines their effect; sending these fields is not a hard reasoning-token cap or a guarantee against output-budget exhaustion.
+4. **Verify-gated transforms (`transform --file <path> --verify <cmd>`, `--verify` required).** Applies the local model's edit, runs the verify command, retries with the end of its output on exit `1`, and **restores the original file byte-for-byte** unless the command reaches exit `0`. Any exit other than `0` or `1` — `2`, a missing command, a signal — is no verdict: the file is restored at once and the runner exits `2`. SIGINT, SIGTERM and SIGHUP, whether they arrive during the model call or while the verify command runs, stop the verify, restore the file and exit `2`. **The limits:** this is a rollback, not an atomic write — any other signal that ends the runner (SIGKILL and SIGQUIT among them) or a power loss between the edit and its verdict leaves the edit in place, and since the verify runs in a process group of its own, such a signal leaves the verify running too, even one sent to the runner's whole group. Signal handling is POSIX's: on Windows no SIGTERM arrives, and stopping the verify stops only its shell.
+5. **Role screening (`eval`).** Grades an installed local model against Portulan's own [`cli/comments.mjs`](../../../cli/comments.mjs), [`cli/doctor.mjs`](../../../cli/doctor.mjs), and [`cli/skill-goldens.mjs`](../../../cli/skill-goldens.mjs) predicates across `code`, `codify`, and `handoff` tasks. It takes **one sample per task and has no baseline arm**, so it is a screen, not the A/B run `0038` requires before a role is routed: a model that fails it is not routed, and passing it earns nothing. It needs `cli/` beside `packs/`, as the repository and the npm package carry it, exiting `2` where only `packs/` was installed. The A/B run is [*The A/B record*](#the-ab-record) below.
+
+Default loopback `/chat/completions` calls use a finite 15-minute deadline over the
+connection, headers and complete response body, with a 256-KiB response limit.
+This avoids Node fetch's five-minute header wait for non-streaming thinking models.
+`localhost` is pinned to 127.0.0.1 without DNS; `[::1]` remains available. Redirects
+are refused. Explicit injected fetches, authorized remote calls and embeddings
+retain their existing transport behavior and response capacity. A caller injecting
+its own transport can explicitly use the exported `localModelFetch`; the default
+repair cannot change a transport hidden inside an injection.
+
+A response ending with `finish_reason: length` is incomplete and is rejected before
+a replacement or verifier can run. Workspaces may enforce stronger completion or
+slice limits without changing earned routing or model profiles. The code extractor
+accepts valid whitespace/language fences, preserves inline backticks, and rejects
+an unfinished fence. Provenance: owner Marius, November 5, 2026; local thinking
+generations were interrupted by a client deadline and valid generated fences were
+misparsed. These repairs change transport/completion handling, not model quality.
+
+A receipt prints `reasoning=` as a subset of `out=` and `cached-in=` as a subset
+of `in=`; neither is added to its parent. Counts must be safe nonnegative integers
+and subsets cannot exceed their reported parent. Missing, invalid or unverifiable
+counts print `?`; reported zero stays zero. Transform attempts sum each measurement
+once, with an unknown attempt making that sum unknown. `time=` is summed model
+request time: the runner does not stream, so it cannot see a first token. The ledger
+accepts both historical and extended receipts and continues counting parent totals.
+Update runner and ledger together: older strict receipt readers do not understand
+the added fields.
+
+On `length`, the diagnostic distinguishes a measured all-reasoning response from
+other combined-budget exhaustion. A short input does not cap thinking. Inspect the
+backend's actual thinking controls and the task contract before another request;
+do not silently lower a pinned profile, retry identical work, or accept truncated
+output. An advertised reasoning-budget parameter needs a binding runtime test,
+not just a short successful response. Provenance: the November 6 Q06 investigation
+on [#578](https://github.com/sleepy-panda-srl/portulan/issues/578#issuecomment-6695966000),
+with accounting on [#571](https://github.com/sleepy-panda-srl/portulan/issues/571).
+
+## Screening a slot for duplicates
+
+`dedup --dir <slot-dir> [--threshold 0.80] [--model <id>] [--max-input-bytes <per file>]` embeds every file directly in the directory whose name ends in `.md`, in any letter case, sorted by name, in one `POST /v1/embeddings` request to the loopback endpoint, with `text-embedding-nomic-embed-text-v1.5` (`PORTULAN_LOCAL_EMBED_MODEL`) unless `--model` names another; `--role` picks a chat model and is refused here. It prints each pair whose cosine similarity is at or above the threshold, highest first and ties in file-name order, one line each, `0.829 0036-….md 0038-….md`, and two lines on stderr: `dedup files=8 pairs=1 threshold=0.8`, followed by the shared receipt `OK mode=dedup model=… attempts=1 in=…t out=? time=…ms rate=? reasoning=? cached-in=? spend=$0.00`. The ledger reads the receipt as embedding work at $0.00; output tokens and rate are unreported, never zero.
+
+- **A slot's `README.md` is left out**, in any letter case: it describes the slot rather than being one of its entries, and would rank against every entry it summarizes. Subdirectories, symbolic links and other files are left out too.
+- **The ceiling is per file**, since the model embeds each input on its own, and it measures the file's text: its size on disk before it is read, and its UTF-8 bytes after. A file over it refuses the run with exit `1`, naming every such file with its size, and nothing is truncated. The request as a whole is not capped: a slot of `n` files sends at most `n` times the ceiling of file text, which JSON escaping can enlarge in the request body, to about six times for control characters. Raise `--max-input-bytes` only as far as the embedding model's context holds.
+- **The threshold is a plain decimal from `-1` to `1`**, the range of a cosine; `-1` lists every pair. Anything else is exit `2`, as every malformed argument to the runner is, and never read as the default.
+- **Exit `0` is a run, with pairs or with none**, since a candidate pair is for a human to judge, not a red. Exit `1` is a refusal before any request: an endpoint off loopback, or a file over the ceiling. Exit `2` is no server, a redirect, an HTTP error, a response that is not one finite vector of one length per file, or a directory with fewer than two entries to compare. A zero vector scores `0` against everything rather than `NaN`.
+
+A score is a candidate, not a verdict: the runner neither merges nor edits a file.
+
+## Delegating to it
+
+The pass is the [`delegate-local`](skills/delegate-local/SKILL.md) skill: check the server with
+`status`, scope the input with [`cli/symbols.mjs`](../../../cli/symbols.mjs) and `--lines`, choose the
+supported role by task shape, gate every edit with `--verify`, and read only the receipt back.
+The acquired [record](../../../evals/ab/local-worker.md) supports `code` at low effort; shared `prose`
+is disabled because codify failed all five local turns. `summarize` defaults to that disabled role and
+refuses. An explicit `--model` without `--role` is an operator-selected candidate, not automatic routing.
+Delegate one complete function or a small test group per call, specify one observable change, prefer
+serial calls, and keep output to the selected replacement. On truncated output inspect the diagnostic and task contract; on transport
+failure inspect the transport; on verifier failure inspect code. For reasoning-only
+exhaustion, investigate backend/profile/task behavior before selecting another span.
+Do not lower an operator-pinned
+model/effort or bypass verification. This advice reduces failures, does not guarantee completion.
+
+## The A/B record
+
+The acquired [capture](../../../evals/ab/local-worker.json) and its generated
+[register](../../../evals/ab/local-worker.md) record five turns per task and arm on 2026-11-03:
+code L5/H5, codify L0/H3, handoff L5/H5. The [acquisition note](../../../evals/ab/local-worker-acquisition.md)
+keeps the failed authentication acquisition and the authorized host-only replacement auditable.
+
+The rule fixed before outcomes requires all five local turns to pass and all five host turns to
+complete, with local passes at least host passes. A positive 5/5 tie qualifies; a 0/5 tie does not.
+Both tasks must qualify for the shared prose role. Thus **code is enabled and prose is disabled**.
+Operational defaults and explicit `--role` calls require the measured model at low effort; a model
+override or environment variable cannot silently substitute another. Explicit `--model` without
+`--role`, `eval` and `eval --ab` remain candidate paths and never enable routing by themselves.
+
+`EARNED_ROUTING` embeds the capture's SHA-256, measured model IDs and support bits, so packs-only
+installs need no root `evals/` directory at runtime. The repository's
+[`cli/local-routing.mjs`](../../../cli/local-routing.mjs) regrades the record, reproduces its register,
+and independently derives those bits and models before comparing them with the shipped table.
+It is repository workshop tooling, excluded from npm like the earlier A/B instruments.
+
+- **`eval --ab --k 5 --seed <s> --host-cmd "<command>" --host-model <id> --write` captures it.** Arm L
+  is each task's model as routed (`code` to `--role code`; `codify` and `handoff` to `--role prose`)
+  at `--effort low`, on loopback, with a sampling seed per turn derived from `--seed`. Arm H is the host
+  agent, given the same prompt as one text, as the last argument of the command `--host-cmd` names, in
+  an empty directory it removes afterwards. Both arms are graded by `eval`'s predicates, `k = 5` per
+  (task, arm) cell, the k ruled for `evals/ab/baseline.md`. The record names both models, the host's
+  version, the commit, the machine and the seed, and the sha256 fingerprints of the loaded runner source, the predicates and the
+  prompts it was graded under, as the capture recorded them, since nothing can later verify the first; the register carries `baseline.md`'s limit: `k = 5` per cell supports a
+  recorded rate and nothing else. The machine is the CPU model, the memory, the platform and
+  architecture, and the Node version, and never the hostname.
+- **Arm H consumes host-account usage and its prompts leave the machine**, fifteen host calls, so it crosses the loopback
+  rail the rest of the runner holds and that `local-llm-off-loopback` records. Usage may be charged
+  through an API account or covered by a subscription; this acquisition used Claude Max sign-in with
+  Sonnet 5.5 and a configured total ceiling of $2.25, not an independently measured invoice amount. It runs only where
+  `--host-cmd` names the host command; without it `--ab` refuses with exit `1`, the answer the runner
+  gives any refused endpoint, before anything is contacted. **It runs under the operator's full
+  environment and credentials**, which is how it reaches the host's account and also how anything the
+  operator's configuration loads reaches arm H. A command carrying `--dangerously-skip-permissions` or
+  `--permission-mode bypassPermissions` is refused with exit `2`, by the check `cli/ab-run.mjs` uses.
+  Arm L runs first, at `$0.00`, so a local failure stops the capture before any host call.
+  `--host-model` is recorded as declared: the runner does not observe the host's model.
+- **The command is split on whitespace, with no shell**, so no argument of it can hold a space, and a
+  quoted one arrives in pieces. Its version is read by running the whole command with `--version`
+  appended, so a wrapper such as `env` reports the host it runs rather than itself; a host that read
+  `--version` as prompt text would answer it as a turn, and the record would show that answer as its
+  version.
+- **Every turn is journalled before the next starts**, under the repository's git directory
+  (`git rev-parse --git-path portulan/local-worker-journal`), where no commit and no `git status` sees
+  it. SIGINT, SIGTERM or SIGHUP stops the running host turn, removes its directory, keeps the journal
+  and exits `2`; the same command run again resumes, re-running only turns the journal lacks, and
+  refuses with exit `2` a journal taken under other conditions (commit, tree, machine, seed, models,
+  endpoint, host command, version and model, or any fingerprint), since its arm-H turns were paid
+  for and removing it is the operator's act. A host that writes more than 16 MiB stops the capture with
+  exit `2` and the turn is re-run on resume. **The limit:** SIGKILL, or a power loss, leaves the running
+  host turn's directory behind and, since the host runs in a process group of its own, the host still
+  running and billing a turn the journal will not hold; the journal still resumes, and that turn runs again. The journal is removed once the
+  record is written, and a capture is refused where `evals/ab/local-worker.json` exists: a record is
+  captured once, and removing it is the operator's deliberate act. The final record is created
+  exclusively too: if another record appears during capture, it and its register are preserved,
+  exit `2` leaves the current journal for recovery. This prevents overwrites; it is not a lock on
+  concurrent journal writers or a crash-atomic publication. Different loaded runner bytes refuse
+  journal resume even at the same commit and with both trees dirty.
+- **`eval --verify` checks it without running a model**: the record's shape, its matrix total over `k`,
+  every arm-L seed against `--seed`, every turn's model against the routing, **every completed turn
+  regraded** by the predicates as they ship, the prompts' fingerprint, the cells re-folded from the
+  turns, and the register byte-compared with a fresh render. A verdict that regrades differently is
+  exit `1`, and the finding says whether the predicates moved since capture or the record no longer
+  matches its own output: either way the figures would not hold under the screen that ships. A
+  predicate file that changed while every verdict regrades the same is no finding. A missing record is
+  exit `2`, and so is a payload with no `cli/` beside `packs/`, since nothing there can regrade. A
+  record carrying the synthetic fixture's `fixture` field is exit `1` from `--verify` and `--write`
+  alike: a synthetic fixture is not a capture.
+  `eval --write` re-renders the register from the record, refusing one `--verify` would red.
+
+The `ab-run` recipe runs the earlier baseline verifier, exact no-model `eval --verify`, and the
+shipped-routing verifier on every pull request. A red or could-not-run result survives later greens;
+exit `2` takes precedence over exit `1`. No model is rerun in CI. `k = 5` supports these recorded rates,
+not general reliability, arbitrary task quality or independently observed host model identity.
+
+Not for multi-file architectural planning, ambiguous requirements (`clarify`), or pre-merge verdicts (`reviewer`).

package/packs/tools/local-llm/pack.json

@@ -0,0 +1,32 @@
+{
+  "portulan": {
+    "pack": "1.0",
+    "version": "0.2.0"
+  },
+  "name": "local-llm",
+  "category": "tools",
+  "summary": "Zero-dependency local LLM runner (LM Studio / Ollama) with a verify-gated red-to-green file transform loop, a zero-floor input budget rail, role evals, an embedding screen for duplicate files in a curated slot, and loopback-only enforcement.",
+  "doc": "README.md",
+  "contributes": {
+    "skills": ["skills/"],
+    "personas": ["personas/local-worker.md"],
+    "verify": [
+      {
+        "id": "runner-rails",
+        "run": "node ${PACK_ROOT}/runner/local-llm.mjs --verify-offline",
+        "requires": ["node"],
+        "doc": "verify/README.md"
+      }
+    ],
+    "gates": [
+      {
+        "id": "local-llm-off-loopback",
+        "tier": "propose",
+        "action": {
+          "none": "No honest shell matcher. Whether a local-model invocation stays on the machine depends on the resolved endpoint (`--base-url`, `PORTULAN_LOCAL_LLM_URL`, or a proxy config), not on the binary name. `runner/local-llm.mjs` refuses non-loopback hosts in code unless `--allow-remote` is passed; this fragment records the policy boundary that a host permission rule cannot see."
+        },
+        "reason": "Routing a task to a local model is a spend and confidentiality boundary only while the endpoint resolves to loopback (`127.0.0.1`, `localhost`, `[::1]`). Pointing a local worker at a remote endpoint sends workspace context off-machine under a local tool's name; propose the remote endpoint explicitly before running."
+      }
+    ]
+  }
+}

package/packs/tools/local-llm/personas/local-worker.md

@@ -0,0 +1,44 @@
+---
+name: local-worker
+description: Executes bounded file transformations, test-fixture drafts, handoff compressions, and research digests on a local OpenAI-compatible model (LM Studio / Ollama) with an omitted host floor and a verify-gated rollback loop. Delegate surgical, single-file or read-parallel summarization work to it; never ask it to own multi-file architecture or merge decisions.
+tools: [read, search, edit-code, run-verify]
+effort: low
+omitClaudeMd: true
+---
+
+# Persona — Local Worker
+
+> A context-firewall worker whose whole value is **what it does not load and what it does not spend**.
+> A frontier session re-reads its entire prefix (`F + g·T`) on every tool call; a cloud subagent inherits
+> the host floor (`F ≈ 44k–85k` tokens). The local worker runs against a loopback model (`127.0.0.1`) with
+> the host instruction floor omitted (`omitClaudeMd: true`), a hard input-byte ceiling, and a verify
+> recipe that gates its output before anything returns to the parent loop.
+
+## Charter
+
+- **For:** executing one bounded, verify-gated code transformation (`transform --file <path> --verify <cmd>`),
+  drafting unit tests or fixtures against an explicit signature, compressing a session log into a
+  bounded handoff, or distilling a file span into a compact digest for the parent session.
+- **Not for:** cross-cutting architectural design, multi-branch coordination, or final pre-merge verdicts
+  (`reviewer` / `supervisor`). It does not accept rules (`librarian`) and it does not take outward actions.
+
+## Autonomy reach
+
+Acts in **Auto** to read scoped inputs, run local loopback inference, apply edits in place, kept only on a
+green verify and rolled back otherwise, and run the workspace's verify recipe. Its output into any curated
+slot (memory, proposals, handoffs) is a draft into a **Propose** or **Gated** decision owned by the
+parent session or human. **Prohibited is not a reach** and does not appear here as a permission: it is
+the tier no role may act in.
+
+## Read / write posture
+
+Reads only the scoped files or symbol spans passed to it — never the full repository tree or the
+always-loaded host floor. Writes are isolated to a single target file under a rollback guard: unless the
+verify command exits `0` within the retry cap, the original bytes are restored before the worker exits,
+short of the signals and power loss the pack's README names.
+
+## Memory scope
+
+Its own `local-worker` scope in the adopting workspace's memory layer: which task shapes, prompt budgets,
+and local model identifiers pass the workspace's verify recipes on the first try versus which require
+frontier escalation. Kept separate from `implementer` and `reviewer` memory.