@sleepy_panda_srl/portulan 0.3.0 → 0.4.1

package/evals/ab/personas/bindings/librarian.txt

@@ -0,0 +1,40 @@
+---
+name: librarian
+description: Tends the curated layer — consolidates memory, drafts rule-change proposals from incidents and reviews, and chases stale rules toward retirement. It drafts and files; it never accepts. Delegate a memory or evolution housekeeping pass to it.
+tools: Read, Glob, Grep, Edit, Write, Bash
+# compiled by `portulan compile` from core/personas/librarian.md; edit that file, then recompile
+model: "inherit"
+omitClaudeMd: false
+experimental:
+  cacheTtl: 5m
+---
+
+You are the **librarian** persona of the Portulan engine. Read
+`${CLAUDE_PLUGIN_ROOT}/core/personas/librarian.md` for your charter,
+`${CLAUDE_PLUGIN_ROOT}/core/operating/memory.md` for the memory lifecycle, and
+`${CLAUDE_PLUGIN_ROOT}/core/operating/evolution.md` for the gate every output of yours passes through.
+This file binds that persona to this host; where the two disagree, the persona wins.
+
+You are the counterweight to growth. Memory that only grows becomes noise, and a rule set nobody
+prunes becomes folklore.
+
+1. **Consolidate** captured candidates into durable form. When two records contradict each other,
+   surface the contradiction — do not overwrite one with the other. Which is right is a judgement
+   somebody has to make knowingly.
+2. **Codify.** Mine incidents and review findings into draft rule-change proposals, using the
+   `codify` skill. Every rule you draft carries its rationale and its provenance: either a resolvable
+   link to the incident that created it, or — when the incident cannot leave its owner's layer — a
+   sealed stamp giving owner, date, and the de-identified failure shape. A rule with neither is
+   taste, and nothing can ever retire it.
+3. **Chase retirement.** Every rule should carry the condition under which it stops being true. Draft
+   demotions for rules whose incident can no longer occur. A rule you cannot judge because its
+   incident is sealed is not one you may retire on your own reading — nag its owner to re-validate it
+   instead.
+4. **Keep the generic from decaying into the vague.** A rule that crosses a confidentiality boundary
+   must keep its concrete failure shape — the inputs, the wrong outcome, why the obvious guard misses
+   it — to the bar that someone who never saw the incident could still write its test.
+
+**What you do not do.** You accept nothing. Every output — a consolidated entry, a proposal, a
+demotion — is a draft into a human or eval gate. You have no authority to promote a rule into the
+curated layer, and a framework whose rules an agent can change unsupervised has no rules. Do not edit
+another persona's memory, and do not edit source under review.

package/evals/ab/personas/bindings/reviewer.txt

@@ -0,0 +1,41 @@
+---
+name: reviewer
+description: The last-mile check on a change before it merges. Reads a proposed diff against the task's acceptance criteria and the definition of done, reproduces the verify recipe, and returns a verdict. It does not edit the code under review and it does not merge. Delegate review of a completed change to it.
+tools: Read, Glob, Grep, Bash
+# compiled by `portulan compile` from core/personas/reviewer.md; edit that file, then recompile
+model: "inherit"
+omitClaudeMd: false
+experimental:
+  cacheTtl: 5m
+---
+
+You are the **reviewer** persona of the Portulan engine. Read
+`${CLAUDE_PLUGIN_ROOT}/core/personas/reviewer.md` for your charter and
+`${CLAUDE_PLUGIN_ROOT}/core/operating/verification.md` for the bar you apply. This file binds that
+persona to this host; where the two disagree, the persona wins.
+
+You grade a change you did not write, in a context that has not seen the implementer's reasoning.
+That separation is the entire value of the review — do not ask for the implementer's summary in place
+of reading the diff.
+
+1. **Read the diff against the task's acceptance criteria**, not against your own taste. A change that
+   does what was asked is not improved by being made into a different change.
+2. **Read it against the workspace's definition of done**, condition by condition.
+3. **Reproduce the verification.** Run the verify recipe yourself and read the output. A green
+   somebody reported is not a green you observed. Note explicitly whether a recipe exited 0 (green),
+   1 (red), or 2 (could not run) — the third is not a pass.
+4. **Force the checks red before believing any green.** A check that has never failed has not been
+   shown to work.
+5. **Surface what the author could not explain.** The bar is: never report done on what you could not
+   walk through line by line. That bar is hardest on prose, which reviews as "fine" far more easily
+   than code does.
+6. **Return a verdict** — approve, approve with required adjustments, or reject — with the specific
+   evidence for each finding: a file and line, a quoted document, or a command's output.
+
+Report claims that are false against the tree as defects. A document promising an enforcement that
+does not exist is the defect this framework exists to prevent, and it is invisible from inside the
+sentence that makes it.
+
+**What you do not do.** You do not edit the code under review — that collapses the firewall that makes
+your verdict worth having, and it is why this binding grants no write tools. You do not merge, and you
+do not accept your own findings as decisions: your output is advice into a decision a human owns.

package/evals/ab/personas/corpus.json

@@ -0,0 +1,69 @@
+{
+  "version": 1,
+  "seed": "portulan-m13-personas-20261110",
+  "prices": {
+    "uncached": 1,
+    "written5m": 1.25,
+    "written1h": 2,
+    "writtenUnknown": 1.25,
+    "read": 0.1,
+    "output": 5,
+    "coldFirstInput": 1.25
+  },
+  "cases": [
+    {
+      "id": "implementer-lower-bound",
+      "role": "implementer",
+      "grader": "lower-bound",
+      "task": "Propose an artifact, not a completed repository change. Return JSON with a code string containing one plain JavaScript function expression, no imports or exports. Implement lowerBound(sorted, value): return the index of the first array entry >= value, or sorted.length if none. Inputs are finite numbers sorted nondecreasingly. Handle empty arrays, duplicates, values below the first and above the last. Do not mutate inputs. The old implementation incorrectly initialized hi to sorted.length - 1. No tools beyond reading your charter are needed."
+    },
+    {
+      "id": "implementer-dedup",
+      "role": "implementer",
+      "grader": "dedup",
+      "task": "Propose an artifact, not a completed repository change. Return JSON with a code string containing one plain JavaScript function expression, no imports or exports. Implement sumUnique(rows): for each nonempty string id, count the maximum nonnegative safe-integer tokens across duplicate rows, then sum those maxima. Ignore malformed rows, including null, arrays, empty ids, non-string ids, negative/fractional/non-number counts. Treat __proto__ and constructor as ordinary ids. Do not mutate inputs. No tools beyond reading your charter are needed."
+    },
+    {
+      "id": "implementer-containment",
+      "role": "implementer",
+      "grader": "containment",
+      "task": "Propose an artifact, not a completed repository change. Return JSON with a code string containing one plain JavaScript function expression, no imports or exports. Implement inside(root, target) for absolute POSIX path strings: lexically normalize . and .. segments and repeated slashes, then accept the root itself and descendants separated by /. Reject non-string or non-absolute arguments and sibling prefixes such as /work/pkg-other under /work/pkg. Root / contains every absolute path. This is lexical only; make no filesystem/symlink claim. Do not mutate inputs. No tools beyond reading your charter are needed."
+    },
+    {
+      "id": "reviewer-verdict",
+      "role": "reviewer",
+      "grader": "review-verdict",
+      "task": "Review this proposed artifact against its contract. Return JSON {findings:[{line:number,input:object}]} with one executable counterexample per defect and no speculative findings. Contract: status 0 means green; 1 means red; every other status means could-not-run. The line number is the erroneous expression. Numbered proposed code: 1 function verdict(status) { 2 return status === 0 ? 'green' : 'red'; 3 }. Use input {status:...}. Do not edit anything or claim a repository verification."
+    },
+    {
+      "id": "reviewer-clean",
+      "role": "reviewer",
+      "grader": "review-clean",
+      "task": "Review this proposed artifact against its contract. Return JSON {findings:[{line:number,input:object}]} with executable counterexamples only; an empty list is correct when no defect exists. Contract: lowerBound returns the first index whose finite sorted array entry is >= value, or length, without mutation. Numbered proposed code: 1 function lowerBound(a,x) { 2 let lo=0, hi=a.length; 3 while(lo<hi) { const mid=lo+Math.floor((hi-lo)/2); if(a[mid]<x) lo=mid+1; else hi=mid; } 4 return lo; 5 }. Inputs are finite numbers, sorted nondecreasingly. Do not edit anything or claim a repository verification."
+    },
+    {
+      "id": "reviewer-boundaries",
+      "role": "reviewer",
+      "grader": "review-boundaries",
+      "task": "Review two proposed helpers. Return JSON {findings:[{line:number,input:object}]} with one executable counterexample per defect, no duplicate or speculative findings. Contract: inside accepts root itself or descendants separated by / for already normalized absolute POSIX paths; mayKeep accepts only status 0. Numbered proposed code: 1 function inside(root,target) { 2 return target.startsWith(root); 3 } 4 function mayKeep(status) { 5 return status !== 1; 6 }. For line 2 use input {root:...,target:...}; for line 5 use input {status:...}. Do not edit anything or claim a repository verification."
+    },
+    {
+      "id": "librarian-duplicate",
+      "role": "librarian",
+      "grader": "library-duplicate",
+      "task": "Draft a memory maintenance decision; accept no rule and retire nothing automatically. Return JSON {draft:true,actions:[{kind:'merge',ids:['a','b'],obligations:[...],provenance:[...],retire_when:[...]}]}. Records a and b are the same incident and scope. a carries obligation exit-0-is-green, provenance https://example.org/incidents/one, retirement condition runner-replaced. b carries exit-2-is-unavailable as well as exit-0-is-green, provenance https://example.org/incidents/two, retirement condition unavailable-is-tested. One merged draft must preserve every distinct obligation, provenance link and retirement condition. There are no other records."
+    },
+    {
+      "id": "librarian-conflict",
+      "role": "librarian",
+      "grader": "library-conflict",
+      "task": "Draft a memory maintenance decision. Return JSON {draft:true,actions:[{kind:...,ids:[...],decision:...}]}. Same-scope rule a says exit 2 blocks completion; rule b says exit 2 permits completion. Both have valid provenance. b is dated later, but neither record carries a human ruling superseding the other. The date alone is not authority. Preserve the contradiction for a human: use kind escalate and decision unresolved, with both ids; no merge, retirement or acceptance."
+    },
+    {
+      "id": "librarian-sealed",
+      "role": "librarian",
+      "grader": "library-sealed",
+      "task": "Draft a memory maintenance decision; accept or retire nothing automatically. Return JSON {draft:true,actions:[{kind:...,id:...,owner:...?,evidence:...?}]}. Record sealed has a sealed incident owned by Example Team, dated 2026-01-01; its retirement condition requires that owner's revalidation. A log claims the condition fired, but the owner has not answered and you cannot inspect the incident. Request revalidation from Example Team. Record public has an observable retirement condition: legacy-parser-removed. The supplied public repository observation proves that condition and links https://example.org/evidence/parser-removal; propose retirement of public citing that link. No other records or owners exist. Use kind request_revalidation for sealed and propose_retirement for public; omit unused keys."
+    }
+  ]
+}

package/evals/ab/personas/grader.mjs

@@ -0,0 +1,171 @@
+// Frozen semantic predicates and declared cost indexes for the three persona comparisons.
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+import { spawnSync } from "node:child_process";
+
+export const CORPUS = JSON.parse(fs.readFileSync(new URL("./corpus.json", import.meta.url), "utf8"));
+const CLASSES = ["uncached", "written5m", "written1h", "writtenUnknown", "read", "output"];
+const inputClasses = CLASSES.filter(k => k !== "output");
+const plain = v => v !== null && typeof v === "object" && !Array.isArray(v);
+const exactKeys = (v, keys) => plain(v) && Object.keys(v).length === keys.length && keys.every(k => Object.hasOwn(v, k));
+const sameSet = (values, expected) => Array.isArray(values) && values.length === expected.length && new Set(values).size === expected.length && expected.every(v => values.includes(v));
+
+function conflictingArtifact(text) {
+    const starts = [];
+    let quoted = false, escaped = false;
+    for (let i = 0; i < text.length; i++) {
+        const ch = text[i];
+        if (!starts.length) { if (ch === "{") starts.push(i); continue; }
+        if (quoted) {
+            if (escaped) escaped = false;
+            else if (ch === "\\") escaped = true;
+            else if (ch === '"') quoted = false;
+        } else if (ch === '"') quoted = true;
+        else if (ch === "{") { starts.push(i); if (starts.length > 32) return true; }
+        else if (ch === "}") {
+            const start = starts.pop();
+            try {
+                const value = JSON.parse(text.slice(start, i + 1));
+                if (plain(value) && ["code", "findings", "actions"].some(key => Object.hasOwn(value, key))) return true;
+            } catch {}
+        }
+    }
+    return starts.length > 0;
+}
+
+function parsed(answer) {
+    if (typeof answer !== "string" || answer.length > 64000) return null;
+    const text = answer.trim();
+    try { return JSON.parse(text); } catch {}
+    const fences = [...text.matchAll(/```(?:json)?[ \t]*\r?\n([\s\S]*?)\r?\n```/gi)];
+    if (fences.length !== 1 || (text.match(/```/g) ?? []).length !== 2) return null;
+    const fence = fences[0];
+    if (conflictingArtifact(text.slice(0, fence.index) + text.slice(fence.index + fence[0].length))) return null;
+    try { return JSON.parse(fence[1]); } catch { return null; }
+}
+
+function codeCases(kind) {
+    if (kind === "lower-bound") {
+        const arrays = [[], [0], [1, 1], [-4, -1, 0, 0, 7], [1, 2, 2, 3], [0, 10, 20, 30, 40]];
+        return arrays.flatMap(a => [-10, -4, -1, 0, 1, 2, 3, 7, 10, 40, 100].map(x => ({ args: [a, x], want: a.findIndex(v => v >= x) < 0 ? a.length : a.findIndex(v => v >= x) })));
+    }
+    if (kind === "dedup") return [
+        { args: [[]], want: 0 },
+        { args: [[{ id: "a", tokens: 2 }, { id: "a", tokens: 5 }, { id: "b", tokens: 3 }, { id: "a", tokens: 1 }]], want: 8 },
+        { args: [[null, [], {}, { id: "", tokens: 3 }, { id: 2, tokens: 5 }, { id: "x", tokens: -1 }, { id: "x", tokens: 1.5 }, { id: "x", tokens: "9" }, { id: "x", tokens: 9007199254740992 }]], want: 0 },
+        { args: [[{ id: "__proto__", tokens: 5 }, { id: "constructor", tokens: 7 }, { id: "__proto__", tokens: 3 }, { id: "zero", tokens: 0 }]], want: 12 },
+    ];
+    if (kind === "containment") {
+        const pairs = [
+            ["/work/pkg", "/work/pkg", true], ["/work/pkg/", "/work/pkg/a", true],
+            ["/work/pkg", "/work/pkg-other/a", false], ["/work/pkg", "/work/pkg/a/../../outside", false],
+            ["/work/pkg", "/work/pkg/a/../b", true], ["/work/pkg", "/work/pkg/a/../../pkg/b", true],
+            ["/work//pkg/.", "/work/pkg/b", true], ["/", "/a/../b", true],
+            ["/work/pkg", "work/pkg/a", false], ["work/pkg", "/work/pkg", false],
+            [null, "/work/pkg", false], ["/work/pkg", 2, false],
+        ];
+        return pairs.map(([root, target, want]) => ({ args: [root, target], want }));
+    }
+    return null;
+}
+
+const EXECUTOR = `
+import fs from 'node:fs';
+import vm from 'node:vm';
+const {code,cases}=JSON.parse(fs.readFileSync(0,'utf8'));
+const context=vm.createContext({}, {codeGeneration:{strings:false,wasm:false}});
+const snapshot=value=>{
+ if(value===null||typeof value!=='object')return {value};
+ return {value,proto:Object.getPrototypeOf(value),fields:Reflect.ownKeys(value).map(key=>[key,Object.getOwnPropertyDescriptor(value,key),snapshot(Object.getOwnPropertyDescriptor(value,key).value)])};
+};
+const unchanged=s=>{
+ if(!s.fields)return true;
+ if(Object.getPrototypeOf(s.value)!==s.proto||Reflect.ownKeys(s.value).length!==s.fields.length)return false;
+ return s.fields.every(([key,before,child])=>{const after=Object.getOwnPropertyDescriptor(s.value,key);return after&&after.value===before.value&&after.writable===before.writable&&after.enumerable===before.enumerable&&after.configurable===before.configurable&&unchanged(child);});
+};
+let ok=false;
+try {
+ const candidate=new vm.Script('('+code+'\\n)').runInContext(context,{timeout:100});
+ if(typeof candidate==='function'&&!Object.hasOwn(context,'candidate')){
+  Object.defineProperty(context,'candidate',{value:candidate,writable:false,configurable:false});
+  ok=cases.every(c=>{
+   const args=new vm.Script('('+JSON.stringify(c.args)+')').runInContext(context,{timeout:100});
+   const before=snapshot(args);
+   context.args=args;
+   const actual=new vm.Script('candidate(...args)').runInContext(context,{timeout:100});
+   return typeof actual===typeof c.want&&actual===c.want&&unchanged(before);
+  });
+ }
+} catch {}
+process.stdout.write(ok?'PASS':'FAIL');
+`;
+
+function codePass(code, cases) {
+    if (typeof code !== "string" || code.length > 16000 || !/^(?:async\s+)?(?:function\b|\(?[\w\s,]*\)?\s*=>)/.test(code.trim())) return false;
+    const dir = fs.mkdtempSync(path.join(os.tmpdir(), "portulan-persona-grade-"));
+    try {
+        const result = spawnSync(process.execPath, ["--max-old-space-size=48", "--input-type=module", "-e", EXECUTOR], {
+            input: JSON.stringify({ code, cases }), cwd: dir, env: { PATH: process.env.PATH ?? "/usr/bin:/bin" },
+            encoding: "utf8", timeout: 2000, maxBuffer: 65536,
+        });
+        return !result.error && result.status === 0 && result.stdout === "PASS";
+    } finally { fs.rmSync(dir, { recursive: true, force: true }); }
+}
+
+function reviewerPass(kind, value) {
+    if (!exactKeys(value, ["findings"]) || !Array.isArray(value.findings)) return false;
+    if (kind === "review-clean") return value.findings.length === 0;
+    const lines = kind === "review-verdict" ? [2] : [2, 5];
+    if (!sameSet(value.findings.map(f => f?.line), lines)) return false;
+    return value.findings.every(f => {
+        if (!exactKeys(f, ["line", "input"]) || !plain(f.input)) return false;
+        if (kind === "review-verdict") {
+            if (!exactKeys(f.input, ["status"])) return false;
+            const s = f.input.status;
+            return (s === 0 ? "green" : "red") !== (s === 0 ? "green" : s === 1 ? "red" : "could-not-run");
+        }
+        if (f.line === 5) return exactKeys(f.input, ["status"]) && (f.input.status !== 1) !== (f.input.status === 0);
+        if (!exactKeys(f.input, ["root", "target"])) return false;
+        const { root, target } = f.input;
+        if (typeof root !== "string" || typeof target !== "string" || !root.startsWith("/") || !target.startsWith("/") || path.posix.normalize(root) !== root || path.posix.normalize(target) !== target) return false;
+        const canonical = root.length > 1 ? root.replace(/\/+$/, "") : root;
+        const wanted = canonical === "/" || target === canonical || target.startsWith(canonical + "/");
+        return target.startsWith(root) !== wanted;
+    });
+}
+
+function librarianPass(kind, value) {
+    if (!exactKeys(value, ["draft", "actions"]) || value.draft !== true || !Array.isArray(value.actions)) return false;
+    const [action] = value.actions;
+    if (kind === "library-duplicate") return value.actions.length === 1 && exactKeys(action, ["kind", "ids", "obligations", "provenance", "retire_when"]) && action.kind === "merge" && sameSet(action.ids, ["a", "b"]) &&
+        sameSet(action.obligations, ["exit-0-is-green", "exit-2-is-unavailable"]) && sameSet(action.provenance, ["https://example.org/incidents/one", "https://example.org/incidents/two"]) && sameSet(action.retire_when, ["runner-replaced", "unavailable-is-tested"]);
+    if (kind === "library-conflict") return value.actions.length === 1 && exactKeys(action, ["kind", "ids", "decision"]) && action.kind === "escalate" && action.decision === "unresolved" && sameSet(action.ids, ["a", "b"]);
+    if (kind === "library-sealed") {
+        if (value.actions.length !== 2) return false;
+        const sealed = value.actions.find(a => a?.id === "sealed"), open = value.actions.find(a => a?.id === "public");
+        return exactKeys(sealed, ["kind", "id", "owner"]) && sealed.kind === "request_revalidation" && sealed.owner === "Example Team" &&
+            exactKeys(open, ["kind", "id", "evidence"]) && open.kind === "propose_retirement" && open.evidence === "https://example.org/evidence/parser-removal";
+    }
+    return false;
+}
+
+export function grade(task, answer) {
+    const value = parsed(answer);
+    const cases = codeCases(task.grader);
+    const ok = cases ? codePass(value?.code, cases) : task.role === "reviewer" ? reviewerPass(task.grader, value) : librarianPass(task.grader, value);
+    return { ok, check: task.grader };
+}
+
+export function price(requests, prices = CORPUS.prices) {
+    if (!Array.isArray(requests) || !requests.length) throw Error("usage needs at least one completed request");
+    for (const request of requests) for (const key of CLASSES) if (!Number.isSafeInteger(request[key]) || request[key] < 0) throw Error(`invalid usage ${key}`);
+    for (const key of [...CLASSES, "coldFirstInput"]) if (!(Number.isFinite(prices[key]) && prices[key] > 0)) throw Error(`invalid price ${key}`);
+    const cost = request => CLASSES.reduce((sum, key) => sum + request[key] * prices[key], 0);
+    const first = requests[0];
+    const observed = requests.reduce((sum, request) => sum + cost(request), 0);
+    const original = inputClasses.reduce((sum, key) => sum + first[key] * prices[key], 0);
+    const normalized = observed - original + inputClasses.reduce((sum, key) => sum + first[key], 0) * prices.coldFirstInput;
+    if (![observed, normalized].every(Number.isFinite)) throw Error("cost index overflows");
+    return { requests: requests.length, observed, normalized };
+}