bantamkit-mcp 0.35.6 → 0.35.8

package/README.md

@@ -400,13 +400,17 @@ feature lands in both in one change, and a conformance case compares their answe
 are deliberate and each is ruled in the
 [divergence table](https://github.com/Ink01101011/bantamkit/blob/main/docs/porting.md#where-the-two-runtimes-deliberately-differ):
 
-- **Concurrent writes are safe for the memory store and lossy for the checkpoint, in both
-  runtimes.** `index.md` is derived from `facts/`, so the last writer re-enumerates everything
-  and the index self-heals: 8 trials of two processes × 15 saves, zero disagreements. A
-  checkpoint is not: `shiftwork_clock_out` is read-modify-write, so two concurrent clock-outs on
-  the *same cursor unit* both answer `ok` and one history entry is lost — 10/10 trials on Node
-  **and** 10/10 on the Python reference. The cursor check serialises the normal case; only the
-  same-unit collision loses.
+- **Concurrent writes are safe for the memory store and, since job65, for the checkpoint, in
+  both runtimes.** `index.md` is derived from `facts/`, so the last writer re-enumerates
+  everything and the index self-heals: 8 trials of two processes × 15 saves, zero
+  disagreements. A checkpoint is read-modify-write, and until job65 (#116) every writer shared
+  one temp file, so two concurrent clock-outs — on the same unit OR on two different units of
+  one wave — lost `ok` answers: 10 of 20 in 10 trials, on Node, on Python and across the two.
+  `shiftwork_clock_out` now holds `<checkpoint>.lock` (O_EXCL, the same protocol in both
+  runtimes) around the whole call and writes through a per-writer temp name: two clock-outs of
+  different units both land, and a second clock-out of the same unit is refused as not the
+  cursor unit. A writer that cannot take the lock within 5 s answers `result: error` and writes
+  nothing.
 - **Do not hand-edit a fact file's `name`, `description` or `type` to a bare non-string.**
   `description: 2026` is a YAML integer. Python interpolates it and carries on; this port refuses
   the whole store with `malformed fact file …`. Quote it: `description: '2026'`. Every fact the

package/assets/tools/memory_compact.json

@@ -10,8 +10,7 @@
     "properties": {
       "reserve": {
         "type": "integer",
-        "minimum": 0,
-        "description": "Bytes of index headroom to free below the budget. Omit for the default: the size of the largest index line the store holds, capped at half the budget."
+        "description": "Bytes of index headroom to free below the budget: the target is `budget - reserve`, a negative value counts as 0, any value is capped at half the budget, and 0 compacts to the budget itself. Omit for the default: the room the degraded-index warning demands (the budget minus `(90 * budget - 1) // 100`) plus the size of the largest index line the store holds -- at the default 24000-byte budget with a 39-byte largest line that is 2401 + 39 = 2440, a 21560-byte target."
       }
     }
   },

package/assets/tools/memory_recall.json

@@ -17,9 +17,7 @@
       },
       "k": {
         "type": "integer",
-        "minimum": 1,
-        "maximum": 5,
-        "description": "Facts wanted. Raised to the store default of 3 when smaller; never lowered below it."
+        "description": "Facts wanted, clamped to 1..5. Raised to the store default of 3 when smaller; never lowered below it."
       }
     }
   },

package/assets/tools/shiftwork_clock_in.json

@@ -1,6 +1,6 @@
 {
   "name": "shiftwork_clock_in",
-  "description": "Shift-work clock-in: schema-validate the checkpoint file and return the brief for the unit at plan.cursor — {unit, role, invariants, handoff, do_not, files} — to hand to the spawned agent verbatim. The unit briefed is the one at plan.cursor, or, when unit_id is given, that unit — which must appear in shiftwork_plan's `ready`, the batch the dependency graph says may run now. A unit_id that is not ready, including one naming no unit, is refused with {\"result\": \"error\", \"reason\": \"unit <id> is not ready; ready is <a, b>\"}, and the refusals of the batch view itself (cycle, unknown dependency, unreadable checkpoint) pass through verbatim. Clock-in still never moves plan.cursor; clock_out does. Structured refusals, never exceptions: result=escalate when handoff.open_questions is non-empty, result=success when every unit is done or dropped, result=error when the checkpoint fails validation. Clock-in also WRITES: on result=brief, and only then, it appends one line {event: brief, ts, unit, role} to <checkpoint>.log.jsonl — best-effort: a log it cannot write costs the brief nothing (the brief still returns, no error, only the record is lost). clock_out reads those lines back to write `briefed` on every accounting line, so the ledger is no longer one line per clock-out; a brief line carries `event` and no `status`. See docs/shiftwork.md.",
+  "description": "Shift-work clock-in: schema-validate the checkpoint file and return the brief for the unit at plan.cursor — {unit, role, invariants, handoff, do_not, files} — to hand to the spawned agent verbatim. The unit briefed is the one at plan.cursor, or, when unit_id is given, that unit — either way it must appear in shiftwork_plan's `ready`, the batch the dependency graph says may run now: the cursor unit is held to the same ready test as unit_id. A unit that is not ready (a done or dropped cursor, a cursor whose dependencies have not run, a unit_id naming no unit) is refused with {\"result\": \"error\", \"reason\": \"unit <id> is not ready; ready is <a, b>\"}, and on both paths the refusals of the batch view itself (cycle, unknown dependency, unreadable checkpoint) pass through verbatim; a cursor naming no unit still escalates. Clock-in still never moves plan.cursor; clock_out does. Structured refusals, never exceptions: result=escalate when handoff.open_questions is non-empty, result=success when every unit is done or dropped, result=error when the checkpoint fails validation. Clock-in also WRITES: on result=brief, and only then, it appends one line {event: brief, ts, unit, role} to <checkpoint>.log.jsonl — best-effort: a log it cannot write costs the brief nothing (the brief still returns, no error, only the record is lost). clock_out reads those lines back to write `briefed` on every accounting line, so the ledger is no longer one line per clock-out; a brief line carries `event` and no `status`. See docs/shiftwork.md.",
   "surfaces": [
     "mcp"
   ],

package/assets/tools/shiftwork_clock_out.json

@@ -87,7 +87,7 @@
         "anyOf": [
           {
             "type": "object",
-            "description": "Per-unit cost. Written verbatim, beside ts/unit/role/status, as one line of <checkpoint>.log.jsonl - the ledger the N-sessions experiment reads. The keys named here have fixed meanings so a reader holding only the ledger knows what each number counts, in which unit, and which model produced it; any other key passes through unchanged (a refused key is friction, not safety). Lines written before this schema existed may spell these differently; the schema governs new writes only.",
+            "description": "Per-unit cost. Written verbatim, beside ts/unit/role/status, as one line of <checkpoint>.log.jsonl - the ledger the N-sessions experiment reads. The keys named here have fixed meanings so a reader holding only the ledger knows what each number counts, in which unit, and which model produced it; any other key passes through unchanged, except the five the runtime writes itself — unit, role, status, ts, event — which are refused (result=error) before anything is written; a self-reported briefed is overwritten, not refused. Lines written before this schema existed may spell these differently; the schema governs new writes only.",
             "properties": {
               "tokens": {
                 "type": "integer",

package/assets/tools/validate_json.json

@@ -1,6 +1,6 @@
 {
   "name": "validate_json",
-  "description": "Validate candidate output text against a JSON Schema. Returns {valid, feedback}; when invalid, feed the feedback back to the model and retry.",
+  "description": "Validate candidate output text against a JSON Schema. Returns {valid, feedback}; when invalid, feed the feedback back to the model and retry. The output may be any JSON value; otherwise the first fenced block, or the first object or array in the text, is used. $schema selects draft-04, draft-06, draft-07, 2019-09 or 2020-12 (default); $ref resolves only inside the one schema (\"#\", \"#/...\", \"#name\"), so a remote reference or an embedded $id is refused. pattern is Python re syntax, without inline flags, look-behind, backreferences, atomic or possessive groups, conditional groups or \\N{...}; a document's patterns get 500 ms in total.",
   "surfaces": [
     "mcp"
   ],

package/assets/tools/work_plan.json

@@ -1,6 +1,6 @@
 {
   "name": "work_plan",
-  "description": "Work plan: turn a dependency graph into the batches that may run in parallel. Takes `nodes`, each `{id, depends_on, priority}` — `priority` defaults to 0 — and answers `{\"result\": \"plan\", \"batches\", \"sequence\", \"width\"}`: batch k holds every node whose dependencies all appear in batches below k, `sequence` is those batches flattened in order, and `width` is the largest batch length — the widest fan-out, which is the number an orchestrator needs to decide whether it can afford the batch. Order inside a batch is priority descending, then the order the nodes were given; both halves are contract, because determinism inside a batch is what two implementations are held to. Empty `nodes` is an ANSWER, not a refusal: no batches, no sequence, width 0. Three refusals, checked in this order and one sentence each: `duplicate node id <id>`, `node <id> depends on <dep>, which no node declares`, and `the graph has a cycle: <a> -> <b> -> <a>`. It reports what the graph permits and decides nothing about what may actually be dispatched together — `depends_on` encodes logical order, not file contention, so a batch this tool calls parallel may still hold work that writes one file. Pure computation over the nodes it was handed: it opens no file and takes no path. Never mutates.",
+  "description": "Work plan: turn a dependency graph into the batches that may run in parallel. Takes `nodes`, each `{id, depends_on, priority}` — `depends_on` absent or null means none, `priority` defaults to 0 — and answers `{\"result\": \"plan\", \"batches\", \"sequence\", \"width\"}`: batch k holds every node whose dependencies all appear in batches below k, `sequence` is those batches flattened in order, and `width` is the largest batch length — the widest fan-out, which is the number an orchestrator needs to decide whether it can afford the batch. Order inside a batch is priority descending, then the order the nodes were given; both halves are contract, because determinism inside a batch is what two implementations are held to. Empty `nodes` is an ANSWER, not a refusal: no batches, no sequence, width 0. Four refusals, checked in this order and one sentence each: `nodes[<i>].<field> must be ...` for a node whose `id` is not a string, whose `depends_on` is not a list of strings or whose `priority` is not an integer (named by position, never coerced), `duplicate node id <id>`, `node <id> depends on <dep>, which no node declares`, and `the graph has a cycle: <a> -> <b> -> <a>`. It reports what the graph permits and decides nothing about what may actually be dispatched together — `depends_on` encodes logical order, not file contention, so a batch this tool calls parallel may still hold work that writes one file. Pure computation over the nodes it was handed: it opens no file and takes no path. Never mutates.",
   "surfaces": [
     "mcp"
   ],
@@ -17,11 +17,18 @@
               "description": "The node's identifier, unique across `nodes`"
             },
             "depends_on": {
-              "type": "array",
-              "items": {
-                "type": "string"
-              },
-              "description": "Ids this node waits for; every one of them must be declared by some node"
+              "anyOf": [
+                {
+                  "type": "array",
+                  "items": {
+                    "type": "string"
+                  }
+                },
+                {
+                  "type": "null"
+                }
+              ],
+              "description": "Ids this node waits for; every one of them must be declared by some node. Optional: absent or null means none"
             },
             "priority": {
               "type": "integer",
@@ -29,8 +36,7 @@
             }
           },
           "required": [
-            "id",
-            "depends_on"
+            "id"
           ]
         }
       }

package/dist/cli.js

@@ -497,8 +497,36 @@ function buildMemory(options) {
         checkStoreFlag(options.store);
         return new Memory(options.store, shared);
     }
+    if (options.start !== null)
+        checkStartFlag(options.start);
     return Memory.layered(options.start, shared);
 }
+/**
+ * job65/#142: `--start` must name a directory that is there.
+ *
+ * WAS: unchecked. `pyResolve` of a path that does not exist answers the path anyway, the walk
+ * then climbed its nonexistent parents to the first real one and bound whatever store sat
+ * above it — a typo'd `--start` silently bound an ancestor's store as the WRITABLE project
+ * layer. Unlike `--store`, a missing `--start` is not a designation: nothing is ever created
+ * at a start directory, so a missing one can only be a mistake, and it is refused by name
+ * before a transport exists. Same `stat` and the same sentence family as `checkStoreFlag`.
+ */
+function checkStartFlag(raw) {
+    const start = pyJoin(raw);
+    let isDir;
+    try {
+        isDir = pyStatIsDir(start);
+    }
+    catch (e) {
+        if (!(e instanceof PyOSError))
+            throw e;
+        if (e.name === 'FileNotFoundError')
+            throw new Refusal(`--start does not exist: ${start}`);
+        throw new Refusal(`--start is unreachable: ${start}: ${e.strerror}`);
+    }
+    if (!isDir)
+        throw new Refusal(`--start is not a directory: ${start}`);
+}
 /**
  * `--store` may name a store that is missing, never one that is not a store.
  *

package/dist/contract.d.ts

@@ -120,15 +120,6 @@ export declare function documentError(detail: unknown): string;
  * `Error`'s `message` is that string here, which is what `SkillAuditError` carries.
  */
 export declare function toolFailed(tool: string, detail: unknown): string;
-/**
- * `extract_json`: a fenced block if there is one, otherwise the first `{`/`[` onward.
- *
- * Both arms are here because both reach the model. The fence is `re.search` with `DOTALL`
- * and a non-greedy body, so it takes the FIRST fence and prose on either side is discarded;
- * with no fence, `find` takes the earliest brace or bracket — including one inside a string
- * of prose, which is faithful and occasionally wrong in exactly Python's way. `raw_decode`
- * then stops at the end of that one value, which is why `{"a": 1} — anything else?` parses.
- */
 export declare function extractJson(text: string): PyValue;
 /**
  * `schema_error`: a pointed validation error for `output`, or `null` if it satisfies

package/dist/contract.js

@@ -35,8 +35,8 @@ import { BantamError } from './errors.js';
 import { pyStrip } from './memory/factfile.js';
 import { pyReadText } from './memory/pyfs.js';
 import { resolveImplicitTag } from './memory/pyyaml.js';
-import { fromJs, PyValueError, rawDecode } from './pyjson.js';
-import { absolutePath, validate } from './pyjsonschema.js';
+import { fromJs, parseJson, PyValueError, rawDecode, reprValue } from './pyjson.js';
+import { absolutePath, clipText, PyJsonSchemaRefusal, validate } from './pyjsonschema.js';
 /**
  * `contract.REQUIRED_KEYS`, in Python's order — the order the error message lists them.
  *
@@ -541,7 +541,8 @@ export function toolFailed(tool, detail) {
  * of prose, which is faithful and occasionally wrong in exactly Python's way. `raw_decode`
  * then stops at the end of that one value, which is why `{"a": 1} — anything else?` parses.
  */
-export function extractJson(text) {
+/** `_unfenced`: the text `extract_json` searches — stripped, and the first fence's body if any. */
+function unfenced(text) {
     let body = pyStrip(text);
     // `re.DOTALL` — JS needs `[\s\S]`, and `\s*` after the opening fence is Python's `\s*`.
     const fence = /```(?:json)?[\s\S]*?```/.exec(body);
@@ -549,6 +550,10 @@ export function extractJson(text) {
         const inner = /```(?:json)?\s*([\s\S]*?)```/.exec(body);
         body = pyStrip(inner[1]);
     }
+    return body;
+}
+export function extractJson(text) {
+    const body = unfenced(text);
     const startBrace = body.indexOf('{');
     const startBracket = body.indexOf('[');
     let start;
@@ -580,24 +585,58 @@ export function extractJson(text) {
  * loss.
  */
 export function schemaError(output, schema) {
+    const isValueError = (e) => e instanceof Error && (e.name === 'PyValueError' || e.name === 'PyJSONDecodeError');
     let data;
     try {
-        data = extractJson(output);
+        // #152: a whole-text JSON document of any type first — `"x"`, `1.0` and `null` are JSON
+        // texts — and only then the object/array search `extract_json` does (`json.loads` then
+        // `extract_json` on the reference).
+        data = parseJson(unfenced(output));
     }
-    catch (e) {
-        // `except ValueError` — and `json.JSONDecodeError` IS a `ValueError`, which is how
-        // CPython's decoder text reaches the model.
-        if (e instanceof Error && (e.name === 'PyValueError' || e.name === 'PyJSONDecodeError')) {
-            return parseErrorMessage(e);
+    catch (whole) {
+        if (!isValueError(whole))
+            throw whole;
+        try {
+            data = extractJson(output);
+        }
+        catch (e) {
+            // `except ValueError` — and `json.JSONDecodeError` IS a `ValueError`, which is how
+            // CPython's decoder text reaches the model.
+            if (isValueError(e))
+                return parseErrorMessage(e);
+            throw e;
         }
-        throw e;
     }
     const asPy = isPyValue(schema) ? schema : fromJs(schema);
-    const error = validate(data, asPy);
+    let error;
+    try {
+        error = validate(data, asPy);
+    }
+    catch (e) {
+        // A schema the validator will not resolve — a reference outside the one resource, a
+        // draft-3 `$schema` — is ANSWERED with its sentence, not raised (#125). The Python side
+        // returns the same sentence from `_unsupported_schema` and from the `referencing`
+        // exceptions, so `validate_json` says `{"valid": false, "feedback": …}` on both.
+        if (e instanceof PyJsonSchemaRefusal)
+            return e.message;
+        throw e;
+    }
     if (error === null)
         return null;
     const where = absolutePath(error).map(String).join('/') || 'root';
-    return validationErrorMessage(where, error.message);
+    return validationErrorMessage(clipText(where), clipDetail(error.message, error.instance));
+}
+/**
+ * #153: the offending value quoted at most 200 codepoints, the sentence at most 1000 —
+ * `contract._clip_detail`. Every `repr` of the instance is clipped first, so a `maxLength`
+ * failure on a 1 MiB string still says `is too long`; the whole-sentence cap then bounds
+ * what a message quotes from PARTS of the instance (an unexpected property's name).
+ */
+function clipDetail(message, instance) {
+    const quoted = reprValue(instance);
+    if ([...quoted].length > 200)
+        message = message.split(quoted).join(clipText(quoted));
+    return clipText(message, 1000);
 }
 function isPyValue(value) {
     return (typeof value === 'object' &&

package/dist/eventlog.d.ts

@@ -98,7 +98,20 @@ export declare class EventLog {
      * two runtimes.
      */
     writeFailed: boolean;
-    constructor(path: string | null, capBytes?: number, clock?: () => number);
+    /**
+     * job65/#154: the directory that must already exist before a record is written, or `null`.
+     *
+     * `fromEnv` sets it to the store root when the switch is `on` (the default file lives
+     * INSIDE the store). WAS: the `mkdir` in `append` brought a never-saved store into
+     * existence — `--store <absent>` plus one `work_plan` call left `<store>/events/mcp.jsonl`
+     * behind, and the next layered session walking up through it bound that directory as its
+     * store. The log does not create the store: while the store is not there, a record has
+     * nowhere to go and is DROPPED — not redirected, and not counted as `writeFailed`, because
+     * nothing was offered to the filesystem. The first save creates the store; records after it
+     * land. A literal path is the operator's own choice and keeps `requiresDir === null`.
+     */
+    readonly requiresDir: string | null;
+    constructor(path: string | null, capBytes?: number, clock?: () => number, requiresDir?: string | null);
     static fromEnv(storeRoot: string, env?: Record<string, string | undefined>, clock?: () => number): EventLog;
     get enabled(): boolean;
     /**

package/dist/eventlog.js

@@ -58,7 +58,7 @@
  * notice — the failure mode a diagnostic can least afford.
  */
 import { appendFileSync, mkdirSync, renameSync, statSync } from 'node:fs';
-import { cmpCodepoint, osErrorClassName, pyExists, PyOSError, pyJoin, pyName, pyParent } from './memory/pyfs.js';
+import { cmpCodepoint, osErrorClassName, pyExists, pyIsDir, PyOSError, pyJoin, pyName, pyParent } from './memory/pyfs.js';
 import { bantamkitDirFor, ensureBantamkitGitignore } from './memory/store.js';
 /**
  * Environment switch. Unset or `off`/`0`/`false`/`no`/empty -> disabled. `on`/`1`/`true`/
@@ -185,13 +185,29 @@ export class EventLog {
      * two runtimes.
      */
     writeFailed = false;
-    constructor(path, capBytes = CAP_BYTES, clock = defaultClock) {
+    /**
+     * job65/#154: the directory that must already exist before a record is written, or `null`.
+     *
+     * `fromEnv` sets it to the store root when the switch is `on` (the default file lives
+     * INSIDE the store). WAS: the `mkdir` in `append` brought a never-saved store into
+     * existence — `--store <absent>` plus one `work_plan` call left `<store>/events/mcp.jsonl`
+     * behind, and the next layered session walking up through it bound that directory as its
+     * store. The log does not create the store: while the store is not there, a record has
+     * nowhere to go and is DROPPED — not redirected, and not counted as `writeFailed`, because
+     * nothing was offered to the filesystem. The first save creates the store; records after it
+     * land. A literal path is the operator's own choice and keeps `requiresDir === null`.
+     */
+    requiresDir;
+    constructor(path, capBytes = CAP_BYTES, clock = defaultClock, requiresDir = null) {
         this.path = path;
         this.capBytes = capBytes;
         this.clock = clock;
+        this.requiresDir = requiresDir;
     }
     static fromEnv(storeRoot, env = process.env, clock = defaultClock) {
-        return new EventLog(resolvePath(storeRoot, env[EVENT_LOG_ENV]), CAP_BYTES, clock);
+        const raw = env[EVENT_LOG_ENV];
+        const inStore = raw !== undefined && raw !== null && ON.has(raw.trim().toLowerCase());
+        return new EventLog(resolvePath(storeRoot, raw), CAP_BYTES, clock, inStore ? storeRoot : null);
     }
     get enabled() {
         return this.path !== null;
@@ -246,6 +262,10 @@ export class EventLog {
      * by construction, so that branch is a statement of intent, not a live path.
      */
     append(path, payload) {
+        // job65/#154: `pyIsDir` raises for EACCES exactly as `Path.is_dir()` does, and that lands
+        // in `record`'s catch as a lost record — the same verdict the reference reaches.
+        if (this.requiresDir !== null && !pyIsDir(this.requiresDir))
+            return;
         // AUDIT FINDING (J51-1, mirrored from `runtime-py/src/bantamkit/eventlog.py`): the
         // `mkdirSync` below can bring a whole `.bantamkit` directory into existence on its own
         // -- `BANTAMKIT_EVENT_LOG=on` with a project store that was never saved to reaches here