@sema-agent/core 7.38.1 → 7.38.3

package/CHANGELOG.md

@@ -1,3 +1,37 @@
+## 7.38.3 — 2026-10-02
+
+**Hard-breaking changes:** none.
+
+**Closed sets:** none. **Package exports:** +0 / −0.
+
+### The JSONL tail repair reads the file's last segment, not the whole file (#1286; behavior (widen: a store file of 2 GiB or more opens and takes appends; an unterminated last segment past the engine's string bound no longer throws at open); +0 exports) (@server @test)
+
+- **Before (v7.38.2).** `AppendLog` repaired a JSONL file's tail at three stations (open, the lazy reopen after a compaction swap, the append door when the file had moved off a record boundary) by reading the whole file to look at its last line. A file of 2 GiB or more could not be opened at all (`ERR_FS_FILE_TOO_LARGE`), so an append-only dataset that grows without bound (an outcomes log) refused every write once it crossed that size; an unterminated last segment whose text was longer than V8's maximum string (`buffer.constants.MAX_STRING_LENGTH`, 536,870,888 characters; for ASCII that is 512 MiB) threw `ERR_STRING_TOO_LONG` at open.
+- **Now.** One function, `readTailSegment(fd, size)` (`src/stores/file/fs-atomic.ts`), walks back from the end in 64 KiB blocks to the last newline and returns the segment after it; the three stations read their tail through it, so a repair costs the length of the last line, not of the file. What the segment IS stays `jsonlTextFormOf`'s call, unchanged: a whole record is ended with its newline, a half record or a non-object is cut back to the last newline, a file that ends on a newline is not touched. A segment longer than that bound, MEASURED IN BYTES (536,870,888 bytes), is not judged: it is ended as a line (one `\n` appended, its bytes untouched, never cut) and one `console.warn` line names the file; the next append stands on its own line behind it, where leaving it open would have glued the next acknowledged record onto it.
+- **Who is hurt.** Nobody who worked before: every file that opened still gets the same repair, byte for byte, with one exception: an unterminated last segment of more than 536,870,888 BYTES of multi-byte text (up to about three times that) could still be decoded before, and a torn one was cut back; it is now ended as a line unjudged like an ASCII one, so a torn one stays as an unparseable line (a reader skips it) instead of being cut. A file that used to refuse to open (2 GiB or more, or an ASCII last segment above the string bound) now opens and appends. The repair is only half of what such a file needs: a store that replays its whole file when it boots (`readJsonlRecords`, still one whole-file read) still cannot read a file of 2 GiB or more, nor a line above the string bound; only a consumer that appends without replaying (a sink) benefits end to end. That read is tracked as its own backlog row (1287), not changed here.
+- **Compensation.** None needed; there is no switch for the old whole read.
+- **Known limit (KL-1286-a).** A last segment longer than the string bound is ended as a line and not judged: if it was a real record, it stays on disk and is not read by the repair (a reader meets it as an ordinary line; what the whole-file read does with it is backlog row 1287's). The bound is the runtime's `buffer.constants.MAX_STRING_LENGTH` applied to the segment's length in BYTES: a segment within it always decodes (a byte is at most one UTF-16 unit), one past it might still decode when it is multi-byte text, and is left unjudged either way (judging by decoded length would mean reading and decoding up to three times the bound to find out). A record that long is half a gigabyte of text: this store's writers do not produce one in practice, so the shape is a foreign writer's or damage.
+- **Known limit (KL-1286-b).** The repair is not serialized with an appender that is writing the same file: it samples the last segment, judges it, then cuts by offset. A peer that completes the half record after the sample and before the cut has that record cut with the remnant. The window is the same read-to-cut one v7.38.2 had (it read the whole file first); it is now the length of the last line, not of the file. The directory fence (one writer per ledger directory) is what keeps a peer from being there; closing the window itself needs the repair to run under that fence.
+- Pins: `test/backlog1286-jsonl-tail-large-file.test.ts` — a 2.2 GB sparse file (nothing is written; `truncateSync` extends it) with ① a whole unterminated last line (ended, one byte added, the next append is its own line), ② a half last line / a non-object tail (cut back), ③ a clean file (not one byte moved), and the other two stations on it (lazy reopen after a swap; append door met by a peer's whole / half tail); ④ tails around read-block edges (100 KiB; every length one byte either side of one and two blocks), ⑤ a line longer than three blocks, ⑥ a file with no newline at all (small, and larger than a block), whole and torn; ⑦ a segment one byte past the string bound (ended, never cut, one warning, the next append not glued); ⑨ the repair reads at most a couple of blocks and never goes through `readFileSync` on the log; ⑪ the present form of KL-1286-b (a peer completing the half record between the read and the cut loses it); `readTailSegment` itself (segment starts, newline at a block's first / last byte, the bound judged at its exact edge). `test/backlog1278-jsonl-tail-newline.test.ts` PIN 14 (a multi-byte character straddling a read-block edge does not tear a whole record) and PINs 1–13 unchanged and green.
+
+## 7.38.2 — 2026-10-02
+
+**Hard-breaking changes:** none.
+
+**Closed sets:** none. **Package exports:** +0 / −0.
+
+### A JSONL tail that parses to an object is a record, newline or not (#1278; behavior (widen: the file backends' JSONL read; narrow: what the open-time repair discards; a failed append leaves no bytes); +0 exports) (@server @test)
+
+- **Before (v7.38.1).** `readJsonlRecords` served nothing after a file's last newline, and `AppendLog` cut that tail away when it opened and again at the append door whenever the file had moved off a record boundary. A record that was whole on disk and lacked only its final `\n` was invisible to every replay and then physically removed. Every file store on the format shared it: session, mailbox, memory, workflow journal, and the shared-ledger stores (checkpoint, background agent, workflow run).
+- **Now.** One function, `jsonlTextFormOf` (`src/stores/file/fs-atomic.ts`), judges a segment, and the three stations that must answer "is the unterminated tail a record" all ask it: the read, the open-time repair, the append door's boundary repair. A file on this format holds one JSON object per line and no strict prefix of a JSON object parses, so a tail that parses to an object is a whole record that lost only its newline: the read serves it; the open-time repair and the append door end its line (the bytes stay); nothing is disclosed (it is not damage). A tail that does not parse is a crash's partial write: not served, cut back to the last newline, silent (unchanged). A tail that parses to a non-object (number, string, boolean, `null`, array) is bytes the store never wrote: skipped and cut, and now disclosed once under `not-an-object` exactly as the same bytes with a newline are (it was silent). The consuming store's own floor (`judge`) applies to the tail like to any line. `truncateTornTail` is renamed `repairJsonlTail` and cuts by byte offset. And an append whose write fails (the write throws, or makes no progress) leaves no bytes of its own: `AppendLog.append` cuts the file back to where the write began, provided the file is exactly that size plus the bytes it wrote (a peer that appended after it owns what follows; a size that cannot be read, or a cut that fails, leaves the file for the next append's tail repair), then rethrows the write's own error. A caller that retries the record writes it once; without this, a failure that left every byte but the newline would, under the rule above, be kept and then written again by the retry.
+- **Who is hurt.** A consumer that relied on an unterminated last line never being a record: it now reads one. No store in this package does (the replays read what a record carries, not how its line ended). On the operator side, a host that listens on a store's corrupt-read seat (`onCorruptRead`) hears one more `not-an-object` disclosure when a file's unterminated last segment parses to a non-object (a number, string, boolean, `null` or array), which was silent before; the same bytes with a newline already disclosed. A caller that relied on a failed append's partial bytes staying on disk: none can, the next append's tail repair cut them.
+- **Compensation.** None needed; there is no switch for the old drop, because a whole record on disk is a record.
+- **Known limit (KL-1278-a).** A whole object followed by a half one with no newline between (`{"a":1}{"b"`), or two whole objects with no newline between, is one segment that does not parse: it is cut whole, before and after, because recovering its first object would be parsing a prefix.
+- **Known limit (KL-1278-b).** The rollback of a failed append has two preconditions: nobody appended after it, and the cut itself works. A peer that appended after our partial bytes leaves its line glued behind them, one unparseable interior line (disclosed `torn-json`, both pieces lost), as before; a failed cut (EIO) that leaves a whole-but-unterminated record is kept by the next open or append door, so a retry writes it twice.
+- **Known limit (KL-1278-c).** A tail whose bytes are invalid UTF-8 inside a closed object (`{"id":"\xE4"}`) is decoded with U+FFFD like an interior line always was, so it is served and its bytes kept; a crash leaves a prefix, never that shape.
+- **Known limit (KL-1278-d).** A non-object tail met first by the open-time repair or the append door, with no read before it, is cut without disclosure (`AppendLog` has no disclosure seat); every file store reads before it opens.
+- Pins: `test/backlog1278-jsonl-tail-newline.test.ts` — ① a whole tail without its newline is read, silently; ② open ends its line; ③ an append after that open is its own line (byte-exact, three records); ④ a half tail is still cut (incl. one cut inside a multi-byte character, and a whole multi-byte tail); ⑤ non-object tails (`123` `[1]` `"x"` `null` `true`) disclosed once and cut, read identically with and without the newline; ⑥ empty / lone-newline / clean / blank-interior files change no byte; ⑧ a 3 MiB tail judged whole or cut; ⑨ a peer's whole unterminated record met by the append door is kept, a peer's half record is cut; ⑩ `{"a":1}{"b"` and two glued objects are cut whole; ⑪ the store's floor applies to the tail; ⑫ the present form of KL-1278-c; 9c a peer's non-object tail is cut by the append door; ⑬ a failed append leaves no bytes and the retry writes the record once (13 partial write then an error, 13b every byte but the newline then an error, 13d a short write then zero progress), 13c a peer's whole record landed before the error is not cut, 13e a peer appended after our partial bytes is not cut (present form of KL-1278-b). `test/file-checkpoint-store.test.ts` PIN 7 — a real checkpoint ledger hand-stripped of its final newline replays, and the next write does not glue. `test/defectscan-truncation-invariant-surface.test.ts` — the exempt entry for the renamed function removed.
+
 ## 7.38.1 — 2026-10-02
 
 **Hard-breaking changes:** none.
@@ -6,9 +40,9 @@
 
 ### Home-anchored read-deny rows are placed under the account home as well as the child's `HOME` (#1244; behavior (tighten: the read deny table); +0 exports) (@server @cli @client-core @test)
 
-- **Before (v7.38.0).** The home-anchored rows of the read deny table (`~/.sema/settings.json`, `~/.ssh`, `~/.aws`, …) were placed under one home: the `HOME` the shell child receives (`ExecutionEnv.homeDir`; `NodeExecutionEnv` reads it off `shellEnv` over the inherited environment). A host that handed the child a temporary `HOME` moved the whole fence with it — the user's real home directory (the account database's, `os.userInfo().homedir`) held no row, and the structured read tools and the shell read station served `~/.sema/settings.json` there.
+- **Before (v7.38.0).** The home-anchored rows of the read deny table (`~/.sema/settings.json`, `~/.claude/settings.json`, `~/.mcp.json`, …; the credential rows `.ssh` / `.aws` match by basename anywhere and were never home-bound) were placed under one home: the `HOME` the shell child receives (`ExecutionEnv.homeDir`; `NodeExecutionEnv` reads it off `shellEnv` over the inherited environment). A host that handed the child a temporary `HOME` moved the whole fence with it — the user's real home directory (the account database's, `os.userInfo().homedir`) held no row, and the structured read tools and the shell read station served `~/.sema/settings.json` there.
 - **Now.** `declaredReadDenyHomes` returns the declared home and, when its lexically normalized spelling differs, the account home. The home set is fixed ONCE per placement — the Runner's (per run), the library-direct toolkit's own, or a host's explicit `readDenyHomes` taken as the home set (the account database is then not asked and no home is added) — and every home of the set, looked up or listed, is anchored at its spelling and at its canonical spelling where the environment answered one — and every judgment reads that set: a judge carries the home views of the placement it was compiled at, so a path spelled through a firmlink or a bind mount of a home is respelled at the home without asking the account again. The Runner hands its tool band that placement itself — the very home list it returned carries its views — so the band never reads the homes a second time; a band handed any other list with the placement's links (a copy) holds its home-anchored rows at every location and says so on `console.warn` (an engine defect, never a silent narrower view). The DECLARED home keeps the v7.38.0 law: its probes wait the gate bound, a placement whose declared home did not answer holds the rows at every location and is asked again, and its directory chain is asked again by every judgment until it answers (a home created after the run began is read once it exists); its views stay under every answer of the account lookup, also under no answer, so a user's deny rule and the write face meet its aliases as in v7.38.0. The ACCOUNT home (and every entry of an explicit list that is not the declared home) is best effort and bounded: its canonical spelling, its identity chain and where its rows' heads lead are read once per placement under one 2000 ms bound; what answered is kept and only what did not is lost, that home's alone — a canonical spelling that answered anchors the rows there at once; a chain that did not answer leaves a path through an alias of the home judged as spelled, a head that did not answer adds no linked location, until the next placement; a canonical spelling that did not answer leaves the rows at the home's spelling alone — the placement stays whole, and one `console.warn` line says so. In the chain a directory that is a home (read off the home's own canonical directory, where its identity answered) keeps the home's own spelling over an ancestor's. A head under a spelling that is no directory — a file, a symlink to a file, a device node, a socket: the environment's path resolution answers `not_directory` — has nothing below it (an answer; before, only a plain file counted: a `HOME` that is a symlink to a file left the placement not whole, replayed on every call). The `~` expansion base is unchanged: `ExecutionEnv.homeDir` stays the directory the child's own `~` names. An environment that declares no absolute home still places nothing (a home-anchored row then holds its run anywhere); the account home is no stand-in for it.
-- **Who is hurt.** A read of a file under the account home that matches a home-anchored row (`~/.sema/settings.json`, `~/.ssh/…`, `~/.aws/…`) while the child's `HOME` is elsewhere: refused (`read_path_denied`) where it was served. A host that lists `readDenyHomes` itself sees the same placement, and its views now cover every listed home (v7.38.0 respelled aliases of the declared home only: an alias spelling of another listed home was served); its explicit list stays a legal channel and can be withdrawn once it carries nothing the default does not. A remote or sandboxed leg gets the engine host's account home as one more anchor: a row over a path that may not exist there. A deployment whose account lookup fails for a reason other than "no passwd row" gets the over-strict placement above (rows held at every location) until the lookup answers. A hung account home makes a placement wait up to 2 s more, once per placement (the Runner's tool band judges with the run's placement and reads no home again). A host that passes BOTH `readDenyHomes` and `readDenyHomeLinks` (a hand-built pair) is now taken for the engine's own handoff of a placement it cannot find: the home-anchored rows are held at every location and `console.warn` names it (v7.38.0 placed the rows at the listed homes); no sibling host makes that call — pass `readDenyHomes` alone and let the toolkit resolve where the heads lead. No sibling host lists one today: server's only home seat is the execution environment's `homeDir` (`boot/execution-env.ts`), so its deployments get the account home from this default with no change on its side.
+- **Who is hurt.** A read of a file under the account home that matches a home-anchored row (`~/.sema/settings.json`, `~/.claude/settings.json`, `~/.mcp.json`) while the child's `HOME` is elsewhere: refused (`read_path_denied`) where it was served. A host that lists `readDenyHomes` itself sees the same placement, and its views now cover every listed home (v7.38.0 respelled aliases of the declared home only: an alias spelling of another listed home was served); its explicit list stays a legal channel and can be withdrawn once it carries nothing the default does not. A remote or sandboxed leg gets the engine host's account home as one more anchor: a row over a path that may not exist there. A deployment whose account lookup fails for a reason other than "no passwd row" gets the over-strict placement above (rows held at every location) until the lookup answers. A hung account home makes a placement wait up to 2 s more, once per placement (the Runner's tool band judges with the run's placement and reads no home again). A host that passes BOTH `readDenyHomes` and `readDenyHomeLinks` (a hand-built pair) is now taken for the engine's own handoff of a placement it cannot find: the home-anchored rows are held at every location and `console.warn` names it (v7.38.0 placed the rows at the listed homes); no sibling host makes that call — pass `readDenyHomes` alone and let the toolkit resolve where the heads lead. No sibling host lists one today: server's only home seat is the execution environment's `homeDir` (`boot/execution-env.ts`), so its deployments get the account home from this default with no change on its side.
 - **Compensation.** None needed for the common case (`HOME` equals the account home: one home, nothing changes). A deployment that really reads a file under its own account home that a row names lists the row in `readDenyBuiltinExclude`.
 - **Known limit (KL-1244-a).** The account lookup has three answers, the split the environment's `canonicalPath` already meets, and NOTHING is kept for the process: every placement asks afresh, and a placement's judgments never ask. A home places. **No account home** — `os.userInfo()` throws the libuv "no passwd row" error (`info.code === "ENOENT"`, a container that runs an arbitrary uid; also a directory service that is down and says so) or answers a non-absolute path — places the rows under the declared home alone, a whole placement; the missing row is not a fault and does not warn (a non-absolute answer warns once). A library-direct toolkit does not place again after a whole placement, so an account home that answers later is not placed for that toolkit's lifetime; the Runner's next run places it. **No answer** on the first placement — any other failure, whatever the error object looks like — says so once on `console.warn` and yields an EMPTY home list (`readDenyHomesOf`, `declaredReadDenyHomes`), the engine's rule for "no home known": the home-anchored rows hold their run at every location until the lookup answers (the project's own `.sema/settings.json` is refused too — over-strict, recoverable) and the placement is asked again at the next placement; the home views keep the declared home meanwhile. `readDenyHomesOf` returns `[]` both when the environment declares no home and when the lookup has no answer, so a host cannot tell them apart and should not cache the list: take it at every mount, or omit `readDenyHomes` and let the toolkit place itself. A toolkit whose table has no home-anchored row places nothing and its views read the declared home (or the list it is handed) without asking the account. A structured notice would be a closed-set row; a minor release carries it.
 - **Known limit (KL-1244-b).** An auxiliary home loses, until the next placement, exactly the parts that did not answer within its 2000 ms bound: a chain that did not answer — paths through an alias of it are judged as spelled; a head that did not answer — no linked location for it; a canonical spelling that did not answer — the rows sit at its spelling alone (neither its canonical spelling nor its linked targets are guarded). The next placement is the Runner's next run; a library-direct toolkit does not place again for that. A directory the probes may not enter (EACCES) is no answer: as the declared home it leaves the placement not whole (replayed per call; the rows stay at its spellings). **Known limit (KL-1244-d).** The public `judgeShellRead` has no placement to read — the boundary's deny seats are the host's own compile and the input carries no home list — so its views read the declared home and the account home looked up for that call; rows the host placed at another home get no alias views there (as in v7.38.0, whose public judgment read the declared home only); an optional input seat comes with a minor. **Known limit (KL-1244-c).** A user's own `~/` persisted permission rule (`Read(~/.aws/**)` …) still resolves against the environment's declared home only (`prepare-task.ts`, `prepare-policy-chain.ts`, `permission-rule-lanes.ts`); under a temporary `HOME` such a rule does not cover the real home directory. The built-in read deny table does.

package/dist/core/store-contracts/read-back-shape-contract.d.ts

@@ -13,6 +13,11 @@ import { type ContractAssertionRunner } from "./contract-harness.js";
  * turns a recoverable incident into a silent one. Which read-back failures a store TOLERATES (an
  * enumeration that skips one unreadable row and serves the rest, a torn trailing write read as absent)
  * is a per-store fact registered beside that store's own contract, never assumed from another store's.
+ * On the JSONL file format the unterminated last segment is judged by the same parse as every other line
+ * (`readJsonlRecords`): one that parses to an object is a whole record that lost only its newline (served,
+ * and its line ended at the next open or append), one that does not parse is a crash's partial write (absent,
+ * cut back to the last newline), so a newline is the boundary's spelling and never the condition of a record
+ * existing.
  */
 export declare const CORRUPT_BODY_FORMS: readonly ["not-an-object", "missing-required", "wrong-type", "torn-json"];
 /** One way a persisted body can fail to be the shape its store writes. */

package/dist/stores/file/fs-atomic.d.ts

@@ -36,8 +36,9 @@ export declare function ensureDir(dir: string): void;
 export declare function atomicWriteFile(tmpDir: string, target: string, bytes: string): void;
 /**
  * Read a JSONL file and return every COMPLETE, parseable record — the universal torn-tail recovery (§2.2):
- *  - a trailing line with no terminating `\n` (a crash mid-write) is discarded,
- *  - any line (interior or tail) that fails `JSON.parse` is skipped (never throws — `listRemoteAgentMetadata`
+ *  - the last segment, when no `\n` ends it, is judged as every line is (`@contract store.read_back_shape_judged`):
+ *    an object is served, an unparseable remnant is a crash's partial write and is discarded,
+ *  - any interior line that fails `JSON.parse` is skipped (never throws — `listRemoteAgentMetadata`
  *    parity), so a single corrupt interior line never poisons the whole replay.
  * A missing file → `[]`. This is why JSONL beats one big JSON blob: a torn tail can never corrupt the store.
  *
@@ -48,10 +49,11 @@ export declare function atomicWriteFile(tmpDir: string, target: string, bytes: s
  * carry my members at all?"); a record it rejects is skipped as `missing-required` on the same road, so
  * the count below covers every skip.
  *
- * `onCorrupt` (optional): one `record` disclosure per skipped INTERIOR record, then — iff any was skipped —
- * one `replay` tally. The torn TAIL stays out of both by design: dying mid-write is the crash shape this
- * format exists to absorb. The callback is invoked as given — a seat that must not throw swallow-guards on
- * ITS side.
+ * `onCorrupt` (optional): one `record` disclosure per skipped record, then — iff any was skipped — one
+ * `replay` tally. The unparseable unterminated tail stays out of both by design: dying mid-write is the
+ * crash shape this format exists to absorb. An unterminated tail that parses to a non-object is not that
+ * shape (no prefix of an object parses to one), so it is disclosed like any other foreign line. The callback
+ * is invoked as given — a seat that must not throw swallow-guards on ITS side.
  */
 export declare function readJsonlRecords<T>(path: string, onCorrupt?: (info: CorruptRecordRead | CorruptReplayTally) => void, judge?: JsonlRecordJudge): T[];
 /** A consuming store's floor on a parsed JSONL object: `accepts` = one of its records; `what` names a
@@ -114,7 +116,7 @@ export declare class AppendLog {
      *
      * It is the cheap arbiter for "did the file change under us since we last left it whole": equal ⇒
      * nothing to check, different ⇒ read the last byte and repair if it is not a newline. See
-     * {@link repairTornTail}.
+     * {@link repairTail}.
      */
     private boundary;
     constructor(path: string);
@@ -122,7 +124,7 @@ export declare class AppendLog {
     /**
      * RB-438 (2026-07-31, hardening probe R2) — re-establish the last-complete-record boundary before a write.
      *
-     * The constructor's {@link truncateTornTail} (and the RB-167 lazy reopen) only ever ran at OPEN time, and
+     * The constructor's {@link repairJsonlTail} (and the RB-167 lazy reopen) only ever ran at OPEN time, and
      * the RB-66 loop below only repairs a SHORT write — not a `writeSync` that THROWS mid-loop (ENOSPC/EDQUOT/
      * EIO after a partial `write(2)`, the classic full-volume shape) and not the `n <= 0` guard. Both leave
      * `written` bytes on disk with no terminating newline while the fd and this object stay live, and every
@@ -131,11 +133,13 @@ export declare class AppendLog {
      * onto the remnant, so replay dropped BOTH — after that second append had returned and fsync'd. Measured:
      * records [1,2,3] + a torn remnant + appends 5,6 replayed as [1,2,3,6].
      *
-     * Cost on the healthy path is ONE `fstat` per append (the sizes match and we return); the whole-file read
-     * only happens when the file actually moved off a boundary. A legitimate concurrent appender (another
-     * process under the boot-lock handoff) leaves the file newline-terminated, so it is adopted, not truncated.
+     * Cost on the healthy path is ONE `fstat` per append (the sizes match and we return); the tail read only
+     * happens when the file actually moved off a boundary, and it reads the last segment, not the file. A
+     * legitimate concurrent appender (another process under the boot-lock handoff) leaves the file
+     * newline-terminated, so it is adopted, not truncated; a whole record it left without its newline is kept
+     * and ended, and only a partial one is cut.
      */
-    private repairTornTail;
+    private repairTail;
     /**
      * Append one record (`json + "\n"`). `fsync:true` makes the record durable before return.
      *
@@ -145,8 +149,18 @@ export declare class AppendLog {
      * line with no trailing newline, so the NEXT append concatenates onto it and replay drops BOTH records.
      * Loop until every byte lands (write(2) returns how many it took) — the O_APPEND atomicity per call is
      * unchanged, and a genuine I/O error still throws.
+     *
+     * A write that fails leaves no bytes of its own: the file is cut back to where the write began, so a caller
+     * that retries the record writes it once — a record that failed is never also half-present.
      */
     append(record: unknown, fsync: boolean): void;
+    /**
+     * Cut the file back to `startedAt` after a write that threw, provided the bytes past it are exactly this
+     * write's own (`written` of them): a peer that appended after us owns what follows, and its bytes are not
+     * ours to cut. Best effort — the error the caller must see is the write's, so a rollback that cannot run
+     * answers false and the next append's tail repair judges what is left.
+     */
+    private rollBackFailedWrite;
     close(): void;
     /**
      * RB-167 — release the descriptor for a compaction SWAP. The next write reopens it lazily.
@@ -158,6 +172,30 @@ export declare class AppendLog {
     closeForSwap(): void;
     private releaseFd;
 }
+/** Bytes read per positioned read while a file's tail is walked back to its last newline. */
+export declare const JSONL_TAIL_READ_BLOCK: number;
+/**
+ * The longest last segment (in BYTES) a repair station will judge: V8's maximum string length in UTF-16 code units
+ * (Node's `buffer.constants.MAX_STRING_LENGTH`). A segment of N bytes decodes to at most N code units, so one within
+ * the bound can never make `toString` throw. One past it MAY still decode (multi-byte text has fewer units than
+ * bytes) but is left unjudged either way: telling would mean reading and decoding up to three times the bound.
+ */
+export declare const JSONL_TAIL_SEGMENT_LIMIT: number;
+/** A file's last segment (the bytes after its last `\n`, or the whole file when it holds none) — or the fact that it is
+ *  too long to judge. `start` is the segment's offset; a file that ends on a newline has an empty segment at `size`. */
+export type JsonlTailSegment = {
+    unjudgeable: false;
+    start: number;
+    bytes: Buffer;
+} | {
+    unjudgeable: true;
+};
+/**
+ * The last segment of an open file of `size` bytes, found by walking back from the end one block at a time to the
+ * last newline: a read is bounded by the segment's length, never by the file's. The one reader of "what is the tail"
+ * for every repair station; whether the segment is a record is {@link jsonlTextFormOf}'s to say.
+ */
+export declare function readTailSegment(fd: number, size: number, limit?: number): JsonlTailSegment;
 /** Why a {@link BootLock}/{@link acquireStoreDirLock} acquisition was refused, as a branchable code. */
 export type FileStoreLockErrorCode = 
 /** A LIVE process (this one or another) already holds the directory. Steady state: it ends when that

package/dist/stores/file/fs-atomic.js

@@ -1,4 +1,5 @@
-import { closeSync, constants as FS, existsSync, fstatSync, linkSync, mkdirSync, openSync, readFileSync, readSync, renameSync, truncateSync, unlinkSync, writeFileSync, writeSync, fsyncSync, } from "node:fs";
+import { appendFileSync, closeSync, constants as FS, existsSync, fstatSync, ftruncateSync, linkSync, mkdirSync, openSync, readFileSync, readSync, renameSync, truncateSync, unlinkSync, writeFileSync, writeSync, fsyncSync, } from "node:fs";
+import { constants as bufferConstants } from "node:buffer";
 import { homedir, hostname, uptime } from "node:os";
 import { hrtime } from "node:process";
 import { basename, dirname, join, resolve as resolvePath } from "node:path";
@@ -106,6 +107,18 @@ function basenameOf(p) {
 function dirnameOf(p) {
     return dirname(p);
 }
+function jsonlTextFormOf(text) {
+    let value;
+    try {
+        value = JSON.parse(text);
+    }
+    catch {
+        return { form: "unparseable" };
+    }
+    if (value === null || typeof value !== "object" || Array.isArray(value))
+        return { form: "not-an-object", value };
+    return { form: "record", record: value };
+}
 export function readJsonlRecords(path, onCorrupt, judge) {
     let raw;
     try {
@@ -119,7 +132,6 @@ export function readJsonlRecords(path, onCorrupt, judge) {
     if (raw === "")
         return [];
     const parts = raw.split("\n");
-    const endedClean = parts[parts.length - 1] === "";
     const out = [];
     let dropped = 0;
     const discloseSkip = (form, line, what) => {
@@ -127,28 +139,25 @@ export function readJsonlRecords(path, onCorrupt, judge) {
         onCorrupt?.({ kind: "record", path, reason: `${form}: ${what} at line ${line} (skipped)`, line, form });
     };
     for (let i = 0; i < parts.length; i++) {
-        if (i === parts.length - 1 && !endedClean)
-            break;
         const line = parts[i];
         if (line === "")
             continue;
-        let record;
-        try {
-            record = JSON.parse(line);
-        }
-        catch {
+        const read = jsonlTextFormOf(line);
+        if (read.form === "unparseable") {
+            if (i === parts.length - 1)
+                break;
             discloseSkip("torn-json", i + 1, "unparseable jsonl record");
             continue;
         }
-        if (record === null || typeof record !== "object" || Array.isArray(record)) {
-            discloseSkip("not-an-object", i + 1, `jsonl record is ${jsonTypeName(record)}, not an object`);
+        if (read.form === "not-an-object") {
+            discloseSkip("not-an-object", i + 1, `jsonl record is ${jsonTypeName(read.value)}, not an object`);
             continue;
         }
-        if (judge !== undefined && !judge.accepts(record)) {
+        if (judge !== undefined && !judge.accepts(read.record)) {
             discloseSkip("missing-required", i + 1, judge.what);
             continue;
         }
-        out.push(record);
+        out.push(read.record);
     }
     if (dropped > 0) {
         const total = out.length + dropped;
@@ -175,7 +184,7 @@ export class AppendLog {
     constructor(path) {
         this.path = path;
         ensureDir(dirnameOf(path));
-        truncateTornTail(path);
+        repairJsonlTail(path);
         this.fd = openSync(path, "a+", 0o600);
         this.boundary = this.sizeOrUnknown();
     }
@@ -187,7 +196,7 @@ export class AppendLog {
             return -1;
         }
     }
-    repairTornTail() {
+    repairTail() {
         const size = this.sizeOrUnknown();
         if (size < 0) {
             this.boundary = -1;
@@ -207,14 +216,15 @@ export class AppendLog {
             }
         }
         catch {
+            this.boundary = -1;
             return;
         }
-        truncateTornTail(this.path);
+        repairJsonlTail(this.path);
         this.boundary = this.sizeOrUnknown();
     }
     append(record, fsync) {
         if (this.fd < 0 && !this.disposed) {
-            truncateTornTail(this.path);
+            repairJsonlTail(this.path);
             this.fd = openSync(this.path, "a+", 0o600);
             this.boundary = this.sizeOrUnknown();
         }
@@ -223,8 +233,9 @@ export class AppendLog {
             e.code = "log_closed";
             throw e;
         }
-        this.repairTornTail();
+        this.repairTail();
         const buf = Buffer.from(`${JSON.stringify(record)}\n`, "utf8");
+        const startedAt = this.boundary;
         let written = 0;
         try {
             while (written < buf.length) {
@@ -235,7 +246,7 @@ export class AppendLog {
             }
         }
         catch (err) {
-            this.boundary = -1;
+            this.boundary = this.rollBackFailedWrite(startedAt, written) ? startedAt : -1;
             throw err;
         }
         if (this.boundary >= 0)
@@ -243,6 +254,19 @@ export class AppendLog {
         if (fsync)
             fsyncSync(this.fd);
     }
+    rollBackFailedWrite(startedAt, written) {
+        if (startedAt < 0)
+            return false;
+        try {
+            if (fstatSync(this.fd).size !== startedAt + written)
+                return false;
+            ftruncateSync(this.fd, startedAt);
+            return true;
+        }
+        catch {
+            return false;
+        }
+    }
     close() {
         this.disposed = true;
         this.releaseFd();
@@ -262,20 +286,71 @@ export class AppendLog {
         }
     }
 }
-function truncateTornTail(path) {
-    let raw;
+export const JSONL_TAIL_READ_BLOCK = 64 * 1024;
+export const JSONL_TAIL_SEGMENT_LIMIT = bufferConstants.MAX_STRING_LENGTH;
+function readExactly(fd, into, length, position) {
+    let got = 0;
+    while (got < length) {
+        const n = readSync(fd, into, got, length - got, position + got);
+        if (n <= 0)
+            throw new Error(`jsonl tail: the file ended ${length - got} bytes early while its last line was read — it shrank under the reader`);
+        got += n;
+    }
+}
+export function readTailSegment(fd, size, limit = JSONL_TAIL_SEGMENT_LIMIT) {
+    const block = Buffer.allocUnsafe(Math.min(JSONL_TAIL_READ_BLOCK, size));
+    let start = 0;
+    for (let end = size; end > 0;) {
+        const from = Math.max(0, end - JSONL_TAIL_READ_BLOCK);
+        readExactly(fd, block, end - from, from);
+        const newline = block.subarray(0, end - from).lastIndexOf(0x0a);
+        if (newline >= 0) {
+            start = from + newline + 1;
+            break;
+        }
+        end = from;
+        if (size - end > limit)
+            return { unjudgeable: true };
+    }
+    if (size - start > limit)
+        return { unjudgeable: true };
+    const bytes = Buffer.allocUnsafe(size - start);
+    readExactly(fd, bytes, bytes.length, start);
+    return { unjudgeable: false, start, bytes };
+}
+function repairJsonlTail(path) {
+    let fd;
     try {
-        raw = readFileSync(path, "utf8");
+        fd = openSync(path, "r");
     }
     catch (err) {
         if (err.code === "ENOENT")
             return;
         throw err;
     }
-    if (raw === "" || raw.endsWith("\n"))
+    let segment;
+    try {
+        segment = readTailSegment(fd, fstatSync(fd).size);
+    }
+    finally {
+        closeSync(fd);
+    }
+    if (segment.unjudgeable) {
+        appendFileSync(path, "\n");
+        try {
+            console.warn(`[sema] file store: a tail segment larger than the judgeable bound (${JSONL_TAIL_SEGMENT_LIMIT} bytes) was ended as a line, not judged — ` +
+                `${path}. Its bytes are untouched; a reader meets it as an ordinary line.`);
+        }
+        catch {
+        }
+        return;
+    }
+    if (segment.bytes.length === 0)
         return;
-    const lastNl = raw.lastIndexOf("\n");
-    truncateSync(path, lastNl === -1 ? 0 : Buffer.byteLength(raw.slice(0, lastNl + 1), "utf8"));
+    if (jsonlTextFormOf(segment.bytes.toString("utf8")).form === "record")
+        appendFileSync(path, "\n");
+    else
+        truncateSync(path, segment.start);
 }
 export class FileStoreLockError extends Error {
     code;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@sema-agent/core",
-  "version": "7.38.1",
+  "version": "7.38.3",
   "description": "Stateless, task-oriented AI agent core",
   "type": "module",
   "license": "BUSL-1.1",

package/wire-changes/core@7.38.2.json

@@ -0,0 +1,44 @@
+{
+  "repo": "core",
+  "version": "7.38.2",
+  "from": "a8493dd6305fe6f8b3fea1174b56c30667f3ff1b",
+  "to": "25b09b0232b290da289a6d9697f0d3b72826713c",
+  "added": [],
+  "removed": [],
+  "changed": [],
+  "narrowed": [],
+  "closedSetMembers": [],
+  "closedSetsUnreadable": [],
+  "typeKeys": [],
+  "typeKeysUnreadableArms": [],
+  "typeKeysFlattenedCount": 220,
+  "declarationMoves": [],
+  "mintMoves": [],
+  "criteriaObserves": [
+    {
+      "kind": "typeMember",
+      "name": "AppendLog.append",
+      "assertions": [
+        "G1.b",
+        "G1.c",
+        "G1.d"
+      ]
+    },
+    {
+      "kind": "typeMember",
+      "name": "CheckpointStore.get",
+      "assertions": [
+        "G1.a",
+        "G1.c"
+      ]
+    }
+  ],
+  "consumersHint": {
+    "cli": [],
+    "server": [],
+    "sdk": [],
+    "note": "scanned cli@64a76326 server@61d9a1d1 sdk@925ffec — a name listed is MENTIONED there today, which is evidence of consumption, not proof of it"
+  },
+  "breakingCount": 0,
+  "breakingItems": []
+}

package/wire-changes/core@7.38.3.json

@@ -0,0 +1,36 @@
+{
+  "repo": "core",
+  "version": "7.38.3",
+  "from": "25b09b0232b290da289a6d9697f0d3b72826713c",
+  "to": "e0ab70decfe37f3dcdcb4aff4f8e694df14a7efe",
+  "added": [],
+  "removed": [],
+  "changed": [],
+  "narrowed": [],
+  "closedSetMembers": [],
+  "closedSetsUnreadable": [],
+  "typeKeys": [],
+  "typeKeysUnreadableArms": [],
+  "typeKeysFlattenedCount": 220,
+  "declarationMoves": [],
+  "mintMoves": [],
+  "criteriaObserves": [
+    {
+      "kind": "typeMember",
+      "name": "AppendLog.append",
+      "assertions": [
+        "G1.a",
+        "G1.b",
+        "G1.c"
+      ]
+    }
+  ],
+  "consumersHint": {
+    "cli": [],
+    "server": [],
+    "sdk": [],
+    "note": "scanned cli@64a76326 server@f4ce4085 sdk@2756723 — a name listed is MENTIONED there today, which is evidence of consumption, not proof of it"
+  },
+  "breakingCount": 0,
+  "breakingItems": []
+}