bareloop 0.24.0 → 0.25.0

package/CHANGELOG.md

@@ -5,7 +5,103 @@ All notable changes to bareloop are documented here. Format:
 [SemVer](https://semver.org/spec/v2.0.0.html). Pre-1.0: **minor** = a ladder rung or
 feature lands, **patch** = docs, fixes, scaffolding.
 
-## [Unreleased]
+## [0.25.0] — 2026-09-12
+
+### Added
+
+- **The source front door** (PRD item 33/M2) — a plain folder, a plain file, or one
+  `http(s)` URL now runs without a `--patient` git checkout: `prepareSource`
+  (`src/source.js`) freezes it into a hidden, scratch git tree bareloop owns (the person
+  never sees the repo), `proveDestination` proves a drop-off file is landable at $0 before
+  any token spends, and `copyOut` delivers the run's own output there once, on a minted
+  green, never overwriting. New CLI `scripts/prep-source.mjs`. `scripts/run-u.mjs` wires
+  both: a manifest-declared destination refuses before any token, and copies out on a
+  green (a refused copy-out never changes the verdict). The destination is a PER-RUN value
+  recorded in the manifest (`<into>/source.json`), never a signed job-spec field — a job
+  spec is a repeatable shape, signed once; nothing was added to `job.js`/`jobSpecHash`.
+  Exported: `prepareSource`, `proveDestination`, `copyOut`, `readSourceManifest`,
+  `frontDoorFromManifest`. `git()` (`src/kinds.js`) is now exported for reuse by this
+  module rather than re-spawned a second way.
+
+### Fixed
+
+- **Source front door (PRD item 33/M2 review):** `output` is now validated the ONE way
+  everywhere it is read or written (new `output-invalid` refusal) — a bare
+  `resolve(tree, output)` previously let a shape like `output/../../x` escape the frozen
+  tree; a hand-edited manifest carrying such a value is now treated as no front door at
+  all by `frontDoorFromManifest`, never handed to `copyOut` unvalidated. `prepareSource`
+  now proves the destination (`proveDestination`) BEFORE creating `into`, so a refused
+  destination leaves nothing on disk. `copyOut` takes an optional `into` (the run's
+  scratch root); `scripts/run-u.mjs`'s two call sites now pass `dirname(wd)` instead of
+  `wd`, so a destination inside the scratch area but outside the frozen tree is caught as
+  `destination-contained` instead of slipping through.
+- **Source front door hard-line defect (PRD item 33/M2, live-proven):** every frozen
+  file's content — a folder, a single file, or a fetched URL body — is now scanned for a
+  known secret shape (the same `scanSecrets` inventory, `src/validate.js`) BEFORE
+  anything is written under `into`; a hit refuses `source-carries-secret`, naming the
+  file(s) and pattern name(s) only, never the matched text, and leaves nothing on disk.
+  Previously the door scanned only the URL string, so a plain folder or file carrying a
+  real API key (e.g. an `.env`) was frozen into the tree and committed to the hidden git
+  seed untouched.
+- **Source front door, M2b review fixes 2–8 (PRD item 33):** `.env`/`.env.*` refused by
+  NAME whatever it contains (`source-env-file`), belt-and-braces beside the content scan;
+  a per-file `MAX_BUFFER` ceiling for folder and single-file sources (`source-file-
+  oversize`; repo sources exempt); the delivered file now carries the date
+  (`profile-2026-09-12.md`, `-2`/`-3` the same day, cap 99) — new exports
+  `datedDestination`/`pickDelivery`, used by both `proveDestination` and `copyOut` so the
+  two can never disagree; `copyOut` returns `{bytes, sha256, path}` and `run-u.mjs`
+  spine-records the real landed path (`destination-written {path, declared, bytes,
+  sha256}`); a URL's redirects are followed but never invisible — the final URL rides in
+  the manifest (`finalUrl`) and is printed by `prep-source`, and is secret-scanned like
+  the typed one; a nested `.git` anywhere below the root refuses `source-nested-repo`,
+  `git add` runs `-f` so a source `.gitignore` cannot drop a file from the seed, and the
+  seed is read back with `git ls-tree` and diffed against the manifest's file list
+  (`source-seed-incomplete` on a shortfall); a git repo root is now an allowed source
+  (`kind: 'repo'`) — copied with its history, working files at the tree root, seed on top
+  of existing history; a `.git` FILE (linked worktree/submodule) refuses
+  `source-is-linked-worktree`; `prep-source` prints a folder-blast-radius note (mirrored
+  in `bareloop.context.md`).
+- **Repo-source hard-line defect (live-proven during the M2b 7 review):** a copied
+  `.git/hooks` carried the SOURCE repo's own `pre-commit`/`commit-msg`/`post-commit`
+  scripts, and `git commit` ran them — arbitrary code from the source repo, executed
+  inside bareloop's own process, before a single token spent. Fixed: the copied hooks
+  directory is stripped after the `.git` copy, and every git call the door makes pins
+  `core.hooksPath` to a path that is never created, so a `core.hooksPath` set in the
+  copied `.git/config` cannot reopen the hole either.
+
+- **Source front door D1/D2/D3 rework (PRD item 33, hamr's rulings, 2026-09-12):**
+  - **D1 — repo sources copy only what git tracks** (hamr, verbatim: "copy only what git
+    tracks"; closes F164/F165): repo sources are enumerated with `git ls-files --stage` in
+    the source, never a filesystem walk; each tracked path's working-tree content is
+    frozen, and nothing untracked or gitignored is ever a candidate (npm
+    `node_modules/.bin/*` symlinks, this repo's own `.claude/`, a gitignored `.env`,
+    none of it reaches the tree or the seed). `git add -A` drops `-f` for repo sources
+    (kept for plain folder/file/URL sources). A gitlink (a real submodule) refuses
+    `source-nested-repo`; a tracked symlink refuses `source-symlink` only when it
+    resolves outside the source root — one staying inside is copied verbatim as a link.
+  - **D2 — secrets scanned inside binary content too** (closes the F166 residual): every
+    frozen file's content is scanned for secrets, binary files included, decoded `latin1`
+    (byte-preserving) instead of `utf8` — the same `SECRET_PATTERNS` inventory, no second
+    pattern list.
+  - **D3 — destination is a DIRECTORY, never a filename** (hamr's ruling, 2026-09-12): it
+    may already exist, need not be empty, and may sit inside the source itself
+    (`destination-in-source` is gone). A job may produce more than one file; `copyOut`
+    delivers every non-empty file under `output/` (excluding `output/.gitkeep`), each
+    under its own dated name, and returns `{files: [{path, bytes, sha256}]}` instead of a
+    single object. The single-file `output` manifest field and `output-invalid` validator
+    are removed — the escape vector they guarded is now closed structurally, since
+    `copyOut` reads `tree/output/` itself rather than a manifest-declared path. A repo
+    source's `destination` is recorded as the declared write fence and never proven or
+    copied into (`frontDoorFromManifest` returns `null` for a repo manifest) — wiring it
+    into the signed `writeScope` field is left to a later milestone. New refusal codes:
+    `destination-not-directory`, `destination-not-writable`, `destination-invalid`
+    (repo-destination type guard); retired: `output-invalid`,
+    `destination-output-required`, `destination-in-source`. `scripts/prep-source.mjs`
+    drops `--output`; `scripts/run-u.mjs`'s copy-out call site emits one
+    `destination-written` per delivered file.
+  - **D4 — measured a real 15 MB repo** (`~/PycharmProjects/adaptlearn`, the repo that
+    broke F164): 1296 files, 3.16 MB copied, 0.89s elapsed, ~84 MB maxRSS, no gitignored
+    or untracked content reached the tree or the seed. No size cap added.
 
 ## [0.24.0] — 2026-09-10
 

package/bareloop.context.md

@@ -2907,6 +2907,183 @@ provider but `anthropic-api`, spine bundling for `formatReplay` on the minting r
 `runid` is noted), and tarball packing — a bundle is a plain directory; use `npm pack` on it
 yourself if you want a tarball.
 
+### The source front door — a plain folder/file/URL, no `--patient` (PRD item 33/M2, `src/source.js`)
+
+A repo job still gives `runJob` a `--patient` checkout it owns. A NON-code job (a resume, a
+CSV, a page) has no repo to point at — this door is what lets it run anyway, WITHOUT the
+person ever seeing a git repo (H1's fix): the source is frozen into a hidden, scratch git
+tree bareloop creates and owns, before any token spends, so the run and its close judge the
+SAME bytes.
+
+```js
+import { prepareSource, proveDestination, copyOut, readSourceManifest, frontDoorFromManifest } from 'bareloop';
+
+const prepared = await prepareSource({
+  source: '/home/me/resume.md',           // an http(s) URL, an existing file, or an existing plain folder/repo
+  into: '/home/me/.bareloop-jobs/j1',     // must NOT already exist — fresh, never reused
+  destination: '/home/me/profile-drafts', // optional — a DIRECTORY (D3), need not exist or be empty
+});
+// prepared.tree === '/home/me/.bareloop-jobs/j1/tree' — hand THIS to runJob as `workdir`
+// prepared.manifest.seed — the commit `seedAtHead` will read at run start
+```
+
+Or the CLI: `node scripts/prep-source.mjs --source <path-or-url> --into <dir> [--destination
+<absolute-directory>]` — prints the tree path, the seed, and the exact next command.
+
+> **If your source is a folder, read this first (hamr's ruling):** make a new folder, put
+> only the file(s) this job needs in it, point bareloop at that — never your original
+> folder. Everything in the folder you name is frozen, read, and paid for. `prep-source`
+> prints the same line on every folder prep, and the panel (N6) shows it beside the Source
+> field.
+
+**Layout.** `<into>/tree/input/…` is the frozen copy; `<into>/tree/output/` is empty (a
+`.gitkeep` seeds it) and is where the run writes; `<into>/source.json` is the manifest,
+OUTSIDE the tree — no worker verb can ever read or edit it. One hidden commit, authored
+`bareloop <bareloop@localhost>` with `commit.gpgsign` pinned off — never the operator's
+identity, never global git config (this repo's own tests run with an empty `HOME`).
+
+**What is admitted today (hamr: "start with already supported files").** A URL is fetched
+ONCE with plain `fetch` — no credentials, no cookies, no custom headers — under
+`PROVIDER_TIMEOUT_MS` and capped at `MAX_BUFFER`, both reused from elsewhere in this library,
+never a second number. Content-type must be `text/*`, `application/json`, or
+`application/xml`. A local file or folder must be plain text (a NUL byte in the first 8KB of
+any file refuses the whole prep, naming every offending file) and must contain no symlinks
+(named, never followed). Each file has its OWN `MAX_BUFFER` ceiling (`source-file-oversize`)
+— there is no folder TOTAL cap, hamr's call. A file named `.env` or `.env.*` is refused by
+NAME whatever it contains (`source-env-file`), belt-and-braces beside the content scan below.
+A `.git` found anywhere BELOW the root is a nested repo and refuses (`source-nested-repo`):
+its objects are a second history nobody declared, and the seed could not hold them honestly.
+PDF and Word input have no reader here (hole H7, PRD item 33) — logged, not built.
+
+**A git repo IS an allowed source** (hamr: yes — the PR-review job). It is COPIED with its
+history, never used in place (the patients-are-copies rule): the working files land at the
+TREE ROOT rather than under `input/`, `.git` is copied verbatim, `output/` is added, and the
+seed commit goes on TOP of the existing history — so `manifest.kind === 'repo'` and the
+worker gets a real repo to review. Its guards (the input stays untouched) arrive with M4. A
+`.git` that is a FILE, not a directory — a linked worktree or a submodule — refuses
+`source-is-linked-worktree`: its real git directory lives elsewhere, and a copy would
+silently commit into the original. Text-only and the per-file ceiling do not apply to a repo
+source (a real repo legitimately carries binaries).
+
+**A repo source copies only what git tracks** (D1 rework, hamr's ruling verbatim: "copy only
+what git tracks" — closes F164/F165). Files are enumerated with `git ls-files --stage` in the
+SOURCE, never a filesystem walk: each tracked path's WORKING-TREE content is frozen (so
+uncommitted edits to a tracked file come along), and nothing untracked or gitignored is ever a
+candidate — real npm `node_modules/.bin/*` symlinks, this repo's own `.claude/`, a gitignored
+`.env`, none of it is read or written. `git add -A` drops `-f` for a repo source (kept for a
+plain folder/file/URL, where a `.gitignore` has no honest meaning yet). A gitlink entry (a real
+submodule) refuses `source-nested-repo`; a tracked symlink refuses `source-symlink` only when
+it resolves OUTSIDE the source root — one that stays inside is legal tracked content, copied
+verbatim as a link, never dereferenced. Measured live against a real 15 MB repo carrying
+`node_modules` (`~/PycharmProjects/adaptlearn`, the repo that broke F164): 1296 files, 3.16 MB,
+0.89s elapsed, ~84 MB maxRSS — no gitignored or untracked content reached the tree or the seed.
+
+**Every frozen file's content is secret-scanned, binary files included** (D2, closes the F166
+residual). A binary file (a NUL byte in its first 8KB) used to be hashed but never scanned —
+now it runs through the SAME `SECRET_PATTERNS` inventory, decoded `latin1` (byte-preserving)
+instead of `utf8`, so an ASCII key embedded in a compiled artifact is still found. This only
+matters where binaries are admitted at all (a repo source today; a plain folder/file still
+refuses binary content outright via `source-not-text`, unchanged).
+
+**A URL's redirects are followed but never invisible.** The URL the body actually came from
+rides into the manifest as `finalUrl` and is printed by `prep-source` whenever it differs
+from what was typed, so a link that quietly lands on a login or error page is something a
+person can see before a token spends. The final URL is secret-scanned the same way the typed
+one is.
+
+**The seed holds every frozen file, and that is CHECKED, not hoped.** `git add` runs with
+`-f`, so a `.gitignore` inside the source cannot drop files from the seed (a file present on
+disk but absent from the seed would read to a close as the worker having written it); then
+the seed is read back with `git ls-tree -r --name-only HEAD` and diffed against the manifest's
+own file list by name. A shortfall refuses `source-seed-incomplete`.
+
+**Every frozen file's content is scanned for a known secret shape before anything is
+written under `into`** (a folder, a single file, or a fetched URL body — the SAME inventory
+`scanSecrets` reads, `src/validate.js`; live-proven fix — the door used to scan only the URL
+string, so a plain `.env` with a real API key was frozen into the tree and committed to the
+hidden git seed untouched). A hit refuses `source-carries-secret`, naming the file path(s)
+and the pattern name(s) only — the matched text itself never appears in the refusal, the
+manifest, or any file, and nothing is written to disk (`into` does not exist afterward). This
+is a shape-based check, not a filename denylist: a secret whose shape is not in the inventory
+(a plain database password in a `.env`, say) still passes through — a named, accepted
+residual, not this fix's job to close.
+
+**Every refusal is a named `{stop, code}`, never a throw and never silent:** `source-
+unreadable`, `source-symlink`, `source-env-file`, `source-file-oversize`,
+`source-nested-repo`, `source-is-linked-worktree`, `source-seed-incomplete`, `source-git-failed`, `source-not-text`, `source-fetch-failed`,
+`source-fetch-timeout`, `source-fetch-oversize`, `source-carries-secret`, `into-exists`,
+`destination-invalid`, `destination-not-absolute`, `destination-not-directory`,
+`destination-not-writable`, `destination-contained`, `destination-parent-missing`,
+`destination-parent-unwritable`, `destination-exists`, `destination-output-missing`,
+`destination-write-failed`, `source-manifest-unreadable`, `source-manifest-invalid`. A
+refusal never throws, never overwrites a person's file, and never lands a destination inside
+the run's own scratch tree.
+
+**Destination is a DIRECTORY, never a filename** (D3 rework, hamr's ruling, 2026-09-12,
+condensed in `docs/product/ITEM33-BUILD.md`). It may already exist and need not be empty; it
+may sit inside the source itself (the same-dir case: "i can limit your actions to a certain
+fix in a certain dir and your output will also be there" — `destination-in-source` is gone). A
+job may produce MORE THAN ONE file (his example: a flight search producing one sheet for
+"SFO→LAX red-eye" and another for "SFO under $700"); the agent names them, and there is no
+`output` field in the manifest any more — `copyOut` always reads `tree/output/` itself, never
+a manifest-declared path, which closes the old escape vector (`output/../../x`) structurally
+rather than by validation.
+
+**Destination is proven at prepare time, not just in-run** (hamr's ruling: source AND
+destination are proven at job start, $0): when `destination` is given, `prepareSource` calls
+`proveDestination` BEFORE creating `into` — a refused destination leaves nothing on disk.
+`proveDestination` only proves the directory is USABLE (exists as a directory and is writable,
+or its parent exists and is writable so it can be created) — it cannot foresee individual
+delivered FILE names, since those are not known until the run actually produces them.
+`copyOut` takes an optional `into` (the run's scratch root, `<into>`, of which `tree` is the
+`tree/` subdirectory; defaults to `tree` for direct callers with no `into`) so a destination
+sitting inside the scratch area but outside the frozen tree — e.g. `<into>/profile.md` — is
+also caught as `destination-contained`, not just one strictly inside `tree`. `run-u.mjs`'s two
+call sites pass `dirname(wd)` (`wd` IS `<into>/tree`) as `into` for exactly this reason.
+
+**A `kind: 'repo'` source's destination is a DIFFERENT thing** (hamr: "destination if for code
+on a green then that would be the place agent allowed to do changes") — it names the WRITE
+FENCE inside the copied repo, the signed `writeScope` field's job (`src/job.js:363`), never a
+filesystem drop-off point. It is recorded as declared, never proven by `proveDestination` and
+never handed to `copyOut` (`frontDoorFromManifest` returns `null` for a repo manifest).
+Wiring it into `writeScope` is M3/M4's job, not this door's.
+
+**Every non-empty file under `output/` is delivered, each under its OWN dated name, and
+nothing is ever overwritten.** A file named `profile.md` LANDS as `profile-2026-09-12.md`
+inside the destination directory; a second delivery of the same name the same day lands as
+`profile-2026-09-12-2.md`, then `-3` (the work-branch collision rule), up to `-99` before
+`destination-exists`. An empty file is silently skipped (not a delivered result); the batch
+refuses `destination-output-missing` only when NOTHING non-empty exists under `output/` at
+all (excluding the `output/.gitkeep` seed placeholder). `datedDestination(destination, now,
+n)` and `pickDelivery(destination, now?)` are exported and unchanged from M2b — `copyOut`
+calls them per file now, at delivery time, rather than once against a single declared path.
+`copyOut` returns `{files: [{path, bytes, sha256}]}` and `run-u.mjs` spine-records ONE
+`destination-written` per delivered file — the declared directory is not the path anything
+wrote.
+
+**The destination is a PER-RUN value, never a signed job-spec field.** A job spec is a
+repeatable SHAPE (goal, checks, judge rules), signed once; source and destination vary per
+run — the same way `bareloop run <bundle> --repo <path>` varies the repo instance without
+touching the bundle's own signature. So `job.js`/`jobSpecHash` carry nothing about it: the
+run's own instance lives entirely in the manifest `prepareSource` writes.
+
+`scripts/run-u.mjs` reads that manifest via `readSourceManifest(dirname(workdir))` (`workdir`
+IS `<into>/tree`, so its parent is `<into>`) and reduces it with `frontDoorFromManifest`. No
+manifest → no front-door behaviour at all — every existing repo-patient job is untouched. A
+manifest that exists but cannot be read or parsed is a named stop, never a silent skip. When
+a manifest declares a destination, `proveDestination` runs BEFORE any token spends (a job
+that cannot land its output should never buy a worker turn first), and `copyOut` runs once,
+on a minted `green` — the ONE outcome string a graded close ever mints; `soft-green` jobs
+mint the same string, since the class lives on `spec.verdictType`, never on `runJob`'s
+return value. A refused copy-out is spine-recorded (`destination-refused {code, detail}`)
+beside the green it could not deliver and NEVER changes that verdict — a delivery failure is
+not a grading failure.
+
+Not wired yet, named rather than silently absent: `bareloop run` (the exported bundle CLI,
+`src/cli.js`) still requires `--repo`; a bundled front-door job is parked past PRD item
+33/M5. The intake form that lets a person fill in Source/Destination through the authoring
+interview (rather than the CLI above) is item 33/M3, not yet built.
+
 ## Architecture
 
 Three layers. An **outer shell** (dumb, permanent): per-run budget cap via bareguard,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bareloop",
-  "version": "0.24.0",
+  "version": "0.25.0",
   "description": "Workflows that earn their own design, with receipts — agent-authored workflows for repeated, long, verifiable jobs that improve across runs under an un-gameable gate.",
   "type": "module",
   "main": "./src/index.js",

package/src/index.js

@@ -285,3 +285,12 @@ export {
 // exists so an adopter driving `bareloop run` from their own script can do so
 // without spawning `bin/bareloop.mjs` as a child process.
 export { main as cliMain } from './cli.js';
+// PRD item 33/M2 (`docs/product/ITEM33-BUILD.md`) — the source front door. A
+// plain folder, a plain file, or one URL frozen into a hidden git tree before
+// any token spends (H1's fix: the person never sees the repo), and the
+// destination proven/copied at the two moments a runner needs them ($0
+// preflight, and once on a minted green). Exported because `scripts/prep-
+// source.mjs` and `scripts/run-u.mjs` are two separate call sites reaching
+// for the SAME functions — an adopter building a third has to reach the same
+// module, not a re-derivation of it.
+export { prepareSource, proveDestination, copyOut, readSourceManifest, frontDoorFromManifest } from './source.js';

package/src/kinds.js

@@ -485,9 +485,15 @@ async function sh(cmd, args, { cwd, env: declared = {}, timeoutMs = DEFAULT_TIME
 /**
  * git, read-only, in a repository. Faults come back as `ok:false` with the
  * stderr — a caller decides whether that is a stop.
+ *
+ * Exported (PRD item 33/M2, `src/source.js`) so the hidden-git seed a plain
+ * folder/file/URL job runs against goes through the SAME spawn primitive as
+ * every other git read in this file — a second `spawn('git', …)` for the
+ * source front door would be a second instrument with its own timeout and
+ * buffer ceiling to keep in sync with this one.
  * @param {string} cwd @param {string[]} args
  */
-async function git(cwd, args) {
+export async function git(cwd, args) {
   const r = await sh('git', args, { cwd, timeoutMs: 120_000 });
   // git refusing IS "the close cannot run": the changed-set primitive is not an
   // optional enrichment, so a non-zero git carries FAILED rather than inheriting