factoryhub 0.1.0 → 0.1.1

package/spec/factory-format.md

@@ -1,7 +1,11 @@
 # The factory format, v1alpha1
 
 Status: alpha. This document describes the format as the reference engine in
-[`packages/engine`](../packages/engine) reads and runs it at this commit.
+[`packages/engine`](../packages/engine) reads and runs it at this commit,
+except the pack v0 additions that
+[section 12](#pack-v0-additions-not-yet-in-the-reference-engine) lists and
+the single-file form of [section 13](#13-the-single-file-form): the format
+defines them, and the reference engine does not implement them yet.
 Changes go through the [RFC process](rfcs/README.md). The key words MUST,
 MUST NOT, SHOULD and MAY are used as in RFC 2119.
 
@@ -24,6 +28,7 @@ Contents:
 10. [Companion files](#10-companion-files)
 11. [What the format leaves out](#11-what-the-format-leaves-out)
 12. [JSON Schemas and conformance](#12-json-schemas-and-conformance)
+13. [The single-file form](#13-the-single-file-form)
 
 ## 1. Files
 
@@ -33,7 +38,8 @@ flow.yaml                  inputs, steps, who does each, what each needs
 agents/<name>/agent.md     one file per agent: its role and standing instructions
 ```
 
-These three are the core. A factory MAY carry companion files. Only paths of
+These three are the core. A factory MAY carry companion files. The same files
+can also be written as one YAML file ([section 13](#13-the-single-file-form)). Only paths of
 these forms are part of a factory; a runner MUST ignore every other path:
 
 | Path | What it is |
@@ -230,7 +236,7 @@ done:
 | Field | Meaning |
 |---|---|
 | `flow` | `mindburn.flow/v1`. Another value is read with a warning |
-| `requires` | execution features the flow depends on: `scoped-context`, `multiple-outputs`. A runner MUST refuse a flow that requires a feature it does not know |
+| `requires` | execution features the flow depends on: `scoped-context`, `multiple-outputs`, `blocking-checks` ([5.3](#blocking-checks)). A runner MUST refuse a flow that requires a feature it does not know |
 | `title` | the human title. Default: the name, with hyphens as spaces |
 | `author`, `license`, `time` | text: who wrote it, its license (an SPDX identifier is recommended), how long a run takes |
 | `version` | the factory's version, as text. Quote it: `"1.0.0"`. Default `1.0.0` |
@@ -287,6 +293,7 @@ input names; write them normalized, such as `check_kit`.
 | `checks` | for a `check`: the work or deliver step it reviews |
 | `shows` | for an `approve`: the work or deliver step it shows the person |
 | `rounds` | how often a check or person may send the work back: 0 to 5. Default 1 for a check, 3 for an approval |
+| `on_fail` | for a `check` only: `flag` (default) or `stop`, what happens when the work still fails after its redos ([blocking checks](#blocking-checks)) |
 | `tools` | tools the step may use ([5.5](#55-tools)) |
 | `context` | scoped context ([5.7](#57-scoped-context)) |
 | `criteria` | a list of statements this step's result is held to, in place of `done` |
@@ -294,7 +301,7 @@ input names; write them normalized, such as `check_kit`.
 | Kind | What happens |
 |---|---|
 | `work` | The agent does one job and hands its result on. |
-| `check` | A reviewer judges the result named in `checks` against the criteria in `task`. If it fails, that work goes back to be redone with the reasons, and everything built on it runs again. After the allowed redos the run continues and the receipt says the check failed (`flagged`). A reviewer that returns malformed output gets one format-repair attempt; if that also fails, the result is `unreviewed`, never passed. |
+| `check` | A reviewer judges the result named in `checks` against the criteria in `task`. If it fails, that work goes back to be redone with the reasons, and everything built on it runs again. After the allowed redos the run continues and the receipt says the check failed (`flagged`); with `on_fail: stop` the work is held for the owner instead ([blocking checks](#blocking-checks)). A reviewer that returns malformed output gets one format-repair attempt; if that also fails, the result is `unreviewed`, never passed. |
 | `approve` | The run stops and shows the person the result named in `shows`. They approve; say what to change, and that work is redone; or reject, which ends the run with no result. Nothing else counts as a decision, and no runner makes one for them. |
 | `deliver` | The agent writes the final deliverable. |
 
@@ -303,6 +310,34 @@ is the first step in its `needs` that is earlier work or deliver, otherwise
 the last earlier work or deliver step. The subject MUST be work or deliver
 that the step depends on, directly or through other steps.
 
+#### Blocking checks
+
+Requires `requires: [blocking-checks]`. A `check` that says `on_fail: stop`
+does not let refused work through:
+
+- When the work still fails after its redos (`flagged`), or the reviewer gives
+  no valid verdict (`unreviewed`), the run **holds** it for the owner
+  ([section 7](#7-how-a-run-works)). Nothing built on it runs and nothing is
+  delivered; steps that do not depend on it finish first. The owner releases
+  the work as it is (`release`) or discards it and ends the run (`discard`).
+- A blocking check gates its subject: every step that needs the subject, other
+  than another check of it, waits until the check passes or the owner releases
+  the work, whether it needs the check or only the subject. A flow in which the
+  blocking check itself needs such a step is invalid, as it would wait for
+  itself.
+- `on_fail` is allowed only on a `check`, with the value `stop` or `flag`.
+  Without it, or with `flag`, the check behaves as described above.
+- A flow with an `on_fail: stop` check MUST declare
+  `requires: [blocking-checks]`, so a runner without blocking checks refuses
+  the flow ([5.1](#51-top-level-fields)) instead of delivering work its
+  checker refused.
+
+An agent following the factory without a runner, such as an exported skill,
+gets this text for the check in place of the "carry on" text, word for word:
+
+- with `rounds` 1 or more: `If it fails, send it back to be redone <once|up to N times> with the reasons, and redo every step that used it. If it still fails, stop and ask the person; do not deliver. Nothing that uses "<subject title>" starts until this check passes or the person lets the work through.`
+- with `rounds: 0`: `If it fails, do not redo it. Stop and ask the person; do not deliver. Nothing that uses "<subject title>" starts until this check passes or the person lets the work through.`
+
 Fields that other harnesses use for tools and connections (`mcp`,
 `mcpServers`, `mcp_servers`, `connectors`, `plugins`, `skills`) are read as
 tools this step needs ([5.5](#55-tools)). Any other unknown field is kept, has
@@ -394,14 +429,48 @@ composition:
   compiler: mindburn.compose/v1
   intent: The person's original request.
   sources:
-    - { id: research, name: market-research, version: "1.0.0", sha256: <64 hex> }
+    - { id: research, name: market-research, title: Market research, version: "1.0.0", sha256: <64 hex> }
   unmetCapabilities: []
 ```
 
-All four fields are required and nothing else is allowed. `composeFactories`
-in [`compose.mjs`](../packages/engine/src/compose.mjs) produces these
-factories from members pinned by hash; the result is an ordinary factory and
-needs no registry to run. A source hash establishes identity, not trust.
+All four fields are required and nothing else is allowed. Each source has
+`id`, `name`, `version` and `sha256`, and MAY have `title`, the member
+factory's title, at most 120 characters. `composeFactories` in
+[`compose.mjs`](../packages/engine/src/compose.mjs) produces these factories
+from members pinned by hash; the result is an ordinary factory and needs no
+registry to run. A source hash establishes identity, not trust, and a
+`composition` block is text anyone can write: a runner names the members of a
+run in its receipt only for a composition it compiled itself
+([section 8](#8-receipts)).
+
+Each member's steps run under the member's id (member `posts`, step `write`
+runs as `posts_write`). A compiler:
+
+- refuses member ids where one begins with another followed by `_`, so every
+  step belongs to exactly one member;
+- keeps a member check's `on_fail`, and adds `blocking-checks` to `requires`
+  when any member check says `on_fail: stop`;
+- MAY take `maxCalls`, a positive whole number, and then refuses a
+  composition whose worst-case run needs more model calls (`worstCaseCalls`).
+  It returns `worstCaseCalls` with the result. The worst case counts the
+  calls one run can make: a work or deliver step with tools may take every
+  model turn (8 in the reference runner), one without tools makes one call; a
+  check makes two, its verdict and one format repair; an approval makes none.
+  Each check or approval may send its subject back as often as its `rounds`
+  allow, and each time the subject, the sender and every step built on the
+  subject that does not wait for the sender run again.
+
+A member MAY carry `rounds: { <step>: <n> }`, with the member's own step
+names. It lowers the revision rounds of its own check or approve steps in the
+composed flow, so a composition fits `maxCalls` without changing the member.
+The compiler refuses something other than a map, a step that is not one of
+the member's checks or approvals, and a fraction, a negative number or more
+rounds than the step has. The member's pin is unchanged.
+
+A member's `mindburn.tests.yaml` tests that member alone, so the compiler
+leaves it out of the composed factory. The member's pin stays the bundle hash
+of all its files, the tests file included. Any other file the compiler cannot
+compile is refused.
 
 ### 5.10 Limits
 
@@ -456,14 +525,30 @@ run; an agent can follow its foreman.
    differs for every occurrence, so a delayed decision cannot land on text that
    changed since or on a later review of the same text. With no decision the
    runner returns the run still waiting.
-6. A provider answer that is truncated, incomplete, or asks for a tool the
+6. A [blocking check](#blocking-checks) that holds work makes the run wait the
+   same way, once nothing else can run, with
+   `approval: { kind: "blocked", step, title, artifact, artifact_sha256, ask, issues, note, choices: ["release", "discard"], id, revisionsLeft: 0 }`.
+   The only answers are `release` and `discard`, with `seen` and the
+   approval's id as above; `approve`, `revise` and `reject` are refused at a
+   hold, and `release` and `discard` at an approval. `release` lets the run go
+   on and the receipt flags the release. `discard` ends the run as `stopped`
+   with nothing delivered. A release answers one attempt of its check: when the
+   work is redone and the check runs again, the new verdict stands. A runner
+   never delivers a run whose blocking check refused work the owner did not
+   release; such a run fails instead. An agent app acting for the owner is
+   shown the hold's `artifact_sha256` and `issues`, never the held text, and
+   cannot release or discard it.
+7. A provider answer that is truncated, incomplete, or asks for a tool the
    step was not offered stops the run with a reason. Failed page reads, missing
    search and partial pages are recorded as limitations when usable work
    continues.
-7. The reference runner's budgets: 60 model calls and 20 page reads per run, 4
-   searches and 8 model turns per step, 14,000 characters of text per page.
-8. A finished run ends with the deliverable and a [receipt](#8-receipts). A
-   run that was rejected at a decision has no deliverable and no receipt.
+8. The reference runner's budgets: 60 model calls and 20 page reads per run, 4
+   searches and 8 model turns per step, 14,000 characters of text per page,
+   read from at most 1,500,000 bytes of a text, HTML, XML or JSON page.
+9. A finished run ends with the deliverable and a [receipt](#8-receipts). A
+   run that was rejected at a decision has no deliverable and no receipt. A
+   run whose owner discarded held work has no deliverable and a receipt whose
+   `outcome` says so.
 
 A repeated run follows the same declared workflow; model outputs can differ.
 
@@ -478,11 +563,41 @@ A completed run produces a receipt (`kind: factory-run-receipt`, `v: 2`):
 | `inputs_sha256` | hash of the canonical JSON of the accepted inputs |
 | `runner` | mode, provider and model |
 | `started`, `finished` | ISO 8601 times |
-| `steps` | every step attempt: status, tools used and targets, verdicts, limitations, decisions with `seen_sha256` and `approval_id`, `output_sha256`; for reviews, the target hash, reviewer provider and model, input and evidence hashes and the request hash |
-| `flags` | what the person should know: failed checks, unreviewed results, limitations |
+| `steps` | every step attempt: status, tools used and targets, verdicts, limitations, decisions with `seen_sha256`, `approval_id` and `decided_by`, a check's `on_fail`, `output_sha256`; for reviews, the target hash, reviewer provider and model, input and evidence hashes and the request hash |
+| `flags` | what the person should know: failed checks, unreviewed results, limitations, released holds |
 | `outputs` | with declared outputs: each output's hash and review |
-| `deliverable_review` | `passed`, `limited` (a passing review with missing evidence), `flagged`, `unreviewed` or `not_checked`, for the delivered bytes |
-| `deliverable_sha256`, `usage` | hash of the deliverable; model calls and tokens |
+| `members` | for a composition the runner compiled itself: one entry per member (below) |
+| `outcome` | for a run whose owner discarded held work: `{ status: "blocked", by: "owner", step, target, decision: "discard", at, decided_by }` |
+| `deliverable_review` | `passed`, `limited` (a passing review with missing evidence), `flagged`, `unreviewed` or `not_checked`, for the delivered bytes; `flagged` whenever a release of held work is in effect, whatever a later check says; `not_delivered`, with `sha256: null` and no checks, when the owner discarded held work |
+| `deliverable_sha256`, `usage` | hash of the deliverable, `null` when nothing was delivered; model calls and tokens. Each entry of `usage.metering.calls` names the `step` it was made for |
+
+**Decisions.** Every decision record (an approval, and the owner's answer to
+a hold) carries `decided_by: { account, surface }`. `account` is a stable,
+salted hash of the deciding account, never its id, or `null` when no account
+stands behind the decision (a run key). `surface` is how the decision arrived:
+`web`, `mcp`, `a2a`, `email` or `api`. It is a label taken from the request,
+not proof of the channel; who had authority is `account`. No handle or name is
+signed: a reader looks up who an account is while it exists.
+
+**Holds.** The owner's answer to a [blocking check](#blocking-checks) is a
+step of `kind: hold`: `{ status: released | discarded, decision, target, seen_sha256, approval_id, decided_by }`,
+kept apart from the check record it answers, which keeps its `flagged` or
+`unreviewed` status.
+
+**Members.** A runner adds `members` only for a composition it compiled
+itself, from the sources it hashed, and never from a bundle's own
+`composition` block, which anyone can write. Each entry, in compile order:
+`{ id, name, title, version, source_sha256, status, steps, send_backs, released, usage }`.
+`title` is at most 120 characters; `steps` are the member's namespaced step
+ids; `send_backs` counts its check records that read `sent_back`; `released`
+is true when the owner released work one of its blocking checks refused and
+that check has not run again since; `usage` has the shape of a run's `usage`,
+holding the calls made for its steps. A receipt's `status` is `done`,
+`blocked` (held work, discarded or left held when the run ended), `failed`,
+`stopped` or `skipped`; a runner's live view MAY also show `pending`,
+`running` and `waiting`. Receipts keep counters, not prices.
+
+A discarded run's receipt verifies no result: check only its signature.
 
 A runner MAY sign a receipt: `signature: { key_id, sig }`, an Ed25519
 signature (base64url) over the canonical JSON of the receipt without its
@@ -601,6 +716,10 @@ engine on the fixtures in [`fixtures/`](fixtures):
 - `semantic/`: the schemas accept them and the engine rejects them, by a rule
   of [section 6](#6-validity-and-runnability) that a per-file schema cannot
   express, such as a missing agent file or a circle of steps.
+- `unimplemented/`: the schemas accept them and the engine refuses them,
+  because they require an execution feature it does not implement yet
+  (`blocking-checks`). A runner MUST refuse such a flow
+  ([5.1](#51-top-level-fields)); the test checks that this is the only reason.
 - `lenient/`: the engine accepts them and the schemas reject them. The engine
   converts or ignores values the format does not define. The fixtures cover:
   an uppercase `kind`; an unknown input `type` (read as `text`); a number
@@ -609,13 +728,139 @@ engine on the fixtures in [`fixtures/`](fixtures):
   warning); `rounds` outside 0 to 5 (clamped); an uppercase `agent` (read in
   lowercase); `checks` on a `work` step (ignored); a `done` item that YAML
   reads as a map (joined into text); an input given as plain text (read as
-  its help); step names that are not written normalized. The engine also
+  its help); step names that are not written normalized; `on_fail: stop`
+  without `requires: [blocking-checks]` (the engine at this commit ignores
+  `on_fail` with a warning; it will reject this once it implements blocking
+  checks). The engine also
   accepts, without fixtures here, non-boolean `required` and `optional`,
   non-integer `rounds` (the default is used), and scalars where
   `repositories`, `agentDefaults`, `author` or `tags` items belong. Authors
   SHOULD NOT rely on any of these; a future version may reject them.
 
+[`schema/factory.yaml.json`](schema/factory.yaml.json) (JSON Schema draft-07)
+describes the single-file form of [section 13](#13-the-single-file-form); its
+fixtures and test are listed there.
+
 A conforming factory validates against both schemas and passes the rules of
 [section 6](#6-validity-and-runnability). The engine is the reference: where
 this document and the engine disagree, that is a bug in one of them, and an
 issue is welcome.
+
+### Pack v0 additions not yet in the reference engine
+
+The format defines these, and the engine and CLI at this commit do not
+implement them yet:
+
+- [blocking checks](#blocking-checks): `on_fail`, `requires: [blocking-checks]`,
+  the gate on the subject, holds and their answers (section 7, item 6), and the
+  export text. The engine refuses a flow that requires `blocking-checks`, as
+  5.1 requires, and ignores `on_fail` elsewhere with a warning;
+- in [5.9](#59-composition): recording `title`, the nested member-id refusal,
+  carrying `on_fail`, `maxCalls`, `worstCaseCalls`, a member's `rounds` and
+  leaving out a member's tests file;
+- in [section 8](#8-receipts): `members`, `outcome`, `decided_by`, `on_fail`,
+  `kind: hold`, `deliverable_review.status: not_delivered`, the `flagged`
+  status for a release in effect, and `usage.metering.calls[].step`.
+
+The schemas already describe the new fields: `on_fail` (only on a `check`,
+`stop` or `flag`), `blocking-checks` in `requires` (required by any
+`on_fail: stop` check), and `title` (at most 120 characters) in a composition
+source.
+
+[RFC 0020](rfcs/0020-pack-v0.md) is where these additions are discussed and
+decided. It also proposes who paid for a run and for each call (`funding`,
+`payer`), which this document does not define until the RFC settles them.
+
+## 13. The single-file form
+
+Proposed in [RFC 0023](rfcs/0023-single-file-factory.md), a draft, which
+gives the full rules. The reference engine and CLI at this commit do not read
+this form yet: `factoryhub validate` and `factoryhub run` read factory folders
+and bundle `.json` files, and `factoryhub push` refuses a single file.
+
+A whole factory MAY be written as one YAML file, `factory.yaml`, with
+`format: mindburn.factory.yaml/v1` on a line of its own. Each top-level
+section is one file of the bundle, and its text is that file's exact bytes:
+
+```yaml
+# yaml-language-server: $schema=https://raw.githubusercontent.com/Mindburn-Labs/factoryhub/main/spec/schema/factory.yaml.json
+format: mindburn.factory.yaml/v1
+
+factory:
+  schemaVersion: v1alpha1
+  name: one-step-summary
+  description: Writes a three-sentence summary of the text you give it.
+
+flow:
+  flow: mindburn.flow/v1
+  inputs:
+    text:
+      type: longtext
+      help: The text to summarise.
+  steps:
+    summary:
+      agent: writer
+      task: Summarise the text in three sentences.
+
+agents:
+  writer:
+    description: Writes plain summaries.
+    instructions: |
+      You write short, plain summaries. You add nothing the text does not say.
+```
+
+| Section | Bundle file |
+|---|---|
+| `format` | none; required, `mindburn.factory.yaml/v1` |
+| `factory`, `flow` | `factory.yaml`, `flow.yaml` |
+| `agents` | `agents/<name>/agent.md`, one entry per agent: its settings, then `instructions: \|` |
+| `tests`, `requirements`, `package` | `mindburn.tests.yaml`, `mindburn.requirements.yaml`, `mindburn.package.yaml` |
+| `samples` | `examples/<path>`, one entry per file |
+| `files` | every other file, by its path |
+
+- **Sections are files.** The lines of a YAML section, less two spaces of
+  indent, are its file, plus one final line break; comments and formatting
+  inside a section are the file's own bytes. An agent entry is its
+  frontmatter settings, then its body under `instructions: |` (or `|-` for no
+  final line break). A file a block cannot carry byte for byte is one JSON
+  string, its exact text, on its key's line.
+- **The coordinator.** A writer leaves out `agents/foreman/agent.md` while it
+  is exactly what the engine writes from the flow (`withForeman`), and a
+  reader writes it again. An authored or stale foreman is kept as written;
+  `foreman: ~` says the bundle has none.
+- **Presentation.** Comments and blank lines between sections, section order,
+  comments on a section's own line and top-level `x-` keys are no file's
+  content. They never change the bundle or its hash.
+- **Unknown keys.** Any other unknown top-level key is an error at its line,
+  never dropped. Inside a section every key is kept byte for byte, whether a
+  runner reads it or not; `requires` gates features exactly as in
+  [5.1](#51-top-level-fields).
+- **Identity.** A single file is a view of its bundle. Its hash is the
+  [bundle hash](#9-identity-and-the-transfer-bundle) of the bundle it expands
+  to; no byte outside the files counts. Writing a bundle as one file and
+  reading it back MUST give the same bundle.
+- **Size.** A file in `samples` or `files` over 32 KiB (32,768 UTF-8 bytes)
+  MAY be written as a reference, `{ sha256: "<64 hex>", bytes: <n> }`: the
+  same file of the version the text came from. A reader resolves it only
+  against that version, and an editor shows it read-only. The bundle the file
+  expands to keeps the limits of [section 9](#9-identity-and-the-transfer-bundle).
+- **Versions.** `format` versions the layout; `schemaVersion` and `flow` keep
+  versioning their files. A reader MUST refuse a `format` it does not know.
+  A file with a line `format: mindburn.factory.yaml/v1` at column 1 is a
+  single file, not a bundle's `factory.yaml`.
+
+[`schema/factory.yaml.json`](schema/factory.yaml.json) (draft-07) gives
+editors completion, hover text and errors. It is open wherever a file is and
+does not decide validity, which the reader and
+[section 6](#6-validity-and-runnability) do. Its `$id`,
+`https://factory.mindburn.org/schemas/factory.yaml.json`, is its stable
+address on the factoryhub site, which generates the same document. That site
+is in private preview, so point editors at this repository's copy, as the
+example's first line does ([editor setup](../docs/editor-setup.md)).
+
+[`fixtures/single-file/`](fixtures/single-file) holds `valid/` (the schema
+accepts them and each is a valid factory; two are house factories, equal to
+their folders byte for byte), `invalid/` (the schema rejects each at the place
+it names) and `reader/` (the schema accepts them, and the rules above reject
+them). [`test/single-file.test.mjs`](test/single-file.test.mjs) checks them,
+and checks the reference engine against them once it reads this form.