@toon-protocol/rig 4.6.0 → 4.7.0

package/README.md

@@ -459,10 +459,58 @@ rig ci serve --relay wss://<relay> --repo <owner-npub>/<repo-id> [--repo …]
 | `--gateway <url>` | `RIG_ARWEAVE_GATEWAY`, else rig's preferred Arweave gateway | the gateway that serves the store's raw bytes: prefix for log/artifact URLs (`<gateway>/raw/<txId>`) **and** the first gateway the coordinator reads a commit's objects from (same `/raw/` route), ahead of the shared public list — name a private or local gateway (the sandbox's) and the coordinator needs nothing else to read |
 | `--act-bin <path>` | `act` on PATH (or `RIG_ACT_BIN`) | the act binary |
 | `--platform <label>=<image>` | `ubuntu-latest=catthehacker/ubuntu:act-latest` | `runs-on` label → Docker image (repeatable) |
+| `--pull` / `--no-pull` | act's default (pull before each run) | whether act pulls the runner image named by `--platform` before each run |
 | `--workdir <dir>` | `<state-dir>/work` | where commits are materialized |
 | `--once` | | stop after the first run concludes (its 9842 and final 39842 are on the relay) — one `rig ci trigger` answered by one bounded serve; ignored or refused triggers do not count |
 | `--json` | | one JSON document with the coordinator, relay, repos and state dir on start |
 
+**Two act behaviours decide whether a real workflow passes**, and a workflow
+whose steps only `echo` hides both:
+
+- **Write `actions/checkout@v4`, exactly as you would on GitHub.** act copies
+  the materialized tree into the job container *at that step*, not when the
+  container starts, and short-circuits it to a local `docker cp` — so the step
+  needs no network and no node in the image. A workflow without it
+  runs its tools against an **empty** container — the steps execute, the files
+  are not there:
+
+  ```
+  error: could not find `Cargo.toml` in `/…/work/<repo>-<sha>-1` or any parent directory
+  ```
+
+  That path is the checkout the coordinator materialized under `--workdir`
+  (`<repo-id>-<short sha>-<n>`), as the job container sees it.
+- **The default image is not GitHub's hosted runner.** `ubuntu-latest` maps to
+  `catthehacker/ubuntu:act-latest`, which carries no language toolchain, so a
+  workflow that expects a preinstalled `cargo`, `go` or JDK fails at its first
+  step. Either point `--platform` at an image that has one
+  (`--platform ubuntu-latest=rust:1-bookworm`) or install it in a step. The
+  official `rust` images ship neither rustfmt nor clippy —
+  `rustup component add rustfmt clippy` is a step, not a given — and an image
+  you built locally needs `--no-pull` (below).
+
+A workflow that survives both, run by a coordinator started with
+`--platform ubuntu-latest=rust:1-bookworm`:
+
+```yaml
+# .github/workflows/ci.yml
+name: ci
+on: [push]
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4 # act resolves this to the local copy
+      - run: rustup component add rustfmt clippy
+      - run: cargo fmt --check
+      - run: cargo clippy -- -D warnings
+      - run: cargo test
+```
+
+A runner image built or loaded on the coordinator host and never pushed to a
+registry needs `--no-pull`: without it act force-pulls the image named by
+`--platform` and every run fails in *Set up job* with `pull access denied`.
+
 On start the coordinator publishes an **Advertisement** (19843: runner family
 `act`, its selectors, admission `maintainer-request`, execution
 `request-required`, billing `out-of-band`, and a fresh `secrets-key`) and renews
@@ -510,9 +558,10 @@ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v rig-ci-state:/st
 The entrypoint is `rig`, so every verb works (`docker run --rm rig-ci --version`),
 and every environment variable rig honours means the same inside the image
 (`TOON_CLIENT_HOME` is `/state`). act launches each job as a container on the
-daemon behind the mounted socket and copies the checkout in, so no host path
-has to match. Against the [sandbox](#dogfooding-rigs-own-ciyml-on-the-sandbox)
-on the same host, run on the host network with the sandbox's own `localhost`
+daemon behind the mounted socket and copies the checkout in at the
+`actions/checkout` step, so no host path has to match. Against the
+[sandbox](#dogfooding-rigs-own-ciyml-on-the-sandbox) on the same host, run on
+the host network with the sandbox's own `localhost`
 addresses — a client dials the endpoint a node **advertises**, and the sandbox
 hub advertises `http://127.0.0.1:3200/ilp` so host-run smokes can reach it
 (sandbox-only; a real connector advertises its public URL, which the default
@@ -565,6 +614,11 @@ git commit --allow-empty -m 'ci: dogfood' && rig push --yes
 rig ci status HEAD --json | jq .summary
 ```
 
+`RIG_ARWEAVE_GATEWAY` in that common env also steers the READ path: `rig clone`
+and `rig fetch` try `http://localhost:3000/raw/<txId>` first, so a repo pushed to
+the sandbox clones back out of it even though the public gateways never see those
+objects (see [Reading through your own gateway](#reading-through-your-own-gateway)).
+
 The same flow, end to end and asserted, is
 `src/__integration__/ci-sandbox.integration.test.ts`; it runs only with
 `RIG_CI_SANDBOX=1` and skips itself unless the sandbox's relay, hub, store edge
@@ -774,6 +828,37 @@ clone never leaves a partial repo. `rig fetch [remote]` is the same pipeline as
 delta: only locally-missing objects are downloaded, and `refs/remotes/<remote>/*`
 (tags → `refs/tags/*`) move with a `git fetch`-style report.
 
+### Reading through your own gateway
+
+By default both commands read from rig's shared public gateway list. A repository
+whose objects live only on a **private, local or air-gapped** gateway is invisible
+there — so `--gateway <url>` (or `RIG_ARWEAVE_GATEWAY`, the same variable `rig push`,
+`rig site` and `rig ci serve` honour) puts that gateway **first**, on the store's
+raw-bytes route `<url>/raw/<txId>`, with the public list kept behind it as the
+fallback. It is the read-path twin of `rig ci serve --gateway`.
+
+```sh
+export RIG_ARWEAVE_GATEWAY=http://localhost:3000   # the dev sandbox's gateway
+rig clone ws://localhost:7100 <owner-npub>/<repo-id>
+rig fetch                                          # or: rig fetch --gateway http://localhost:3000
+```
+
+`/raw/` and not `/<txId>`: an ar.io node serves raw transaction bytes there whether
+or not it does sandboxed-subdomain redirects, and the TOON store's own gateway
+answers on that route alone. Public gateways serve the same bytes on both, so with
+nothing configured the list is exactly as it always was.
+
+The **`Git-SHA` lookup follows the gateway too**. A kind:30618 object map is capped,
+so a SHA it does not carry is resolved by asking a gateway's GraphQL endpoint which
+transaction carries that object — and a configured gateway is asked at
+`<url>/graphql`, never `arweave.net` (which would answer about a *different*
+network, or leak the query off an air-gapped stack). Object bytes are
+content-addressed and SHA-verified, so any mirror may serve them; a tag query is a
+statement about one network, so it goes only to the network you named. Configure
+nothing and it is the shared `arweave.net` resolver, unchanged. A gateway that does
+not serve GraphQL therefore resolves nothing — name one that does (every ar.io node
+does), or leave the variable unset.
+
 `rig issue list|show` and `rig pr list|show` are pure relay reads (kind:1621/1617 by
 the repo `#a` tag; state from kind:1630-1633, latest wins; comments under `show` —
 NIP-22 kind:1111 matched on the uppercase `E` root tag, merged with legacy kind:1622

package/dist/{chunk-DCGV6ALH.js → chunk-3IVO2MSZ.js}

@@ -12,7 +12,7 @@ import {
   parseMaintainers,
   queryRelay,
   resolveIdentity
-} from "./chunk-JBFBD7HV.js";
+} from "./chunk-S3KKQ4IC.js";
 import {
   defaultDaemonPort,
   standaloneForced,
@@ -1677,10 +1677,16 @@ var CI_MANUAL_TRIGGER_KIND = 9840;
 var CI_JOB_RESULT_KIND = 9841;
 var CI_WORKFLOW_RESULT_KIND = 9842;
 var CI_WORKFLOW_PROGRESS_KIND = 39842;
+var CI_LIVE_LOG_TAIL_KIND = 39841;
 var CI_RUNNER_FAMILY = "act";
 var CI_SOFTWARE = "rig";
 var CI_MAX_ADVERTISEMENT_TTL = 30 * 60;
 var CI_MAX_PROGRESS_TTL = 30 * 60;
+var CI_MAX_LIVE_LOG_TAIL_TTL = 30 * 60;
+var CI_LIVE_LOG_TAIL_INTERVAL_MS = 1e4;
+var CI_LIVE_LOG_TAIL_JOB_BYTES = 16 * 1024;
+var CI_LIVE_LOG_TAIL_MAX_BYTES = 64 * 1024;
+var CI_RUNNER_CHANNEL_KEY = "~runner";
 var REPO_ANNOUNCEMENT_KIND = 30617;
 var CONCLUSIONS = /* @__PURE__ */ new Set([
   "success",
@@ -2412,6 +2418,138 @@ function parseCiWorkflowProgress(ev) {
   if (provenance) result.provenance = provenance;
   return result;
 }
+function liveLogTailBudget(entries) {
+  if (!Number.isFinite(entries) || entries <= 1) {
+    return CI_LIVE_LOG_TAIL_JOB_BYTES;
+  }
+  const share = Math.floor(CI_LIVE_LOG_TAIL_MAX_BYTES / entries);
+  return Math.max(1, Math.min(CI_LIVE_LOG_TAIL_JOB_BYTES, share));
+}
+function liveLogTailEventBudget(runTimeoutMs) {
+  if (!Number.isFinite(runTimeoutMs) || runTimeoutMs <= 0) return 1;
+  return Math.ceil(runTimeoutMs / CI_LIVE_LOG_TAIL_INTERVAL_MS) + 1;
+}
+function sliceLogTail(text, maxBytes) {
+  const bytes = Buffer.from(text, "utf8");
+  if (bytes.length <= maxBytes) return { tail: text, omitted: 0 };
+  if (maxBytes <= 0) return { tail: "", omitted: bytes.length };
+  let start = bytes.length - maxBytes;
+  while (start < bytes.length && (bytes[start] & 192) === 128) {
+    start += 1;
+  }
+  return { tail: bytes.subarray(start).toString("utf8"), omitted: start };
+}
+function assertOmitted(omitted, what) {
+  if (!Number.isSafeInteger(omitted) || omitted < 0) {
+    throw new Error(`${what}: omitted must be a non-negative byte count`);
+  }
+}
+function buildCiLiveLogTail(input, createdAt = now()) {
+  if (input.runId === "") throw new Error("a live log tail needs a run id");
+  assertExpiration(createdAt, input.expiresAt, CI_MAX_LIVE_LOG_TAIL_TTL);
+  const seen = /* @__PURE__ */ new Set();
+  for (const job of input.jobs) {
+    if (job.job === "") throw new Error("a live log tail job needs a job id");
+    if (job.job === CI_RUNNER_CHANNEL_KEY) {
+      throw new Error(
+        `the runner channel is not a job: ${CI_RUNNER_CHANNEL_KEY} may not appear in jobs`
+      );
+    }
+    if (seen.has(job.job)) {
+      throw new Error(`job ${job.job} appears twice in one live log tail`);
+    }
+    seen.add(job.job);
+    assertOmitted(job.omitted, `job ${job.job}`);
+  }
+  if (input.runner) assertOmitted(input.runner.omitted, "runner channel");
+  const content = {
+    jobs: input.jobs.map((j) => ({
+      job: j.job,
+      tail: j.tail,
+      omitted: j.omitted
+    }))
+  };
+  if (input.runner) {
+    content.runner = {
+      tail: input.runner.tail,
+      omitted: input.runner.omitted
+    };
+  }
+  return {
+    kind: CI_LIVE_LOG_TAIL_KIND,
+    content: JSON.stringify(content),
+    tags: [
+      ...commonTriggerTags(input.trigger),
+      ["d", input.runId],
+      ["expiration", String(input.expiresAt)]
+    ],
+    created_at: createdAt
+  };
+}
+function isRecord(value) {
+  return typeof value === "object" && value !== null && !Array.isArray(value);
+}
+function parseOmitted(value) {
+  if (value === void 0) return 0;
+  if (!Number.isSafeInteger(value) || value < 0) return null;
+  return value;
+}
+function parseTailChannel(value) {
+  if (!isRecord(value)) return null;
+  const omitted = parseOmitted(value["omitted"]);
+  const tail = value["tail"];
+  if (typeof tail !== "string" || omitted === null) return null;
+  return { tail, omitted };
+}
+function parseCiLiveLogTail(ev) {
+  if (ev.kind !== CI_LIVE_LOG_TAIL_KIND) return null;
+  const { tags } = ev;
+  const trigger = parseCiTriggerContext(tags);
+  if (!trigger) return null;
+  const d = single(tags, "d");
+  const exp = single(tags, "expiration");
+  if (!d || !d[1] || !exp) return null;
+  const expiresAt = Number(exp[1]);
+  if (!Number.isInteger(expiresAt) || expiresAt <= ev.created_at || expiresAt - ev.created_at > CI_MAX_LIVE_LOG_TAIL_TTL) {
+    return null;
+  }
+  let content;
+  try {
+    content = JSON.parse(ev.content);
+  } catch {
+    return null;
+  }
+  if (!isRecord(content) || !Array.isArray(content["jobs"])) return null;
+  const jobs = [];
+  const seen = /* @__PURE__ */ new Set();
+  for (const entry of content["jobs"]) {
+    if (!isRecord(entry)) return null;
+    const job = entry["job"];
+    if (typeof job !== "string" || job === "") return null;
+    if (job === CI_RUNNER_CHANNEL_KEY || seen.has(job)) return null;
+    const channel = parseTailChannel(entry);
+    if (!channel) return null;
+    seen.add(job);
+    jobs.push({ job, tail: channel.tail, omitted: channel.omitted });
+  }
+  let runner;
+  if (content["runner"] !== void 0) {
+    const parsed = parseTailChannel(content["runner"]);
+    if (!parsed) return null;
+    runner = parsed;
+  }
+  const result = {
+    eventId: ev.id,
+    pubkey: ev.pubkey.toLowerCase(),
+    createdAt: ev.created_at,
+    trigger,
+    runId: d[1],
+    jobs,
+    expiresAt
+  };
+  if (runner) result.runner = runner;
+  return result;
+}
 
 // src/ci/secrets.ts
 import { v2 as nip44 } from "nostr-tools/nip44";
@@ -2651,7 +2789,7 @@ var WORKFLOW_FILE_RE = /\.ya?ml$/i;
 function sha256Hex(bytes) {
   return createHash2("sha256").update(bytes).digest("hex");
 }
-function isRecord(value) {
+function isRecord2(value) {
   return value !== null && typeof value === "object" && !Array.isArray(value);
 }
 function stringList(value) {
@@ -2662,7 +2800,7 @@ function stringList(value) {
   return void 0;
 }
 function refFilters(value) {
-  if (!isRecord(value)) return {};
+  if (!isRecord2(value)) return {};
   const out = {};
   const branches = stringList(value["branches"]);
   const tags = stringList(value["tags"]);
@@ -2680,17 +2818,17 @@ function parseTriggers(on) {
   };
   if (typeof on === "string") enable(on, void 0);
   else if (Array.isArray(on)) for (const name of on) enable(name, void 0);
-  else if (isRecord(on)) {
+  else if (isRecord2(on)) {
     for (const [name, options] of Object.entries(on)) enable(name, options);
   }
   return triggers;
 }
 function parseJobs(jobs) {
-  if (!isRecord(jobs)) return [];
+  if (!isRecord2(jobs)) return [];
   const out = [];
   for (const [id, spec] of Object.entries(jobs)) {
     const job = { id };
-    if (isRecord(spec)) {
+    if (isRecord2(spec)) {
       if (typeof spec["name"] === "string") job.name = spec["name"];
       const runsOn = stringList(spec["runs-on"]);
       if (runsOn) job.runsOn = runsOn;
@@ -2716,7 +2854,7 @@ function parseWorkflow(path, content) {
       parseError: err instanceof Error ? err.message : String(err)
     };
   }
-  if (!isRecord(doc)) {
+  if (!isRecord2(doc)) {
     return { ...base, parseError: "workflow is not a YAML mapping" };
   }
   const on = doc["on"] ?? doc["true"];
@@ -4098,10 +4236,16 @@ export {
   CI_JOB_RESULT_KIND,
   CI_WORKFLOW_RESULT_KIND,
   CI_WORKFLOW_PROGRESS_KIND,
+  CI_LIVE_LOG_TAIL_KIND,
   CI_RUNNER_FAMILY,
   CI_SOFTWARE,
   CI_MAX_ADVERTISEMENT_TTL,
   CI_MAX_PROGRESS_TTL,
+  CI_MAX_LIVE_LOG_TAIL_TTL,
+  CI_LIVE_LOG_TAIL_INTERVAL_MS,
+  CI_LIVE_LOG_TAIL_JOB_BYTES,
+  CI_LIVE_LOG_TAIL_MAX_BYTES,
+  CI_RUNNER_CHANNEL_KEY,
   isCiConclusion,
   repoAddress,
   parseRepoAddress,
@@ -4124,6 +4268,11 @@ export {
   parseCiWorkflowResult,
   buildCiWorkflowProgress,
   parseCiWorkflowProgress,
+  liveLogTailBudget,
+  liveLogTailEventBudget,
+  sliceLogTail,
+  buildCiLiveLogTail,
+  parseCiLiveLogTail,
   SECRET_NAME_RE,
   RESERVED_SECRET_NAMES,
   MAX_SECRET_NAMES,
@@ -4174,4 +4323,4 @@ export {
   assembleCiStatus,
   runCiStatus
 };
-//# sourceMappingURL=chunk-DCGV6ALH.js.map
+//# sourceMappingURL=chunk-3IVO2MSZ.js.map