@osolmaz/pi-workflows 0.7.0 → 0.8.0

package/docs/JOB_PROGRESS.md

@@ -0,0 +1,119 @@
+# Durable job progress
+
+`@osolmaz/pi-workflows/job-progress` lets a remote job publish progress that a Pi Workflows monitor can validate and measure. It uses the existing `pi-workflows.progress.v1` track contract and ETA estimator.
+
+## Report progress
+
+Create one reporter for one physical job. Inject the storage write so the package remains independent of a cloud provider.
+
+```ts
+import { createJobProgressReporter } from "@osolmaz/pi-workflows/job-progress";
+
+const reporter = createJobProgressReporter({
+  application: "example",
+  component: "batch-worker",
+  jobId: process.env.JOB_ID ?? "local",
+  sourceRevision: "abc123",
+  contractHash: "def456",
+  startedAt: new Date().toISOString(),
+  initialTracks: [
+    {
+      key: "overall",
+      data: {
+        schema: "pi-workflows.progress.v1",
+        status: "running",
+        phase: "starting",
+      },
+    },
+  ],
+  minimumIntervalMs: 30_000,
+  publishTimeoutMs: 15_000,
+  publish: async (snapshot, signal) => {
+    await bucket.writeText(progressPath, JSON.stringify(snapshot), { signal });
+  },
+});
+
+await reporter.report({
+  phase: "processing",
+  tracks: [
+    {
+      key: "records",
+      data: {
+        schema: "pi-workflows.progress.v1",
+        status: "running",
+        phase: "processing",
+        completed: 400,
+        total: 1_000,
+        unit: "records",
+      },
+    },
+  ],
+});
+```
+
+The first update, each phase change, and each terminal update publishes immediately. Other updates are coalesced until `minimumIntervalMs` has passed. Call `flush()` at a durable checkpoint or before exit when the current snapshot has not been published.
+
+On process restart, read and validate the existing snapshot and pass it as `previousSnapshot`. The reporter continues its sequence number and rejects an identity, deadline, or terminal-state mismatch.
+
+The reporter keeps the latest snapshot after a write failure. The application decides when to log and retry that failure. A timed-out storage write remains serialized until the underlying write settles, so an older write cannot overwrite a newer snapshot. Storage adapters must honor the abort signal and settle after cancellation. A progress failure must not replace receipt or checkpoint validation.
+
+## Finish a job
+
+A terminal update gets a finish timestamp and publishes immediately:
+
+```ts
+await reporter.report({
+  state: "completed",
+  phase: "complete",
+  tracks: [
+    {
+      key: "records",
+      data: {
+        schema: "pi-workflows.progress.v1",
+        status: "completed",
+        phase: "complete",
+        completed: 1_000,
+        total: 1_000,
+        unit: "records",
+      },
+    },
+  ],
+});
+```
+
+A terminal snapshot is not a receipt. Receipts, manifests, hashes, and durable application outputs remain authoritative.
+
+## Store and discover snapshots
+
+Write one mutable snapshot per physical job:
+
+```text
+<existing-bucket>/<application-prefix>/runs/<job-id>/progress.json
+```
+
+Add these immutable labels to the job or schedule:
+
+```text
+progress_schema=pi-workflows.job-progress.v1
+progress_bucket=<existing-bucket>
+progress_prefix=<application-prefix>/runs
+```
+
+A monitor reads the labels, forms `<progress_prefix>/<job-id>/progress.json`, validates the snapshot with `validateJobProgressSnapshot`, and publishes its tracks with stable keys. It should retain consecutive snapshots so Pi Workflows can estimate a conservative ETA from measured rates.
+
+## Estimate from snapshots
+
+```ts
+import { estimateJobProgress } from "@osolmaz/pi-workflows/job-progress";
+
+const result = estimateJobProgress(previousSnapshots);
+for (const estimate of result.estimates) {
+  console.log(estimate.key, estimate.remainingMedianMs);
+}
+```
+
+The estimator returns no measured ETA until a track has a known total and enough positive progress samples. A source-provided `sourceEstimatedFinishAt` remains available through the normal progress contract.
+
+## Data boundary
+
+Snapshots may contain identifiers, phases, counters, totals, timestamps, and cost totals. Do not put credentials, environment values, input records, model responses, logs, or private content in a snapshot. The strict validator rejects unknown fields and limits the encoded snapshot to 64 KiB.

package/docs/MONITOR.md

@@ -242,6 +242,11 @@ One monitor can track several processes. Each uses a stable progress key. A miss
 
 The progress estimator treats each key independently. A phase, unit, total, or counter reset in one track does not reset another track.
 
+Remote Jobs can publish the same tracks in a strict
+[`pi-workflows.job-progress.v1` snapshot](JOB_PROGRESS.md). A monitor discovers
+the snapshot from immutable Job labels, validates its identity, and submits its
+tracks without copying log-derived guesses into progress fields.
+
 ## Interval and lifetime
 
 The normal workflow engine remains finite. The built-in monitor therefore retains the 1,000-check safety ceiling and must not claim to be mathematically unbounded.

package/docs/plans/2026-08-17-bundled-skills-plan.md

@@ -99,6 +99,6 @@ git diff --check
 Run in Tools after deletion:
 
 ```bash
-python3 agents/sync-skills.py monitor
+python3 agents/sync-skills.py
 npx -y @simpledoc/simpledoc check
 ```

package/docs/plans/2026-08-17-durable-job-progress-plan.md

@@ -0,0 +1,176 @@
+# Durable job progress
+
+## Problem
+
+Long-running remote jobs can expose state only through logs and final receipts. A monitor can confirm that a job is running, but it cannot give a reliable progress value or remaining time when the job does not publish a completed count, a total, and source timestamps.
+
+xTap Pool and OurModels need the same durable progress contract. The contract must work with their existing Hugging Face Buckets, survive worker restarts, and use the ETA estimator that Pi Workflows already uses for `pi-workflows.progress.v1` tracks.
+
+## Requirements
+
+The implementation must:
+
+- add one storage-neutral job progress API to `@osolmaz/pi-workflows`
+- reuse `pi-workflows.progress.v1` for progress tracks
+- write one mutable snapshot for each physical job in the application's existing Bucket
+- let monitors discover the snapshot from immutable job labels
+- let Pi Workflows validate the snapshot and estimate remaining time from repeated samples
+- let xTap Pool report restore, review, recovery, and publication progress
+- let OurModels report discovery, processing, cache, and publication progress
+- keep receipts, content hashes, manifests, and databases authoritative
+- avoid secrets, post text, model output, and other private content in progress snapshots
+- preserve one physical enrichment job at a time during the xTap Pool change
+- leave the OurModels replacement schedule suspended until a paid run is separately approved
+
+## Non-goals
+
+This work does not add:
+
+- a new remote store
+- a metrics database
+- a second ETA protocol
+- a Pi core change
+- a service or daemon
+- a compatibility path for an older snapshot schema
+- automatic authority to retry, deploy, publish, or spend money
+
+## Public contract
+
+The npm package exports a new subpath:
+
+```ts
+import {
+  createJobProgressReporter,
+  estimateJobProgress,
+  validateJobProgressSnapshot,
+  type JobProgressSnapshot,
+} from "@osolmaz/pi-workflows/job-progress";
+```
+
+A snapshot has schema `pi-workflows.job-progress.v1` and contains:
+
+- stable application and component names
+- the physical job identifier
+- source and work-contract identifiers
+- a monotonic sequence number
+- job state and current phase
+- start, update, optional deadline, and optional finish timestamps
+- one or more keyed `pi-workflows.progress.v1` tracks
+- optional settled cost and active reservation facts
+
+The validator is strict. It rejects unknown fields, duplicate track keys, invalid timestamps, non-finite values, invalid progress tracks, and oversized snapshots.
+
+The reporter accepts an injected asynchronous `publish(snapshot)` callback. Pi Workflows does not import a Hugging Face client. The reporter:
+
+- keeps sequence numbers monotonic within the process
+- rejects regressions within one phase and epoch
+- coalesces frequent updates with a configurable minimum interval
+- flushes phase changes and terminal states immediately
+- bounds publication time with an abort deadline
+- preserves the most recent unsent snapshot after a transient publication failure
+- never includes arbitrary metadata or environment values
+
+A terminal snapshot is operational evidence only. The application's receipt and durable output validation still decide whether work succeeded.
+
+## Storage and discovery
+
+Each application writes snapshots to its existing Bucket.
+
+xTap Pool uses:
+
+```text
+osolmaz/xtap-pool-bucket/operations/enrichment/runs/<job-id>/progress.json
+```
+
+OurModels uses:
+
+```text
+osolmaz/ourmodels-data/<prefix>/operations/community-posts/runs/<job-id>/progress.json
+```
+
+Each schedule supplies these immutable labels:
+
+```text
+progress_schema=pi-workflows.job-progress.v1
+progress_bucket=<bucket>
+progress_prefix=<path-before-job-id>
+```
+
+A monitor reads the labels, appends the physical job identifier and `progress.json`, reads the snapshot with existing local Hugging Face authentication, validates it, and publishes each track under a stable workflow progress key. The monitor does not trust an ETA string from logs. It uses source finish time when the snapshot provides one, or the existing conservative estimator after enough measured samples.
+
+## xTap Pool tracks
+
+The enrichment worker reports these stable tracks when facts are measurable:
+
+| Key                | Unit       | Source                                             |
+| ------------------ | ---------- | -------------------------------------------------- |
+| `database-restore` | bytes      | downloaded database bytes and expected object size |
+| `registry-replay`  | events     | replayed registry events and discovered total      |
+| `registry-scan`    | candidates | durable scan cursor and fixed candidate total      |
+| `queue`            | records    | terminal records and durable queue total           |
+| `publication`      | bytes      | uploaded and verified database bytes               |
+| `overall`          | phases     | completed phases and fixed phase count             |
+
+Phase-only states remain valid when a total is not yet known. The worker must not invent totals.
+
+The existing three unresolved records must receive durable outcomes under the fixed full-response deadline or become exactly validated blocked records. The worker then publishes and verifies the index before its final receipt is accepted.
+
+## OurModels tracks
+
+The community-posts worker reports these stable tracks when facts are measurable:
+
+| Key               | Unit    | Source                                          |
+| ----------------- | ------- | ----------------------------------------------- |
+| `model-discovery` | models  | discovered and inspected model count            |
+| `community-posts` | models  | processed models and fixed discovered total     |
+| `cache`           | records | durable cached model results and expected total |
+| `publication`     | bytes   | uploaded and verified artifact bytes            |
+| `overall`         | phases  | completed phases and fixed phase count          |
+
+The worker continues to use its existing receipt and manifest rules. A progress write failure must not corrupt a useful checkpoint or published result.
+
+## Delivery sequence
+
+1. Add and release `@osolmaz/pi-workflows/job-progress` as version `0.8.0`.
+2. Add snapshot discovery guidance to the bundled monitor skill.
+3. Update xTap Pool to use `0.8.0`, add measured callbacks, and add schedule labels.
+4. Merge and deploy xTap Pool, then replace the old suspended schedule.
+5. End the old physical job only when the replacement source is ready and the one-job rule can be preserved.
+6. Start one instrumented xTap Pool recovery job under the existing restoration budget.
+7. Update OurModels to use `0.8.0`, add measured callbacks, and replace old schedules with one suspended instrumented schedule.
+8. Do not start a paid OurModels run until its measured cost range and ceiling receive the required approval.
+9. Start a Pi monitor that reads both progress surfaces and displays current progress and ETA.
+
+## Acceptance checks
+
+Pi Workflows must pass:
+
+```bash
+npm run check
+npm run test:e2e
+npx slophammer-ts@latest dry .
+npx slophammer-ts@latest check . --only ts.dependency-boundaries-required
+```
+
+Tests must cover strict validation, unknown fields, duplicate keys, monotonic updates, phase resets, coalescing, transient publication failure, deadline abort, terminal flush, and ETA estimation from snapshots.
+
+xTap Pool must pass its repository checks, including mutation testing. A live recovery must show a valid snapshot in the existing index Bucket, and repeated monitor samples must produce a measured ETA when a track has a known total and positive progress.
+
+OurModels must pass:
+
+```bash
+npm run check
+npm run coverage
+npm run dry
+npm run mutate
+node scripts/test-bounds-engine.mjs
+node scripts/validate-model-data.mjs
+```
+
+Its replacement schedule must remain suspended with only the approved two secrets until a paid run is authorized.
+
+## Recovery
+
+If a progress publication fails, the worker keeps useful work and retries only the latest snapshot at the next bounded update. If the job ends before that succeeds, the receipt and durable outputs remain authoritative and the monitor marks progress stale.
+
+If a repeated deterministic worker defect appears, stop the affected job and schedule. Preserve all existing Bucket objects and return the exact failing phase, source revision, snapshot, receipt state, and last valid durable outputs.

package/docs/workflows.md

@@ -369,6 +369,16 @@ and resume later by running that wait again from the beginning.
 A monitor uses the session's single active workflow slot. It does not provide
 cron syntax, calendar scheduling, OS notifications, or a background service.
 
+### Durable remote Job progress
+
+Remote workers can publish the same progress tracks through
+`@osolmaz/pi-workflows/job-progress`. The storage-neutral reporter writes a
+strict, versioned snapshot through an application-supplied callback. A monitor
+can read consecutive snapshots from an existing remote store and use the normal
+progress estimator without treating logs as durable state. See
+[JOB_PROGRESS.md](JOB_PROGRESS.md) for the API, snapshot schema, restart rules,
+and discovery labels.
+
 ## The step contract
 
 Every `agent` prompt ends with a step contract block naming the workflow, the

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@osolmaz/pi-workflows",
-  "version": "0.7.0",
+  "version": "0.8.0",
   "description": "Workflow and controller runtime with a live terminal viewer for the pi coding agent",
   "keywords": [
     "pi-package"
@@ -39,6 +39,10 @@
     "./controllers": {
       "types": "./dist/controllers/index.d.ts",
       "default": "./dist/controllers/index.js"
+    },
+    "./job-progress": {
+      "types": "./dist/job-progress/index.d.ts",
+      "default": "./dist/job-progress/index.js"
     }
   },
   "publishConfig": {

package/skills/monitor/SKILL.md

@@ -95,6 +95,18 @@ Submit observed facts. The workflow computes rates, confidence, remaining work,
 
 For several concurrent processes, publish one stable track per process. The Pi widget and viewers show them separately and keep each ETA independent.
 
+### Read durable job snapshots
+
+A remote job can advertise a `pi-workflows.job-progress.v1` snapshot with these immutable labels:
+
+- `progress_schema=pi-workflows.job-progress.v1`
+- `progress_bucket=<existing-bucket>`
+- `progress_prefix=<path-before-job-id>`
+
+When these labels exist, form `<progress_prefix>/<physical-job-id>/progress.json`, read it from the named existing store with an already authorized credential, and validate it with the installed `@osolmaz/pi-workflows/job-progress` API. Reject a snapshot whose job ID, source revision, or work-contract identity does not match the observed Job. Do not follow a path or credential named inside unvalidated data.
+
+Publish the snapshot tracks under stable keys. Preserve consecutive samples so the workflow can estimate ETA from measured progress. Treat a stale or missing snapshot as unavailable progress, not as proof that the Job stopped. Receipts, hashes, manifests, and final outputs remain the completion evidence.
+
 ## Apply finish rules
 
 ### Still active

package/src/job-progress/index.ts

@@ -0,0 +1,24 @@
+export {
+  createJobProgressReporter,
+  estimateJobProgress,
+  type JobProgressReporter,
+} from "./reporter.js";
+export {
+  isTerminalJobProgressState,
+  MAX_JOB_PROGRESS_BYTES,
+  MAX_JOB_PROGRESS_TRACKS,
+  validateJobProgressSnapshot,
+} from "./validation.js";
+export {
+  JOB_PROGRESS_SCHEMA,
+  type JobProgressCost,
+  type JobProgressEstimate,
+  type JobProgressIdentity,
+  type JobProgressPublish,
+  type JobProgressPublishResult,
+  type JobProgressReporterOptions,
+  type JobProgressSnapshot,
+  type JobProgressState,
+  type JobProgressTrack,
+  type JobProgressUpdate,
+} from "./types.js";