settld 0.1.2 → 0.2.0

package/docs/OVERVIEW.md

@@ -0,0 +1,190 @@
+# Settld overview
+
+Settld (as shipped in this repo) is **two products** that deliberately share the same “truth engine”:
+
+1. **Settld Protocol (open)**: a cryptographically verifiable artifact protocol (bundles + manifests + attestations + receipts) that can be verified offline by someone who does not trust the producer.
+2. **Settld Verify Cloud (commercial)**: a hosted workflow controller (“Magic Link”) that runs the same verifier server-side and turns verifiable artifacts into approvals, inbox views, exports, and automation hooks.
+
+The core design principle is: the hosted product must never be “the only judge.” Everything it shows should be reproducible offline using the open verifier + explicit trust anchors.
+
+## What Settld solves
+
+Delegated autonomous work (agents, automation services, and human-assisted workflows) produces disputes because evidence is messy and non-portable:
+
+- “Prove the workflow actually completed under agreed terms.”
+- “We’re withholding payment until evidence and settlement terms are clear.”
+- “SLA breach—show deterministic evidence and evaluation outputs.”
+
+Settld makes the invoice and its evidence a self-contained, verifiable bundle:
+
+- Evidence artifacts are committed by hashes (integrity).
+- Pricing terms can be buyer-approved by signature (authorization over terms).
+- Invoice math is recomputable and deterministic (no “trust me” totals).
+- A verifier can later prove pass/fail under explicit policy (strict vs compat).
+- Verify Cloud makes this usable by buyers without requiring installs.
+
+## The end-to-end artifact story (protocol truth)
+
+A producer emits a bundle directory tree (or a zip of that tree).
+
+The bundle includes:
+
+- `manifest.json`: commits to a list of file paths + hashes, excluding `verify/**` (no circular hashing).
+- `attestation/bundle_head_attestation.json`: binds to the manifest hash.
+- Optional `verify/verification_report.json`: a signed receipt that is not listed in the manifest but is cryptographically bound (by hashes) to the manifest and head attestation.
+
+A verifier later:
+
+- recomputes file hashes,
+- recomputes manifest hash using canonical JSON rules,
+- validates signatures,
+- enforces governance and trust anchors,
+- returns deterministic machine output with stable warning/error codes.
+
+## Protocol truth sources (what “counts”)
+
+When docs disagree, the contract is:
+
+1. `docs/spec/` (human spec)
+2. `docs/spec/schemas/` (JSON Schemas)
+3. `test/fixtures/` (fixture corpus) + `conformance/v1/` (language-agnostic oracle)
+4. the reference verifier implementations (Node + Python), as constrained by conformance
+
+## Bundle kinds implemented
+
+These are “directory-level protocols” with distinct manifest rules and verification logic:
+
+- Proof bundles
+  - JobProofBundle.v1
+  - MonthProofBundle.v1
+- Finance pack
+  - FinancePackBundle.v1
+- Invoice bundle (work → terms → metering → claim)
+  - InvoiceBundle.v1 embeds JobProof under `payload/job_proof_bundle/**`
+- ClosePack (pre-dispute “wedge pack”)
+  - ClosePack.v1 embeds an InvoiceBundle and adds deterministic recomputable indexing/evaluation surfaces for self-serve dispute resolution (evidence index + optional SLA/acceptance evaluation surfaces).
+
+## Toolchain CLIs
+
+- `settld-produce`: deterministic bundle production (JobProof/MonthProof/FinancePack/InvoiceBundle/ClosePack).
+- `settld-verify`: bundle verification (strict/compat), emits deterministic JSON output.
+- `settld-release`: release authenticity verification for distribution artifacts.
+- `settld-trust`: bootstrap trust materials for local testing/dev flows.
+
+## Verify Cloud (Magic Link)
+
+Verify Cloud is a hosted controller that accepts bundle zip uploads and produces:
+
+- View-only buyer report page (Green/Red/Amber with stable codes)
+- Downloads:
+  - original bundle zip
+  - hosted verify JSON output
+  - producer receipt (if present in bundle)
+  - PDF summary (compat surface)
+  - audit packet zip (monthly)
+- Workflow features:
+  - inbox listing/filtering for buyers
+  - vendor-scoped ingest keys (upload-only, vendor-stamped)
+  - tenant settings (mode defaults, policies, retention, quotas, webhook config)
+  - quotas + usage metering + billing invoice export
+  - approvals/holds with OTP gating and audit trail
+  - signed webhooks (or record-mode delivery in restricted environments)
+
+Security posture for hosted ingestion is part of the product contract:
+
+- safe zip extraction is centralized and shared by CLI + hosted ingestion
+- budgets enforced during unzip and hashing
+- hostile zip features are rejected (zip-slip, symlinks, duplicates, path attacks, bombs/ratios)
+
+## Quick “show me” commands
+
+Protocol and conformance:
+
+- `npm test`
+- `node scripts/fixtures/generate-bundle-fixtures.mjs`
+- `node conformance/v1/run.mjs --node-bin packages/artifact-verify/bin/settld-verify.js`
+
+Local verify examples:
+
+- `node packages/artifact-verify/bin/settld-verify.js --about --format json`
+- `node packages/artifact-verify/bin/settld-verify.js --strict --format json --invoice-bundle <dir>`
+- `node packages/artifact-verify/bin/settld-verify.js --strict --format json --close-pack <dir>`
+
+Run Verify Cloud locally:
+
+- `MAGIC_LINK_API_KEY=dev_key MAGIC_LINK_DATA_DIR=/tmp/settld-magic-link MAGIC_LINK_PORT=8787 node services/magic-link/src/server.js`
+
+Upload a bundle zip:
+
+- `node packages/magic-link-cli/bin/settld-magic-link.js upload <path-to-zip> --url http://localhost:8787 --mode auto --tenant <tenant>`
+
+## Gotchas that surprise new engineers
+
+- Trust anchors are out-of-band by design (no trust loops).
+- `verify/**` is excluded from manifests; receipts are validated by binding + signature, not by inclusion.
+- Codes are the API, not logs (warnings and errors are stable identifiers).
+- Canonical JSON is a hard contract; numeric semantics drift breaks cross-language parity.
+- Safe zip ingestion is centralized so CLI + hosted don’t drift on security posture.
+- When docs lag code, trust spec + conformance + fixtures.
+
+## Reading paths (10 files each)
+
+### A) New engineer (2–3 hours to become dangerous)
+
+Goal: understand the “truth engine,” then the hosted controller.
+
+1. `docs/spec/README.md`
+2. `docs/spec/INVARIANTS.md`
+3. `docs/spec/CANONICAL_JSON.md`
+4. `docs/spec/STRICTNESS.md`
+5. `docs/spec/VerifyCliOutput.v1.md`
+6. `conformance/v1/README.md`
+7. `packages/artifact-verify/bin/settld-verify.js`
+8. `packages/artifact-verify/src/invoice-bundle.js`
+9. `packages/artifact-verify/src/safe-unzip.js`
+10. `services/magic-link/README.md`
+
+### B) Auditor / partner security reviewer
+
+Goal: can we independently verify, and is ingestion safe?
+
+1. `docs/spec/README.md`
+2. `docs/spec/CRYPTOGRAPHY.md`
+3. `docs/spec/TRUST_ANCHORS.md`
+4. `docs/spec/BundleHeadAttestation.v1.md`
+5. `docs/spec/VerificationReport.v1.md`
+6. `docs/spec/WARNINGS.md`
+7. `docs/spec/ERRORS.md`
+8. `conformance/v1/README.md`
+9. `packages/artifact-verify/src/safe-unzip.js`
+10. `test/zip-security.test.js`
+
+### C) Buyer (AP / finance ops) viewpoint
+
+Goal: what does this change in our invoice workflow?
+
+1. `docs/pilot-kit/README.md`
+2. `docs/pilot-kit/buyer-email.txt`
+3. `docs/pilot-kit/buyer-one-pager.md`
+4. `services/magic-link/README.md`
+5. `docs/spec/InvoiceClaim.v1.md`
+6. `docs/spec/PricingMatrix.v1.md`
+7. `docs/spec/MeteringReport.v1.md`
+8. `docs/spec/VerifyCliOutput.v1.md`
+9. `docs/spec/WARNINGS.md` (top-level meanings)
+10. `docs/spec/STRICTNESS.md` (strict vs compat posture)
+
+### D) Vendor CTO / operator engineering
+
+Goal: how do I generate bundles and integrate?
+
+1. `docs/QUICKSTART_PRODUCE.md`
+2. `docs/QUICKSTART_VERIFY.md`
+3. `docs/spec/InvoiceBundleManifest.v1.md`
+4. `docs/spec/InvoiceClaim.v1.md`
+5. `docs/spec/PricingMatrix.v1.md`
+6. `docs/spec/MeteringReport.v1.md`
+7. `packages/artifact-produce/bin/settld-produce.js`
+8. `src/core/invoice-bundle.js`
+9. `packages/magic-link-cli/bin/settld-magic-link.js`
+10. `docs/pilot-kit/README.md`

package/docs/PERF_BASELINE.md

@@ -0,0 +1,85 @@
+# Performance Baseline (Local)
+
+This doc is the repeatable “truth under load” baseline for Settld. Update it when hot-path behavior changes (indexes, worker concurrency, timeouts).
+
+## Prereqs
+
+- Postgres running (recommended: `docker compose up -d postgres`)
+- API running in PG mode with workers enabled:
+
+  ```sh
+  export STORE=pg
+  export DATABASE_URL=postgres://proxy:proxy@localhost:5432/proxy
+  export PROXY_AUTOTICK=1
+  export PROXY_OPS_TOKENS="dev:ops_read,ops_write,finance_read,finance_write,audit_read"
+  npm run dev:api
+  ```
+
+- `k6` installed on your PATH.
+
+Optional (high-signal):
+
+- `PROXY_PG_LOG_SLOW_MS=100` to log `pg.query.slow` events.
+- `PROXY_PG_WORKER_STATEMENT_TIMEOUT_MS=5000` to prevent worker “hung query” pileups.
+- `PROXY_WORKER_CONCURRENCY_ARTIFACTS` / `PROXY_WORKER_CONCURRENCY_DELIVERIES` to tune throughput.
+
+## Scenario A: Ingest burst + ops reads
+
+Runs job lifecycles at a constant arrival rate, while also hammering ops read endpoints.
+
+```sh
+OPS_TOKEN=dev BASE_URL=http://localhost:3000 \
+  TENANTS=10 ROBOTS_PER_TENANT=3 JOBS_PER_MIN_PER_TENANT=50 DURATION=2m \
+  k6 run scripts/load/ingest-burst.k6.js
+```
+
+Record:
+
+- k6 summary p50/p95/p99 for `http_req_duration`
+- `/healthz` over time: `outboxPending`, `deliveriesPending`, `deliveriesFailed`
+
+## Scenario B: Delivery stress (webhook failures + timeouts)
+
+1) Start the webhook receiver:
+
+```sh
+PORT=4010 TIMEOUT_RATE_PCT=5 ERROR_RATE_PCT=5 TIMEOUT_DELAY_MS=10000 \
+  node scripts/load/webhook-receiver.js
+```
+
+2) Run the API with an export destination pointing at the receiver:
+
+```sh
+export PROXY_EXPORT_DESTINATIONS='{
+  "tenant_default": [
+    { "destinationId": "dst", "kind": "webhook", "url": "http://127.0.0.1:4010/hook", "secret": "devsecret" }
+  ]
+}'
+export PROXY_DELIVERY_HTTP_TIMEOUT_MS=1000
+```
+
+3) Run load:
+
+```sh
+OPS_TOKEN=dev BASE_URL=http://localhost:3000 \
+  TENANTS=3 ROBOTS_PER_TENANT=3 JOBS_PER_MIN_PER_TENANT=100 DURATION=2m \
+  k6 run scripts/load/delivery-stress.k6.js
+```
+
+Record:
+
+- `/healthz` backlog signals (steady state vs unbounded growth)
+- `/ops/deliveries?state=failed` size and retry behavior
+
+## Current baseline (fill in)
+
+- Date:
+- Machine:
+- Scenario A:
+  - p95 ingest/job endpoints:
+  - outboxPending steady state:
+- Scenario B:
+  - deliveriesPending steady state:
+  - deliveriesFailed steady state:
+  - notes:
+

package/docs/PRD.md

@@ -0,0 +1,77 @@
+# Settld PRD (v0)
+
+## One-line
+
+Settld lets a person or business delegate real-world work to an agent (robot + optional operator assist) with strict limits, proof of what happened, and a clear “who pays if something goes wrong” answer.
+
+## Who it’s for
+
+- **Requesters**: Households and Businesses that purchase outcomes (“Reset my apartment”).
+- **Owners**: Entities that provide robot capacity (time/location/reliability) and receive payouts.
+- **Operators**: Trained humans that provide remote assist and exception handling.
+- **Developers**: Publish skills; Settld certifies and distributes them.
+- **Trust Counterparty**: Insurance/guarantee/claims partner (initially) and later first-party.
+
+## Core promise
+
+Delegation with accountability:
+
+- Explicit authorization and revocation (where it can go / what it can touch / what it can record).
+- Observable execution (telemetry, checkpoints, operator actions).
+- Economic finality (escrow, payouts, refunds, chargebacks) with double-entry correctness.
+- Claims-ready evidence (“black box”) that is privacy-respecting and tamper-evident.
+
+## MVP scope (first shippable)
+
+Target environments: managed or semi-managed (apartments common areas, hotels/serviced apartments, offices after-hours).
+
+MVP capabilities:
+
+1. **Task templates**: at least one narrow template (e.g., `reset_lite`).
+2. **Booking**: quote → book → schedule window.
+3. **Dispatch**: match robot + reserve; operator coverage optional but supported.
+4. **Execution control**: job state machine + step checkpoints.
+5. **Exception handling**: abort/assist/customer approval request path (logged).
+6. **Telemetry black box**: append-only, hash-chained events; incident evidence bundles.
+7. **Settlement ledger**: escrow, fee splits, refunds.
+8. **Ops console primitives**: ability to list active jobs, view timeline, see incidents.
+
+## Key user journeys
+
+### 1) Requester books outcome
+
+1. Select template + enter constraints (rooms allowed, pets, privacy mode, fragile items).
+2. Choose time window.
+3. Receive quote (transparent internally; auditable adjustments).
+4. Book (payment hold/escrow).
+5. Track execution (statuses + optional media based on privacy policy).
+6. Receive completion report; rate.
+
+### 2) Execution with assist
+
+1. Robot begins; emits heartbeats + checkpoints.
+2. Failure mode triggers exception policy:
+   - request customer approval, OR
+   - request operator assist, OR
+   - abort and exit safely.
+3. Operator intervention is structured and fully audited.
+
+### 3) Incident & claims
+
+1. Impact/complaint/flag triggers incident workflow.
+2. Evidence bundle generated (timeline + key frames + operator actions).
+3. Claim opened, triaged, resolved; refunds/payout adjustments via ledger.
+
+## Non-goals for MVP
+
+- Full autonomy in random single-family homes.
+- Unbounded, free-form “LLM runs the robot” behavior.
+- Decentralized economics / blockchain dependence.
+
+## Success metrics (early)
+
+- Completion rate (with assist) and incidents per job-hour.
+- Operator minutes per job.
+- On-time start and job duration variance.
+- Claims rate, time-to-resolution, and net loss ratio.
+- Gross margin per job (after ops + reserve).

package/docs/QUICKSTART_KERNEL_V0.md

@@ -0,0 +1,96 @@
+# Kernel v0 Quickstart (Local)
+
+Goal: run the full “economic loop” locally and inspect artifacts (holdback, disputes, deterministic adjustments, replay-evaluate).
+
+## No-Clone CLI Path
+
+Registry form (recommended once package is published):
+
+```sh
+npx settld --version
+npx settld dev up
+npx settld conformance kernel --ops-token tok_ops
+```
+
+Release tarball fallback:
+
+If you downloaded a release asset like `settld-<version>.tgz`, you can run CLI commands without cloning this repo:
+
+```sh
+npx --yes --package ./settld-<version>.tgz settld --version
+npx --yes --package ./settld-<version>.tgz settld dev up
+npx --yes --package ./settld-<version>.tgz settld conformance kernel --ops-token tok_ops
+```
+
+Use the same pattern for all commands in this doc.
+
+## 1) Start The Dev Stack
+
+Recommended (one command):
+
+```sh
+./bin/settld.js dev up
+```
+
+Equivalent (manual):
+
+```sh
+docker compose --profile app up -d --build
+docker compose --profile init run --rm minio-init
+```
+
+Defaults:
+
+- API: `http://127.0.0.1:3000`
+- tenant: `tenant_default`
+- ops token: `tok_ops`
+
+## 2) Run Kernel Conformance
+
+This will exercise:
+
+- tool-call holdback disputes (freeze maintenance tick, issue verdict, deterministic adjustment)
+- marketplace run replay-evaluate (`/runs/:runId/settlement/replay-evaluate`)
+- deterministic verifier plugin selection (`verifier://settld/deterministic/latency-threshold-v1` or `verifier://settld/deterministic/schema-check-v1`)
+
+```sh
+./bin/settld.js conformance kernel --ops-token tok_ops --json-out /tmp/settld-kernel-v0-report.json
+```
+
+The runner prints `INFO ...` lines with `agreementHash` and `runId`.
+
+## 3) Open Kernel Explorer
+
+Open:
+
+`http://127.0.0.1:3000/ops/kernel/workspace?opsToken=tok_ops`
+
+Then paste the `agreementHash` from conformance into the “Tool Call Agreement” panel.
+
+## 4) Verify Replay Evaluate
+
+Use the `runId` printed by conformance:
+
+```sh
+curl -sS "http://127.0.0.1:3000/runs/<runId>/settlement/replay-evaluate" \
+  -H "x-proxy-tenant-id: tenant_default" \
+  -H "x-proxy-ops-token: tok_ops" | jq
+```
+
+## Shutdown
+
+```sh
+./bin/settld.js dev down
+```
+
+To wipe volumes (fresh DB + buckets):
+
+```sh
+./bin/settld.js dev down --wipe
+```
+
+## Product Surfaces
+
+- Kernel v0 contract surface and guarantees: `docs/KERNEL_V0.md`
+- Kernel Compatible badge criteria + listing flow: `docs/KERNEL_COMPATIBLE.md`
+- Reference capability listing JSON: `docs/kernel-compatible/capabilities.json`