tracegrove 0.3.0 → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,126 +0,0 @@
1
- # Tracegrove: architecture, current UX flow, and adapter gaps
2
-
3
- Status as of 2026-10-16, for this repository. "Intended" means what the protocol schemas already declare (enums for surfaces, lanes, boundaries, hosts) and what the README lists as not implemented yet.
4
-
5
- ## 1. High-level architecture
6
-
7
- ```text
8
- you write compiled (derived) kernel (unchanged) evidence
9
- ───────── ────────────────── ────────────────── ────────
10
- tracegrove.config.yaml ─┐
11
- tracegrove/**/*.flow.yaml ├─► authoring compiler ─► .tracegrove/compiled/ ─► corpus loader (freeze + sha256 refs)
12
- tracegrove/fixtures/*.yaml┘ (src/authoring) workflow.yaml, cases, ─► planner (side-effect free, blockers w/ owner+action)
13
- host, persona, recipe… ─► admission + local ledger (single writer, idempotent)
14
- ─► executor
15
- host adapter: local_process │ mock_loopback
16
- fixtures: registered API actions, readback, cleanup
17
- surface driver: http │ Chromium browser
18
- ─► trusted reducer (outcome from observations)
19
- ─► bundle writer ─► verify-bundle ───────► .tracegrove/runs/<run>/
20
- summary.md/json, junit.xml,
21
- request.json, bundle/
22
- tracegrove ui ─► reads config, flows, runs, git; runs `test`; scans routes for discovery
23
- ```
24
-
25
- The [Chromium browser runner](browser-driver.md) supports short flows and the low-level corpus, with typed UI steps, DOM assertions, authenticated local personas, and sealed recordings in Grove. Supporting modules that exist but aren't wired into the everyday flow yet include shared coordination, signed cleanup follow-ups, source citations, and the plan review API.
26
-
27
- ## 2. The current UX flow, end to end
28
-
29
- | Step | Today (this branch) | Intended |
30
- | --- | --- | --- |
31
- | Install | Private package; install a packed archive | `npm i -D tracegrove`, `pipx`/`uvx` shim for Python repos |
32
- | Set up | `tracegrove init` detects FastAPI, Django or Node and writes config + example flow | Same, plus discovery proposes the first flows |
33
- | Discover what to test | `tracegrove ui` scans routes (FastAPI, Django, Express/Fastify/Hono) and shows which have no flow, at a folder or a git branch | Source-selection resolver: diff-aware selection, coverage policy, AI-drafted flows reviewed through the plan review API |
34
- | Write tests | YAML/TypeScript flows with HTTP, browser, native job and local MCP steps, personas, saved values, fixtures | Hosted integration steps and widget-host qualification |
35
- | Run | `tracegrove test` (local process host, real lane) | Lanes: mock contract, recorded replay, hybrid, full live sandbox, canary; hosts: container, CI, cloud sandbox |
36
- | Read results | Terminal table, run folder, JUnit, `tracegrove show`, UI run history | Trace/video media, flaky-history, PR comments |
37
- | Prove it | `verify-bundle` re-derives the verdict from sealed evidence | Importer with issuer trust; hosted qualification lifecycle |
38
-
39
- ## 3. Gaps versus intended, by layer
40
-
41
- | Layer | Supported now | Declared but not implemented |
42
- | --- | --- | --- |
43
- | Surfaces (step kinds) | HTTP, Chromium UI, local `mcp.call` through the optional SDK, and `job.submit`/`job.observe`/`job.cancel` through an opted-in local Node IPC bridge | Hosted MCP/job providers, `integration.action`, `widget.action`, native desktop and computer-use |
44
- | Hosts | `local_process` (POSIX), `mock_loopback` | container, managed sandbox, cloud VM, customer CI, device service; Windows process trees |
45
- | Fidelity lanes | `full_live_sandbox`, `mock_contract` | `recorded_replay`, `hybrid_live_product`, `full_live_canary` |
46
- | Boundaries | `application`, `storage` observed as real/stubbed | per-provider boundaries with interceptors and readback oracles |
47
- | Auth | host-issued bearer tokens per persona | `env_credential` recipes, login flows (form/cookie/OAuth), account bindings and persona leases in use |
48
- | Fixtures/seeding | API-created entities with readback and confirmed 404/410 cleanup; deterministic generators | DB-level seeding, snapshots/restore, factories, datasets, relation bindings in use, `update` lifecycle |
49
- | Oracles | HTTP status/JSON path, native job observations with real worker/store sample, browser DOM, step result | Generic DB readback, hosted queue/job state, email/webhook capture, visual baseline, LLM judge (policy exists, all stages disabled) |
50
- | Environments | `isolated_candidate` | deployed preview, canary, production observation |
51
- | Discovery | Route scan in `tracegrove ui` (heuristic) | source selection resolver, intelligence-policy-gated drafting |
52
- | Media | step events, network metadata | trace and video capture (`required` currently unsatisfiable) |
53
-
54
- ### Integration with the verified core
55
-
56
- The integration branch also includes the immutable Git pair resolver, source citation
57
- validation, durable local plan review, qualification reduction and activation contracts.
58
- Those APIs do not make heuristic route discovery independent source confirmation or
59
- hosted qualification. The workbench remains a local authoring and replay interface.
60
-
61
- Node and Python presets use the same bounded interpreter observation during compilation
62
- and launch. Django runs migrations and starts its server through that Python executable.
63
- Generic shell wrappers remain blocked until a qualified host can observe their actual
64
- application runtime. A corrupt exported evidence bundle marks the overall run inconclusive,
65
- uses an inconclusive run directory and adds an evidence-integrity JUnit error, even when
66
- individual HTTP checks passed.
67
-
68
- ## 4. Adapter gap analysis for full-stack and data-engineering work
69
-
70
- "Adapter" here means anything tracegrove needs to start, seed, fake, observe or clean up. Priority is what blocks the most real repos first.
71
-
72
- ### Seeding
73
-
74
- | Need | Now | Gap and proposed adapter | Priority |
75
- | --- | --- | --- | --- |
76
- | Create records through the app's API | Yes (fixtures) | — | — |
77
- | Seed straight into the database (fast, bypasses API) | No | `seed.sql` adapter: run SQL/fixtures against a per-run DB, readback by SELECT, cleanup by transaction rollback or DB drop | P0 |
78
- | Framework factories (factory_boy, Django fixtures, Prisma seed) | No | `seed.command` adapter: run a repo command with `{dataDir}`/`DATABASE_URL`, capture emitted ids as JSON | P0 |
79
- | Snapshot and restore a known state | No | `seed.snapshot`: SQLite file copy / `pg_dump` template DB / `CREATE DATABASE … TEMPLATE` | P1 |
80
- | Large synthetic datasets for data pipelines | Generators only (words, slug, uuid, int) | dataset recipes: CSV/Parquet generators with seeds, row counts, schemas; `datasetRef` is already in the case schema | P1 |
81
-
82
- ### Databases
83
-
84
- | Need | Now | Gap and proposed adapter | Priority |
85
- | --- | --- | --- | --- |
86
- | SQLite per run | Works if the app reads `TRACEGROVE_DATA_DIR` (Django/FastAPI samples) | Built-in `db: sqlite` that sets `DATABASE_URL` | P0 |
87
- | Postgres/MySQL per run | No | `db: postgres` service adapter: start a container or use a local server, create a throwaway database per run, inject `DATABASE_URL`, drop it on cleanup, record server version in identity | P0 |
88
- | Migrations | Django preset runs `migrate` | `app.setup:` commands (alembic upgrade, prisma migrate, knex) run before readiness | P0 |
89
- | Assert on DB state | No | `observe: { sql: "select … ", expect: … }` read-only oracle with a separate read-only connection | P1 |
90
- | Redis, Mongo, Elasticsearch | No | generic `services:` block with per-service health + isolation recipe | P1 |
91
-
92
- ### Asynchronous jobs
93
-
94
- | Need | Now | Gap and proposed adapter | Priority |
95
- | --- | --- | --- | --- |
96
- | Start workers next to the app | `report-jobs` app starts a separate local worker; aggregate source identity includes its code | `components:` list in config: app + `celery worker`, RQ, Sidekiq, BullMQ, Dramatiq, Temporal worker; each its own supervised process, attested identity, health check | P0 |
97
- | Broker | No | `services: redis/rabbitmq` with per-run vhost/db index | P0 |
98
- | Wait for a job to finish | HTTP polling qualified against a persisted queue and separate worker; independent report readback catches false success, with cancellation, idempotency and failure controls | `job.submit` + `observe: { job: id, state: SUCCESS }` via Celery result backend / Flower API, BullMQ queue API; eager mode is explicitly not a real boundary | P1 |
99
- | Scheduled jobs (beat, cron) | No | logical clock control (`clockProfileRef` exists in the schema) plus a "tick" step | P2 |
100
-
101
- ### Third-party services and mocks
102
-
103
- | Need | Now | Gap and proposed adapter | Priority |
104
- | --- | --- | --- | --- |
105
- | Whole-app mock (contract lane) | `mock_loopback` host with declared routes, not in the authoring layer yet | `lanes: { mock: { from: recordings/ } }` in config | P1 |
106
- | Fake one dependency while the app is real (Stripe, SendGrid, S3, OpenAI) | No | outbound interceptor proxy: app gets `HTTPS_PROXY`/base URL env; routes are stubbed, recorded or passed through per boundary; observations sealed as boundary evidence (`interceptorRef` exists) | P0 |
107
- | Record once, replay in CI | No | `recorded_replay` lane: record real calls in a sandbox account, redact, replay deterministically | P1 |
108
- | Provider sandboxes (Stripe test mode, etc.) | No | `env_credential` auth + `test_sandbox` account class + provider cleanup actions with signed follow-ups (verifier exists) | P2 |
109
- | Email/webhook capture | No | built-in inbox (SMTP sink) and webhook catcher with assertions | P1 |
110
- | Object storage | No | MinIO/localstack service adapter | P2 |
111
-
112
- ### Data engineering
113
-
114
- | Need | Now | Gap and proposed adapter | Priority |
115
- | --- | --- | --- | --- |
116
- | Run a pipeline (Airflow DAG, Dagster job, dbt run, Spark job) | No | `job` surface with a `runner: command` that runs the pipeline against seeded inputs and records exit state and logs | P1 |
117
- | Assert on outputs | No | table oracles: row counts, schema, uniqueness, not-null, freshness, golden-file diff on CSV/Parquet | P1 |
118
- | Warehouses (BigQuery, Snowflake, DuckDB) | No | DuckDB local first; warehouse adapters with per-run schema/dataset and cleanup | P2 |
119
-
120
- ## 5. Suggested order
121
-
122
- 1. **P0 for full-stack repos:** `app.setup` commands, `services:` (Postgres, Redis), `components:` (workers), `db: sqlite|postgres` that injects `DATABASE_URL`, SQL and command seeding, and the outbound interceptor for third parties.
123
- 2. **P1 for coverage:** job observe, DB-state oracle, mock lane in the authoring layer, record/replay, email/webhook capture, data-pipeline job surface with table oracles.
124
- 3. **Qualify browser execution beyond local runs**, including staging personas, controller-death cleanup, trace/redaction collectors, and host network confinement.
125
-
126
- Each adapter keeps the kernel's rules: launched or reserved per run, identity attested, effects inside the declared envelope, cleanup confirmed by an independent read, and evidence sealed in the bundle.
@@ -1,102 +0,0 @@
1
- # Retained evidence to Agent Work Receipt
2
-
3
- This optional adapter exports a retained result as redacted AWR v0.2 and saves
4
- an immutable delivery intent in the existing local ledger. Delivery uses OrgX's
5
- existing workspace-scoped AWR importer. Import does not execute a campaign,
6
- complete work, qualify scenarios, or observe native outcomes. The separate
7
- [work-report attachment stage](work-report-attachment.md) links the imported
8
- report to its original work target and preserves its own retry state.
9
-
10
- Install the optional peer only when using this integration:
11
-
12
- ```sh
13
- npm install --save-exact @useorgx/agent-work-receipt@0.3.0
14
- ```
15
-
16
- Core SDK and ordinary CLI startup do not import that package. The optional adapter
17
- uses its canonical validator and content-hash helpers, without copying AWR schemas
18
- or implementing a competing canonicalizer.
19
-
20
- After `result-ingest`, create a protected operator-owned destination file:
21
-
22
- ```json
23
- {"origin":"https://useorgx.com","workspaceId":"00000000-0000-4000-8000-000000000001"}
24
- ```
25
-
26
- Replace the example UUID with your authorized workspace. The HTTPS origin must
27
- have no path, query, fragment, credentials, or trailing slash. The bundle cannot
28
- choose its destination.
29
-
30
- ```sh
31
- tracegrove awr-prepare --sink ./retained-results --id <result-id> --destination ./destination.json --json
32
- tracegrove awr-inspect --sink ./retained-results --id <result-id> --json
33
- tracegrove awr-deliver --sink ./retained-results --id <result-id> --json
34
- ```
35
-
36
- Preparation and inspection make no network requests. Supply the credential through
37
- a protected `TRACEGROVE_ORGX_TOKEN` environment variable. `--credential-env` selects
38
- a different variable name; never pass a credential as an argument. Tokens are not
39
- included in prepared records, journal entries, or transport diagnostics. Redirects
40
- are refused. Responses have a 32 KiB bound and 15-second deadline.
41
-
42
- The local operator controls configuration. The authenticated importer independently
43
- checks workspace access and admission. SDK hosts must implement
44
- `AwrDeliveryAuthority` against actual consumer-to-workspace policy. Local owner
45
- labels and no-op callbacks do not authenticate a remote tenant or grant approval.
46
-
47
- ## Evidence and acceptance
48
-
49
- The receipt describes export effort. A successful export of a failing campaign is
50
- successful export work; the original product verdict remains visible in its summary
51
- and `extensions["org.tracegrove.workflow"]`. A target regression is not assigned to
52
- the evidence exporter as an operational failure.
53
-
54
- The shared `workflow.evidence-projection/v1` schema preserves exact request,
55
- selection, campaign and attempt refs, separate expected/observed target digests,
56
- actual recorded provenance, planned lanes, and the frozen work binding. An unbound
57
- run stays unbound. Raw observations, excerpts, local paths, media locations and
58
- credentials are withheld. Refs/digests remain potentially sensitive metadata; use
59
- authorized destinations. Full evidence stays in the private result archive.
60
- Oversize exports fail rather than silently omitting attempts.
61
-
62
- Verification remains `unverified`; acceptance remains `pending`; qualification is
63
- not evaluated; native outcomes are not observed. The runtime is `tracegrove`, never
64
- the reserved `orgx-verify`. The adapter cannot manufacture native proof, a second
65
- verifier lens, task acceptance, or outcome attribution.
66
-
67
- Per-attempt estimates, reconciled costs and unknown-charge counts remain in the
68
- projection. AWR v0.2 requires a numeric total even when unknown. The export uses
69
- zero with `estimated: true` and explicit `status: unknown` /
70
- `zero_is_schema_placeholder: true` metadata. It is not a measured zero, campaign
71
- total, invoice cap, or authoritative economic observation.
72
-
73
- ## Recovery
74
-
75
- One retained result has one immutable destination and receipt. Preparing again
76
- returns the same envelope. Changing workspace or origin conflicts; editing the
77
- configuration cannot redirect a saved delivery. A new effort requires a new result.
78
-
79
- Before network I/O, the outbox syncs an attempt record. Missing or invalid responses
80
- leave `acknowledgement_unknown`. Retry with the same result ID: it reuses the exact
81
- receipt, destination, and digest-bound key. Receiver admission followed by connection
82
- loss reconciles one import. The same recovery covers a successful response followed
83
- by failed local acknowledgement publication. No retry path calls a runner, seeds
84
- data, or reruns product actions. Corrupt retained evidence blocks fresh delivery.
85
-
86
- A validated response records `imported_unverified`. Repeating delivery returns the
87
- saved acknowledgement without another network call; it does not renew archive
88
- integrity or permission. Use `results --id` for a fresh archive check. The prototype
89
- blocks at 1,000 attempts without evicting history. It inherits protected filesystem
90
- and single-writer requirements, not distributed fencing.
91
-
92
- The SDK subpath `workflow-verify/integrations/awr` exports `LocalAwrOutbox`, typed
93
- destination/envelope/acknowledgement contracts, `AwrDeliveryAuthority`,
94
- `AwrTransport`, and `OrgxAwrHttpTransport`. Supply the sink's same fenced ledger and
95
- scope. Credentials are retrieved only for network delivery. Trusted hosts may
96
- inject a transport; producers cannot select one through evidence.
97
-
98
- Conformance uses real loopback HTTP with simulated authority and admission. The
99
- receiver destroys the first connection after admission; reopen/retry must retain
100
- one receiver record, identical request bytes, and no credentials in the journal.
101
- This does not establish live OrgX import or native-work attachment; those require
102
- their own authenticated provider receipts.
@@ -1,74 +0,0 @@
1
- # Signed cleanup follow-ups
2
-
3
- An executing worker's cleanup report cannot release a hosted persona. A separate
4
- collector may sign a follow-up after observing the execution-owned resources.
5
- The follow-up is a new immutable record; it does not rewrite the attempt receipt.
6
-
7
- The ownership ledger supplies a `CleanupExpectation` containing the exact scope,
8
- attempt, worker, fencing generation, coordination version, admitted effects,
9
- unresolved effects and owned resource identities. HTTP absence and host stop are
10
- distinct obligations. The collector signs the canonical follow-up with an
11
- Ed25519 key kept outside the executing worker. Its scope-bound observer grant
12
- comes from protected control-plane configuration, not from the submitted proof.
13
-
14
- `verifyCleanupFollowup` validates the signature against that configuration and
15
- the complete expected manifest. It rejects unknown/revoked keys, executing-worker
16
- signers, altered bindings, changed resource/effect closure, unresolved effects,
17
- missing or wrong-kind observations, duplicate identities and stale/future evidence.
18
- The evidence window is at most five minutes. Status-only HTTP absence requires a
19
- GET/HEAD observation returning 404 or 410; a 200 response cannot qualify.
20
-
21
- The signature includes the algorithm, key ID and follow-up under the
22
- `tracegrove.signed-cleanup-followup/v1` domain separator. The signature field is
23
- excluded from signed bytes; the returned proof digest covers the complete envelope.
24
- The expectation digest includes every expectation field. Base64url signatures
25
- must use canonical unpadded encoding. Key IDs select configured public keys;
26
- the verifier never trusts a public key carried by the proof itself.
27
-
28
- `deriveCleanupExpectation` combines the current coordination snapshot with the
29
- protected ownership record. Missing host inventory prevents derivation. Admitted
30
- effects without independent resolution remain unresolved; extra effects and
31
- duplicate resources are refused. Takeover derives the current worker and fence
32
- while retaining prior resource identities. `ownershipLedgerHeadDigest` anchors
33
- the expectation to the exact canonical ledger event, including issuer receipts.
34
- The helper cannot establish that caller-supplied files came from that ledger.
35
-
36
- ## Offline inspection
37
-
38
- ```sh
39
- workflow-verify verify-cleanup --proof followup.json \
40
- --expectation expected.json --trust observers.json --json
41
- ```
42
-
43
- The trust file uses this shape, containing public keys only:
44
-
45
- ```json
46
- {
47
- "schemaVersion": "workflow.cleanup-observers/v1",
48
- "observers": [{
49
- "keyId": "configured-collector-key",
50
- "scopeId": "workspace-scope",
51
- "observerId": "independent-collector",
52
- "publicKeyPem": "-----BEGIN PUBLIC KEY-----\n...\n-----END PUBLIC KEY-----\n"
53
- }]
54
- }
55
- ```
56
-
57
- Exit 0 means the supplied files pass signature and expectation checks. Exit 2
58
- means validation failed. JSON output always reports `hostedReleaseAdmitted: false`.
59
- Inputs are limited to 4 MiB per file. A local trust file is useful for inspection
60
- and cannot configure a hosted authority or provision a signing key.
61
-
62
- ## Remaining hosted requirements
63
-
64
- The control plane must independently establish ownership and provider receipts,
65
- qualify the collector and provision/revoke its protected signing authority.
66
- The storage owner must load the current expectation and lease, recheck time and
67
- fencing under its admission lock, then append cleanup admission and persona
68
- release atomically. This verifier does not implement that storage transaction,
69
- prove provider absence from a digest alone, or authorize compensation actions.
70
- Worker release through the ordinary coordination port remains refused.
71
-
72
- Local tests use ephemeral keys. One native HTTP case rejects observed presence,
73
- deletes an owned test resource and verifies a separately observed absence; it is
74
- local conformance evidence, not a qualified staging or provider receipt.
@@ -1,315 +0,0 @@
1
- # Tracegrove developer UX redesign
2
-
3
- Status: phase 1 is implemented in the draft PR (see Phasing). Node, FastAPI and Django samples pass end to end through it.
4
-
5
- ## The problem today
6
-
7
- Writing one HTTP test for the notes sample takes **19 YAML files across 13 folders** (`workflow.yaml`, suite, case, persona, auth recipe, component profile with a hand-pinned sha256, environment plan, host plan, runner, surface pack, surface binding, fidelity profile, intelligence policy, action registry, seed recipe, fixture plan, a materialized payload with a pinned digest, …). Every file carries `schemaVersion`, `ref.id` and `ref.revision`. Running it is four commands (`doctor`, `plan --output request.json`, `run --request`, `verify-bundle`) and the results land in `.workflow-verify/executions/<uuid>/`, a folder name nobody can find again.
8
-
9
- The kernel underneath (frozen corpus, digests, independent readback, owned cleanup, sealed bundles) is the valuable part, and it stays exactly as it is. What is missing is a front door.
10
-
11
- ## Design principles
12
-
13
- 1. **One config, one file per test.** Everything else is derived, with an escape hatch to the strict v1 corpus when you need it.
14
- 2. **Convention over configuration.** Paths, ids, names and defaults come from where files live. Nothing has an id you have to invent.
15
- 3. **Compile, don't replace.** The new files compile into the existing `workflow.*/v1` documents, so every guarantee (digests, attestation, cleanup proof, `verify-bundle`) still holds and the evidence is identical in kind.
16
- 4. **One command to run.** `tracegrove test` plans, runs, reports and verifies. The frozen request is still written, just not by you.
17
- 5. **Framework presets, not framework plugins.** FastAPI, Django and Node are a few lines of launch defaults, not a dependency on the user's stack.
18
- 6. **Results you can find.** Human-named run folders, a `latest` pointer, Markdown + JSON + JUnit next to the sealed bundle.
19
-
20
- ## What a user's repo looks like
21
-
22
- ```text
23
- my-app/
24
- tracegrove.config.yaml # the only config file
25
- tracegrove/ # tests live here (configurable: testDir)
26
- notes/
27
- create-read-delete.flow.yaml
28
- authz.flow.yaml
29
- fixtures/
30
- existing-note.fixture.yaml
31
- .tracegrove/ # generated, gitignored
32
- runs/
33
- 2026-10-16T14-22-03Z_pass_3f9a1c/
34
- summary.md summary.json junit.xml bundle/
35
- latest -> 2026-10-16T14-22-03Z_pass_3f9a1c
36
- compiled/ # the derived v1 corpus, for inspection or ejection
37
- state/ # ledger
38
- ```
39
-
40
- ## Naming conventions
41
-
42
- | Thing | Convention | Derived id |
43
- | --- | --- | --- |
44
- | Test file | `tracegrove/<area>/<behavior>.flow.yaml`, kebab-case | case id `<area>.<behavior>` |
45
- | Fixture file | `tracegrove/fixtures/<name>.fixture.yaml` | entity key `<name>` |
46
- | Step | `name:` in plain words, optional | `snake_case(name)`, else `<method>_<n>` |
47
- | Assertion | from a step's `expect:` | `<step>.expect` |
48
- | Persona | key under `personas:` | credential `<persona>_token` |
49
- | Saved value | `save: { noteId: note.id }` | referenced as `{noteId}` anywhere later |
50
- | Fixture output | `{existing.noteId}` | input from fixture `existing`, output `noteId` |
51
- | Tags | `tags: [smoke, authz]` | coverage tags; filter with `--tag` |
52
- | Run folder | `<UTC timestamp>_<outcome>_<short execution id>` | sortable, greppable |
53
-
54
- ## The config file
55
-
56
- ```yaml
57
- # tracegrove.config.yaml
58
- project: notes
59
- app:
60
- adapter: node # node | fastapi | django | command
61
- entry: app/app.mjs # node: script; fastapi: module:app; django: manage.py dir
62
- health: /health
63
- sources: [app/**] # files whose digest is attested as "what was tested"
64
- env:
65
- NOTES_BUILD: sample
66
- inject: # where tracegrove hands the app its port, data dir and tokens
67
- port: PORT
68
- dataDir: NOTES_DIR
69
- tokens: NOTES_TOKENS
70
- personas:
71
- owner: { subject: alice } # first persona is the default caller
72
- intruder: { subject: bob }
73
- ```
74
-
75
- Defaults when omitted: `testDir: tracegrove`, `inject` = `PORT`, `TRACEGROVE_DATA_DIR`, `TRACEGROVE_TOKENS`, `health: /health`, deadline 60 s, sources = all tracked-looking files under the app dir excluding `node_modules`, `.venv`, `__pycache__`, `.git`, `.tracegrove`.
76
-
77
- ## Adapter presets
78
-
79
- | `adapter` | Launch it compiles to | Notes |
80
- | --- | --- | --- |
81
- | `node` | `node <entry>` with `PORT` set | Any Express/Fastify/Hono/Nest app that reads `PORT`. |
82
- | `fastapi` | `<python> -m uvicorn <entry> --host 127.0.0.1 --port {port}` | `python:` defaults to `.venv/bin/python` if present, else `python3`. |
83
- | `django` | `<python> manage.py migrate --noinput && <python> manage.py runserver 127.0.0.1:{port} --noreload` | Point `DATABASES` at `TRACEGROVE_DATA_DIR` for a fresh database per run (the sample does). |
84
- | `command` | Reserved for a qualified application-runtime adapter | Currently BLOCKED: a shell wrapper cannot attest the application interpreter. |
85
-
86
- `{port}` and `{dataDir}` are substituted in `args` and `env` values. Tracegrove picks a free loopback port itself (the "assigned" port mode), so apps no longer need to print `listening <port>`.
87
-
88
- **Auth hook.** Tracegrove issues a random bearer token per persona per run and passes `{token: subject}` as JSON in the `tokens` env var. The app adds about five lines in test mode to accept them:
89
-
90
- ```python
91
- # FastAPI
92
- tokens = json.loads(os.environ.get("TRACEGROVE_TOKENS", "{}"))
93
- def current_user(authorization: str = Header("")):
94
- sub = tokens.get(authorization.removeprefix("Bearer "))
95
- if not sub: raise HTTPException(401)
96
- return sub
97
- ```
98
-
99
- ```python
100
- # Django middleware
101
- TOKENS = json.loads(os.environ.get("TRACEGROVE_TOKENS", "{}"))
102
- def tracegrove_auth(get_response):
103
- def mw(request):
104
- sub = TOKENS.get(request.headers.get("Authorization", "").removeprefix("Bearer "))
105
- if sub: request.user = User.objects.get_or_create(username=sub)[0]
106
- return get_response(request)
107
- return mw
108
- ```
109
-
110
- ```js
111
- // Express
112
- const tokens = JSON.parse(process.env.TRACEGROVE_TOKENS ?? '{}');
113
- app.use((req, _res, next) => { req.user = tokens[(req.headers.authorization ?? '').slice(7)]; next(); });
114
- ```
115
-
116
- `tracegrove init` prints the right snippet for the detected framework.
117
-
118
- ## A test
119
-
120
- ```yaml
121
- # tracegrove/notes/create-read-delete.flow.yaml
122
- title: Create a note, read it back, keep other notes intact, delete it
123
- risk: critical # critical | standard (default) | exploratory
124
- tags: [notes]
125
- given:
126
- existing: existing-note # tracegrove/fixtures/existing-note.fixture.yaml
127
- steps:
128
- - name: create
129
- POST: /notes
130
- json: { title: Fresh note, body: Written through the real API. }
131
- expect: { status: 201 }
132
- save: { noteId: note.id }
133
-
134
- - name: read back
135
- GET: /notes/{noteId}
136
- observe: true # independent readback, polled until it holds
137
- expect: { status: 200, json: { note.title: Fresh note } }
138
-
139
- - name: intruder is denied
140
- GET: /notes/{noteId}
141
- as: intruder
142
- expect: { status: 403, absent: [note] }
143
-
144
- - name: delete
145
- DELETE: /notes/{noteId}
146
- expect: { status: 204 }
147
-
148
- - name: gone
149
- GET: /notes/{noteId}
150
- observe: true
151
- expect: { status: 404 }
152
-
153
- - name: existing note untouched
154
- GET: /notes/{existing.noteId}
155
- observe: true
156
- expect: { status: 200, json: { note.title: Existing note } }
157
- ```
158
-
159
- Rules: one HTTP verb key per step; use block style (or quotes) for values containing `{` or `[`, because YAML reads them as inline maps and lists; `json:` is the request body; `expect.json` keys are dot paths into the response body; `absent` lists paths that must not exist; the effect envelope (allowed methods, path prefixes, max creates) is derived from the steps and fixtures, so the safety rails stay on without being written by hand.
160
-
161
- ## A fixture
162
-
163
- ```yaml
164
- # tracegrove/fixtures/existing-note.fixture.yaml
165
- create:
166
- POST: /notes
167
- json: { title: Existing note, body: { words: 6 } }
168
- expect: 201
169
- save: { noteId: note.id }
170
- readback:
171
- GET: /notes/{noteId}
172
- expect: { status: 200, json: { note.title: Existing note } }
173
- cleanup:
174
- DELETE: /notes/{noteId}
175
- expect: 204
176
- gone: { GET: "/notes/{noteId}", status: 404 }
177
- ```
178
-
179
- `{ words: 6 }` uses the deterministic generator; the logical payload is materialized at compile time, so there is no digest to paste.
180
-
181
- ## TypeScript authoring
182
-
183
- Use `tracegrove.config.ts` instead of the YAML config for editor completion and
184
- shared constants. Flows and fixtures can independently use `.flow.ts` and
185
- `.fixture.ts`; YAML and TypeScript files can coexist when their logical names
186
- differ. They all compile through the same Zod schemas into the same corpus.
187
-
188
- ```ts
189
- // tracegrove.config.ts
190
- import { defineConfig } from 'workflow-verify/authoring';
191
-
192
- export default defineConfig({
193
- project: 'notes',
194
- app: { adapter: 'node', entry: 'app/app.mjs' },
195
- personas: { owner: { subject: 'alice' } },
196
- });
197
- ```
198
-
199
- ```ts
200
- // tracegrove/notes/health.flow.ts
201
- import { flow } from 'workflow-verify/authoring';
202
-
203
- export default flow({
204
- title: 'The application is ready',
205
- steps: [{ GET: '/health', expect: 200 }],
206
- });
207
- ```
208
-
209
- `fixture({...})` supplies the same editor types for fixture recipes. The helpers
210
- preserve input values; the ordinary compiler validates and fills defaults. Run
211
- your TypeScript checker for editor contracts and `tracegrove check` for canonical
212
- runtime validation. `test`, `list`, `compile`, discovery and Grove use either
213
- config format. `init` preserves an existing TypeScript config.
214
-
215
- TypeScript authoring executes trusted project code, including during `check` and
216
- discovery; it is outside the application sandbox. Each module loads in a fresh
217
- Node process so edits to imported helpers are seen on the next compilation.
218
- Evaluation is limited to five seconds and a 1 MiB JSON-compatible default export.
219
- Console output does not contaminate CLI JSON. Errors report the failing module
220
- without forwarding arbitrary module diagnostics.
221
-
222
- Use Node 22's erasable TypeScript: include extensions on relative imports and use
223
- `import type` for types. Enums, parameter properties, JSX, and tsconfig path aliases
224
- are not transformed. The package's Node version requirement still applies. Keep
225
- one project config; discovery rejects competing YAML and TypeScript configs.
226
- An explicit `--config` on `test`, `list`, or `compile` selects a format when both
227
- are intentionally present.
228
-
229
- ## Command reference
230
-
231
- ```text
232
- tracegrove init detect FastAPI / Django / Node, write config + an example flow + .gitignore entry
233
- tracegrove test [--grep notes] [--tag smoke] [--json]
234
- compile, plan, run, write the run folder, verify the bundle
235
- tracegrove list list flows with their derived ids, tags and steps
236
- tracegrove show [--run latest] print a run's summary
237
- tracegrove compile write the derived v1 corpus to .tracegrove/compiled (to inspect or eject)
238
- ```
239
-
240
- Exit codes are unchanged (0 pass, 1 assertion failure, 2 blocked, 3 infrastructure, 4 cancelled). `workflow-verify` and all of its low-level commands keep working; the compiled folder is a normal `workflow.yaml` project, so `workflow-verify plan --config .tracegrove/compiled/workflow.yaml` still works.
241
-
242
- ## Output a human reads first
243
-
244
- ```text
245
- tracegrove · notes · app sha 3e8d13bc9bd1
246
-
247
- PASS notes.create-read-delete 6 steps 28 ms
248
- FAIL notes.authz 3 steps 14 ms
249
- step "intruder is denied": expected status 403, got 200
250
- GET /notes/9c1e… -> 200 {"note":{"id":"9c1e…","title":"Fresh note"}}
251
- ERROR notes.crud 5 steps 0 ms
252
- (app startup): application exited before becoming ready; last app output:
253
- | ERROR: Error loading ASGI app. Could not import module "app.mian".
254
-
255
- 1 passed, 1 failed · run .tracegrove/runs/2026-10-16T14-22-03Z_fail_3f9a1c
256
- ```
257
-
258
- `summary.md` carries the same table plus per-step request/response excerpts; `junit.xml` lets GitHub Actions, GitLab and Buildkite annotate failures with no plugin.
259
-
260
- ## Forking it into your own repo
261
-
262
- 1. `npm i -D tracegrove` once it is published (today: install the packed archive, as the `tracegrove-notes-api` repo does).
263
- 2. `npx tracegrove init` writes `tracegrove.config.yaml`, `tracegrove/example.flow.yaml` and adds `.tracegrove/` to `.gitignore`.
264
- 3. Paste the printed auth snippet into the app's test-mode auth.
265
- 4. `npx tracegrove test`.
266
- 5. CI: one step, `npx tracegrove test`, then upload `.tracegrove/runs/latest` as an artifact.
267
-
268
- ## Phasing
269
-
270
- | Phase | Scope | In the draft PR |
271
- | --- | --- | --- |
272
- | 1 | Config + flow + fixture compiler, node/fastapi/django presets, assigned port mode, app-output-on-startup-failure, `test`/`list`/`compile`/`show`/`init`, run folders with summary.md/json/junit, Node/FastAPI/Django samples | Yes; generic command host remains unqualified |
273
- | 2 | `lanes:` with an offline mock lane compiled from recorded responses; `--update-snapshots` to record them | No |
274
- | 3 | `tracegrove.config.ts` with `defineConfig`, `flow()` and `fixture()` for editor completion; canonical JSON Schema for YAML | Implemented through the shared compiler; no alternative executor |
275
- | 4 | Login-recipe personas (POST /login, cookie or token capture) for apps whose auth can't be patched; external `url:` targets with declared, unverified identity | No |
276
- | 5 | Browser surface using shared typed `browser:` commands, independent DOM/API checks, and verified Grove recording playback | Implemented locally; staging qualification remains open |
277
-
278
- ## Structured command failures
279
-
280
- An uncaught command failure with `--json`, `--json true` or `--json=true` writes one
281
- `tracegrove.cli-error/v1` result to stdout and no exception diagnostic to stderr. The strict
282
- schema is exported as `schema/tracegrove_cli-error_v1.json` and `CliErrorReport` from the
283
- protocol package. For example, `tracegrove run --json` returns exit 2 and:
284
-
285
- ```json
286
- {"schemaVersion":"tracegrove.cli-error/v1","command":"run","errorClass":"blocked","exitCode":2,"reasons":[{"code":"missing_argument","message":"A required option or option value is missing.","recovery":"Run tracegrove help and correct the command options."}],"omittedReasonCount":0}
287
- ```
288
-
289
- Messages and recovery actions come from a controlled public vocabulary. Raw exception text,
290
- stacks, supplied argument values, configuration contents and arbitrary internal reason codes
291
- are omitted. At most 32 reasons are returned; `omittedReasonCount` reports any remainder.
292
- Unknown prerequisites use `prerequisite_unavailable`. Unknown command names become `command: null`.
293
- This trades detailed candidate diagnostics for a bounded machine result; keep private evidence
294
- under its existing access policy when investigating a failure.
295
-
296
- Existing process exit statuses are preserved: blocked/conflict/schema failures use 2;
297
- infrastructure and unexpected failures use 3. A top-level JSON parse failure retains its legacy
298
- exit 3 with `errorClass: invalid_input` and `invalid_json`. The report's exit code equals the
299
- process status. It does not assert that execution never started or authorize a retry: a failure
300
- can follow partial effects, and uncertain ownership must be reconciled before replay.
301
-
302
- Doctor retains `tracegrove.doctor/v1`. Completed test/run results and assertion failures retain
303
- their existing JSON and exit codes 0/1/2/3/4. Returned check/verification results keep their existing
304
- schemas. `--json false` and `--json=false` keep prose errors, and `--jsonl` keeps its existing event
305
- stream and top-level error behavior. Successful reports, doctor reports and artifacts are outside
306
- this error envelope's omission policy; this change establishes no artifact redaction guarantee.
307
-
308
- ## Found along the way
309
-
310
- - The bundle verifier re-derived "infrastructure failed" only from step reason codes, so any run whose app never started produced a bundle that failed its own verification. The PR fixes this by also reading the sealed `host.failed` event; that evidence can only move a cell away from passed.
311
-
312
- ## Open questions for Hope
313
-
314
- - Package and binary name: `tracegrove` for both (recommended), keeping `workflow-verify` as an alias.
315
- - Default test folder: `tracegrove/` (recommended) versus `tests/tracegrove/` to sit beside pytest/jest.