@pipefy/pipefy-process-coder 0.1.0

package/README.md

@@ -0,0 +1,808 @@
+# Pipefy Process Coder
+
+Pull a Pipefy pipe and everything it connects to down to disk as readable JSON,
+edit it with Claude, diff it, push it back.
+
+A CLI, and only a CLI. `PLAN.md` describes an Electron app with a CLI seam inside
+it; this build inverts that — `pipe` **is** the product. The desktop shell is
+gone rather than deferred: the one thing a GUI bought was *seeing* a diff, and
+`pipe diff --open` renders the same page in the browser with nothing to install.
+
+> **Working on this repo with an agent?** Read [`AGENTS.md`](AGENTS.md) first —
+> what the tool is for, the doctrines that make it trustworthy, the specific ways
+> this API lies, and what is still unbuilt.
+
+```bash
+pipe login                          # browser sign-in via the Pipefy AI toolkit
+pipe doctor                         # what can this endpoint actually do?
+pipe pull <pipeId> --out ./my-pipe     # snapshot if available, GraphQL if not
+cd my-pipe
+pipe validate                       # offline, instant
+pipe diff --open                    # see it
+pipe plan --dry-run                 # refuses what cannot be applied
+pipe apply                          # snapshot, apply, verify
+```
+
+## The two workflows
+
+An existing pipe starts with **`pipe pull`**. A new one starts with
+**`pipe create`**, which creates the pipe and then pulls it, so both land in the
+same place: a workspace on disk with a baseline to diff against.
+
+```
+  EXISTING PIPE                          NEW PIPE
+  ─────────────                          ────────
+
+  pipe pull <pipeId> --out ./my-pipe        pipe create "New pipe" --org <orgId>
+        │                                      │
+        │  snapshot, or GraphQL                │  createPipe, then pulls it —
+        │  reconstruction if the org           │  a new pipe is not empty, it
+        │  has no snapshots                    │  arrives with a start form
+        │                                      │  and three default phases
+        └──────────────┬───────────────────────┘
+                       ▼
+        ./my-pipe/  pipes/<id>--<name>/…      the tree you edit
+                .ppc/baseline/<id>.json   what you diff against
+                AGENTS.md · MAPPING.md    how to edit it, and what every id means
+                       │
+                       ▼
+        ┌──── edit the JSON ─────────────────────────────────┐
+        │      by hand, or with an agent reading AGENTS.md   │
+        │                                                    │
+        │  pipe validate      shape + referential integrity  │  offline,
+        │  pipe diff          against the baseline           │  instant,
+        │  pipe plan --dry-run  is it all applicable?        │  no network
+        │                                                    │
+        └──── both clean? ───────────────────────────────────┘
+                       │
+                       ▼
+        pipe apply
+          1. snapshot the pipe, labelled pre-apply/<ts>      the rollback point
+          2. execute step by step, persisting each status    resumable
+          3. adopt created ids from the responses            no read-back needed
+          4. verify for free (GraphQL), or ask first          before a 2nd snapshot
+                       │
+          ┌────────────┴────────────┐
+          ▼                         ▼
+        clean                    something failed
+        done                     pipe apply --resume <plan>   replay as compiled
+                                 pipe apply                   recompile and retry
+                                 (offered) revert everything   pipe, flows, agents —
+                                                               whatever this run
+                                                               actually changed
+```
+
+### `pull` vs `export`
+
+**They perform the same read.** Snapshot if the organization has them, GraphQL
+reconstruction if not. The difference is entirely in what they leave on disk.
+
+```
+  pipe export <pipeId> --out pipe.json   pipe pull <pipeId> --out ./my-pipe
+  ───────────────────────────────────     ─────────────────────────────
+
+  pipe.json                        76K   ./my-pipe/
+                                           pipes/<pipeId>--<slug>/
+  ONE file. The structure exactly as         pipe.json
+  Pipefy's snapshot format has it:           phases/00--start-form.json
+  twenty flat arrays named after             phases/01--new-request.json
+  database tables.                           …7 more phase files
+                                             labels.json  webhooks.json
+  payload.relations.fields[0]                public-form.json  relations.json
+    .label = "Some field in a late phase"    preferences.json
+    .phase_id = <a phase id>                 _readonly/  _non-snapshot/
+                                           .ppc/baseline/<pipeId>.json
+  All 48 fields in one array, in           .ppc/lock.json
+  whatever order the database gave         AGENTS.md  MAPPING.md  CAPABILITIES.md
+                                          REFERENCE.md
+  them, each pointing at its phase        .gitignore  workspace.json
+  by id.                                                            29 files
+
+                                          The same 48 fields, but each one inside
+                                          the phase file it belongs to, ordered by
+                                          index:
+
+                                            phases/00--start-form.json
+                                              .name = "Start form"
+                                              .fields[0].label = "Request title"
+```
+
+So:
+
+| | `pipe export` | `pipe pull` |
+| --- | --- | --- |
+| Writes | one JSON file | a workspace directory |
+| Shape | the raw payload, 20 flat arrays | files organised the way you would edit them |
+| Baseline to diff against | no | yes, `.ppc/baseline/<id>.json` |
+| Docs for agents | no | `AGENTS.md`, `MAPPING.md`, `CAPABILITIES.md`, `REFERENCE.md` |
+| Connected repos | no, one repo only | yes, `--depth` walks them |
+| `validate` / `diff` / `plan` / `apply` work on it | **no** | yes |
+
+That last row is the whole distinction. Run `pipe diff` in a directory holding
+only an export and you get *"no pipe workspace found"* — because there is no
+workspace, no baseline, and nothing organised to edit. **`export` gives you the
+data; `pull` gives you somewhere to work.**
+
+### So why does `export` exist?
+
+To carry a read from a machine that can reach the pipe to one that cannot.
+
+An FDE often gets one session on a client's laptop, or one token that expires, or
+a VPN they will not have tomorrow. `export` captures the structure as a file they
+can take away. On the other side, `pull --from-file` turns that file into a real
+workspace — the same 29 files, no network, no token, no second snapshot:
+
+```bash
+# on the machine with access, once
+pipe export <pipeId> --envelope --pretty --out pipe.json
+
+# anywhere, afterwards
+pipe pull --from-file pipe.json --out ./my-pipe
+cd cs && pipe validate && pipe diff        # the normal loop, offline
+```
+
+It is a transport format, not a step in the workflow. If you can reach the pipe,
+use `pipe pull` and skip it.
+
+**When you do use it, pass `--envelope`.** Without it the file is a bare payload
+with no coverage map, and `--from-file` then has to assume every entity group was
+read in full. On an endpoint with snapshots that is true. On one without, it is
+false in the dangerous direction: a group the reconstruction could not read looks
+*empty*, and empty means delete. Both commands warn about it now, but the fix is
+the flag.
+
+## Does it save tokens?
+
+Yes, and the saving grows with the size of the pipe. Measured against the live
+FDE pipe — 8 phases, 58 fields, 14 automations — with
+`research/measure-read-cost.ts`, which issues the same queries the Pipefy MCP
+tools issue and weighs the responses.
+
+**Learning the pipe through MCP tool calls:**
+
+```
+                                        minified    as the MCP sends it
+  get_pipe                  1 call    ~3,931 tok    ~7,432 tok
+  get_automations           1 call      ~405 tok      ~786 tok
+  get_automation × each    14 calls   ~2,488 tok    ~4,218 tok
+  get_webhooks              1 call       ~87 tok      ~164 tok
+  get_field_conditions      1 call       ~85 tok      ~187 tok
+  get_pipe_relations        1 call       ~75 tok      ~142 tok
+  get_pipe_members          1 call       ~64 tok      ~131 tok
+  ─────────────────────────────────────────────────────────────
+  TOTAL                    20 calls   ~7,135 tok   ~13,060 tok
+```
+
+The right-hand column is the one that matters. An MCP result is pretty-printed
+with a 2-space indent and then embedded as an escaped string inside its own JSON
+envelope, which is roughly **1.8× the minified data**. Both steps are
+deterministic, so that number is measured, not estimated.
+
+**Learning the same pipe through the CLI:**
+
+```
+  pipe pull <pipeId> --out ./my-pipe        ~138 tok   its whole terminal output
+
+  then read only what the task needs:
+    MAPPING.md   every id → label + phase   ~2,179 tok
+    AGENTS.md    the editing rules          ~1,210 tok
+    one phase file, to edit its fields      ~1,089 tok
+```
+
+A pull moves 115 KB of pipe structure onto disk and puts **~138 tokens** in the
+context — a fifteen-line summary. Nothing else is read until something needs it,
+and then it is one file, not twenty API responses.
+
+For a typical structural edit — orient with `MAPPING.md`, open the phase you are
+changing, edit it — that is around **3,300 tokens against ~13,000**. About a
+quarter. Add a second edit in the same session and the gap widens: the CLI
+rereads one file, while the MCP path either re-queries or holds the whole
+structure in context for the rest of the conversation.
+
+### Where the rest of the saving comes from
+
+| | |
+| --- | --- |
+| **Diffs instead of dumps** | `pipe diff` on a real 3-field change: **148 tokens**. `pipe plan --dry-run`: **73**. `pipe validate`: **7**. The model sees what changed, never the pipe. |
+| **No tool schemas** | The Pipefy MCP exposes ~250 tools. Their JSON schemas sit in context for the whole session whether used or not. `pipe` is one Bash call. |
+| **Refusals cost nothing** | `pipe plan` says "`start_form_title` has no input" offline, before a single request. The MCP path finds out by sending a mutation that succeeds and silently changes nothing. |
+| **Failures explain themselves** | A failed apply writes its step statuses to the plan file. Resuming reads one line, not a re-derivation of what happened. |
+
+The honest counter-case: for **one** small read — "what phases does this pipe
+have?" — the MCP is cheaper, because a pull costs a snapshot (~60s) and writes 29
+files you did not need. Use the MCP to look; use `pipe` to change.
+
+## Does it handle integrations?
+
+The ones with a public write path, yes — all of them, exercised live and
+reverted. See `VALIDATION.md`.
+
+| Integration | Coverage |
+| --- | --- |
+| **Webhooks** | create, update, delete. The full row: url, actions, headers, filters. |
+| **Pipe relations** (connections) | create, update, delete, including all eight behaviour flags |
+| **Cross-pipe automations** | create and delete, and update by delete+recreate. A `create_card` into another pipe with its field map is what wires the FDE pipe to the CS pipe. |
+| **HTTP request automations** | the whole `send_http_request` surface: url, method, headers, body, and the four authentication params |
+| **Connector fields** | read and diffed; edited inside their phase file |
+| **Public form** | 14 of the 16 `PublicFormSettingsInput` fields; `html` and `css` have no input |
+| **Email templates** | **read-only.** `EmailTemplate` has no Input type on this endpoint at all |
+| **Email inboxes** | **read-only.** No create, update or delete exists |
+| **iPaaS flows** | pulled, diffed, planned and applied by the same commands — see **iPaaS flows** below |
+| **iPaaS connections** | read and referenced, never written. A missing one is created as an inert placeholder so a flow can still be built |
+| **AI agents** | read separately (they are absent from the snapshot payload) and never written back |
+
+Every run also writes the integration surface out on its own when `--report` is
+passed — webhooks, relations, outbound automations, connector fields, inbound
+email, and every repo the pipe reaches. See **Runs** below.
+
+What makes this trustworthy rather than a claim: the tool refuses at plan time
+when a write path does not exist, per column, rather than sending a mutation that
+reports success and changes nothing. `pipe capabilities` prints the whole
+registry — 29 supported paths, and the reason for each gap.
+
+## iPaaS flows
+
+Pipefy's iPaaS — "Advanced Automations" — is **Activepieces** underneath, and it
+is where a process does what the automation engine cannot: reach external
+systems, and read pipes other than the one that triggered.
+
+**A flow is an entity of its pipe.** Every pipe has its own iPaaS workspace, and
+a pipe's own webhook URLs carry the ids of its flows — so `pipe pull` reads them
+without a flag, and they land under the pipe:
+
+```
+pipes/<pipeId>--<slug>/flows/
+  _meta.json                  coverage, projectId, counts, read time
+  connections.json            externalId, pieceName, displayName, status
+  pieces.json                 pieceName -> version this instance offers
+  _readonly/flow-runs.json    recent run outcomes; never written back
+  <flowId>--<slug>/
+    flow.json                 the flow row
+    version.json              displayName, valid, schemaVersion, state
+    nodes/NN--<name>.json     one node per file, in execution order
+```
+
+A flow is a linked list on the wire — `nextAction`, `firstLoopAction`,
+`children[]` — and one file per node on disk. That fold is a real
+transformation rather than a move, so `link(unlink(F)) == F` is proven per flow
+on every pull and in `test/flow-codec.test.ts`.
+
+Authentication needs no new credential: the internal API mints a pipe-scoped
+token and one exchange turns it into an iPaaS session, which also hands back the
+project id. A pipe whose iPaaS is off gets `coverage: "unknown"` — never read as
+"this pipe has no flows".
+
+### The two rules the flow path enforces
+
+**Enabling a flow is refused** unless `--allow-flow-enable` is passed, and
+refused outright while the project holds a placeholder connection. Enabling
+points a flow at live traffic; that is never incidental.
+
+**Every datapill is compared against what was sent.** The whole-tree write the
+API also offers, `IMPORT_FLOW`, runs a schema migration on whatever it is given
+— so a flow already in live form comes back with every reference doubled,
+`{{step_1['output'].x}}` becoming `{{step_1['output']['output'].x}}`. That is
+structurally valid, passes Pipefy's own validator, and resolves to nothing at
+runtime. `pipe` never sends it, compiles to the per-node operations instead, and
+reads the stored bytes back to prove nothing was rewritten.
+
+### Flags
+
+```bash
+pipe pull <pipeId>          # flows included
+pipe diff                   # flows diffed offline, against the recorded baseline
+pipe diff --against live    # re-read the pipe instead
+pipe apply                  # flows applied after the pipe, then verified
+pipe apply --allow-flow-enable   # permit a status change to ENABLED
+<any of the above> --no-flows    # skip the flow half entirely
+```
+
+`PLAN-IPAAS.md` has the design and the evidence behind it; the run logs under
+`workspaces/*/flows/BUILD-LOG.md` record what each build found.
+
+## Install
+
+One requirement: **Node 22.18 or newer**. That version strips TypeScript types
+natively, which is why there is no build step and no `dist/`.
+
+```bash
+git clone https://github.com/Guilherme-CA-Dias/pipefy-process-coder
+cd pipefy-process-coder
+npm link                 # puts `pipe` on PATH, pointing at this folder
+pipe --version           # confirms which clone you are running
+pipe doctor              # confirms it can reach Pipefy
+pipe skills install      # installs this repo's agent skills for every agent you use
+```
+
+The skills under [`.agents/skills/`](.agents/skills) — named `ppc-pipefy-*`, not
+just `pipefy-*`, to stay out of the namespace the official
+[pipefy/ai-toolkit](https://github.com/pipefy/ai-toolkit) already ships skills
+in — are what teaches an agent (Claude Code or otherwise) how to actually use
+this tool: the pull/edit/diff/apply loop, the pipe-authoring rules, the iPaaS
+flow traps, when to reach for which mechanism. They only trigger for a session
+opened inside this repo until installed somewhere every session reads from,
+because a pulled workspace (**Where `pipe` can run**, below) usually lives
+somewhere else entirely.
+
+`pipe skills install` doesn't copy them itself — it hands off to
+[`npx skills add`](https://github.com/vercel-labs/skills), which already
+solves "which agent's directory" for 75+ agents (Claude Code, Cursor, Codex,
+…), picking each one's own path and symlinking by default so there is one
+canonical copy. Every flag after `install` is forwarded to it verbatim —
+`pipe skills install -g -a claude-code -y`, `--skill`, `--copy`, and the rest
+of its README apply as-is. `pipe skills list` still shows what ships with this
+install, locally, with no network call.
+
+The source it hands `npx skills add` is a **local path** — this package's own
+install location on disk, resolved the same way `pipe --version` locates
+`package.json` — not a GitHub reference. That means `pipe skills install`
+only works for someone who already has this CLI installed some other way
+(the clone above, or `npm install -g` once published); it is not a way to get
+the skills without it, and this repo's visibility (public or private) is
+irrelevant to it. `pipefy/ai-toolkit`'s own `npx skills add pipefy/ai-toolkit`
+is the other pattern — a GitHub reference, installable with no CLI at all —
+which would need this repo to be public (or private with the installer's own
+git credentials already granted access) if we ever want that decoupled path
+too. Nothing here does that yet.
+
+```
+$ pipe --version
+pipefy-process-coder 0.1.0
+  node    v24.19.0 (needs >=22.18 for native type stripping)
+  running C:UsersyouCodepipefy-process-codersrccli.ts
+```
+
+**No `npm install` needed to run it.** Every import in `src/` is a `node:`
+builtin — child_process, crypto, fs, os, path, readline, url, util, zlib and
+nothing else. Verified by deleting `node_modules` entirely and running `pipe
+capabilities`, which worked. The two devDependencies (`typescript`,
+`@types/node`) are only for `npm run typecheck`.
+
+That matters for the actual use case: an FDE on a client-issued laptop, behind a
+proxy that blocks the npm registry, can clone and run.
+
+### Where `pipe` can run
+
+Anywhere. `npm link` symlinks a launcher into the global npm bin directory, so
+`pipe` is on PATH for every shell and every folder:
+
+```
+C:\Users\you\AppData\Roaming\npm\pipe   →   <the cloned repo>\bin\pipe.js
+~/.npm-global/bin/pipe                      →   <the cloned repo>/bin/pipe.js
+```
+
+Two consequences of it being a symlink rather than a copy: moving or deleting the
+clone breaks `pipe`, and `git pull` upgrades it instantly with nothing to rebuild.
+If you would rather have a copy, `npm i -g .` installs one.
+
+Commands that need no workspace — `login`, `doctor`, `pull`, `create`, `export`,
+`capabilities` — run from any directory. The rest (`validate`, `diff`, `plan`,
+`apply`, `verify`, `status`, `runs`) work on a workspace, and find it by walking
+**up** from the current directory looking for `workspace.json`. So either `cd`
+into the workspace, or pass its path:
+
+```bash
+cd ./my-pipe && pipe diff          # found by walking up
+pipe diff ./my-pipe               # or name it from anywhere
+```
+
+### Other machines, other platforms
+
+The package is `private: true`, so it is not on npm — distribution is the git
+clone above. Same three commands on any machine.
+
+| | |
+| --- | --- |
+| Node | ≥ 22.18, for native type stripping. Older Node cannot run `.ts` at all. |
+| npm install | not required to run; only for `npm run typecheck` |
+| Auth, easy path | `uv tool install pipefy-cli`, then `pipe login` opens a browser |
+| Auth, no-uv path | `pipe login --token <api token>` — a Pipefy personal token, stored in `~/.ppc/credentials.json` |
+
+**Not Windows-only.** Nothing in the tool assumes a shell, and every
+platform-specific branch is explicit: `.exe` suffixes and `Scripts/` vs `bin/`
+when locating the toolkit, `cmd /c start` vs `open` vs `xdg-open` for
+`pipe diff --open`, `USERPROFILE` vs `HOME`, and `~/.ppc` from `os.homedir()`.
+macOS `~/Library/Application Support/uv/tools` is looked in too.
+
+Being honest about the limit of that claim: it has been **run** only on Windows
+11 with Node 24. The code is written for all three platforms and the tests are
+pure Node, but macOS and Linux are untested rather than known-good.
+
+## Authentication
+
+`pipe` does not implement Pipefy login. It borrows the session from the official
+[Pipefy AI toolkit](https://github.com/pipefy/ai-toolkit), so one browser
+sign-in serves this tool, the `pipefy` CLI and the Pipefy MCP server.
+
+```bash
+uv tool install pipefy-cli          # once
+pipe login                          # runs `pipefy auth login` for you
+pipe login --status                 # who am I, and via which source
+```
+
+Resolution order:
+
+| | Source | When |
+| --- | --- | --- |
+| 1 | `PIPEFY_TOKEN` / `PPC_TOKEN` | explicit override; CI |
+| 2 | the toolkit | its own precedence: static token → service account → keychain session |
+| 3 | `~/.ppc/credentials.json` | `pipe login --token <token>`, for machines without the toolkit |
+
+The toolkit issues **five-minute** access tokens, so the credential is a
+provider, not a string: it is fetched per process and re-fetched once when the
+API answers 401, which is the difference between a long apply finishing and
+failing at step 90. Nothing long-lived is ever written to disk by this tool.
+
+[`src/pipefy/toolkit_bearer.py`](src/pipefy/toolkit_bearer.py) runs inside the
+toolkit's own environment and prints one bearer as JSON, so the keychain
+backend, the refresh grant and the credential precedence stay the toolkit's
+rather than being reimplemented against a private store shape.
+
+> **Windows note.** `pipefy auth login` fails with `WinError 1783 CredWrite` out
+> of the box: Credential Manager caps a credential blob at ~2.5 KB and a Keycloak
+> session is larger. The toolkit's documented unblock is
+> `keychain_backend = "file"` in `%APPDATA%\pipefy\config.toml`, which stores the
+> session in **plaintext** at `%APPDATA%\pipefy\keyring.cfg`. `pipefy auth logout`
+> clears it.
+
+No build step and no runtime dependencies: Node ≥ 22.18 strips the types itself,
+so `node src/cli.ts` works on a clean checkout. That matters for a tool an FDE
+runs on a client's laptop.
+
+## Two read paths, one shape
+
+`pipe pull` uses whichever is available, and both produce the identical
+snapshot-shaped payload, so the codec, diff, plan and apply are written once.
+
+| | Snapshot | Reconstruction |
+| --- | --- | --- |
+| When | the org has `createRepoSnapshot` | it does not, or `--no-snapshot`, or the repo is a table |
+| Cost | ~63 s and a new S3 object per pull | a handful of queries, no snapshot rows |
+| Coverage | all 20 entity arrays | 12 groups; the rest are marked **unknown** |
+
+The reconstruction path is the Change Migrator recipes, generalised. Everything
+it reads, it reads the way they do:
+
+- structure from `pipe { phases { fields } start_form_fields }` — with
+  `internal_id` as the payload's `id` and the public `id` as the slug, which is
+  what every field reference in every automation depends on
+- field conditions from `pipe.fieldConditions`, split back into the four payload
+  arrays a snapshot would have produced
+- automations from the internal `getAutomations` plus `loadAutomationToEdit` per
+  automation — the only read path that returns `action_params.field_map`,
+  `searchFor`, `responseSchema` and `aiParams`
+- phase jumps from `GET /internal_api/settings/phases/:id`
+
+**Coverage is tracked per read, and absence is not emptiness.** A reconstructed
+pull cannot tell "this pipe has no webhooks" from "I have no way to read
+webhooks", so absence from an incomplete read is treated as no information at
+three levels:
+
+| Level | Measured on a real pipe (8 phases, 58 fields, 14 automations) | Without the rule |
+| --- | --- | --- |
+| group | `visibilities` has no public query | every reconstructed tree "empties" them |
+| column | the public API has no `color_uuid`, `only_admin_can_move_to_previous` | ~10 phantom changes per phase, each proposing null |
+| row | the internal automations list returns **13 of 15** | two live automations proposed for deletion |
+| collection | no `field_map` for `update_card_field` automations | "delete every field mapping" |
+
+Suppressions are reported, never silent: they appear under *not diffed* with the
+row named. The rules are the subject of [test/coverage.test.ts](test/coverage.test.ts).
+
+### Verified by reading the same pipe twice
+
+The two paths are checked against each other — snapshot vs reconstruction of one
+live pipe. They now agree on the repo row, all 9 phases, all 55 fields, field
+conditions, labels and webhooks. The only remaining difference is real: the
+snapshot payload carries **orphaned `phase_jumps`** pointing at phases that no
+longer exist, and the live endpoint does not. The compiler drops those out of the
+set-replacing write rather than resending them.
+
+## Progress
+
+`pull`, `apply` and `apply --dry-run` show a live task list: one line per repo
+or plan step, with a spinner, elapsed time and the current stage. It exists
+because a snapshot takes ~63 s and silence for a minute is indistinguishable from
+a hang.
+
+Two renderers, chosen by whether stderr is a terminal. The plain one is not a
+fallback — it is what CI, `2>file` and a backgrounded run get:
+
+- **live**: repaints a block in place, capped to a window so a 200-step plan
+  cannot scroll the terminal, and restores the cursor on Ctrl-C
+- **plain**: one line per task outcome, plus a heartbeat every 15 s for work that
+  is still running, and no cursor control at all
+
+While a display is up it owns the screen: `log.ts` routes ordinary output through
+it, so an interleaved write cannot land inside a repaint. Set `PPC_NO_TUI=1` to
+force plain. Everything goes to stderr, so `--json` on stdout stays clean.
+
+## The diff
+
+`pipe diff` is modelled on what the recipes do: one change list per entity type,
+each row tagged create / update / delete, walked in dependency order.
+
+```bash
+pipe diff                          # against the last pull
+pipe diff --against live           # re-read the pipe now
+pipe diff --against 302208097      # cross-repo: a dev→prod migration diff
+pipe diff --open                   # the HTML view
+```
+
+`--against <another pipeId>` is the recipes' actual job. It runs in **cross-repo** mode,
+where matching falls back to the natural key (phase name, phase + field label,
+automation name) exactly as the recipes match, and the resulting id map is what
+lets embedded field references be rewritten on apply — the work the recipes do
+with a Pipefy table keyed `dev_pipe_field_internal_id` and one API call per id.
+
+Same-repo mode matches on uuid, then id, which is exact and needs no lookups.
+
+## What it refuses, and why that is the point
+
+`pipe plan` classifies every change against a hand-maintained registry
+(`src/apply/registry.ts`) and **blocks** on anything unsupported, naming the
+entity and the reason. The worst failure mode available to this tool is an apply
+that reports success having silently ignored something.
+
+| Refused | Why |
+| --- | --- |
+| a field's `type_id` | `updatePhaseField` has no `type_id`; delete + create destroys every value |
+| a field's `phase_id` | same, and the diff reconciles the delete/create pair into one refusal |
+| reordering phases | `createPhase` takes an index, `updatePhase` does not |
+| anything under `_readonly/` | no public write path — rejected on the path alone |
+| deleting the start form | it is a real phase at index 0 |
+| deleting a phase | needs `--allow-card-moves`: cards must move first |
+
+`pipe capabilities` prints the whole registry. Each refusal names the platform ask
+that would remove it (`ROUTE-B-V2.md` §8).
+
+## Safety
+
+- `pack(unpack(P)) ≡ P` under a canonical form, checked on every pull and
+  runnable as `pipe roundtrip`. A codec that loses data is caught before it can
+  corrupt a pipe, not after.
+- Validation is two passes — shape, then referential integrity — and
+  distinguishes **pre-existing** problems from introduced ones. Pipe <pipeId>
+  ships with five `phase_jumps` pointing at phases that no longer exist; without
+  that distinction a freshly pulled, unedited workspace fails validation.
+- `pipe apply` snapshots first (labelled `pre-apply/<ts>`), persists every step's
+  status to `.ppc/plans/`, so a failure is resumable with `--resume`.
+- Ids that do not exist yet are placeholders resolved at send time, deep-walked
+  through the opaque JSON that `action_params`, `event_params`, `search_for` and
+  condition values really are.
+- **Verification asks whether the pipe satisfies the intent**, not whether two
+  trees are identical. A newly authored entity is a partial specification: the
+  author writes what they care about and the server fills `slug`, `card_synced`,
+  `unique` and the connector flags. Comparing full rows failed a correct apply
+  with seven differences per created field.
+- **The loop is idempotent, and no longer needs a read-back to stay that way.**
+  A create mutation's own response already carries the id — the same one
+  `%{_new:<uuid>}` substitution for later steps in the same run gets it from —
+  so `pipe apply` writes it straight into the file and the baseline, no network
+  read required. A created entity otherwise still reads `"id": null, "_new":
+  true"` where it was written, and the next apply would make a duplicate.
+- **A second, live-read snapshot is never taken on its own initiative.** That
+  read-back is exactly what used to make the pre-apply rollback snapshot stop
+  being restorable the moment the apply that took it finished (`pipe rollback`
+  only restores the *latest* one). Verification instead defaults to a free
+  GraphQL reconstruction; a full snapshot-based verify-and-adopt is offered
+  as a yes/no prompt (default no), or `--post-snapshot` to skip asking.
+- **On a failed or partial apply, `pipe apply` offers to roll back instead** —
+  to the pre-apply snapshot, which is normally still the latest one now that
+  nothing else has snapshotted the repo in between. Declining (the default)
+  leaves the record for `pipe apply --resume` or a manual `pipe rollback`.
+  The "is it still the latest" check this depends on was broken until it was
+  actually exercised against a real failure — see `Pipe.snapshots(last: N)`
+  above — and is now verified live: a real API rejection, a real restorable
+  check, a real `restoreRepoToSnapshot` that reaches `SUCCEEDED`.
+- **One apply, one revert question — not three mechanisms scoped separately.**
+  On any failure partway — pipe, flow, or agent — `pipe apply` asks exactly
+  once whether to revert *everything* that run changed, not just whichever
+  half happened to error. Each part is reverted the way its platform actually
+  allows: pipe structure via `restoreRepoToSnapshot` (Pipefy's own mechanism,
+  only offered while that snapshot is still the latest); flows and agents —
+  neither ever had a Pipefy snapshot — by diffing the current live state
+  against the read taken right before this apply touched it and sending the
+  reverse plan through the same compiler every ordinary apply uses. Rendered
+  in full before the one question is asked; reverted, on confirmation, in the
+  reverse of the order they were applied. Still true regardless: no revert
+  un-sends an email or un-fires a webhook an automation already triggered —
+  that's the platform's own irreversibility, not something a bigger rollback
+  fixes. Verified live: a real API rejection, the correct reverse plan, and
+  the flow back to its exact original chain once applied.
+- No draft rehearsal: `createRepoDraft` does not resolve on this endpoint, and
+  the draft's relation semantics could mutate production connected repos. It
+  slots in ahead of the pre-apply snapshot when it becomes available.
+
+## What the live endpoint actually answers
+
+Checked against `api.pipefy.com` and pipe <pipeId>, not assumed. Several of
+these contradict `PLAN.md`, which is why the read paths are introspection-driven
+rather than hard-coded.
+
+| | Reality |
+| --- | --- |
+| `createRepoSnapshot`, `renameRepoSnapshot`, `createPresignedUrl` | present |
+| `restoreRepoSnapshot`, `createRepoDraft`, `importRepoSnapshot` | **absent** — no atomic rollback, no rehearsal, no Route B |
+| the 28 mutations the registry needs | all present |
+| `Pipe.snapshots` | a Relay `RepoSnapshotConnection` (`nodes`), not a flat array |
+| `Pipe.snapshots(last: N)` | **not the most recent N.** Measured on a pipe with 86 snapshots: `last: 1` returned `sequenceIndex: 1`, the *oldest* one; `first: 1` correctly returned `sequenceIndex: 86`. The connection is ordered newest-first internally, the opposite of the usual Relay convention — `SnapshotClient.list()` now asks with `first`. This silently broke `pipe rollback`'s and `pipe apply`'s "is this still the latest snapshot" check, which used `last` and compared against the oldest one instead |
+| `Pipe.snapshot(versionId:)` | takes `ID`; declaring `String!` fails the query |
+| `RepoSnapshotStatus` | `PENDING \| UPLOADED \| FAILED` — **UPLOADED** is success |
+| `pipe(id:)` vs `table(id:)` | **both answer for every repo**; `Table.url` (`/pipes/` vs `/apollo_databases/`) is the only discriminator |
+| phase fields | type `PhaseField`; `synced_with_card`, `canCreateNewConnected`, `canConnectExisting`, `canConnectMultiples`, no `unique` |
+| `Field.connectedRepo` | a `PublicRepoUnion` of `PublicPipe \| PublicTable` |
+| `Phase.next_phase_ids` | public, but **forward jumps only** — not the payload's `phase_jumps` |
+| `GET settings/phases/:id` | answers `data.attributes.jump_targets`; the **PUT** answers `data.next_phase_ids` — different shapes |
+| internal `AutomationActionParams` | has `taskParams`, `aiBehaviorParams`, `authenticationType` that the Change Migrator query omits |
+| `CreatePhaseFieldInput` | has **no `uuid`** — the server mints it, so a client uuid is only a local handle |
+| renaming a field | does **not** regenerate the slug. `triage_owner` survived a rename to "Triage owner (FDE)", so `PLAN.md` §9's worry about breaking slug-keyed integrations does not apply |
+
+## Deviations from PLAN.md
+
+Three, all deliberate:
+
+1. **CLI only.** The Electron shell of `PLAN.md` phase 3 was dropped, not built:
+   `pipe diff --open` opens the HTML view in the browser.
+2. **A phase's jump set lives in its phase file** as `jump_target_ids`, not in a
+   separate `phase-jumps.json`. The internal endpoint is per-phase and
+   set-replacing, so a set is the honest shape and there is no second file to
+   keep consistent.
+3. **`webhooks.json` is one array file**, matching `labels.json`, rather than a
+   directory.
+
+## Layout
+
+```
+src/
+  cli.ts            command dispatch
+  config.ts         endpoints, token resolution, limits
+  pipefy/           graphql client · internal API adapter · snapshots ·
+                    reconstruction · discovery · capability detection
+  codec/            unpack · pack · round-trip comparator
+  workspace/        layout · write · read · lock · generated docs
+  validate/         shape · referential integrity
+  diff/             change set · text render · HTML view
+  apply/            registry · mutation builders · compiler · id map · executor ·
+                    flowops (iPaaS operations)
+  report/           run records · the integration surface of a pipe
+  commands/         one file per command
+test/               codec, workspace, diff/plan, id map
+research/snap.json  the pipe <pipeId> fixture every test runs against
+```
+
+## Commands
+
+| | |
+| --- | --- |
+| `pipe --version` | version, Node version, and which clone is on PATH |
+| `pipe login` | store a token in `~/.ppc/credentials.json` |
+| `pipe doctor` | capabilities, missing mutations, internal-API status |
+| `pipe skills` | the agent skills this install ships with; `install` hands off to `npx skills add` |
+| `pipe create <name>` | create an empty pipe and pull it, ready to be built from files |
+| `pipe pull <id>` | pull into a workspace; `--depth`, `--reuse-snapshot`, `--from-file` |
+| `pipe export <id>` | one repo as a single payload, no workspace, no token needed to consume |
+| `pipe validate` | offline shape + integrity |
+| `pipe diff` | baseline, live, or cross-repo; `--html`, `--open`, `--json` |
+| `pipe plan` | classify and order; works offline for classification |
+| `pipe apply` | snapshot, execute, adopt ids, verify for free; `--resume`, `--dry-run`, `--post-snapshot` |
+| `pipe verify` | read back and diff against the workspace |
+| `pipe rollback` | restore a snapshot (needs `restoreRepoSnapshot`) |
+| `pipe status` | what is in the workspace and how it was read |
+| `pipe capabilities` | the write-coverage registry |
+| `pipe runs` | every run recorded in the workspace; `--last`, `--show`, `--failed` |
+| `pipe roundtrip` | codec self-check |
+
+`pipe export` and `pipe pull --from-file` pair up — see **The two workflows**
+above for when that is the right shape, and why `--envelope` matters.
+
+## Runs
+
+Off by default. Pass `--report` to a pull, apply, dry run or verify and it
+writes a folder into the workspace recording what happened:
+
+```
+runs/index.md                         one line per run, oldest first
+runs/<when>--<kind>/report.md         what happened, readable, no credentials
+runs/<when>--<kind>/run.json          the same thing structured
+runs/<when>--<kind>/pipe.json         the payload as of the end of that run
+runs/<when>--<kind>/integrations.json webhooks, relations, cross-repo automations
+```
+
+The reason this is a file and not terminal scrollback is that the write path is
+non-atomic. When an apply half-succeeds, the plan file already says what the
+tool meant to send; the run folder says which of those calls landed, in what
+order, with what error, and what the pipe looked like on either side — an apply
+also writes `pipe.before.json`, so both sides of the change sit in one place.
+
+`integrations.json` is the outward-facing surface pulled out on its own:
+webhooks, pipe relations, automations that reach another repo, connector fields,
+inbound email. Those are the parts of a change whose blast radius leaves the
+pipe, and the parts that carry credentials — so `report.md` names hosts and ids
+and never a secret, while the two payload copies are covered by the generated
+`.gitignore`.
+
+It is opt-in because a run folder is a full copy of the pipe: one per command
+would grow the workspace without bound and put webhook URLs and HTTP-request
+credentials in one more place. The record an apply *needs* is the plan file
+under `.ppc/plans/`, which is written either way and is what `--resume` reads.
+
+A finished plan carries its outcome in the filename, so the directory reads as a
+history without opening anything:
+
+```
+.ppc/plans/
+  2026-09-01T20-14-12Z--945e060b--applied.json
+  2026-09-01T20-04-15Z--29374430--partial.json
+  2026-09-01T20-00-46Z--9588c4ac--failed.json
+  2026-09-01T19-58-02Z--cdf84d72.json            <- no suffix: never applied
+```
+
+The plan id does not change, so `pipe apply --resume <id>` keeps working whatever
+the file is called.
+`runs/` is for the times you want the human account kept — a migration someone
+will ask about later, a change with an audit trail, a sequence of runs compared
+side by side.
+
+```bash
+pipe apply --report       # keep the account of this one
+pipe runs                 # what has been recorded in this workspace
+pipe runs --last          # the newest report in full
+pipe runs --failed        # only the ones that did not finish
+```
+
+## Secrets
+
+A pulled workspace contains webhook URLs — possession is authorization — and
+HTTP-request automations can carry live credentials in `action_params`. A
+`.gitignore` is generated on pull and the first pull warns. Do not commit a
+workspace.
+
+## Tests
+
+```bash
+npm test          # node --test, no test framework to install
+npm run typecheck # tsc --noEmit, strict
+```
+
+Every test runs against `research/snap.json`, the real pipe <pipeId> snapshot
+captured during the research in `PLAN.md`.
+
+
+
+## Internal
+```bash
+pipe pull <pipeId> --out ./my-pipe   # ~63s per repo (snapshot generation)
+cd my-pipe-request
+# ...edit the JSON, with Claude or by hand...
+pipe validate        # offline, instant
+pipe diff            # or: pipe diff --open  for the HTML view
+pipe plan --dry-run  # refuses what can't be applied, before anything is sent
+pipe apply           # snapshot → apply → verify → sync the files back
+```
+
+Three things that differ from a plain git workflow
+
+The diff has three modes, and only the first is "what did I change":
+pipe diff                      # vs .ppc/baseline — your edits
+pipe diff --against live       # re-read the pipe now — did someone else touch it?
+pipe diff --against 302208097  # cross-repo: a dev→prod migration diff
+pipe diff --open               # HTML view
+
+pipe plan will refuse things, by design — a field's type_id, a field's phase_id, reordereadonly/. It names the entity and the reason rather than applying a partial change andreporting success.
+
+A successful pipe apply rewrites your files. It adopts each created entity's real id straight from the create response — no read-back needed — because a created entity is written as "id": null, "_new": true and a second apply would otherwise duplicate it. Verification then runs for free against a GraphQL reconstruction unless you ask for a full snapshot-based one (pipe apply --post-snapshot), since a second snapshot is what used to make the pre-apply rollback point stop being restorable. A failed or partial apply offers one combined revert covering everything that run actually changed — pipe structure via the pre-apply snapshot, plus flows and agents via their own reverse plan, since neither ever had a Pipefy snapshot to begin with. Rendered in full before asking once; declining leaves the files untouched so --resume works. It cannot un-send an email or un-fire a webhook an automation already triggered — that's the platform's own irreversibility, not something the revert can reach.
+
+
+## TODO
+
+[] - Define dynamic content in the construction
+[] - Index the workspace faster so it doesnt consume much tokens
+[x] - After applying, update the baseline
+[] - Add verbose config
+[x] - add Databases
+[] - add possibility to add pipe to a workspace
+[] - Make the apply verbose on plan items
+[] - Check the baseline is not stale before planning: compare the pulled version id against the live one, and refuse to propose deletions when the pipe has moved since the pull. A stale baseline turns someone else's changes into a diff — seen live, where a verification diff listed five changes made elsewhere as deletions the workspace wanted
+[] - pipe pull inside a workspace pulls the pipes automatically
+[] - One design point worth stating
+Each repo still applies independently — own plan file, own pre-apply snapshot, own verify. That's deliberate: they're separate repos with separate baselines, and a failure in one must not roll back or abandon another that already succeeded. So it's a loop over the real thing, not a merged mega-plan. Exit code is the worst of them. --plan and --resume stay single-repo, since a saved plan belongs to exactly one.