@kici-dev/compiler 0.1.21 → 0.1.23

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/dist/cli.js +25 -7
  2. package/dist/commands/compile.d.ts +6 -0
  3. package/dist/commands/compile.js +6 -3
  4. package/dist/commands/docs.d.ts +8 -8
  5. package/dist/commands/docs.js +35 -16
  6. package/dist/commands/held-run-client.d.ts +7 -2
  7. package/dist/commands/held-run-client.js +9 -3
  8. package/dist/commands/held-run-resolve.d.ts +5 -0
  9. package/dist/commands/login.d.ts +2 -0
  10. package/dist/commands/login.js +15 -7
  11. package/dist/commands/org.js +2 -2
  12. package/dist/commands/run-hold-watch.d.ts +57 -0
  13. package/dist/commands/run-hold-watch.js +87 -0
  14. package/dist/commands/run.d.ts +23 -0
  15. package/dist/commands/run.js +150 -17
  16. package/dist/commands/test.d.ts +14 -0
  17. package/dist/commands/types.d.ts +2 -0
  18. package/dist/commands/types.js +1 -1
  19. package/dist/fixtures/describe-event.d.ts +6 -0
  20. package/dist/fixtures/describe-event.js +18 -0
  21. package/dist/fixtures/picker.d.ts +19 -0
  22. package/dist/fixtures/picker.js +64 -0
  23. package/dist/llm-context/llms-architecture.txt +1440 -0
  24. package/dist/llm-context/llms-cli.txt +2386 -0
  25. package/dist/llm-context/llms-features.txt +2389 -0
  26. package/dist/llm-context/llms-full.txt +1304 -349
  27. package/dist/llm-context/llms-getting-started.txt +519 -0
  28. package/dist/llm-context/llms-patterns.txt +1324 -0
  29. package/dist/llm-context/llms-providers.txt +805 -0
  30. package/dist/llm-context/llms-sdk.txt +3725 -0
  31. package/dist/llm-context/llms.txt +15 -1
  32. package/dist/local-executor/index.js +40 -3
  33. package/dist/local-executor/job-runner.d.ts +2 -0
  34. package/dist/local-executor/job-runner.js +37 -4
  35. package/dist/local-executor/types.d.ts +2 -0
  36. package/dist/lockfile/generator.js +46 -20
  37. package/dist/remote/config.d.ts +2 -0
  38. package/dist/remote/config.js +1 -0
  39. package/dist/remote/platform-client.d.ts +12 -1
  40. package/dist/remote/uploader.js +1 -0
  41. package/dist/templates/package-json.js +1 -1
  42. package/dist/test-runner/rule-evaluator.d.ts +1 -1
  43. package/dist/test-runner/rule-evaluator.js +2 -1
  44. package/dist/test-runner/step-context.d.ts +1 -1
  45. package/dist/test-runner/step-context.js +8 -2
  46. package/dist/types.d.ts +15 -6
  47. package/package.json +4 -4
  48. package/sbom.spdx.json +35 -35
@@ -12,75 +12,75 @@ Documentation for workflow authors -- people writing CI/CD pipelines in TypeScri
12
12
 
13
13
  ## Pages
14
14
 
15
- ### [Getting started with KiCI](getting-started.md)
15
+ ### [Getting started with KiCI](https://docs.kici.dev/user/getting-started/)
16
16
 
17
17
  Install the SDK and compiler, write your first workflow, compile it to a lock file, and test it locally with simulated events. Covers prerequisites (Node.js 24+, pnpm), the `kici init` command for scaffolding, and the relationship between workflows, the lock file, and the three-tier runtime.
18
18
 
19
- ### [SDK reference](sdk-reference.md)
19
+ ### [SDK reference](https://docs.kici.dev/user/sdk-reference/)
20
20
 
21
21
  Complete API reference for `@kici-dev/sdk`. Covers all factory functions (`workflow()`, `job()`, `step()`), trigger builders (`pr()`, `push()`), rule functions (`rule()`, `skip()`), matrix configuration (static arrays, static objects, dynamic functions, include/exclude), and the `StepContext` interface that steps receive at runtime.
22
22
 
23
- ### [Lock file and workflow drift](lock-file-and-drift.md)
23
+ ### [Lock file and workflow drift](https://docs.kici.dev/user/lock-file-and-drift/)
24
24
 
25
25
  Why the lock file must stay in sync with workflow source, how to commit both together, using pre-commit and CI to catch drift early, and the agent-side hash verification when compiling from source.
26
26
 
27
- ### [CLI reference](cli-reference.md)
27
+ ### [CLI reference](https://docs.kici.dev/user/cli-reference/)
28
28
 
29
29
  All CLI commands provided by `@kici-dev/compiler`: `kici compile` (with watch mode and check mode), `kici run` (local and remote execution), `kici test` (event simulation with dry-run, filtering, debug output, and custom payloads), `kici login`/`logout`/`org` (authentication and org management), `kici diagnostics` (infrastructure tree) and `kici runs` (`list`/`show`/`logs`/`rerun`/`cancel` — run management), `kici secrets` (secret listing), `kici types` (type generation), `kici fixture` (generate test payloads), `kici init` (interactive project scaffolding), `kici hook` (pre-commit hook installation), `kici endpoints` (webhook entrypoints), and `kici workflows` (workflow listing). Includes environment variables and exit codes.
30
30
 
31
- ### [Workflow patterns](workflow-patterns.md)
31
+ ### [Workflow patterns](https://docs.kici.dev/user/workflow-patterns/)
32
32
 
33
33
  Common patterns for building real-world CI/CD workflows. Includes examples for basic CI pipelines with job dependencies, monorepo path-based triggering, conditional jobs with rules, matrix builds across Node versions, dynamic jobs generated at runtime, Docker-based step execution, and parallel test splitting.
34
34
 
35
- ### [Dashboard](dashboard.md)
35
+ ### [Dashboard](https://docs.kici.dev/user/dashboard/)
36
36
 
37
37
  Guide to the KiCI web dashboard. Covers navigation (sidebar, org switcher, mobile bottom tabs), run list (table columns, filters, pagination, empty states), run detail (resizable two-panel layout, job tree, step selection, metadata tabs), log viewer (ANSI color rendering, search, permalink, copy), settings page (tabbed layout), theme toggle, keyboard shortcuts, and error pages.
38
38
 
39
- ### [Testing guide](testing-guide.md)
39
+ ### [Testing guide](https://docs.kici.dev/user/testing-guide/)
40
40
 
41
41
  How to run and write tests for KiCI workflows, including remote test execution with `kici run remote`, fixture-based testing, and overlay mode for uncommitted changes.
42
42
 
43
- ### [Environments](environments.md)
43
+ ### [Environments](https://docs.kici.dev/user/environments/)
44
44
 
45
45
  Configure deployment environments (staging, production, review/\*) with variables, scoped secrets, and protection rules. Covers the SDK API (`environment`, `env`, `concurrencyGroup` on jobs), the 8-layer variable merge precedence, protection rules (branch restrictions, required reviewers, wait timers, concurrency), dashboard management, type generation, and migration from the legacy contexts system.
46
46
 
47
- ### [Environment variables](env-vars.md)
47
+ ### [Environment variables](https://docs.kici.dev/user/env-vars/)
48
48
 
49
49
  Reference for all `KICI_*` environment variables supported by the CLI. Covers authentication overrides (OIDC issuer, client ID, project ID), browser behavior (custom browser command, fixed callback port), development mode, and usage examples for CI/CD, self-hosted, and headless environments.
50
50
 
51
- ### [CLI authentication](cli-auth.md)
51
+ ### [CLI authentication](https://docs.kici.dev/user/cli-auth/)
52
52
 
53
53
  Authenticate the KiCI CLI with browser-based OAuth (default), device authorization flow (for headless environments), or API key paste (for CI/CD pipelines). Covers org management and PATs.
54
54
 
55
- ### [Event system](events.md)
55
+ ### [Event system](https://docs.kici.dev/user/events/)
56
56
 
57
57
  Event model concepts: event types, the registration model, event matching, and circuit breaker protection. Understanding this distinction is key to working with non-git triggers like schedules, custom events, and generic webhooks.
58
58
 
59
- ### [Lifecycle hooks](hooks.md)
59
+ ### [Lifecycle hooks](https://docs.kici.dev/user/hooks/)
60
60
 
61
61
  SDK hook API for cancel, cleanup, success, failure, and step-level callbacks. Hooks run at specific points in the execution lifecycle to react to outcomes and perform cleanup.
62
62
 
63
- ### [Concurrency groups](concurrency.md)
63
+ ### [Concurrency groups](https://docs.kici.dev/user/concurrency/)
64
64
 
65
65
  Control parallel execution with auto-cancel and queue modes. Prevent multiple workflow runs from executing in parallel when they target the same resource.
66
66
 
67
- ### [Dynamic values](dynamic-values.md)
67
+ ### [Dynamic values](https://docs.kici.dev/user/dynamic-values/)
68
68
 
69
69
  Compute `environment`, `env`, and `concurrencyGroup` at runtime based on the incoming event payload. Instead of hardcoding static strings, pass a function that receives the webhook event and returns the resolved value.
70
70
 
71
- ### [Secrets](secrets.md)
71
+ ### [Secrets](https://docs.kici.dev/user/secrets/)
72
72
 
73
73
  Access encrypted secrets in workflow steps via the explicit secrets API. Secrets are never auto-injected into `process.env` -- you must explicitly request each secret by name.
74
74
 
75
- ### [GitHub App provider](providers/github.md)
75
+ ### [GitHub App provider](https://docs.kici.dev/user/providers/github/)
76
76
 
77
77
  The flagship source. Covers creating the GitHub App on GitHub's side (permissions, webhook URL, private key), registering it with the orchestrator via `kici-admin source add github`, routing keys (`github:<appId>`), global-workflow policy, enriched Check runs on pull requests, private-key and webhook-secret rotation, and troubleshooting.
78
78
 
79
- ### [Universal-git provider](providers/universal-git.md)
79
+ ### [Universal-git provider](https://docs.kici.dev/user/providers/universal-git/)
80
80
 
81
81
  Connect a non-GitHub-App forge (Forgejo, Gitea, Gogs, GitLab, plain GitHub) to KiCI via its webhook. Covers preset selection, PAT and SSH credential wiring, credential rotation, global workflow policy against `generic:<orgId>:<sourceId>` routing keys, and troubleshooting.
82
82
 
83
- ### [Global workflows](global-workflows.md)
83
+ ### [Global workflows](https://docs.kici.dev/user/global-workflows/)
84
84
 
85
85
  Cross-repo workflows that let a single workflow repo define jobs which run on events from many source repos in the same org. Covers the mental model (workflow repo vs. source repo, authoring axis vs. source axis), SDK syntax for declaring globals via `repos:` patterns, the dashboard opt-in flow and per-setting semantics (master toggle, author allow-list, source deny-list, elevated-access list), the security model, and troubleshooting skipped dispatches.
86
86
 
@@ -136,7 +136,7 @@ npx kici init --mjs
136
136
 
137
137
  This creates `.mjs` workflow files that run directly without a build step.
138
138
 
139
- After running `kici init`, jump straight to [Compile the workflow](#compile-the-workflow) below to compile and preview the scaffolded workflow.
139
+ After running `kici init`, jump straight to [Compile the workflow](https://docs.kici.dev/user/getting-started/#compile-the-workflow) below to compile and preview the scaffolded workflow.
140
140
 
141
141
  ## Manual setup
142
142
 
@@ -215,7 +215,7 @@ This workflow:
215
215
  - Runs a `lint` job first
216
216
  - Runs a `test` job after lint succeeds (`needs: [lint]`)
217
217
 
218
- `runsOn` selects which agents may run a job. Every agent self-reports `kici:os:<platform>`, `kici:arch:<cpu>`, and `kici:host:<hostname>`, so `runsOn: 'kici:os:linux'` targets any connected Linux agent with zero configuration — `kici init` scaffolds workflows with exactly that. Use a custom label such as `'linux'` or `'gpu'` (defined in your scaler's `labelSet`) to target a specific pool instead. See the [runsOn forms](./sdk/core.md#runson-forms) reference for the full label model.
218
+ `runsOn` selects which agents may run a job. Every agent self-reports `kici:os:<platform>`, `kici:arch:<cpu>`, and `kici:host:<hostname>`, so `runsOn: 'kici:os:linux'` targets any connected Linux agent with zero configuration — `kici init` scaffolds workflows with exactly that. Use a custom label such as `'linux'` or `'gpu'` (defined in your scaler's `labelSet`) to target a specific pool instead. See the [runsOn forms](https://docs.kici.dev/user/sdk/core/#runson-forms) reference for the full label model.
219
219
 
220
220
  **Single-step shortcut.** If a job only has one step, pass `run` directly to `job()` instead of building a `steps: [step(...)]` array:
221
221
 
@@ -229,7 +229,7 @@ const deploy = job('deploy', {
229
229
  });
230
230
  ```
231
231
 
232
- `run` and `steps` are mutually exclusive. The shorthand is ideal for deploy/notify/smoke-test jobs. See [Single-step job shorthand](sdk/core.md#single-step-job-shorthand) in the SDK reference for details (output access on the resulting `job.result` is flat -- no step-name nesting).
232
+ `run` and `steps` are mutually exclusive. The shorthand is ideal for deploy/notify/smoke-test jobs. See [Single-step job shorthand](https://docs.kici.dev/user/sdk/core/#single-step-job-shorthand) in the SDK reference for details (output access on the resulting `job.result` is flat -- no step-name nesting).
233
233
 
234
234
  ## Compile the workflow
235
235
 
@@ -245,7 +245,7 @@ Expected output:
245
245
  ✓ Compiled workflows → .kici/kici.lock.json (1 workflow)
246
246
  ```
247
247
 
248
- The lock file (`kici.lock.json`) is a JSON representation of your workflow that the KiCI agent uses for execution. Commit this file alongside your workflow source. See [Lock file and workflow drift](lock-file-and-drift.md) for why and how to keep them in sync.
248
+ The lock file (`kici.lock.json`) is a JSON representation of your workflow that the KiCI agent uses for execution. Commit this file alongside your workflow source. See [Lock file and workflow drift](https://docs.kici.dev/user/lock-file-and-drift/) for why and how to keep them in sync.
249
249
 
250
250
  ## Preview trigger matching
251
251
 
@@ -316,7 +316,7 @@ Every `.kici/` dependency must be resolvable from the **single cloned repository
316
316
 
317
317
  In practice:
318
318
 
319
- - **From a registry** — the common case. Pin a published version (a private registry works — see [Private registries](./private-registries.md)). Available for any package manager.
319
+ - **From a registry** — the common case. Pin a published version (a private registry works — see [Private registries](https://docs.kici.dev/user/private-registries/)). Available for any package manager.
320
320
  - **From an in-repo workspace sibling** — if your `.kici/` is a member of a **pnpm workspace** or a **yarn berry workspace** (a `workspaces` array in the repo-root `package.json`), it can depend on a sibling package in the same repo via `workspace:*` (yarn berry also accepts `portal:`). The whole repo is cloned, so the sibling is present and resolves; the agent also builds your `.kici/` dependency closure after install, so a sibling's build output exists before the workflow that imports it loads. A `file:`/`link:`/`portal:` path is allowed only when it stays inside the repository.
321
321
 
322
322
  What fails fast (with an actionable error naming the dependency, not a raw package-manager error): a `workspace:` dependency in an **npm** project (npm has no workspace protocol — pin a published version or switch to pnpm), a `workspace:`/`portal:` dependency in a **yarn classic** project (v1 has neither — use a version range, pnpm, or yarn berry), a `workspace:` dependency in a **yarn berry** project whose repo-root `package.json` has no `workspaces` array, and any `file:`/`link:`/`portal:` path that points outside the cloned repo.
@@ -450,10 +450,10 @@ The compiler watches `.kici/workflows/*.ts` and recompiles on every save.
450
450
 
451
451
  ## Next steps
452
452
 
453
- - **[5-minute quickstart](quickstart.md)** -- ready to run your workflow on real infrastructure? Stand up an orchestrator + agent (Docker / Podman or bare metal)
454
- - **[SDK reference](sdk-reference.md)** -- complete API for workflows, jobs, steps, triggers, rules, and matrix
455
- - **[CLI reference](cli-reference.md)** -- all CLI commands with options and examples
456
- - **[Workflow patterns](workflow-patterns.md)** -- common patterns for real-world CI/CD workflows
453
+ - **[5-minute quickstart](https://docs.kici.dev/user/quickstart/)** -- ready to run your workflow on real infrastructure? Stand up an orchestrator + agent (Docker / Podman or bare metal)
454
+ - **[SDK reference](https://docs.kici.dev/user/sdk-reference/)** -- complete API for workflows, jobs, steps, triggers, rules, and matrix
455
+ - **[CLI reference](https://docs.kici.dev/user/cli-reference/)** -- all CLI commands with options and examples
456
+ - **[Workflow patterns](https://docs.kici.dev/user/workflow-patterns/)** -- common patterns for real-world CI/CD workflows
457
457
 
458
458
  ## How KiCI works
459
459
 
@@ -475,10 +475,10 @@ The lock file approach means the orchestrator stays git-agnostic -- it only need
475
475
 
476
476
  ## See also
477
477
 
478
- - [SDK reference](sdk-reference.md) -- complete API for workflows, jobs, steps, triggers, rules, and matrix
479
- - [CLI reference](cli-reference.md) -- all CLI commands with options and examples
480
- - [Workflow patterns](workflow-patterns.md) -- common patterns for real-world CI/CD workflows
481
- - [Architecture overview](../architecture/overview.md) -- how the three-tier runtime executes your workflows
478
+ - [SDK reference](https://docs.kici.dev/user/sdk-reference/) -- complete API for workflows, jobs, steps, triggers, rules, and matrix
479
+ - [CLI reference](https://docs.kici.dev/user/cli-reference/) -- all CLI commands with options and examples
480
+ - [Workflow patterns](https://docs.kici.dev/user/workflow-patterns/) -- common patterns for real-world CI/CD workflows
481
+ - [Architecture overview](https://docs.kici.dev/architecture/overview/) -- how the three-tier runtime executes your workflows
482
482
 
483
483
  ---
484
484
 
@@ -494,13 +494,13 @@ Each guide is split into two parts. **Part 1** gets you a green run against your
494
494
 
495
495
  Two containers brought up with `docker compose up -d` (orchestrator + PostgreSQL), plus one short-lived agent container spawned per job by the container scaler. Minimal host setup, perfect for a laptop, home server, or a tiny VM. No need to install PostgreSQL or any other system service.
496
496
 
497
- [Start with the Docker / Podman quickstart →](./quickstart/compose.md)
497
+ [Start with the Docker / Podman quickstart →](https://docs.kici.dev/user/quickstart/compose/)
498
498
 
499
499
  ## Option B — Bare-metal install
500
500
 
501
501
  Native systemd services managed by `kici-admin orchestrator install` / `kici-admin agent install` — the orchestrator and agents run as native processes. The backing PostgreSQL runs as a single container by default (one `docker compose up -d`), or you can install it natively if you'd rather not run a container runtime at all. Best for a long-lived Linux host.
502
502
 
503
- [Start with the bare-metal quickstart →](./quickstart/bare-metal.md)
503
+ [Start with the bare-metal quickstart →](https://docs.kici.dev/user/quickstart/bare-metal/)
504
504
 
505
505
  ## Which should I pick?
506
506
 
@@ -516,7 +516,7 @@ If you're not sure, pick Docker / Podman.
516
516
 
517
517
  ## Looking for the laptop-only path?
518
518
 
519
- Both quickstarts deploy a real orchestrator + agent. If you only want to write a workflow and dry-run it on your laptop with no infrastructure, [Getting started](./getting-started.md) covers `kici test` and `kici run local` instead.
519
+ Both quickstarts deploy a real orchestrator + agent. If you only want to write a workflow and dry-run it on your laptop with no infrastructure, [Getting started](https://docs.kici.dev/user/getting-started/) covers `kici test` and `kici run local` instead.
520
520
 
521
521
  ---
522
522
 
@@ -576,7 +576,7 @@ export default workflow('ci', {
576
576
  });
577
577
  ```
578
578
 
579
- The `test` and `typecheck` jobs both depend on `lint`, so they run in parallel after lint succeeds. KiCI validates the dependency graph at compile time -- cycles and missing references are caught before you commit. At runtime, jobs are gated on upstream completion: a job only dispatches after every entry in its `needs` array reaches a terminal state. If an upstream fails, downstream jobs skip by default (override per-edge with `ifFailed: 'run'`). See [Job dependencies (`needs`)](../sdk/core.md#job-dependencies-needs) in the SDK reference for the full matrix of `needs` forms (string, `Job` ref, `{ name, ifFailed }`, `dynamicGroup()`) and [needs-scheduler](../../architecture/execution/needs-scheduler.md) for the dispatch semantics.
579
+ The `test` and `typecheck` jobs both depend on `lint`, so they run in parallel after lint succeeds. KiCI validates the dependency graph at compile time -- cycles and missing references are caught before you commit. At runtime, jobs are gated on upstream completion: a job only dispatches after every entry in its `needs` array reaches a terminal status that satisfies the edge. If an upstream fails, downstream jobs skip by default (override per-edge with `when: 'always'`). See [Job dependencies (`needs`)](https://docs.kici.dev/user/sdk/core/#job-dependencies-needs) in the SDK reference for the full matrix of `needs` forms (string, `Job` ref, `{ name, when }`, `dynamicGroup()`) and [needs-scheduler](https://docs.kici.dev/architecture/execution/needs-scheduler/) for the dispatch semantics.
580
580
 
581
581
  **Single-step jobs don't need a `steps` array.** When a job only does one thing, pass `run` to `job()` instead of wrapping it in `steps: [step(...)]`:
582
582
 
@@ -597,7 +597,7 @@ export default workflow('smoke', {
597
597
  });
598
598
  ```
599
599
 
600
- `run` is mutually exclusive with `steps` (throws at compile time if both are set). Outputs are flat on `job.result` (no step-name nesting). See [Single-step job shorthand](../sdk/core.md#single-step-job-shorthand) in the SDK reference.
600
+ `run` is mutually exclusive with `steps` (throws at compile time if both are set). Outputs are flat on `job.result` (no step-name nesting). See [Single-step job shorthand](https://docs.kici.dev/user/sdk/core/#single-step-job-shorthand) in the SDK reference.
601
601
 
602
602
  ## PR-only workflow with branch filters
603
603
 
@@ -744,7 +744,7 @@ npx kici compile # regenerate .kici/kici.lock.json
744
744
  npx kici run local dispatch
745
745
  ```
746
746
 
747
- `kici run local` compiles the workflow, matches triggers against a simulated `dispatch` event, and executes the matched jobs directly on your machine with DAG-based scheduling. No webhook, no GitHub, no deployed orchestrator involved. See [`kici run local`](../cli-reference.md#kici-run-local) for options like `--job`, `--env`, `--json`, and `--junit`.
747
+ `kici run local` compiles the workflow, matches triggers against a simulated `dispatch` event, and executes the matched jobs directly on your machine with DAG-based scheduling. No webhook, no GitHub, no deployed orchestrator involved. See [`kici run local`](https://docs.kici.dev/user/cli-reference/#kici-run-local) for options like `--job`, `--env`, `--json`, and `--junit`.
748
748
 
749
749
  ### Scoping to a single workflow
750
750
 
@@ -1109,6 +1109,116 @@ This workflow:
1109
1109
 
1110
1110
  ---
1111
1111
 
1112
+ ## Host restart & wait-for-alive
1113
+
1114
+ Source: https://docs.kici.dev/user/patterns/host-restart/
1115
+
1116
+ When a KiCI agent runs on a host you are provisioning, a workflow can reboot
1117
+ that host and resume work once it comes back — the Ansible `reboot` +
1118
+ `wait_for_connection` pattern, expressed as two jobs pinned to the same host.
1119
+
1120
+ ## The two-job pattern
1121
+
1122
+ Host restart is a **job-boundary** capability: the reboot is the last step of a
1123
+ "restart" job, and the post-restart work lives in a **separate job** pinned to
1124
+ the same host that `needs` the restart job. The orchestrator holds the
1125
+ post-restart job until the host completes a reboot cycle, then dispatches it.
1126
+
1127
+ ```typescript
1128
+ import { workflow, job, step, restartHost, waitForHostAlive } from '@kici-dev/sdk';
1129
+
1130
+ export default workflow('patch-and-verify', {
1131
+ on: [
1132
+ /* ... */
1133
+ ],
1134
+ jobs: [
1135
+ // Restart job: apply updates, then reboot. restartHost() MUST be the last step.
1136
+ job('patch', {
1137
+ runsOn: 'kici:host:box-01',
1138
+ steps: [
1139
+ step('upgrade', async (ctx) => {
1140
+ await ctx.$`apt-get upgrade -y`;
1141
+ }),
1142
+ restartHost(),
1143
+ ],
1144
+ }),
1145
+ // Post-restart job: pinned to the SAME host, needs the restart job.
1146
+ job('verify', {
1147
+ runsOn: 'kici:host:box-01',
1148
+ needs: ['patch'],
1149
+ steps: [
1150
+ waitForHostAlive(() => fetch('http://localhost:8080/health')),
1151
+ step('check-service', async (ctx) => {
1152
+ await ctx.$`systemctl is-active myservice`;
1153
+ }),
1154
+ ],
1155
+ }),
1156
+ ],
1157
+ });
1158
+ ```
1159
+
1160
+ ## `restartHost()`
1161
+
1162
+ `restartHost(opts?)` reboots the host the job runs on. It signals the
1163
+ orchestrator that a reboot is pending (which holds the post-restart job and
1164
+ treats the agent's imminent disconnect as expected, not a failure), reports the
1165
+ step success, and the agent issues the OS reboot once the step completes.
1166
+
1167
+ - **Must be the last step** of its job — the job completes before the box goes
1168
+ down.
1169
+ - `deadlineMs` (optional) overrides how long the orchestrator waits for the host
1170
+ to return after the reboot. The default is the orchestrator's
1171
+ `KICI_HOST_REBOOT_DEADLINE_MS` (15 minutes). If the host does not reconnect by
1172
+ the deadline, the held post-restart job fails with a clear "host did not
1173
+ return after reboot" reason.
1174
+ - The reboot command is chosen per operating system (Linux `systemctl reboot`,
1175
+ macOS `shutdown -r now`, Windows `shutdown /r /t 0`).
1176
+
1177
+ ## `waitForHostAlive(probe, opts?)`
1178
+
1179
+ `waitForHostAlive()` is the optional first step of the post-restart job. The
1180
+ baseline "the host is back" guarantee comes for free — the post-restart job only
1181
+ dispatches after the agent reconnects. `waitForHostAlive(probe)` adds a
1182
+ **service-readiness** gate on top: it polls `probe` until it resolves, for hosts
1183
+ where "agent connected" does not yet mean "services ready".
1184
+
1185
+ - The probe can return anything (an HTTP response, an open port check, a marker
1186
+ file). Any non-null resolution means "ready"; a throw or rejection keeps
1187
+ polling.
1188
+ - `intervalMs` (default 3000) and `timeoutMs` (default 300000) tune the poll. If
1189
+ the probe never succeeds within `timeoutMs`, the step fails with "services did
1190
+ not come up".
1191
+
1192
+ ## Same-host pinning
1193
+
1194
+ The "same host" relationship is the pin: both jobs target the same host via
1195
+ `runsOn` (a `kici:host:<id>` label or another label the host carries), and the
1196
+ post-restart job `needs` the restart job. Durable provisioning hosts MUST set a
1197
+ stable agent id (`KICI_AGENT_ID`) so the host re-registers under the same
1198
+ identity after the reboot — that stable identity is what lets the orchestrator
1199
+ recognise the down-then-up cycle and release the held job.
1200
+
1201
+ ## Failure behavior
1202
+
1203
+ - **Host never returns by the deadline** → the held post-restart job fails.
1204
+ - **Reboot privilege denied** → the restart step fails with a clear privilege
1205
+ error (see the operator note below), and the orchestrator clears the
1206
+ reboot-pending hold.
1207
+ - **`waitForHostAlive` probe never succeeds** → that step fails.
1208
+
1209
+ The orchestrator refuses to reboot the host it runs on, so a co-located agent
1210
+ cannot take down the orchestrator's own box.
1211
+
1212
+ ## Operator prerequisite: reboot privilege
1213
+
1214
+ Rebooting needs host privilege. An agent used for host provisioning must be able
1215
+ to run the OS reboot primitive — run the agent service with reboot privilege, or
1216
+ grant a narrow `shutdown` / `systemctl reboot` permission. Agents used for
1217
+ provisioning generally need broad, near-root host privileges; see the operator
1218
+ agent documentation for the full posture.
1219
+
1220
+ ---
1221
+
1112
1222
  ## Integration patterns
1113
1223
 
1114
1224
  Source: https://docs.kici.dev/user/patterns/integrations/
@@ -1157,7 +1267,7 @@ export const postDeploy = workflow('post-deploy', {
1157
1267
 
1158
1268
  ### Same-run discovery → fan-out
1159
1269
 
1160
- A result-aware [`dynamicJob(group, { needs, generate })`](../sdk/rules-matrix-dynamic.md#dynamicjob--result-aware-generation) is deferred until its declared upstreams complete, then runs with their frozen outputs as `ctx.needs` — so a discovery job can emit a list at runtime and the generator fans out one follow-up job per item, all in the same run:
1270
+ A result-aware [`dynamicJob(group, { needs, generate })`](https://docs.kici.dev/user/sdk/rules-matrix-dynamic/#dynamicjob--result-aware-generation) is deferred until its declared upstreams complete, then runs with their frozen outputs as `ctx.needs` — so a discovery job can emit a list at runtime and the generator fans out one follow-up job per item, all in the same run:
1161
1271
 
1162
1272
  ```typescript
1163
1273
  import { workflow, job, step, push, dynamicJob, z } from '@kici-dev/sdk';
@@ -1263,7 +1373,7 @@ export default workflow('on-argocd-deploy', {
1263
1373
  });
1264
1374
  ```
1265
1375
 
1266
- See the [Operator guide: event routing](../../operator/event-routing.md) for how to set up generic webhook sources, verification methods, and trust relationships.
1376
+ See the [Operator guide: event routing](https://docs.kici.dev/operator/event-routing/) for how to set up generic webhook sources, verification methods, and trust relationships.
1267
1377
 
1268
1378
  ## Stripe webhook handler
1269
1379
 
@@ -1305,9 +1415,9 @@ export default workflow('stripe-invoice-handler', {
1305
1415
 
1306
1416
  **Prerequisites:**
1307
1417
 
1308
- - An operator must create a generic webhook source named `stripe` via the admin API. See [Operator guide: creating a source](../../operator/event-routing.md#creating-a-source).
1418
+ - An operator must create a generic webhook source named `stripe` via the admin API. See [Operator guide: creating a source](https://docs.kici.dev/operator/event-routing/#creating-a-source).
1309
1419
  - The `stripe-signing-key` secret must contain your Stripe webhook signing secret.
1310
- - This workflow uses the [registration model](../events.md#the-registration-model) -- it will not trigger until you push to your default branch.
1420
+ - This workflow uses the [registration model](https://docs.kici.dev/user/events/#the-registration-model) -- it will not trigger until you push to your default branch.
1311
1421
 
1312
1422
  ## Self-hosted git forge (Gogs, Forgejo, Gitea)
1313
1423
 
@@ -1356,7 +1466,7 @@ export default workflow('on-forgejo-push', {
1356
1466
 
1357
1467
  **Caveat — cloning:** generic webhook sources deliver only the payload; they do not carry a clone token, and KiCI's automatic pre-step clone (`packages/agent/src/checkout/git-clone.ts`) is GitHub-only today (HTTPS + `http.extraHeader` Basic auth with a GitHub installation token). Three practical patterns:
1358
1468
 
1359
- - **Mirror to GitHub and fan out.** Keep the repo on GitHub, register the workflow via a GitHub default-branch push, and have Gogs/Forgejo webhooks fan out via [cross-source delivery](../../architecture/webhooks/webhook-delivery.md#cross-source-delivery). The clone runs against the GitHub mirror using the GitHub App's token.
1469
+ - **Mirror to GitHub and fan out.** Keep the repo on GitHub, register the workflow via a GitHub default-branch push, and have Gogs/Forgejo webhooks fan out via [cross-source delivery](https://docs.kici.dev/architecture/webhooks/webhook-delivery/#cross-source-delivery). The clone runs against the GitHub mirror using the GitHub App's token.
1360
1470
  - **Clone yourself using a secret.** Set `checkout: false` on the job to skip the framework clone, store an SSH private key or forge personal access token as a secret, and run `git clone` explicitly in the first step. This works for any forge the agent can reach, no mirror needed. You still need a way to **register** the workflow — either keep a one-file GitHub repo whose only job is to own the registration, or bootstrap the registration manually against the orchestrator DB.
1361
1471
  - **Self-contained workflow.** No clone at all. The step reads whatever it needs from `rawPayload` (e.g., `rawPayload.after`, `rawPayload.repository.clone_url`) and drives external systems — notifications, deploys, third-party CI triggers.
1362
1472
 
@@ -1390,7 +1500,7 @@ HTTPS with a forge PAT works the same way — store the token as a secret, `awai
1390
1500
 
1391
1501
  - An operator must create a generic webhook source via `kici-admin source add generic` (see above).
1392
1502
  - The forge's webhook secret must match the `--secret` value.
1393
- - The workflow uses the [registration model](../events.md#the-registration-model) -- push to the default branch of a registered repo before the first webhook fires.
1503
+ - The workflow uses the [registration model](https://docs.kici.dev/user/events/#the-registration-model) -- push to the default branch of a registered repo before the first webhook fires.
1394
1504
 
1395
1505
  ## Plain GitHub repo webhooks (no GitHub App)
1396
1506
 
@@ -1452,12 +1562,12 @@ export default workflow('on-github-repo-push', {
1452
1562
  **What you lose compared to the GitHub App** (these are the same cloning / metadata caveats that apply to the Gogs/Forgejo pattern, plus GitHub-specific integrations):
1453
1563
 
1454
1564
  - No auto-clone — `packages/agent/src/checkout/git-clone.ts` uses GitHub App installation tokens to fetch the repo; a generic source has none. Either set `checkout: false` and clone yourself with a PAT/Deploy Key secret (same pattern as the Forgejo manual-clone example above), or keep the workflow self-contained.
1455
- - No lock-file fetch — the orchestrator cannot fetch `.kici/kici.lock.json` at the pushed SHA via the GitHub API. The workflow must be pre-registered via the [registration model](../events.md#the-registration-model); ad-hoc per-commit workflow discovery that a GitHub App push gives you is not available.
1565
+ - No lock-file fetch — the orchestrator cannot fetch `.kici/kici.lock.json` at the pushed SHA via the GitHub API. The workflow must be pre-registered via the [registration model](https://docs.kici.dev/user/events/#the-registration-model); ad-hoc per-commit workflow discovery that a GitHub App push gives you is not available.
1456
1566
  - No changed-files enrichment — `event.changedFiles` is empty. Use JSONPath `match` on `rawPayload.commits[*].added/modified/removed` if you need path filters.
1457
1567
  - No check-run integration — KiCI cannot post Check Run results back to GitHub.
1458
1568
  - Workflow authors must use `genericWebhook()`, not `push()` / `pr()` / `webhook()` — the latter three only match events delivered through the native GitHub App provider.
1459
1569
 
1460
- **When to use it anyway:** trigger-only workflows that don't need the cloned repo — posting Slack messages, kicking off external deploys, forwarding to downstream systems, or exposing GitHub repo events as `genericWebhook` for same-org [cross-source fan-out](../../architecture/webhooks/webhook-delivery.md#cross-source-delivery). For anything that compiles, tests, or checks code, install the GitHub App instead.
1570
+ **When to use it anyway:** trigger-only workflows that don't need the cloned repo — posting Slack messages, kicking off external deploys, forwarding to downstream systems, or exposing GitHub repo events as `genericWebhook` for same-org [cross-source fan-out](https://docs.kici.dev/architecture/webhooks/webhook-delivery/#cross-source-delivery). For anything that compiles, tests, or checks code, install the GitHub App instead.
1461
1571
 
1462
1572
  ## Nightly cron build
1463
1573
 
@@ -1590,7 +1700,7 @@ export default workflow('nightly-build', {
1590
1700
  **Notes:**
1591
1701
 
1592
1702
  - The `cron` field uses standard 5-field cron syntax. Use the `timezone` option (defaults to `'UTC'`) to control evaluation in a specific timezone: `schedule({ cron: '0 2 * * *', timezone: 'America/New_York' })`.
1593
- - Schedule workflows use the [registration model](../events.md#the-registration-model) -- the cron will not start firing until you push to your default branch.
1703
+ - Schedule workflows use the [registration model](https://docs.kici.dev/user/events/#the-registration-model) -- the cron will not start firing until you push to your default branch.
1594
1704
  - In clustered orchestrator deployments, only the Raft leader evaluates cron schedules. If the leader changes, the new leader recovers missed schedules.
1595
1705
 
1596
1706
  **Timing precision and scaling:**
@@ -1654,7 +1764,7 @@ export const deploy = workflow('deploy-on-success', {
1654
1764
 
1655
1765
  - `workflowComplete()` is a system event trigger -- the orchestrator automatically emits these events when workflows finish. You do not need to call `ctx.emit()`.
1656
1766
  - The `status` filter accepts `'success'`, `'failed'`, and `'cancelled'`. Omit `status` to trigger on any completion.
1657
- - The `deploy-on-success` workflow uses the [registration model](../events.md#the-registration-model) -- it will not trigger until you push to your default branch. The `build` workflow (using `push()`) works immediately.
1767
+ - The `deploy-on-success` workflow uses the [registration model](https://docs.kici.dev/user/events/#the-registration-model) -- it will not trigger until you push to your default branch. The `build` workflow (using `push()`) works immediately.
1658
1768
  - You can also use `jobComplete()` to trigger on individual job completions within a workflow.
1659
1769
 
1660
1770
  ## Custom event chaining
@@ -1726,8 +1836,8 @@ export const autoDeploy = workflow('auto-deploy', {
1726
1836
  - `defineEvent()` creates a typed contract using Zod. This is optional but recommended for documenting event payloads.
1727
1837
  - Custom events are delivered immediately when `ctx.emit()` is called (mid-workflow), not queued until the workflow completes.
1728
1838
  - Payload matching is available via the `match` option: `kiciEvent({ name: 'tests-passed', match: { '$.branch': 'main' } })`.
1729
- - The `auto-deploy` workflow uses the [registration model](../events.md#the-registration-model) -- it will not trigger until you push to your default branch.
1730
- - The [circuit breaker](../events.md#circuit-breaker) limits chain depth (default: 10) and rate (default: 100/min per workflow) to prevent infinite loops.
1839
+ - The `auto-deploy` workflow uses the [registration model](https://docs.kici.dev/user/events/#the-registration-model) -- it will not trigger until you push to your default branch.
1840
+ - The [circuit breaker](https://docs.kici.dev/user/events/#circuit-breaker) limits chain depth (default: 10) and rate (default: 100/min per workflow) to prevent infinite loops.
1731
1841
 
1732
1842
  ## Step context
1733
1843
 
@@ -1746,7 +1856,7 @@ Two surfaces drive the same cache:
1746
1856
  - **Declarative** — a `cache` field on a job or a step. The runtime restores before the work runs and saves after it succeeds, with no code in your step body.
1747
1857
  - **Imperative** — `ctx.cache.restore(spec)` / `ctx.cache.save(spec)` inside a step body, for fine-grained control over when restore and save happen.
1748
1858
 
1749
- The cache is backed by the orchestrator's object storage. Entries are isolated per organization and per ref scope (see [Isolation](#isolation)); no other tenant can read your cache, and an untrusted/fork ref can never poison the cache a trusted branch reads.
1859
+ The cache is backed by the orchestrator's object storage. Entries are isolated per organization and per ref scope (see [Isolation](https://docs.kici.dev/user/sdk/caching/#isolation)); no other tenant can read your cache, and an untrusted/fork ref can never poison the cache a trusted branch reads.
1750
1860
 
1751
1861
  ## CacheSpec
1752
1862
 
@@ -1863,18 +1973,18 @@ Cache storage is bounded per organization. Two mechanisms keep it bounded:
1863
1973
  - **Quota** — when a save pushes the org over its byte quota (`KICI_USER_CACHE_QUOTA_BYTES`, default 5 GiB), the oldest entries are evicted until the org is back under quota.
1864
1974
  - **TTL** — entries unused for `KICI_USER_CACHE_TTL_MS` (default 7 days) expire. The TTL refreshes on read (touch-on-read), so an actively used cache stays warm.
1865
1975
 
1866
- Both knobs are operator-configured on the orchestrator — see [orchestrator storage layout](../../operator/orchestrator/storage-layout.md).
1976
+ Both knobs are operator-configured on the orchestrator — see [orchestrator storage layout](https://docs.kici.dev/operator/orchestrator/storage-layout/).
1867
1977
 
1868
1978
  ## Observability
1869
1979
 
1870
- Each cache restore and save surfaces in the run timeline as a `cache:restore` / `cache:save` pseudo-step, reporting the outcome (hit/miss/saved, the matched key, bytes). The same outcomes are recorded as `cache.restore` / `cache.save` run events. See [data flows](../../architecture/data-flows.md#user-facing-cache-flow) for the restore/save protocol.
1980
+ Each cache restore and save surfaces in the run timeline as a `cache:restore` / `cache:save` pseudo-step, reporting the outcome (hit/miss/saved, the matched key, bytes). The same outcomes are recorded as `cache.restore` / `cache.save` run events. See [data flows](https://docs.kici.dev/architecture/data-flows/#user-facing-cache-flow) for the restore/save protocol.
1871
1981
 
1872
1982
  ## See also
1873
1983
 
1874
- - [Core](./core.md) -- `job()` / `step()` factories the `cache` field attaches to
1875
- - [Runtime](./runtime.md) -- `StepContext`, where `ctx.cache` lives
1876
- - [Orchestrator storage layout](../../operator/orchestrator/storage-layout.md) -- cache prefix, quota, TTL, and eviction
1877
- - [Data flows](../../architecture/data-flows.md#user-facing-cache-flow) -- restore/save protocol and trust→scope mapping
1984
+ - [Core](https://docs.kici.dev/user/sdk/core/) -- `job()` / `step()` factories the `cache` field attaches to
1985
+ - [Runtime](https://docs.kici.dev/user/sdk/runtime/) -- `StepContext`, where `ctx.cache` lives
1986
+ - [Orchestrator storage layout](https://docs.kici.dev/operator/orchestrator/storage-layout/) -- cache prefix, quota, TTL, and eviction
1987
+ - [Data flows](https://docs.kici.dev/architecture/data-flows/#user-facing-cache-flow) -- restore/save protocol and trust→scope mapping
1878
1988
 
1879
1989
  ---
1880
1990
 
@@ -1908,8 +2018,8 @@ function workflow(name: string, options: WorkflowOptions): Workflow;
1908
2018
  | `options.cleanup` | `HookInput` | no | Always runs after the workflow (success, failure, or cancel) |
1909
2019
  | `options.onSuccess` | `HookInput` | no | Runs on workflow success |
1910
2020
  | `options.onFailure` | `HookInput` | no | Runs on workflow failure |
1911
- | `options.concurrency` | `{ group: (ctx) => string; cancelInProgress?: boolean; max?: number }` | no | Workflow-scoped concurrency. See [Concurrency](../concurrency.md). |
1912
- | `options.timeout` | `number` | no | Whole-run wall-clock timeout in milliseconds across all jobs. On breach the orchestrator cancels the run and marks it timed out. See [Timeouts](#timeouts). |
2021
+ | `options.concurrency` | `{ group: (ctx) => string; cancelInProgress?: boolean; max?: number }` | no | Workflow-scoped concurrency. See [Concurrency](https://docs.kici.dev/user/concurrency/). |
2022
+ | `options.timeout` | `number` | no | Whole-run wall-clock timeout in milliseconds across all jobs. On breach the orchestrator cancels the run and marks it timed out. See [Timeouts](https://docs.kici.dev/user/sdk/core/#timeouts). |
1913
2023
 
1914
2024
  **Returns:** `Workflow` -- an immutable workflow definition.
1915
2025
 
@@ -1922,7 +2032,7 @@ export default workflow('ci', {
1922
2032
  });
1923
2033
  ```
1924
2034
 
1925
- Secret scoping happens at the job level via `environment` (see [job options](#jobname-options--joboptions) and [Secrets](../secrets.md)) — the workflow itself does not declare which secret environments it can read.
2035
+ Secret scoping happens at the job level via `environment` (see [job options](https://docs.kici.dev/user/sdk/core/#jobname-options--joboptions) and [Secrets](https://docs.kici.dev/user/secrets/)) — the workflow itself does not declare which secret environments it can read.
1926
2036
 
1927
2037
  ### job(name, options) / job(options)
1928
2038
 
@@ -1940,8 +2050,8 @@ function job(options: JobOptions): Job;
1940
2050
  | `name` | `string` | no | Job name (auto-generated UUID if omitted) |
1941
2051
  | `options.runsOn` | `RunsOn` | yes | Runner label(s) and optional exclusions (see below) |
1942
2052
  | `options.steps` | `StepInput[]` | yes (or use `run`) | Steps to execute in order. Mutually exclusive with `run`. |
1943
- | `options.run` | `(ctx) => Promise<unknown>` | yes (or use `steps`) | Single-step shorthand -- see [Single-step job shorthand](#single-step-job-shorthand). Mutually exclusive with `steps`. |
1944
- | `options.needs` | `NeedsEntry[]` | no | Job dependencies (must complete first) -- see [Job dependencies (`needs`)](#job-dependencies-needs) |
2053
+ | `options.run` | `(ctx) => Promise<unknown>` | yes (or use `steps`) | Single-step shorthand -- see [Single-step job shorthand](https://docs.kici.dev/user/sdk/core/#single-step-job-shorthand). Mutually exclusive with `steps`. |
2054
+ | `options.needs` | `NeedsEntry[]` | no | Job dependencies (must complete first) -- see [Job dependencies (`needs`)](https://docs.kici.dev/user/sdk/core/#job-dependencies-needs) |
1945
2055
  | `options.rules` | `Rule[]` | no | Conditions for conditional execution |
1946
2056
  | `options.description` | `string` | no | Human-readable description |
1947
2057
  | `options.matrix` | `Matrix` | no | Matrix configuration for job expansion |
@@ -1949,19 +2059,19 @@ function job(options: JobOptions): Job;
1949
2059
  | `options.exclude` | `MatrixExclude[]` | no | Matrix combinations to remove |
1950
2060
  | `options.checkout` | `boolean` | no (default: `true`) | When `false`, agent skips git clone. Useful for deploy/notify jobs. |
1951
2061
  | `options.container` | `string \| ContainerConfig` | no | Docker image for job execution. String form is the image name; object form adds `env`. All steps run inside the container. |
1952
- | `options.environment` | `string \| ((event) => string \| Promise<string>)` | no | Deployment environment for this job. Static string or async/dynamic function -- see [Dynamic values](../dynamic-values.md). |
1953
- | `options.env` | `Record<string, string> \| ((event) => Record<string, string>)` | no | Environment variables. Static object or async/dynamic function -- see [Dynamic values](../dynamic-values.md). |
1954
- | `options.concurrencyGroup` | `string \| ((event) => string \| Promise<string>)` | no | Concurrency group name (defaults to environment name) -- see [Concurrency](../concurrency.md). |
2062
+ | `options.environment` | `string \| ((event) => string \| Promise<string>)` | no | Deployment environment for this job. Static string or async/dynamic function -- see [Dynamic values](https://docs.kici.dev/user/dynamic-values/). |
2063
+ | `options.env` | `Record<string, string> \| ((event) => Record<string, string>)` | no | Environment variables. Static object or async/dynamic function -- see [Dynamic values](https://docs.kici.dev/user/dynamic-values/). |
2064
+ | `options.concurrencyGroup` | `string \| ((event) => string \| Promise<string>)` | no | Concurrency group name (defaults to environment name) -- see [Concurrency](https://docs.kici.dev/user/concurrency/). |
1955
2065
  | `options.onCancel` | `HookInput` | no | Hook that runs when the job is cancelled |
1956
2066
  | `options.cleanup` | `HookInput` | no | Hook that always runs after completion |
1957
2067
  | `options.onSuccess` | `HookInput` | no | Hook that runs when the job succeeds |
1958
2068
  | `options.onFailure` | `HookInput` | no | Hook that runs when the job fails |
1959
2069
  | `options.beforeStep` | `HookInput` | no | Hook that runs before each step |
1960
2070
  | `options.afterStep` | `HookInput` | no | Hook that runs after each step |
1961
- | `options.gracePeriod` | `number` | no | Seconds before SIGKILL after SIGTERM during cancellation -- see [Hooks](../hooks.md#hook-timeout). |
1962
- | `options.timeout` | `number` | no | Total job wall-clock timeout in milliseconds (init + all steps + hooks). On breach the job is aborted and reported timed out. See [Timeouts](#timeouts). |
1963
- | `options.resources` | `ResourceRequest` | no | Per-job CPU / memory request and limit. See [Per-job resources](#per-job-resources) below. |
1964
- | `options.init` | `InitConfig` | no | Per-job initialization run after clone, before steps -- provisions a toolchain. A generic config, a typed preset (`'mise'` / `{ mise }`), `'auto'`, or `false`. See [Per-job init](#per-job-init) below. |
2071
+ | `options.gracePeriod` | `number` | no | Seconds before SIGKILL after SIGTERM during cancellation -- see [Hooks](https://docs.kici.dev/user/hooks/#hook-timeout). |
2072
+ | `options.timeout` | `number` | no | Total job wall-clock timeout in milliseconds (init + all steps + hooks). On breach the job is aborted and reported timed out. See [Timeouts](https://docs.kici.dev/user/sdk/core/#timeouts). |
2073
+ | `options.resources` | `ResourceRequest` | no | Per-job CPU / memory request and limit. See [Per-job resources](https://docs.kici.dev/user/sdk/core/#per-job-resources) below. |
2074
+ | `options.init` | `InitConfig` | no | Per-job initialization run after clone, before steps -- provisions a toolchain. A generic config, a typed preset (`'mise'` / `{ mise }`), `'auto'`, or `false`. See [Per-job init](https://docs.kici.dev/user/sdk/core/#per-job-init) below. |
1965
2075
 
1966
2076
  **Returns:** `Job` -- an immutable job definition.
1967
2077
 
@@ -1982,7 +2092,7 @@ const build = job({
1982
2092
 
1983
2093
  #### runsOn forms
1984
2094
 
1985
- A job's `runsOn` selects which agents may run it. Every label listed must be present on the agent (a subset match). It accepts three forms, and each label can be an exact string, a glob, or a regular expression (see [Targeting by pattern](#targeting-by-pattern) below):
2095
+ A job's `runsOn` selects which agents may run it. Every label listed must be present on the agent (a subset match). It accepts three forms, and each label can be an exact string, a glob, or a regular expression (see [Targeting by pattern](https://docs.kici.dev/user/sdk/core/#targeting-by-pattern) below):
1986
2096
 
1987
2097
  ```typescript
1988
2098
  // 1. Simple string -- agent must have this label
@@ -2008,7 +2118,7 @@ runsOn: { labels: ['kici:os:linux'], exclude: ['kici:host:box-01'] }
2008
2118
  - **Required labels:** The agent must have every label in the `labels` array (or the string/array form).
2009
2119
  - **Excluded labels:** The agent must NOT have any label in the `exclude` array. This includes auto-derived labels like `kici:arch:arm64`, `kici:os:linux`, etc.
2010
2120
  - **Compile-time validation:** The compiler will error if any label appears in both `labels` and `exclude` (overlap detection).
2011
- - **Operator-declared mandatory labels:** Operators may mark a scaler with `mandatoryLabels` (Kubernetes-taint-style opt-in). When a scaler declares a mandatory label, a job is only allowed to land on it if `runsOn.labels` includes that label. A workflow targeting such a scaler must explicitly list the mandatory label in `runsOn`. See the [auto-scaler mandatory labels](../../operator/orchestrator/auto-scaler/common-config.md#mandatory--exclude-labels) for details.
2121
+ - **Operator-declared mandatory labels:** Operators may mark a scaler with `mandatoryLabels` (Kubernetes-taint-style opt-in). When a scaler declares a mandatory label, a job is only allowed to land on it if `runsOn.labels` includes that label. A workflow targeting such a scaler must explicitly list the mandatory label in `runsOn`. See the [auto-scaler mandatory labels](https://docs.kici.dev/operator/orchestrator/auto-scaler/common-config/#mandatory--exclude-labels) for details.
2012
2122
 
2013
2123
  ```typescript
2014
2124
  // Route to any Linux agent that does NOT have the 'gpu' label
@@ -2024,6 +2134,33 @@ const deploy = job('deploy', {
2024
2134
  });
2025
2135
  ```
2026
2136
 
2137
+ #### Single-host selection: `pick`
2138
+
2139
+ When more than one agent matches a `runsOn` selector, the object form's `pick`
2140
+ field controls **which** one runs the job:
2141
+
2142
+ ```typescript
2143
+ // Always the same host across re-runs (default — can be omitted)
2144
+ runsOn: { labels: ['role:db'], pick: 'deterministic' }
2145
+
2146
+ // Any available host (load spread)
2147
+ runsOn: { labels: ['role:db'], pick: 'any' }
2148
+ ```
2149
+
2150
+ - **`'deterministic'` (the default)** — the orchestrator sorts the matching
2151
+ agents by their agent id and picks the lowest. A job that must run exactly
2152
+ once on one stable host — a database migration, a backup dump — lands on the
2153
+ **same** host every run. The string and array shorthand forms
2154
+ (`runsOn: 'role:db'`, `runsOn: ['role:db', 'linux']`) inherit this default.
2155
+ - **`'any'`** — pick any available matching agent. Use this for jobs that don't
2156
+ need a stable host and benefit from spreading load across an equivalent pool.
2157
+
2158
+ **Trade-off:** `'deterministic'` can hot-spot — if many jobs target the same
2159
+ label set, they all pin to the same lowest-id agent. Use `'any'` to spread those
2160
+ across the pool; keep `'deterministic'` when reproducibility matters more than
2161
+ balance. (`pick` selects among **single-agent** candidates; to fan a job out to
2162
+ **every** matching host, use [`runsOnAll`](https://docs.kici.dev/user/sdk/runs-on-all/) instead.)
2163
+
2027
2164
  #### Targeting by pattern
2028
2165
 
2029
2166
  Every selector element — in `runsOn`, in `runsOnAll`, on both the include and the exclude side — can be a plain string, a glob pattern, or a regular expression. KiCI picks the matching mode from the value itself:
@@ -2128,7 +2265,7 @@ const configureNginx = step('configure-nginx', {
2128
2265
  ```
2129
2266
 
2130
2267
  A checked step can run in apply mode (converge) or `--check` preview mode (report
2131
- drift, change nothing). See [Idempotent steps and check mode](../idempotent-steps.md).
2268
+ drift, change nothing). See [Idempotent steps and check mode](https://docs.kici.dev/user/idempotent-steps/).
2132
2269
 
2133
2270
  ### Per-job resources
2134
2271
 
@@ -2211,7 +2348,7 @@ export const build = workflow('build', {
2211
2348
  | --------- | ----------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
2212
2349
  | `run` | `string` | yes | Command run after clone, before steps. Runs in the job's sandbox at the clone root. Must be a non-empty command. |
2213
2350
  | `shell` | `string` | no | Shell used to run `run`. Defaults to `bash`. |
2214
- | `cache` | `CacheSpec` | no | Cache spec for binaries the command installs -- restored before the command, saved after on a key miss. See [Caching](./caching.md). |
2351
+ | `cache` | `CacheSpec` | no | Cache spec for binaries the command installs -- restored before the command, saved after on a key miss. See [Caching](https://docs.kici.dev/user/sdk/caching/). |
2215
2352
  | `timeout` | `number` | no | Max wall-clock for this init command in milliseconds. Defaults to 10 minutes. On breach the init is aborted and the job is reported timed out. |
2216
2353
  | `env` | `Record<string,string>` | no | Static environment variables available to the command. |
2217
2354
 
@@ -2394,6 +2531,43 @@ export default workflow('ci', {
2394
2531
 
2395
2532
  Workflow and job timeouts surface with a distinct "timed out" reason so the dashboard labels the run or job as timed out rather than a generic failure or cancel.
2396
2533
 
2534
+ ### Retries
2535
+
2536
+ A step can declare a `retry` policy so a thrown attempt is re-run automatically instead of failing the job on the first error. Use it for genuinely transient failures — a flaky network call, an occasional 503, a dependency that is briefly not ready.
2537
+
2538
+ ```typescript
2539
+ step('publish', {
2540
+ retry: 3, // shorthand for { maxAttempts: 3 } with the defaults below
2541
+ run: async (ctx) => {
2542
+ await ctx.$`pnpm publish`;
2543
+ },
2544
+ });
2545
+
2546
+ step('fetch-token', {
2547
+ retry: {
2548
+ maxAttempts: 5, // total attempts including the first; must be >= 1
2549
+ delayMs: 500, // base delay between attempts (default 1000)
2550
+ backoff: 'exponential', // 'exponential' (default) or 'fixed'
2551
+ maxDelayMs: 30_000, // cap for exponential growth (default 30000)
2552
+ retryIf: (err) => err instanceof TransientError, // default: retry on any throw
2553
+ },
2554
+ run: async (ctx) => {
2555
+ await fetchToken();
2556
+ },
2557
+ });
2558
+ ```
2559
+
2560
+ - **`retry: N`** is shorthand for `{ maxAttempts: N }` with all defaults applied.
2561
+ - **Defaults:** `delayMs: 1000`, `backoff: 'exponential'`, `maxDelayMs: 30000`, and "retry on any throw" when no `retryIf` is given.
2562
+ - **Backoff.** With `'exponential'`, the wait after the `n`-th attempt (1-based) is `min(delayMs * 2 ** (n - 1), maxDelayMs)` — 1s, 2s, 4s, … capped at `maxDelayMs`. With `'fixed'`, the wait is always `delayMs`.
2563
+ - **`retryIf(err)`** runs against the thrown error before each retry; return `false` to stop retrying immediately and let the failure stand.
2564
+ - **Timeout is per-attempt.** Each attempt gets the step's full `timeout` budget — a timed-out attempt counts as one failed attempt and is retried while attempts remain. The total wall-clock can therefore approach `maxAttempts * (timeout + delay)`, so keep `maxAttempts` and `maxDelayMs` sane (the job-level `timeout` still bounds the whole job).
2565
+ - **Retries exhaust before `continueOnError`.** A step with both retries first; only the _final_ failure is then softened to a warning by `continueOnError`.
2566
+
2567
+ `retry` works identically under `kici run local` and on a remote agent, and applies to dynamically-generated job steps too. The `retryIf` predicate is an in-memory function: it is honored at execution time but never serialized into the lock file.
2568
+
2569
+ > **Retry vs. wait-until-condition.** `retry` re-runs a step that _throws_. To poll until a condition becomes true (a port listening, a `/health` endpoint returning 200, a unit becoming active), use [`waitForStep`](https://docs.kici.dev/user/sdk/wait-for/) instead — it is purpose-built for declarative wait-for-condition with intervals, a timeout, and on-timeout handling.
2570
+
2397
2571
  ### Output chaining
2398
2572
 
2399
2573
  Steps and jobs can access outputs from preceding steps/jobs using two patterns.
@@ -2476,7 +2650,7 @@ const deploy = job('deploy', {
2476
2650
 
2477
2651
  **Important:** `needs` must be declared explicitly. Output chaining does not auto-infer dependencies -- you must list job dependencies in `needs` even if you access their outputs via `.result`.
2478
2652
 
2479
- Cross-job output chaining works in both local test mode (`kici test`) and remote pipeline execution. The orchestrator's needs-aware dispatch scheduler guarantees upstream jobs reach a terminal state before downstream jobs dispatch, and upstream outputs are transported to the downstream agent sandbox via the `upstreamJobOutputs` field on `job.dispatch`. See [needs-scheduler](../../architecture/execution/needs-scheduler.md) for the full dispatch semantics.
2653
+ Cross-job output chaining works in both local test mode (`kici test`) and remote pipeline execution. The orchestrator's needs-aware dispatch scheduler guarantees upstream jobs reach a terminal state before downstream jobs dispatch, and upstream outputs are transported to the downstream agent sandbox via the `upstreamJobOutputs` field on `job.dispatch`. See [needs-scheduler](https://docs.kici.dev/architecture/execution/needs-scheduler/) for the full dispatch semantics.
2480
2654
 
2481
2655
  ### Job dependencies (`needs`)
2482
2656
 
@@ -2489,9 +2663,9 @@ const test = job('test', { needs: [lint], ... });
2489
2663
  // 2. Reference by string name
2490
2664
  const test = job('test', { needs: ['lint'], ... });
2491
2665
 
2492
- // 3. Object form with per-edge failure policy
2666
+ // 3. Object form with a per-edge run condition (`when`)
2493
2667
  const cleanup = job('cleanup', {
2494
- needs: [{ name: 'build', ifFailed: 'run' }],
2668
+ needs: [{ name: 'build', when: 'always' }],
2495
2669
  ...
2496
2670
  });
2497
2671
 
@@ -2502,16 +2676,36 @@ const deploy = job('deploy', {
2502
2676
  });
2503
2677
  ```
2504
2678
 
2505
- **Failure policy (`ifFailed`):** controls what happens to a downstream job when an upstream reaches a non-success terminal state (`failed`, `cancelled`, `drift_dropped`).
2679
+ **Run condition (`when`):** controls when a downstream edge is satisfied, based on the upstream's terminal status. `when` is keyword sugar (or a raw status-set) that resolves at compile time to the set of upstream terminal statuses that satisfy the edge. The downstream edge is satisfied when the upstream's terminal status is a member of that set.
2506
2680
 
2507
- | Value | Behavior |
2508
- | ------ | ------------------------------------------------------------------------------------------- |
2509
- | `skip` | (Default) Downstream transitions directly to `skipped`. Failures cascade through the DAG. |
2510
- | `run` | Downstream dispatches anyway. Use for cleanup, notification, or "always-run" teardown jobs. |
2681
+ | Keyword | Satisfied when the upstream is… | Use for |
2682
+ | ------------------------ | ------------------------------- | ------------------------------------------- |
2683
+ | `'on-success'` (default) | `success` | normal dependencies |
2684
+ | `'always'` | any terminal status | cleanup / notification / teardown jobs |
2685
+ | `'on-skip'` | `success` or `skipped` | continue when an upstream was narrowed out |
2686
+ | `'on-failure'` | `failed` or `timed_out_stale` | error-handler jobs that run only on failure |
2511
2687
 
2512
- String and `Job`-reference entries default to `ifFailed: 'skip'`. To override, use the object form (`{ name, ifFailed }` for static upstreams, `{ group, ifFailed }` for dynamic groups -- `dynamicGroup(name, { ifFailed: 'run' })` produces the latter).
2688
+ For full control, pass a raw status-set instead of a keyword: `when: ['skipped', 'failed', 'timed_out_stale']`. The valid members are the terminal job statuses: `success`, `failed`, `cancelled`, `skipped`, `timed_out_stale`, `drift_dropped`.
2513
2689
 
2514
- **Dispatch gate:** `needs` is a hard dispatch gate. A job only dispatches after every upstream in its `needs` array reaches a terminal state (success, or failure with `ifFailed: 'run'`). Root jobs (empty `needs`, no dynamic group refs) dispatch immediately. The scheduler is DB-backed and fully recovers across orchestrator restarts.
2690
+ String and `Job`-reference entries default to `when: 'on-success'`. To override, use the object form (`{ name, when }` for static upstreams, `{ group, when }` for dynamic groups -- `dynamicGroup(name, { when: 'always' })` produces the latter).
2691
+
2692
+ When an upstream's terminal status is **not** in the edge's set, the downstream transitions directly to `skipped`. Because a skipped job is itself terminal, this propagates transitively: each downstream's `when` set governs whether the skip cascades further.
2693
+
2694
+ **Dispatch gate:** `needs` is a hard dispatch gate. A job dispatches only after every upstream in its `needs` array reaches a terminal status that satisfies that edge's `when` set. Root jobs (empty `needs`, no dynamic group refs) dispatch immediately. The scheduler is DB-backed and fully recovers across orchestrator restarts.
2695
+
2696
+ **Reading an upstream's status in a step:** inside a running job, `ctx.needs.<job>.status` exposes each upstream's terminal status (`success`, `failed`, `skipped`, …) and `ctx.needs.<job>.result` its outputs. A group / matrix / `runsOnAll` fan-out upstream is an ordered array of `{ name, result, status }`, one per child. Use this to branch in TypeScript:
2697
+
2698
+ ```typescript
2699
+ job('report', {
2700
+ needs: [{ name: 'probe', when: 'always' }],
2701
+ run: async (ctx) => {
2702
+ if (ctx.needs.probe.status === 'failed') await fileIncident(ctx.needs.probe.result);
2703
+ else await publish(ctx.needs.probe.result);
2704
+ },
2705
+ });
2706
+ ```
2707
+
2708
+ For an arbitrary outcome-based gate that prevents a job from dispatching at all, use a result-aware `dynamicJob` that returns `[]` or `[job]` based on `ctx.needs.<job>.status` — see [Dynamic jobs](https://docs.kici.dev/architecture/execution/dynamic-jobs/).
2515
2709
 
2516
2710
  **DAG validation:** three-layer cycle detection.
2517
2711
 
@@ -2524,7 +2718,10 @@ String and `Job`-reference entries default to `ifFailed: 'skip'`. To override, u
2524
2718
  Create a reference to a dynamic job group, for use inside a static job's `needs` array.
2525
2719
 
2526
2720
  ```typescript
2527
- function dynamicGroup(name: string, options?: { ifFailed?: 'skip' | 'run' }): DynamicGroupRef;
2721
+ function dynamicGroup(
2722
+ name: string,
2723
+ options?: { when?: 'on-success' | 'always' | 'on-skip' | 'on-failure' | string[] },
2724
+ ): DynamicGroupRef;
2528
2725
  ```
2529
2726
 
2530
2727
  Use when a static downstream must wait for every generated job tagged with a given group name to complete. If the dynamic group produces zero jobs, the downstream dispatches immediately (empty group satisfies all upstreams).
@@ -2553,7 +2750,7 @@ Tag a dynamic job generator function with a group name so other jobs can referen
2553
2750
  function dynamicJob(groupName: string, fn: DynamicJobFn): DynamicJobFn;
2554
2751
  ```
2555
2752
 
2556
- The generator runs twice: once in the init phase (to register expected job names) and once inside the executing agent (to produce the actual jobs). Mismatches between the two evaluations are detected as determinism drift -- see [dynamic-jobs](../../architecture/execution/dynamic-jobs.md).
2753
+ The generator runs twice: once in the init phase (to register expected job names) and once inside the executing agent (to produce the actual jobs). Mismatches between the two evaluations are detected as determinism drift -- see [dynamic-jobs](https://docs.kici.dev/architecture/execution/dynamic-jobs/).
2557
2754
 
2558
2755
  ### Auto-generated IDs
2559
2756
 
@@ -2918,13 +3115,13 @@ The typed `payload` shapes above reference these partial GitHub object types. Ea
2918
3115
 
2919
3116
  Source: https://docs.kici.dev/user/sdk/idempotent/
2920
3117
 
2921
- The SDK exposes two idempotency helpers — a generic function `idempotent()` and a step factory `idempotentStep()` — for the common case where a workflow step should:
3118
+ The SDK exposes three idempotency helpers — a generic function `idempotent()`, the step factory `idempotentStep()`, and its check-mode-aware sibling `checkStep()` — for the common case where a workflow step should:
2922
3119
 
2923
3120
  1. **Check** whether the desired state is already in place.
2924
3121
  2. **Apply** the change only when drift is detected.
2925
3122
  3. **Surface** the resource (or its identifier) on both branches, so downstream steps don't need to know whether work happened or was skipped.
2926
3123
 
2927
- Both helpers wrap the same underlying runner, so they share semantics and return shape. Pick `idempotentStep()` when the operation is the whole job of a step; use `idempotent()` from anywhere — inside a multi-action step, a hook, or a bare async function.
3124
+ `idempotent()` and `idempotentStep()` wrap the same underlying runner and always apply on drift. Pick `idempotentStep()` when the operation is the whole job of a step; use `idempotent()` from anywhere — inside a multi-action step, a hook, or a bare async function. Pick `checkStep()` when the step should respect the run-level check mode — `kici run --check` previews the drift without applying it.
2928
3125
 
2929
3126
  ## `idempotent(options)`
2930
3127
 
@@ -3017,6 +3214,61 @@ export const setup = job('setup', {
3017
3214
  });
3018
3215
  ```
3019
3216
 
3217
+ ## `checkStep(name, options)`
3218
+
3219
+ The check-mode-aware sibling of `idempotentStep()`. It takes the **same option shape**, but behaves differently when a run is started in check mode (`kici run --check`):
3220
+
3221
+ | Factory | Behavior under `kici run --check` |
3222
+ | ---------------- | ----------------------------------------- |
3223
+ | `idempotentStep` | always applies on drift |
3224
+ | `checkStep` | reports drift, applies only in apply mode |
3225
+
3226
+ Use `checkStep()` for deploy-style steps where you want a dry-run preview of pending changes before committing them, and `idempotentStep()` for steps that must always converge (for example inside a hook). A `checkStep()` desugars to the first-class step check facet (`check` / `summarize` / `run(ctx, drift)` / `whenInSync`), so it participates in run-level check mode automatically: `kici run --check` reports the drift and skips `apply`, `kici run --check --fail-on-drift` exits non-zero when drift is detected, and apply mode applies the change.
3227
+
3228
+ ### Parameters
3229
+
3230
+ | Name | Type | Required | Description |
3231
+ | ----------------- | ------------------------------------------- | -------- | --------------------------------------------------------------------------------------------------- |
3232
+ | `name` | `string` | Yes | Step name. Appears in the run timeline and in log lines. |
3233
+ | `check` | `(ctx) => Promise<TDrift \| null>` | Yes | Read-only inspection. Return `null` when the system is already in the desired state. |
3234
+ | `apply` | `(ctx, drift: TDrift) => Promise<TApplied>` | Yes | Brings the system to the desired state. Runs only in apply mode (skipped under `kici run --check`). |
3235
+ | `summarize` | `(drift: TDrift) => string` | Yes | Human-readable summary of what `apply()` would do; shown in check-mode drift output. |
3236
+ | `whenInSync` | `(ctx) => Promise<TInSync>` | No | Runs when `check()` returned `null` (already in sync). |
3237
+ | `continueOnError` | `boolean` | No | When true, the job proceeds even if this step fails. |
3238
+ | `timeout` | `number` | No | Step-level timeout in milliseconds. |
3239
+
3240
+ The one signature difference from `idempotentStep`: `apply` and `whenInSync` receive `ctx` as their first argument, so the apply logic has access to `ctx.$`, `ctx.log`, and `ctx.secrets`.
3241
+
3242
+ ### Result
3243
+
3244
+ `checkStep(...)` returns `Step<TApplied | TInSync>` — the output is whichever of `apply` / `whenInSync` ran.
3245
+
3246
+ ### Example
3247
+
3248
+ ```typescript
3249
+ import { checkStep, job } from '@kici-dev/sdk';
3250
+
3251
+ const ensureDnsRecord = checkStep('ensure-dns-record', {
3252
+ check: async (ctx) => {
3253
+ const existing = await ctx.$`dig +short api.example.com`;
3254
+ return existing.stdout.trim() ? null : { fqdn: 'api.example.com', target: '203.0.113.10' };
3255
+ },
3256
+ summarize: (drift) => `Create A record ${drift.fqdn} → ${drift.target}`,
3257
+ apply: async (ctx, drift) => {
3258
+ await ctx.$`dns-cli create ${drift.fqdn} ${drift.target}`;
3259
+ return { created: true };
3260
+ },
3261
+ whenInSync: async () => ({ created: false }),
3262
+ });
3263
+
3264
+ export const deploy = job('deploy', {
3265
+ runsOn: 'linux',
3266
+ steps: [ensureDnsRecord],
3267
+ });
3268
+ ```
3269
+
3270
+ Run `kici run --check` against this workflow to see the drift summary without touching DNS; run it without `--check` to apply.
3271
+
3020
3272
  ## Worked example: create-if-missing returning a resource id
3021
3273
 
3022
3274
  The typical use case is **resource provisioning that should be safe to re-run**. The helper guarantees the same downstream typed shape whether the resource already existed or was just created:
@@ -3059,8 +3311,8 @@ The caller never has to branch on outcome — `result.result` is always a `Bucke
3059
3311
 
3060
3312
  ## See also
3061
3313
 
3062
- - [Core SDK reference](./core.md) — the `step()`, `job()`, and `workflow()` factories that `idempotentStep()` builds on.
3063
- - [Runtime types](./runtime.md) — `StepContext`, `Logger`, and other surface used inside the helpers.
3314
+ - [Core SDK reference](https://docs.kici.dev/user/sdk/core/) — the `step()`, `job()`, and `workflow()` factories that `idempotentStep()` builds on.
3315
+ - [Runtime types](https://docs.kici.dev/user/sdk/runtime/) — `StepContext`, `Logger`, and other surface used inside the helpers.
3064
3316
 
3065
3317
  ---
3066
3318
 
@@ -3187,7 +3439,7 @@ You can also narrow directly with `if (ctx.event.type === 'pull_request')` — T
3187
3439
 
3188
3440
  `EventPayload` is a discriminated union over the `type` field. Each variant provides typed access to the normalized event fields and the raw webhook payload.
3189
3441
 
3190
- Every variant carries the shared `EventBase` fields — `type`, `action`, `targetBranch`, `sourceBranch`, `provider`, `isForkPR`, `baseBranch`, `senderUsername`, `sourceRepo`, `changedFiles`, and the raw `payload` — plus a per-type `payload` shape for the typed variants. The complete field-by-field schema, including every typed `payload` shape and the shared GitHub object types, is in the [event payload reference](./event-payloads.md).
3442
+ Every variant carries the shared `EventBase` fields — `type`, `action`, `targetBranch`, `sourceBranch`, `provider`, `isForkPR`, `baseBranch`, `senderUsername`, `sourceRepo`, `changedFiles`, and the raw `payload` — plus a per-type `payload` shape for the typed variants. The complete field-by-field schema, including every typed `payload` shape and the shared GitHub object types, is in the [event payload reference](https://docs.kici.dev/user/sdk/event-payloads/).
3191
3443
 
3192
3444
  **Typed variants** (with GitHub-specific payload fields): `pull_request`, `push`, `tag`, `comment`, `review`, `review_comment`, `release`, `dispatch`, `create`, `delete`, `status`, `workflow_run`, `fork`, `star`, `watch`.
3193
3445
 
@@ -3297,7 +3549,7 @@ interface MatrixValues {
3297
3549
 
3298
3550
  A matrix fan-out runs every combination at once by default. The fan-out-generic
3299
3551
  `maxParallel` and `failFast` job options bound it the same way they bound a
3300
- [`runsOnAll`](./runs-on-all.md#rolling-rollout-maxparallel--failfast) host fan-out:
3552
+ [`runsOnAll`](https://docs.kici.dev/user/sdk/runs-on-all/#rolling-rollout-maxparallel--failfast) host fan-out:
3301
3553
 
3302
3554
  ```typescript
3303
3555
  const test = job('test', {
@@ -3445,12 +3697,12 @@ export default workflow('discovery-fan-out', { jobs: [discover, reports] });
3445
3697
 
3446
3698
  `ctx.needs` shape:
3447
3699
 
3448
- | Need form | `ctx.needs[...]` value |
3449
- | ------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
3450
- | `'jobName'` / `{ name, ifFailed }` | `{ result }` — `result` is an `OutputProxy` (`ctx.needs.<job>.result.<step>.<field>`; single-step `run` jobs flatten to `ctx.needs.<job>.result.<field>`) |
3451
- | `dynamicGroup('g')` / `dynamicGroup('g', { ifFailed })` | ordered array of `{ name, result }`, one per group member |
3700
+ | Need form | `ctx.needs[...]` value |
3701
+ | --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
3702
+ | `'jobName'` / `{ name, when }` | `{ result, status }` — `result` is an `OutputProxy` (`ctx.needs.<job>.result.<step>.<field>`; single-step `run` jobs flatten to `ctx.needs.<job>.result.<field>`); `status` is the upstream's terminal status |
3703
+ | `dynamicGroup('g')` / `dynamicGroup('g', { when })` | ordered array of `{ name, result, status }`, one per group member |
3452
3704
 
3453
- `ctx.needs` is deterministic — a snapshot of upstream outputs frozen at first eval and replayed unchanged on re-eval, like `ctx.event`. Use result-aware generation for same-run fan-out from a prior job's result; use [`jobComplete()`](./triggers.md) for cross-workflow reactions to a job finishing. See the architecture deep-dive in [dynamic jobs](../../architecture/execution/dynamic-jobs.md#result-aware-generation).
3705
+ `ctx.needs` is deterministic — a snapshot of upstream outputs frozen at first eval and replayed unchanged on re-eval, like `ctx.event`. Use result-aware generation for same-run fan-out from a prior job's result; use [`jobComplete()`](https://docs.kici.dev/user/sdk/triggers/) for cross-workflow reactions to a job finishing. See the architecture deep-dive in [dynamic jobs](https://docs.kici.dev/architecture/execution/dynamic-jobs/#result-aware-generation).
3454
3706
 
3455
3707
  ### JobOrFactory
3456
3708
 
@@ -3491,7 +3743,7 @@ roster that matches a label predicate — one pinned execution per host. Use it
3491
3743
  fleet-wide operations: patch every web tier, smoke-test every node, collect uptime
3492
3744
  from the fleet.
3493
3745
 
3494
- `runsOnAll` is mutually exclusive with [`runsOn`](/user/sdk/core/): a job declares one
3746
+ `runsOnAll` is mutually exclusive with [`runsOn`](https://docs.kici.dev/user/sdk/core/): a job declares one
3495
3747
  or the other. Where `runsOn` picks a **single** agent that satisfies the labels,
3496
3748
  `runsOnAll` targets **all** matching hosts and runs the job once on each, pinned to
3497
3749
  that specific host.
@@ -3540,7 +3792,7 @@ const patch = job('patch', {
3540
3792
  #### Targeting by pattern
3541
3793
 
3542
3794
  Every entry in any of these forms — include or exclude — can be an exact string, a
3543
- glob, or a regular expression, exactly like [`runsOn`](./core.md#targeting-by-pattern):
3795
+ glob, or a regular expression, exactly like [`runsOn`](https://docs.kici.dev/user/sdk/core/#targeting-by-pattern):
3544
3796
 
3545
3797
  - **Plain string → exact match** (`'role:web'`).
3546
3798
  - **String with glob metacharacters (`*`, `?`, `[]`, `{}`) → glob** (`'kici:host:web-*'`).
@@ -3592,6 +3844,66 @@ run: async (ctx) => {
3592
3844
 
3593
3845
  Both are `undefined` for jobs that do not use `runsOnAll`.
3594
3846
 
3847
+ ### ctx.fanout — fan-out position
3848
+
3849
+ Every fan-out child — a `runsOnAll` host **or** a matrix combination — also
3850
+ carries its **position** within the fan-out:
3851
+
3852
+ ```typescript
3853
+ ctx.fanout?: {
3854
+ index: number; // 0-based position in the deterministically-ordered fan-out
3855
+ total: number; // number of children in this fan-out
3856
+ first: boolean; // index === 0
3857
+ last: boolean; // index === total - 1
3858
+ };
3859
+ ```
3860
+
3861
+ The order is a **documented guarantee**: host fan-out is ordered by agent id,
3862
+ matrix fan-out by its combination label. So `ctx.fanout.first` is always the
3863
+ same (lowest-agent-id) host across re-runs, and `ctx.fanout.last` the same final
3864
+ one. `ctx.fanout` is `undefined` on a job that is not fanned out.
3865
+
3866
+ ### Run-once steps: onlyOnFirstHost / onlyOnLastHost / onlyOnFanoutIndex
3867
+
3868
+ For ordered, stateful rollouts you often need a step that runs on exactly **one**
3869
+ host — enable a leader before the rest join, run a one-time migration, take a
3870
+ single dump. Three rule helpers express this by reading `ctx.fanout`:
3871
+
3872
+ ```typescript
3873
+ import { job, step, onlyOnFirstHost, onlyOnLastHost, onlyOnFanoutIndex } from '@kici-dev/sdk';
3874
+
3875
+ const rollout = job('rollout', {
3876
+ runsOnAll: 'role:db',
3877
+ maxParallel: 1, // serial, so "first" runs before the rest
3878
+ steps: [
3879
+ // Runs only on the first (lowest-agent-id) host — KiCI's run-once primitive.
3880
+ step('enable-sync-mode', { rules: [onlyOnFirstHost()] }, async (ctx) => {
3881
+ /* configure the leader before standbys join */
3882
+ }),
3883
+ // Runs on every host.
3884
+ step('apply', async (ctx) => {
3885
+ /* ... */
3886
+ }),
3887
+ // Runs only on the last host.
3888
+ step('finalize', { rules: [onlyOnLastHost()] }, async (ctx) => {
3889
+ /* ... */
3890
+ }),
3891
+ ],
3892
+ });
3893
+ ```
3894
+
3895
+ - A step gated this way is **skipped** (not failed) on non-matching hosts — its
3896
+ outputs exist only on the host where it ran.
3897
+ - `onlyOnFanoutIndex(n)` targets the host at a specific position.
3898
+ - **Non-fan-out safety:** on a job that is not fanned out, `ctx.fanout` is
3899
+ `undefined` and these helpers treat the job as a single implicit child at
3900
+ index 0 — so `onlyOnFirstHost()` runs normally there (there is one host, which
3901
+ is the first). This means you can author a step with `onlyOnFirstHost()` and it
3902
+ behaves correctly whether or not the job ends up fanning out.
3903
+ - The helpers are host-flavored by name (the dominant use case) but read
3904
+ `ctx.fanout`, so they work for matrix fan-out too — `onlyOnFirstHost()` runs on
3905
+ the first combination.
3906
+
3595
3907
  ### byHost outputs
3596
3908
 
3597
3909
  A downstream that `needs:` a `runsOnAll` job receives a **byHost** envelope instead
@@ -3624,7 +3936,7 @@ to a single scalar: `summary.outputs[key]` is an array of every host's value, an
3624
3936
  ### onUnreachable: skip | fail | hold
3625
3937
 
3626
3938
  Resolution is backed by the **declared host roster** (see the operator
3627
- [host roster](/operator/orchestrator/host-roster/) doc), not just the live registry.
3939
+ [host roster](https://docs.kici.dev/operator/orchestrator/host-roster/) doc), not just the live registry.
3628
3940
  This lets KiCI surface an expected-but-absent host instead of silently fanning out to a
3629
3941
  partial fleet. The `onUnreachable` policy controls what happens when a **durable**
3630
3942
  (static) host in the roster is matched but not currently connected:
@@ -3650,6 +3962,37 @@ independent of `onUnreachable` — a scaled-down node may never return. A `runsO
3650
3962
  that matches zero usable hosts fails the run rather than reporting a silent zero-child
3651
3963
  success.
3652
3964
 
3965
+ ### includeUninitialized: converge a fresh fleet
3966
+
3967
+ `onUnreachable` governs declared hosts that _had_ an agent and are momentarily absent.
3968
+ A **never-initialized** host — a freshly-provisioned box reachable over SSH but with no
3969
+ agent yet — is a different case: there is nothing to run on. Set
3970
+ `includeUninitialized: true` to widen the fan-out to those hosts and bring them up:
3971
+
3972
+ ```typescript
3973
+ const converge = job('converge', {
3974
+ runsOnAll: 'kici:group:prod',
3975
+ includeUninitialized: true,
3976
+ steps: [partitionDisk, formatLuks, debootstrap, installAgent],
3977
+ });
3978
+ ```
3979
+
3980
+ For each un-agented declared host (one carrying SSH reach metadata), KiCI brings up a
3981
+ temporary init-runner over SSH and runs the **same steps** on it; hosts that already
3982
+ have a live agent run the steps on their own agent. One workflow converges the whole
3983
+ fleet — fresh boxes get built, live boxes run the same phases.
3984
+
3985
+ Because the steps run on already-initialized hosts too, the bootstrap phases **must be
3986
+ idempotent [check-steps](https://docs.kici.dev/user/sdk/core/)**: each step's `check()` reports in-sync on a
3987
+ live box so the partition / format / install steps **skip** there and run only on fresh
3988
+ boxes. This is the safety guard — an OS or disk-format step must never re-run on a host
3989
+ that is already built. Re-running the workflow is a no-op everywhere. See the operator
3990
+ [fresh-box bootstrap](https://docs.kici.dev/operator/orchestrator/host-roster/) doc for the bring-up,
3991
+ capability gating, and lifecycle details.
3992
+
3993
+ `includeUninitialized` is only meaningful alongside `runsOnAll`; it is ignored on a
3994
+ single-agent `runsOn` job.
3995
+
3653
3996
  ### Rolling rollout: maxParallel + failFast
3654
3997
 
3655
3998
  By default a `runsOnAll` fan-out dispatches to every matched host at once — fine for
@@ -3685,6 +4028,32 @@ A held host occupies a wave slot indefinitely while it waits to reconnect, stall
3685
4028
  roll behind an absent box. `skip` (run only reachable hosts) or `fail` (refuse the roll
3686
4029
  if any expected host is down) keep the window moving.
3687
4030
 
4031
+ ### Narrowing the roster at run time with `--target`
4032
+
4033
+ A `runsOnAll` predicate is authored once in the workflow, but you can narrow it for a
4034
+ single run with `kici run --target <selector>` — an Ansible-`--limit`-style runtime
4035
+ filter. The effective host set becomes `runsOnAll ∩ target`: the selector can only
4036
+ _remove_ hosts from the matched roster, never add them. The narrowing is **run-global**
4037
+ (it applies to every `runsOnAll` job) and **`runsOnAll`-only** (single `runsOn`-pinned
4038
+ jobs are untouched). Repeated `--target` values AND-combine — a host must satisfy every
4039
+ selector to survive.
4040
+
4041
+ ```bash
4042
+ # Patch only the role:web subset of whatever role:* hosts the job would match
4043
+ kici run remote deploy --target role:web
4044
+
4045
+ # Intersect two selectors: hosts must be BOTH role:web AND dc:eu
4046
+ kici run remote deploy --target role:web --target dc:eu
4047
+ ```
4048
+
4049
+ When `--target` narrows a `runsOnAll` job to zero hosts, the run **fails** by default
4050
+ (a mistyped selector should be loud, not silently no-op). Pass `--target-allow-empty`
4051
+ to **skip** the zeroed job instead — it records a `skipped` status, and downstream jobs
4052
+ gated with `when: 'on-skip'` (or `when: 'always'`) still run, exactly as for an
4053
+ `onUnreachable: 'skip'` zero-host fan-out. See the [CLI reference](https://docs.kici.dev/user/cli-reference/#host-narrowing-with---target)
4054
+ for the full flag behavior and the [`needs` gating model](https://docs.kici.dev/user/sdk/core/#job-dependencies-needs)
4055
+ for how a skipped upstream propagates.
4056
+
3688
4057
  ### Limits (v0)
3689
4058
 
3690
4059
  - Per-host secret scoping is not yet available — all hosts receive the job's resolved
@@ -3716,9 +4085,9 @@ All types are exported from `@kici-dev/sdk` as type-only imports.
3716
4085
  | `OutputSchema` | Record of Zod types for step outputs |
3717
4086
  | `InferOutputs<T>` | Infer output type from output schema |
3718
4087
  | `ContainerConfig` | Container config for job execution (`image`, `env?`) |
3719
- | `RunsOn` | Union of `runsOn` forms: `string \| RegExp \| (string \| RegExp)[] \| RunsOnSelector`. A plain string matches exactly, a string with glob metacharacters (`*?[]{}`) is a glob, and a `RegExp` is a regular expression. See [Targeting by pattern](./core.md#targeting-by-pattern). |
4088
+ | `RunsOn` | Union of `runsOn` forms: `string \| RegExp \| (string \| RegExp)[] \| RunsOnSelector`. A plain string matches exactly, a string with glob metacharacters (`*?[]{}`) is a glob, and a `RegExp` is a regular expression. See [Targeting by pattern](https://docs.kici.dev/user/sdk/core/#targeting-by-pattern). |
3720
4089
  | `RunsOnSelector` | Object form for `runsOn` with `labels` (required) and `exclude` (optional) properties. Each element accepts the exact / glob / regex forms on both sides. |
3721
- | `RunsOnAllInput` | Union of `runsOnAll` host fan-out forms: `string \| RegExp \| (string \| RegExp)[] \| { include: { all: (string \| RegExp)[] }[]; exclude?: (string \| RegExp)[] }`. Same exact / glob / regex semantics per element. See [runsOnAll](./runs-on-all.md#targeting-by-pattern). |
4090
+ | `RunsOnAllInput` | Union of `runsOnAll` host fan-out forms: `string \| RegExp \| (string \| RegExp)[] \| { include: { all: (string \| RegExp)[] }[]; exclude?: (string \| RegExp)[] }`. Same exact / glob / regex semantics per element. See [runsOnAll](https://docs.kici.dev/user/sdk/runs-on-all/#targeting-by-pattern). |
3722
4091
  | `Fixture` | Test fixture definition returned by `fixture()` |
3723
4092
  | `FixtureOptions` | Options for `fixture()` factory |
3724
4093
  | `Registry` | Private npm registry declaration used in `WorkflowOptions.registries` |
@@ -3905,7 +4274,7 @@ step('example', async ({ $, log, env, matrix, workflow, job }) => {
3905
4274
  - **Inside a dynamic `environment` / `env` / `concurrencyGroup` function** on a static job — captured to the `__init__` job's synthetic step-0 log, which appears in the timeline as "Init: _jobname_".
3906
4275
  - **Inside a `DynamicJobFn` body and the per-generated-job `environment` / `env` / `concurrencyGroup` / `matrix` functions** — captured to the `__dynamic__` job's synthetic step-0 log ("Evaluate: _jobname_" in the timeline). The `$` parameter in that context is a scoped zx shell, so `await $\`...\`` subprocess output is captured too.
3907
4276
 
3908
- Use whichever style is convenient — you don't have to wrap `console.log` in the provided `log` parameter to make it visible. One limitation applies to in-process contexts only (init, build, dynamic-eval): direct `process.stdout.write` / `printf` is not captured there, because the agent's own logger uses that path and we don't want agent-internal output leaking into your step logs. Use `console.*` or the `log` parameter instead. See [Log streaming](../../architecture/execution/job-execution.md#log-streaming) for the full capture surface and limits (default 10 MB per step, backpressure behavior).
4277
+ Use whichever style is convenient — you don't have to wrap `console.log` in the provided `log` parameter to make it visible. One limitation applies to in-process contexts only (init, build, dynamic-eval): direct `process.stdout.write` / `printf` is not captured there, because the agent's own logger uses that path and we don't want agent-internal output leaking into your step logs. Use `console.*` or the `log` parameter instead. See [Log streaming](https://docs.kici.dev/architecture/execution/job-execution/#log-streaming) for the full capture surface and limits (default 10 MB per step, backpressure behavior).
3909
4278
 
3910
4279
  ### setEnv(key, value)
3911
4280
 
@@ -3930,7 +4299,7 @@ step('use', async (ctx) => {
3930
4299
  - Last-write-wins -- if multiple steps set the same key, the last value is used
3931
4300
  - Cannot override operator-injected secrets (the operator value takes precedence)
3932
4301
  - Changes take effect immediately in the current step and persist for all subsequent steps
3933
- - Shell commands export the same way by appending to `$KICI_ENV` (see [Exporting env from shell commands](#exporting-env-from-shell-commands-kici_env--kici_path) below)
4302
+ - Shell commands export the same way by appending to `$KICI_ENV` (see [Exporting env from shell commands](https://docs.kici.dev/user/sdk/runtime/#exporting-env-from-shell-commands-kici_env--kici_path) below)
3934
4303
 
3935
4304
  ### addPath(dir)
3936
4305
 
@@ -4143,7 +4512,7 @@ const publish = job('publish', {
4143
4512
  - The returned `{ storageKey, subjectDigest, bundleMediaType }` identifies the stored bundle.
4144
4513
  - Only available inside a running job step; calling it outside one (for example, during local execution) rejects with a clear error.
4145
4514
 
4146
- See the [build provenance guide](../provenance.md) for the end-to-end attest →
4515
+ See the [build provenance guide](https://docs.kici.dev/user/provenance/) for the end-to-end attest →
4147
4516
  verify → view journey, including how to verify a bundle with `kici verify-attestation`.
4148
4517
 
4149
4518
  ## Secrets
@@ -4238,7 +4607,7 @@ step('discover', async (ctx) => {
4238
4607
 
4239
4608
  ### File-mounted secrets (ctx.secrets.mountFile / exposeFile)
4240
4609
 
4241
- Tools that require a file path on disk (sops `SOPS_AGE_KEY_FILE`, kubectl `KUBECONFIG`, gcloud `GOOGLE_APPLICATION_CREDENTIALS`) get a typed step-side API: `ctx.secrets.mountFile(opts)` writes the concatenation of one or more existing secrets to a per-step tmpfile and returns the path; `ctx.secrets.exposeFile(envVar, opts)` additionally sets `process.env[envVar] = path`. Files are removed and env vars are unset automatically when the step completes (success, failure, or timeout) — no manual cleanup. See [Mounting secrets as files](../secrets.md#mounting-secrets-as-files) for the full options table, lifecycle details, and the canonical sops example.
4610
+ Tools that require a file path on disk (sops `SOPS_AGE_KEY_FILE`, kubectl `KUBECONFIG`, gcloud `GOOGLE_APPLICATION_CREDENTIALS`) get a typed step-side API: `ctx.secrets.mountFile(opts)` writes the concatenation of one or more existing secrets to a per-step tmpfile and returns the path; `ctx.secrets.exposeFile(envVar, opts)` additionally sets `process.env[envVar] = path`. Files are removed and env vars are unset automatically when the step completes (success, failure, or timeout) — no manual cleanup. See [Mounting secrets as files](https://docs.kici.dev/user/secrets/#mounting-secrets-as-files) for the full options table, lifecycle details, and the canonical sops example.
4242
4611
 
4243
4612
  ### Local test mode secrets
4244
4613
 
@@ -4491,13 +4860,73 @@ Create a repository_dispatch trigger. Returns a frozen `DispatchTriggerConfig`.
4491
4860
  function dispatch(config?: DispatchConfigInput): DispatchTriggerConfig;
4492
4861
  ```
4493
4862
 
4494
- **Config options:** `types` (string[]), `description`
4863
+ **Config options:** `types` (string[]), `description`, `inputs` (typed dispatch inputs map)
4495
4864
 
4496
4865
  ```typescript
4497
4866
  dispatch(); // Any dispatch
4498
4867
  dispatch({ types: ['deploy', 'rollback'] }); // Specific event types
4499
4868
  ```
4500
4869
 
4870
+ #### Typed dispatch inputs
4871
+
4872
+ A `dispatch()` trigger can declare a typed `inputs` schema. Operators supply
4873
+ values with `kici run --input key=value`; KiCI validates, coerces, defaults, and
4874
+ exposes them to steps and rules as `ctx.dispatchInputs`. The values are validated
4875
+ on the orchestrator from the compiled lock file — a missing required input or a
4876
+ bad value is rejected before any agent runs, without cloning the repository.
4877
+
4878
+ ```typescript
4879
+ import { workflow, job, step, dispatch, defineDispatchInputs, z } from '@kici-dev/sdk';
4880
+
4881
+ const inputs = defineDispatchInputs({
4882
+ target: z.string().optional(),
4883
+ skipCveScan: z.boolean().default(false),
4884
+ skipCveScanReason: z.string().min(1).optional(),
4885
+ mode: z.enum(['full', 'edge-only']).default('full'),
4886
+ retries: z.number().int().min(0).max(10).default(3),
4887
+ });
4888
+
4889
+ export default workflow('deploy-prod', {
4890
+ on: dispatch({ types: ['deploy-prod'], inputs }),
4891
+ jobs: [
4892
+ job('gates', {
4893
+ runsOn: 'kici:group:ops',
4894
+ steps: [
4895
+ step('cve-gate', async (ctx) => {
4896
+ const i = inputs.from(ctx); // fully typed per declared key
4897
+ if (i.skipCveScan) {
4898
+ ctx.log.warn(`CVE gate skipped: ${i.skipCveScanReason ?? '(no reason)'}`);
4899
+ return;
4900
+ }
4901
+ await ctx.$`pnpm scan:cve:gate`;
4902
+ }),
4903
+ ],
4904
+ }),
4905
+ ],
4906
+ });
4907
+ ```
4908
+
4909
+ - **`defineDispatchInputs(map)`** is the single declaration site. It returns a
4910
+ handle that `dispatch({ inputs })` accepts and exposes `inputs.from(ctx)` — a
4911
+ typed reader over `ctx.dispatchInputs`, typed per declared key. `dispatch({ inputs })`
4912
+ also accepts a bare `{ name: schema }` map directly when you don't need the reader.
4913
+ - **`ctx.dispatchInputs`** is always present (a validated map of
4914
+ `string | number | boolean | null`), distinct from `ctx.inputs` (typed outputs
4915
+ from `needs` dependencies). Rules see the same values via `ctx.dispatchInputs`,
4916
+ so `skipUnless(ctx => !ctx.dispatchInputs.skipCveScan)` works.
4917
+ - **Defaults are applied once**, on the orchestrator (the authoritative side); the
4918
+ CLI pre-validates `--input` for fast feedback and forwards the raw operator pairs.
4919
+
4920
+ **Allowed input types (closed subset):** `z.string()`, `z.number()`,
4921
+ `z.boolean()`, `z.enum([...])`, `z.literal(v)`, with the modifiers `.optional()`,
4922
+ `.nullable()`, `.default(v)`, `.min(n)`, `.max(n)`, `.regex(re)`, `.int()`.
4923
+ Anything outside this set (`.refine()`, `.transform()`, `.pipe()`, `z.object()`,
4924
+ `z.array()`, `z.union()`, `z.record()`, `z.coerce.*`) is a **compile error** —
4925
+ the closed set is what guarantees the schema survives the trip to the
4926
+ orchestrator's lock file without silently dropping any validation. CLI strings
4927
+ are coerced for you (`--input retries=3` becomes the number `3`; booleans accept
4928
+ `true`/`false`/`1`/`0`/`yes`/`no`), so author your schema with clean types.
4929
+
4501
4930
  ### create()
4502
4931
 
4503
4932
  Create a ref creation trigger (branches/tags). Returns a frozen `CreateTriggerConfig`.
@@ -4611,7 +5040,7 @@ Create a catch-all webhook trigger for any GitHub event. Returns a frozen `Webho
4611
5040
  function webhook(config: WebhookConfigInput): WebhookTriggerConfig;
4612
5041
  ```
4613
5042
 
4614
- **Config options:** `events` (required string[]), `actions` (optional string[]), `repos` (optional cross-repo source patterns -- see [global workflows](../global-workflows.md)), `description`
5043
+ **Config options:** `events` (required string[]), `actions` (optional string[]), `repos` (optional cross-repo source patterns -- see [global workflows](https://docs.kici.dev/user/global-workflows/)), `description`
4615
5044
 
4616
5045
  ```typescript
4617
5046
  webhook({ events: ['deployment'] }); // Deployment events
@@ -4716,7 +5145,7 @@ jobComplete({ workflow: 'CI', job: 'build', status: ['success'] }); // Success o
4716
5145
  jobComplete({ workflow: 'CI', job: 'build', source: 'org/repo' }); // Cross-repo
4717
5146
  ```
4718
5147
 
4719
- `jobComplete()` starts a **new** workflow run that reacts to another job finishing (gated on the prior job's status). For same-run fan-out — generating follow-up jobs from a prior job's _outputs_ within the same run — use a result-aware [`dynamicJob(group, { needs, generate })`](./rules-matrix-dynamic.md#dynamicjob--result-aware-generation) instead.
5148
+ `jobComplete()` starts a **new** workflow run that reacts to another job finishing (gated on the prior job's status). For same-run fan-out — generating follow-up jobs from a prior job's _outputs_ within the same run — use a result-aware [`dynamicJob(group, { needs, generate })`](https://docs.kici.dev/user/sdk/rules-matrix-dynamic/#dynamicjob--result-aware-generation) instead.
4720
5149
 
4721
5150
  ### genericWebhook()
4722
5151
 
@@ -5096,9 +5525,9 @@ If `check()` throws while polling, the error is logged and polling continues —
5096
5525
 
5097
5526
  ## See also
5098
5527
 
5099
- - [Core SDK reference](./core.md) — the `step()`, `job()`, and `workflow()` factories that `waitForStep()` builds on.
5100
- - [Idempotent helpers](./idempotent.md) — `idempotent()` and `idempotentStep()` for check / apply patterns.
5101
- - [Runtime types](./runtime.md) — `StepContext`, `Logger`, and other surface used inside the helpers.
5528
+ - [Core SDK reference](https://docs.kici.dev/user/sdk/core/) — the `step()`, `job()`, and `workflow()` factories that `waitForStep()` builds on.
5529
+ - [Idempotent helpers](https://docs.kici.dev/user/sdk/idempotent/) — `idempotent()` and `idempotentStep()` for check / apply patterns.
5530
+ - [Runtime types](https://docs.kici.dev/user/sdk/runtime/) — `StepContext`, `Logger`, and other surface used inside the helpers.
5102
5531
 
5103
5532
  ---
5104
5533
 
@@ -5110,14 +5539,14 @@ Reference documentation for `@kici-dev/sdk`. The reference is split across five
5110
5539
 
5111
5540
  | Page | Covers |
5112
5541
  | ------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
5113
- | [Core](./sdk/core.md) | `workflow()`, `job()`, `step()` factory functions and step / job authoring patterns (bare functions, output chaining, `needs`, dynamic groups). |
5114
- | [Triggers](./sdk/triggers.md) | All 22 trigger factories -- GitHub events (`pr`, `push`, `tag`, `comment`, ...), event triggers (`kiciEvent`, `workflowComplete`, `jobComplete`), `genericWebhook`, `schedule`, `lifecycle`, plus branch-pattern semantics. |
5115
- | [Rules, matrix, dynamic jobs](./sdk/rules-matrix-dynamic.md) | `rule()`, `skip()`, matrix builds (static + dynamic), and `dynamicJob()` / `dynamicGroup()`. |
5116
- | [Caching](./sdk/caching.md) | `CacheSpec`, declarative `cache` on jobs/steps, imperative `ctx.cache.restore()` / `ctx.cache.save()`, immutable keys, `restoreKeys` prefix fallback, per-org + per-ref isolation. |
5117
- | [Validation & events](./sdk/validation-events.md) | `validateDag()`, `defineEvent()`, event emission patterns. |
5118
- | [Runtime](./sdk/runtime.md) | Types index, `StepContext`, secrets, and fixtures. |
5119
- | [Idempotent helpers](./sdk/idempotent.md) | `idempotent()` and `idempotentStep()` — check / apply pattern with typed results on both the skipped and applied branches. |
5120
- | [Wait-for helpers](./sdk/wait-for.md) | `waitFor()` and `waitForStep()` — poll a condition on an interval, run an optional success action, recover gracefully on timeout. |
5542
+ | [Core](https://docs.kici.dev/user/sdk/core/) | `workflow()`, `job()`, `step()` factory functions and step / job authoring patterns (bare functions, output chaining, `needs`, dynamic groups). |
5543
+ | [Triggers](https://docs.kici.dev/user/sdk/triggers/) | All 22 trigger factories -- GitHub events (`pr`, `push`, `tag`, `comment`, ...), event triggers (`kiciEvent`, `workflowComplete`, `jobComplete`), `genericWebhook`, `schedule`, `lifecycle`, plus branch-pattern semantics. |
5544
+ | [Rules, matrix, dynamic jobs](https://docs.kici.dev/user/sdk/rules-matrix-dynamic/) | `rule()`, `skip()`, matrix builds (static + dynamic), and `dynamicJob()` / `dynamicGroup()`. |
5545
+ | [Caching](https://docs.kici.dev/user/sdk/caching/) | `CacheSpec`, declarative `cache` on jobs/steps, imperative `ctx.cache.restore()` / `ctx.cache.save()`, immutable keys, `restoreKeys` prefix fallback, per-org + per-ref isolation. |
5546
+ | [Validation & events](https://docs.kici.dev/user/sdk/validation-events/) | `validateDag()`, `defineEvent()`, event emission patterns. |
5547
+ | [Runtime](https://docs.kici.dev/user/sdk/runtime/) | Types index, `StepContext`, secrets, and fixtures. |
5548
+ | [Idempotent helpers](https://docs.kici.dev/user/sdk/idempotent/) | `idempotent()`, `idempotentStep()`, and the check-mode-aware `checkStep()` — check / apply pattern with typed results on both the skipped and applied branches. |
5549
+ | [Wait-for helpers](https://docs.kici.dev/user/sdk/wait-for/) | `waitFor()` and `waitForStep()` — poll a condition on an interval, run an optional success action, recover gracefully on timeout. |
5121
5550
 
5122
5551
  The `@kici-dev/sdk` package re-exports the entire surface from a single entry point. Pick what you need:
5123
5552
 
@@ -5129,12 +5558,12 @@ For the complete list of every named export (factory functions, triggers, rules,
5129
5558
 
5130
5559
  ## See also
5131
5560
 
5132
- - [Getting started](getting-started.md) -- install the SDK, write your first workflow, test locally
5133
- - [CLI reference](cli-reference.md) -- compile, test, and manage workflows from the command line
5134
- - [Workflow patterns](workflow-patterns.md) -- common patterns using the SDK features documented above
5135
- - [Secrets management (operator)](../operator/security/secrets.md) -- configure encrypted secret storage and admin API
5136
- - [Secrets architecture](../architecture/security/secrets.md) -- encryption model, multi-backend, and data flow
5137
- - [State machine](../architecture/execution/state-machine.md) -- how execution states map to the lifecycle of jobs and steps
5561
+ - [Getting started](https://docs.kici.dev/user/getting-started/) -- install the SDK, write your first workflow, test locally
5562
+ - [CLI reference](https://docs.kici.dev/user/cli-reference/) -- compile, test, and manage workflows from the command line
5563
+ - [Workflow patterns](https://docs.kici.dev/user/workflow-patterns/) -- common patterns using the SDK features documented above
5564
+ - [Secrets management (operator)](https://docs.kici.dev/operator/security/secrets/) -- configure encrypted secret storage and admin API
5565
+ - [Secrets architecture](https://docs.kici.dev/architecture/security/secrets/) -- encryption model, multi-backend, and data flow
5566
+ - [State machine](https://docs.kici.dev/architecture/execution/state-machine/) -- how execution states map to the lifecycle of jobs and steps
5138
5567
 
5139
5568
  ---
5140
5569
 
@@ -5299,13 +5728,13 @@ Every request to `/api/v1/*` carries an `Authorization: Bearer <token>` header.
5299
5728
  | `kici_sa_` | Service account key | Dashboard → Settings → Service accounts | Org |
5300
5729
  | (other) | OIDC JWT or opaque OIDC token | OIDC login (browser SPA) | User (cross-org) |
5301
5730
 
5302
- JWT and opaque OIDC tokens are validated against the configured OIDC issuer (JWKS for JWTs, the issuer's UserInfo endpoint for opaque ones). All `kici_*` tokens are validated by SHA-256 hash lookup against the Platform DB. See [RBAC: authentication methods](../architecture/security/rbac.md#authentication-methods) for the full model.
5731
+ JWT and opaque OIDC tokens are validated against the configured OIDC issuer (JWKS for JWTs, the issuer's UserInfo endpoint for opaque ones). All `kici_*` tokens are validated by SHA-256 hash lookup against the Platform DB. See [RBAC: authentication methods](https://docs.kici.dev/architecture/security/rbac/#authentication-methods) for the full model.
5303
5732
 
5304
5733
  > **Note:** `kici_ok_` keys are **not** for the HTTP API — they authenticate orchestrator-to-Platform WebSocket connections only. Use `kici_sk_` (or `kici_pat_`) for HTTP calls.
5305
5734
 
5306
5735
  ### Permissions
5307
5736
 
5308
- Tokens authenticate; RBAC authorizes. Every org-scoped route runs `orgContextMiddleware` (verifies you are a member of the target org) followed by `requirePermission(resource, level)`. The 15 resources and 5 levels are documented in [RBAC](../architecture/security/rbac.md#permission-model). User API keys carry their own permission matrix bounded above by the creator's effective permissions; PATs inherit the user's role permissions (or are capped further by their `scopes` field).
5737
+ Tokens authenticate; RBAC authorizes. Every org-scoped route runs `orgContextMiddleware` (verifies you are a member of the target org) followed by `requirePermission(resource, level)`. The 17 resources and 5 levels are documented in [RBAC](https://docs.kici.dev/architecture/security/rbac/#permission-model). User API keys carry their own permission matrix bounded above by the creator's effective permissions; PATs inherit the user's role permissions (or are capped further by their `scopes` field).
5309
5738
 
5310
5739
  ### Configurable surfaces
5311
5740
 
@@ -5350,7 +5779,7 @@ There is currently no per-token rate limit on `/api/v1/*`. A single global body-
5350
5779
 
5351
5780
  ### Audit trail
5352
5781
 
5353
- Every `/api/v1/*` mutation that touches tenant-plane data is recorded in the upstream tenant-plane audit log, stamped with the actor (user, API key, service account, or upstream operator on a break-glass support read). Reads on customer data go through the orchestrator over the WebSocket proxy and land in the orchestrator's `access_log` table. See [Audit log](../operator/security/audit-log.md) for the orchestrator schema and the dashboard's "Activity" page for the federated view.
5782
+ Every `/api/v1/*` mutation that touches tenant-plane data is recorded in the upstream tenant-plane audit log, stamped with the actor (user, API key, service account, or upstream operator on a break-glass support read). Reads on customer data go through the orchestrator over the WebSocket proxy and land in the orchestrator's `access_log` table. See [Audit log](https://docs.kici.dev/operator/security/audit-log/) for the orchestrator schema and the dashboard's "Activity" page for the federated view.
5354
5783
 
5355
5784
  ## Token storage
5356
5785
 
@@ -5496,26 +5925,27 @@ kici run local [event] [options]
5496
5925
 
5497
5926
  **Options:**
5498
5927
 
5499
- | Option | Default | Description |
5500
- | ------------------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
5501
- | `-p, --pick` | `false` | Interactively pick a workflow + trigger (see below) |
5502
- | `--workflow <name>` | none | Run only the specified workflow (mutex with `--pick`) |
5503
- | `--job <name>` | none | Run only the specified job (and its dependencies) |
5504
- | `--branch <name>` | detected | Override detected git branch |
5505
- | `--sha <hash>` | detected | Override detected git SHA |
5506
- | `--payload <path>` | none | Path to explicit event payload JSON file |
5507
- | `--concurrency <n>` | CPU cores | Max parallel jobs **within one run** (job-level only). Cross-run [concurrency groups](concurrency.md) declared in `workflow({ concurrency: ... })` are enforced separately — see "Concurrency enforcement" below. |
5508
- | `--keep-going` | `false` | Continue after job failure |
5509
- | `--container` | `false` | Use Podman container isolation |
5510
- | `--env <KEY=VALUE>` | none | Environment variable override (repeatable) |
5511
- | `--files <path>` | git diff | Override changed file paths (repeatable, default: git diff) |
5512
- | `--quiet` | `false` | Suppress streaming output (summary only) |
5513
- | `--json` | `false` | Output structured JSON result |
5514
- | `--junit <path>` | none | Output JUnit XML result to file |
5515
- | `--debug` | `false` | Verbose internals |
5516
- | `--kici-dir <path>` | `.kici` | Path to .kici directory |
5517
- | `--in-place` | `false` | Run against the real working directory instead of an isolated tmp checkout (see "Execution isolation" below) |
5518
- | `--keep` | `false` | Always retain the isolated tmp checkout (default: keep only on failure) |
5928
+ | Option | Default | Description |
5929
+ | --------------------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
5930
+ | `-p, --pick` | `false` | Interactively pick a workflow + trigger (see below) |
5931
+ | `--workflow <name>` | none | Run only the specified workflow (mutex with `--pick`) |
5932
+ | `--job <name>` | none | Run only the specified job (and its dependencies) |
5933
+ | `--branch <name>` | detected | Override detected git branch |
5934
+ | `--sha <hash>` | detected | Override detected git SHA |
5935
+ | `--payload <path>` | none | Path to explicit event payload JSON file |
5936
+ | `--concurrency <n>` | CPU cores | Max parallel jobs **within one run** (job-level only). Cross-run [concurrency groups](https://docs.kici.dev/user/concurrency/) declared in `workflow({ concurrency: ... })` are enforced separately — see "Concurrency enforcement" below. |
5937
+ | `--keep-going` | `false` | Continue after job failure |
5938
+ | `--container` | `false` | Use Podman container isolation |
5939
+ | `--env <KEY=VALUE>` | none | Environment variable override (repeatable) |
5940
+ | `--input <KEY=VALUE>` | none | Typed workflow-dispatch input (repeatable) — coerced + validated against the workflow's `dispatch({ inputs })` schema, exposed as `ctx.dispatchInputs` (see [triggers → typed dispatch inputs](https://docs.kici.dev/user/sdk/triggers/#typed-dispatch-inputs)) |
5941
+ | `--files <path>` | git diff | Override changed file paths (repeatable, default: git diff) |
5942
+ | `--quiet` | `false` | Suppress streaming output (summary only) |
5943
+ | `--json` | `false` | Output structured JSON result |
5944
+ | `--junit <path>` | none | Output JUnit XML result to file |
5945
+ | `--debug` | `false` | Verbose internals |
5946
+ | `--kici-dir <path>` | `.kici` | Path to .kici directory |
5947
+ | `--in-place` | `false` | Run against the real working directory instead of an isolated tmp checkout (see "Execution isolation" below) |
5948
+ | `--keep` | `false` | Always retain the isolated tmp checkout (default: keep only on failure) |
5519
5949
 
5520
5950
  **Interactive workflow selection (`--pick` / `-p`):**
5521
5951
 
@@ -5619,9 +6049,11 @@ kici run local push --keep-going
5619
6049
 
5620
6050
  Execute fixtures remotely through the full CI pipeline. Fixtures are defined in `.kici/tests/*.ts` using the `fixture()` factory function. Without arguments, lists available fixtures.
5621
6051
 
5622
- Remote runs route through the Platform. Authenticate with a personal access token (`kici login`), then target an organization with `kici org use <org>` or the `--org` flag. The Platform relays the run to the org's orchestrator, while your working-tree overlay uploads directly to object storage — see [How the run is routed](#how-the-run-is-routed) and [The two planes](#the-two-planes) below.
6052
+ Remote runs route through the Platform. Authenticate with a personal access token (`kici login`), then target an organization with `kici org use <org>` or the `--org` flag. The Platform relays the run to the org's orchestrator, while your working-tree overlay uploads directly to object storage — see [How the run is routed](https://docs.kici.dev/user/cli-reference/#how-the-run-is-routed) and [The two planes](https://docs.kici.dev/user/cli-reference/#the-two-planes) below.
5623
6053
 
5624
- The orchestrator must have **cache storage configured** (`KICI_STORAGE_TYPE` = `s3` or `filesystem`) with a dev-reachable upload endpoint so the CLI's direct upload succeeds; see the [testing guide](testing-guide.md) and [Storage layout](../operator/orchestrator/storage-layout.md) for setup.
6054
+ Like `kici run local`, `kici run remote` recompiles your workflows (`.kici/workflows` `kici.lock.json`) before dispatching, so the orchestrator matches and dispatches against your current workflow definitions — a brand-new or edited workflow takes effect without a separate `kici compile`. A compile or validation error aborts the run before anything is uploaded.
6055
+
6056
+ The orchestrator must have **cache storage configured** (`KICI_STORAGE_TYPE` = `s3` or `filesystem`) with a dev-reachable upload endpoint so the CLI's direct upload succeeds; see the [testing guide](https://docs.kici.dev/user/testing-guide/) and [Storage layout](https://docs.kici.dev/operator/orchestrator/storage-layout/) for setup.
5625
6057
 
5626
6058
  ```bash
5627
6059
  kici run remote [fixture] [options]
@@ -5635,22 +6067,26 @@ kici run remote [fixture] [options]
5635
6067
 
5636
6068
  **Options:**
5637
6069
 
5638
- | Option | Default | Description |
5639
- | --------------------------- | ------- | ------------------------------------------------------------------------------------------------- |
5640
- | `--org <id>` | active | Target organization for this run (overrides `kici org use`) |
5641
- | `--orchestrator <name>` | default | Target orchestrator cluster within the org (overrides the per-org default) |
5642
- | `--all` | `false` | Run all fixtures |
5643
- | `--workflow <name>` | none | Run a specific workflow directly (bypass triggers) |
5644
- | `--parallel` | `false` | Run multiple fixtures concurrently |
5645
- | `--no-wait` | - | Fire and forget (print runIds, don't stream) |
5646
- | `--quiet` | `false` | Minimal output (only final result) |
5647
- | `--json` | `false` | Machine-readable JSON output |
5648
- | `--junit <path>` | none | JUnit XML output to file for CI integration |
5649
- | `--history` | `false` | Show table of recent test runs |
5650
- | `--context <ctx.key=value>` | none | Inject a namespaced context secret, uploaded encrypted (repeatable) |
5651
- | `--env <KEY=VALUE>` | none | Provide a per-run secret, uploaded encrypted (repeatable) — see [testing guide](testing-guide.md) |
5652
- | `--debug` | `false` | Verbose internals |
5653
- | `--kici-dir <path>` | `.kici` | Path to .kici directory |
6070
+ | Option | Default | Description |
6071
+ | --------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
6072
+ | `--org <id>` | active | Target organization for this run (overrides `kici org use`) |
6073
+ | `--orchestrator <name>` | default | Target orchestrator cluster within the org (overrides the per-org default) |
6074
+ | `--all` | `false` | Run all fixtures |
6075
+ | `-p, --pick` | `false` | Interactively pick fixtures to run (multi-select; mutex with a fixture arg, `--all`, and `--workflow`) |
6076
+ | `--workflow <name>` | none | Run a specific workflow directly (bypass triggers) |
6077
+ | `--parallel` | `false` | Run multiple fixtures concurrently |
6078
+ | `--no-wait` | - | Fire and forget (print runIds, don't stream) |
6079
+ | `--quiet` | `false` | Minimal output (only final result) |
6080
+ | `--json` | `false` | Machine-readable JSON output |
6081
+ | `--junit <path>` | none | JUnit XML output to file for CI integration |
6082
+ | `--history` | `false` | Show table of recent test runs |
6083
+ | `--context <ctx.key=value>` | none | Inject a namespaced context secret, uploaded encrypted (repeatable) |
6084
+ | `--env <KEY=VALUE>` | none | Provide a per-run secret, uploaded encrypted (repeatable) — see [testing guide](https://docs.kici.dev/user/testing-guide/) |
6085
+ | `--input <KEY=VALUE>` | none | Typed workflow-dispatch input (repeatable) — validated + coerced + defaulted on the orchestrator from the lock descriptor, exposed as `ctx.dispatchInputs` (see [triggers typed dispatch inputs](https://docs.kici.dev/user/sdk/triggers/#typed-dispatch-inputs)) |
6086
+ | `--target <selector>` | none | Narrow `runsOnAll` jobs to hosts matching this label selector (repeatable, AND-combined) |
6087
+ | `--target-allow-empty` | `false` | A `--target` that narrows a `runsOnAll` job to zero hosts skips it instead of failing |
6088
+ | `--debug` | `false` | Verbose internals |
6089
+ | `--kici-dir <path>` | `.kici` | Path to .kici directory |
5654
6090
 
5655
6091
  **Examples:**
5656
6092
 
@@ -5687,8 +6123,59 @@ kici run remote push-main --no-wait
5687
6123
 
5688
6124
  # View recent test run history
5689
6125
  kici run remote --history
6126
+
6127
+ # Interactively pick which fixtures to run (multi-select)
6128
+ kici run remote --pick
6129
+
6130
+ # Narrow runsOnAll jobs to a subset of the host roster
6131
+ kici run remote deploy --target role:web
6132
+
6133
+ # AND-combine repeated --target values (hosts must match every selector)
6134
+ kici run remote deploy --target role:web --target dc:eu
6135
+
6136
+ # Skip a runsOnAll job instead of failing it when the target matches no host
6137
+ kici run remote deploy --target role:gpu --target-allow-empty
5690
6138
  ```
5691
6139
 
6140
+ **Interactive fixture selection (`--pick` / `-p`):**
6141
+
6142
+ Pass `--pick` (or `-p`) to open an interactive checkbox menu of the available
6143
+ fixtures. Toggle one or more with space, confirm with enter, and the selected
6144
+ fixtures run through the normal remote pipeline (honoring `--parallel`,
6145
+ `--no-wait`, and the other run flags). Notes:
6146
+
6147
+ - `--pick` is mutually exclusive with a fixture argument, `--all`, and
6148
+ `--workflow`. Passing any together exits with code 2.
6149
+ - When `stdin` is not a TTY, `--pick` prints the available fixtures and exits
6150
+ without running anything — pass a fixture name (or `--all`) in scripts.
6151
+
6152
+ #### Host narrowing with `--target`
6153
+
6154
+ `--target <selector>` is a runtime narrowing for `runsOnAll` jobs, analogous to
6155
+ Ansible's `--limit`. A `runsOnAll` job normally fans out to **every** roster host
6156
+ matching its predicate, one pinned execution per host. `--target` intersects that
6157
+ matched roster with a label selector, so the effective host set is
6158
+ `runsOnAll ∩ target`:
6159
+
6160
+ - **Narrow-only.** `--target` can only _remove_ hosts from the matched set, never
6161
+ add them. The widening dimension (OR across host groups) lives in the workflow's
6162
+ `runsOnAll`; `--target` only subtracts.
6163
+ - **Run-global, `runsOnAll`-only.** A single `--target` applies to every
6164
+ `runsOnAll` job in the run. Jobs pinned to a single host with `runsOn` are
6165
+ untouched.
6166
+ - **Repeatable and AND-combined.** Each `--target` value is its own selector; a
6167
+ host must satisfy **all** of them to survive the narrowing. Use a single value
6168
+ for an OR-style match within one selector and repeated values for AND.
6169
+ - **Selector syntax** matches `runsOn`: an exact label (`role:web`), a glob
6170
+ (`role:*`), or a regex (`/^box-0[1-3]$/`).
6171
+
6172
+ When `--target` narrows a `runsOnAll` job to zero hosts, the default is to **fail**
6173
+ the run (fail-loud — a typo in the selector shouldn't silently skip work). Pass
6174
+ `--target-allow-empty` to **skip** the zeroed job instead; the job records a
6175
+ `skipped` status, and any downstream job that needs it with `when: 'on-skip'` (or
6176
+ `when: 'always'`) still runs. See [Job dependencies](https://docs.kici.dev/user/sdk/core/#job-dependencies-needs)
6177
+ for the `when` gating model.
6178
+
5692
6179
  **Exit codes:**
5693
6180
 
5694
6181
  | Code | Meaning |
@@ -5718,9 +6205,9 @@ When an org has more than one connected orchestrator cluster, the CLI picks the
5718
6205
  `kici run remote` uses two independent paths:
5719
6206
 
5720
6207
  - **Control plane** — run initiation, trigger, status, log retrieval, and cancellation flow from your machine through the Platform, which relays them over a WebSocket connection to the org's orchestrator. Logs are delivered by the CLI polling the Platform for log chunks (tracked by a monotonic line cursor) and run status until the run reaches a terminal state; there is no direct streaming socket to the orchestrator.
5721
- - **Data plane** — your working-tree overlay tarball uploads **directly** from your machine to the orchestrator's object store via a presigned PUT URL. The overlay never passes through the Platform. This is why the orchestrator's object-store upload endpoint must be reachable from your machine; see [Storage layout](../operator/orchestrator/storage-layout.md).
6208
+ - **Data plane** — your working-tree overlay tarball uploads **directly** from your machine to the orchestrator's object store via a presigned PUT URL. The overlay never passes through the Platform. This is why the orchestrator's object-store upload endpoint must be reachable from your machine; see [Storage layout](https://docs.kici.dev/operator/orchestrator/storage-layout/).
5722
6209
 
5723
- An orchestrator with no Platform connection cannot serve remote runs — the Platform is the service that offers them. For executing workflow steps on your own machine without an orchestrator (no scaler, agents, or environments), use [`kici run local`](#kici-run-local).
6210
+ An orchestrator with no Platform connection cannot serve remote runs — the Platform is the service that offers them. For executing workflow steps on your own machine without an orchestrator (no scaler, agents, or environments), use [`kici run local`](https://docs.kici.dev/user/cli-reference/#kici-run-local).
5724
6211
 
5725
6212
  #### Fresh repos (no GitHub remote)
5726
6213
 
@@ -5734,7 +6221,7 @@ An orchestrator with no Platform connection cannot serve remote runs — the Pla
5734
6221
 
5735
6222
  Destination routing is unchanged for fresh repos: the run still goes to your active org through the Platform.
5736
6223
 
5737
- For a detailed guide on writing fixtures, configuring secrets, and understanding the upload flow, see [Testing guide](testing-guide.md).
6224
+ For a detailed guide on writing fixtures, configuring secrets, and understanding the upload flow, see [Testing guide](https://docs.kici.dev/user/testing-guide/).
5738
6225
 
5739
6226
  #### kici orchestrators
5740
6227
 
@@ -5829,18 +6316,21 @@ By default, `kici login` opens your browser for OIDC authentication using PKCE.
5829
6316
 
5830
6317
  After OAuth, the CLI exchanges the OIDC token for a personal access token (PAT) stored in the config directory (`~/.kici/config` by default, overridable with `KICI_CONFIG_DIR`).
5831
6318
 
6319
+ `kici login` targets the hosted KiCI Platform by default. To authenticate against another environment (a self-hosted Platform, for example), pass `--platform-endpoint` / `--oidc-issuer` or set `KICI_PLATFORM_URL` / `KICI_OIDC_ISSUER`. Login persists the platform endpoint and OIDC issuer it authenticated against alongside the PAT, so a saved PAT always matches its endpoint. Because the config describes one environment at a time, **switching the endpoint resets the active organization and default clusters** — re-run `kici org use <name>` after switching environments.
6320
+
5832
6321
  ```bash
5833
6322
  kici login [options]
5834
6323
  ```
5835
6324
 
5836
6325
  **Options:**
5837
6326
 
5838
- | Option | Default | Description |
5839
- | --------------------------- | ------- | ---------------------------------------------- |
5840
- | `--token <key>` | none | API key for direct authentication (legacy) |
5841
- | `--device` | false | Force device authorization flow (headless/SSH) |
5842
- | `--platform-endpoint <url>` | none | Platform relay URL |
5843
- | `--routing-key <key>` | none | Routing key for webhook source identification |
6327
+ | Option | Default | Description |
6328
+ | --------------------------- | ------- | --------------------------------------------------- |
6329
+ | `--token <key>` | none | API key for direct authentication (legacy) |
6330
+ | `--device` | false | Force device authorization flow (headless/SSH) |
6331
+ | `--platform-endpoint <url>` | none | Platform relay URL |
6332
+ | `--oidc-issuer <url>` | none | OIDC issuer URL (selects a non-default environment) |
6333
+ | `--routing-key <key>` | none | Routing key for webhook source identification |
5844
6334
 
5845
6335
  **Environment variables:**
5846
6336
 
@@ -5866,7 +6356,8 @@ kici login --device
5866
6356
  kici login --token kici_sk_abc123...
5867
6357
 
5868
6358
  # Log in against a self-hosted Platform
5869
- kici login --platform-endpoint https://platform.example.com
6359
+ kici login --platform-endpoint https://platform.example.com \
6360
+ --oidc-issuer https://auth.example.com/realms/kici-internal
5870
6361
 
5871
6362
  # Suppress browser opening (print authorize URL to stdout)
5872
6363
  KICI_BROWSER_CMD=none kici login
@@ -6115,7 +6606,7 @@ for `--quiet`.
6115
6606
 
6116
6607
  ### kici approve
6117
6608
 
6118
- Approve a held [approval gate](approvals.md) so the run resumes. Identify the held element by run ID, optionally narrowed to a job and step.
6609
+ Approve a held [approval gate](https://docs.kici.dev/user/approvals/) so the run resumes. Identify the held element by run ID, optionally narrowed to a job and step.
6119
6610
 
6120
6611
  ```bash
6121
6612
  kici approve <run-id> [options]
@@ -6151,7 +6642,7 @@ You must be eligible for at least one unsatisfied clause (a member of a named te
6151
6642
 
6152
6643
  ### kici reject
6153
6644
 
6154
- Reject a held [approval gate](approvals.md). A rejection fails the held element and the run. A reason is required.
6645
+ Reject a held [approval gate](https://docs.kici.dev/user/approvals/). A rejection fails the held element and the run. A reason is required.
6155
6646
 
6156
6647
  ```bash
6157
6648
  kici reject <run-id> --reason <text> [options]
@@ -6230,7 +6721,7 @@ After generating types, `ctx.secrets.get('MY_KEY')` and `ctx.secrets.expose('DB_
6230
6721
 
6231
6722
  **Git workflow:** Commit the generated `.kici/types/secrets.d.ts` so team members get type checking without needing orchestrator access. Run `kici types` to refresh when environments change.
6232
6723
 
6233
- **Auto-regeneration:** `kici compile` automatically runs `kici types` after successful compilation when authenticated. See the [kici compile](#kici-compile) section for details.
6724
+ **Auto-regeneration:** `kici compile` automatically runs `kici types` after successful compilation when authenticated. See the [kici compile](https://docs.kici.dev/user/cli-reference/#kici-compile) section for details.
6234
6725
 
6235
6726
  **Escape hatch:** For dynamic keys not in the generated types, use a cast: `(ctx.secrets as any).DYNAMIC_KEY`.
6236
6727
 
@@ -6449,9 +6940,10 @@ Open the KiCI documentation site in the default browser. With the `llm` subcomma
6449
6940
  ```bash
6450
6941
  kici docs # open https://kici.dev/docs/
6451
6942
  kici docs --no-open # print the URL instead of opening a browser
6452
- kici docs llm # print llms-full.txt (the full bundle) to stdout
6453
- kici docs llm --index # print llms.txt (the curated link index) to stdout
6454
- kici docs llm --out path/to/file.md # write the bundle to a file
6943
+ kici docs llm # print the llms.txt index (a router over the task bundles)
6944
+ kici docs llm sdk # print the SDK task bundle
6945
+ kici docs llm full # print llms-full.txt (every page in one file)
6946
+ kici docs llm sdk --out sdk-context.md # write a bundle to a file
6455
6947
  ```
6456
6948
 
6457
6949
  **Examples:**
@@ -6460,14 +6952,14 @@ kici docs llm --out path/to/file.md # write the bundle to a file
6460
6952
  # Open the docs site in your browser
6461
6953
  kici docs
6462
6954
 
6463
- # Pipe the full LLM bundle into a coding agent
6464
- kici docs llm | claude -- "Read this and help me author a deploy workflow"
6955
+ # Pipe just the SDK bundle into a coding agent (small, task-scoped context)
6956
+ kici docs llm sdk | claude -- "Read this and help me author a deploy workflow"
6465
6957
 
6466
- # Save the curated index for offline reference
6467
- kici docs llm --index --out kici-llms.txt
6958
+ # Save the router index for offline reference
6959
+ kici docs llm --out kici-llms-index.txt
6468
6960
  ```
6469
6961
 
6470
- The bundle is regenerated from `docs/` every time `@kici-dev/compiler` is built, so it always matches your installed CLI version. The same content is available online at <https://kici.dev/llms.txt> and <https://kici.dev/llms-full.txt> following the [llms.txt convention](https://llmstxt.org/).
6962
+ Bundles are regenerated from `docs/` every time `@kici-dev/compiler` is built, so they always match your installed CLI version. The index lists each task bundle `getting-started`, `sdk`, `cli`, `patterns`, `features`, `providers`, `architecture` — with its size and a one-line purpose; pass the bundle id as the topic. Every cross-reference link inside a bundle is an absolute `docs.kici.dev` URL. The same files are published online following the [llms.txt convention](https://llmstxt.org/).
6471
6963
 
6472
6964
  ### kici admin
6473
6965
 
@@ -6506,7 +6998,7 @@ kici admin drain-worker --url http://worker-2.internal:10143
6506
6998
 
6507
6999
  ### kici verify-attestation
6508
7000
 
6509
- Verify a KiCI build-provenance attestation bundle offline. A bundle is the signed package a workflow step produces via `ctx.attestProvenance(...)`: a DSSE-wrapped SLSA in-toto statement, the ephemeral public key that signed it, and the KiCI identity token that anchors the build context. For the end-to-end attest → verify → view journey, see the [build provenance guide](./provenance.md). Verification establishes the full chain — the identity token verifies against the trusted issuer's JWKS, the DSSE signature verifies against the bundled key, and the statement's build context must match the token's claims (a mismatch is a hard failure). When an `[artifact]` is given, its SHA-256 digest is also matched against the attestation subject.
7001
+ Verify a KiCI build-provenance attestation bundle offline. A bundle is the signed package a workflow step produces via `ctx.attestProvenance(...)`: a DSSE-wrapped SLSA in-toto statement, the ephemeral public key that signed it, and the KiCI identity token that anchors the build context. For the end-to-end attest → verify → view journey, see the [build provenance guide](https://docs.kici.dev/user/provenance/). Verification establishes the full chain — the identity token verifies against the trusted issuer's JWKS, the DSSE signature verifies against the bundled key, and the statement's build context must match the token's claims (a mismatch is a hard failure). When an `[artifact]` is given, its SHA-256 digest is also matched against the attestation subject.
6510
7002
 
6511
7003
  ```bash
6512
7004
  kici verify-attestation [artifact] --bundle <path-or-url> --trust-root <url-or-file> [options]
@@ -6595,7 +7087,7 @@ The `kici compile` command produces `.kici/kici.lock.json` inside the `.kici` di
6595
7087
  - Should be committed to version control
6596
7088
  - Is regenerated on every `kici compile` run
6597
7089
 
6598
- Use `kici compile --check` in CI to validate that workflows are correct without writing files. For the full story on drift, pre-commit/CI, and agent-side verification, see [Lock file and workflow drift](lock-file-and-drift.md).
7090
+ Use `kici compile --check` in CI to validate that workflows are correct without writing files. For the full story on drift, pre-commit/CI, and agent-side verification, see [Lock file and workflow drift](https://docs.kici.dev/user/lock-file-and-drift/).
6599
7091
 
6600
7092
  ## Exit codes
6601
7093
 
@@ -6633,10 +7125,10 @@ Set `KICI_DEBUG=true` for additional internal debug output across all commands.
6633
7125
 
6634
7126
  ## See also
6635
7127
 
6636
- - [Getting started](getting-started.md) -- install the SDK and write your first workflow
6637
- - [Testing guide](testing-guide.md) -- writing fixtures, remote test runs, secret contexts, and repo state transfer
6638
- - [SDK reference](sdk-reference.md) -- complete API for the workflow definitions that the CLI compiles
6639
- - [Workflow patterns](workflow-patterns.md) -- example workflows to compile and test with these commands
7128
+ - [Getting started](https://docs.kici.dev/user/getting-started/) -- install the SDK and write your first workflow
7129
+ - [Testing guide](https://docs.kici.dev/user/testing-guide/) -- writing fixtures, remote test runs, secret contexts, and repo state transfer
7130
+ - [SDK reference](https://docs.kici.dev/user/sdk-reference/) -- complete API for the workflow definitions that the CLI compiles
7131
+ - [Workflow patterns](https://docs.kici.dev/user/workflow-patterns/) -- example workflows to compile and test with these commands
6640
7132
 
6641
7133
  ---
6642
7134
 
@@ -6917,12 +7409,12 @@ Each workflow entry includes:
6917
7409
  | `jobs` | Job definitions with scheduling metadata (runsOn, needs, matrix, environment, concurrency, container, checkout, gracePeriod, label routing, dynamic fields, etc.). |
6918
7410
  | `rules` | Workflow-level conditional rules (optional). Stored as dynamic references since rule functions cannot be serialized. |
6919
7411
  | `description` | Optional workflow description. |
6920
- | `hashFiles` | Declared glob patterns for extra files included in the content hash (optional). See [extra files in the content hash](#extra-files-in-the-content-hash-hashfiles). |
7412
+ | `hashFiles` | Declared glob patterns for extra files included in the content hash (optional). See [extra files in the content hash](https://docs.kici.dev/user/lock-file-and-drift/#extra-files-in-the-content-hash-hashfiles). |
6921
7413
  | `resolvedHashFiles` | Resolved file paths from `hashFiles` at compile time (optional). Recorded so the agent can verify without re-discovering. |
6922
7414
  | `contexts` | Secret contexts declared by the workflow (optional). The orchestrator validates access to each context before dispatch. |
6923
- | `registries` | Private npm registry declarations the agent authenticates against before install (optional): `url`, `scope`, `tokenSecret` reference, `alwaysAuth`. Resolved token bytes never appear in the lock file. See [private registries](private-registries.md). |
6924
- | `installEnv` | Extra qualified secret refs (`<environment>:<secret-name>`) projected as env vars on the install subprocess for use with a committed `.kici/.npmrc` (optional). See [private registries](private-registries.md). |
6925
- | `concurrency` | Workflow-level concurrency config: `hasGroup`, `cancelInProgress`, `max` (optional). See [concurrency groups](concurrency.md). |
7415
+ | `registries` | Private npm registry declarations the agent authenticates against before install (optional): `url`, `scope`, `tokenSecret` reference, `alwaysAuth`. Resolved token bytes never appear in the lock file. See [private registries](https://docs.kici.dev/user/private-registries/). |
7416
+ | `installEnv` | Extra qualified secret refs (`<environment>:<secret-name>`) projected as env vars on the install subprocess for use with a committed `.kici/.npmrc` (optional). See [private registries](https://docs.kici.dev/user/private-registries/). |
7417
+ | `concurrency` | Workflow-level concurrency config: `hasGroup`, `cancelInProgress`, `max` (optional). See [concurrency groups](https://docs.kici.dev/user/concurrency/). |
6926
7418
  | `timeout` | Whole-run wall-clock timeout in milliseconds (optional). The orchestrator reads this at run creation to set the run deadline. |
6927
7419
  | Hook flags | Boolean flags (`hasOnCancel`, `hasCleanup`, `hasOnSuccess`, `hasOnFailure`) indicating which lifecycle hooks are defined. Job entries additionally have `hasBeforeStep` and `hasAfterStep`. |
6928
7420
 
@@ -6951,7 +7443,7 @@ Install a hook that compiles and stages the lock file before each commit:
6951
7443
  npx kici hook install
6952
7444
  ```
6953
7445
 
6954
- This runs `kici compile && git add .kici/kici.lock.json` before each commit: if compilation fails the commit is blocked; if it succeeds the updated lock file is automatically staged. See [CLI Reference — kici hook](cli-reference.md#kici-hook) for options (husky, lefthook, pre-commit, prek, raw git).
7446
+ This runs `kici compile && git add .kici/kici.lock.json` before each commit: if compilation fails the commit is blocked; if it succeeds the updated lock file is automatically staged. See [CLI Reference — kici hook](https://docs.kici.dev/user/cli-reference/#kici-hook) for options (husky, lefthook, pre-commit, prek, raw git).
6955
7447
 
6956
7448
  ### CI check
6957
7449
 
@@ -7001,9 +7493,9 @@ So even without a pre-commit or CI check, a stale lock file will cause the run t
7001
7493
 
7002
7494
  ## See also
7003
7495
 
7004
- - [Getting Started](getting-started.md) — compile and commit the lock file
7005
- - [CLI Reference](cli-reference.md) — `kici compile`, `kici compile --check`, `kici hook`
7006
- - [Architecture — Data flows](../architecture/data-flows.md) — how the lock file is used in the pipeline
7496
+ - [Getting Started](https://docs.kici.dev/user/getting-started/) — compile and commit the lock file
7497
+ - [CLI Reference](https://docs.kici.dev/user/cli-reference/) — `kici compile`, `kici compile --check`, `kici hook`
7498
+ - [Architecture — Data flows](https://docs.kici.dev/architecture/data-flows/) — how the lock file is used in the pipeline
7007
7499
 
7008
7500
  ---
7009
7501
 
@@ -7025,14 +7517,14 @@ Test your workflows remotely against the full CI pipeline from your local machin
7025
7517
  The command is remote-only -- all execution happens on the orchestrator and agent. For local-only trigger matching previews, use `kici test <event>`.
7026
7518
 
7027
7519
  :::note[Orchestrator prerequisite: cache storage]
7028
- `kici run remote` uploads your working-tree overlay to the orchestrator's **cache storage** via a pre-signed URL, and the agent fetches it from there (see [Repo state transfer](#repo-state-transfer)). The target orchestrator must therefore have cache storage enabled (`KICI_STORAGE_TYPE` = `s3` or `filesystem`).
7520
+ `kici run remote` uploads your working-tree overlay to the orchestrator's **cache storage** via a pre-signed URL, and the agent fetches it from there (see [Repo state transfer](https://docs.kici.dev/user/testing-guide/#repo-state-transfer)). The target orchestrator must therefore have cache storage enabled (`KICI_STORAGE_TYPE` = `s3` or `filesystem`).
7029
7521
 
7030
- - **Both quickstarts wire this up for you** — the [Docker / Podman quickstart](quickstart/compose.md) and the [bare-metal quickstart](quickstart/bare-metal.md) each ship a SeaweedFS object store and pre-fill the orchestrator's `KICI_STORAGE_*` block, so `kici run remote` works out of the box (see each guide's "run a workflow without pushing" step).
7522
+ - **Both quickstarts wire this up for you** — the [Docker / Podman quickstart](https://docs.kici.dev/user/quickstart/compose/) and the [bare-metal quickstart](https://docs.kici.dev/user/quickstart/bare-metal/) each ship a SeaweedFS object store and pre-fill the orchestrator's `KICI_STORAGE_*` block, so `kici run remote` works out of the box (see each guide's "run a workflow without pushing" step).
7031
7523
  - **A hand-rolled orchestrator deploy does not configure storage by default** — enable a backend before using `kici run remote`:
7032
7524
  - **`filesystem`** — simplest for a single-host orchestrator: set `KICI_STORAGE_TYPE=filesystem` and `KICI_STORAGE_FS_PATH=/var/lib/kici/cache`. No external service needed; blobs are served through the orchestrator's own HMAC-signed HTTP route.
7033
7525
  - **`s3`** — any S3-compatible bucket. **A non-public / self-hosted endpoint works**: set `KICI_STORAGE_TYPE=s3`, `KICI_STORAGE_BUCKET`, `KICI_STORAGE_ENDPOINT=https://your-endpoint` and (for most self-hosted services) `KICI_STORAGE_FORCE_PATH_STYLE=true`. If the developer machine running `kici run remote` reaches the bucket at a different address than the orchestrator, set `KICI_STORAGE_UPLOAD_ENDPOINT` to the developer-reachable address; if agents reach it at yet another address (e.g. agents in containers), set `KICI_STORAGE_EXTERNAL_ENDPOINT` to the agent-routable URL.
7034
7526
 
7035
- See [Storage layout](../operator/orchestrator/storage-layout.md) for the full env-var reference.
7527
+ See [Storage layout](https://docs.kici.dev/operator/orchestrator/storage-layout/) for the full env-var reference.
7036
7528
  :::
7037
7529
 
7038
7530
  ## Getting started
@@ -7043,7 +7535,7 @@ See [Storage layout](../operator/orchestrator/storage-layout.md) for the full en
7043
7535
  kici login
7044
7536
  ```
7045
7537
 
7046
- This opens your browser for OAuth authentication and stores a personal access token in `~/.kici/config`. For CI/CD pipelines or headless environments, use `kici login --token <your-api-key>` or `kici login --device` instead. See [CLI authentication](cli-auth.md) for details.
7538
+ This opens your browser for OAuth authentication and stores a personal access token in `~/.kici/config`. For CI/CD pipelines or headless environments, use `kici login --token <your-api-key>` or `kici login --device` instead. See [CLI authentication](https://docs.kici.dev/user/cli-auth/) for details.
7047
7539
 
7048
7540
  ### 2. Write a test fixture
7049
7541
 
@@ -7151,8 +7643,8 @@ This maps the `db` secret context to the `test-database` context, and `api` to `
7151
7643
 
7152
7644
  This mapping is honored by **both** `kici run local` and `kici run remote`:
7153
7645
 
7154
- - For **`kici run local`** (see [`kici run local`](cli-reference.md#kici-run-local)), each named context is resolved from your local secret files (`.kici/.secrets`, `.env.local`, `secrets.yaml`, and `--env` flags).
7155
- - For **`kici run remote`**, each named context maps to an orchestrator **environment**, and the orchestrator resolves that environment's secrets for the run. The target environment must be flagged `allowLocalExecution: true` — mapping a context to a missing or non-test environment rejects the run (see [Secret contexts for testing](#secret-contexts-for-testing) below).
7646
+ - For **`kici run local`** (see [`kici run local`](https://docs.kici.dev/user/cli-reference/#kici-run-local)), each named context is resolved from your local secret files (`.kici/.secrets`, `.env.local`, `secrets.yaml`, and `--env` flags).
7647
+ - For **`kici run remote`**, each named context maps to an orchestrator **environment**, and the orchestrator resolves that environment's secrets for the run. The target environment must be flagged `allowLocalExecution: true` — mapping a context to a missing or non-test environment rejects the run (see [Secret contexts for testing](https://docs.kici.dev/user/testing-guide/#secret-contexts-for-testing) below).
7156
7648
 
7157
7649
  ### Async fixtures
7158
7650
 
@@ -7307,7 +7799,7 @@ Because these values originate on your machine, they are the natural place to pu
7307
7799
 
7308
7800
  In addition to your uploaded values, the orchestrator resolves test-scoped secrets from its own store for a remote test run:
7309
7801
 
7310
- - The job's own declared `environment` contributes its resolved secrets (flat). Static strings and **pure dynamic functions** both participate: a pure `environment:` function (see [Dynamic values](dynamic-values.md)) is evaluated against the fixture's simulated event, and the resolved name is gated and resolved like a static one. Impure dynamic functions (those requiring an init job) are not evaluated for test runs — use a fixture `secrets:` mapping (or `--context`) to supply such a job's secrets.
7802
+ - The job's own declared `environment` contributes its resolved secrets (flat). Static strings and **pure dynamic functions** both participate: a pure `environment:` function (see [Dynamic values](https://docs.kici.dev/user/dynamic-values/)) is evaluated against the fixture's simulated event, and the resolved name is gated and resolved like a static one. Impure dynamic functions (those requiring an init job) are not evaluated for test runs — use a fixture `secrets:` mapping (or `--context`) to supply such a job's secrets.
7311
7803
  - Each fixture `secrets: { ctx: envName }` mapping resolves the named environment's secrets under the namespaced context `ctx`.
7312
7804
 
7313
7805
  Both paths are restricted to environments flagged `allowLocalExecution: true`. A production environment left at the default `false` is never resolvable for a test run.
@@ -7352,7 +7844,7 @@ or via the dashboard's "Test runs" toggle on the environment detail page. `kici
7352
7844
 
7353
7845
  ### Local execution as an alternative
7354
7846
 
7355
- `kici run local` resolves the same local secret files entirely on your machine and honors the fixture `secrets: { ... }` mapping to pick which local context backs each name (see [`kici run local`](cli-reference.md#kici-run-local)). Because the values never leave your machine, it's a good fit when you want to exercise secret-dependent steps without involving the orchestrator at all.
7847
+ `kici run local` resolves the same local secret files entirely on your machine and honors the fixture `secrets: { ... }` mapping to pick which local context backs each name (see [`kici run local`](https://docs.kici.dev/user/cli-reference/#kici-run-local)). Because the values never leave your machine, it's a good fit when you want to exercise secret-dependent steps without involving the orchestrator at all.
7356
7848
 
7357
7849
  ### Discovering available contexts
7358
7850
 
@@ -7429,9 +7921,9 @@ export const pushMain = fixture('push-main', {
7429
7921
 
7430
7922
  ## See also
7431
7923
 
7432
- - [CLI reference](cli-reference.md) -- complete command reference for all `kici` commands
7433
- - [SDK reference](sdk-reference.md) -- trigger functions, step context, and workflow API
7434
- - [Workflow patterns](workflow-patterns.md) -- example workflows to test against
7924
+ - [CLI reference](https://docs.kici.dev/user/cli-reference/) -- complete command reference for all `kici` commands
7925
+ - [SDK reference](https://docs.kici.dev/user/sdk-reference/) -- trigger functions, step context, and workflow API
7926
+ - [Workflow patterns](https://docs.kici.dev/user/workflow-patterns/) -- example workflows to test against
7435
7927
 
7436
7928
  ---
7437
7929
 
@@ -7439,24 +7931,24 @@ export const pushMain = fixture('push-main', {
7439
7931
 
7440
7932
  Source: https://docs.kici.dev/user/workflow-patterns/
7441
7933
 
7442
- Practical patterns for building real-world KiCI workflows in TypeScript. The patterns are organised across five pages -- start with [Basic CI](./patterns/basic.md) if you're new, or jump to [Integrations](./patterns/integrations.md) if you're wiring up a non-GitHub forge or a generic webhook.
7934
+ Practical patterns for building real-world KiCI workflows in TypeScript. The patterns are organised across five pages -- start with [Basic CI](https://docs.kici.dev/user/patterns/basic/) if you're new, or jump to [Integrations](https://docs.kici.dev/user/patterns/integrations/) if you're wiring up a non-GitHub forge or a generic webhook.
7443
7935
 
7444
7936
  | Page | Covers |
7445
7937
  | ---------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
7446
- | [Basic CI](./patterns/basic.md) | Single-job CI, PR-only / push-only filters, multiple triggers on one workflow, manual-only workflows. |
7447
- | [Conditionals & matrix](./patterns/conditionals-matrix.md) | Conditional execution with rules, matrix builds (static + dynamic), and dynamic job generation. |
7448
- | [Integrations](./patterns/integrations.md) | Workflow chaining, generic webhooks, Stripe handlers, self-hosted git forges (Forgejo / Gitea / Gogs), plain GitHub repo webhooks (no GitHub App). |
7449
- | [Scheduling & events](./patterns/scheduling-and-events.md) | Nightly cron, workflow-complete-triggered deploys, custom event chaining. |
7450
- | [Pattern reference](./patterns/reference.md) | Step context, the examples repository, and GitHub check run output -- cross-cutting reference shared by every pattern above. |
7938
+ | [Basic CI](https://docs.kici.dev/user/patterns/basic/) | Single-job CI, PR-only / push-only filters, multiple triggers on one workflow, manual-only workflows. |
7939
+ | [Conditionals & matrix](https://docs.kici.dev/user/patterns/conditionals-matrix/) | Conditional execution with rules, matrix builds (static + dynamic), and dynamic job generation. |
7940
+ | [Integrations](https://docs.kici.dev/user/patterns/integrations/) | Workflow chaining, generic webhooks, Stripe handlers, self-hosted git forges (Forgejo / Gitea / Gogs), plain GitHub repo webhooks (no GitHub App). |
7941
+ | [Scheduling & events](https://docs.kici.dev/user/patterns/scheduling-and-events/) | Nightly cron, workflow-complete-triggered deploys, custom event chaining. |
7942
+ | [Pattern reference](https://docs.kici.dev/user/patterns/reference/) | Step context, the examples repository, and GitHub check run output -- cross-cutting reference shared by every pattern above. |
7451
7943
 
7452
7944
  ## See also
7453
7945
 
7454
- - [Event system](events.md) -- event model concepts, registration model, circuit breaker
7455
- - [SDK reference](sdk-reference.md) -- complete API reference for all functions used in these patterns
7456
- - [CLI reference](cli-reference.md) -- how to compile and test these workflows locally
7457
- - [Getting started](getting-started.md) -- installation and first workflow setup
7458
- - [Job execution lifecycle](../architecture/execution/job-execution.md) -- how agents execute the jobs defined in these patterns
7459
- - [GitHub checks architecture](../architecture/webhooks/github-checks.md) -- deep dive into the check run system
7946
+ - [Event system](https://docs.kici.dev/user/events/) -- event model concepts, registration model, circuit breaker
7947
+ - [SDK reference](https://docs.kici.dev/user/sdk-reference/) -- complete API reference for all functions used in these patterns
7948
+ - [CLI reference](https://docs.kici.dev/user/cli-reference/) -- how to compile and test these workflows locally
7949
+ - [Getting started](https://docs.kici.dev/user/getting-started/) -- installation and first workflow setup
7950
+ - [Job execution lifecycle](https://docs.kici.dev/architecture/execution/job-execution/) -- how agents execute the jobs defined in these patterns
7951
+ - [GitHub checks architecture](https://docs.kici.dev/architecture/webhooks/github-checks/) -- deep dive into the check run system
7460
7952
 
7461
7953
  ---
7462
7954
 
@@ -7476,7 +7968,7 @@ your organizations, roles, and API keys stay attached to the same identity.
7476
7968
  Sign-in methods and passwords are managed in your **account console**, provided
7477
7969
  by the identity provider that handles single sign-on for KiCI. The dashboard's
7478
7970
  **Linked accounts** page does not control how you sign in — see
7479
- [Linked accounts vs sign-in methods](#linked-accounts-vs-sign-in-methods) below.
7971
+ [Linked accounts vs sign-in methods](https://docs.kici.dev/user/account-and-login/#linked-accounts-vs-sign-in-methods) below.
7480
7972
 
7481
7973
  You can open the account console from the dashboard: go to your personal
7482
7974
  settings, open **Linked accounts**, and use the **Account console** link.
@@ -7520,13 +8012,15 @@ Source: https://docs.kici.dev/user/approvals/
7520
8012
 
7521
8013
  An **approval gate** pauses execution until an authorized person approves it. Execution resumes from exactly where it paused; a rejection (or an expired hold) fails the run.
7522
8014
 
7523
- You declare a gate in your workflow with `requireApproval`. It is available at three levels of granularity:
8015
+ You declare a gate in your workflow with `approval`. It is available at three levels of granularity:
7524
8016
 
7525
8017
  - **Step** — pause mid-job, before a specific step runs. The agent holds the live workspace (with all prior-step state intact) for the duration of the wait.
7526
8018
  - **Job** — hold the job before any of its steps run.
7527
8019
  - **Workflow** — hold the whole run before any job is dispatched.
7528
8020
 
7529
- Approvers are named as **teams** and **users**. A team is an operator-defined group of org members; your workflow code may name a team but can never change its membership, which is what makes a team clause a real gate rather than a suggestion. See [Approval gates (operator guide)](../operator/approvals.md) for how operators define teams, the approval queue, and expiry; see [the architecture overview](../architecture/approvals.md) for how a hold is evaluated and resumed.
8021
+ A step-level gate can also fire **only when a check/apply step finds drift** Terraform's plan→apply, per step. See [Drift gates](https://docs.kici.dev/user/approvals/#drift-gates-whendrift) below.
8022
+
8023
+ Approvers are named as **teams** and **users**. A team is an operator-defined group of org members; your workflow code may name a team but can never change its membership, which is what makes a team clause a real gate rather than a suggestion. See [Approval gates (operator guide)](https://docs.kici.dev/operator/approvals/) for how operators define teams, the approval queue, and expiry; see [the architecture overview](https://docs.kici.dev/architecture/approvals/) for how a hold is evaluated and resumed.
7530
8024
 
7531
8025
  ## Quick start
7532
8026
 
@@ -7540,37 +8034,37 @@ export default workflow('deploy', {
7540
8034
  jobs: [
7541
8035
  job('deploy-production', {
7542
8036
  runsOn: 'default',
7543
- requireApproval: [{ team: 'leads' }],
8037
+ approval: [{ team: 'leads' }],
7544
8038
  steps: [step('deploy', async (ctx) => ctx.$`deploy --prod`)],
7545
8039
  }),
7546
8040
  ],
7547
8041
  });
7548
8042
  ```
7549
8043
 
7550
- When the run reaches this job, it is held instead of dispatched. The held run appears in the dashboard approval queue and can be released from there or with the [`kici approve`](#approving-from-the-cli) command. Once a member of `leads` approves, the job dispatches normally.
8044
+ When the run reaches this job, it is held instead of dispatched. The held run appears in the dashboard approval queue and can be released from there or with the [`kici approve`](https://docs.kici.dev/user/approvals/#approving-from-the-cli) command. Once a member of `leads` approves, the job dispatches normally.
7551
8045
 
7552
- ## The `requireApproval` field
8046
+ ## The `approval` field
7553
8047
 
7554
- `requireApproval` accepts three forms.
8048
+ `approval` accepts three forms.
7555
8049
 
7556
8050
  ### Shorthand: `true`
7557
8051
 
7558
8052
  ```typescript
7559
8053
  job('deploy', {
7560
8054
  runsOn: 'default',
7561
- requireApproval: true,
8055
+ approval: true,
7562
8056
  steps: [
7563
8057
  /* ... */
7564
8058
  ],
7565
8059
  });
7566
8060
  ```
7567
8061
 
7568
- `requireApproval: true` holds the element until **any** org member who can act on approvals signs off — anyone with the `environments:write` or `ci_trust:write` permission. Use it when you want a manual gate without restricting who may release it.
8062
+ `approval: true` holds the element until **any** org member who can act on approvals signs off — anyone with the `environments:write` or `ci_trust:write` permission. Use it when you want a manual gate without restricting who may release it.
7569
8063
 
7570
8064
  ### Approver list (AND)
7571
8065
 
7572
8066
  ```typescript
7573
- requireApproval: [{ team: 'leads' }, { user: 'cto' }],
8067
+ approval: [{ team: 'leads' }, { user: 'cto' }],
7574
8068
  ```
7575
8069
 
7576
8070
  A list of approver clauses is an **AND** list: every clause must be satisfied before the element is released.
@@ -7582,27 +8076,29 @@ A single approver may satisfy more than one clause. If `cto` is also a member of
7582
8076
 
7583
8077
  There is no OR or nested logic — clauses are always a flat AND list.
7584
8078
 
7585
- ### Object form: reason and timeout
8079
+ ### Object form: when, reason, and timeout
7586
8080
 
7587
8081
  ```typescript
7588
- requireApproval: {
8082
+ approval: {
8083
+ when: 'always',
7589
8084
  approvers: [{ team: 'security' }, { team: 'leads' }],
7590
8085
  reason: 'Production deploy requires security + leads sign-off',
7591
8086
  timeout: 7200, // seconds
7592
8087
  },
7593
8088
  ```
7594
8089
 
7595
- | Field | Type | Description |
7596
- | ----------- | ------------------ | --------------------------------------------------------------------------------------------------- |
7597
- | `approvers` | `ApproverClause[]` | The AND list of `{ team }` / `{ user }` clauses. An empty list means "any approval-capable member". |
7598
- | `reason` | `string` | A human-readable label shown in the dashboard queue and the held-for-approval status check. |
7599
- | `timeout` | `number` | Per-gate expiry in **seconds**, overriding the org default. On expiry the element is rejected. |
8090
+ | Field | Type | Description |
8091
+ | ----------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
8092
+ | `when` | `'always' \| 'drift'` | When the gate fires. `'always'` (default) gates before the element; `'drift'` gates a check/apply step only when it finds drift. See [Drift gates](https://docs.kici.dev/user/approvals/#drift-gates-whendrift). |
8093
+ | `approvers` | `ApproverClause[]` | The AND list of `{ team }` / `{ user }` clauses. An empty list means "any approval-capable member". |
8094
+ | `reason` | `string` | A human-readable label shown in the dashboard queue and the held-for-approval status check. |
8095
+ | `timeout` | `number` | Per-gate expiry in **seconds**, overriding the org default. On expiry the element is rejected. |
7600
8096
 
7601
- When `timeout` is omitted, the gate uses the org's default approval expiry (set by the operator). On expiry, the held element is rejected and the run fails — see [expiry](../operator/approvals.md#expiry).
8097
+ When `timeout` is omitted, the gate uses the org's default approval expiry (set by the operator). On expiry, the held element is rejected and the run fails — see [expiry](https://docs.kici.dev/operator/approvals/#expiry).
7602
8098
 
7603
8099
  ## Granularity
7604
8100
 
7605
- The same `requireApproval` field is accepted on a workflow, a job, and a step.
8101
+ The same `approval` field is accepted on a workflow, a job, and a step.
7606
8102
 
7607
8103
  ### Workflow-level
7608
8104
 
@@ -7611,7 +8107,7 @@ A workflow-level gate holds the entire run before any job is dispatched:
7611
8107
  ```typescript
7612
8108
  export default workflow('release', {
7613
8109
  on: [push({ branches: ['main'] })],
7614
- requireApproval: [{ team: 'release-managers' }],
8110
+ approval: [{ team: 'release-managers' }],
7615
8111
  jobs: [buildJob, publishJob],
7616
8112
  });
7617
8113
  ```
@@ -7623,7 +8119,7 @@ A job-level gate holds just that job; other jobs in the run proceed normally:
7623
8119
  ```typescript
7624
8120
  job('publish', {
7625
8121
  runsOn: 'default',
7626
- requireApproval: [{ team: 'leads' }],
8122
+ approval: [{ team: 'leads' }],
7627
8123
  steps: [
7628
8124
  /* ... */
7629
8125
  ],
@@ -7640,7 +8136,7 @@ job('migrate-and-deploy', {
7640
8136
  steps: [
7641
8137
  step('build-plan', async (ctx) => ctx.$`./gen-migration-plan.sh`),
7642
8138
  step('apply-migration', {
7643
- requireApproval: [{ team: 'dba' }],
8139
+ approval: [{ team: 'dba' }],
7644
8140
  run: async (ctx) => ctx.$`./apply-migration.sh`,
7645
8141
  }),
7646
8142
  step('deploy', async (ctx) => ctx.$`deploy --prod`),
@@ -7650,11 +8146,41 @@ job('migrate-and-deploy', {
7650
8146
 
7651
8147
  Here `build-plan` runs, then the job pauses for a `dba` approval. On approval, `apply-migration` runs against the exact workspace `build-plan` produced, followed by `deploy`. A rejection or expiry fails the job.
7652
8148
 
7653
- Because a step-level hold keeps an agent and its workspace occupied for the whole human wait, prefer job- or workflow-level gates when you do not need prior-step state, and keep step-level timeouts short. See the [operator note on agent occupancy](../operator/approvals.md#agent-occupancy-during-step-level-holds).
8149
+ Because a step-level hold keeps an agent and its workspace occupied for the whole human wait, prefer job- or workflow-level gates when you do not need prior-step state, and keep step-level timeouts short. See the [operator note on agent occupancy](https://docs.kici.dev/operator/approvals/#agent-occupancy-during-step-level-holds).
8150
+
8151
+ ## Drift gates (`when: 'drift'`)
8152
+
8153
+ A `when: 'drift'` gate is **step-scope only** and requires a [check/apply step](https://docs.kici.dev/user/idempotent-steps/). Instead of pausing unconditionally, it fires **between the step's `check` and `run`, only when `check` finds drift in apply mode** — exactly Terraform's plan→apply, scoped to one step. When the step is already in sync (no drift), nothing pauses and the step skips.
8154
+
8155
+ When the gate fires, the held run carries the **computed drift** as a payload: the rendering your `summarize(drift)` produced (the per-file diff, the commands that would run), plus the structured drift. The dashboard approval queue and the [CLI](https://docs.kici.dev/user/approvals/#approving-from-the-cli) show the actual diff the operator is approving — not a static reason string.
8156
+
8157
+ ```typescript
8158
+ job('patch-prod', {
8159
+ runsOn: 'default',
8160
+ steps: [
8161
+ step('apply-nginx-config', {
8162
+ check: async (ctx) => ((await inSync(ctx)) ? null : { want: DESIRED_CONF }),
8163
+ summarize: (drift) => `would rewrite nginx.conf (${drift.want.length} bytes)`,
8164
+ run: async (ctx, drift) => {
8165
+ await writeConfig(drift.want);
8166
+ },
8167
+ approval: { when: 'drift', approvers: [{ team: 'ops' }], reason: 'prod patch' },
8168
+ }),
8169
+ ],
8170
+ });
8171
+ ```
8172
+
8173
+ Behavior:
8174
+
8175
+ - The gate fires **only in apply mode**. In `--check` mode nothing applies, so nothing gates — drift is just reported.
8176
+ - **Approve** → the step's `run(ctx, drift)` applies the change.
8177
+ - **Reject** → fail-stop: the step fails, the job fails, and the `needs:` skip-cascade aborts everything downstream.
8178
+
8179
+ A `when: 'drift'` gate on a step without a `check` facet, or at job/workflow scope, is a compile error.
7654
8180
 
7655
8181
  ## Mandatory vs. explicit gates
7656
8182
 
7657
- `requireApproval` is the **explicit** gate — a deliberate "pause for a human here" written by the workflow author. It composes with the **mandatory** gate an operator can attach to a protected environment via required reviewers (see [Environments](environments.md#required-reviewers)). When both apply to the same job, all clauses from both sources must be satisfied before the job is released. The two funnel into one held-element mechanism, so the dashboard queue and `kici approve` work the same way regardless of which source held the element.
8183
+ `approval` is the **explicit** gate — a deliberate "pause for a human here" written by the workflow author. It composes with the **mandatory** gate an operator can attach to a protected environment via required reviewers (see [Environments](https://docs.kici.dev/user/environments/#required-reviewers)). When both apply to the same job, all clauses from both sources must be satisfied before the job is released. The two funnel into one held-element mechanism, so the dashboard queue and `kici approve` work the same way regardless of which source held the element.
7658
8184
 
7659
8185
  ## Approving from the CLI
7660
8186
 
@@ -7674,15 +8200,28 @@ kici approve <run-id> --job migrate-and-deploy --step apply-migration
7674
8200
  kici reject <run-id> --job deploy-production --reason "Wrong release branch"
7675
8201
  ```
7676
8202
 
7677
- You must be eligible for at least one unsatisfied clause — being a member of a named team or being a named user. The orchestrator verifies eligibility against the operator-defined teams, so naming a team in your workflow can never let an ineligible person release the gate. The command reports whether the element was released, how many clauses remain, or that it was rejected. See [`kici approve`](cli-reference.md#kici-approve) for the full command reference.
8203
+ You must be eligible for at least one unsatisfied clause — being a member of a named team or being a named user. The orchestrator verifies eligibility against the operator-defined teams, so naming a team in your workflow can never let an ineligible person release the gate. The command reports whether the element was released, how many clauses remain, or that it was rejected. See [`kici approve`](https://docs.kici.dev/user/cli-reference/#kici-approve) for the full command reference.
7678
8204
 
7679
- You can also approve from the dashboard approval queue. See [Dashboard](dashboard/environments-and-secrets.md#approval-queue).
8205
+ ### Inline approval and `--approve-all` in `kici run remote`
8206
+
8207
+ When you trigger a run with `kici run remote` and it pauses on a gate, the CLI prints the gate (and, for a drift gate, the computed drift) and — in an interactive terminal — prompts you to approve or reject right there, without leaving the watch. In a non-interactive shell it prints how to approve out of band and keeps watching.
8208
+
8209
+ To auto-approve **every gate of the run you just dispatched**, pass `--approve-all` (alias `--yes`):
8210
+
8211
+ ```bash
8212
+ kici run remote deploy-prod --approve-all
8213
+ ```
8214
+
8215
+ `--approve-all` is **run-scoped** — it only auto-approves holds belonging to the run this invocation dispatched; there is no fleet-wide or account-wide auto-approve. Eligibility is still enforced per hold: if you are not eligible for a gate, that gate still blocks. Each auto-approved gate prints its payload before resolving and is recorded distinctly in the audit trail (`held_run.auto_approve`).
8216
+
8217
+ You can also approve from the dashboard approval queue. See [Dashboard](https://docs.kici.dev/user/dashboard/environments-and-secrets/#approval-queue).
7680
8218
 
7681
8219
  ## See also
7682
8220
 
7683
- - [Environments](environments.md) — operator-required reviewers on protected environments.
7684
- - [Approval gates (operator guide)](../operator/approvals.md) — teams, the approval queue, expiry, and self-approval.
7685
- - [Approval gates (architecture)](../architecture/approvals.md) — the unified hold model and the step-level round-trip.
8221
+ - [Idempotent steps](https://docs.kici.dev/user/idempotent-steps/) — the check/apply step facet that drift gates build on.
8222
+ - [Environments](https://docs.kici.dev/user/environments/) — operator-required reviewers on protected environments.
8223
+ - [Approval gates (operator guide)](https://docs.kici.dev/operator/approvals/) — teams, the approval queue, expiry, and self-approval.
8224
+ - [Approval gates (architecture)](https://docs.kici.dev/architecture/approvals/) — the unified hold model and the step-level round-trip.
7686
8225
 
7687
8226
  ---
7688
8227
 
@@ -7742,7 +8281,7 @@ group: () => 'deploy';
7742
8281
  group: (ctx) => `deploy-${ctx.event.targetBranch ?? 'default'}`;
7743
8282
  ```
7744
8283
 
7745
- The workflow-level group function is always evaluated **agent-side** at runtime -- the lock file records only that a group function exists (`hasGroup: true`), not the function itself. The agent loads the workflow source, calls the group function with `{ branch, event }`, and reports the evaluated key back to the orchestrator before step execution begins. This differs from job-level `concurrencyGroup` (see [Environments](environments.md#concurrency-groups)), where the compiler performs purity analysis and can inline pure functions for orchestrator-side evaluation.
8284
+ The workflow-level group function is always evaluated **agent-side** at runtime -- the lock file records only that a group function exists (`hasGroup: true`), not the function itself. The agent loads the workflow source, calls the group function with `{ branch, event }`, and reports the evaluated key back to the orchestrator before step execution begins. This differs from job-level `concurrencyGroup` (see [Environments](https://docs.kici.dev/user/environments/#concurrency-groups)), where the compiler performs purity analysis and can inline pure functions for orchestrator-side evaluation.
7746
8285
 
7747
8286
  ## cancelInProgress mode
7748
8287
 
@@ -7893,7 +8432,7 @@ Queued runs can be cancelled before they start executing. The cancel request rem
7893
8432
 
7894
8433
  ## Job-level concurrency groups
7895
8434
 
7896
- In addition to workflow-level concurrency, individual jobs can define their own concurrency group via the `concurrencyGroup` property. This controls concurrent execution at the job level rather than the workflow level. See [Environments — concurrency groups](environments.md#concurrency-groups) for details.
8435
+ In addition to workflow-level concurrency, individual jobs can define their own concurrency group via the `concurrencyGroup` property. This controls concurrent execution at the job level rather than the workflow level. See [Environments — concurrency groups](https://docs.kici.dev/user/environments/#concurrency-groups) for details.
7897
8436
 
7898
8437
  ## Local execution
7899
8438
 
@@ -7901,7 +8440,7 @@ In addition to workflow-level concurrency, individual jobs can define their own
7901
8440
 
7902
8441
  Coordination is local only. Running the same workflow on two different machines does not serialize across them — that requires the orchestrator. For full cross-host enforcement (queueing across agents, dashboard visibility, `max > 1`), use `kici run remote` against a deployed orchestrator.
7903
8442
 
7904
- Lock files live under `$XDG_RUNTIME_DIR/kici-local-locks/` on Linux, falling back to `os.tmpdir()/kici-local-locks-<uid>/`. A workflow whose `group` callback throws aborts the run with a clear error rather than running unprotected. See [`kici run local` — Concurrency enforcement](cli-reference.md#concurrency-enforcement) for the `KICI_LOCAL_LOCK_KILL_GRACE_MS` override and the diagnostic output emitted while contending on a busy lock.
8443
+ Lock files live under `$XDG_RUNTIME_DIR/kici-local-locks/` on Linux, falling back to `os.tmpdir()/kici-local-locks-<uid>/`. A workflow whose `group` callback throws aborts the run with a clear error rather than running unprotected. See [`kici run local` — Concurrency enforcement](https://docs.kici.dev/user/cli-reference/#concurrency-enforcement) for the `KICI_LOCAL_LOCK_KILL_GRACE_MS` override and the diagnostic output emitted while contending on a busy lock.
7905
8444
 
7906
8445
  The `kici run local --concurrency <n>` flag is a separate concept — it caps **job-level** parallelism within a single run (how many jobs from one workflow run at once), not cross-run serialization.
7907
8446
 
@@ -7921,14 +8460,14 @@ This guide is split across the following pages:
7921
8460
 
7922
8461
  | Page | Covers |
7923
8462
  | ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
7924
- | [Getting started](./dashboard/getting-started.md) | Onboarding checklist and your organizations list |
7925
- | [Navigation and layout](./dashboard/navigation.md) | Sidebar, mobile nav, theme, time display, shortcuts, error pages |
7926
- | [Runs and logs](./dashboard/runs.md) | Run list, run detail, the log viewer |
7927
- | [Settings](./dashboard/settings.md) | Members, roles, teams, keys, sources, billing, security, support access |
7928
- | [Workflows, diagnostics, and orchestrators](./dashboard/workflows-and-diagnostics.md) | Registered workflows, infra health, per-cluster views |
7929
- | [Environments, secrets, and approvals](./dashboard/environments-and-secrets.md) | Environments, secret scopes, approval queue |
7930
- | [Activity and DLQ](./dashboard/activity-and-dlq.md) | Forensic activity log and dead-letter queue |
7931
- | [Account](./dashboard/account.md) | Personal account settings |
8463
+ | [Getting started](https://docs.kici.dev/user/dashboard/getting-started/) | Onboarding checklist and your organizations list |
8464
+ | [Navigation and layout](https://docs.kici.dev/user/dashboard/navigation/) | Sidebar, mobile nav, theme, time display, shortcuts, error pages |
8465
+ | [Runs and logs](https://docs.kici.dev/user/dashboard/runs/) | Run list, run detail, the log viewer |
8466
+ | [Settings](https://docs.kici.dev/user/dashboard/settings/) | Members, roles, teams, keys, sources, billing, security, support access |
8467
+ | [Workflows, diagnostics, and orchestrators](https://docs.kici.dev/user/dashboard/workflows-and-diagnostics/) | Registered workflows, infra health, per-cluster views |
8468
+ | [Environments, secrets, and approvals](https://docs.kici.dev/user/dashboard/environments-and-secrets/) | Environments, secret scopes, approval queue |
8469
+ | [Activity and DLQ](https://docs.kici.dev/user/dashboard/activity-and-dlq/) | Forensic activity log and dead-letter queue |
8470
+ | [Account](https://docs.kici.dev/user/dashboard/account/) | Personal account settings |
7932
8471
 
7933
8472
  ---
7934
8473
 
@@ -8046,7 +8585,7 @@ env: async (event) => {
8046
8585
  - **Write pure functions whenever possible** to avoid the init-job delay. Most environment and env computations only need the event payload data.
8047
8586
  - **Check compiler warnings** -- the compiler tells you when a function is classified as impure and explains why.
8048
8587
  - **Runtime errors in inline expressions cause immediate job failure.** There is no fallback to the init-job path. If your pure function throws at runtime (e.g., accessing a property on `undefined`), the job fails immediately.
8049
- - **The event parameter is the normalized event envelope** — the same shape rules receive as `ctx.event`: `{ type, action, targetBranch, sourceBranch, changedFiles, payload, … }` (see the [event payload reference](./sdk/event-payloads.md) for the complete schema). Narrow on `event.type` (`'push'`, `'pull_request'`, `'tag'`, …) to branch per trigger kind. The raw provider webhook body is nested at `event.payload` (for GitHub pushes: `payload.ref`, `payload.after`, `payload.repository`, …).
8588
+ - **The event parameter is the normalized event envelope** — the same shape rules receive as `ctx.event`: `{ type, action, targetBranch, sourceBranch, changedFiles, payload, … }` (see the [event payload reference](https://docs.kici.dev/user/sdk/event-payloads/) for the complete schema). Narrow on `event.type` (`'push'`, `'pull_request'`, `'tag'`, …) to branch per trigger kind. The raw provider webhook body is nested at `event.payload` (for GitHub pushes: `payload.ref`, `payload.after`, `payload.repository`, …).
8050
8589
 
8051
8590
  ---
8052
8591
 
@@ -8213,7 +8752,7 @@ job('deploy-review', {
8213
8752
  });
8214
8753
  ```
8215
8754
 
8216
- A pure function like the one above (see [Dynamic values](dynamic-values.md)) is evaluated inline at dispatch with no init-job overhead. Dynamic environments that match a glob pattern (e.g., `review/*`) inherit the pattern's configuration, variables, and protection rules.
8755
+ A pure function like the one above (see [Dynamic values](https://docs.kici.dev/user/dynamic-values/)) is evaluated inline at dispatch with no init-job overhead. Dynamic environments that match a glob pattern (e.g., `review/*`) inherit the pattern's configuration, variables, and protection rules.
8217
8756
 
8218
8757
  ### Job-level environment variables
8219
8758
 
@@ -8237,7 +8776,7 @@ job('deploy', {
8237
8776
 
8238
8777
  ### Concurrency groups
8239
8778
 
8240
- Jobs can define their own concurrency groups to control concurrent execution within an environment. For workflow-level concurrency (which applies to all jobs in a workflow), see [Concurrency groups](concurrency.md).
8779
+ Jobs can define their own concurrency groups to control concurrent execution within an environment. For workflow-level concurrency (which applies to all jobs in a workflow), see [Concurrency groups](https://docs.kici.dev/user/concurrency/).
8241
8780
 
8242
8781
  Control concurrent deployments to the same environment:
8243
8782
 
@@ -8286,7 +8825,7 @@ When a job targets an environment, variables are merged in this order (last wins
8286
8825
  6. **Job env** -- from the `env` property in the SDK
8287
8826
  7. **`setEnv()` calls** -- runtime modifications within steps
8288
8827
 
8289
- > **Note:** Secrets are NOT part of the environment variable merge. They are delivered to the step context via IPC and accessed through `ctx.secrets`, not through `process.env`. See the [step context](#step-context) section above.
8828
+ > **Note:** Secrets are NOT part of the environment variable merge. They are delivered to the step context via IPC and accessed through `ctx.secrets`, not through `process.env`. See the [step context](https://docs.kici.dev/user/environments/#step-context) section above.
8290
8829
 
8291
8830
  ## Protection rules
8292
8831
 
@@ -8310,9 +8849,9 @@ Require manual approval before a job can proceed:
8310
8849
  Required reviewers: alice, bob
8311
8850
  ```
8312
8851
 
8313
- When reviewers are required, the job enters a "held" state. Reviewers can approve or reject via the dashboard, the [`kici approve`](cli-reference.md#kici-approve) command, or the API. Held runs expire after a configurable timeout.
8852
+ When reviewers are required, the job enters a "held" state. Reviewers can approve or reject via the dashboard, the [`kici approve`](https://docs.kici.dev/user/cli-reference/#kici-approve) command, or the API. Held runs expire after a configurable timeout.
8314
8853
 
8315
- This operator-set rule is the **mandatory** form of an approval gate. Workflow authors can also declare gates in code with `requireApproval` at step, job, or workflow level — see [Approval gates](approvals.md). Both forms use the same held-element mechanism and the same queue.
8854
+ This operator-set rule is the **mandatory** form of an approval gate. Workflow authors can also declare gates in code with `approval` at step, job, or workflow level — see [Approval gates](https://docs.kici.dev/user/approvals/). Both forms use the same held-element mechanism and the same queue.
8316
8855
 
8317
8856
  ### Wait timer
8318
8857
 
@@ -8347,7 +8886,7 @@ Trust tier is determined by the contributor's identity link and CI trust RBAC le
8347
8886
 
8348
8887
  The trust tier also affects which lock file is used for PR-triggered runs: trusted contributors use the PR head lock file, while known and unknown contributors use the base branch lock file. This prevents untrusted workflow modifications from affecting execution.
8349
8888
 
8350
- See the [CI security architecture docs](../architecture/security/ci-security.md) for the full trust resolution flow.
8889
+ See the [CI security architecture docs](https://docs.kici.dev/architecture/security/ci-security/) for the full trust resolution flow.
8351
8890
 
8352
8891
  ### Security approval queue
8353
8892
 
@@ -8381,7 +8920,7 @@ Navigate to **Settings > Environments** in the dashboard. Click **New environmen
8381
8920
  - **Fixed** -- applies to jobs that declare exactly this environment name, like `staging` or `production`
8382
8921
  - **Glob** -- applies to any environment name a job declares that matches the pattern, e.g. `review/*` matches a job with `environment: 'review/PR-123'`
8383
8922
 
8384
- The environments list shows each environment's type, whether test runs may use it (the `allowLocalExecution` flag -- see the [testing guide](./testing-guide.md)), and whether it is enabled.
8923
+ The environments list shows each environment's type, whether test runs may use it (the `allowLocalExecution` flag -- see the [testing guide](https://docs.kici.dev/user/testing-guide/)), and whether it is enabled.
8385
8924
 
8386
8925
  ### Environment detail page
8387
8926
 
@@ -8540,7 +9079,7 @@ genericWebhook({
8540
9079
 
8541
9080
  **Config options:** `source` (required), `events`, `match`, `not`, `auth`, `path`, `description`.
8542
9081
 
8543
- The `source` field MUST match the `--name` that an operator passed to `kici-admin source add generic --name <name>` when the source was created — that string is the source's identifier in the orchestrator. Generic webhook sources must be created by an operator before events can be received; see [Operator guide: event routing](../operator/event-routing.md) for setup instructions.
9082
+ The `source` field MUST match the `--name` that an operator passed to `kici-admin source add generic --name <name>` when the source was created — that string is the source's identifier in the orchestrator. Generic webhook sources must be created by an operator before events can be received; see [Operator guide: event routing](https://docs.kici.dev/operator/event-routing/) for setup instructions.
8544
9083
 
8545
9084
  ### Schedule events
8546
9085
 
@@ -8650,7 +9189,7 @@ This lookup is fast because the registration index is held in memory and refresh
8650
9189
 
8651
9190
  ### Cross-source webhook delivery
8652
9191
 
8653
- The catch-all `webhook()` trigger (see [SDK reference: webhook()](sdk/triggers.md#webhook)) participates in this same registration lookup, but with one twist: it fires for matching events arriving via **any** inbound webhook source in the same org, not just the source the workflow's repo is bound to. The orchestrator maintains a `(customerId, eventName)` index over webhook trigger registrations and consults it on every inbound generic webhook.
9192
+ The catch-all `webhook()` trigger (see [SDK reference: webhook()](https://docs.kici.dev/user/sdk/triggers/#webhook)) participates in this same registration lookup, but with one twist: it fires for matching events arriving via **any** inbound webhook source in the same org, not just the source the workflow's repo is bound to. The orchestrator maintains a `(customerId, eventName)` index over webhook trigger registrations and consults it on every inbound generic webhook.
8654
9193
 
8655
9194
  The lookup is structurally org-isolated — a generic webhook delivered to org A can never reach a workflow registered against org B, because foreign-org rows live in a different bucket of the index. When a webhook fires across sources, the runtime clone token, repo URL, and check-status posting all come from the **registration's** source bundle, not the inbound source. The inbound source contributes only the event payload.
8656
9195
 
@@ -8787,15 +9326,15 @@ export default workflow('post-deploy', {
8787
9326
  });
8788
9327
  ```
8789
9328
 
8790
- Custom events are delivered immediately when emitted (mid-workflow, not queued until workflow completion). See the [SDK reference: emitting events](sdk/validation-events.md#emitting-events) section for the full `ctx.emit()` API.
9329
+ Custom events are delivered immediately when emitted (mid-workflow, not queued until workflow completion). See the [SDK reference: emitting events](https://docs.kici.dev/user/sdk/validation-events/#emitting-events) section for the full `ctx.emit()` API.
8791
9330
 
8792
9331
  ## See also
8793
9332
 
8794
- - [SDK reference: event triggers](sdk/triggers.md#event-triggers) -- complete API signatures for all trigger builders
8795
- - [SDK reference: emitting events](sdk/validation-events.md#emitting-events) -- `ctx.emit()` and `defineEvent()` API
8796
- - [Workflow patterns: workflow chaining](patterns/integrations.md#workflow-chaining) -- examples of event-driven workflow chains
8797
- - [Operator guide: event routing](../operator/event-routing.md) -- configuring generic webhook sources, trust relationships, and event routing
8798
- - [Architecture: event system](../architecture/webhooks/event-system.md) -- internal event routing design, registration model, cluster synchronization
9333
+ - [SDK reference: event triggers](https://docs.kici.dev/user/sdk/triggers/#event-triggers) -- complete API signatures for all trigger builders
9334
+ - [SDK reference: emitting events](https://docs.kici.dev/user/sdk/validation-events/#emitting-events) -- `ctx.emit()` and `defineEvent()` API
9335
+ - [Workflow patterns: workflow chaining](https://docs.kici.dev/user/patterns/integrations/#workflow-chaining) -- examples of event-driven workflow chains
9336
+ - [Operator guide: event routing](https://docs.kici.dev/operator/event-routing/) -- configuring generic webhook sources, trust relationships, and event routing
9337
+ - [Architecture: event system](https://docs.kici.dev/architecture/webhooks/event-system/) -- internal event routing design, registration model, cluster synchronization
8799
9338
 
8800
9339
  ---
8801
9340
 
@@ -8915,9 +9454,9 @@ Non-push triggers work too — `pr()`, `tag()`, `comment()`, `release()`, `workf
8915
9454
 
8916
9455
  ## See also
8917
9456
 
8918
- - [Architecture — global workflows](../architecture/global-workflows.md) — dual-query dispatch flow, cross-provider auth, security model, lock-file schema.
8919
- - [Universal-git provider](providers/universal-git.md#global-workflows) — how global workflows interact with `generic:<orgId>:<sourceId>` routing keys.
8920
- - [SDK reference](sdk-reference.md) — the full set of triggers that accept `repos:`.
9457
+ - [Architecture — global workflows](https://docs.kici.dev/architecture/global-workflows/) — dual-query dispatch flow, cross-provider auth, security model, lock-file schema.
9458
+ - [Universal-git provider](https://docs.kici.dev/user/providers/universal-git/#global-workflows) — how global workflows interact with `generic:<orgId>:<sourceId>` routing keys.
9459
+ - [SDK reference](https://docs.kici.dev/user/sdk-reference/) — the full set of triggers that accept `repos:`.
8921
9460
 
8922
9461
  ---
8923
9462
 
@@ -8987,7 +9526,7 @@ const configureNginx = step('configure-nginx', {
8987
9526
  `summarize` is **required** whenever `check` is declared. `run` and `whenInSync`
8988
9527
  both produce the same output type — one output shape per step, whichever path
8989
9528
  runs. Every other step facet (`cache`, `rules`, `continueOnError`, `timeout`,
8990
- `requireApproval`, `onCancel`, `cleanup`, `outputs`) composes unchanged.
9529
+ `approval`, `onCancel`, `cleanup`, `outputs`) composes unchanged.
8991
9530
 
8992
9531
  A plain `step()` without `check` keeps its exact current behavior — the check
8993
9532
  facet is fully optional.
@@ -9044,9 +9583,9 @@ describing what would change. The rendering is read-only.
9044
9583
 
9045
9584
  ## See also
9046
9585
 
9047
- - [Idempotent SDK helpers](./sdk/idempotent.md) — the `idempotent()` / `idempotentStep()` convenience wrappers, which always apply on drift inside a single step (no run-level check mode).
9048
- - [Core SDK reference](./sdk/core.md) — the `step()`, `job()`, and `workflow()` factories the check facet extends.
9049
- - [Lock file and drift](./lock-file-and-drift.md) — how the lock file carries step capability flags.
9586
+ - [Idempotent SDK helpers](https://docs.kici.dev/user/sdk/idempotent/) — the `idempotent()` / `idempotentStep()` convenience wrappers (always apply on drift), plus `checkStep()`, the clean-shape sibling that respects the run-level check mode.
9587
+ - [Core SDK reference](https://docs.kici.dev/user/sdk/core/) — the `step()`, `job()`, and `workflow()` factories the check facet extends.
9588
+ - [Lock file and drift](https://docs.kici.dev/user/lock-file-and-drift/) — how the lock file carries step capability flags.
9050
9589
 
9051
9590
  ---
9052
9591
 
@@ -9099,7 +9638,7 @@ export default workflow('build', {
9099
9638
 
9100
9639
  Per-field rules:
9101
9640
 
9102
- - **`url`** — Must be HTTPS. HTTP is permitted only for `localhost` / `127.0.0.0/8` / `::1` / `*.local` hosts, or when an operator has flipped the org-level `allow_http_npm_registries` toggle (see [`kici-admin org-settings allow-http-npm`](/operator/kici-admin-cli#allow-http-npm--permit-non-https-private-npm-registries)).
9641
+ - **`url`** — Must be HTTPS. HTTP is permitted only for `localhost` / `127.0.0.0/8` / `::1` / `*.local` hosts, or when an operator has flipped the org-level `allow_http_npm_registries` toggle (see [`kici-admin org-settings allow-http-npm`](https://docs.kici.dev/operator/kici-admin-cli#allow-http-npm--permit-non-https-private-npm-registries)).
9103
9642
  - **`scope`** — Optional. When present, the registry serves only that scope (`@my-org`). When absent, this entry becomes the **default** registry — at most one entry may omit `scope`.
9104
9643
  - **`tokenSecret`** — Mandatory `<environment>:<secret-name>`. The orchestrator looks up the secret in the named environment via the per-environment secret resolver. The bare name **must not** contain a colon.
9105
9644
  - **`alwaysAuth`** — Defaults to `true`. Forces npm to send the token on every request (even GETs), which is what most managed-registry providers require.
@@ -9264,7 +9803,7 @@ registries: [
9264
9803
 
9265
9804
  ## Security model
9266
9805
 
9267
- - **Per-environment scoping.** Every `tokenSecret` and `installEnv` entry is qualified with an environment name. The orchestrator runs the same protection-rule pipeline (branch / trust / concurrency / reviewer / wait-timer) against each named environment **before** resolving any secret, so a workflow that wants a `production` token from a feature branch is rejected exactly like a job that tries to deploy to `production` from a feature branch. A reviewer-gated install environment **pauses** the whole workflow dispatch as a workflow-scoped held run instead of resolving the token — see [Reviewer-gated installs](#reviewer-gated-installs) below.
9806
+ - **Per-environment scoping.** Every `tokenSecret` and `installEnv` entry is qualified with an environment name. The orchestrator runs the same protection-rule pipeline (branch / trust / concurrency / reviewer / wait-timer) against each named environment **before** resolving any secret, so a workflow that wants a `production` token from a feature branch is rejected exactly like a job that tries to deploy to `production` from a feature branch. A reviewer-gated install environment **pauses** the whole workflow dispatch as a workflow-scoped held run instead of resolving the token — see [Reviewer-gated installs](https://docs.kici.dev/user/private-registries/#reviewer-gated-installs) below.
9268
9807
  - **Untrusted contributors get no tokens.** When a fork PR is dispatched and the contributor-trust resolution returns anything other than `trusted`, the orchestrator strips both `npmRegistries` and `installEnvSecrets` out of the dispatch. The install runs without auth and fails naturally on the first private dep — fork PRs cannot ever observe a registry token, even if a misconfigured environment lacks an explicit `requiredTrustTier`.
9269
9808
  - **Lifecycle scripts disabled.** Whenever a private registry is in scope, the agent runs the install with `--ignore-scripts` (npm or pnpm alike). A malicious `preinstall` / `postinstall` hook in committed `package.json` cannot read the synthesized token env vars, even though they exist in the install subprocess. For a pnpm workspace, the agent builds your in-repo dependency closure as a separate step **after** the install's auth is torn down, so build scripts never see the tokens either.
9270
9809
  - **Stderr is redacted.** If the install fails, the agent masks every token literal out of the surfaced stderr / stdout chunks before logging.
@@ -9300,9 +9839,9 @@ The dashboard JSON lives at `infra/terraform/modules/grafana/dashboards/install-
9300
9839
 
9301
9840
  ## See also
9302
9841
 
9303
- - [Secrets](secrets.md) — how to seed the `<environment>:<secret-name>` values referenced by `tokenSecret` / `installEnv`.
9304
- - [Environments](environments.md) — protection rules (`branch_restrictions`, `requires_review`, `minimum_trust`) that the install gate inherits.
9305
- - [Operator: `kici-admin org-settings`](/operator/kici-admin-cli#org-settings----org-level-security-policy) — the `allow_http_npm_registries` toggle and other org-scoped knobs.
9842
+ - [Secrets](https://docs.kici.dev/user/secrets/) — how to seed the `<environment>:<secret-name>` values referenced by `tokenSecret` / `installEnv`.
9843
+ - [Environments](https://docs.kici.dev/user/environments/) — protection rules (`branch_restrictions`, `requires_review`, `minimum_trust`) that the install gate inherits.
9844
+ - [Operator: `kici-admin org-settings`](https://docs.kici.dev/operator/kici-admin-cli#org-settings----org-level-security-policy) — the `allow_http_npm_registries` toggle and other org-scoped knobs.
9306
9845
 
9307
9846
  ---
9308
9847
 
@@ -9461,7 +10000,7 @@ kici verify-attestation ./dist/app.tgz \
9461
10000
  --trust-root ./kici-trust-root.json
9462
10001
  ```
9463
10002
 
9464
- The full flag reference is in the [CLI reference](./cli-reference.md#kici-verify-attestation).
10003
+ The full flag reference is in the [CLI reference](https://docs.kici.dev/user/cli-reference/#kici-verify-attestation).
9465
10004
 
9466
10005
  ## Viewing attestations in the dashboard
9467
10006
 
@@ -9491,9 +10030,9 @@ attestations shows an empty state.
9491
10030
 
9492
10031
  ## See also
9493
10032
 
9494
- - [SDK runtime reference](./sdk/runtime.md) — the `ctx.attestProvenance` and
10033
+ - [SDK runtime reference](https://docs.kici.dev/user/sdk/runtime/) — the `ctx.attestProvenance` and
9495
10034
  `ctx.kici.oidc.token` step APIs in full.
9496
- - [CLI reference](./cli-reference.md#kici-verify-attestation) — every
10035
+ - [CLI reference](https://docs.kici.dev/user/cli-reference/#kici-verify-attestation) — every
9497
10036
  `kici verify-attestation` flag and exit code.
9498
10037
 
9499
10038
  ---
@@ -9506,7 +10045,7 @@ KiCI provides an explicit secrets API that gives workflow steps controlled acces
9506
10045
 
9507
10046
  ## Overview
9508
10047
 
9509
- Secrets are managed per-environment in the orchestrator (see [operator docs](/operator/orchestrator/configuration) for setup). When a job runs with an `environment` binding, the agent receives the secret keys available for that environment but does **not** inject their values into the step's process environment. Instead, steps access secrets through the `ctx.secrets` API.
10048
+ Secrets are managed per-environment in the orchestrator (see [operator docs](https://docs.kici.dev/operator/orchestrator/configuration) for setup). When a job runs with an `environment` binding, the agent receives the secret keys available for that environment but does **not** inject their values into the step's process environment. Instead, steps access secrets through the `ctx.secrets` API.
9510
10049
 
9511
10050
  This design prevents accidental secret leakage through child processes, log output, or error messages. Only secrets you explicitly request are loaded into memory.
9512
10051
 
@@ -9525,7 +10064,7 @@ Use whichever fits the workflow — most small teams stay on the dashboard; ops
9525
10064
 
9526
10065
  ### When the operator has disabled dashboard writes
9527
10066
 
9528
- The orchestrator operator can flip `secrets.set` (and `variables.set`) to **CLI-only** as part of the [dashboard-write policy](/operator/security/dashboard-write-policy). When that flip is on:
10067
+ The orchestrator operator can flip `secrets.set` (and `variables.set`) to **CLI-only** as part of the [dashboard-write policy](https://docs.kici.dev/operator/security/dashboard-write-policy). When that flip is on:
9529
10068
 
9530
10069
  - The dashboard's "Add secret" / "Edit value" controls render with a lock icon. Clicking them shows a tooltip with the exact `kici-admin secret set` invocation needed.
9531
10070
  - The dashboard's secrets page still lists secret **names**, scopes, and bindings — only the value-entry path moves to the CLI.
@@ -9562,7 +10101,7 @@ Two cross-cutting flags help every mode:
9562
10101
 
9563
10102
  `kici-admin variable set` uses the same flags for non-encrypted variables, plus `--locked` to mark a variable as immutable from subsequent dashboard writes.
9564
10103
 
9565
- A full reference of input modes — including the default-mode resolution rules and the security trade-offs of each — lives in [Dashboard-write policy → CLI input modes](/operator/security/dashboard-write-policy#cli-input-modes-for-the-plaintext-path).
10104
+ A full reference of input modes — including the default-mode resolution rules and the security trade-offs of each — lives in [Dashboard-write policy → CLI input modes](https://docs.kici.dev/operator/security/dashboard-write-policy#cli-input-modes-for-the-plaintext-path).
9566
10105
 
9567
10106
  ## Accessing secrets
9568
10107
 
@@ -9720,7 +10259,7 @@ export default workflow('deploy', {
9720
10259
 
9721
10260
  ### Injecting decrypted sops values into the environment
9722
10261
 
9723
- KiCI does **not** scan your repository for `*.enc.yaml` files and auto-decrypt them into the environment at job init — nothing in a job runs `sops` on your behalf, and resolved secrets are never auto-injected as environment variables (see [Security notes](./sdk/runtime.md#security-notes)). Decryption is always something your workflow does explicitly: provision the age (or other) decryption key as a KiCI secret, expose it for the step, run `sops -d`, and decide what to do with the output.
10262
+ KiCI does **not** scan your repository for `*.enc.yaml` files and auto-decrypt them into the environment at job init — nothing in a job runs `sops` on your behalf, and resolved secrets are never auto-injected as environment variables (see [Security notes](https://docs.kici.dev/user/sdk/runtime/#security-notes)). Decryption is always something your workflow does explicitly: provision the age (or other) decryption key as a KiCI secret, expose it for the step, run `sops -d`, and decide what to do with the output.
9724
10263
 
9725
10264
  When you want the decrypted values available as environment variables — not just written to a file — decrypt early and export the values through `$KICI_ENV` (or `ctx.setEnv`). Anything appended to `$KICI_ENV` becomes an environment variable for every later step in the same job, so a single decrypt step can populate the environment for the whole job:
9726
10265
 
@@ -9753,9 +10292,9 @@ export default workflow('deploy', {
9753
10292
  });
9754
10293
  ```
9755
10294
 
9756
- Decrypted values exported this way follow the same rules as any other `$KICI_ENV` / `ctx.setEnv` export: last-write-wins on a repeated key, and a key that collides with an operator-injected secret is ignored (the operator value wins). See [Exporting env from shell commands](./sdk/runtime.md#exporting-env-from-shell-commands-kici_env--kici_path) for the full `$KICI_ENV` contract.
10295
+ Decrypted values exported this way follow the same rules as any other `$KICI_ENV` / `ctx.setEnv` export: last-write-wins on a repeated key, and a key that collides with an operator-injected secret is ignored (the operator value wins). See [Exporting env from shell commands](https://docs.kici.dev/user/sdk/runtime/#exporting-env-from-shell-commands-kici_env--kici_path) for the full `$KICI_ENV` contract.
9757
10296
 
9758
- If you only need the decrypted material as a file on disk (the common `kubectl` / `gcloud` case), skip the env hop and redirect to a file instead — see the [canonical sops example](#canonical-sops-example) above.
10297
+ If you only need the decrypted material as a file on disk (the common `kubectl` / `gcloud` case), skip the env hop and redirect to a file instead — see the [canonical sops example](https://docs.kici.dev/user/secrets/#canonical-sops-example) above.
9759
10298
 
9760
10299
  ## API reference
9761
10300
 
@@ -9797,7 +10336,7 @@ Note that `get()` is async -- you must `await` the result.
9797
10336
 
9798
10337
  When you run `kici types`, the compiler generates a `.kici/secrets.d.ts` file that provides type-safe autocompletion for your secret keys. The generated types augment the `StepSecrets` interface so that `ctx.secrets.get('...')` and `ctx.secrets.has('...')` offer suggestions for known keys.
9799
10338
 
9800
- See [CLI reference](/user/cli) for the `kici types` command.
10339
+ See [CLI reference](https://docs.kici.dev/user/cli) for the `kici types` command.
9801
10340
 
9802
10341
  ---
9803
10342
 
@@ -9812,11 +10351,11 @@ The **GitHub App** is KiCI's flagship source. A single App:
9812
10351
  1. receives `push`, `pull_request`, and related events from every repo it's installed on,
9813
10352
  2. clones repos with a short-lived installation token (no deploy key to manage),
9814
10353
  3. posts workflow / job / step Check runs back to the pull request (see
9815
- [GitHub checks architecture](../../architecture/webhooks/github-checks.md)).
10354
+ [GitHub checks architecture](https://docs.kici.dev/architecture/webhooks/github-checks/)).
9816
10355
 
9817
10356
  You don't need an App for every scenario — if you only care about `push`
9818
10357
  events, don't want to install an App, or are using a non-GitHub forge,
9819
- use the [universal-git provider](universal-git.md) instead.
10358
+ use the [universal-git provider](https://docs.kici.dev/user/providers/universal-git/) instead.
9820
10359
 
9821
10360
  ## GitHub App vs. `github-repo` preset
9822
10361
 
@@ -9886,11 +10425,12 @@ GitHub App "my-org" is live.
9886
10425
 
9887
10426
  **Flags:**
9888
10427
 
9889
- | Flag | Effect |
9890
- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
9891
- | `--name <name>` | The App name on GitHub (required). |
9892
- | `--github-org <slug>` | Create the App under a GitHub organization instead of your personal account. |
9893
- | `--no-browser` | Headless mode: the CLI prints a `kici.dev` URL to open, then reads the setup code you paste back. The page is pure client-side it only displays the short-lived code, which is useless once the CLI exchanges it. |
10428
+ | Flag | Effect |
10429
+ | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
10430
+ | `--name <name>` | The App name you _request_ on GitHub (required). GitHub assigns the final name + slug; the stored, displayed name always comes from GitHub (see [Display name and slug](https://docs.kici.dev/user/providers/github/#display-name-and-slug)). |
10431
+ | `--github-org <slug>` | Create the App under a GitHub organization instead of your personal account. |
10432
+ | `--webhook-url <url>` | **Advanced / self-hosted.** Bake this `https://` URL into the App's webhook verbatim and skip the platform-mode webhook-URL resolution (so it works even where the auto-resolved KiCI Platform URL is unavailable). See [Self-hosted webhook URL](https://docs.kici.dev/user/providers/github/#self-hosted-webhook-url-override). |
10433
+ | `--no-browser` | Headless mode: the CLI prints a `kici.dev` URL to open, then reads the setup code you paste back. The page is pure client-side — it only displays the short-lived code, which is useless once the CLI exchanges it. |
9894
10434
 
9895
10435
  The manifest flow always creates a **new** App on GitHub. If a source for
9896
10436
  that App id already exists on the orchestrator, the command refuses — use
@@ -9952,7 +10492,7 @@ the manifest flow — follow these steps.
9952
10492
 
9953
10493
  The first four rows cover the core flow (clone, trigger matching,
9954
10494
  Check runs). The **Organization -> Members** row is only relevant if
9955
- you use [CI trust tiers](../../architecture/security/ci-security.md)
10495
+ you use [CI trust tiers](https://docs.kici.dev/architecture/security/ci-security/)
9956
10496
  on an org-level install — see the event note below.
9957
10497
 
9958
10498
  6. **Subscribe to events.** At minimum: `push`, `pull_request`,
@@ -9964,7 +10504,7 @@ the manifest flow — follow these steps.
9964
10504
  KiCI caches each pull-request author's repository access level (used
9965
10505
  to decide whether workflow changes take effect immediately or are
9966
10506
  held for approval — see
9967
- [CI security](../../architecture/security/ci-security.md)). These
10507
+ [CI security](https://docs.kici.dev/architecture/security/ci-security/)). These
9968
10508
  events let the orchestrator drop stale cache entries the moment a
9969
10509
  contributor's access changes. They are not required for correctness:
9970
10510
  without them the cache simply ages out on its own 15-minute TTL, so a
@@ -10035,7 +10575,7 @@ kici-admin source get-webhook-secret github:12345 # Fetch the secret (for debug
10035
10575
  ```
10036
10576
 
10037
10577
  For the full CLI reference see the `source` section of the
10038
- [kici-admin CLI reference](../../operator/orchestrator/kici-admin-cli.md).
10578
+ [kici-admin CLI reference](https://docs.kici.dev/operator/orchestrator/kici-admin-cli/).
10039
10579
 
10040
10580
  ## Routing keys
10041
10581
 
@@ -10054,6 +10594,53 @@ its own source record and the orchestrator looks up the right one by
10054
10594
  combining the URL's `<orgId>` with the App ID from the
10055
10595
  `X-GitHub-Hook-Installation-Target-ID` header.
10056
10596
 
10597
+ ## Display name and slug
10598
+
10599
+ For a GitHub App source, **GitHub is the source of truth for the displayed
10600
+ name**. `--name` is only the name you _request_ when the App is created;
10601
+ GitHub assigns the final display name and a URL-safe **slug**
10602
+ (`my-org` → `my-org-1` if the name was taken). KiCI captures both at creation
10603
+ and shows them in the dashboard **Sources** tab — the display name prominently,
10604
+ with the slug as dimmed secondary text.
10605
+
10606
+ If you later rename the App in GitHub's UI, KiCI keeps the displayed name in
10607
+ sync two ways:
10608
+
10609
+ - **Automatically**, on a daily schedule. The orchestrator re-reads each GitHub
10610
+ source's name + slug from GitHub and updates the dashboard if they changed.
10611
+ The interval is configurable via `KICI_GITHUB_APP_NAME_REFRESH_INTERVAL_MS`
10612
+ (default 24h).
10613
+ - **On demand**, with `source refresh`:
10614
+
10615
+ ```bash
10616
+ kici-admin source refresh github:<appId> # one source
10617
+ kici-admin source refresh --all # every GitHub source
10618
+ ```
10619
+
10620
+ It prints `old → new` for any name or slug that changed, and is a no-op when
10621
+ GitHub already matches what KiCI has stored. Non-GitHub routing keys are
10622
+ rejected — name/slug sync applies only to GitHub App sources.
10623
+
10624
+ ## Self-hosted webhook URL override
10625
+
10626
+ By default the manifest flow bakes the KiCI Platform webhook endpoint
10627
+ (`https://<platform-host>/webhook/<orgId>/github`) into the App. If you run
10628
+ your own ingress and want GitHub to deliver events to it instead, pass
10629
+ `--webhook-url` when creating the App:
10630
+
10631
+ ```bash
10632
+ kici-admin source add github --manifest --name my-org \
10633
+ --webhook-url https://hooks.my-infra.example/github
10634
+ ```
10635
+
10636
+ The supplied URL must be an absolute `https://` URL; it is written into the
10637
+ App's webhook configuration **verbatim**. This is the operator asserting "I own
10638
+ webhook delivery": KiCI adds **no** ingress at this URL and does **not** receive
10639
+ events there — your own infrastructure is responsible for accepting GitHub's
10640
+ deliveries and routing them onward. Supplying the flag also decouples App
10641
+ creation from platform-mode URL resolution, so it works even in a configuration
10642
+ where the auto-resolved KiCI Platform URL is unavailable.
10643
+
10057
10644
  ## Global workflows
10058
10645
 
10059
10646
  A GitHub App source opts in to org-wide global workflows using the
@@ -10083,7 +10670,7 @@ kici-admin org-settings global-workflows deny-add 'my-org/contrib/*' \
10083
10670
  Global workflows authored in a GitHub App repo can dispatch against
10084
10671
  events from universal-git sources in the same org, and vice versa,
10085
10672
  with each clone using its own source's credentials. See
10086
- [Global workflows](../../architecture/global-workflows.md) for the
10673
+ [Global workflows](https://docs.kici.dev/architecture/global-workflows/) for the
10087
10674
  policy model and cross-source dispatch contract.
10088
10675
 
10089
10676
  ## Check runs
@@ -10101,7 +10688,7 @@ configuration is required beyond installing the App with the
10101
10688
  `checks: write` permission.
10102
10689
 
10103
10690
  For architecture details see
10104
- [GitHub checks architecture](../../architecture/webhooks/github-checks.md).
10691
+ [GitHub checks architecture](https://docs.kici.dev/architecture/webhooks/github-checks/).
10105
10692
 
10106
10693
  ## Rotation
10107
10694
 
@@ -10184,12 +10771,12 @@ and confirm the App is installed on that repo.
10184
10771
 
10185
10772
  ## See also
10186
10773
 
10187
- - [Universal-git provider](universal-git.md) — for Forgejo / Gitea /
10774
+ - [Universal-git provider](https://docs.kici.dev/user/providers/universal-git/) — for Forgejo / Gitea /
10188
10775
  Gogs / GitLab, and for plain-GitHub repos without an App
10189
- - [GitHub checks architecture](../../architecture/webhooks/github-checks.md)
10190
- - [Global workflows](../../architecture/global-workflows.md)
10191
- - [kici-admin CLI reference](../../operator/orchestrator/kici-admin-cli.md)
10192
- - [Event routing](../../operator/event-routing.md) — operator-level
10776
+ - [GitHub checks architecture](https://docs.kici.dev/architecture/webhooks/github-checks/)
10777
+ - [Global workflows](https://docs.kici.dev/architecture/global-workflows/)
10778
+ - [kici-admin CLI reference](https://docs.kici.dev/operator/orchestrator/kici-admin-cli/)
10779
+ - [Event routing](https://docs.kici.dev/operator/event-routing/) — operator-level
10193
10780
  routing-key mechanics
10194
10781
 
10195
10782
  ---
@@ -10208,8 +10795,8 @@ from GitHub or GitLab; the operator drives runs explicitly with the
10208
10795
  > globally-registered or policy workflows the operator controls — a vendored
10209
10796
  > repo baked into an agent image, a repo synced onto a host out-of-band, a
10210
10797
  > golden internal pipeline. For ordinary per-developer CI driven by pull
10211
- > requests and pushes, use the [GitHub App provider](./github.md) or the
10212
- > [universal-git provider](./universal-git.md) against a real forge.
10798
+ > requests and pushes, use the [GitHub App provider](https://docs.kici.dev/user/providers/github/) or the
10799
+ > [universal-git provider](https://docs.kici.dev/user/providers/universal-git/) against a real forge.
10213
10800
 
10214
10801
  ## Trust caveat (read first)
10215
10802
 
@@ -10304,9 +10891,9 @@ the run is created and then fails, rather than being silently dropped.
10304
10891
 
10305
10892
  ## See also
10306
10893
 
10307
- - [Universal-git provider](./universal-git.md) — for a remote forge (or any
10894
+ - [Universal-git provider](https://docs.kici.dev/user/providers/universal-git/) — for a remote forge (or any
10308
10895
  `http://` git server) when there is no shared filesystem.
10309
- - [GitHub App provider](./github.md) — the flagship source for pull-request
10896
+ - [GitHub App provider](https://docs.kici.dev/user/providers/github/) — the flagship source for pull-request
10310
10897
  CI with Checks.
10311
10898
 
10312
10899
  ---
@@ -10321,7 +10908,7 @@ Gitea, Gogs, GitLab, plain GitHub (without the App), and any custom
10321
10908
  webhook-driven forge you can describe in JSONPath.
10322
10909
 
10323
10910
  > **Want Check runs on pull requests?** Use the [GitHub App
10324
- > provider](github.md) instead — it clones via short-lived installation
10911
+ > provider](https://docs.kici.dev/user/providers/github/) instead — it clones via short-lived installation
10325
10912
  > tokens and drives KiCI's enriched Checks UI out of the box. The
10326
10913
  > universal-git `github-repo` preset is the right fallback when you
10327
10914
  > can't install an App.
@@ -10341,7 +10928,7 @@ that back the GitHub App source also serve universal-git sources.
10341
10928
  > the right choice for the **remote-agent** case — point it at an `http://`
10342
10929
  > git server and the agent clones over the network. When the repo instead
10343
10930
  > lives on the agent's own filesystem (a vendored / operator-curated repo),
10344
- > use a [local `file://` source](./local-file.md) and drive it with the
10931
+ > use a [local `file://` source](https://docs.kici.dev/user/providers/local-file/) and drive it with the
10345
10932
  > `kici-admin` CLI.
10346
10933
 
10347
10934
  ## Which preset do I need?
@@ -10515,7 +11102,7 @@ kici-admin org-settings global-workflows deny-add \
10515
11102
  --source "generic:<orgId>:<sourceId>"
10516
11103
  ```
10517
11104
 
10518
- See [Global workflows](../../architecture/global-workflows.md) for the
11105
+ See [Global workflows](https://docs.kici.dev/architecture/global-workflows/) for the
10519
11106
  policy model (`isWorkflowRepoAllowed` + `isSourceRepoAllowed` +
10520
11107
  `isElevatedAccessAllowed`) and the cross-provider dispatch contract.
10521
11108
 
@@ -10559,6 +11146,374 @@ with the right PEM.
10559
11146
 
10560
11147
  # Architecture overview
10561
11148
 
11149
+ ## Configuration architecture
11150
+
11151
+ Source: https://docs.kici.dev/architecture/configuration/
11152
+
11153
+ This document describes the internal design of the orchestrator's configuration management system. For operator-facing documentation, see [Configuration Reference](https://docs.kici.dev/operator/orchestrator/configuration/) and [Config Management Guide](https://docs.kici.dev/operator/orchestrator/config-management/).
11154
+
11155
+ ## Config type system
11156
+
11157
+ The configuration is modeled as three distinct types that merge into a final application config:
11158
+
11159
+ ### LocalConfig
11160
+
11161
+ Per-orchestrator settings loaded from a YAML file. These are instance-specific and never shared:
11162
+
11163
+ ```typescript
11164
+ interface LocalConfig {
11165
+ database: { url: string };
11166
+ instance?: { id?: string; mode?: 'platform' | 'hybrid' | 'independent' };
11167
+ server?: { port?: number; basePath?: string; logLevel?: string };
11168
+ scaler?: { configPath?: string; configDir?: string };
11169
+ }
11170
+ ```
11171
+
11172
+ **Key property:** Every field except `database.url` is optional. An orchestrator can run entirely from env vars with no YAML file.
11173
+
11174
+ ### SharedConfig
11175
+
11176
+ Shared settings stored in the PostgreSQL `config_versions` table. Defined once, shared across all instances:
11177
+
11178
+ ```typescript
11179
+ interface SharedConfig {
11180
+ platform?: { url?: string; token?: string };
11181
+ storage?: { type?: 's3'; bucket?: string; ... };
11182
+ agentAuth?: 'token' | 'none';
11183
+ agentTokenTtlMs?: number;
11184
+ queue?: { maxDepth?: number; timeoutMs?: number };
11185
+ lockfileCache?: { max?: number; ttlMs?: number };
11186
+ staleDetector?: { scanIntervalMs?: number; ... };
11187
+ secrets?: { key?: string; keyFile?: string; bootstrapAdminToken?: string };
11188
+ pgCustomerSecrets?: boolean;
11189
+ cluster?: { joinToken?: string; raftElectionTimeoutMinMs?: number; ... };
11190
+ // ... tuning fields
11191
+ }
11192
+ ```
11193
+
11194
+ **Key property:** All top-level fields are optional. The DB may store a partial config.
11195
+
11196
+ ### AppConfig
11197
+
11198
+ The merged result type used throughout the codebase. Combines `LocalConfig` + `SharedConfig` with resolved defaults:
11199
+
11200
+ ```typescript
11201
+ interface AppConfig {
11202
+ instanceId: string; // From local config or auto-generated
11203
+ mode: 'platform' | 'hybrid' | 'independent';
11204
+ databaseUrl: string; // Flattened from database.url
11205
+ port: number; // Flattened from server.port
11206
+ basePath: string;
11207
+ platformUrl?: string; // Flattened from platform.url
11208
+ platformToken?: string;
11209
+ agentAuth: 'token' | 'none'; // With defaults applied
11210
+ queueMaxDepth: number; // Flattened from queue.maxDepth
11211
+ cluster: { instanceId: string; credentialFile: string; autoRotateCredentials: boolean; peers: string[]; ... };
11212
+ // ... all other fields with defaults
11213
+ }
11214
+ ```
11215
+
11216
+ **Key property:** `AppConfig` uses flat field names (e.g., `databaseUrl` instead of `database.url`) for backward compatibility with the existing codebase. A `flattenToAppConfig()` function handles the mapping.
11217
+
11218
+ ### How they merge
11219
+
11220
+ ```
11221
+ defaults (getDefaults())
11222
+ |
11223
+ v
11224
+ SharedConfig (from DB) ──deepMerge──> merged layer 1+2
11225
+ |
11226
+ v
11227
+ LocalConfig (from YAML) ──deepMerge──> merged layer 1+2+3
11228
+ |
11229
+ v
11230
+ Env var overrides ──apply──> merged layer 1+2+3+4
11231
+ |
11232
+ v
11233
+ flattenToAppConfig() ──flatten──> flat AppConfig shape
11234
+ |
11235
+ v
11236
+ appConfigSchema.safeParse() ──validate──> typed AppConfig
11237
+ ```
11238
+
11239
+ The `deepMerge` function merges objects recursively, replaces arrays (does not merge item-by-item), and skips `undefined`/`null` source values (they do not override existing values).
11240
+
11241
+ ## Resolution chain
11242
+
11243
+ ### Two-phase design
11244
+
11245
+ ```
11246
+ Phase 1 (local-only):
11247
+ YAML file + KICI_ env vars
11248
+ |
11249
+ v
11250
+ resolveLocalConfig() -> { databaseUrl, instanceId, port, mode }
11251
+ |
11252
+ v
11253
+ Connect to PostgreSQL
11254
+ |
11255
+ v
11256
+ Phase 2 (full merge):
11257
+ defaults -> DB -> YAML -> env
11258
+ |
11259
+ v
11260
+ resolveFullConfig() -> AppConfig
11261
+ ```
11262
+
11263
+ **Why two phases?** The database URL must come from local config (YAML or env var) because we need it to connect to PostgreSQL. But the shared config is stored in PostgreSQL. This circular dependency is broken by resolving local config first (Phase 1), connecting to the DB, then doing the full merge (Phase 2).
11264
+
11265
+ ### Env var processing
11266
+
11267
+ Environment variables are processed in two stages:
11268
+
11269
+ 1. **Direct mappings:** `KICI_DATABASE_URL` -> `database.url`, etc. A lookup table in `env-overlay.ts` maps known env var suffixes to config path arrays.
11270
+
11271
+ 2. **Multi-app GitHub provider:** `KICI_PROVIDERS_GITHUB_<APP_NAME>_<FIELD>` is parsed by stripping the `PROVIDERS_GITHUB_` prefix, finding the field suffix (`APP_ID`, `PRIVATE_KEY`, `WEBHOOK_SECRET`), and deriving the app name from the middle segment. App names are lowercased with underscores converted to hyphens (`MAIN_ORG` -> `main-org`).
11272
+
11273
+ Type coercion is applied based on known field types: numeric fields are parsed as numbers, boolean fields are compared against `"true"`, all others remain strings.
11274
+
11275
+ ## Two-phase bootstrap
11276
+
11277
+ ```
11278
+ ┌─────────────────┐
11279
+ │ Process Start │
11280
+ └────────┬────────┘
11281
+
11282
+
11283
+ ┌─────────────────┐
11284
+ │ Load YAML + │ resolveLocalConfig()
11285
+ │ Env Overrides │ -> databaseUrl, instanceId, port, mode
11286
+ └────────┬────────┘
11287
+
11288
+
11289
+ ┌─────────────────┐
11290
+ │ Connect to │ PostgreSQL
11291
+ │ Database │ Run migrations
11292
+ └────────┬────────┘
11293
+
11294
+
11295
+ ┌─────────────────┐
11296
+ │ Load Shared │ SharedConfigStore.getLatest()
11297
+ │ Config from DB │ -> decrypt -> SharedConfig
11298
+ └────────┬────────┘
11299
+
11300
+
11301
+ ┌─────────────────┐
11302
+ │ Full Merge │ resolveFullConfig(local, db, env)
11303
+ │ + Validate │ -> AppConfig
11304
+ └────────┬────────┘
11305
+
11306
+
11307
+ ┌─────────────────┐
11308
+ │ Start Server │ HTTP, WS, scaler, cluster
11309
+ └─────────────────┘
11310
+ ```
11311
+
11312
+ ## DB schema
11313
+
11314
+ ### config_versions table
11315
+
11316
+ ```sql
11317
+ CREATE TABLE config_versions (
11318
+ id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
11319
+ version SERIAL NOT NULL UNIQUE,
11320
+ config JSONB NOT NULL,
11321
+ created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
11322
+ created_by TEXT NOT NULL,
11323
+ description TEXT,
11324
+ encrypted_paths TEXT[] NOT NULL DEFAULT '{}'
11325
+ );
11326
+
11327
+ CREATE INDEX idx_config_versions_version ON config_versions(version DESC);
11328
+ CREATE INDEX idx_config_versions_created_at ON config_versions(created_at DESC);
11329
+ ```
11330
+
11331
+ **Design choices:**
11332
+
11333
+ - **SERIAL version:** Auto-incrementing integer provides a total ordering of config changes. Simple to compare in heartbeats.
11334
+ - **JSONB config:** Stores the full shared config document. JSONB allows future querying/indexing if needed, though we always read the full document.
11335
+ - **Immutable rows:** Each change creates a new version. Old versions are never modified, providing a full audit trail.
11336
+ - **encrypted_paths:** Array of concrete dot-separated paths (e.g., `platform.token`, `cluster.joinToken`) that contain encrypted values. Stored alongside the config so the system knows exactly which fields to decrypt without runtime path list dependency.
11337
+ - **created_by:** Identifies the source of the change (e.g., `cli:seed`, `api:set`, `api:rollback`).
11338
+
11339
+ ### Versioning strategy
11340
+
11341
+ - Version numbers are auto-incrementing integers managed by PostgreSQL SERIAL
11342
+ - Rollback creates a new version (copy of target) rather than reverting to the old version number
11343
+ - Example: versions 1, 2, 3 exist. Rollback to 1 creates version 4 with the content of version 1
11344
+ - This preserves the audit trail: version 4 records when and why the rollback happened
11345
+
11346
+ ## Encryption
11347
+
11348
+ ### Algorithm
11349
+
11350
+ AES-256-GCM via the existing `secrets/crypto.ts` module:
11351
+
11352
+ - **Key:** 32-byte AES-256 key derived from the master key (`KICI_SECRET_KEY`)
11353
+ - **IV:** Random 12-byte initialization vector per encryption
11354
+ - **Auth tag:** 16-byte GCM authentication tag
11355
+ - **AAD:** `config-field:<path>` (e.g., `config-field:platform.token`) -- binds ciphertext to its specific location
11356
+ - **Wire format:** base64(IV || AuthTag || Ciphertext)
11357
+ - **Key version:** Integer stamp for the master-key generation that sealed the row. Every new row is written under the active generation (hydrated from `MAX(key_version)` at startup). `kici-admin rotate-key` bumps the stamp atomically — the decrypt path accepts the current generation, and during the grace window also the previous one (`KICI_SECRET_KEY_OLD`), so historical rows and rollbacks continue to work seamlessly across rotations.
11358
+
11359
+ ### Sensitive field paths
11360
+
11361
+ The following glob patterns define sensitive fields:
11362
+
11363
+ ```typescript
11364
+ const SENSITIVE_FIELD_PATHS = [
11365
+ 'platform.token',
11366
+ 'secrets.key',
11367
+ 'secrets.bootstrapAdminToken',
11368
+ 'cluster.joinToken',
11369
+ ] as const;
11370
+ ```
11371
+
11372
+ ### Encryption flow
11373
+
11374
+ ```
11375
+ Save:
11376
+ config -> resolveGlobPaths(SENSITIVE_FIELD_PATHS)
11377
+ -> for each concrete path: encrypt(value, key, "config-field:<path>")
11378
+ -> store { encrypted_config, encrypted_paths[] }
11379
+
11380
+ Load:
11381
+ row -> for each path in encrypted_paths: decrypt(value, key, "config-field:<path>")
11382
+ -> SharedConfig
11383
+
11384
+ Export (redacted):
11385
+ row -> decrypt (if master key available) -> replace encrypted_paths values with "***REDACTED***"
11386
+ ```
11387
+
11388
+ ### Rollback optimization
11389
+
11390
+ When rolling back, the target version's encrypted config is copied as-is to the new version. No re-encryption is needed because:
11391
+
11392
+ - The same master key applies (all orchestrators share the same key)
11393
+ - The same AAD applies (paths are identical)
11394
+ - The `encrypted_paths` array is preserved from the target version
11395
+
11396
+ ## Hot-Reload
11397
+
11398
+ ### ConfigReloader design
11399
+
11400
+ The `ConfigReloader` class manages the full reload lifecycle:
11401
+
11402
+ ```mermaid
11403
+ flowchart TD
11404
+ SIGHUP["SIGHUP (signal)"] --> trigger
11405
+ HTTP["HTTP POST /reload"] --> trigger
11406
+ Cluster["Cluster auto-fix"] --> trigger["triggerReload()<br/>500ms debounce"]
11407
+
11408
+ trigger --> execute["executeReload()<br/>boolean mutex"]
11409
+
11410
+ execute --> resolveLocal["resolveLocalConfig()"]
11411
+ execute --> getLatest["getLatest (DB)"]
11412
+ execute --> resolveFull["resolveFullConfig()<br/>(merge + validate)"]
11413
+
11414
+ resolveLocal --> check["Check restart-required fields"]
11415
+ getLatest --> check
11416
+ resolveFull --> check
11417
+
11418
+ check --> swap["Atomic swap<br/>onConfigApplied()"]
11419
+
11420
+ swap --> onProvider["onProviderChange (if changed)"]
11421
+ swap --> onScaler["onScalerReload (always)"]
11422
+ swap --> onPlatform["onPlatformReconnect (if changed)"]
11423
+ ```
11424
+
11425
+ ### Safety guarantees
11426
+
11427
+ - **Mutex:** Boolean flag prevents concurrent reloads. Second reload returns `{ success: false, errors: ["Reload already in progress"] }`.
11428
+ - **Debounce:** Rapid triggers (e.g., multiple SIGHUP signals) are collapsed into a single reload with a 500ms window.
11429
+ - **Validation before swap:** The new config must pass full schema validation. On failure, the old config is preserved and an error is logged.
11430
+ - **Restart-required detection:** Fields like `databaseUrl`, `port`, `instanceId` are compared. If changed, the old values are preserved in the applied config and a warning is logged.
11431
+ - **No crash on failure:** The orchestrator always keeps running with the old config if anything goes wrong during reload.
11432
+
11433
+ ### Subsystem callbacks
11434
+
11435
+ The `ConfigReloader` uses a dependency injection pattern with callbacks for subsystem re-initialization:
11436
+
11437
+ | Callback | When Called | Purpose |
11438
+ | --------------------- | ----------------------------- | -------------------------------------------------------------------------------------------- |
11439
+ | `onProviderChange` | Provider config changed | Reserved callback (providers are now DB-managed via sources table; currently always a no-op) |
11440
+ | `onScalerReload` | Always on successful reload | Reload scaler YAML config |
11441
+ | `onPlatformReconnect` | Platform URL or token changed | Reconnect WS to Platform relay |
11442
+ | `onConfigApplied` | Always on successful reload | Atomic config reference swap, increment local config version |
11443
+
11444
+ ### Prometheus metrics
11445
+
11446
+ | Metric | Type | Labels | Description |
11447
+ | ------------------------------- | ------- | ----------------------------------------------------------------------- | ------------------------------------- |
11448
+ | `kici_orch_config_reload_total` | Counter | `result` (attempted/success/failed), `source` (sighup/http/cluster/cli) | Config reload attempts and outcomes |
11449
+ | `kici_orch_config_version` | Gauge | -- | Current shared config version from DB |
11450
+
11451
+ ## Multi-Provider
11452
+
11453
+ ### ProviderRegistry
11454
+
11455
+ The `ProviderRegistry` maps routing keys to provider bundles. Each routing key (e.g., `github:12345`) is associated with a `ProviderBundle` containing:
11456
+
11457
+ - `WebhookNormalizer` -- normalizes incoming webhooks to a standard format
11458
+ - `LockFileFetcher` -- fetches lock files from the repository
11459
+ - `ChangedFilesFetcher` -- determines which files changed
11460
+ - `CloneTokenProvider` -- generates clone tokens for agents
11461
+ - `RepoUrlBuilder` -- builds clone URLs and raw file URLs
11462
+ - `ContributorResolver` -- resolves contributor permissions for trust-tier gating
11463
+ - `CheckStatusPoster` -- posts check statuses (approval/hold) to the git provider
11464
+
11465
+ Provider registrations are managed via the `sources` database table, not via `SharedConfig`. When the orchestrator connects to the Platform relay, it reads source records from the DB and sends `source.register` messages. Changes to sources (add/remove) are detected via PostgreSQL LISTEN/NOTIFY on the `sources_change` channel and pushed to the Platform via `source.secrets` and `source.register`/`source.deregister`.
11466
+
11467
+ ### Per-App Credentials
11468
+
11469
+ Each source record contains its own `appId` and `privateKey` (stored as scoped secrets). When processing a webhook, the orchestrator looks up the routing key to find the matching source and uses its credentials for JWT generation, clone tokens, and check run updates.
11470
+
11471
+ ## Cluster sync
11472
+
11473
+ ### Heartbeat config version
11474
+
11475
+ In clustered deployments, each orchestrator includes its current config version in Raft heartbeat metadata via the `configVersion` optional field on the `peerHeartbeatSchema`.
11476
+
11477
+ When the `PeerRegistry` processes a heartbeat:
11478
+
11479
+ 1. Compare `localConfigVersion` with `peer.configVersion`
11480
+ 2. If `peer.configVersion > localConfigVersion` AND both are > 0:
11481
+ - Invoke the `onConfigVersionBehind` callback
11482
+ - This triggers a config reload from the database
11483
+
11484
+ ### Auto-remediation flow
11485
+
11486
+ ```
11487
+ Orchestrator A (version 5) Orchestrator B (version 3)
11488
+ │ │
11489
+ │──── heartbeat(configVersion=5) ────>│
11490
+ │ │
11491
+ │ compare: 5 > 3
11492
+ │ trigger reload from DB
11493
+ │ │
11494
+ │ resolveFullConfig()
11495
+ │ -> version 5
11496
+ │ │
11497
+ │<── heartbeat(configVersion=5) ──────│
11498
+ │ │
11499
+ │ both at version 5 ✓ │
11500
+ ```
11501
+
11502
+ ### Guard conditions
11503
+
11504
+ - Version comparison only triggers when **both** local and peer versions are > 0
11505
+ - This prevents false triggers from:
11506
+ - Legacy orchestrators that do not report `configVersion` (field is optional, defaults to 0)
11507
+ - Newly started orchestrators before their first config load
11508
+ - The `localConfigVersion` is a monotonically incrementing local counter (incremented on each successful reload)
11509
+
11510
+ ## See also
11511
+
11512
+ - [Configuration Reference](https://docs.kici.dev/operator/orchestrator/configuration/) -- operator guide
11513
+ - [Config Management Guide](https://docs.kici.dev/operator/orchestrator/config-management/) -- CLI and API guide
11514
+
11515
+ ---
11516
+
10562
11517
  ## Data flows
10563
11518
 
10564
11519
  Source: https://docs.kici.dev/architecture/data-flows/
@@ -10596,7 +11551,7 @@ GitHub --> Platform Relay --> Orchestrator --> Agent
10596
11551
  15. **Orchestrator matches triggers** against lock file using `matchAllWorkflows()` from `@kici-dev/engine`.
10597
11552
  16. **Orchestrator checks caches** for source tarballs and dependency tarballs.
10598
11553
  17. **Orchestrator dispatches jobs** to agents via the job queue and WebSocket.
10599
- 18. **Orchestrator persists a delivery row** keyed by `(org_id, delivery_id)` to its own `event_log`, including a pointer to the gzipped payload in object storage. The orchestrator's delivery log is surfaced in the dashboard's Settings → Event log tab. See [`webhook-delivery.md`](./webhooks/webhook-delivery.md#delivery-log).
11554
+ 18. **Orchestrator persists a delivery row** keyed by `(org_id, delivery_id)` to its own `event_log`, including a pointer to the gzipped payload in object storage. The orchestrator's delivery log is surfaced in the dashboard's Settings → Event log tab. See [`webhook-delivery.md`](https://docs.kici.dev/architecture/webhooks/webhook-delivery/#delivery-log).
10600
11555
 
10601
11556
  ## Job execution flow
10602
11557
 
@@ -10624,7 +11579,7 @@ Orchestrator Agent Sandbox (child pro
10624
11579
 
10625
11580
  ### Agent pipeline
10626
11581
 
10627
- The agent delegates job execution to an `ExecutionSandbox` (container, bare-metal, or firecracker). The sandbox runs customer code in an isolated child process -- never in the agent's V8 isolate. Four job types are handled: execution jobs (sandbox), build-only jobs (in-process, cache population), init-only jobs (in-process, dynamic field resolution), and DynamicJobFn evaluation jobs (in-process, runtime job generation). See [Job execution lifecycle](./execution/job-execution.md) for details.
11582
+ The agent delegates job execution to an `ExecutionSandbox` (container, bare-metal, or firecracker). The sandbox runs customer code in an isolated child process -- never in the agent's V8 isolate. Four job types are handled: execution jobs (sandbox), build-only jobs (in-process, cache population), init-only jobs (in-process, dynamic field resolution), and DynamicJobFn evaluation jobs (in-process, runtime job generation). See [Job execution lifecycle](https://docs.kici.dev/architecture/execution/job-execution/) for details.
10628
11583
 
10629
11584
  1. **Report running** -- Send `job.status: running` immediately upon accepting the dispatch
10630
11585
  2. **Sandbox selection** -- Determine execution mode (container, bare-metal, firecracker) from job config and environment
@@ -10662,7 +11617,7 @@ Run initiation (`upload-init`), the trigger, status polling, log retrieval, and
10662
11617
 
10663
11618
  ### Data plane
10664
11619
 
10665
- The working-tree overlay tarball uploads **directly** from the developer machine to the orchestrator's object store via a pre-signed PUT URL minted during `upload-init`. The overlay bytes never pass through the Platform. Because of this split, only the object store needs to be reachable from the developer machine — the orchestrator can sit behind a private network. See [Storage layout](../operator/orchestrator/storage-layout.md) for the upload-endpoint configuration.
11620
+ The working-tree overlay tarball uploads **directly** from the developer machine to the orchestrator's object store via a pre-signed PUT URL minted during `upload-init`. The overlay bytes never pass through the Platform. Because of this split, only the object store needs to be reachable from the developer machine — the orchestrator can sit behind a private network. See [Storage layout](https://docs.kici.dev/operator/orchestrator/storage-layout/) for the upload-endpoint configuration.
10666
11621
 
10667
11622
  ### Org anchor
10668
11623
 
@@ -10785,7 +11740,7 @@ If cache storage is unavailable or a download fails:
10785
11740
  - **Source tarball download failure:** Hard failure today — the agent does not fall back to `git clone` on the execution path. (The build path is where cloning happens.) In practice this is rare because the same orchestrator that issued the pre-signed URL controls the cache backend.
10786
11741
  - **Dep tarball download failure:** Agent falls back to running `npm ci` / `npm install` inline.
10787
11742
  - **Dep tarball hash mismatch:** Agent retries the download twice (3 total attempts), then fails the job (no fallback for integrity failures).
10788
- - **Source tarball drift (extracted `contentHash` ≠ lock file):** Hard failure with "Lock file is out of date: workflow source changed without regenerating kici.lock.json" — see [Lock file and drift](../user/lock-file-and-drift.md).
11743
+ - **Source tarball drift (extracted `contentHash` ≠ lock file):** Hard failure with "Lock file is out of date: workflow source changed without regenerating kici.lock.json" — see [Lock file and drift](https://docs.kici.dev/user/lock-file-and-drift/).
10789
11744
  - **Build failure:** Execution is skipped entirely with a "Build failed" check status. Workflows that contain dynamic job entries (DynamicJobFn) are allowed to proceed with their dynamic eval jobs since those compile from source.
10790
11745
  - **No cache configured:** Agent runs inline install for every job (pre-caching behavior).
10791
11746
 
@@ -10817,9 +11772,9 @@ The orchestrator derives the target platform/arch for dep cache lookups by probi
10817
11772
 
10818
11773
  ### TTL and eviction (touch-on-read)
10819
11774
 
10820
- Both caches refresh TTL on read via `touch-on-read`. An entry's lifetime is reset every time an orchestrator issues a pre-signed GET URL for it. Default TTL is `KICI_CACHE_TTL_DAYS=30`; entries unused for 30 days expire at the storage level. Actively used sources and deps stay in cache indefinitely as long as they continue to be referenced by inbound webhooks or reruns. See [`docs/operator/dependency-caching.md`](../operator/dependency-caching.md#cache-behavior) for configuration.
11775
+ Both caches refresh TTL on read via `touch-on-read`. An entry's lifetime is reset every time an orchestrator issues a pre-signed GET URL for it. Default TTL is `KICI_CACHE_TTL_DAYS=30`; entries unused for 30 days expire at the storage level. Actively used sources and deps stay in cache indefinitely as long as they continue to be referenced by inbound webhooks or reruns. See [`docs/operator/dependency-caching.md`](https://docs.kici.dev/operator/dependency-caching/#cache-behavior) for configuration.
10821
11776
 
10822
- For the full per-package bucket and prefix inventory — cache, logs, cold-store, and the observability sidecar buckets — see [orchestrator storage layout](../operator/orchestrator/storage-layout.md).
11777
+ For the full per-package bucket and prefix inventory — cache, logs, cold-store, and the observability sidecar buckets — see [orchestrator storage layout](https://docs.kici.dev/operator/orchestrator/storage-layout/).
10823
11778
 
10824
11779
  ### Pre-signed URL upload flow
10825
11780
 
@@ -10857,7 +11812,7 @@ Agents receive pre-signed S3 GET URLs (15-minute expiry) directly in `job.dispat
10857
11812
 
10858
11813
  ## User-facing cache flow
10859
11814
 
10860
- The source/dep cache above is internal: the orchestrator owns its keys and decides when to hit or build. The **user-facing cache** is driven by the workflow author — the declarative `cache: { key, paths, restoreKeys? }` on a job/step, or the imperative `ctx.cache.restore()` / `ctx.cache.save()` API (see [SDK caching reference](../user/sdk/caching.md)). It reuses the same object-storage backend and the same direct-to-storage presigned-URL transport, but the agent — not the orchestrator — initiates each restore and save over WebSocket.
11815
+ The source/dep cache above is internal: the orchestrator owns its keys and decides when to hit or build. The **user-facing cache** is driven by the workflow author — the declarative `cache: { key, paths, restoreKeys? }` on a job/step, or the imperative `ctx.cache.restore()` / `ctx.cache.save()` API (see [SDK caching reference](https://docs.kici.dev/user/sdk/caching/)). It reuses the same object-storage backend and the same direct-to-storage presigned-URL transport, but the agent — not the orchestrator — initiates each restore and save over WebSocket.
10861
11816
 
10862
11817
  The agent's cache module archives `paths` into a gzipped tarball (computing a SHA-256 over the bytes) and streams downloads back through a checksum-verified extract pipeline. The orchestrator's `UserCache` owns the `cache/<orgId>/<repoId>/<scope>/<key>` namespacing, the immutable first-save check, the `restoreKeys` prefix scan, the two-phase atomic save, and per-org quota/TTL eviction.
10863
11818
 
@@ -10911,7 +11866,7 @@ The save is **immutable** and **atomic**. The orchestrator declines (`skip: true
10911
11866
 
10912
11867
  ### Trust → scope mapping
10913
11868
 
10914
- The orchestrator threads a `cacheRefScope` onto each `job.dispatch`. A **trusted** ref (the repo's own branches, default branch) maps to the `shared` write scope; any other ref (a fork PR) maps to `isolated`, writing to a per-run `iso/<runId>/` scope. This is the cache-isolation model: a fork can restore from the trusted `shared/` cache but can never write into it, so it cannot poison the entries a trusted branch later restores. The org segment of the key namespace (`cache/<orgId>/`) is the per-tenant boundary — no tenant can read another tenant's cache. See [orchestrator storage layout](../operator/orchestrator/storage-layout.md#user-cache) for the full prefix map and quota/TTL knobs.
11869
+ The orchestrator threads a `cacheRefScope` onto each `job.dispatch`. A **trusted** ref (the repo's own branches, default branch) maps to the `shared` write scope; any other ref (a fork PR) maps to `isolated`, writing to a per-run `iso/<runId>/` scope. This is the cache-isolation model: a fork can restore from the trusted `shared/` cache but can never write into it, so it cannot poison the entries a trusted branch later restores. The org segment of the key namespace (`cache/<orgId>/`) is the per-tenant boundary — no tenant can read another tenant's cache. See [orchestrator storage layout](https://docs.kici.dev/operator/orchestrator/storage-layout/#user-cache) for the full prefix map and quota/TTL knobs.
10915
11870
 
10916
11871
  ## Internal event routing flow
10917
11872
 
@@ -11405,15 +12360,15 @@ The Platform tier exposes a `/ws/browser` WebSocket endpoint for dashboard clien
11405
12360
 
11406
12361
  ## See also
11407
12362
 
11408
- - [Architecture overview](overview.md) -- three-tier model and component responsibilities
11409
- - [Protocol messages](protocol-messages.md) -- WebSocket message schemas
11410
- - [Event system internals](./webhooks/event-system.md) -- event router, registration model, cron scheduler
11411
- - [State machine](./execution/state-machine.md) -- job execution state transitions
11412
- - [Webhook delivery](./webhooks/webhook-delivery.md) -- detailed webhook processing pipeline
11413
- - [Operator: dependency caching](../operator/dependency-caching.md) -- configuration guide
11414
- - [Operator: monitoring & tracing](../operator/observability/monitoring.md) -- trace fields and Loki queries
11415
- - [Operator: event routing & generic webhooks](../operator/event-routing.md) -- generic source setup and trust management
11416
- - [SDK reference: output chaining](../user/sdk/core.md#output-chaining) -- user-facing output chaining API
12363
+ - [Architecture overview](https://docs.kici.dev/architecture/overview/) -- three-tier model and component responsibilities
12364
+ - [Protocol messages](https://docs.kici.dev/architecture/protocol-messages/) -- WebSocket message schemas
12365
+ - [Event system internals](https://docs.kici.dev/architecture/webhooks/event-system/) -- event router, registration model, cron scheduler
12366
+ - [State machine](https://docs.kici.dev/architecture/execution/state-machine/) -- job execution state transitions
12367
+ - [Webhook delivery](https://docs.kici.dev/architecture/webhooks/webhook-delivery/) -- detailed webhook processing pipeline
12368
+ - [Operator: dependency caching](https://docs.kici.dev/operator/dependency-caching/) -- configuration guide
12369
+ - [Operator: monitoring & tracing](https://docs.kici.dev/operator/observability/monitoring/) -- trace fields and Loki queries
12370
+ - [Operator: event routing & generic webhooks](https://docs.kici.dev/operator/event-routing/) -- generic source setup and trust management
12371
+ - [SDK reference: output chaining](https://docs.kici.dev/user/sdk/core/#output-chaining) -- user-facing output chaining API
11417
12372
 
11418
12373
  ---
11419
12374
 
@@ -11468,7 +12423,7 @@ The orchestrator is the execution brain. It decides what to run and dispatches w
11468
12423
  - **Agent registry** -- Tracks connected agents with label-based routing for job dispatch.
11469
12424
  - **Job queue** -- PostgreSQL-backed FIFO queue for reliable dispatch.
11470
12425
  - **Webhook pipeline** -- Dedup, event mapping, lock file fetch, trigger matching, and job dispatch in a single pipeline.
11471
- - **Multi-orchestrator clustering** -- Optional peer-to-peer coordination via direct WebSocket connections. Enables cross-architecture job routing (e.g., x64 coordinator reroutes arm64 jobs to a peer), high availability, and dedicated coordinator topologies. Uses Raft consensus for leader election (orphan recovery). See [Multi-Orchestrator Architecture](./clustering/multi-orchestrator.md).
12426
+ - **Multi-orchestrator clustering** -- Optional peer-to-peer coordination via direct WebSocket connections. Enables cross-architecture job routing (e.g., x64 coordinator reroutes arm64 jobs to a peer), high availability, and dedicated coordinator topologies. Uses Raft consensus for leader election (orphan recovery). See [Multi-Orchestrator Architecture](https://docs.kici.dev/architecture/clustering/multi-orchestrator/).
11472
12427
  - **Auto-scaler** -- Optional pluggable module for ephemeral agent provisioning. Supports containers (Docker/Podman), bare-metal processes, and Firecracker microVMs as backends. Spawns agents on demand when no matching agent is connected, with label-based routing, two-level capacity limits (global + per-backend), warm pools, YAML configuration (`scalers.d/` directory support), and SIGHUP reload. Disabled by default -- orchestrator works without it.
11473
12428
  - **Independent database** -- Has its own PostgreSQL database separate from the Platform. Stores execution runs/jobs/steps, dispatch queue, webhook secrets, dedup cache, and scaler state. The orchestrator's `execution_runs` and `execution_jobs` are the authoritative source of truth. The Platform receives execution status updates via WebSocket messages (`execution.status`, `job.status.forward`).
11474
12429
 
@@ -11607,13 +12562,13 @@ The orchestrator connects outbound to the Platform WebSocket endpoint. After aut
11607
12562
 
11608
12563
  When multiple orchestrators are deployed, they establish direct WebSocket connections to each other on the `/ws/peer` endpoint. Peers are discovered via the Platform matchmaker (Platform/hybrid modes) or static configuration (`KICI_CLUSTER_PEERS` env var, independent mode). Connections are authenticated with a mutual pre-shared key (PSK). Traffic includes agent inventory heartbeats, job rerouting, progress reporting, cancel propagation, and Raft leader election. These messages never transit the Platform tier.
11609
12564
 
11610
- > See [Multi-Orchestrator Architecture](./clustering/multi-orchestrator.md) for clustering details and [Protocol Messages](protocol/dashboard.md#orchestrator-orchestrator-messages-peer-to-peer) for message schemas.
12565
+ > See [Multi-Orchestrator Architecture](https://docs.kici.dev/architecture/clustering/multi-orchestrator/) for clustering details and [Protocol Messages](https://docs.kici.dev/architecture/protocol/dashboard/#orchestrator-orchestrator-messages-peer-to-peer) for message schemas.
11611
12566
 
11612
12567
  ### Orchestrator ↔ Agent
11613
12568
 
11614
12569
  The agent connects outbound to the orchestrator WebSocket endpoint. After registration (agent ID, labels, concurrency), the connection is used for job dispatch, status reporting, and log streaming.
11615
12570
 
11616
- > See [Protocol Messages](protocol-messages.md) and [Webhook Delivery](./webhooks/webhook-delivery.md) for detailed message flows and schemas.
12571
+ > See [Protocol Messages](https://docs.kici.dev/architecture/protocol-messages/) and [Webhook Delivery](https://docs.kici.dev/architecture/webhooks/webhook-delivery/) for detailed message flows and schemas.
11617
12572
 
11618
12573
  ## Authentication and multi-tenancy
11619
12574
 
@@ -11621,9 +12576,9 @@ KiCI uses application-level tenant isolation. The Platform dashboard API accepts
11621
12576
 
11622
12577
  ## See also
11623
12578
 
11624
- - [Multi-Orchestrator Architecture](./clustering/multi-orchestrator.md) -- P2P clustering, Raft consensus, job rerouting
11625
- - [State Machine](./execution/state-machine.md) -- execution lifecycle tracking across all tiers
11626
- - [Protocol Messages](protocol-messages.md) -- WebSocket message schemas for all three layers
11627
- - [Webhook Delivery](./webhooks/webhook-delivery.md) -- end-to-end trace of a webhook through all three tiers
12579
+ - [Multi-Orchestrator Architecture](https://docs.kici.dev/architecture/clustering/multi-orchestrator/) -- P2P clustering, Raft consensus, job rerouting
12580
+ - [State Machine](https://docs.kici.dev/architecture/execution/state-machine/) -- execution lifecycle tracking across all tiers
12581
+ - [Protocol Messages](https://docs.kici.dev/architecture/protocol-messages/) -- WebSocket message schemas for all three layers
12582
+ - [Webhook Delivery](https://docs.kici.dev/architecture/webhooks/webhook-delivery/) -- end-to-end trace of a webhook through all three tiers
11628
12583
 
11629
12584
  ---