@kindgi/sdk 0.0.0-bootstrap.0 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +201 -0
- package/README.md +187 -1
- package/dist/build.d.ts +10 -0
- package/dist/build.d.ts.map +1 -0
- package/dist/build.js +11 -0
- package/dist/build.js.map +1 -0
- package/dist/client.d.ts +33 -0
- package/dist/client.d.ts.map +1 -0
- package/dist/client.js +55 -0
- package/dist/client.js.map +1 -0
- package/dist/define.d.ts +25 -0
- package/dist/define.d.ts.map +1 -0
- package/dist/define.js +37 -0
- package/dist/define.js.map +1 -0
- package/dist/index.d.ts +17 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +19 -0
- package/dist/index.js.map +1 -0
- package/dist/runtime-config.d.ts +53 -0
- package/dist/runtime-config.d.ts.map +1 -0
- package/dist/runtime-config.js +114 -0
- package/dist/runtime-config.js.map +1 -0
- package/dist/types.d.ts +16 -0
- package/dist/types.d.ts.map +1 -0
- package/dist/types.js +4 -0
- package/dist/types.js.map +1 -0
- package/dist/webhooks.d.ts +15 -0
- package/dist/webhooks.d.ts.map +1 -0
- package/dist/webhooks.js +16 -0
- package/dist/webhooks.js.map +1 -0
- package/package.json +88 -4
- package/skills/kindgi-authoring-agents/SKILL.md +252 -0
- package/skills/kindgi-authoring-flows/SKILL.md +302 -0
- package/skills/kindgi-authoring-guardrails/SKILL.md +297 -0
- package/skills/kindgi-authoring-mcp-servers/SKILL.md +289 -0
- package/skills/kindgi-authoring-providers/SKILL.md +705 -0
- package/skills/kindgi-authoring-tools/SKILL.md +298 -0
- package/skills/kindgi-framework-feedback/SKILL.md +211 -0
- package/skills/kindgi-getting-started/SKILL.md +189 -0
- package/skills/kindgi-python-authoring-agents/SKILL.md +205 -0
- package/skills/kindgi-python-authoring-flows/SKILL.md +325 -0
- package/skills/kindgi-python-authoring-guardrails/SKILL.md +176 -0
- package/skills/kindgi-python-authoring-tools/SKILL.md +305 -0
- package/skills/kindgi-python-getting-started/SKILL.md +242 -0
- package/src/build.ts +18 -0
- package/src/client.ts +177 -0
- package/src/define.ts +75 -0
- package/src/index.ts +20 -0
- package/src/runtime-config.ts +180 -0
- package/src/types.ts +86 -0
- package/src/webhooks.ts +34 -0
|
@@ -0,0 +1,302 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: kindgi-authoring-flows
|
|
3
|
+
description: >
|
|
4
|
+
Covers writing flows for a Kindgi pack with @kindgi/sdk: defineFlow,
|
|
5
|
+
tool and agent steps, edges and `when` conditions, branches that join
|
|
6
|
+
again, inputMapping from runInput / nodeOutputs, typed agent output in
|
|
7
|
+
a flow, the flow's declared output, loops (foreach / while) and fanout,
|
|
8
|
+
per-edge retry and timeout, and running a flow (kindgi runs start
|
|
9
|
+
--flow, in the background, as a dry run) and reading its journal. Load
|
|
10
|
+
this whenever you are authoring or editing code inside a pack's flows/
|
|
11
|
+
directory, defining a flow, or when the user asks to add, change or
|
|
12
|
+
debug one. Tools are covered by kindgi-authoring-tools, agents by
|
|
13
|
+
kindgi-authoring-agents.
|
|
14
|
+
type: core
|
|
15
|
+
library: "@kindgi/sdk"
|
|
16
|
+
version: "0.1.2"
|
|
17
|
+
sdk_version: "0.0.0"
|
|
18
|
+
pack_languages: [node]
|
|
19
|
+
sources:
|
|
20
|
+
- packages/flow/src/types.ts
|
|
21
|
+
- packages/flow/src/define.ts
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
# Authoring Kindgi flows
|
|
25
|
+
|
|
26
|
+
> **Running `kindgi`:** the CLI is a devDependency of the project (`@kindgi/cli`),
|
|
27
|
+
> not a global command. Run it through the project's package manager —
|
|
28
|
+
> `pnpm exec kindgi …`, `npx --no kindgi …` (npm), `yarn kindgi …` or
|
|
29
|
+
> `bun run kindgi …`. Commands below are written `kindgi …` for brevity.
|
|
30
|
+
|
|
31
|
+
A **flow** is a versioned, durable graph of steps: tools (your code) and
|
|
32
|
+
agents (a model's judgment), joined by edges that can carry conditions.
|
|
33
|
+
It is **data**, not code: `defineFlow({...})` in `flows/<name>/index.ts`.
|
|
34
|
+
The runtime runs it step by step, journals every step, and can resume a
|
|
35
|
+
run that was interrupted. A run pins the flow version it started on.
|
|
36
|
+
|
|
37
|
+
Use a flow when the order of the work is known: parse, then classify,
|
|
38
|
+
then branch, then write. Use a single agent when the model should decide
|
|
39
|
+
the order.
|
|
40
|
+
|
|
41
|
+
## Ask before building
|
|
42
|
+
|
|
43
|
+
- **What goes in, and what comes out?** The run input's shape and the
|
|
44
|
+
output the caller reads. They become `runInput.*` paths and `output`.
|
|
45
|
+
- **Which steps are code, which are judgment?** Deterministic work
|
|
46
|
+
(parse, rank, look up, write) is a tool; judgment (classify, draft,
|
|
47
|
+
summarize) is an agent with a typed `output`.
|
|
48
|
+
- **Where does it branch?** Every branch needs a condition, and the
|
|
49
|
+
steps after a branch must cope with the branch that didn't run.
|
|
50
|
+
- **What does it change outside Kindgi?** A tool that only reads
|
|
51
|
+
declares `mutating: false`, and a dry run runs it. A tool that writes
|
|
52
|
+
doesn't (leaving it out counts as mutating), and a dry run stops there.
|
|
53
|
+
|
|
54
|
+
## A flow
|
|
55
|
+
|
|
56
|
+
```ts
|
|
57
|
+
// flows/triage-ticket/index.ts
|
|
58
|
+
import { defineFlow } from '@kindgi/sdk/define';
|
|
59
|
+
|
|
60
|
+
const isBilling = {
|
|
61
|
+
op: 'eq',
|
|
62
|
+
left: { path: 'nodeOutputs.classify.output.category' },
|
|
63
|
+
right: { literal: 'billing' },
|
|
64
|
+
} as const;
|
|
65
|
+
|
|
66
|
+
const defined = defineFlow({
|
|
67
|
+
id: 'acme.triage-ticket',
|
|
68
|
+
version: '0.1.0',
|
|
69
|
+
name: 'Triage a support ticket',
|
|
70
|
+
description: 'Parses a ticket, classifies it, looks up billing when needed, drafts a reply.',
|
|
71
|
+
nodes: [
|
|
72
|
+
{
|
|
73
|
+
id: 'parse',
|
|
74
|
+
kind: 'tool',
|
|
75
|
+
ref: 'acme.parse-ticket',
|
|
76
|
+
inputMapping: { ticket: { path: 'runInput.ticket' } },
|
|
77
|
+
},
|
|
78
|
+
{
|
|
79
|
+
id: 'classify',
|
|
80
|
+
kind: 'agent',
|
|
81
|
+
ref: 'acme.ticket-classifier', // an agent with a typed `output`
|
|
82
|
+
inputMapping: { text: { path: 'nodeOutputs.parse.text' } },
|
|
83
|
+
config: { parameters: { product: 'acme-cloud' } },
|
|
84
|
+
},
|
|
85
|
+
{
|
|
86
|
+
id: 'billing',
|
|
87
|
+
kind: 'tool',
|
|
88
|
+
ref: 'acme.lookup-invoice',
|
|
89
|
+
inputMapping: { customerId: { path: 'runInput.ticket.customerId' } },
|
|
90
|
+
},
|
|
91
|
+
{
|
|
92
|
+
id: 'reply',
|
|
93
|
+
kind: 'tool',
|
|
94
|
+
ref: 'acme.draft-reply',
|
|
95
|
+
inputMapping: {
|
|
96
|
+
category: { path: 'nodeOutputs.classify.output.category' },
|
|
97
|
+
invoice: { path: 'nodeOutputs.billing.invoice' }, // absent when billing didn't run
|
|
98
|
+
},
|
|
99
|
+
},
|
|
100
|
+
],
|
|
101
|
+
edges: [
|
|
102
|
+
{ id: 'e0', from: '$start', to: 'parse' },
|
|
103
|
+
{ id: 'e1', from: 'parse', to: 'classify' },
|
|
104
|
+
{ id: 'e2', from: 'classify', to: 'billing', when: isBilling },
|
|
105
|
+
{ id: 'e3', from: 'classify', to: 'reply', when: { op: 'not', child: isBilling } },
|
|
106
|
+
{ id: 'e4', from: 'billing', to: 'reply' },
|
|
107
|
+
{ id: 'e5', from: 'reply', to: '$end' },
|
|
108
|
+
],
|
|
109
|
+
output: {
|
|
110
|
+
mapping: {
|
|
111
|
+
category: { path: 'nodeOutputs.classify.output.category' },
|
|
112
|
+
reply: { path: 'nodeOutputs.reply.text' },
|
|
113
|
+
},
|
|
114
|
+
schema: {
|
|
115
|
+
type: 'object',
|
|
116
|
+
properties: { category: { type: 'string' }, reply: { type: 'string' } },
|
|
117
|
+
required: ['category', 'reply'],
|
|
118
|
+
},
|
|
119
|
+
},
|
|
120
|
+
});
|
|
121
|
+
|
|
122
|
+
if (defined.kind === 'err') {
|
|
123
|
+
throw new Error(`acme.triage-ticket failed to compile: ${defined.error.message}`);
|
|
124
|
+
}
|
|
125
|
+
|
|
126
|
+
export default defined.value;
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
`defineFlow` checks the shape where it is written: ids, the version,
|
|
130
|
+
edges between nodes that exist, `$start` / `$end`, no cycles, paths
|
|
131
|
+
rooted where they may be. It doesn't check that `acme.parse-ticket`
|
|
132
|
+
exists. That is checked when a run starts (see "Running a flow").
|
|
133
|
+
|
|
134
|
+
## Nodes
|
|
135
|
+
|
|
136
|
+
- **`kind: 'tool'`** runs the tool `ref` (its id). The tool's input is
|
|
137
|
+
what the node's `inputMapping` builds, else the output of the node's
|
|
138
|
+
single upstream node (the run input after `$start`). It is validated
|
|
139
|
+
against the tool's input schema, so a mismatch fails the step with
|
|
140
|
+
`input-validation-failed`. The node's output is the tool's return value.
|
|
141
|
+
- **`kind: 'agent'`** runs one turn of the agent `ref` as a child run of
|
|
142
|
+
the flow run. The agent gets the node's input in two ways:
|
|
143
|
+
- as **structured input**: `{{ input.text }}` in its instructions;
|
|
144
|
+
- as its user message (the input as JSON).
|
|
145
|
+
|
|
146
|
+
`config.parameters` fills the agent's `parameters` (string, number or
|
|
147
|
+
boolean values). `config.version` pins an agent version; without it
|
|
148
|
+
the latest active version runs.
|
|
149
|
+
|
|
150
|
+
The node's output:
|
|
151
|
+
- `output` is the agent's typed answer (its `output` schema);
|
|
152
|
+
- `text` is the answer as text;
|
|
153
|
+
- `runId` and `conversationId` belong to the child run.
|
|
154
|
+
|
|
155
|
+
Read a field as `nodeOutputs.<node>.output.<field>`. An answer that
|
|
156
|
+
doesn't fit the agent's `output` schema, after its repairs, fails the
|
|
157
|
+
step with `output-schema-violation`.
|
|
158
|
+
|
|
159
|
+
An approval inside the agent's turn parks the flow until it's decided.
|
|
160
|
+
- **`kind: 'loop'`** repeats a body: `loopKind: 'foreach'` once per
|
|
161
|
+
element of `iterateOver` (`concurrency` up to 32 in parallel), or
|
|
162
|
+
`loopKind: 'while'` until `exitCondition`.
|
|
163
|
+
- The body has its own nodes and edges, with `$loop-start` /
|
|
164
|
+
`$loop-end`; the element is the body's input.
|
|
165
|
+
- `maxIterations` and `outputSchema` are required.
|
|
166
|
+
- The loop's output is `finalOutput`, plus `outputs` with
|
|
167
|
+
`collectAllIterations: true`.
|
|
168
|
+
- Node ids must be unique across the whole flow, bodies included.
|
|
169
|
+
- **`kind: 'fanout'`** runs several handlers on the same input at once,
|
|
170
|
+
each a `branch` with an `outputSchema`. `convergence` decides the
|
|
171
|
+
result:
|
|
172
|
+
- `'all-succeed'`: every branch must succeed;
|
|
173
|
+
- `'any-succeed'`: the first success wins;
|
|
174
|
+
- `'settle-all'`: wait for every branch and report each.
|
|
175
|
+
- **`kind: 'subgraph'`** (a sub-flow) is part of the flow schema, but a
|
|
176
|
+
run refuses it today (`flow-unbound`). Inline the steps instead.
|
|
177
|
+
|
|
178
|
+
## Edges and conditions
|
|
179
|
+
|
|
180
|
+
An edge goes from a node (or `$start`) to a node (or `$end`). Without
|
|
181
|
+
`when` it fires when its source completes; with `when` it fires only if
|
|
182
|
+
the condition is true. Conditions are JSON:
|
|
183
|
+
|
|
184
|
+
| Operator | Shape |
|
|
185
|
+
|---|---|
|
|
186
|
+
| `eq` `ne` `lt` `lte` `gt` `gte` | `{ op, left, right }` |
|
|
187
|
+
| `in` `notIn` | `{ op, value, set }` |
|
|
188
|
+
| `exists` `notExists` `truthy` `falsy` | `{ op, value }` |
|
|
189
|
+
| `and` `or` | `{ op, children: [...] }` |
|
|
190
|
+
| `not` | `{ op, child }` |
|
|
191
|
+
|
|
192
|
+
Each operand is `{ literal: … }` or `{ path: … }`. When a path doesn't
|
|
193
|
+
resolve, `eq`, `lt`, `lte`, `gt` and `gte` are false and `ne` is true. So
|
|
194
|
+
for the "otherwise" branch, write `not` around the condition (as above),
|
|
195
|
+
rather than a second comparison: it covers exactly what the first edge
|
|
196
|
+
doesn't.
|
|
197
|
+
|
|
198
|
+
**Joining branches.** A node with several incoming edges runs once every
|
|
199
|
+
one of them is decided and at least one fired. In the example, `reply`
|
|
200
|
+
runs after `billing` on the billing branch, and straight after `classify`
|
|
201
|
+
otherwise. A node none of whose incoming edges fired is skipped, and so
|
|
202
|
+
is everything only it leads to.
|
|
203
|
+
|
|
204
|
+
**Edge policy** (`policy` on the edge into a node with a single incoming
|
|
205
|
+
edge):
|
|
206
|
+
- `retry: { maxAttempts, delayMs?, backoff?, maxDelayMs? }`: up to 10
|
|
207
|
+
attempts in all;
|
|
208
|
+
- `timeoutMs`: a step that takes longer fails with `reason: 'timeout'`;
|
|
209
|
+
- `concurrencyKey`: at most one such step at a time in the tenant;
|
|
210
|
+
- `priority`: −100 to 100.
|
|
211
|
+
|
|
212
|
+
A node with several incoming edges ignores them.
|
|
213
|
+
|
|
214
|
+
## Inputs and the output
|
|
215
|
+
|
|
216
|
+
`inputMapping` maps each key to a `{ literal }` or a `{ path }`. Paths are
|
|
217
|
+
dot-separated (a number segment indexes an array: `items.0.sku`), rooted at:
|
|
218
|
+
- `runInput.…`: the input the run was started with;
|
|
219
|
+
- `nodeOutputs.<nodeId>.…`: a step's output. For an agent step, add
|
|
220
|
+
`.output.<field>` to read its typed answer;
|
|
221
|
+
- `state.…`: values written by the runtime's own handlers. Pack tools
|
|
222
|
+
don't write it, so use `nodeOutputs`.
|
|
223
|
+
|
|
224
|
+
A path that doesn't resolve leaves its key out. A step after a branch
|
|
225
|
+
that didn't run gets no `invoice` key at all, rather than `invoice:
|
|
226
|
+
undefined`. Make that key optional in the tool's input schema.
|
|
227
|
+
|
|
228
|
+
`output` is what the run returns: a `mapping` resolved when the run
|
|
229
|
+
finishes, checked against `schema` if you give one. A run whose output
|
|
230
|
+
doesn't match fails. Without `output`, the run returns the output of the
|
|
231
|
+
step that reached `$end`.
|
|
232
|
+
|
|
233
|
+
## Running a flow
|
|
234
|
+
|
|
235
|
+
From another terminal in the pack directory, while `kindgi dev` runs:
|
|
236
|
+
|
|
237
|
+
```sh
|
|
238
|
+
kindgi runs start --flow=acme.triage-ticket --input='{"ticket":{"customerId":"c-1","body":"Charged twice"}}'
|
|
239
|
+
kindgi runs start --flow=acme.triage-ticket --input=@ticket.json --no-wait # the run id now; it finishes in the background
|
|
240
|
+
kindgi runs start --flow=acme.triage-ticket --input=@ticket.json --dry-run # runs only read-only tools
|
|
241
|
+
kindgi runs get <run-id> # status, output, failureMessage
|
|
242
|
+
kindgi runs journal <run-id> # every step.started / step.completed / edge.evaluated
|
|
243
|
+
kindgi runs stream <run-id> # follow a running one
|
|
244
|
+
kindgi runs cancel <run-id>
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
- **A refusal before the run exists:** `422 flow-unbound` names the
|
|
248
|
+
nodes a run can't bind: a tool or agent id the tenant doesn't have, or
|
|
249
|
+
a sub-flow. Fix the ids; nothing ran.
|
|
250
|
+
- **A failed step fails the run**, and `failureMessage` says which step
|
|
251
|
+
and why. Retry it on its edge with `policy.retry` only if running the
|
|
252
|
+
step twice is safe.
|
|
253
|
+
- **`--no-wait`** is how an application starts runs (`options: { wait:
|
|
254
|
+
false }` with `@kindgi/sdk/client`). It answers with the run id at once;
|
|
255
|
+
poll `GET /v1/runs/<id>` or follow the stream.
|
|
256
|
+
- **`--dry-run`** runs a tool only if it's declared read-only:
|
|
257
|
+
`mutating: false`, and no `writes`, `deletes`, `spawns-run`,
|
|
258
|
+
`emits-event` or `external-side-effect` effect. The first other tool
|
|
259
|
+
stops the run with `dry-run-effectful-tool`, and everything before it
|
|
260
|
+
really ran. That's useful for checking the wiring without the writes.
|
|
261
|
+
|
|
262
|
+
## Iterating on a flow
|
|
263
|
+
|
|
264
|
+
Save the file and `kindgi dev` re-indexes; the next run uses the new
|
|
265
|
+
definition, with no restart. A run already in flight keeps the version it
|
|
266
|
+
started on. Bump `version` when callers' contract changes (the input or
|
|
267
|
+
the output), not on every save.
|
|
268
|
+
|
|
269
|
+
## Common mistakes
|
|
270
|
+
|
|
271
|
+
1. **Building a flow without asking what goes in and comes out.** The
|
|
272
|
+
pack's `echo-flow` proves the runtime works. It isn't a template for
|
|
273
|
+
the user's flow.
|
|
274
|
+
2. **A second comparison for "otherwise".** On a path that may be
|
|
275
|
+
missing, `eq` is false and `ne` is true, and `lt`/`gt` are both false,
|
|
276
|
+
so a hand-written opposite can miss a case or overlap. Use `not` around
|
|
277
|
+
the positive condition: it covers exactly what the first edge doesn't.
|
|
278
|
+
3. **Reading an agent step's answer at `nodeOutputs.<step>.<field>`.**
|
|
279
|
+
The typed answer is under `.output`: `nodeOutputs.<step>.output.<field>`.
|
|
280
|
+
An agent without an `output` schema has only `text`.
|
|
281
|
+
4. **A required input key fed by a branch that may not run.** The key is
|
|
282
|
+
omitted, the tool's input check fails, and so does the step. Make the
|
|
283
|
+
key optional in the tool's input schema.
|
|
284
|
+
5. **`mutating: false` on a tool that writes.** A dry run runs it for
|
|
285
|
+
real unless its `effects` declare the write, and an agent's tool
|
|
286
|
+
gates (when on, with no rule for it) let it through unasked.
|
|
287
|
+
6. **A read-only tool without `mutating: false`.** Leaving it out counts
|
|
288
|
+
as mutating: a dry run stops at the tool, and an agent's tool gates
|
|
289
|
+
ask before it. kindgi-authoring-tools has the details.
|
|
290
|
+
7. **A sub-flow node.** A run refuses it (`flow-unbound`) until
|
|
291
|
+
sub-flows are supported.
|
|
292
|
+
8. **Duplicate node ids inside a loop body.** Ids are unique across the
|
|
293
|
+
whole flow, bodies included.
|
|
294
|
+
9. **Missing `Result` unwrap.** Check `defined.kind === 'err'` and throw,
|
|
295
|
+
so a broken flow fails when its module loads, not on the first run.
|
|
296
|
+
|
|
297
|
+
## When the framework itself is the problem
|
|
298
|
+
|
|
299
|
+
If the bug is in Kindgi or `@kindgi/sdk` (a step's output missing a
|
|
300
|
+
field, a condition that evaluates wrongly, a misleading error) and not in
|
|
301
|
+
the pack's code, load `kindgi-framework-feedback` and file it with
|
|
302
|
+
`kindgi feedback write`.
|
|
@@ -0,0 +1,297 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: kindgi-authoring-guardrails
|
|
3
|
+
description: >
|
|
4
|
+
Covers writing guardrails (safety checks) for a Kindgi pack: the check
|
|
5
|
+
implementation via defineCheck from @kindgi/sdk/define, the guardrail
|
|
6
|
+
declaration a pack file default-exports (the indexer's shape), the
|
|
7
|
+
three kinds (zero-llm / llm-judge / external), action semantics
|
|
8
|
+
(halt / retry / escalate / log-only / compensate), severity levels,
|
|
9
|
+
scope selectors, config schemas via Zod or JSON Schema, validating a
|
|
10
|
+
declaration with defineGuardrail from @kindgi/guardrails, and how
|
|
11
|
+
guardrails reach agents. Load this whenever you are authoring or
|
|
12
|
+
editing code inside a pack's guardrails/ directory, defining a check,
|
|
13
|
+
or wiring a guardrail onto an agent. Authoring tools is covered by
|
|
14
|
+
kindgi-authoring-tools; authoring agents is covered by
|
|
15
|
+
kindgi-authoring-agents.
|
|
16
|
+
type: core
|
|
17
|
+
library: "@kindgi/sdk"
|
|
18
|
+
version: "0.3.6"
|
|
19
|
+
sdk_version: "0.0.0"
|
|
20
|
+
pack_languages: [node]
|
|
21
|
+
sources:
|
|
22
|
+
- packages/guardrails/src/types.ts
|
|
23
|
+
- packages/guardrails/src/define-check.ts
|
|
24
|
+
- packages/guardrails/src/define.ts
|
|
25
|
+
- packages/guardrails/src/judge.ts
|
|
26
|
+
- packages/guardrails/src/checks.ts
|
|
27
|
+
- packages/handler-runtime/src/kindgi-index.ts
|
|
28
|
+
- packages/handler-runtime/src/handler-runner.ts
|
|
29
|
+
---
|
|
30
|
+
|
|
31
|
+
# Authoring Kindgi guardrails
|
|
32
|
+
|
|
33
|
+
> **Running `kindgi`:** the CLI is a devDependency of the project (`@kindgi/cli`),
|
|
34
|
+
> not a global command. Run it through the project's package manager —
|
|
35
|
+
> `pnpm exec kindgi …`, `npx --no kindgi …` (npm), `yarn kindgi …` or
|
|
36
|
+
> `bun run kindgi …`. Commands below are written `kindgi …` for brevity.
|
|
37
|
+
|
|
38
|
+
A **guardrail** is a safety rule an agent turn must satisfy. It combines
|
|
39
|
+
a **check** (the function that inspects the turn's trace) with an
|
|
40
|
+
**action** (what happens when the check fails). In a pack, a guardrail
|
|
41
|
+
lives at `guardrails/<name>/index.ts`. For an agent turn, the runtime
|
|
42
|
+
evaluates every guardrail the agent lists once, on the final response,
|
|
43
|
+
before the response is stored.
|
|
44
|
+
|
|
45
|
+
## Mental model: guardrail vs check
|
|
46
|
+
|
|
47
|
+
- **Check** — the implementation. `defineCheck({ id, kind, configSchema?,
|
|
48
|
+
evaluate })` from `@kindgi/sdk/define` returns a registered check whose
|
|
49
|
+
`evaluate(config, trace, bindings)` resolves to `{ passed, reason? }`.
|
|
50
|
+
Zero-llm checks are pure over the trace.
|
|
51
|
+
- **Guardrail** — the declaration: `id`, `kind`, the `check` it uses,
|
|
52
|
+
the check's `config`, and `action` / `severity` / `scope`. Its type is
|
|
53
|
+
`Guardrail` from `@kindgi/guardrails`. One check can back many
|
|
54
|
+
guardrails with different configs.
|
|
55
|
+
- **Built-in checks** (`BUILT_IN_CHECK_IDS` in `@kindgi/guardrails`):
|
|
56
|
+
`must-cite`, `never-call-tool`, `max-tool-calls`, `output-matches`,
|
|
57
|
+
`tool-order`, `required-substring`, `forbidden-substring`. A guardrail
|
|
58
|
+
can name one of these instead of shipping its own check.
|
|
59
|
+
|
|
60
|
+
`@kindgi/sdk` exports `defineCheck` but no helper for the guardrail
|
|
61
|
+
itself: a pack file default-exports the declaration as a plain object.
|
|
62
|
+
|
|
63
|
+
## A pack guardrail file (zero-llm)
|
|
64
|
+
|
|
65
|
+
```ts
|
|
66
|
+
// guardrails/no-fabricated-quotes/index.ts
|
|
67
|
+
import { defineCheck } from '@kindgi/sdk/define';
|
|
68
|
+
import { z } from 'zod';
|
|
69
|
+
|
|
70
|
+
// The check implementation. The pack service calls `check.evaluate`.
|
|
71
|
+
export const check = defineCheck({
|
|
72
|
+
id: 'acme.checks.no-fabricated-quotes',
|
|
73
|
+
kind: 'zero-llm',
|
|
74
|
+
configSchema: z.object({ minPrecedentCalls: z.number().int().min(0).optional() }),
|
|
75
|
+
evaluate: async (config, trace) => {
|
|
76
|
+
const needed = config.minPrecedentCalls ?? 1;
|
|
77
|
+
const precedentCalls = trace.toolCalls.filter((c) => c.toolName === 'acme.fetch-precedent');
|
|
78
|
+
if (precedentCalls.length < needed) {
|
|
79
|
+
return {
|
|
80
|
+
passed: false,
|
|
81
|
+
reason: `Only ${precedentCalls.length} precedent lookups (need ${needed}+).`,
|
|
82
|
+
};
|
|
83
|
+
}
|
|
84
|
+
return { passed: true };
|
|
85
|
+
},
|
|
86
|
+
});
|
|
87
|
+
|
|
88
|
+
// The guardrail declaration the indexer reads.
|
|
89
|
+
export default {
|
|
90
|
+
id: 'acme.no-fabricated-quotes',
|
|
91
|
+
name: 'No fabricated quotations',
|
|
92
|
+
kind: 'zero-llm',
|
|
93
|
+
check,
|
|
94
|
+
action: { 'on-violation': 'halt' },
|
|
95
|
+
severity: 'critical',
|
|
96
|
+
};
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
How the pack tooling reads this file:
|
|
100
|
+
|
|
101
|
+
- The **indexer** (`packages/handler-runtime/src/kindgi-index.ts`)
|
|
102
|
+
recognises a guardrail by a default export with a `kind` and an
|
|
103
|
+
`action` object carrying `on-violation`. It records `id`, `name`,
|
|
104
|
+
`kind`, `action`, `severity`, `scope`, `sandbox` / `limits` /
|
|
105
|
+
`network`, and the check: its id (`check` may be the check id as a
|
|
106
|
+
string, or the check object) and its config schema (from the check's
|
|
107
|
+
`configZod`, `configSchema` or `configJsonSchema`, or a top-level
|
|
108
|
+
`configZod` / `configSchema`), and the declaration's `config` — what
|
|
109
|
+
the check runs with. It does not record `description`, `budget` or
|
|
110
|
+
`judgeCapabilities`.
|
|
111
|
+
- The **pack service** loads the same module to run the check. It uses
|
|
112
|
+
the module's `evaluate` export, or the `default` / `check` export when
|
|
113
|
+
that is a function or has an `evaluate` method — here, the named
|
|
114
|
+
`check` export.
|
|
115
|
+
|
|
116
|
+
## Validating a declaration in-process
|
|
117
|
+
|
|
118
|
+
`defineGuardrail(spec, checks)` from `@kindgi/guardrails` validates a
|
|
119
|
+
`Guardrail` against the wire schema and a check registry: the check id
|
|
120
|
+
must be registered, the check's `kind` must match, and `config` must
|
|
121
|
+
pass the check's config schema. It returns a `Result`; use it in tests
|
|
122
|
+
or wherever guardrails are registered in-process.
|
|
123
|
+
|
|
124
|
+
```ts
|
|
125
|
+
import { createCheckRegistry, defineGuardrail } from '@kindgi/guardrails';
|
|
126
|
+
import type { GuardrailId } from '@kindgi/sdk/types';
|
|
127
|
+
|
|
128
|
+
import { check } from './index.js';
|
|
129
|
+
|
|
130
|
+
const checks = createCheckRegistry([check]); // built-in checks are included
|
|
131
|
+
const defined = defineGuardrail(
|
|
132
|
+
{
|
|
133
|
+
id: 'acme.no-fabricated-quotes' as GuardrailId,
|
|
134
|
+
kind: 'zero-llm',
|
|
135
|
+
check: check.id,
|
|
136
|
+
config: { minPrecedentCalls: 2 },
|
|
137
|
+
action: { 'on-violation': 'halt' },
|
|
138
|
+
severity: 'critical',
|
|
139
|
+
},
|
|
140
|
+
checks,
|
|
141
|
+
);
|
|
142
|
+
if (defined.kind === 'err') {
|
|
143
|
+
throw new Error(`acme.no-fabricated-quotes: ${defined.error.message}`);
|
|
144
|
+
}
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
Registering a guardrail through the API (`POST /v1/guardrails`, or
|
|
148
|
+
`client.guardrails.author(spec, { projectId })` in `@kindgi/sdk/client`)
|
|
149
|
+
stores the declaration only; the check it names must already be
|
|
150
|
+
available to the runtime that evaluates it.
|
|
151
|
+
|
|
152
|
+
## Field-by-field
|
|
153
|
+
|
|
154
|
+
- **`id`** — `<pack-id>.<guardrail-name>` (kebab-case, dot-namespaced).
|
|
155
|
+
Name the ASSERTION as a positive rule (e.g. `no-fabricated-quotes`,
|
|
156
|
+
`response-not-empty`, `must-cite-source`).
|
|
157
|
+
- **`kind`**:
|
|
158
|
+
- `'zero-llm'` — pure function over the trace. Fast, deterministic,
|
|
159
|
+
free. **The default choice for most safety rules.**
|
|
160
|
+
- `'llm-judge'` — a model scores the turn against a rubric. Costs
|
|
161
|
+
money; requires `judgeCapabilities`. See below.
|
|
162
|
+
- `'external'` — evaluated outside the engine. The built-in
|
|
163
|
+
`external` strategy returns an `invalid-guardrail` error; a caller
|
|
164
|
+
that wants external evaluation registers its own strategy.
|
|
165
|
+
- **`check`** — the id of a registered check (built-in, or one built
|
|
166
|
+
with `defineCheck`). In a pack file it may also be the check object.
|
|
167
|
+
- **`config`** — the check's parameters, validated against the check's
|
|
168
|
+
`configSchema` by `defineGuardrail`. In a pack, the declaration's
|
|
169
|
+
`config` goes into the index and the check runs with it; without one
|
|
170
|
+
it runs with `{}`. A declaration a pack file default-exports isn't run
|
|
171
|
+
through `defineGuardrail`, so nothing validates its `config`: keep it
|
|
172
|
+
valid against the schema yourself. `evaluate` receives the config as
|
|
173
|
+
declared —
|
|
174
|
+
schema defaults are not filled in — so handle absent optional fields.
|
|
175
|
+
- **`action.on-violation`** — `'halt'`, `'retry'` (with
|
|
176
|
+
`retry.maxAttempts`, 1–10), `'escalate'` (with `escalateTo`),
|
|
177
|
+
`'log-only'`, `'compensate'` (with `compensateWith`, a tool id). In an
|
|
178
|
+
agent turn, a failed `halt` guardrail fails the turn with
|
|
179
|
+
`guardrail-violation` and the response is not stored; failures with
|
|
180
|
+
any other action are reported in `AgentTurnResult.violations` and the
|
|
181
|
+
turn completes. The action handlers in `@kindgi/guardrails`
|
|
182
|
+
(`retryHandler`, `escalateHandler`, `compensateHandler`, …) record the
|
|
183
|
+
intent for callers that act on it. In 0.1 the runtime acts only on
|
|
184
|
+
`halt`: `retry`, `escalate` and `compensate` are recorded on the
|
|
185
|
+
violation, with no second attempt, escalation or compensating call.
|
|
186
|
+
- **`severity`** — `'info'` / `'warn'` / `'error'` (the default) /
|
|
187
|
+
`'critical'`. Orthogonal to `action`: logs and dashboards group by
|
|
188
|
+
severity; execution follows the action. A `log-only` guardrail can
|
|
189
|
+
still be `'critical'`.
|
|
190
|
+
- **`scope`** — when the guardrail applies. `{ when: 'always' }` fires
|
|
191
|
+
everywhere; `{ when: 'ci-only' }` blocks CI but not runtime;
|
|
192
|
+
`{ when: 'runtime-only' }` enforces at runtime but not CI. `agents`,
|
|
193
|
+
`flows` and `tenants` lists narrow it further.
|
|
194
|
+
- **`budget`** — `{ maxCostUsd?, maxLatencyMs? }`, relevant to
|
|
195
|
+
`llm-judge`. Declarative: the runtime does not enforce it.
|
|
196
|
+
- **`judgeCapabilities`** — for `llm-judge`: the capability
|
|
197
|
+
declaration used to route the judge model.
|
|
198
|
+
|
|
199
|
+
## LLM-judge guardrail (costs money)
|
|
200
|
+
|
|
201
|
+
An `llm-judge` guardrail does not run custom check code: the engine's
|
|
202
|
+
`llm-judge` strategy sends the turn's trace and the rubric in `config`
|
|
203
|
+
(`{ rubric, responseFormat?, threshold?, temperature? }`) to a model
|
|
204
|
+
routed through `judgeCapabilities`, and parses a PASS/FAIL or a score.
|
|
205
|
+
|
|
206
|
+
```ts
|
|
207
|
+
import type { Guardrail } from '@kindgi/guardrails';
|
|
208
|
+
import type { GuardrailId } from '@kindgi/sdk/types';
|
|
209
|
+
|
|
210
|
+
export const toneProfessional: Guardrail = {
|
|
211
|
+
id: 'acme.tone-professional' as GuardrailId,
|
|
212
|
+
kind: 'llm-judge',
|
|
213
|
+
// Required by the `Guardrail` type; the llm-judge strategy judges
|
|
214
|
+
// with `config` and does not call this check.
|
|
215
|
+
check: 'acme.checks.tone-professional',
|
|
216
|
+
config: {
|
|
217
|
+
rubric: 'The response is professional in tone and contains no slang.',
|
|
218
|
+
responseFormat: 'pass-fail',
|
|
219
|
+
},
|
|
220
|
+
judgeCapabilities: { needs: [{ feature: 'structured-output' }] },
|
|
221
|
+
action: { 'on-violation': 'log-only' },
|
|
222
|
+
severity: 'warn',
|
|
223
|
+
budget: { maxCostUsd: 0.01, maxLatencyMs: 5000 }, // declarative, not enforced
|
|
224
|
+
};
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
The judge is resolved from the provider registry passed in the
|
|
228
|
+
evaluation bindings (or a pinned `judgeProvider`), under the tenant
|
|
229
|
+
policy in those bindings when one is passed. Checks that run in a pack are called with empty
|
|
230
|
+
`bindings` — no provider registry — so a pack check cannot call a model
|
|
231
|
+
itself; use `kind: 'llm-judge'` for model-based rules.
|
|
232
|
+
|
|
233
|
+
## Wiring the guardrail onto an agent
|
|
234
|
+
|
|
235
|
+
Agents reference guardrails by id:
|
|
236
|
+
|
|
237
|
+
```ts
|
|
238
|
+
// agents/brief-writer/index.ts
|
|
239
|
+
guardrails: ['acme.no-fabricated-quotes'],
|
|
240
|
+
```
|
|
241
|
+
|
|
242
|
+
At the start of each turn, the runtime resolves these ids against the
|
|
243
|
+
guardrails available to the run. An id that isn't registered fails the
|
|
244
|
+
turn before the model is called (`Error [invalid-request]: Agent "…"
|
|
245
|
+
references guardrails not in the registry: <id>`), so register the
|
|
246
|
+
guardrail before an agent references it.
|
|
247
|
+
|
|
248
|
+
## Changing a guardrail
|
|
249
|
+
|
|
250
|
+
Guardrails have no `version` field; the id is the stable identifier.
|
|
251
|
+
Changing a guardrail's check, config, severity or action changes
|
|
252
|
+
behavior for every agent that references it. When you tighten a rule
|
|
253
|
+
(raise severity from `warn` to `error`, switch the action from
|
|
254
|
+
`log-only` to `halt`), check whether the agents that reference it are
|
|
255
|
+
ready for the stricter enforcement.
|
|
256
|
+
|
|
257
|
+
## Common mistakes
|
|
258
|
+
|
|
259
|
+
1. **Confusing guardrail and check.** The id in `agent.guardrails: [...]`
|
|
260
|
+
is the GUARDRAIL id, not the check id. The agent binds to
|
|
261
|
+
guardrails; guardrails reference checks.
|
|
262
|
+
|
|
263
|
+
2. **A check whose `evaluate` always returns `passed: true`.** If you
|
|
264
|
+
are stubbing the check, give the guardrail `action: { 'on-violation':
|
|
265
|
+
'log-only' }` so it is honest about not being enforced.
|
|
266
|
+
|
|
267
|
+
3. **Calling a model from a pack check.** Pack checks receive empty
|
|
268
|
+
`bindings`; there is no provider registry to route through. Declare
|
|
269
|
+
an `llm-judge` guardrail with a rubric instead.
|
|
270
|
+
|
|
271
|
+
4. **Not declaring `configSchema`.** Without it, `config` is
|
|
272
|
+
`Record<string, unknown>` — no validation, no editor completion,
|
|
273
|
+
silent typos. Prefer Zod for TS-side inference on
|
|
274
|
+
`evaluate(config, ...)`.
|
|
275
|
+
|
|
276
|
+
5. **Ignoring the `Result` from `defineGuardrail`.** It returns
|
|
277
|
+
`Result<Guardrail, …>`; check `kind` and throw at load time.
|
|
278
|
+
`defineCheck` itself throws when its `configSchema` can't be
|
|
279
|
+
compiled.
|
|
280
|
+
|
|
281
|
+
## References
|
|
282
|
+
|
|
283
|
+
- Type surface: hover any `@kindgi/sdk/define` export for full JSDoc;
|
|
284
|
+
`Guardrail`, `defineGuardrail` and the built-in checks are in
|
|
285
|
+
`@kindgi/guardrails`.
|
|
286
|
+
- API reference: https://docs.kindgi.com/v0.1/reference/typescript/sdk/kindgi/sdk/define/
|
|
287
|
+
- Built-in check implementations: `packages/guardrails/src/checks.ts`.
|
|
288
|
+
|
|
289
|
+
## When the framework itself is the problem
|
|
290
|
+
|
|
291
|
+
If you diagnose that the bug lives in Kindgi/`@kindgi/sdk` itself
|
|
292
|
+
(guardrail runtime dropping context fields, check-sandbox dispatch
|
|
293
|
+
regression, misleading error message, CLI friction) — not in the
|
|
294
|
+
pack's own code — load the `kindgi-framework-feedback` skill and file
|
|
295
|
+
a structured report with `kindgi feedback write`. That diagnostic is
|
|
296
|
+
high-signal input the maintainers can act on; don't let it disappear
|
|
297
|
+
into the transcript.
|