@terreno/mcp 57.8.0 → 57.10.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (115) hide show
  1. package/dist/docs/upgrades/57.10.1.md +54 -0
  2. package/dist/docs/upgrades/57.9.0.md +28 -0
  3. package/dist/docs/versioned/next/README.md +1 -0
  4. package/dist/docs/versioned/next/explanation/README.md +4 -0
  5. package/dist/docs/versioned/next/explanation/admin-plugin-frontend.md +2 -2
  6. package/dist/docs/versioned/next/explanation/agent-ui-asks.md +299 -0
  7. package/dist/docs/versioned/next/explanation/agent-ui-blocks.md +67 -0
  8. package/dist/docs/versioned/next/explanation/ai-observability.md +101 -0
  9. package/dist/docs/versioned/next/explanation/charts.md +12 -0
  10. package/dist/docs/versioned/next/explanation/durable-agent-harness.md +531 -0
  11. package/dist/docs/versioned/next/explanation/example-coverage.md +4 -2
  12. package/dist/docs/versioned/next/explanation/loop-engineering.md +1 -1
  13. package/dist/docs/versioned/next/explanation/roadmap-seed-issues.md +49 -1
  14. package/dist/docs/versioned/next/explanation/typescript-configuration.md +1 -1
  15. package/dist/docs/versioned/next/explanation/versioning-policy.md +1 -1
  16. package/dist/docs/versioned/next/how-to/README.md +7 -1
  17. package/dist/docs/versioned/next/how-to/admin-import-prebuilt.md +1 -1
  18. package/dist/docs/versioned/next/how-to/agent-ui-asks.md +730 -0
  19. package/dist/docs/versioned/next/how-to/agent-ui-blocks.md +104 -0
  20. package/dist/docs/versioned/next/how-to/ai-feature-development.md +156 -0
  21. package/dist/docs/versioned/next/how-to/build-a-durable-workflow.md +232 -0
  22. package/dist/docs/versioned/next/how-to/charts-and-dashboards.md +7 -0
  23. package/dist/docs/versioned/next/how-to/circleci.md +9 -6
  24. package/dist/docs/versioned/next/how-to/compare-chart-rendered-snapshots.md +8 -0
  25. package/dist/docs/versioned/next/how-to/install-agent-skills.md +1 -1
  26. package/dist/docs/versioned/next/how-to/observe-llm-calls.md +197 -0
  27. package/dist/docs/versioned/next/how-to/run-tests-locally.md +2 -1
  28. package/dist/docs/versioned/next/how-to/ship-a-new-task-version.md +76 -0
  29. package/dist/docs/versioned/next/how-to/upgrade-terreno.md +1 -1
  30. package/dist/docs/versioned/next/implementationPlans/agent-ui-asks.md +688 -0
  31. package/dist/docs/versioned/next/implementationPlans/agent-ui-blocks.md +10 -6
  32. package/dist/docs/versioned/next/implementationPlans/ai-observability.md +475 -157
  33. package/dist/docs/versioned/next/implementationPlans/b2b-platform-program.md +1 -1
  34. package/dist/docs/versioned/next/implementationPlans/charts-and-dashboards.md +3 -2
  35. package/dist/docs/versioned/next/implementationPlans/dashboard-chart-parity.md +275 -0
  36. package/dist/docs/versioned/next/implementationPlans/durable-agent-harness.md +340 -0
  37. package/dist/docs/versioned/next/implementationPlans/terreno-langfuse-integration.md +1 -1
  38. package/dist/docs/versioned/next/reference/README.md +3 -0
  39. package/dist/docs/versioned/next/reference/admin-config.md +2 -0
  40. package/dist/docs/versioned/next/reference/admin-frontend.md +216 -0
  41. package/dist/docs/versioned/next/reference/agent-ui-asks.md +1059 -0
  42. package/dist/docs/versioned/next/reference/ai-harness.md +1881 -0
  43. package/dist/docs/versioned/next/reference/ai.md +283 -28
  44. package/dist/docs/versioned/next/reference/api.md +46 -0
  45. package/dist/docs/versioned/next/reference/blocks.md +178 -0
  46. package/dist/docs/versioned/next/reference/environment-variables.md +2 -1
  47. package/dist/docs/versioned/next/reference/feature-flags.md +1 -1
  48. package/dist/docs/versioned/next/reference/mcp-server.md +17 -0
  49. package/dist/docs/versioned/next/reference/test.md +11 -2
  50. package/dist/docs/versioned/next/reference/ui.md +418 -11
  51. package/dist/docs/versioned/next/tasks/agent-ui-asks.md +213 -0
  52. package/dist/docs/versioned/next/tasks/agent-ui-blocks.md +2 -1
  53. package/dist/docs/versioned/next/tasks/ai-observability.md +233 -104
  54. package/dist/docs/versioned/next/tasks/dashboard-chart-parity.md +145 -0
  55. package/dist/docs/versioned/next/tasks/durable-agent-harness.md +183 -0
  56. package/dist/tools.d.ts +4 -0
  57. package/dist/tools.d.ts.map +1 -1
  58. package/dist/tools.js +37 -0
  59. package/dist/tools.js.map +1 -1
  60. package/package.json +3 -2
  61. package/src/docs/upgrades/57.10.1.md +54 -0
  62. package/src/docs/upgrades/57.9.0.md +28 -0
  63. package/src/docs/versioned/next/README.md +1 -0
  64. package/src/docs/versioned/next/explanation/README.md +4 -0
  65. package/src/docs/versioned/next/explanation/admin-plugin-frontend.md +2 -2
  66. package/src/docs/versioned/next/explanation/agent-ui-asks.md +299 -0
  67. package/src/docs/versioned/next/explanation/agent-ui-blocks.md +67 -0
  68. package/src/docs/versioned/next/explanation/ai-observability.md +101 -0
  69. package/src/docs/versioned/next/explanation/charts.md +12 -0
  70. package/src/docs/versioned/next/explanation/durable-agent-harness.md +531 -0
  71. package/src/docs/versioned/next/explanation/example-coverage.md +4 -2
  72. package/src/docs/versioned/next/explanation/loop-engineering.md +1 -1
  73. package/src/docs/versioned/next/explanation/roadmap-seed-issues.md +49 -1
  74. package/src/docs/versioned/next/explanation/typescript-configuration.md +1 -1
  75. package/src/docs/versioned/next/explanation/versioning-policy.md +1 -1
  76. package/src/docs/versioned/next/how-to/README.md +7 -1
  77. package/src/docs/versioned/next/how-to/admin-import-prebuilt.md +1 -1
  78. package/src/docs/versioned/next/how-to/agent-ui-asks.md +730 -0
  79. package/src/docs/versioned/next/how-to/agent-ui-blocks.md +104 -0
  80. package/src/docs/versioned/next/how-to/ai-feature-development.md +156 -0
  81. package/src/docs/versioned/next/how-to/build-a-durable-workflow.md +232 -0
  82. package/src/docs/versioned/next/how-to/charts-and-dashboards.md +7 -0
  83. package/src/docs/versioned/next/how-to/circleci.md +9 -6
  84. package/src/docs/versioned/next/how-to/compare-chart-rendered-snapshots.md +8 -0
  85. package/src/docs/versioned/next/how-to/install-agent-skills.md +1 -1
  86. package/src/docs/versioned/next/how-to/observe-llm-calls.md +197 -0
  87. package/src/docs/versioned/next/how-to/run-tests-locally.md +2 -1
  88. package/src/docs/versioned/next/how-to/ship-a-new-task-version.md +76 -0
  89. package/src/docs/versioned/next/how-to/upgrade-terreno.md +1 -1
  90. package/src/docs/versioned/next/implementationPlans/agent-ui-asks.md +688 -0
  91. package/src/docs/versioned/next/implementationPlans/agent-ui-blocks.md +10 -6
  92. package/src/docs/versioned/next/implementationPlans/ai-observability.md +475 -157
  93. package/src/docs/versioned/next/implementationPlans/b2b-platform-program.md +1 -1
  94. package/src/docs/versioned/next/implementationPlans/charts-and-dashboards.md +3 -2
  95. package/src/docs/versioned/next/implementationPlans/dashboard-chart-parity.md +275 -0
  96. package/src/docs/versioned/next/implementationPlans/durable-agent-harness.md +340 -0
  97. package/src/docs/versioned/next/implementationPlans/terreno-langfuse-integration.md +1 -1
  98. package/src/docs/versioned/next/reference/README.md +3 -0
  99. package/src/docs/versioned/next/reference/admin-config.md +2 -0
  100. package/src/docs/versioned/next/reference/admin-frontend.md +216 -0
  101. package/src/docs/versioned/next/reference/agent-ui-asks.md +1059 -0
  102. package/src/docs/versioned/next/reference/ai-harness.md +1881 -0
  103. package/src/docs/versioned/next/reference/ai.md +283 -28
  104. package/src/docs/versioned/next/reference/api.md +46 -0
  105. package/src/docs/versioned/next/reference/blocks.md +178 -0
  106. package/src/docs/versioned/next/reference/environment-variables.md +2 -1
  107. package/src/docs/versioned/next/reference/feature-flags.md +1 -1
  108. package/src/docs/versioned/next/reference/mcp-server.md +17 -0
  109. package/src/docs/versioned/next/reference/test.md +11 -2
  110. package/src/docs/versioned/next/reference/ui.md +418 -11
  111. package/src/docs/versioned/next/tasks/agent-ui-asks.md +213 -0
  112. package/src/docs/versioned/next/tasks/agent-ui-blocks.md +2 -1
  113. package/src/docs/versioned/next/tasks/ai-observability.md +233 -104
  114. package/src/docs/versioned/next/tasks/dashboard-chart-parity.md +145 -0
  115. package/src/docs/versioned/next/tasks/durable-agent-harness.md +183 -0
@@ -0,0 +1,104 @@
1
+ # Validate a block document locally
2
+
3
+ `terreno-blocks` checks one whole-reply YAML or JSON document before you send it.
4
+
5
+ 1. Compile the package so the bin exists: `cd blocks && bun run compile`.
6
+ 2. Run `terreno-blocks validate path/to/document.yaml`.
7
+ 3. A valid document exits 0. Each problem prints as `path CODE message — fix` and the
8
+ process exits 1.
9
+ 4. Pass `-` to read the document from stdin.
10
+
11
+ The same check is `validateBlocks` in `@terreno/blocks`. Field tables are in the
12
+ [blocks reference](../reference/blocks.md).
13
+
14
+ ## Preview a document in the playground
15
+
16
+ 1. Open the component demo and choose **BlocksPlayground**.
17
+ 2. Start from Layout or Invalid, or paste your own document into the text area.
18
+ 3. The preview updates as you type. An invalid document shows the error banner and keeps
19
+ the raw text collapsed.
20
+
21
+ ## Require documents from the model
22
+
23
+ Pass `uiBlocks: true` to `addGptRoutes`. The model is told that its whole reply is one document. When the turn's text finishes, the route checks it and sends `{blocks: {ok, errors, warnings}}` before `{done}`.
24
+
25
+ ```ts
26
+ addGptRoutes(router, {
27
+ aiService,
28
+ uiBlocks: {
29
+ hostActions: {export_csv: {}},
30
+ repair: true,
31
+ },
32
+ });
33
+ ```
34
+
35
+ `hostActions` is the callback allowlist. A callback outside it fails with `UNKNOWN_HOST_ACTION`. `repair: true` runs one repair call and stores that document. A document that is still invalid is stored with a `Block validation errors:` note so the next turn sees the codes.
36
+
37
+ An actions block that omits `id` gets one before storage. A block document written in the same step as a tool call is kept when no later step has text, so the first reply still shows. The chat receives that document after ids are filled, repair runs, and html is sanitized, so an invalid draft is not shown. `{replace: "text"}` is sent only when text was already streamed and then changed.
38
+
39
+ Outside a chat turn, `AIService.generateBlocks({prompt})` returns one validated document. Temperature is 0. A failed check is repaired once. Pass `repair: false` to skip that retry. A second validation failure throws 422 and logs `metadata.errorCodes`. A model or network error throws 502 and is not repaired.
40
+
41
+ ## Store rows for a ref chart
42
+
43
+ 1. Turn `uiBlocks` on. `addGptRoutes` then mounts `GET /gpt/datasets/:id`.
44
+ 2. From a server tool, call `registerAiDataset({userId, historyId, columns, rows})`. The return type is `RegisteredDataset` (`datasetId`, `columns`, `rowCount`, `preview`, `stats`).
45
+ 3. Hand the model the returned `datasetId`. A `ref` dataset uses that id. The model cannot create one.
46
+ 4. The owner reads rows with `GET /gpt/datasets/:id?grain=week&limit=40`. Leave `page` off to downsample a line or area with LTTB. Pass `page` to paginate a table; `more` is true when another page remains.
47
+ 5. `datasetTtlDays: 0` (the default) stores no `expiresAt`. `7` sets `expiresAt` to seven days after `created`. More than `datasetMaxRows` (default 50,000) returns 413. Another user's id returns 404.
48
+
49
+ ## Register a server callback
50
+
51
+ 1. Pass `hostActions` on `uiBlocks`. Each name has a Zod `payload` and a `handler`.
52
+ 2. The client posts `{historyId, messageId, blockId, elementId, name, payload}` to `POST /gpt/actions`.
53
+ 3. Return `{replace: "block", blocks}` to swap the block, or `{text}` to append an assistant message. `blocks` must be a valid document (`v` then `blocks`).
54
+ 4. An unknown name is 404. A payload that fails the schema is 400 with `meta.fields`. Another user's history is 403. The handler stops at 10 seconds with 504. An invalid document from the handler is 500.
55
+
56
+ ```ts
57
+ addGptRoutes(router, {
58
+ aiService,
59
+ uiBlocks: {
60
+ hostActions: {
61
+ export_csv: {
62
+ payload: z.object({format: z.literal("csv")}).strict(),
63
+ handler: async () => ({
64
+ replace: "block",
65
+ blocks: {v: 1, blocks: [{type: "badge", text: "Exporting", status: "info"}]},
66
+ }),
67
+ },
68
+ },
69
+ },
70
+ });
71
+ ```
72
+
73
+ The path is `/gpt/actions`, so a limiter on `/gpt` covers it. Each call is stored as an `AIRequest` with `requestType: "ui_action"`.
74
+
75
+ ## Wire the example chat
76
+
77
+ 1. Turn `uiBlocks` on in `addGptRoutes` so `GET /gpt/datasets/:id` and `POST /gpt/actions` are in the OpenAPI spec.
78
+ 2. Regenerate the frontend SDK: `cd example-frontend && bun run sdk`.
79
+ 3. Set `uiBlocks` and `hostActions` on `GPTChat`. `reply` already calls `onSubmit`.
80
+ 4. `onBlockCallback` calls `usePostGptActionsMutation`. Return that result so a `{replace: "block"}` swaps the block.
81
+ 5. `resolveDataset` calls `useLazyGetGptDatasetsByIdQuery` and returns `{columns, rows, source: "inline"}`.
82
+ 6. An `open` action uses `router.push` for a `/…` route and `Linking.openURL` for an `https` URL.
83
+ 7. When the `{blocks}` event is `ok`, set `blockNote` to the top-level block count, such as `3 components`. Recompute that caption from stored assistant YAML when a history is loaded again.
84
+
85
+ ## Turn blocks on in the example backend
86
+
87
+ `example-backend` passes `uiBlocks` with `repair: true` and one host callback, `exportDataset`. Its payload is `{dataset: string}`. The handler returns `{replace: "block", blocks}` with a badge that names that dataset. Dataset TTL stays at the default, so stored rows are kept.
88
+
89
+ The per-request tool `todoStats` counts the signed-in user's open and completed todos, calls `registerAiDataset`, and returns `datasetId`. It stores the rows on `req.body.historyId` when that id belongs to the caller. With no history id it uses the caller's newest history, and with no history it returns `{datasetId: null, rowCount: 0}`. A chart dataset uses `source: ref` and that id. Columns are `status` and `count`.
90
+
91
+ `GPTChat` lists `exportDataset` in `hostActions`, so that callback stays enabled. The e2e mock also allows `export_csv`.
92
+
93
+ ## Render documents in chat
94
+
95
+ Set `uiBlocks` on `GPTChat`. Assistant `content` is the YAML document. While `isStreaming`
96
+ is true, finished top-level blocks render and a spinner marks the block still arriving.
97
+
98
+ - `reply` calls `onSubmit` with the button's text.
99
+ - `open` and `select` call `onBlockAction`.
100
+ - `callback` calls `onBlockCallback`. Return `{replace: "block", blocks}` to swap that
101
+ block, or `{text}` to append an assistant message. The button shows loading until the
102
+ promise settles.
103
+ - Pass `hostActions` to disable callbacks the host does not run, `resolveDataset` to
104
+ load `ref` datasets, and `resolveImage` to turn a `file:` image id into a URL.
@@ -0,0 +1,156 @@
1
+ # Develop an AI feature
2
+
3
+ Work this loop in **Terreno admin** when observability primaries are `local`. If prompts/datasets/experiments are Langfuse, the same screens proxy the vendor, or use **Open in Langfuse**.
4
+
5
+ Architecture: [AI observability](../explanation/ai-observability.md). Routes and models: [AI reference](../reference/ai.md). Register plugins: [Observe LLM calls](./observe-llm-calls.md).
6
+
7
+ Volume targets (how many gold items, class balance) live in the operator spreadsheet, not in Terreno. Use **tags** and **outcome class** so you can filter for balance.
8
+
9
+ For a walkable example, run `bun run backend:seed`: it installs
10
+ `examples/example-summarize` with production on v1 and candidate v2, the human
11
+ `correctness-human` and automatic `schema-assert` evaluators, and the proofread
12
+ `example-gold` dataset. It also installs `chat-safety-synthetic`, twelve proofread
13
+ synthetic chats between two people. Each item is the new message plus a few earlier
14
+ turns. Gold labels cover 988 (immediate danger to self), care-team follow-up,
15
+ toxicity, a privacy leak, and a dismissive reply. Compare `chat-safety-screen` v1
16
+ (new message only) with v2 (earlier turns) using evaluator `chat-safety-agreement`.
17
+ These fixtures are proofread on purpose so experiments include them. The Generate
18
+ button still lands drafts as `proofread: false` until a human accepts them. Load the
19
+ same set into a running database, including a PR preview, with Admin → Scripts →
20
+ `seedChatSafetyDataset` (wet run).
21
+
22
+ ## 1. Gather a gold dataset
23
+
24
+ 1. Open **Datasets** → create a named set. Optionally set `inputSchemaPromptName` to a registry prompt whose `inputSchema` should validate imported rows, and `expectedOutputSchema` if product has agreed a label shape.
25
+ 2. Add items in three ways:
26
+ - **From production traces** (preferred for open-ended features): Traces list → filter → **Add to dataset** (`POST /ai/observability/traces/add-to-dataset`). Sensitive traces always land `proofread: false`.
27
+ - **Import JSON or CSV** (`POST /ai/observability/datasets/:id/import`): JSON can be an array of bare input objects or structured rows with `input`, `expectedOutput`, `proofread`, `tags`, `outcomeClass`, and `metadata`. CSV accepts `Content-Type: text/csv` or `{format: "csv", content: "..."}`; plain columns map to `input`, `input.foo` nests fields, `expectedOutput.foo` maps labels, and reserved `proofread` / `tags` / `outcomeClass` columns map metadata. Invalid rows against the bound input schema return 400 with the row number and JSON path.
28
+ - **Manual**: `POST /datasets/:id/items` with a production-shaped `input`.
29
+ - **Synthetic** (phase 2.4): **Generate** drafts. They land as `origin: synthetic` and `proofread: false`. Do not run experiments on them until a human proofreads and sets `proofread: true`.
30
+ 3. Prefer a mix of true/false positives and negatives on classification features. Tag critical cases.
31
+
32
+ In-app **thumbs** / **false positive|negative** on a live feature write feedback scores and can **flag for dataset** (step 7). Those traces show up with `flaggedForDataset`.
33
+
34
+ ## 2. Label expected output
35
+
36
+ 1. Open the dataset item.
37
+ 2. Fill `expectedOutput` to match the schema from step 1.
38
+ 3. Set `outcomeClass` (`tp` / `fp` / `tn` / `fn`) when the feature is a flag.
39
+ 4. Add `tags` for categories you will filter on later.
40
+ 5. Set `proofread: true` on synthetic items you accept.
41
+
42
+ Skip unlabeled items in experiments unless you only need generation smoke (still prefer labels before scoring).
43
+
44
+ ## 3. Create the prompt
45
+
46
+ 1. Open **Admin → AI Observability → Prompts**. The folder rail shows counts; search filters name and folder. If your role lacks `aiPrompt:create`, **Create prompt** is hidden (API writes still 403).
47
+ 2. **Create prompt** (folder, name, system, user template with `{{variables}}`). That write is immutable **v1**. Open the row to the prompt hub (**Overview**, **Versions**, **Traces**, **Experiments**).
48
+ 3. On **Versions**, edit system/template/variables/temperature when `aiPrompt:update` is allowed, then **Save as vN+1**. There is no in-place save. **Playground → Run once** (when `aiPrompt:playground` is allowed) compiles and calls the model; it does **not** create a version. **Traces** and **Experiments** show recent items for this prompt from prompt detail relationships.
49
+ 4. Do not paste that string into app routes. Apps call `AIService` with `promptName` + `promptLabel` (step 6).
50
+
51
+ Read-only auditors with `aiPrompt:list` and `read` can review overview metadata and relationship tabs but do not see create, save, promote, or playground controls. Operators with the full `aiPrompt` action set see those controls. UI gating follows `permissions` on `GET /ai/observability/status`; assign roles in **Admin → Roles** per [API reference — AI observability RBAC](../reference/api.md#ai-observability-rbac).
52
+
53
+ Judge prompts for evaluators are named constants or registry prompts — never inline in `AIService`.
54
+
55
+ ## 4. Set up evaluators
56
+
57
+ **Evaluators** → **Create from template** (`POST /ai/observability/evaluators/templates/:name`) or custom (`POST /ai/observability/evaluators`):
58
+
59
+ | Feature shape | Template | Score |
60
+ | --- | --- | --- |
61
+ | Boolean flags (urgent, etc.) | `correctness` (`llm-judge`) or `correctness-human` (review queue) | boolean `correct` |
62
+ | Open-ended text | `hallucination` / `hallucination-human` | boolean `hallucinated` / `contains_hallucination` |
63
+ | Open-ended text | `helpfulness` / `helpfulness-human` | numeric 0–1 |
64
+ | Open-ended text | `toxicity` / `toxicity-human` | boolean `is_toxic` / `toxic` |
65
+ | Structured JSON output | `schema-assert` (`json-assert`) | boolean `schema_valid` |
66
+ | Open-ended text (if useful) | `conciseness` | numeric 0–1 (human) |
67
+
68
+ `llm-judge` templates require a registry judge prompt (`eval-judge-*`) whose `outputSchema` declares every required dimension. Create returns 400 naming any missing key. `confidenceAlertBelow` defaults to **0.7**.
69
+
70
+ A human evaluator with `runModes.liveSampleRate > 0` is rejected (400). Live sampling for judges is task 2.2, capped by `AI_OBS_SAMPLE_RATE`.
71
+
72
+ Add more dimensions with product/engineering. Attach them to the experiment in step 5.
73
+
74
+ ## 5. Run experiments
75
+
76
+ 1. **Experiments** → `POST /ai/observability/experiments` with dataset id, **2–3 prompt version numbers**, evaluator ids, optional `modelOverride`, and optional `thresholds[]`.
77
+ 2. Thresholds default to the SOP gates (override per run if product disagrees):
78
+
79
+ | Dimension | Gate |
80
+ | --- | --- |
81
+ | correctness true-rate | = 100% |
82
+ | hallucination true-rate | = 0% |
83
+ | helpfulness mean | ≥ 90% |
84
+ | toxicity true-rate | = 0% |
85
+
86
+ 3. Wait for the run (`BackgroundTask` when local — even for a one-item dataset).
87
+ 4. `GET /ai/observability/experiments/:id` returns gate pass/fail, aggregates (`trueRate` for booleans, `mean` for numerics), `outlierItemIds`, `lowConfidenceItemIds`, and per-item outputs (**failed rows first**).
88
+ 5. Skim **per-item outputs**. If they are nonsense, fix the prompt before trusting aggregates.
89
+ 6. Read **pass/fail vs thresholds**. Open outliers and low-confidence items (default alert below 0.7). Use those items to rewrite the prompt.
90
+ 7. Save a new prompt version. Repeat this step until gates pass (or product accepts a documented override).
91
+ 8. `POST /ai/observability/experiments/:id/promote` with `{version}` moves `production` when all gates pass; **409** while any gate fails. Experiments never auto-promote without this call.
92
+
93
+ Unproofread synthetic items are excluded unless you set `includeUnproofread`.
94
+
95
+ ## 6. Deploy to production
96
+
97
+ 1. In the prompt editor, select the winning version and **Set vN as production…**. The confirm modal names the **outgoing** production version (or none). Experiments never auto-promote.
98
+ 2. In the app, resolve that label at call time:
99
+
100
+ ```typescript
101
+ await aiService.generateText({
102
+ prompt: text,
103
+ promptLabel: "production",
104
+ promptName: "example-summarize",
105
+ userId: req.user?.id,
106
+ });
107
+ ```
108
+
109
+ Missing `production` fails the request (400). Do not fall back to a hardcoded string.
110
+
111
+ ## 7. Production tracing and feedback
112
+
113
+ Do this in parallel with steps 1–6 once the feature is reachable.
114
+
115
+ 1. Register `ObservabilityApp`. Every `generate*` emits a trace unless `skipTrace`.
116
+ 2. Pass `userId` and `sessionId` from the request.
117
+ 3. From the product UI, call `POST /ai/observability/traces/:id/feedback` as the signed-in user:
118
+ - `kind: "thumbs"` + `up` / `down` (helpfulness of suggestions)
119
+ - `kind: "outcome"` + `tp` / `fp` / `tn` / `fn` (flags)
120
+ - `flagDataset: true` to queue the trace as a dataset candidate
121
+ 4. In **Admin → AI Observability → Traces**, filter by time, prompt dropdown, status, user, session, score presence, or data sensitivity. Score and sensitivity dropdowns include both positive and negative states. Rows show a status dot, `sensitive` badge, error line, `N prompts`, tokens, cost, and latency. Open a row for the span tree (kind, indent, duration bar). Sensitive I/O disclosures start **collapsed**.
122
+ 5. Select one or more traces. The bulk bar warns when any selected row is `sensitive`. Choose **Send to human review**, read the queue explanation, and pick a human evaluator. The evaluator's dimensions become the score fields and its instructions guide the reviewer. Confirming posts `POST /ai/observability/traces/review` with `reason: "manual"`. Automatic (`llm-judge` / `json-assert`) evaluators cannot enter this queue.
123
+ 6. Open **Review queue**. Tabs show Pending / In progress / Done / Skipped counts, and pending
124
+ items are oldest-first. Select **Start reviewing — oldest first**.
125
+ 7. On each item, compare **What the AI was given** with **What the AI wrote**. Long fields begin
126
+ collapsed with a word count; structured outputs retain field labels and reviewer notes. Use
127
+ **Raw JSON** only when the field view is insufficient.
128
+ 8. Score every required evaluator dimension: numeric dimensions use the declared range slider,
129
+ boolean dimensions use Pass / Fail, and categorical dimensions use pills. Add an optional
130
+ comment, then **Submit & next**. **Skip** records no score. For the second phase-1 intake path,
131
+ use **Assign to me** to manually claim an item and move it to In progress. Continue until the
132
+ toast says **Queue clear**.
133
+
134
+ Feedback is a score (`source: "user-feedback"`) fanned out to every ScoreSink. Non-owners who are not admin get 403.
135
+
136
+ ## 8. Add weak production traces back to the dataset
137
+
138
+ 1. **Traces**: filter `thumbs=down`, `flaggedForDataset`, `failedEval`, or `lowConfidence` (filters land with task 2.6 feedback).
139
+ 2. **Add to dataset** (`POST /ai/observability/traces/add-to-dataset`, bulk allowed).
140
+ 3. Label the new items (step 2).
141
+ 4. Repeat steps 5–8.
142
+
143
+ Phase 1 also supports **Send to review queue** from the Traces bulk bar (step 7) when human scoring is enough before dataset intake.
144
+
145
+ ## Checklist (one feature)
146
+
147
+ - [ ] Gold set exists with mixed origins and labels
148
+ - [ ] Prompt versions in the registry, not inlined in routes
149
+ - [ ] SOP (or custom) evaluators attached
150
+ - [ ] Experiment gates green or explicitly overridden
151
+ - [ ] `production` label on the winning version
152
+ - [ ] App calls `promptName` + `promptLabel`
153
+ - [ ] Live traces include user/session
154
+ - [ ] Product can send thumbs / outcome / flag-for-dataset
155
+ - [ ] Human review queue reaches Queue clear
156
+ - [ ] Weak traces can be added back to the dataset
@@ -0,0 +1,232 @@
1
+ # Build a durable workflow
2
+
3
+ Build a multi-step task that survives restarts, asks a person to sign off, and performs its
4
+ side effect exactly once. The one worked example is the example backend's clinical tracer,
5
+ `clinic.intakeSummary`: fetch a chart, summarize it with an agent, wait for a clinician,
6
+ file a note. API: [AI harness reference](../reference/ai-harness.md). Why it is shaped this
7
+ way: [Durable agent harness](../explanation/durable-agent-harness.md).
8
+
9
+ | File | What it holds |
10
+ | --- | --- |
11
+ | `example-backend/src/harness/clinicalIntake.ts` | The task, the summarizer agent, the approvers |
12
+ | `example-backend/src/harness/fakeEhr.ts` | The EHR client (`getChart`, `writeNote`) over `FakePatientChart` / `FakeClinicalNote` |
13
+ | `example-backend/src/harness/clinicModels.ts` | Model resolver: Gemini when configured, else the demo model |
14
+ | `example-backend/src/harness/clinicDemoModel.ts` | Deterministic local `LanguageModel` for dev and e2e |
15
+ | `example-backend/src/harness/exampleHarness.ts` | `Harness.open` + runner, started in the API process |
16
+ | `example-backend/src/harness/startClinicalIntake.ts` | Admin script that starts a run |
17
+
18
+ ## Try it (5 minutes)
19
+
20
+ 1. Start a replica-set `mongod`, the backend (`bun run backend:dev`), and the frontend
21
+ (`bun run frontend:web`). Steps: [Run tests locally](run-tests-locally.md).
22
+ 2. Seed: `bun run backend:seed`. It creates charts `p-1001`, `p-1002`, `p-1003`, the admin
23
+ `admin@example.com`, and the clinician `clinician@example.com` (password
24
+ `testpassword123`).
25
+ 3. Start a run: `cd example-backend && bun run script startClinicalIntake --wet --patientId p-1001`,
26
+ or run `startClinicalIntake` from the admin **Scripts** screen.
27
+ 4. Sign in as the admin and open **AI Harness → Approvals**
28
+ (`http://localhost:8082/admin/harness-approvals`). Open the item and approve it.
29
+ 5. Open **AI Observability → Traces**: the newest `clinic.intakeSummary@1` trace shows every
30
+ phase, the summarizer with its model call, and the sign-off.
31
+
32
+ ## 1. Define the task
33
+
34
+ Split the work at every point where a crash would need a different answer. Each phase is a
35
+ checkpoint; `replay` says what to do when a phase is cut off.
36
+
37
+ | Phase | `replay` | Work | Why that policy |
38
+ | --- | --- | --- | --- |
39
+ | `fetch` | `safe` | `ehr.getChart(patientId)`; no chart fails the task | A read; running it twice is harmless |
40
+ | `summarize` | `safe` | `rt.runAgent(summarizer, {input: chart, output: IntakeSummarySchema})` | A replay finds the same subagent turn; finished model calls are not repeated |
41
+ | `review` | `safe` | `rt.approval("clinician-signoff", {...})`; branch on the decision | A replay finds the same approval row |
42
+ | `write` | default (`never`) | `ehr.writeNote(..., {idempotencyKey: \`note-${task.id}\`})` | A write cut off mid-call needs a person to check the EHR |
43
+
44
+ ```typescript
45
+ const intakeSummary = defineTask<IntakeSummaryInput, IntakeSummaryState, IntakeSummaryResult>({
46
+ name: "clinic.intakeSummary",
47
+ version: 1,
48
+ approvals: {"clinician-signoff": {approvers: [isAdminOrClinician]}},
49
+ initial: () => ({phase: "fetch", state: {}}),
50
+ retry: {maxAttempts: 3},
51
+ phases: {
52
+ fetch: {
53
+ replay: "safe",
54
+ run: async (task, rt) => {
55
+ const chart = await ehr.getChart(task.input.patientId);
56
+ if (!chart) {
57
+ await rt.commit({terminal: {status: "failed", error: `No chart for patient ${task.input.patientId}`}});
58
+ return;
59
+ }
60
+ await rt.commit({phase: "summarize", state: {chart}});
61
+ },
62
+ },
63
+ // summarize, review, write: see the sections below
64
+ },
65
+ });
66
+ ```
67
+
68
+ Keep `state` plain JSON: it is stored on every checkpoint. Bump `version` when phases or
69
+ state change: [Ship a new task version](ship-a-new-task-version.md).
70
+
71
+ ## 2. Add the agent
72
+
73
+ ```typescript
74
+ const summarizer = defineAgent({
75
+ name: "clinic.summarizer",
76
+ model, // {provider: "google", modelId: "gemini-2.5-flash"} or the demo model
77
+ fallbackModels,
78
+ instructions: CLINIC_SUMMARIZER_INSTRUCTIONS, // a named constant, never inline
79
+ tools: [],
80
+ });
81
+
82
+ summarize: {
83
+ replay: "safe",
84
+ run: async (task, rt) => {
85
+ const summary = await rt.runAgent<NoteContent>(summarizer, {
86
+ input: task.state.chart,
87
+ output: IntakeSummarySchema, // {summary, risk, sources}
88
+ });
89
+ await rt.commit({phase: "review", state: {...task.state, summary}});
90
+ },
91
+ },
92
+ ```
93
+
94
+ `Harness.open({models})` turns the agent's `{provider, modelId}` into a model. The example
95
+ resolver (`resolveExampleModel`) maps:
96
+
97
+ | Ref | Model |
98
+ | --- | --- |
99
+ | `google/*` | The server's Gemini / Vertex model (`GEMINI_API_KEY` or `GOOGLE_VERTEX_PROJECT`) |
100
+ | `demo/clinic-summarizer-demo` | The demo model: derives the summary from the chart by fixed rules, prefixes it `[Demo model: no LLM was called]`, calls no network |
101
+
102
+ The summarizer uses the demo model when no provider is configured or `CLINIC_DEMO_MODEL=true`,
103
+ so dev and e2e runs need no API key. Its `LLM` spans are named `demo/clinic-summarizer-demo`.
104
+
105
+ ## 3. Ask for approval
106
+
107
+ Declare approvers on the definition, keyed by the approval key. Approvers in one list are
108
+ ANDed, so "admin **or** clinician" is one approver:
109
+
110
+ ```typescript
111
+ const isAdminOrClinician = anyApprover(
112
+ Permissions.IsAdmin,
113
+ access.permission({clinicalIntake: ["signoff"]}) // RBAC: the `clinician` role grants it
114
+ );
115
+
116
+ review: {
117
+ replay: "safe",
118
+ run: async (task, rt) => {
119
+ const decision = await rt.approval("clinician-signoff", {
120
+ title: `Sign off intake summary for ${chart.name}`,
121
+ summary: signoffSummary(chart, summary), // markdown shown in the inbox
122
+ payload: {patientId: chart.patientId, ...summary},
123
+ timeout: {hours: 24},
124
+ });
125
+ if (!decision.approved) {
126
+ const reason = decision.expired
127
+ ? "Sign-off expired before anyone decided"
128
+ : (decision.reason ?? "Rejected");
129
+ await rt.commit({terminal: {status: "completed", result: {status: "rejected", reason}}});
130
+ return;
131
+ }
132
+ await rt.commit({phase: "write", state: {...task.state, signedOffBy: decision.decidedBy}});
133
+ },
134
+ },
135
+ ```
136
+
137
+ The example adds the `clinicalIntake.signoff` statement and a `clinician` role
138
+ (`example-backend/src/access.ts`, `rbacRoles.ts`). Admins decide in the admin inbox.
139
+ Clinicians without admin access decide over HTTP (`GET /harness/approvals`,
140
+ `POST /harness/approvals/:id/approve`) or in a `HarnessApprovalInbox` mounted outside the
141
+ admin shell ([admin-frontend](../reference/admin-frontend.md#ai-harness-approvals-inbox)).
142
+
143
+ ## 4. Perform the side effect once
144
+
145
+ Leave the side-effecting phase at the default `replay: "never"` and pass an idempotency key
146
+ the EHR enforces:
147
+
148
+ ```typescript
149
+ write: {
150
+ run: async (task, rt) => {
151
+ rt.signal.throwIfAborted();
152
+ const note = await ehr.writeNote(task.input.patientId, task.state.summary, {
153
+ idempotencyKey: `note-${task.id}`,
154
+ signedOffBy: task.state.signedOffBy,
155
+ taskId: task.id,
156
+ });
157
+ await rt.commit({terminal: {status: "completed", result: {status: "filed", noteId: String(note._id)}}});
158
+ },
159
+ },
160
+ ```
161
+
162
+ `FakeClinicalNote.idempotencyKey` is unique, so even a retried write returns the first note.
163
+
164
+ ## 5. Register and run
165
+
166
+ ```typescript
167
+ const harness = await Harness.open({
168
+ models: resolveExampleModel,
169
+ registry: [intakeSummary, summarizer],
170
+ runner: new InProcessRunner(exampleRunnerOptions()),
171
+ });
172
+ terraApp.register(new HarnessApp({harness})); // before AdminApp: adds the approvals inbox
173
+ await harness.start(); // after listen: the runner claims work
174
+
175
+ await harness.createTask(intakeSummary, {patientId}, {requestId: `intake-${patientId}`});
176
+ ```
177
+
178
+ `requestId` makes starting idempotent: the script returns the first run for a repeated key
179
+ (pass `--requestId` to start another). Every process that runs or decides tasks registers
180
+ the same definitions.
181
+
182
+ ## 6. Observe
183
+
184
+ | Where | What you see |
185
+ | --- | --- |
186
+ | **AI Harness → Approvals** | The pending sign-off: title, markdown summary, payload |
187
+ | **AI Observability → Traces** (`/admin/ai-trace-detail?id=<traceId>`) | The span waterfall |
188
+ | `GET /ai/observability/traces/:traceId` | The same tree as JSON (admin) |
189
+ | `GET /harness/tasks/:id/events` | Live `task.status` and `approval.*` events (SSE) |
190
+
191
+ `HarnessTask.traceId` links a run to its trace. A completed run's tree:
192
+
193
+ ```
194
+ CHAIN clinic.intakeSummary@1
195
+ ├── CHAIN fetch
196
+ ├── CHAIN summarize (waiting on the summarizer)
197
+ ├── AGENT clinic.summarizer
198
+ │ ├── CHAIN request
199
+ │ └── LLM demo/clinic-summarizer-demo
200
+ ├── CHAIN summarize (summary committed)
201
+ ├── CHAIN review (waiting on sign-off)
202
+ ├── CHAIN wait:terreno.approval:clinician-signoff:<step>:0
203
+ ├── CHAIN approval:clinician-signoff (decision, decidedBy)
204
+ ├── CHAIN review
205
+ └── CHAIN write
206
+ ```
207
+
208
+ ## 7. Recover an interrupted run
209
+
210
+ | Cut off during | What happens | You do |
211
+ | --- | --- | --- |
212
+ | `fetch`, `summarize`, `review` | After the lease expires (`HARNESS_LEASE_SECONDS`, default 30), the next runner replays the phase. A cut-off model request is re-requested; an error `request` span records the interruption. | Nothing |
213
+ | `write` | The task goes `interrupted`. | Check the EHR for `note-<taskId>`, then `POST /harness/tasks/:id/resolveInterrupted` with `{action: "retry" \| "complete" \| "abort", reason}` |
214
+
215
+ The proof is `example-backend/src/harness/clinicalIntake.crash.test.ts`. It starts the real
216
+ API process (`bun run src/index.ts`) against a replica set with
217
+ `CLINIC_DEMO_MODEL_DELAY_MS=600000`, starts a run, waits until the summarizer's model
218
+ request is in flight, sends `SIGKILL`, starts a fresh process, approves, and asserts:
219
+ the run ends `filed`, exactly one `FakeClinicalNote` exists, and the trace has one `LLM`
220
+ span plus an interruption span naming the dead process's lease. Run it with
221
+ `cd example-backend && bun test src/harness/clinicalIntake.crash.test.ts` (about 50 s).
222
+ The Playwright spec `example-frontend/e2e/harness-intake.spec.ts` covers approve-in-inbox →
223
+ `filed` → span tree in the Traces screen.
224
+
225
+ ## Configuration
226
+
227
+ | Variable | Default | Effect |
228
+ | --- | --- | --- |
229
+ | `GEMINI_API_KEY` / `GOOGLE_VERTEX_PROJECT` | unset | When set, the summarizer uses `google/gemini-2.5-flash` (fallback `gemini-2.5-flash-lite`) |
230
+ | `CLINIC_DEMO_MODEL` | unset | `true` forces the demo model even with a provider |
231
+ | `CLINIC_DEMO_MODEL_DELAY_MS` | `0` | Demo model waits this long per request (crash drills) |
232
+ | `HARNESS_LEASE_SECONDS` | `30` | Owner and task lease; heartbeat is a third of it. Short values speed up crash recovery in tests |
@@ -35,3 +35,10 @@ export const OpsDashboard: FC = () => {
35
35
  Charts have no width prop: each one fills the container you put it in and redraws when that width changes, so size the `Card` or `Box` around it. `height` is the chart's whole height, tick row and tooltip row included, so `height={140}` fits a 140px slot; the tooltip row is always reserved so hovering never reflows the page. Long x labels truncate inside their band rather than widening the chart.
36
36
 
37
37
  Prove paint with `bun run ui:charts:compare` against `demo/rendered-snapshots/`, not with JSON snapshots. See [Compare rendered chart snapshots](compare-chart-rendered-snapshots.md).
38
+
39
+ A chat reply can draw the same charts. `BlocksView` turns a `chart` block into one series of `{label, value}` points from an inline dataset or from `points`. It does not pass a per-point `color`. A `ref` dataset is loaded with `resolveDataset` and the chart stays in `loading` until those rows arrive.
40
+
41
+ The example app admin home (`example-frontend/components/AdminCharts.tsx`, mounted
42
+ below the admin dashboard at `/admin`) shows the same pieces together: scorecards with comparison
43
+ sparklines, a multi-series line, a donut with a center label, a bar chart, and an
44
+ area chart inside a spanning `DashboardGrid`.
@@ -80,7 +80,8 @@ for `.circleci/**`); the kitchen-sink does not start a second copy. If the same
80
80
  pipeline already set `run-api`, `run-ui`, `run-e2e`, `run-example-backend`, or
81
81
  `run-admin-spa`, that kitchen-sink workflow is skipped so jobs are not doubled.
82
82
  `comms/**` also sets `run-example-backend` and `run-example-backend-script`,
83
- matching the GitHub Actions twins.
83
+ matching the GitHub Actions twins. `blocks/**` sets `run-blocks` plus every
84
+ parameter `ai/**` sets, because `@terreno/ai` depends on `@terreno/blocks`.
84
85
 
85
86
  The example-backend Docker job runs only when its image recipe changes
86
87
  (`Dockerfile`, `.dockerignore`, package manifests, or `bun.lock`). API/source
@@ -165,7 +166,7 @@ non-PR branch builds.
165
166
  | Path (examples) | Parameter | PR job | `master` job |
166
167
  | --- | --- | --- | --- |
167
168
  | `demo/**`, `ui/**` | `run-deploy-demo` | `deploy-demo-preview` | `deploy-demo` |
168
- | `example-frontend/**`, `admin-frontend/**`, `rtk/**`, `ui/**`, `ai/**`, `syncdb/**`, `bun.lock` | `run-deploy-frontend` | `deploy-frontend-preview` | `deploy-frontend` |
169
+ | `example-frontend/**`, `admin-frontend/**`, `rtk/**`, `ui/**`, `ai/**`, `blocks/**`, `syncdb/**`, `bun.lock` | `run-deploy-frontend` | `deploy-frontend-preview` | `deploy-frontend` |
169
170
  | `docs/**`, `website/**` | `run-deploy-docs` | `deploy-docs-preview` | `deploy-docs` |
170
171
  | `example-backend/**`, `api/**`, `comms/**`, `jobs/**`, `admin-spa/**`, `feature-flags/**`, `announcements/**`, plus every frontend path above | `run-cd-backend` | `gcp-cd-preview` | `gcp-cd-prod` |
171
172
  | `terraform/**` | `run-cd-terraform` | `gcp-cd-preview` | `gcp-cd-prod` |
@@ -242,9 +243,10 @@ restrict `terreno-release` and `terreno-npm` to tag/manual release pipelines.
242
243
  Until contexts exist, e2e jobs may use **project env vars for `E2E_*` secrets only**
243
244
  (or the in-job `ci-e2e-*-secret` fallbacks). **Do not** put `GITHUB_TOKEN` (or any
244
245
  GitHub PAT) in project env vars — those are injected into every job, including
245
- `bun` scripts from the PR. Create `terreno-github-api`, restrict it to this
246
- project, leave fork-PR secret passing off, and attach that context **only** to
247
- `dco` and `architectural-pr-review`. DCO skips if `GITHUB_TOKEN` is unset.
246
+ `bun` scripts from the PR. `terreno-github-api` must hold the PAT as
247
+ `GITHUB_TOKEN`; restrict it to this project and leave fork-PR secret passing off.
248
+ The `always` workflow attaches it **only** to `dco` and `architectural-pr-review`
249
+ (the latter also gets `terreno-agentic`). DCO skips if `GITHUB_TOKEN` is unset.
248
250
  `architectural-pr-review` also skips if `GITHUB_TOKEN` or `CURSOR_API_KEY` is
249
251
  unset, and it skips fork PRs. The job checks out `origin/master` before running
250
252
  the review script so a PR cannot rewrite the reviewer.
@@ -268,7 +270,7 @@ ambient OIDC token for GCP impersonation.
268
270
  Branch protection must require the CircleCI job names below. Remove disabled
269
271
  GitHub check names or pull requests will wait for checks that can no longer run.
270
272
 
271
- Dedicated package jobs (`api-ci`, `ai-ci`, `rtk-ci`, `ui-ci`, `syncdb-ci`,
273
+ Dedicated package jobs (`api-ci`, `ai-ci`, `blocks-ci`, `rtk-ci`, `ui-ci`, `syncdb-ci`,
272
274
  `comms-ci`, `mcp-server-ci`, `admin-spa-ci`) run `bun run test:coverage`
273
275
  (`scripts/check-coverage.ts`, 95% functions and lines). Isolated `syncdb` tests
274
276
  are included by that script. Published packages without a dedicated workflow
@@ -287,6 +289,7 @@ the same commands through the parameterized `packages-ci` job, gated by
287
289
  | Admin backend lint, compile, coverage | `packages-ci` (`admin-backend`) |
288
290
  | Admin frontend lint, compile, coverage | `packages-ci` (`admin-frontend`) |
289
291
  | Run all tests (AI CI) | `ai-ci` |
292
+ | Lint, compile, and test blocks | `blocks-ci` |
290
293
  | RTK Lint and Build | `rtk-ci` |
291
294
  | Syncdb Lint, Build, and Tests | `syncdb-ci` |
292
295
  | UI Lint, Build, Types, and Tests + demo typecheck | `ui-ci` |
@@ -33,4 +33,12 @@ easy-through-hard charts in the demo app and diffs those PNGs against goldens.
33
33
  Agents reviewing a fail should follow the `review-chart-visuals` skill and look at the
34
34
  PNGs, not only the code diff.
35
35
 
36
+ Hard fixture `scorecard-sparkline-comparison` renders five KPI tiles with solid current
37
+ and dotted previous-period sparklines.
38
+ Cartesian parity fixtures are `bar-time-rotated-ticks`, `bar-day-of-week`, and
39
+ `line-three-series`.
40
+ Donut parity fixtures are `donut-center-and-share` and `donut-single-slice`.
41
+ Hard fixture `hows-it-going-dashboard` composes KPI scorecards, time-series charts, donuts,
42
+ and table placeholders in a spanning `DashboardGrid`.
43
+
36
44
  `CHART_VISUAL_BASE_URL` overrides the demo origin when the gallery is already hosted.
@@ -70,7 +70,7 @@ scan stages come from
70
70
  | Docs | `update-docs`, `update-agent-docs`, architecture skills |
71
71
  | GitHub | issues, review, UI verification, release, dependency updates |
72
72
  | Code scans | `terreno-scan-1-aim` … `terreno-scan-5-track`, `terreno-scan-campaign`, `terreno-scan-loop` |
73
- | Expo and native | `track-upstream-expo`, `upgrading-expo`, deployment / EAS skills |
73
+ | Expo and native | `track-upstream-expo`, `upgrading-expo`, `store-assets`, deployment / EAS skills |
74
74
  | Plugin agents | `pre-commit`, `ui-verifier` |
75
75
 
76
76
  `skills.sh.json` at the repo root groups those names on [skills.sh](https://skills.sh).