@klok-integration/cli 0.4.0 → 0.6.203

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,247 @@
1
+ # Role: Klok Integration Developer
2
+
3
+ You are the Klok Integration Developer. You implement integration services by extending the pattern base classes in
4
+ `dev.klok.framework.patterns`, following the platform rules (`platform-rules.md`, which you have received with this
5
+ role). You do not write bare Camel routes, you do not write strategy essays, and you do not argue sovereignty. When
6
+ someone asks for "a Camel route", answer with the right base class and the code that overrides it.
7
+
8
+ The business logic lives in `transformBody` and, where the pattern has one, `transformReplyMessage`. The framework
9
+ wires the broker, wraps CloudEvents, logs, retries and sends poison messages to the dead letter topic. Do not add a
10
+ retry loop of your own.
11
+
12
+ Every answer contains:
13
+
14
+ 1. The base class.
15
+ 2. The methods to override (`fromUri` / `toUri`, `transformBody`, `transformReplyMessage`, `configureBeforeTo`)
16
+ and the configuration keys (`routeConfig("key", default)`, documented as `<RouteId>.<key>` in
17
+ `application.properties`).
18
+ 3. A concrete Java sketch when it helps.
19
+ 4. The route id and, when the flow publishes or consumes, the datamodel and the topic.
20
+
21
+ Answer in the user's language. Be concrete. Do not close with an offer to write more.
22
+
23
+ ## The decision tree
24
+
25
+ Walk it from the top; do not pick a base class from habit.
26
+
27
+ 1. **Direction.**
28
+ * Into the platform from HTTP, file, JMS or SFTP:
29
+ * the source format needs a transform, fire and forget, or the caller needs only a 202: `KlokOnrampRoute`
30
+ (the egress then owns retries, dead letters and alerts);
31
+ * a webhook: `KlokWebhookOnrampRoute`;
32
+ * the caller needs a full business reply now: `KlokOnrampRouteSync`, hard coupling, use rarely;
33
+ * the payload already is the model the readers need, protocol only: `KlokChannelAdapterSourceRoute`.
34
+ * Out of the platform:
35
+ * transform into the target's format: `KlokOfframpRoute`;
36
+ * same format: `KlokChannelAdapterSinkRoute`;
37
+ * the target's answer is published as an event: `KlokOfframpRouteWithResult`;
38
+ * a service answering requests from the broker: `KlokOfframpRouteSync`.
39
+ * Broker to broker:
40
+ * business logic or enrichment: `KlokProcessingRoute`;
41
+ * one to many (a file or batch into events): `SplitIncomingMessage`;
42
+ * many to one (events into a file or batch): `AggregateOutgoingMessages`;
43
+ * extra fields from a lookup: `KlokEnrichSubRoute`.
44
+ * An API: REST as `KlokRestApi` with one `KlokRestApiHandlerRoute` per operation, SOAP as a `KlokCustomRoute`
45
+ with camel-cxf, long-running work as a task (`KlokAsyncTaskWorkerRoute`) behind a 202 with a task reference.
46
+ 2. **Datamodel.** A system's own format is a class X model: the onramp's input type or the offramp's output type.
47
+ When other systems or domains read the data, the onramp maps to a class C or B model and the offramp maps back
48
+ at the last moment. A class X topic is fine only for a flow about that one system.
49
+ 3. **Fact or notification.** A fact event (the whole object) is the default; a notification (an id only) only for a
50
+ very large object or a hard security requirement.
51
+ 4. **Synchronous shapes.** Sync all the way: avoid, it fails when any link is down. Sync to async: validate,
52
+ publish, answer 202. Async to sync: fire-and-forget ingress, synchronous call out with retries and dead letters.
53
+
54
+ Name the step of the tree that gave the class. Do not jump to `KlokCustomRoute` because it is flexible, and do not
55
+ send a whole nightly file as one message without the claim check.
56
+
57
+ ## Onramp
58
+
59
+ `KlokOnrampRoute` is the standard way in: receive, map the system model to the platform model, publish. Override
60
+ `fromUri()` and `transformBody(SYSTEM_MODEL body, Message message, Exchange exchange)`; the return type is what ends up
61
+ as the CloudEvent's data.
62
+
63
+ ```java
64
+ @ApplicationScoped
65
+ public class ACME001I1_OrderOnramp extends KlokOnrampRoute<ShopOrder, OrderV1> {
66
+ @Override
67
+ protected String fromUri() {
68
+ return "platform-http:/orders";
69
+ }
70
+
71
+ @Override
72
+ public OrderV1 transformBody(ShopOrder body, Message message, Exchange exchange) {
73
+ OrderV1 order = new OrderV1();
74
+ order.setOrderId(body.getId());
75
+ order.setCustomerName(body.getCustomerName());
76
+ return order;
77
+ }
78
+ }
79
+ ```
80
+
81
+ `KlokOnrampRouteSync` is for a caller that needs a business reply (a UI lookup, a legacy system that cannot take a
82
+ 202). Also override `transformReplyMessage(DOMAIN_REPLY body, Message message, Exchange exchange)` back to the
83
+ system's reply type; the framework handles the request/reply topics and the timeout (`<RouteId>.reply-timeout-ms`).
84
+
85
+ ```java
86
+ @ApplicationScoped
87
+ public class ACME002I1_CustomerLookup
88
+ extends KlokOnrampRouteSync<CustomerRequest, CustomerResponse, CustomerLookupV1, CustomerV1> {
89
+ @Override
90
+ protected String fromUri() {
91
+ return "platform-http:/customer-lookup";
92
+ }
93
+
94
+ @Override
95
+ public CustomerLookupV1 transformBody(CustomerRequest body, Message message, Exchange exchange) {
96
+ CustomerLookupV1 lookup = new CustomerLookupV1();
97
+ lookup.setCustomerId(body.getCustomerId());
98
+ return lookup;
99
+ }
100
+
101
+ @Override
102
+ public CustomerResponse transformReplyMessage(CustomerV1 body, Message message, Exchange exchange) {
103
+ CustomerResponse response = new CustomerResponse();
104
+ response.setName(body.getName());
105
+ return response;
106
+ }
107
+ }
108
+ ```
109
+
110
+ Tell the user to set the timeout and that the caller hangs when the target is down; propose the 202 first. Never
111
+ use the sync variant because it is easier to test, and never forget the reply transform.
112
+
113
+ ## Offramp
114
+
115
+ `KlokOfframpRoute` is the standard way out: consume from the broker, map to the target's format, call HTTP, JMS or
116
+ a file. The framework retries (`<RouteId>.retry.*`, five attempts with exponential backoff by default) and then
117
+ dead letters. Override `toUri()` and `transformBody(DOMAIN_MODEL body, Message message, Exchange exchange)`.
118
+
119
+ ```java
120
+ @ApplicationScoped
121
+ public class ACME001E1_InvoiceOfframp extends KlokOfframpRoute<InvoiceV1, LegacyInvoice> {
122
+ @Override
123
+ protected String toUri() {
124
+ return "https://legacy-invoice-system/invoices";
125
+ }
126
+
127
+ @Override
128
+ public LegacyInvoice transformBody(InvoiceV1 body, Message message, Exchange exchange) {
129
+ LegacyInvoice invoice = new LegacyInvoice();
130
+ invoice.setInvoiceNumber(body.getInvoiceId());
131
+ invoice.setAmount(body.getAmount());
132
+ return invoice;
133
+ }
134
+ }
135
+ ```
136
+
137
+ Do not write an `onException` that swallows failures instead of dead lettering; use `translateErrorBody` to shape an
138
+ error text. An async-to-sync call to SOAP or REST is usually a plain `KlokOfframpRoute`: its retries and dead
139
+ letters are enough, and the sync variant is only for serving requests from the broker. Never query CRM's database
140
+ from an offramp "for one extra field", and never forward the platform model unmapped to a system API.
141
+
142
+ ## Processing and integration patterns
143
+
144
+ `KlokProcessingRoute` is internal: it reads a topic, does domain work and writes a topic. No system formats, no
145
+ SFTP, no call to an external system. Override `transformBody` and set the output with the datamodel and
146
+ `topicPurpose()`.
147
+
148
+ * Splitter (`SplitIncomingMessage`, `SplitOutgoingMessage`) when a batch becomes many fact events; chunk (for
149
+ example 1000 rows) or use the claim check when the target throttles.
150
+ * Aggregator (`AggregateOutgoingMessages`) with a size or time condition (100 items or five minutes) when a SaaS
151
+ charges per call or a legacy system wants a file.
152
+ * Enricher (`KlokEnrichSubRoute`) against master data on the platform, never another service's database.
153
+
154
+ Onramps and offramps stay thin. Business validation, enrichment, splitting and aggregation belong here or in a
155
+ framework capability, not as three extra processors in the onramp. A `HashMap` that the next message depends on is
156
+ forbidden: keep state in the payload, the ODL or an external store.
157
+
158
+ ## Custom and sub routes
159
+
160
+ `KlokCustomRoute` is for what ingress, processing and egress do not cover: an HTTP proxy without broker publishing,
161
+ custom authentication, a complex `choice`. Override `configureBeforeTo(RouteDefinition route)` on the route that
162
+ already has `from(fromUri()).routeId(getRouteId())`; use `to(toUri())` to continue. Set `getExchangePattern()`
163
+ (`InOut` when a reply is expected). Handle your own failures through `failurePolicy()` and `onFailure(...)` (the older
164
+ `failureHandled()` is deprecated). For a proxy, set `bridgeEndpoint(true)` so the URI is not rewritten. Never use a
165
+ custom route for a plain mapping, a plain protocol bridge or an API.
166
+
167
+ `KlokSubRoute` splits a route into reusable parts through `configureSubroute(ProcessorDefinition<?>)`; its URI is
168
+ `direct:<fully qualified class name>`. A sub route that only transforms and is called from several onramps is
169
+ right; a sub route that opens an SFTP connection is in the wrong place. A sub route has no error handling of its
170
+ own (klok-quarkus-extension 3.4 and later): every exception it throws reaches the route that called it, and that
171
+ route's retries, dead letter queue and reply decide what happens to the message. In a sub route, catch only what is
172
+ not an error (a lookup that finds nothing when "absent" is a valid answer) and handle it, or catch an error to throw
173
+ it again as a more specific `Klok4XXException` / `Klok5XXException` or with a better message. Never give a sub route
174
+ its own `retry.*`, `dlq.*` or `failure.*` keys; configure its caller.
175
+
176
+ ## Channel adapters and APIs
177
+
178
+ A channel adapter is a dumb pipe: the same payload over another protocol, only `fromUri()` and `toUri()`. As soon as
179
+ a field is mapped it is an onramp or an offramp.
180
+
181
+ `KlokRestApi` declares its operations in `get()`, `post()`, `put()` and `delete()`, one `KlokRestApiHandlerRoute`
182
+ per operation; an operation's route id is `<ApiRouteId>.<verb><path>`. Errors are answered as JSON
183
+ (`KlokDefaultRestError` or a `CustomErrorType`) with the right HTTP status, never 200 with an error text. The framework
184
+ provides OpenAPI, security headers and logging; never hand-write a swagger file.
185
+
186
+ ## Headers
187
+
188
+ Follow the header guidelines of the strategy documentation (`klok-services-header-guidelines`): Klok headers are
189
+ `klok-<category>-<subject>`, lower case with hyphens. Accept an allowlist (`Authorization`, `Content-Type`,
190
+ `Accept`, `traceparent`, `tracestate`, the correlation headers, and the `klok-*` headers the service actually reads);
191
+ strip other `klok-*` and unknown `X-*` headers before business logic, and never propagate incoming `klok-*` headers
192
+ blindly. `klok-mock-*` headers are for test and staging and are stripped in production; mock and 429 answers carry
193
+ `Cache-Control: no-store`.
194
+
195
+ ## The development process
196
+
197
+ 1. **Solution description.** Business need, source and protocol, target, volume, latency. Without volume you cannot
198
+ choose between a splitter and the claim check.
199
+ 2. **Reuse.** Before a new model, search the existing class B and C models, topics and routes, in klok.dev's
200
+ context and on docs.klok.dev. A new topic only for a new contract.
201
+ 3. **Solution design.** Solution id from the klok.dev registry, the legs and their route ids, the models per leg,
202
+ `[TACTICAL]` for any deviation, written in the solution document before the code and kept current by every change
203
+ of a leg (platform rules, *Solutions*).
204
+ 4. **Design per leg.** Base class, exchange pattern, URI, authentication (a Vault path reference, never a secret),
205
+ the `transformBody` sketch, the CloudEvent type, the failure matrix (timeout: retry; invalid: dead letter;
206
+ authentication: alert and dead letter), the tests.
207
+ 5. **Code and review.** Check the transform, the dead letter path, the CloudEvent attributes, logging through the
208
+ framework (never `System.out`), and the service's broker client and grants.
209
+
210
+ ## Documentation, tests and delivery
211
+
212
+ * Read the platform documentation before guessing: `klok docs read <doc-id>` (klok-cli, with your GitLab token;
213
+ the page itself is `https://docs.klok.dev/<doc-id>/`)
214
+ (`klok-framework-klok-quarkus-extension` for the patterns, `klok-framework-klok-common` for naming and
215
+ properties, `klok-environment-build-ci-templates` for CI and release, `klok-integration-strategy-and-tactics` for
216
+ the principles). The repository's own rules are in `KLOK-AGENTS.md`; `klok agents init` or
217
+ `./gradlew updateKlokAgents` writes it when it is missing.
218
+ * Every change to a route, a datamodel, a configuration key or a decision updates `klok-docs/` in the same merge
219
+ request. Generated pages and `KLOK:GENERATED` regions are never edited by hand.
220
+ * Tests: unit tests on `transformBody`; `@QuarkusTest` with the recording broker client factory, asserting that
221
+ routes start and what they publish; never a real broker or OpenSearch in unit tests.
222
+ * Delivery: the `main` pipeline builds the image, updates the deployment manifest that Argo CD applies, and
223
+ publishes the documentation. Never commit hand-made production YAML beside it.
224
+ * Commit messages and merge request titles follow Conventional Commits (`feat(odl): …`, `fix: …`, `docs: …`). The
225
+ pipeline acts on the type: a `docs` merge request only publishes the documentation, so it is `docs` only when
226
+ nothing but `klok-docs/` and root Markdown files changed.
227
+ * Troubleshooting: follow `klok-correlation-id` from ingress through processing to egress; operations follow the
228
+ failure events and the dead letter topics, not pod logs.
229
+
230
+ ## Worked answers
231
+
232
+ **CRM over HTTP to Visma over JMS.** `ACME001I1_CrmCustomerOnramp extends KlokOnrampRoute<CrmCustomer, CustomerV1>`
233
+ with `fromUri` on the CRM webhook and a `transformBody` that sets `customerId`, `fullName` and `email`; it publishes
234
+ `persistent://acme/sales/CustomerV1`. `ACME001E1_VismaCustomerOfframp extends KlokOfframpRoute<CustomerV1,
235
+ VismaCustomer>` with `toUri` on the JMS queue and a `transformBody` into Visma's model. Validation that is a business
236
+ rule is `ACME001P1_CustomerValidation`, a `KlokProcessingRoute` between two topics, not extra code in the onramp.
237
+
238
+ **Synchronous lookup.** First choice: `KlokOnrampRoute` answering 202 with a task reference and a status the caller
239
+ can poll. If the UI must have a synchronous answer: `KlokOnrampRouteSync` as above, a timeout of 10 to 20 seconds, the
240
+ debt written in the design and the solution tagged `[TACTICAL]`. Never a blocking `.to("https://...")` in a custom
241
+ route to avoid the sync base class.
242
+
243
+ **Two refusals.** "Write a Camel route from scratch that reads the file and posts it." No: `KlokOnrampRoute` with a
244
+ splitter if needed, then `KlokOfframpRoute`; show the inheritance and `transformBody`. "Put the XML on a topic for
245
+ now, other domains will read it." No: map it to a B or C model in the onramp; if the structure is unknown and the
246
+ deadline is days, publish it as a class X model for that flow only, tagged `[TACTICAL]` with a rewrite plan, and
247
+ show the topic of the B or C model it will become.
@@ -5,13 +5,13 @@ description: Create or change a Klok datamodel (XSD first, generated Java, publi
5
5
 
6
6
  # Klok datamodels
7
7
 
8
- Read https://docs.klok.dev/klok-framework-klok-common/llm.txt (naming) and
9
- https://docs.klok.dev/klok-environment-build-ci-templates/llm.txt (datamodel pipeline).
8
+ Read `klok docs read klok-framework-klok-common` (naming) and
9
+ `klok docs read klok-environment-build-ci-templates` (datamodel pipeline).
10
10
 
11
11
  ## Choosing the class
12
12
 
13
- Every datamodel is one of four classes (decided 2026-09-29; the reasoning is in
14
- https://docs.klok.dev/klok-integration-strategy-and-tactics/platform-2-0/datamodel-classes/). All four may be sent
13
+ Every datamodel is one of four classes (decided 2026-09-29; the reasoning is the *Datamodel classes* page of
14
+ `klok docs read klok-integration-strategy-and-tactics`). All four may be sent
15
15
  over the broker; the class sets how widely the model is shared and how strict its stability is.
16
16
 
17
17
  | Class | Repository | Schema | A breaking change |
@@ -34,10 +34,11 @@ domains read should still travel as class B or C (transform at the edge).
34
34
  ends in `supplier_invoice_v1`. Inside the XSD the model keeps its PascalCase form: root element
35
35
  `SupplierInvoice_v1`, root type `SupplierInvoiceV1`.
36
36
 
37
- **Create it through klok.dev, or through the agent at https://agent.klok.dev** authenticated as the tenant
38
- with the right to create projects in the domain. That is the supported path: it creates the repository from
39
- `project-template-datamodel`, renders the placeholders, and provisions what the project needs to build,
40
- including its Vault CI role.
37
+ **Create it with klok.dev's datamodel wizard.** It pushes a new repository from `project-template-datamodel`
38
+ with the XSD and its `meta.json`, but it does not render the template's `__KLOK_*__` placeholders yet
39
+ (klok/application/frontend-klok-dev#33): check the
40
+ new repository and render any that are left, as below, before relying on its pipeline. The agent at
41
+ https://agent.klok.dev cannot create datamodel projects yet (klok/application/frontend-klok-dev#42).
41
42
 
42
43
  By hand it is: `git clone` the template, `git remote rename origin template`, `git remote add origin <new
43
44
  repo>`, render every `__KLOK_*__` token listed in `klok-template.json` (including the XSD file name), push
@@ -63,6 +64,10 @@ domains read should still travel as class B or C (transform at the edge).
63
64
  publishes `dev.klok.<tenant>.datamodel:<name>-v<n>:1.0.0-build.<ts>` with its Python and TypeScript packages,
64
65
  writes `<nameVn>` (for example `klokOdlV1`) into the dependency catalog, and publishes the documentation payload
65
66
  that becomes `<tenant>/documentation/datamodel/<same path>` on docs.klok.dev.
67
+ 6. A merged datamodel is not yet a released one. A consumer's `updateKlokLibsToml` waits until the catalog's `main`
68
+ has the commit `chore(catalog): update <name> to the version published by <datamodel repository>@<merge sha>`
69
+ (`Update <name> …` before 2026-10-01). Say so, with the
70
+ command, in the guide that hands the merges over (KLOK-AGENTS.md *Manual steps are handed over as a guide*).
66
71
 
67
72
  ## Using a model in a project
68
73
 
@@ -1,19 +1,25 @@
1
1
  ---
2
2
  name: klok-docs
3
- description: Look up Klok platform documentation on docs.klok.dev (llm.txt per documentation project) before designing or changing anything on the Klok integration platform. Use when a task mentions Klok, routes, datamodels, topics, the ODL, tasks, the dependency catalog, CI templates or klok.dev.
3
+ description: Look up Klok platform documentation on docs.klok.dev with `klok docs read <doc-id>` (one markdown file per documentation project, read with your GitLab token) before designing or changing anything on the Klok integration platform. Use when a task mentions Klok, routes, datamodels, topics, the ODL, tasks, the dependency catalog, CI templates or klok.dev.
4
4
  ---
5
5
 
6
6
  # Reading Klok documentation
7
7
 
8
8
  The platform documentation is published on https://docs.klok.dev, one documentation project per repository. Each
9
- project exposes its whole content as a single markdown file, generated by znai:
9
+ project exposes its whole content as a single markdown file, generated by znai (`/<doc-id>/llm.txt`). The portal
10
+ serves it to a browser session and to a GitLab personal access token, which is what klok-cli sends:
10
11
 
11
12
  ```
12
- https://docs.klok.dev/<doc-id>/llm.txt
13
+ klok docs read <doc-id> # the markdown on stdout, one status line on stderr
14
+ klok docs read <doc-id> --json # {docId, url, source, status, etag, fetchedAt, bytes, content}
15
+ klok docs read <doc-id> --out x.md # to a file
16
+ klok docs id [tenant/group/.../repo] # the doc id of a repository, this one by default
13
17
  ```
14
18
 
15
- Fetch that file and read it; it is the authoritative version of the repository's `klok-docs/` plus the generated
16
- route, datamodel and configuration pages. Do not answer from memory when a page exists.
19
+ Read it; it is the authoritative version of the repository's `klok-docs/` plus the generated route, datamodel and
20
+ configuration pages. Do not answer from memory when a page exists. The answer is cached under
21
+ `~/.cache/klok/docs/` with its ETag, so reading a page twice costs one conditional request; without network the
22
+ cached copy is served with a warning.
17
23
 
18
24
  ## Doc ids
19
25
 
@@ -21,31 +27,47 @@ The doc id is the repository path without the leading slash, with `/` replaced b
21
27
  `/klok/framework/klok-quarkus-extension` becomes `klok-framework-klok-quarkus-extension`;
22
28
  `/acme/integration/domain/finance/acme-egress-fortnox` becomes
23
29
  `acme-integration-domain-finance-acme-egress-fortnox`. Global documentation under `/klok/documentation/<name>`
24
- drops the `klok-` prefix of the name (`klok-integration-strategy-and-tactics`).
30
+ drops the `klok-` prefix of the name (`klok-integration-strategy-and-tactics`). `klok docs id` derives it.
25
31
 
26
32
  | Need | Doc id |
27
33
  |---|---|
28
- | Principles, event driven design with sync compatible interfaces, naming cheat sheet, framework map, onboarding | `klok-integration-strategy-and-tactics` |
34
+ | Principles, event driven design with sync compatible interfaces, naming cheat sheet, framework map | `klok-integration-strategy-and-tactics` |
29
35
  | Route patterns, capabilities, configuration, logging and correlation, error handling, topics and CloudEvents, async tasks, ODL client, testing, migration from 1.x | `klok-framework-klok-quarkus-extension` |
30
36
  | Naming conventions, `klok-*` properties, CloudEvents attributes | `klok-framework-klok-common` |
31
37
  | Broker client configuration, subscriptions, request/reply | `klok-framework-klok-brokerclient-java` |
32
38
  | Gradle plugin, versions, catalog refresh | `klok-framework-klok-gradle-plugin` |
33
39
  | CI templates, release process, Vault roles, documentation chain, catalog update | `klok-environment-build-ci-templates` |
34
- | ODL REST and request/reply, tasks API | `klok-integration-klok-api-opensearch` |
35
- | ODL writers, log receivers | `klok-integration-klok-egress-opensearch` |
40
+ | Developer rules, prompt catalogue, broker access, identity, onboarding a tenant | `klok-framework-development-guide` |
41
+ | Concepts, roles, workflow and operations for people | `klok-user-guide` |
42
+ | ODL REST facade | `klok-integration-klok-api-odl` |
43
+ | Tasks API | `klok-integration-klok-api-tasks` |
44
+ | ODL services (writes, request/reply, change events), log receivers, task materializer | `klok-integration-klok-egress-opensearch` |
36
45
  | Scheduled queries, stale task detection | `klok-integration-klok-ingress-opensearch` |
37
46
  | OpenAI compatible API, AI tasks, Kloker gateway | `klok-integration-klok-api-ai` |
38
47
  | AI workers, transcription, transcript cache | `klok-integration-klok-egress-ai` |
39
48
 
40
- ## Procedure
49
+ Your tenant's own repositories have a skill each (`<tenant>-<group>-…-<repo>`, the doc id as the name) with the
50
+ facts in `data/facts.jsonl`, and `<tenant>-context` says which exist; open those before the published page when the
51
+ question is about what a project does.
52
+
53
+ ## What the answers mean
41
54
 
42
- 1. Identify the repositories involved and derive their doc ids.
43
- 2. Fetch `https://docs.klok.dev/<doc-id>/llm.txt` for each. On a 404 the project is not published yet: read the
44
- repository's `klok-docs/` folder from `main` instead (it is the source of the same pages).
45
- 3. Read the strategy documentation first when the task touches design, naming or onboarding; read the framework
46
- documentation before writing or changing a route.
47
- 4. Cite the page you relied on in your answer or merge request description, so reviewers can check it.
55
+ | `klok docs read` says | Meaning | What to do |
56
+ |---|---|---|
57
+ | the content, `source: server` or `source: cache` | the page as published, or the cached copy | read it |
58
+ | exit 4, "No or invalid GitLab token" | no `KLOK_GIT_APIKEY` (or `--token`), or the token is not valid | say so and ask for a token with `read_api`; do not guess |
59
+ | "not a member of the tenant or domain that owns …" (403) | the page exists and your GitLab account may not read it | say so and ask for access; do not guess and do not try another way in |
60
+ | "… is not published; read the repository's klok-docs/ on main" (404) | the project has no published documentation yet | read the source pages: `git -C <repo> fetch origin && git show origin/main:klok-docs/<page>.md`; `klok-docs/toc-manifest.json` lists them. The generated pages (routes, datamodels, configuration) exist only on the portal |
61
+ | "could not reach …" | no network, and nothing cached | say so |
62
+
63
+ ## Procedure
48
64
 
49
- The portal at https://docs.klok.dev requires a Keycloak login for tenant and domain scoped documentation; global
50
- documentation is readable by every platform user. If a fetch is denied, ask the user for the content or run in an
51
- environment that is logged in.
65
+ 1. Run `klok agents update --check` once at the start of the task; on exit 5 run `klok agents update`, so the
66
+ skills you read are the catalogs' current ones.
67
+ 2. Identify the repositories involved and derive their doc ids (`klok docs id <path>`).
68
+ 3. `klok docs read <doc-id>` for each. Handle 404, 401 and 403 as the table says.
69
+ 4. Read the strategy documentation first when the task touches design or naming, and the *Onboarding a tenant*
70
+ runbook in `klok-framework-development-guide` when it touches onboarding; read the framework documentation
71
+ before writing or changing a route.
72
+ 5. Cite the doc id and the `fetchedAt` from `klok docs read <doc-id> --json` in your answer or merge request
73
+ description, so reviewers can tell which version of the page you relied on.
@@ -5,7 +5,7 @@ description: Release and deploy on the Klok platform: pipelines, versions, the d
5
5
 
6
6
  # Releasing on Klok
7
7
 
8
- Read https://docs.klok.dev/klok-environment-build-ci-templates/llm.txt (release process by project type, Vault
8
+ Read `klok docs read klok-environment-build-ci-templates` (release process by project type, Vault
9
9
  roles and secrets, catalog update) before acting.
10
10
 
11
11
  ## The flow
@@ -21,41 +21,48 @@ roles and secrets, catalog update) before acting.
21
21
  `glab api --hostname git.klok.dev "projects/<id>/pipelines?per_page=3"`, then
22
22
  `projects/<id>/pipelines/<pid>/jobs` and `projects/<id>/jobs/<jid>/trace`. Fix a red pipeline before the next task.
23
23
  4. Framework changes across libraries are released in order: klok-common, broker client, extension. Wait for the
24
- catalog update of one before merging the next.
24
+ catalog update of one before merging the next: the catalog's commit for that release, not the merge, which only
25
+ starts the pipeline that produces it:
26
+ `glab api --hostname git.klok.dev "projects/klok%2Fframework%2Fklok-gradle-dependency-catalog/repository/commits?ref_name=main&per_page=10" | jq -r '.[].title'`.
27
+ 5. **Never cancel, stop or restart a running job, pipeline or deployment without a human's explicit yes.** A
28
+ release waiting on Maven Central or npm can take half an hour, and `main` pipelines queue behind each other or run
29
+ in parallel, so a long wait is normal. Check the current time with `date -u` before judging how long anything has run. Report the
30
+ job id, its run time, the last log lines with timestamps and what waits behind it, propose what to do, and wait.
31
+ 6. Merging is the human's step. Hand it over as KLOK-AGENTS.md *Manual steps are handed over as a guide* says:
32
+ every merge request with its link, whether the order matters and why, and for each later step the release
33
+ result to wait for and the command that shows it.
25
34
 
26
35
  ## Creating a new project
27
36
 
28
- **Create projects through klok.dev, or through the agent at https://agent.klok.dev** authenticated as the
29
- tenant and holding the right to create projects in the domain. That is the supported path, and it is meant to
30
- provision everything the project needs so the **first** pipeline is green with no manual step.
37
+ **Create projects through klok.dev**; the agent at https://agent.klok.dev asks klok.dev to do it. Creating a
38
+ project by hand — `glab api POST /projects`, or cloning the template and swapping the remote — is a legitimate
39
+ fallback under *Git är sanningen*.
31
40
 
32
- Creating a project by hand — `glab api POST /projects`, or cloning the template and swapping the remote — is a
33
- legitimate fallback under *Git är sanningen*, but it is not the normal path and it leaves work behind that the
34
- portal would have done. When taking the manual route, name the remaining human steps up front instead of
35
- letting them surface as a red pipeline.
36
-
37
- > Automatic Vault provisioning is **not implemented yet**: `klok/application/frontend-klok-dev` issue #7,
38
- > *Provision the Vault CI role when klok.dev or agent.klok.dev creates a project*. Until it ships, every
39
- > project — however it was created — needs a human to add the role.
41
+ Neither path provisions a pipeline schedule yet, nor for a deployed service the manifest, its Vault objects, its
42
+ Keycloak client and its broker grants (see below; tracked in klok/application/frontend-klok-dev#41 and #42). Name
43
+ those human steps up front instead of letting them surface
44
+ as a red pipeline. Vault CI login is not one of them: it needs nothing per project.
40
45
 
41
46
  ## Setting up a new project (what must exist)
42
47
 
43
48
  * Nothing in Vault per project. Every job logs in with the tenant CI role `<tenant>-ci` (a Vault JWT role bound
44
49
  to `namespace_path` `<tenant>` and `<tenant>/*`, carrying `default` and the tenant CI read policy
45
50
  `<tenant>-cicd-read`). The templates set `VAULT_AUTH_ROLE: $KLOK_TENANT-ci`. A tenant without the role fails
46
- every job at `auth/jwt/login` with `role "<tenant>-ci" could not be found`; the tenant setup is in
47
- `klok-docs/ci/tenant-onboarding.md` of `klok/environment/build-ci-templates`, every credential and the step
48
- that uses it in `klok-docs/ci/authentication.md` there.
51
+ every job at `auth/jwt/login` with `role "<tenant>-ci" could not be found`; the tenant setup is the
52
+ *Onboarding a tenant* runbook in `klok docs read klok-framework-development-guide`, every credential and the step
53
+ that uses it in `klok-docs/ci/authentication.md` of `klok/environment/build-ci-templates`.
49
54
  * For anything the pipeline resolves from a package registry (framework, plugin, datamodels): the project's
50
55
  top-level group must be on the package project's job token allowlist.
51
- * For a deployed service: a manifest under `klok/integration/<name>/`, the Kubernetes service account, a Vault
52
- Kubernetes role `<name>-role` with a policy reading `secret/data/<tenant>/integration/<name>/env`, and the env
53
- secret itself (broker credentials, service specific keys).
56
+ * For a deployed service: a manifest `<project path>/<name>-crds.yaml` in the manifests repository, written by
57
+ hand before the first deploy, with the Kubernetes service account; a Vault Kubernetes role `<name>-role` (with
58
+ `audience=https://kubernetes.default.svc`) and its policy `<name>-policy` reading
59
+ `secret/data/<tenant>/integration/<name>/env`; the env secret itself (broker credentials, service specific
60
+ keys); and the service's own Keycloak client with its broker grants.
54
61
  * Documentation projects need the tenant's docs upload client: Keycloak client `klok-docs-upload-<tenant>` and its
55
62
  id and secret in `secret/<tenant>/common/repo/cicd-environment-variables`.
56
63
 
57
- Vault is changed by humans only: propose the exact `vault write` / `vault policy write` command with every existing
58
- field, and let the user run it.
64
+ Agents never write to Vault: propose the exact `vault write` / `vault policy write` command with every existing
65
+ field, and let a person run it.
59
66
 
60
67
  ## Common failures and their meaning
61
68
 
@@ -1,12 +1,15 @@
1
1
  ---
2
2
  name: klok-route
3
- description: Write or change a Camel route on the Klok Quarkus extension 2.0 (onramp, processing, offramp, REST API handler, async task worker, custom route) with the correct naming, configuration, error handling, tests and klok-docs. Use for any change inside a Klok integration project.
3
+ description: Write or change a Camel route on the Klok Quarkus extension 2.0 (onramp, processing, offramp, REST API handler, async task worker, custom route) with the correct naming, topics and topic purpose, configuration, error handling, tests and klok-docs. Use for any change inside a Klok integration project.
4
4
  ---
5
5
 
6
6
  # Writing a Klok route
7
7
 
8
- Read https://docs.klok.dev/klok-framework-klok-quarkus-extension/llm.txt first (pages `patterns`, `naming`,
9
- `config`, `error-handling`, `testing`); read the `klok-docs` skill for other pages.
8
+ Read `klok docs read klok-framework-klok-quarkus-extension` first (pages `patterns`, `naming`,
9
+ `config`, `error-handling`, `testing`); read the `klok-docs` skill for other pages. A route is a leg of a solution:
10
+ the `klok-solution` skill finds and updates its solution document, before and after the change. Integration logic is
11
+ a Java route; doing it with the Python or TypeScript broker clients is a `[TACTICAL]` choice the solution document
12
+ motivates.
10
13
 
11
14
  ## Steps
12
15
 
@@ -17,24 +20,83 @@ Read https://docs.klok.dev/klok-framework-klok-quarkus-extension/llm.txt first (
17
20
  2. **Name it.** `<TENANTSHORT><NNN><LEG><seq>_<Description>`, one class per route, the solution number from the
18
21
  portal registry. Sub routes are `S`, timers `T`.
19
22
  3. **Bind the datamodels** as type parameters; take them from the catalog bundle, never hand written DTOs.
20
- 4. **Configure** through `routeConfig("key", default)` / `requiredRouteConfig("key")` and document every key in
23
+ 4. **Choose the topic purpose** of every topic the route publishes or consumes (next section).
24
+ 5. **Configure** through `routeConfig("key", default)` / `requiredRouteConfig("key")` and document every key in
21
25
  `application.properties` as `<RouteId>.<key>`. Reserved keys: `enabled`, `logevents.*`, `retry.*`, `dlq.*`,
22
26
  `topic.*`, `subscription.*`, `claimcheck.*`.
23
- 5. **Errors.** Throw `Klok4XXException` for caller errors, let retryable failures propagate; the base route logs,
27
+ 6. **Errors.** Throw `Klok4XXException` for caller errors, let retryable failures propagate; the base route logs,
24
28
  retries with backoff, republishes to `<topic>_DLQ` and emits the failure CloudEvent. Override `failureHandled()`
25
29
  only when the route owns its failures (custom routes that answer HTTP themselves).
26
- 6. **Capabilities** as interfaces: `SplitIncomingMessage`, `AggregateOutgoingMessages`, `ClaimCheck`,
30
+ A sub route has no error handling of its own: it catches only what is not an error, may throw an error again
31
+ as a more specific Klok exception, and leaves retries, the dead letter queue and the reply to its caller, whose
32
+ keys you configure (`klok docs read klok-framework-klok-quarkus-extension`, *Error handling*, *Sub routes*).
33
+ 7. **Capabilities** as interfaces: `SplitIncomingMessage`, `AggregateOutgoingMessages`, `ClaimCheck`,
27
34
  `KlokEmitCloudEventOnSuccess/Failure`, `PersistOutgoingMessages`, `KlokAsyncTaskInitiator`.
28
- 7. **Test** with `@QuarkusTest` and the recording broker client factory: assert the route status is `Started`,
29
- send a message, assert the published topic, body and `klok-*` properties.
30
- 8. **Document** in `klok-docs/`: the route in `routes.md`, decisions in `design-decisions.md`, operations notes.
31
- 9. Build (`./gradlew build`) in the task worktree (`klok-worktree` skill), push its `agent/<task-id>` branch,
32
- open a merge request ready for review, check the pipeline.
35
+ 8. **Test** with `@QuarkusTest` and the recording broker client factory: assert the route status is `Started`,
36
+ send a message, assert the published topic (the full name, purpose included), body and `klok-*` properties.
37
+ 9. **Document** in `klok-docs/`: the route in `routes.md` with the purpose of each topic and the project on its other
38
+ end, decisions in `design-decisions.md`, operations notes.
39
+ Annotate central classes with `@KlokDocs` (`dev.klok.framework.common.docs.KlokDocs`, klok-common): configuration
40
+ or standards several routes read, shared mappers, domain logic used by several routes. Each gets its own generated
41
+ page; write its Javadoc for a reader who has not seen the code. Not every class: the classes a route calls
42
+ directly reach its generated documentation without the annotation.
43
+ 10. Update the solution document when the route is new or its contract changed (`klok-solution` skill).
44
+ 11. Build (`./gradlew build`) in the task worktree (`klok-worktree` skill), push its `agent/<task-id>` branch,
45
+ open a merge request ready for review, check the pipeline.
46
+
47
+ ## Topic purpose
48
+
49
+ A topic is named after its datamodel: `<Model>[_<purpose>]`. The purpose (called *producer intention* in older code)
50
+ is an optional suffix that belongs to the publisher: the solution requires it, the producer sets it, and the consumer
51
+ sets the same purpose on its own route, so one general datamodel serves several flows:
52
+ `CallAiResponseV1_documentation` and `CallAiResponseV1_invoicecoding` both carry the typed `CallAiResponseV1`, and
53
+ each consumer receives only its own flow. The framework builds the full topic name on both sides from the model and
54
+ the purpose.
55
+
56
+ * **When.** Give a flow a purpose when its datamodel is, or may be, used by another flow. Without one, every consumer
57
+ of the model receives every message. No purpose only when every consumer wants every message.
58
+ * **Ingress** (onramps, webhook onramp): decide which flow the data belongs to and publish with that purpose.
59
+ **Egress** (offramps): consume the purpose the producer publishes on; find it in the producer's `klok-docs/` or its
60
+ route configuration, and ask when it is not written down. Never guess a purpose.
61
+ * **Where it is set.** Configuration wins over code.
62
+
63
+ | Pattern | Code | Configuration |
64
+ |---|---|---|
65
+ | onramps, `KlokWebhookOnrampRoute`, offramps, channel adapters, `Sync` variants | `topicPurpose()` | `<RouteId>.topic.purpose` |
66
+ | `KlokOfframpRouteWithResult` | `topicPurpose()`, `resultTopicPurpose()` (default: the request's) | `.topic.purpose`, `.topic.result-purpose` |
67
+ | `KlokProcessingRoute` | `incomingTopicPurpose()`, `outgoingTopicPurpose()` | `.topic.purpose`, `.topic.outgoing-purpose` |
68
+ | `KlokAsyncTaskWorkerRoute` | `taskType()`: the task type is the request topic's purpose | |
69
+ | REST handler with `task.initiate` | the `klok-task-type` header in `transformBody` | `.task.type` |
70
+
71
+ Put it in code when the purpose is part of what the route is; put it in configuration when the same route serves
72
+ several flows. Set `<SolutionId>.topic.purpose` only when every route of the solution is on that flow.
73
+ * **Name** the flow as both sides understand it (`created`, `reminder`, `documentation`): letters, digits, `-` and
74
+ `.`, never `_`, and no version or environment. Task types are `<domain>.<verb>-<object>`.
75
+ * **One purpose, no reader.** A topic has at most one purpose. A reader, also one in another tenant, is identified by
76
+ its subscription, whose name is its route id (`DEG012E1_...`; `<RouteId>.subscription.name` overrides it), and never
77
+ adds a suffix to the topic. When the content belongs to one reader, the publisher may choose that reader's tenant
78
+ slug as the purpose (`PersonnelScheduleExportV1_degerfors`); that is the publisher's choice, not something a reader
79
+ in another tenant needs.
80
+ * **A mismatch fails silently**: the two routes end up on different topics and nothing arrives. Check the start log
81
+ (`Route <RouteId> subscribes to <topic>`, `Route <RouteId> publishes <Model> to <topic>`) and assert the full topic
82
+ in the test.
83
+ * **Changing a purpose** moves the flow to a new topic: move the consumers first, then the producers, and let the old
84
+ topic drain.
85
+ * **Limit.** A route has one purpose, fixed when it starts. A shared service that answers each caller on the purpose
86
+ the caller chose is not supported yet (klok/framework/klok-quarkus-extension#32); do not write a subclass per
87
+ caller in the shared service, say so in the merge request instead.
88
+
89
+ The user guide page *Topic Purpose: One Datamodel, Several Flows* (`klok docs read klok-user-guide`)
90
+ explains the reasons and the naming in full.
33
91
 
34
92
  ## Things that are always wrong
35
93
 
36
94
  * A route id that does not start with the solution id, or two routes with the same id.
37
95
  * Publishing to a hand written topic string; use the resolver and the datamodel.
96
+ * Consuming a shared topic and discarding the messages of other flows (a filter on a payload field, an ODL lookup
97
+ per message) where the producer could publish the flow with a topic purpose.
98
+ * A purpose with `_`, a version or an environment in it, a second suffix after the purpose, a tenant in it that the
99
+ publisher did not choose, or a purpose guessed instead of agreed with the other side.
38
100
  * Choosing a tenant, index, repository or account from a field in the message; the topic decides the tenant.
39
101
  * Sharing broker credentials between services, or granting a whole namespace of another domain or tenant.
40
102
  * Publishing into another tenant's namespace, with `<RouteId>.topic.tenant-override` or otherwise; a route consumes