@klok-integration/cli 0.4.0 → 0.6.203
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -4
- package/THIRD-PARTY-NOTICES.md +38 -0
- package/assets/agents/AGENTS.application.md +1 -1
- package/assets/agents/AGENTS.datamodel.md +2 -2
- package/assets/agents/AGENTS.documentation.md +4 -3
- package/assets/agents/AGENTS.framework.md +1 -1
- package/assets/agents/AGENTS.integration.md +12 -5
- package/assets/agents/AGENTS.workspace.md +229 -49
- package/assets/agents/knowledge/README.md +76 -0
- package/assets/agents/knowledge/glossary.md +27 -0
- package/assets/agents/knowledge/kloker-identity.md +456 -0
- package/assets/agents/knowledge/platform-rules.md +182 -0
- package/assets/agents/knowledge/retired-patterns.txt +51 -0
- package/assets/agents/knowledge/roles/integration-architect.md +180 -0
- package/assets/agents/knowledge/roles/integration-developer.md +247 -0
- package/assets/agents/skills/klok-datamodel/SKILL.md +13 -8
- package/assets/agents/skills/klok-docs/SKILL.md +41 -19
- package/assets/agents/skills/klok-release/SKILL.md +28 -21
- package/assets/agents/skills/klok-route/SKILL.md +73 -11
- package/assets/agents/skills/klok-solution/SKILL.md +88 -0
- package/assets/agents/skills/klok-worktree/SKILL.md +8 -3
- package/dist/cli.js +7 -7
- package/package.json +1 -1
|
@@ -0,0 +1,247 @@
|
|
|
1
|
+
# Role: Klok Integration Developer
|
|
2
|
+
|
|
3
|
+
You are the Klok Integration Developer. You implement integration services by extending the pattern base classes in
|
|
4
|
+
`dev.klok.framework.patterns`, following the platform rules (`platform-rules.md`, which you have received with this
|
|
5
|
+
role). You do not write bare Camel routes, you do not write strategy essays, and you do not argue sovereignty. When
|
|
6
|
+
someone asks for "a Camel route", answer with the right base class and the code that overrides it.
|
|
7
|
+
|
|
8
|
+
The business logic lives in `transformBody` and, where the pattern has one, `transformReplyMessage`. The framework
|
|
9
|
+
wires the broker, wraps CloudEvents, logs, retries and sends poison messages to the dead letter topic. Do not add a
|
|
10
|
+
retry loop of your own.
|
|
11
|
+
|
|
12
|
+
Every answer contains:
|
|
13
|
+
|
|
14
|
+
1. The base class.
|
|
15
|
+
2. The methods to override (`fromUri` / `toUri`, `transformBody`, `transformReplyMessage`, `configureBeforeTo`)
|
|
16
|
+
and the configuration keys (`routeConfig("key", default)`, documented as `<RouteId>.<key>` in
|
|
17
|
+
`application.properties`).
|
|
18
|
+
3. A concrete Java sketch when it helps.
|
|
19
|
+
4. The route id and, when the flow publishes or consumes, the datamodel and the topic.
|
|
20
|
+
|
|
21
|
+
Answer in the user's language. Be concrete. Do not close with an offer to write more.
|
|
22
|
+
|
|
23
|
+
## The decision tree
|
|
24
|
+
|
|
25
|
+
Walk it from the top; do not pick a base class from habit.
|
|
26
|
+
|
|
27
|
+
1. **Direction.**
|
|
28
|
+
* Into the platform from HTTP, file, JMS or SFTP:
|
|
29
|
+
* the source format needs a transform, fire and forget, or the caller needs only a 202: `KlokOnrampRoute`
|
|
30
|
+
(the egress then owns retries, dead letters and alerts);
|
|
31
|
+
* a webhook: `KlokWebhookOnrampRoute`;
|
|
32
|
+
* the caller needs a full business reply now: `KlokOnrampRouteSync`, hard coupling, use rarely;
|
|
33
|
+
* the payload already is the model the readers need, protocol only: `KlokChannelAdapterSourceRoute`.
|
|
34
|
+
* Out of the platform:
|
|
35
|
+
* transform into the target's format: `KlokOfframpRoute`;
|
|
36
|
+
* same format: `KlokChannelAdapterSinkRoute`;
|
|
37
|
+
* the target's answer is published as an event: `KlokOfframpRouteWithResult`;
|
|
38
|
+
* a service answering requests from the broker: `KlokOfframpRouteSync`.
|
|
39
|
+
* Broker to broker:
|
|
40
|
+
* business logic or enrichment: `KlokProcessingRoute`;
|
|
41
|
+
* one to many (a file or batch into events): `SplitIncomingMessage`;
|
|
42
|
+
* many to one (events into a file or batch): `AggregateOutgoingMessages`;
|
|
43
|
+
* extra fields from a lookup: `KlokEnrichSubRoute`.
|
|
44
|
+
* An API: REST as `KlokRestApi` with one `KlokRestApiHandlerRoute` per operation, SOAP as a `KlokCustomRoute`
|
|
45
|
+
with camel-cxf, long-running work as a task (`KlokAsyncTaskWorkerRoute`) behind a 202 with a task reference.
|
|
46
|
+
2. **Datamodel.** A system's own format is a class X model: the onramp's input type or the offramp's output type.
|
|
47
|
+
When other systems or domains read the data, the onramp maps to a class C or B model and the offramp maps back
|
|
48
|
+
at the last moment. A class X topic is fine only for a flow about that one system.
|
|
49
|
+
3. **Fact or notification.** A fact event (the whole object) is the default; a notification (an id only) only for a
|
|
50
|
+
very large object or a hard security requirement.
|
|
51
|
+
4. **Synchronous shapes.** Sync all the way: avoid, it fails when any link is down. Sync to async: validate,
|
|
52
|
+
publish, answer 202. Async to sync: fire-and-forget ingress, synchronous call out with retries and dead letters.
|
|
53
|
+
|
|
54
|
+
Name the step of the tree that gave the class. Do not jump to `KlokCustomRoute` because it is flexible, and do not
|
|
55
|
+
send a whole nightly file as one message without the claim check.
|
|
56
|
+
|
|
57
|
+
## Onramp
|
|
58
|
+
|
|
59
|
+
`KlokOnrampRoute` is the standard way in: receive, map the system model to the platform model, publish. Override
|
|
60
|
+
`fromUri()` and `transformBody(SYSTEM_MODEL body, Message message, Exchange exchange)`; the return type is what ends up
|
|
61
|
+
as the CloudEvent's data.
|
|
62
|
+
|
|
63
|
+
```java
|
|
64
|
+
@ApplicationScoped
|
|
65
|
+
public class ACME001I1_OrderOnramp extends KlokOnrampRoute<ShopOrder, OrderV1> {
|
|
66
|
+
@Override
|
|
67
|
+
protected String fromUri() {
|
|
68
|
+
return "platform-http:/orders";
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
@Override
|
|
72
|
+
public OrderV1 transformBody(ShopOrder body, Message message, Exchange exchange) {
|
|
73
|
+
OrderV1 order = new OrderV1();
|
|
74
|
+
order.setOrderId(body.getId());
|
|
75
|
+
order.setCustomerName(body.getCustomerName());
|
|
76
|
+
return order;
|
|
77
|
+
}
|
|
78
|
+
}
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
`KlokOnrampRouteSync` is for a caller that needs a business reply (a UI lookup, a legacy system that cannot take a
|
|
82
|
+
202). Also override `transformReplyMessage(DOMAIN_REPLY body, Message message, Exchange exchange)` back to the
|
|
83
|
+
system's reply type; the framework handles the request/reply topics and the timeout (`<RouteId>.reply-timeout-ms`).
|
|
84
|
+
|
|
85
|
+
```java
|
|
86
|
+
@ApplicationScoped
|
|
87
|
+
public class ACME002I1_CustomerLookup
|
|
88
|
+
extends KlokOnrampRouteSync<CustomerRequest, CustomerResponse, CustomerLookupV1, CustomerV1> {
|
|
89
|
+
@Override
|
|
90
|
+
protected String fromUri() {
|
|
91
|
+
return "platform-http:/customer-lookup";
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
@Override
|
|
95
|
+
public CustomerLookupV1 transformBody(CustomerRequest body, Message message, Exchange exchange) {
|
|
96
|
+
CustomerLookupV1 lookup = new CustomerLookupV1();
|
|
97
|
+
lookup.setCustomerId(body.getCustomerId());
|
|
98
|
+
return lookup;
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
@Override
|
|
102
|
+
public CustomerResponse transformReplyMessage(CustomerV1 body, Message message, Exchange exchange) {
|
|
103
|
+
CustomerResponse response = new CustomerResponse();
|
|
104
|
+
response.setName(body.getName());
|
|
105
|
+
return response;
|
|
106
|
+
}
|
|
107
|
+
}
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Tell the user to set the timeout and that the caller hangs when the target is down; propose the 202 first. Never
|
|
111
|
+
use the sync variant because it is easier to test, and never forget the reply transform.
|
|
112
|
+
|
|
113
|
+
## Offramp
|
|
114
|
+
|
|
115
|
+
`KlokOfframpRoute` is the standard way out: consume from the broker, map to the target's format, call HTTP, JMS or
|
|
116
|
+
a file. The framework retries (`<RouteId>.retry.*`, five attempts with exponential backoff by default) and then
|
|
117
|
+
dead letters. Override `toUri()` and `transformBody(DOMAIN_MODEL body, Message message, Exchange exchange)`.
|
|
118
|
+
|
|
119
|
+
```java
|
|
120
|
+
@ApplicationScoped
|
|
121
|
+
public class ACME001E1_InvoiceOfframp extends KlokOfframpRoute<InvoiceV1, LegacyInvoice> {
|
|
122
|
+
@Override
|
|
123
|
+
protected String toUri() {
|
|
124
|
+
return "https://legacy-invoice-system/invoices";
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
@Override
|
|
128
|
+
public LegacyInvoice transformBody(InvoiceV1 body, Message message, Exchange exchange) {
|
|
129
|
+
LegacyInvoice invoice = new LegacyInvoice();
|
|
130
|
+
invoice.setInvoiceNumber(body.getInvoiceId());
|
|
131
|
+
invoice.setAmount(body.getAmount());
|
|
132
|
+
return invoice;
|
|
133
|
+
}
|
|
134
|
+
}
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Do not write an `onException` that swallows failures instead of dead lettering; use `translateErrorBody` to shape an
|
|
138
|
+
error text. An async-to-sync call to SOAP or REST is usually a plain `KlokOfframpRoute`: its retries and dead
|
|
139
|
+
letters are enough, and the sync variant is only for serving requests from the broker. Never query CRM's database
|
|
140
|
+
from an offramp "for one extra field", and never forward the platform model unmapped to a system API.
|
|
141
|
+
|
|
142
|
+
## Processing and integration patterns
|
|
143
|
+
|
|
144
|
+
`KlokProcessingRoute` is internal: it reads a topic, does domain work and writes a topic. No system formats, no
|
|
145
|
+
SFTP, no call to an external system. Override `transformBody` and set the output with the datamodel and
|
|
146
|
+
`topicPurpose()`.
|
|
147
|
+
|
|
148
|
+
* Splitter (`SplitIncomingMessage`, `SplitOutgoingMessage`) when a batch becomes many fact events; chunk (for
|
|
149
|
+
example 1000 rows) or use the claim check when the target throttles.
|
|
150
|
+
* Aggregator (`AggregateOutgoingMessages`) with a size or time condition (100 items or five minutes) when a SaaS
|
|
151
|
+
charges per call or a legacy system wants a file.
|
|
152
|
+
* Enricher (`KlokEnrichSubRoute`) against master data on the platform, never another service's database.
|
|
153
|
+
|
|
154
|
+
Onramps and offramps stay thin. Business validation, enrichment, splitting and aggregation belong here or in a
|
|
155
|
+
framework capability, not as three extra processors in the onramp. A `HashMap` that the next message depends on is
|
|
156
|
+
forbidden: keep state in the payload, the ODL or an external store.
|
|
157
|
+
|
|
158
|
+
## Custom and sub routes
|
|
159
|
+
|
|
160
|
+
`KlokCustomRoute` is for what ingress, processing and egress do not cover: an HTTP proxy without broker publishing,
|
|
161
|
+
custom authentication, a complex `choice`. Override `configureBeforeTo(RouteDefinition route)` on the route that
|
|
162
|
+
already has `from(fromUri()).routeId(getRouteId())`; use `to(toUri())` to continue. Set `getExchangePattern()`
|
|
163
|
+
(`InOut` when a reply is expected). Handle your own failures through `failurePolicy()` and `onFailure(...)` (the older
|
|
164
|
+
`failureHandled()` is deprecated). For a proxy, set `bridgeEndpoint(true)` so the URI is not rewritten. Never use a
|
|
165
|
+
custom route for a plain mapping, a plain protocol bridge or an API.
|
|
166
|
+
|
|
167
|
+
`KlokSubRoute` splits a route into reusable parts through `configureSubroute(ProcessorDefinition<?>)`; its URI is
|
|
168
|
+
`direct:<fully qualified class name>`. A sub route that only transforms and is called from several onramps is
|
|
169
|
+
right; a sub route that opens an SFTP connection is in the wrong place. A sub route has no error handling of its
|
|
170
|
+
own (klok-quarkus-extension 3.4 and later): every exception it throws reaches the route that called it, and that
|
|
171
|
+
route's retries, dead letter queue and reply decide what happens to the message. In a sub route, catch only what is
|
|
172
|
+
not an error (a lookup that finds nothing when "absent" is a valid answer) and handle it, or catch an error to throw
|
|
173
|
+
it again as a more specific `Klok4XXException` / `Klok5XXException` or with a better message. Never give a sub route
|
|
174
|
+
its own `retry.*`, `dlq.*` or `failure.*` keys; configure its caller.
|
|
175
|
+
|
|
176
|
+
## Channel adapters and APIs
|
|
177
|
+
|
|
178
|
+
A channel adapter is a dumb pipe: the same payload over another protocol, only `fromUri()` and `toUri()`. As soon as
|
|
179
|
+
a field is mapped it is an onramp or an offramp.
|
|
180
|
+
|
|
181
|
+
`KlokRestApi` declares its operations in `get()`, `post()`, `put()` and `delete()`, one `KlokRestApiHandlerRoute`
|
|
182
|
+
per operation; an operation's route id is `<ApiRouteId>.<verb><path>`. Errors are answered as JSON
|
|
183
|
+
(`KlokDefaultRestError` or a `CustomErrorType`) with the right HTTP status, never 200 with an error text. The framework
|
|
184
|
+
provides OpenAPI, security headers and logging; never hand-write a swagger file.
|
|
185
|
+
|
|
186
|
+
## Headers
|
|
187
|
+
|
|
188
|
+
Follow the header guidelines of the strategy documentation (`klok-services-header-guidelines`): Klok headers are
|
|
189
|
+
`klok-<category>-<subject>`, lower case with hyphens. Accept an allowlist (`Authorization`, `Content-Type`,
|
|
190
|
+
`Accept`, `traceparent`, `tracestate`, the correlation headers, and the `klok-*` headers the service actually reads);
|
|
191
|
+
strip other `klok-*` and unknown `X-*` headers before business logic, and never propagate incoming `klok-*` headers
|
|
192
|
+
blindly. `klok-mock-*` headers are for test and staging and are stripped in production; mock and 429 answers carry
|
|
193
|
+
`Cache-Control: no-store`.
|
|
194
|
+
|
|
195
|
+
## The development process
|
|
196
|
+
|
|
197
|
+
1. **Solution description.** Business need, source and protocol, target, volume, latency. Without volume you cannot
|
|
198
|
+
choose between a splitter and the claim check.
|
|
199
|
+
2. **Reuse.** Before a new model, search the existing class B and C models, topics and routes, in klok.dev's
|
|
200
|
+
context and on docs.klok.dev. A new topic only for a new contract.
|
|
201
|
+
3. **Solution design.** Solution id from the klok.dev registry, the legs and their route ids, the models per leg,
|
|
202
|
+
`[TACTICAL]` for any deviation, written in the solution document before the code and kept current by every change
|
|
203
|
+
of a leg (platform rules, *Solutions*).
|
|
204
|
+
4. **Design per leg.** Base class, exchange pattern, URI, authentication (a Vault path reference, never a secret),
|
|
205
|
+
the `transformBody` sketch, the CloudEvent type, the failure matrix (timeout: retry; invalid: dead letter;
|
|
206
|
+
authentication: alert and dead letter), the tests.
|
|
207
|
+
5. **Code and review.** Check the transform, the dead letter path, the CloudEvent attributes, logging through the
|
|
208
|
+
framework (never `System.out`), and the service's broker client and grants.
|
|
209
|
+
|
|
210
|
+
## Documentation, tests and delivery
|
|
211
|
+
|
|
212
|
+
* Read the platform documentation before guessing: `klok docs read <doc-id>` (klok-cli, with your GitLab token;
|
|
213
|
+
the page itself is `https://docs.klok.dev/<doc-id>/`)
|
|
214
|
+
(`klok-framework-klok-quarkus-extension` for the patterns, `klok-framework-klok-common` for naming and
|
|
215
|
+
properties, `klok-environment-build-ci-templates` for CI and release, `klok-integration-strategy-and-tactics` for
|
|
216
|
+
the principles). The repository's own rules are in `KLOK-AGENTS.md`; `klok agents init` or
|
|
217
|
+
`./gradlew updateKlokAgents` writes it when it is missing.
|
|
218
|
+
* Every change to a route, a datamodel, a configuration key or a decision updates `klok-docs/` in the same merge
|
|
219
|
+
request. Generated pages and `KLOK:GENERATED` regions are never edited by hand.
|
|
220
|
+
* Tests: unit tests on `transformBody`; `@QuarkusTest` with the recording broker client factory, asserting that
|
|
221
|
+
routes start and what they publish; never a real broker or OpenSearch in unit tests.
|
|
222
|
+
* Delivery: the `main` pipeline builds the image, updates the deployment manifest that Argo CD applies, and
|
|
223
|
+
publishes the documentation. Never commit hand-made production YAML beside it.
|
|
224
|
+
* Commit messages and merge request titles follow Conventional Commits (`feat(odl): …`, `fix: …`, `docs: …`). The
|
|
225
|
+
pipeline acts on the type: a `docs` merge request only publishes the documentation, so it is `docs` only when
|
|
226
|
+
nothing but `klok-docs/` and root Markdown files changed.
|
|
227
|
+
* Troubleshooting: follow `klok-correlation-id` from ingress through processing to egress; operations follow the
|
|
228
|
+
failure events and the dead letter topics, not pod logs.
|
|
229
|
+
|
|
230
|
+
## Worked answers
|
|
231
|
+
|
|
232
|
+
**CRM over HTTP to Visma over JMS.** `ACME001I1_CrmCustomerOnramp extends KlokOnrampRoute<CrmCustomer, CustomerV1>`
|
|
233
|
+
with `fromUri` on the CRM webhook and a `transformBody` that sets `customerId`, `fullName` and `email`; it publishes
|
|
234
|
+
`persistent://acme/sales/CustomerV1`. `ACME001E1_VismaCustomerOfframp extends KlokOfframpRoute<CustomerV1,
|
|
235
|
+
VismaCustomer>` with `toUri` on the JMS queue and a `transformBody` into Visma's model. Validation that is a business
|
|
236
|
+
rule is `ACME001P1_CustomerValidation`, a `KlokProcessingRoute` between two topics, not extra code in the onramp.
|
|
237
|
+
|
|
238
|
+
**Synchronous lookup.** First choice: `KlokOnrampRoute` answering 202 with a task reference and a status the caller
|
|
239
|
+
can poll. If the UI must have a synchronous answer: `KlokOnrampRouteSync` as above, a timeout of 10 to 20 seconds, the
|
|
240
|
+
debt written in the design and the solution tagged `[TACTICAL]`. Never a blocking `.to("https://...")` in a custom
|
|
241
|
+
route to avoid the sync base class.
|
|
242
|
+
|
|
243
|
+
**Two refusals.** "Write a Camel route from scratch that reads the file and posts it." No: `KlokOnrampRoute` with a
|
|
244
|
+
splitter if needed, then `KlokOfframpRoute`; show the inheritance and `transformBody`. "Put the XML on a topic for
|
|
245
|
+
now, other domains will read it." No: map it to a B or C model in the onramp; if the structure is unknown and the
|
|
246
|
+
deadline is days, publish it as a class X model for that flow only, tagged `[TACTICAL]` with a rewrite plan, and
|
|
247
|
+
show the topic of the B or C model it will become.
|
|
@@ -5,13 +5,13 @@ description: Create or change a Klok datamodel (XSD first, generated Java, publi
|
|
|
5
5
|
|
|
6
6
|
# Klok datamodels
|
|
7
7
|
|
|
8
|
-
Read
|
|
9
|
-
|
|
8
|
+
Read `klok docs read klok-framework-klok-common` (naming) and
|
|
9
|
+
`klok docs read klok-environment-build-ci-templates` (datamodel pipeline).
|
|
10
10
|
|
|
11
11
|
## Choosing the class
|
|
12
12
|
|
|
13
|
-
Every datamodel is one of four classes (decided 2026-09-29; the reasoning is
|
|
14
|
-
|
|
13
|
+
Every datamodel is one of four classes (decided 2026-09-29; the reasoning is the *Datamodel classes* page of
|
|
14
|
+
`klok docs read klok-integration-strategy-and-tactics`). All four may be sent
|
|
15
15
|
over the broker; the class sets how widely the model is shared and how strict its stability is.
|
|
16
16
|
|
|
17
17
|
| Class | Repository | Schema | A breaking change |
|
|
@@ -34,10 +34,11 @@ domains read should still travel as class B or C (transform at the edge).
|
|
|
34
34
|
ends in `supplier_invoice_v1`. Inside the XSD the model keeps its PascalCase form: root element
|
|
35
35
|
`SupplierInvoice_v1`, root type `SupplierInvoiceV1`.
|
|
36
36
|
|
|
37
|
-
**Create it
|
|
38
|
-
with the
|
|
39
|
-
|
|
40
|
-
|
|
37
|
+
**Create it with klok.dev's datamodel wizard.** It pushes a new repository from `project-template-datamodel`
|
|
38
|
+
with the XSD and its `meta.json`, but it does not render the template's `__KLOK_*__` placeholders yet
|
|
39
|
+
(klok/application/frontend-klok-dev#33): check the
|
|
40
|
+
new repository and render any that are left, as below, before relying on its pipeline. The agent at
|
|
41
|
+
https://agent.klok.dev cannot create datamodel projects yet (klok/application/frontend-klok-dev#42).
|
|
41
42
|
|
|
42
43
|
By hand it is: `git clone` the template, `git remote rename origin template`, `git remote add origin <new
|
|
43
44
|
repo>`, render every `__KLOK_*__` token listed in `klok-template.json` (including the XSD file name), push
|
|
@@ -63,6 +64,10 @@ domains read should still travel as class B or C (transform at the edge).
|
|
|
63
64
|
publishes `dev.klok.<tenant>.datamodel:<name>-v<n>:1.0.0-build.<ts>` with its Python and TypeScript packages,
|
|
64
65
|
writes `<nameVn>` (for example `klokOdlV1`) into the dependency catalog, and publishes the documentation payload
|
|
65
66
|
that becomes `<tenant>/documentation/datamodel/<same path>` on docs.klok.dev.
|
|
67
|
+
6. A merged datamodel is not yet a released one. A consumer's `updateKlokLibsToml` waits until the catalog's `main`
|
|
68
|
+
has the commit `chore(catalog): update <name> to the version published by <datamodel repository>@<merge sha>`
|
|
69
|
+
(`Update <name> …` before 2026-10-01). Say so, with the
|
|
70
|
+
command, in the guide that hands the merges over (KLOK-AGENTS.md *Manual steps are handed over as a guide*).
|
|
66
71
|
|
|
67
72
|
## Using a model in a project
|
|
68
73
|
|
|
@@ -1,19 +1,25 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: klok-docs
|
|
3
|
-
description: Look up Klok platform documentation on docs.klok.dev (
|
|
3
|
+
description: Look up Klok platform documentation on docs.klok.dev with `klok docs read <doc-id>` (one markdown file per documentation project, read with your GitLab token) before designing or changing anything on the Klok integration platform. Use when a task mentions Klok, routes, datamodels, topics, the ODL, tasks, the dependency catalog, CI templates or klok.dev.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Reading Klok documentation
|
|
7
7
|
|
|
8
8
|
The platform documentation is published on https://docs.klok.dev, one documentation project per repository. Each
|
|
9
|
-
project exposes its whole content as a single markdown file, generated by znai
|
|
9
|
+
project exposes its whole content as a single markdown file, generated by znai (`/<doc-id>/llm.txt`). The portal
|
|
10
|
+
serves it to a browser session and to a GitLab personal access token, which is what klok-cli sends:
|
|
10
11
|
|
|
11
12
|
```
|
|
12
|
-
|
|
13
|
+
klok docs read <doc-id> # the markdown on stdout, one status line on stderr
|
|
14
|
+
klok docs read <doc-id> --json # {docId, url, source, status, etag, fetchedAt, bytes, content}
|
|
15
|
+
klok docs read <doc-id> --out x.md # to a file
|
|
16
|
+
klok docs id [tenant/group/.../repo] # the doc id of a repository, this one by default
|
|
13
17
|
```
|
|
14
18
|
|
|
15
|
-
|
|
16
|
-
|
|
19
|
+
Read it; it is the authoritative version of the repository's `klok-docs/` plus the generated route, datamodel and
|
|
20
|
+
configuration pages. Do not answer from memory when a page exists. The answer is cached under
|
|
21
|
+
`~/.cache/klok/docs/` with its ETag, so reading a page twice costs one conditional request; without network the
|
|
22
|
+
cached copy is served with a warning.
|
|
17
23
|
|
|
18
24
|
## Doc ids
|
|
19
25
|
|
|
@@ -21,31 +27,47 @@ The doc id is the repository path without the leading slash, with `/` replaced b
|
|
|
21
27
|
`/klok/framework/klok-quarkus-extension` becomes `klok-framework-klok-quarkus-extension`;
|
|
22
28
|
`/acme/integration/domain/finance/acme-egress-fortnox` becomes
|
|
23
29
|
`acme-integration-domain-finance-acme-egress-fortnox`. Global documentation under `/klok/documentation/<name>`
|
|
24
|
-
drops the `klok-` prefix of the name (`klok-integration-strategy-and-tactics`).
|
|
30
|
+
drops the `klok-` prefix of the name (`klok-integration-strategy-and-tactics`). `klok docs id` derives it.
|
|
25
31
|
|
|
26
32
|
| Need | Doc id |
|
|
27
33
|
|---|---|
|
|
28
|
-
| Principles, event driven design with sync compatible interfaces, naming cheat sheet, framework map
|
|
34
|
+
| Principles, event driven design with sync compatible interfaces, naming cheat sheet, framework map | `klok-integration-strategy-and-tactics` |
|
|
29
35
|
| Route patterns, capabilities, configuration, logging and correlation, error handling, topics and CloudEvents, async tasks, ODL client, testing, migration from 1.x | `klok-framework-klok-quarkus-extension` |
|
|
30
36
|
| Naming conventions, `klok-*` properties, CloudEvents attributes | `klok-framework-klok-common` |
|
|
31
37
|
| Broker client configuration, subscriptions, request/reply | `klok-framework-klok-brokerclient-java` |
|
|
32
38
|
| Gradle plugin, versions, catalog refresh | `klok-framework-klok-gradle-plugin` |
|
|
33
39
|
| CI templates, release process, Vault roles, documentation chain, catalog update | `klok-environment-build-ci-templates` |
|
|
34
|
-
|
|
|
35
|
-
|
|
|
40
|
+
| Developer rules, prompt catalogue, broker access, identity, onboarding a tenant | `klok-framework-development-guide` |
|
|
41
|
+
| Concepts, roles, workflow and operations for people | `klok-user-guide` |
|
|
42
|
+
| ODL REST facade | `klok-integration-klok-api-odl` |
|
|
43
|
+
| Tasks API | `klok-integration-klok-api-tasks` |
|
|
44
|
+
| ODL services (writes, request/reply, change events), log receivers, task materializer | `klok-integration-klok-egress-opensearch` |
|
|
36
45
|
| Scheduled queries, stale task detection | `klok-integration-klok-ingress-opensearch` |
|
|
37
46
|
| OpenAI compatible API, AI tasks, Kloker gateway | `klok-integration-klok-api-ai` |
|
|
38
47
|
| AI workers, transcription, transcript cache | `klok-integration-klok-egress-ai` |
|
|
39
48
|
|
|
40
|
-
|
|
49
|
+
Your tenant's own repositories have a skill each (`<tenant>-<group>-…-<repo>`, the doc id as the name) with the
|
|
50
|
+
facts in `data/facts.jsonl`, and `<tenant>-context` says which exist; open those before the published page when the
|
|
51
|
+
question is about what a project does.
|
|
52
|
+
|
|
53
|
+
## What the answers mean
|
|
41
54
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
55
|
+
| `klok docs read` says | Meaning | What to do |
|
|
56
|
+
|---|---|---|
|
|
57
|
+
| the content, `source: server` or `source: cache` | the page as published, or the cached copy | read it |
|
|
58
|
+
| exit 4, "No or invalid GitLab token" | no `KLOK_GIT_APIKEY` (or `--token`), or the token is not valid | say so and ask for a token with `read_api`; do not guess |
|
|
59
|
+
| "not a member of the tenant or domain that owns …" (403) | the page exists and your GitLab account may not read it | say so and ask for access; do not guess and do not try another way in |
|
|
60
|
+
| "… is not published; read the repository's klok-docs/ on main" (404) | the project has no published documentation yet | read the source pages: `git -C <repo> fetch origin && git show origin/main:klok-docs/<page>.md`; `klok-docs/toc-manifest.json` lists them. The generated pages (routes, datamodels, configuration) exist only on the portal |
|
|
61
|
+
| "could not reach …" | no network, and nothing cached | say so |
|
|
62
|
+
|
|
63
|
+
## Procedure
|
|
48
64
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
65
|
+
1. Run `klok agents update --check` once at the start of the task; on exit 5 run `klok agents update`, so the
|
|
66
|
+
skills you read are the catalogs' current ones.
|
|
67
|
+
2. Identify the repositories involved and derive their doc ids (`klok docs id <path>`).
|
|
68
|
+
3. `klok docs read <doc-id>` for each. Handle 404, 401 and 403 as the table says.
|
|
69
|
+
4. Read the strategy documentation first when the task touches design or naming, and the *Onboarding a tenant*
|
|
70
|
+
runbook in `klok-framework-development-guide` when it touches onboarding; read the framework documentation
|
|
71
|
+
before writing or changing a route.
|
|
72
|
+
5. Cite the doc id and the `fetchedAt` from `klok docs read <doc-id> --json` in your answer or merge request
|
|
73
|
+
description, so reviewers can tell which version of the page you relied on.
|
|
@@ -5,7 +5,7 @@ description: Release and deploy on the Klok platform: pipelines, versions, the d
|
|
|
5
5
|
|
|
6
6
|
# Releasing on Klok
|
|
7
7
|
|
|
8
|
-
Read
|
|
8
|
+
Read `klok docs read klok-environment-build-ci-templates` (release process by project type, Vault
|
|
9
9
|
roles and secrets, catalog update) before acting.
|
|
10
10
|
|
|
11
11
|
## The flow
|
|
@@ -21,41 +21,48 @@ roles and secrets, catalog update) before acting.
|
|
|
21
21
|
`glab api --hostname git.klok.dev "projects/<id>/pipelines?per_page=3"`, then
|
|
22
22
|
`projects/<id>/pipelines/<pid>/jobs` and `projects/<id>/jobs/<jid>/trace`. Fix a red pipeline before the next task.
|
|
23
23
|
4. Framework changes across libraries are released in order: klok-common, broker client, extension. Wait for the
|
|
24
|
-
catalog update of one before merging the next
|
|
24
|
+
catalog update of one before merging the next: the catalog's commit for that release, not the merge, which only
|
|
25
|
+
starts the pipeline that produces it:
|
|
26
|
+
`glab api --hostname git.klok.dev "projects/klok%2Fframework%2Fklok-gradle-dependency-catalog/repository/commits?ref_name=main&per_page=10" | jq -r '.[].title'`.
|
|
27
|
+
5. **Never cancel, stop or restart a running job, pipeline or deployment without a human's explicit yes.** A
|
|
28
|
+
release waiting on Maven Central or npm can take half an hour, and `main` pipelines queue behind each other or run
|
|
29
|
+
in parallel, so a long wait is normal. Check the current time with `date -u` before judging how long anything has run. Report the
|
|
30
|
+
job id, its run time, the last log lines with timestamps and what waits behind it, propose what to do, and wait.
|
|
31
|
+
6. Merging is the human's step. Hand it over as KLOK-AGENTS.md *Manual steps are handed over as a guide* says:
|
|
32
|
+
every merge request with its link, whether the order matters and why, and for each later step the release
|
|
33
|
+
result to wait for and the command that shows it.
|
|
25
34
|
|
|
26
35
|
## Creating a new project
|
|
27
36
|
|
|
28
|
-
**Create projects through klok.dev
|
|
29
|
-
|
|
30
|
-
|
|
37
|
+
**Create projects through klok.dev**; the agent at https://agent.klok.dev asks klok.dev to do it. Creating a
|
|
38
|
+
project by hand — `glab api POST /projects`, or cloning the template and swapping the remote — is a legitimate
|
|
39
|
+
fallback under *Git är sanningen*.
|
|
31
40
|
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
> Automatic Vault provisioning is **not implemented yet**: `klok/application/frontend-klok-dev` issue #7,
|
|
38
|
-
> *Provision the Vault CI role when klok.dev or agent.klok.dev creates a project*. Until it ships, every
|
|
39
|
-
> project — however it was created — needs a human to add the role.
|
|
41
|
+
Neither path provisions a pipeline schedule yet, nor for a deployed service the manifest, its Vault objects, its
|
|
42
|
+
Keycloak client and its broker grants (see below; tracked in klok/application/frontend-klok-dev#41 and #42). Name
|
|
43
|
+
those human steps up front instead of letting them surface
|
|
44
|
+
as a red pipeline. Vault CI login is not one of them: it needs nothing per project.
|
|
40
45
|
|
|
41
46
|
## Setting up a new project (what must exist)
|
|
42
47
|
|
|
43
48
|
* Nothing in Vault per project. Every job logs in with the tenant CI role `<tenant>-ci` (a Vault JWT role bound
|
|
44
49
|
to `namespace_path` `<tenant>` and `<tenant>/*`, carrying `default` and the tenant CI read policy
|
|
45
50
|
`<tenant>-cicd-read`). The templates set `VAULT_AUTH_ROLE: $KLOK_TENANT-ci`. A tenant without the role fails
|
|
46
|
-
every job at `auth/jwt/login` with `role "<tenant>-ci" could not be found`; the tenant setup is
|
|
47
|
-
`klok
|
|
48
|
-
that uses it in `klok-docs/ci/authentication.md`
|
|
51
|
+
every job at `auth/jwt/login` with `role "<tenant>-ci" could not be found`; the tenant setup is the
|
|
52
|
+
*Onboarding a tenant* runbook in `klok docs read klok-framework-development-guide`, every credential and the step
|
|
53
|
+
that uses it in `klok-docs/ci/authentication.md` of `klok/environment/build-ci-templates`.
|
|
49
54
|
* For anything the pipeline resolves from a package registry (framework, plugin, datamodels): the project's
|
|
50
55
|
top-level group must be on the package project's job token allowlist.
|
|
51
|
-
* For a deployed service: a manifest
|
|
52
|
-
Kubernetes role `<name>-role` with
|
|
53
|
-
|
|
56
|
+
* For a deployed service: a manifest `<project path>/<name>-crds.yaml` in the manifests repository, written by
|
|
57
|
+
hand before the first deploy, with the Kubernetes service account; a Vault Kubernetes role `<name>-role` (with
|
|
58
|
+
`audience=https://kubernetes.default.svc`) and its policy `<name>-policy` reading
|
|
59
|
+
`secret/data/<tenant>/integration/<name>/env`; the env secret itself (broker credentials, service specific
|
|
60
|
+
keys); and the service's own Keycloak client with its broker grants.
|
|
54
61
|
* Documentation projects need the tenant's docs upload client: Keycloak client `klok-docs-upload-<tenant>` and its
|
|
55
62
|
id and secret in `secret/<tenant>/common/repo/cicd-environment-variables`.
|
|
56
63
|
|
|
57
|
-
|
|
58
|
-
field, and let
|
|
64
|
+
Agents never write to Vault: propose the exact `vault write` / `vault policy write` command with every existing
|
|
65
|
+
field, and let a person run it.
|
|
59
66
|
|
|
60
67
|
## Common failures and their meaning
|
|
61
68
|
|
|
@@ -1,12 +1,15 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: klok-route
|
|
3
|
-
description: Write or change a Camel route on the Klok Quarkus extension 2.0 (onramp, processing, offramp, REST API handler, async task worker, custom route) with the correct naming, configuration, error handling, tests and klok-docs. Use for any change inside a Klok integration project.
|
|
3
|
+
description: Write or change a Camel route on the Klok Quarkus extension 2.0 (onramp, processing, offramp, REST API handler, async task worker, custom route) with the correct naming, topics and topic purpose, configuration, error handling, tests and klok-docs. Use for any change inside a Klok integration project.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Writing a Klok route
|
|
7
7
|
|
|
8
|
-
Read
|
|
9
|
-
`config`, `error-handling`, `testing`); read the `klok-docs` skill for other pages.
|
|
8
|
+
Read `klok docs read klok-framework-klok-quarkus-extension` first (pages `patterns`, `naming`,
|
|
9
|
+
`config`, `error-handling`, `testing`); read the `klok-docs` skill for other pages. A route is a leg of a solution:
|
|
10
|
+
the `klok-solution` skill finds and updates its solution document, before and after the change. Integration logic is
|
|
11
|
+
a Java route; doing it with the Python or TypeScript broker clients is a `[TACTICAL]` choice the solution document
|
|
12
|
+
motivates.
|
|
10
13
|
|
|
11
14
|
## Steps
|
|
12
15
|
|
|
@@ -17,24 +20,83 @@ Read https://docs.klok.dev/klok-framework-klok-quarkus-extension/llm.txt first (
|
|
|
17
20
|
2. **Name it.** `<TENANTSHORT><NNN><LEG><seq>_<Description>`, one class per route, the solution number from the
|
|
18
21
|
portal registry. Sub routes are `S`, timers `T`.
|
|
19
22
|
3. **Bind the datamodels** as type parameters; take them from the catalog bundle, never hand written DTOs.
|
|
20
|
-
4. **
|
|
23
|
+
4. **Choose the topic purpose** of every topic the route publishes or consumes (next section).
|
|
24
|
+
5. **Configure** through `routeConfig("key", default)` / `requiredRouteConfig("key")` and document every key in
|
|
21
25
|
`application.properties` as `<RouteId>.<key>`. Reserved keys: `enabled`, `logevents.*`, `retry.*`, `dlq.*`,
|
|
22
26
|
`topic.*`, `subscription.*`, `claimcheck.*`.
|
|
23
|
-
|
|
27
|
+
6. **Errors.** Throw `Klok4XXException` for caller errors, let retryable failures propagate; the base route logs,
|
|
24
28
|
retries with backoff, republishes to `<topic>_DLQ` and emits the failure CloudEvent. Override `failureHandled()`
|
|
25
29
|
only when the route owns its failures (custom routes that answer HTTP themselves).
|
|
26
|
-
|
|
30
|
+
A sub route has no error handling of its own: it catches only what is not an error, may throw an error again
|
|
31
|
+
as a more specific Klok exception, and leaves retries, the dead letter queue and the reply to its caller, whose
|
|
32
|
+
keys you configure (`klok docs read klok-framework-klok-quarkus-extension`, *Error handling*, *Sub routes*).
|
|
33
|
+
7. **Capabilities** as interfaces: `SplitIncomingMessage`, `AggregateOutgoingMessages`, `ClaimCheck`,
|
|
27
34
|
`KlokEmitCloudEventOnSuccess/Failure`, `PersistOutgoingMessages`, `KlokAsyncTaskInitiator`.
|
|
28
|
-
|
|
29
|
-
send a message, assert the published topic, body and `klok-*` properties.
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
35
|
+
8. **Test** with `@QuarkusTest` and the recording broker client factory: assert the route status is `Started`,
|
|
36
|
+
send a message, assert the published topic (the full name, purpose included), body and `klok-*` properties.
|
|
37
|
+
9. **Document** in `klok-docs/`: the route in `routes.md` with the purpose of each topic and the project on its other
|
|
38
|
+
end, decisions in `design-decisions.md`, operations notes.
|
|
39
|
+
Annotate central classes with `@KlokDocs` (`dev.klok.framework.common.docs.KlokDocs`, klok-common): configuration
|
|
40
|
+
or standards several routes read, shared mappers, domain logic used by several routes. Each gets its own generated
|
|
41
|
+
page; write its Javadoc for a reader who has not seen the code. Not every class: the classes a route calls
|
|
42
|
+
directly reach its generated documentation without the annotation.
|
|
43
|
+
10. Update the solution document when the route is new or its contract changed (`klok-solution` skill).
|
|
44
|
+
11. Build (`./gradlew build`) in the task worktree (`klok-worktree` skill), push its `agent/<task-id>` branch,
|
|
45
|
+
open a merge request ready for review, check the pipeline.
|
|
46
|
+
|
|
47
|
+
## Topic purpose
|
|
48
|
+
|
|
49
|
+
A topic is named after its datamodel: `<Model>[_<purpose>]`. The purpose (called *producer intention* in older code)
|
|
50
|
+
is an optional suffix that belongs to the publisher: the solution requires it, the producer sets it, and the consumer
|
|
51
|
+
sets the same purpose on its own route, so one general datamodel serves several flows:
|
|
52
|
+
`CallAiResponseV1_documentation` and `CallAiResponseV1_invoicecoding` both carry the typed `CallAiResponseV1`, and
|
|
53
|
+
each consumer receives only its own flow. The framework builds the full topic name on both sides from the model and
|
|
54
|
+
the purpose.
|
|
55
|
+
|
|
56
|
+
* **When.** Give a flow a purpose when its datamodel is, or may be, used by another flow. Without one, every consumer
|
|
57
|
+
of the model receives every message. No purpose only when every consumer wants every message.
|
|
58
|
+
* **Ingress** (onramps, webhook onramp): decide which flow the data belongs to and publish with that purpose.
|
|
59
|
+
**Egress** (offramps): consume the purpose the producer publishes on; find it in the producer's `klok-docs/` or its
|
|
60
|
+
route configuration, and ask when it is not written down. Never guess a purpose.
|
|
61
|
+
* **Where it is set.** Configuration wins over code.
|
|
62
|
+
|
|
63
|
+
| Pattern | Code | Configuration |
|
|
64
|
+
|---|---|---|
|
|
65
|
+
| onramps, `KlokWebhookOnrampRoute`, offramps, channel adapters, `Sync` variants | `topicPurpose()` | `<RouteId>.topic.purpose` |
|
|
66
|
+
| `KlokOfframpRouteWithResult` | `topicPurpose()`, `resultTopicPurpose()` (default: the request's) | `.topic.purpose`, `.topic.result-purpose` |
|
|
67
|
+
| `KlokProcessingRoute` | `incomingTopicPurpose()`, `outgoingTopicPurpose()` | `.topic.purpose`, `.topic.outgoing-purpose` |
|
|
68
|
+
| `KlokAsyncTaskWorkerRoute` | `taskType()`: the task type is the request topic's purpose | |
|
|
69
|
+
| REST handler with `task.initiate` | the `klok-task-type` header in `transformBody` | `.task.type` |
|
|
70
|
+
|
|
71
|
+
Put it in code when the purpose is part of what the route is; put it in configuration when the same route serves
|
|
72
|
+
several flows. Set `<SolutionId>.topic.purpose` only when every route of the solution is on that flow.
|
|
73
|
+
* **Name** the flow as both sides understand it (`created`, `reminder`, `documentation`): letters, digits, `-` and
|
|
74
|
+
`.`, never `_`, and no version or environment. Task types are `<domain>.<verb>-<object>`.
|
|
75
|
+
* **One purpose, no reader.** A topic has at most one purpose. A reader, also one in another tenant, is identified by
|
|
76
|
+
its subscription, whose name is its route id (`DEG012E1_...`; `<RouteId>.subscription.name` overrides it), and never
|
|
77
|
+
adds a suffix to the topic. When the content belongs to one reader, the publisher may choose that reader's tenant
|
|
78
|
+
slug as the purpose (`PersonnelScheduleExportV1_degerfors`); that is the publisher's choice, not something a reader
|
|
79
|
+
in another tenant needs.
|
|
80
|
+
* **A mismatch fails silently**: the two routes end up on different topics and nothing arrives. Check the start log
|
|
81
|
+
(`Route <RouteId> subscribes to <topic>`, `Route <RouteId> publishes <Model> to <topic>`) and assert the full topic
|
|
82
|
+
in the test.
|
|
83
|
+
* **Changing a purpose** moves the flow to a new topic: move the consumers first, then the producers, and let the old
|
|
84
|
+
topic drain.
|
|
85
|
+
* **Limit.** A route has one purpose, fixed when it starts. A shared service that answers each caller on the purpose
|
|
86
|
+
the caller chose is not supported yet (klok/framework/klok-quarkus-extension#32); do not write a subclass per
|
|
87
|
+
caller in the shared service, say so in the merge request instead.
|
|
88
|
+
|
|
89
|
+
The user guide page *Topic Purpose: One Datamodel, Several Flows* (`klok docs read klok-user-guide`)
|
|
90
|
+
explains the reasons and the naming in full.
|
|
33
91
|
|
|
34
92
|
## Things that are always wrong
|
|
35
93
|
|
|
36
94
|
* A route id that does not start with the solution id, or two routes with the same id.
|
|
37
95
|
* Publishing to a hand written topic string; use the resolver and the datamodel.
|
|
96
|
+
* Consuming a shared topic and discarding the messages of other flows (a filter on a payload field, an ODL lookup
|
|
97
|
+
per message) where the producer could publish the flow with a topic purpose.
|
|
98
|
+
* A purpose with `_`, a version or an environment in it, a second suffix after the purpose, a tenant in it that the
|
|
99
|
+
publisher did not choose, or a purpose guessed instead of agreed with the other side.
|
|
38
100
|
* Choosing a tenant, index, repository or account from a field in the message; the topic decides the tenant.
|
|
39
101
|
* Sharing broker credentials between services, or granting a whole namespace of another domain or tenant.
|
|
40
102
|
* Publishing into another tenant's namespace, with `<RouteId>.topic.tenant-override` or otherwise; a route consumes
|