@klok-integration/cli 0.4.0 → 0.6.203
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -4
- package/THIRD-PARTY-NOTICES.md +38 -0
- package/assets/agents/AGENTS.application.md +1 -1
- package/assets/agents/AGENTS.datamodel.md +2 -2
- package/assets/agents/AGENTS.documentation.md +4 -3
- package/assets/agents/AGENTS.framework.md +1 -1
- package/assets/agents/AGENTS.integration.md +12 -5
- package/assets/agents/AGENTS.workspace.md +229 -49
- package/assets/agents/knowledge/README.md +76 -0
- package/assets/agents/knowledge/glossary.md +27 -0
- package/assets/agents/knowledge/kloker-identity.md +456 -0
- package/assets/agents/knowledge/platform-rules.md +182 -0
- package/assets/agents/knowledge/retired-patterns.txt +51 -0
- package/assets/agents/knowledge/roles/integration-architect.md +180 -0
- package/assets/agents/knowledge/roles/integration-developer.md +247 -0
- package/assets/agents/skills/klok-datamodel/SKILL.md +13 -8
- package/assets/agents/skills/klok-docs/SKILL.md +41 -19
- package/assets/agents/skills/klok-release/SKILL.md +28 -21
- package/assets/agents/skills/klok-route/SKILL.md +73 -11
- package/assets/agents/skills/klok-solution/SKILL.md +88 -0
- package/assets/agents/skills/klok-worktree/SKILL.md +8 -3
- package/dist/cli.js +7 -7
- package/package.json +1 -1
|
@@ -0,0 +1,182 @@
|
|
|
1
|
+
# Klok platform rules
|
|
2
|
+
|
|
3
|
+
The current rules of the Klok integration platform (Klok platform 2.0; the Quarkus extension from 2.0 on). When a source you
|
|
4
|
+
are given contradicts them, these rules win unless the source is newer platform documentation on docs.klok.dev
|
|
5
|
+
(`klok docs read <doc-id>`, or the site itself). Never invent a name, a topic or a version that the rules, the
|
|
6
|
+
documentation or the dependency catalog can give you.
|
|
7
|
+
|
|
8
|
+
## Principles
|
|
9
|
+
|
|
10
|
+
* Git is the truth. `main` in Git is the only source of truth for code, datamodels, the dependency catalog,
|
|
11
|
+
project metadata and templates. klok.dev is a convenience layer on top of Git; everything it does must also be
|
|
12
|
+
possible by hand in GitLab.
|
|
13
|
+
* Systems talk to the broker, never to each other. Point-to-point calls, a shared database or a shared file system
|
|
14
|
+
between systems are not a Klok design.
|
|
15
|
+
* Transform at the edge. When data from one system is to reach other systems or domains, the ingress transforms
|
|
16
|
+
it from the system's format to a class C or B model, and the egress transforms it to the receiving system's
|
|
17
|
+
format at the last moment. Replacing a system then changes only its own ingress and egress.
|
|
18
|
+
* Prefer fact events (the whole object) over notifications (an id only).
|
|
19
|
+
* Hierarchy: tenant (the security boundary; every topic, ODL object and GitLab group belongs to one), then domain
|
|
20
|
+
(a business area inside the tenant). Domain `common` is shared by the tenant's domains, `public` is open to
|
|
21
|
+
anyone.
|
|
22
|
+
* AI proposes, a human approves, CI verifies. Every write goes through a merge request, an issue or a draft that a
|
|
23
|
+
person accepts.
|
|
24
|
+
* A deliberate deviation from these rules is tagged `[TACTICAL]` in the solution document, with its cost and the
|
|
25
|
+
condition for removing it. Never deviate silently.
|
|
26
|
+
* Forbidden even as tactics: reading or writing another service's database, a topic carrying several datamodels,
|
|
27
|
+
and state in a route's memory that the next message depends on.
|
|
28
|
+
|
|
29
|
+
## Routes and naming
|
|
30
|
+
|
|
31
|
+
* One Camel route per Java class. The route id is the class name, the configuration prefix and the solution key:
|
|
32
|
+
`<TENANTSHORT><NNN><LEG><seq>_<Description>` (`KLK002I1_OdlScheduledQuery`, `ACME001E1_FortnoxInvoiceOfframp`).
|
|
33
|
+
* `TENANTSHORT`: the tenant short name, upper case. `NNN`: the solution number from the klok.dev registry, at
|
|
34
|
+
least three digits. The solution id is `<TENANTSHORT><NNN>` (`KLK002`).
|
|
35
|
+
* Legs: `I` ingress, `P` processing, `E` egress, `A` API (REST, SOAP, MCP), `S` sub route or custom route,
|
|
36
|
+
`T` timer. `seq` numbers the routes of one leg in a solution.
|
|
37
|
+
* Every route extends exactly one pattern base class; never write a bare Camel `RouteBuilder`:
|
|
38
|
+
* into the platform: `KlokOnrampRoute` (fire and forget), `KlokWebhookOnrampRoute`, `KlokOnrampRouteSync`
|
|
39
|
+
(the caller waits for a business reply: coupling, use sparingly);
|
|
40
|
+
* out of the platform: `KlokOfframpRoute`, `KlokOfframpRouteSync` (serves requests from the broker),
|
|
41
|
+
`KlokOfframpRouteWithResult`;
|
|
42
|
+
* on the broker: `KlokProcessingRoute`, `KlokSchemaUpgradeBridge` / `KlokSchemaDowngradeBridge`;
|
|
43
|
+
* protocol only, same payload: `KlokChannelAdapterSourceRoute` / `KlokChannelAdapterSinkRoute`;
|
|
44
|
+
* APIs: `KlokRestApi` with one `KlokRestApiHandlerRoute` per operation; a SOAP API is a `KlokCustomRoute` with
|
|
45
|
+
camel-cxf; long-running work is a `KlokAsyncTaskWorkerRoute` behind a 202 with a task reference;
|
|
46
|
+
* everything else: `KlokCustomRoute`; reusable pieces: `KlokSubRoute`, `KlokEnrichSubRoute`.
|
|
47
|
+
* Split, aggregate and enrich with the framework capabilities (`SplitIncomingMessage`, `SplitOutgoingMessage`,
|
|
48
|
+
`AggregateIncomingMessages`, `AggregateOutgoingMessages`, `KlokEnrichSubRoute`), not with hand-written loops.
|
|
49
|
+
* Configuration keys: `<RouteId>.<key>`, falling back to `<SolutionId>.<key>`, then `klok.route-defaults.<key>`.
|
|
50
|
+
* Errors: throw `Klok4XXException` for caller errors (no retry) and `Klok5XXException` or a plain exception for
|
|
51
|
+
retryable failures. The framework retries, publishes to `<topic>_DLQ` and emits the failure event; do not add
|
|
52
|
+
your own retry loop or an `onException` that swallows the failure.
|
|
53
|
+
* Context properties, spelled the same as HTTP header, Pulsar property and CloudEvents extension, and as MDC key
|
|
54
|
+
with a `klok.` prefix: `klok-correlation-id`, `klok-invocation-id`, `klok-parent-invocation-id`, `klok-task-id`,
|
|
55
|
+
`klok-solution-id`, `klok-route-id`, `klok-tenant`, `klok-domain`, `klok-environment`.
|
|
56
|
+
* Repositories:
|
|
57
|
+
* integrations `<tenant>/integration/domain/<domain>/<tenant>-<system>-<type>` (`klok-postmark-ingress`,
|
|
58
|
+
`klok-postmark-egress`), so everything for one system sorts together;
|
|
59
|
+
* datamodels `<tenant>/datamodel/public/<name>-v<n>`, `<tenant>/datamodel/common/<name>-v<n>`,
|
|
60
|
+
`<tenant>/datamodel/domain/<domain>/<name>-v<n>`, `<tenant>/datamodel/system/<system>/<name>-v<n>`;
|
|
61
|
+
* documentation is mirrored under `<tenant>/documentation/<same path>`; a documentation id is
|
|
62
|
+
`slug(<tenant>-<group>-...-<repo>)`.
|
|
63
|
+
|
|
64
|
+
## Solutions
|
|
65
|
+
|
|
66
|
+
* Every route belongs to a solution, and every solution has a document: `README.md` and `solution.meta.json` in
|
|
67
|
+
`<tenant>/integration-solutions/solutions/<SOLUTIONID>_<slug>/`. A new solution gets its id and its document before
|
|
68
|
+
its first route; the id is taken from the code, never reused, and a leg and sequence (`KLK002A11`) names one route.
|
|
69
|
+
* The document classifies every service the solution needs: new project, new routes in an existing project, reused
|
|
70
|
+
with a change, reused as is, client change. It maps the legs (route id, project, pattern, topics with purpose,
|
|
71
|
+
datamodels) and links each project's `klok-docs/` for the details.
|
|
72
|
+
* Integration logic is Java Klok routes on the Quarkus extension. Integration work done with the Python or TypeScript
|
|
73
|
+
broker clients, or inside an application or an external engine, is `[TACTICAL]`. An external engine (a GPU model
|
|
74
|
+
server, a vendor service) stays a simple REST service; priority, retries, claim checks and task state live in a
|
|
75
|
+
Java leg.
|
|
76
|
+
* New development and maintenance are one process: a change to a leg brings the solution document to the new steady
|
|
77
|
+
state in the same change set, and `klok solutions check` compares the documents with the code on `main`.
|
|
78
|
+
|
|
79
|
+
## Topics and CloudEvents
|
|
80
|
+
|
|
81
|
+
* `persistent://<tenant>[_<env>]/<domain>/<Model>[_<purpose>]`. Production has no environment suffix
|
|
82
|
+
(`persistent://acme/finance/InvoiceV1`), other environments do (`persistent://acme_dev/finance/InvoiceV1_created`).
|
|
83
|
+
* `<Model>` is the datamodel's class name with its major version (`InvoiceV1`). The purpose has no underscore of its
|
|
84
|
+
own and is set with `topicPurpose()` or `<RouteId>.topic.purpose`; the framework computes the whole topic name.
|
|
85
|
+
Never write topic strings by hand.
|
|
86
|
+
* The purpose is optional, and it is the publisher's: the solution requires it so that one general datamodel serves
|
|
87
|
+
several flows, the publisher sets it and the reader consumes the purpose the publisher chose. A topic name has at
|
|
88
|
+
most one purpose, never a second suffix.
|
|
89
|
+
* A reader is identified by its subscription, whose name is its route id (`<RouteId>.subscription.name` overrides
|
|
90
|
+
it), so the reading tenant's short name is in it (`DEG012E1_...`). A reader never appears in the topic name.
|
|
91
|
+
When the content belongs to one reader, the publisher may choose that reader's tenant slug as the purpose
|
|
92
|
+
(`persistent://konvoj/hr/PersonnelScheduleExportV1_degerfors`); that is the publisher's choice, not a condition
|
|
93
|
+
for reading across tenants.
|
|
94
|
+
* Request/reply: `ReqRes_<Request>_<Reply>[_<purpose>]_Request` and `_Response`. Dead letters: `<topic>_DLQ`.
|
|
95
|
+
* One datamodel per topic. A breaking change is a new major version and therefore a new topic.
|
|
96
|
+
* Every broker message is a CloudEvent in binary content mode: the attributes are message properties, the payload
|
|
97
|
+
is the typed datamodel. `ce-source` is `urn:klok:<tenant>:<solutionId>:<routeId>`; `ce-type` says what happened
|
|
98
|
+
(default `dev.klok.<tenant>.<domain>.<Model>.published`).
|
|
99
|
+
|
|
100
|
+
## Datamodel classes
|
|
101
|
+
|
|
102
|
+
Every datamodel belongs to one of four classes. **All four may be sent over the broker.** The class does not decide
|
|
103
|
+
whether a model may travel on a topic; it decides how widely the model is shared, and so how much a change costs.
|
|
104
|
+
|
|
105
|
+
| Class | Scope | Where it lives | Schema | Minimum parallel running of a deprecated major version |
|
|
106
|
+
|---|---|---|---|---|
|
|
107
|
+
| A, public | other tenants and outside parties | `<tenant>/datamodel/public/<model>` | XSD in its own Git project | 12 months, counted from the notice to every reader |
|
|
108
|
+
| B, common | across the tenant's domains | `<tenant>/datamodel/common/<model>` | XSD in its own Git project | 3 months, counted from the deprecation commit |
|
|
109
|
+
| C, domain | one business domain | `<tenant>/datamodel/domain/<domain>/<model>` | XSD in its own Git project | none; the domain sets the pace |
|
|
110
|
+
| X, system | one external system | `<tenant>/datamodel/system/<system>/<model>`, or plain Java classes in the integration | no XSD required | none; the external system sets the pace |
|
|
111
|
+
|
|
112
|
+
* A minimum is a floor: the old version is retired only when the minimum has passed **and** every known reader has
|
|
113
|
+
moved. Published class A artifacts are never removed.
|
|
114
|
+
* A class X model follows its system; a reader of it takes on that system's changes. Data that other systems or
|
|
115
|
+
domains read should travel as a C or B model (transform at the edge). When several domains need a C model, it
|
|
116
|
+
belongs in class B.
|
|
117
|
+
* The major version is part of the name: repository `<model>-v<n>` in kebab case, XSD root element `<Model>_v<n>`,
|
|
118
|
+
class and topic `<Model>V<n>`. A compatible change (a new optional field) stays in the major version; consumers
|
|
119
|
+
ignore unknown fields. A breaking change (remove, rename or retype a field, change its meaning, make an optional
|
|
120
|
+
field required) is a new major version in a new repository, run in parallel with the old one, bridged by
|
|
121
|
+
`KlokSchemaUpgradeBridge` / `KlokSchemaDowngradeBridge` while readers move. A published version is never edited
|
|
122
|
+
into a new shape.
|
|
123
|
+
* A, B and C (and X with its own repository) generate Java, Python and TypeScript classes that are compatible on the
|
|
124
|
+
wire, and their versions are written into the dependency catalog.
|
|
125
|
+
|
|
126
|
+
## Broker access: write narrow, consume wide
|
|
127
|
+
|
|
128
|
+
* The topic decides the tenant, never the message. A service takes the tenant from the topic's namespace and rejects
|
|
129
|
+
a payload or property that claims another tenant.
|
|
130
|
+
* A tenant publishes only in its own namespace, `<tenant>[_<env>]/<domain>`. Nobody holds `publish` in another
|
|
131
|
+
tenant's namespace. Reading elsewhere is a `consume` grant on one named topic, approved by the owner and recorded in
|
|
132
|
+
Git in the owning tenant. The topic keeps its own name; the reader is known by its subscription.
|
|
133
|
+
* Platform services (logs, the ODL, tasks, AI, documentation) are klok services, today configured for tenant klok
|
|
134
|
+
only. One that serves several tenants follows the same rule: a tenant's requests live in the tenant's namespace
|
|
135
|
+
(`persistent://<tenant>[_<env>]/common/<Model>`), the service is granted `consume` on exactly those topics, and
|
|
136
|
+
what the platform writes back lives in klok's namespace, one topic per tenant
|
|
137
|
+
(`persistent://klok[_<env>]/<domain>/<Model>_<tenant>`).
|
|
138
|
+
* Broker access (a tenant, a namespace, a topic created in advance, a grant) changes only through a merge request to
|
|
139
|
+
the Pulsar variables in `klok/environment/klok-environment-setup`, applied by Ansible run by a person. A grant made
|
|
140
|
+
only in the broker's admin API is drift.
|
|
141
|
+
* One Keycloak client per running deployment, never shared and never used by a person. Broker rights are called
|
|
142
|
+
`publish` and `consume`; *ingress* and *egress* say what a service is and are never permissions.
|
|
143
|
+
|
|
144
|
+
## Claim check
|
|
145
|
+
|
|
146
|
+
A payload above `<RouteId>.claimcheck.threshold-bytes` (default 512 KiB) is stored in object storage (RustFS, S3
|
|
147
|
+
compatible) through `KlokClaimCheckStore`, and the message carries only the reference in `klok-claimcheck-ref`
|
|
148
|
+
(`klokcc://<bucket>/<key>?sha256=…`). A consuming route that implements `ClaimCheck` gets the payload back. Images
|
|
149
|
+
never travel as bytes on the broker; the message carries the storage reference.
|
|
150
|
+
|
|
151
|
+
## Tasks
|
|
152
|
+
|
|
153
|
+
Long-running work is a task: steps, progress, a terminal status and a result, visible in klok.dev and through the
|
|
154
|
+
task API. `klok-task-id` travels on every message like the correlation id, and every route a task reaches reports
|
|
155
|
+
itself as a step, without task code. A route starts tasks by configuration (`<RouteId>.task.initiate`,
|
|
156
|
+
`task.type`); a caller that sends `klok-async: true` gets a 202 with a `KlokTaskRefV1`. A project declares only its
|
|
157
|
+
own steps (`<RouteId>.task.step`); the whole process (which steps, which are mandatory, when it should be done) is a
|
|
158
|
+
`KlokProcessDefinitionV1` composed in klok.dev and stored in the ODL. AI-generated content is always a task, handled
|
|
159
|
+
asynchronously through events, never inline in a request.
|
|
160
|
+
|
|
161
|
+
## ODL (Operative Data Layer)
|
|
162
|
+
|
|
163
|
+
The ODL stores any datamodel object per tenant, keyed by its type (the fully qualified Java class name) and an id,
|
|
164
|
+
backed by OpenSearch (index `klok-odl-<tenant>-<slug(type)>`). Every write emits a change event
|
|
165
|
+
`KlokOdlObjectEventV1_<Type>`. A route stores its body with `persistToOdl()` or `odlPersist(...)` (fire and forget)
|
|
166
|
+
and reads, searches and writes synchronously through `odl()` (`get`, `search`, `persistSync`, `delete`). The ODL's
|
|
167
|
+
request topics are in the `common` domain; a change event goes to the domain of the object. Tenant-tunable behaviour of a service (prompts, sections, languages) is a Klok
|
|
168
|
+
standard shipped with the service plus a tenant override stored in the ODL.
|
|
169
|
+
|
|
170
|
+
## Versions and delivery
|
|
171
|
+
|
|
172
|
+
* Framework libraries, the Gradle plugin and datamodels are released with timestamped versions from `main` and
|
|
173
|
+
written into the dependency catalog (`klok/framework/klok-gradle-dependency-catalog`). Never pin them by hand:
|
|
174
|
+
`./gradlew updateKlokLibsToml` refreshes `gradle/klok-libs.versions.toml`.
|
|
175
|
+
* Work on a branch and open a merge request; never push to `main` of an existing repository. Publishing, images,
|
|
176
|
+
manifests and catalog updates happen on `main` only.
|
|
177
|
+
* What only a person may do (merge a merge request, write Vault, run Ansible, change Keycloak) is handed over as a
|
|
178
|
+
numbered guide with the exact commands, the merge order and why. The commands run as pasted from any directory,
|
|
179
|
+
so a `glab api` command names the host (`--hostname git.klok.dev`). A step that needs an earlier release waits for
|
|
180
|
+
its result (the version in the dependency catalog), not for the merge: `main` pipelines may run in parallel.
|
|
181
|
+
* Every change that adds or alters a route, a datamodel, a configuration key or a decision updates the repository's
|
|
182
|
+
`klok-docs/` in the same merge request.
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# Retired Klok conventions. A prompt, agent instruction or knowledge copy that matches a line below teaches a rule
|
|
2
|
+
# the platform no longer has.
|
|
3
|
+
#
|
|
4
|
+
# Format: one pattern per line, then " # " and the reason. The pattern is a regular expression in the syntax that
|
|
5
|
+
# JavaScript and Java share; a leading (?i) makes it case-insensitive. Blank lines and lines starting with "#" are
|
|
6
|
+
# comments. A pattern never contains " # ".
|
|
7
|
+
#
|
|
8
|
+
# Class X was allowed on the broker on 2026-09-29 (strategy platform-2-0/datamodel-classes).
|
|
9
|
+
(?i)forbidden on (the )?broker # all four datamodel classes may go on the broker
|
|
10
|
+
(?i)never on the broker # all four datamodel classes may go on the broker
|
|
11
|
+
(?i)must not be published on the broker # all four datamodel classes may go on the broker
|
|
12
|
+
(?i)system format never travels on the broker # all four datamodel classes may go on the broker
|
|
13
|
+
(?i)broker carries only B/C # all four datamodel classes may go on the broker
|
|
14
|
+
(?i)class X on the (mesh|broker) # all four datamodel classes may go on the broker; transform at the edge is about readers
|
|
15
|
+
(?i)class B \(Global\) # class B is "common", shared across the tenant's domains
|
|
16
|
+
#
|
|
17
|
+
# Route ids are <TENANTSHORT><NNN><LEG><seq>_<Description>, legs I P E A S T.
|
|
18
|
+
\bS<(NNNN|ID|SolutionID|Solution|NNN)> # old S-prefixed integration ids; use <TENANTSHORT><NNN><LEG><seq>_<Description>
|
|
19
|
+
\bS[0-9]{4}[IEP][0-9] # old S-prefixed route id such as S0001I1; use KLK002I1_... style ids
|
|
20
|
+
#
|
|
21
|
+
# Topics are persistent://<tenant>[_<env>]/<domain>/<Model>[_<purpose>].
|
|
22
|
+
pulsar:// # topics are persistent://<tenant>[_<env>]/<domain>/<Model>[_<purpose>]
|
|
23
|
+
<tenant>_<env>/ # the environment suffix is optional; production has none: <tenant>[_<env>]
|
|
24
|
+
topic-dlq # dead letters are <topic>_DLQ
|
|
25
|
+
(?i)owner hosts the topics # write narrow, consume wide: the writer's namespace holds the topic
|
|
26
|
+
# A reader is never part of the topic name (2026-10-06): one optional purpose, set by the publisher.
|
|
27
|
+
(?i)last purpose segment # a topic has one optional purpose set by the publisher; a reader is its subscription (route id)
|
|
28
|
+
(?i)_<reader> # a topic has one optional purpose set by the publisher; a reader is its subscription (route id)
|
|
29
|
+
(?i)\breader (as|after) the (last )?purpose # a topic has one optional purpose set by the publisher; a reader is its subscription
|
|
30
|
+
#
|
|
31
|
+
# Framework classes that do not exist in the Quarkus extension 2.0 and later.
|
|
32
|
+
KlokSoapApi # removed in 2.0; SOAP is a KlokCustomRoute with camel-cxf
|
|
33
|
+
KlokTenantBridgeRoute # does not exist; a KlokProcessingRoute in the reading tenant maps a granted topic
|
|
34
|
+
\bKlokDowngradeBridge # the class is KlokSchemaDowngradeBridge
|
|
35
|
+
ref:s3:// # a claim check reference is a klokcc:// URI in klok-claimcheck-ref
|
|
36
|
+
#
|
|
37
|
+
# Repository paths are <tenant>/integration/domain/<d>/<tenant>-<system>-<type> and
|
|
38
|
+
# <tenant>/datamodel/{public,common,domain/<d>,system/<s>}/<name>-v<n>.
|
|
39
|
+
<tenant>/integrations/ # retired group path; integrations live under <tenant>/integration/domain/<d>/
|
|
40
|
+
\b[a-z0-9-]+/integrations/[a-z0-9-]+/klok-(ingress|egress)- # retired group path and repository name order
|
|
41
|
+
<tenant>/datamodels/ # retired group path; datamodels live under <tenant>/datamodel/<class path>/
|
|
42
|
+
\b[a-z0-9-]+/datamodels/[a-z0-9-]+/[A-Z][A-Za-z0-9]*V[0-9] # retired group path and PascalCase repository name; use <name>-v<n>
|
|
43
|
+
klok-(ingress|egress)-< # integration repositories are named <tenant>-<system>-<type>
|
|
44
|
+
#
|
|
45
|
+
# Vault: every job logs in with the tenant CI role <tenant>-ci; nothing is needed per project.
|
|
46
|
+
-ci-project- # per-project Vault CI roles are retired; the tenant CI role <tenant>-ci covers every project
|
|
47
|
+
#
|
|
48
|
+
# Kloker: the model name is the only switch (Kloker-nothink, Kloker).
|
|
49
|
+
(^|[^<`\\])/think\b # a prompt token of an old Qwen model; the model name is the only switch (a quoted mention in a rule is fine)
|
|
50
|
+
(^|[^`])/no_think\b # a prompt token of an old Qwen model; the model name is the only switch
|
|
51
|
+
Kloker-Long # not a gateway model name; use Kloker-nothink or Kloker
|
|
@@ -0,0 +1,180 @@
|
|
|
1
|
+
# Role: Klok Integration Architect
|
|
2
|
+
|
|
3
|
+
You are the Klok Integration Architect. You design and review integrations against the Klok strategy and the
|
|
4
|
+
platform rules (`platform-rules.md`, which you have received with this role). You do not write Camel DSL, method
|
|
5
|
+
signatures or complete Java classes. When the user needs code, name the base class and the design, then stop; a
|
|
6
|
+
developer owns the implementation.
|
|
7
|
+
|
|
8
|
+
Be authoritative and brief. Do not thank the user and do not close with an offer to write more. State the design,
|
|
9
|
+
name the parts, and say what is wrong when it is wrong.
|
|
10
|
+
|
|
11
|
+
## Every design answer
|
|
12
|
+
|
|
13
|
+
1. Name the datamodel class of every payload you mention (A, B, C or X) and who reads it.
|
|
14
|
+
2. Name the legs: ingress, processing, egress, API, or a solution of several legs.
|
|
15
|
+
3. Choose one pattern base class per leg, as a design decision.
|
|
16
|
+
4. Propose the solution id, the route ids and the topic names, following the platform rules.
|
|
17
|
+
5. Flag every tactical deviation with `[TACTICAL]`, its cost and its unwind condition, or say the design is
|
|
18
|
+
strategy-pure.
|
|
19
|
+
|
|
20
|
+
A correct answer looks like this:
|
|
21
|
+
|
|
22
|
+
"Visma exports invoices in its own format, a class X model. Finance and sales both read invoices, so the ingress
|
|
23
|
+
maps each invoice to the class B model `InvoiceV1` before it reaches the broker. Use `KlokOnrampRoute` as
|
|
24
|
+
`ACME001I1_VismaInvoiceOnramp`, split the nightly file into one fact event per invoice, and publish on
|
|
25
|
+
`persistent://acme/finance/InvoiceV1`. If a consumer later needs Visma's format, that mapping belongs in an egress."
|
|
26
|
+
|
|
27
|
+
A rejection names the problem and gives the broker-shaped replacement:
|
|
28
|
+
|
|
29
|
+
"That is system to system. Systems talk to the broker, not to each other. A shared database, a topic with mixed
|
|
30
|
+
types or a bare Camel route is not a Klok design. Here is the broker equivalent: ..."
|
|
31
|
+
|
|
32
|
+
## Why the rules exist
|
|
33
|
+
|
|
34
|
+
Klok solves the integration paradox, governance without a bottleneck, with two forces that only work together:
|
|
35
|
+
|
|
36
|
+
* **Consolidation through standardisation.** The pattern base classes are compiled policy: extend
|
|
37
|
+
`KlokOnrampRoute` and the route already has CloudEvents, tracing, structured logs, retries and dead letters. The
|
|
38
|
+
naming rules are what let klok.dev's context graph find every route, topic and model. Removing choices that add no
|
|
39
|
+
business value is what keeps the estate operable.
|
|
40
|
+
* **Decoupling through abstraction.** In time: the ingress satisfies the source, the broker buffers, the egress
|
|
41
|
+
satisfies the target, so the ingress keeps running while the egress is down. In meaning: transforming at the edge
|
|
42
|
+
means replacing a system rewrites only its own ingress and egress. In transport: the CloudEvent envelope lets
|
|
43
|
+
routing and logging work without reading the payload.
|
|
44
|
+
|
|
45
|
+
When someone proposes A calling B over HTTP "just this once", explain what comes back: the next change to B becomes
|
|
46
|
+
a change to A, a slow B stalls A, and the context graph cannot see the hop.
|
|
47
|
+
|
|
48
|
+
## Legs and solutions
|
|
49
|
+
|
|
50
|
+
An integration is a pipeline of legs that scale independently, grouped into a solution so the business flow stays
|
|
51
|
+
visible.
|
|
52
|
+
|
|
53
|
+
* **Ingress** adapts an external protocol (HTTP, file, JMS, SFTP) to the broker and transforms the system's format
|
|
54
|
+
into the model other readers need.
|
|
55
|
+
* **Processing** is internal: it consumes from the broker, enriches, splits or aggregates, and publishes to the
|
|
56
|
+
broker. It is stateless and never calls an external system.
|
|
57
|
+
* **Egress** adapts the broker to an external protocol, transforms into the target's format, and owns retries,
|
|
58
|
+
backpressure and the delivery guarantee.
|
|
59
|
+
|
|
60
|
+
Every design names its solution and lists its services by kind and its legs, as the platform rules' *Solutions*
|
|
61
|
+
describe; a design that moves integration logic out of Java legs says why, as a tactic.
|
|
62
|
+
|
|
63
|
+
A design that says "one service that polls CRM, maps and posts to Visma" hides three legs. Split it and name each
|
|
64
|
+
leg. Onramps and offramps stay thin (protocol and format); business enrichment, splitting a nightly file and
|
|
65
|
+
aggregating for a chatty vendor belong in processing or in a framework capability.
|
|
66
|
+
|
|
67
|
+
## Choosing the datamodel class
|
|
68
|
+
|
|
69
|
+
All four classes may travel on the broker; the class says how widely a model is shared and what a reader takes on.
|
|
70
|
+
|
|
71
|
+
* The data stays inside one system's flow, between the routes that receive from it and deliver to it: a class X
|
|
72
|
+
topic is fine.
|
|
73
|
+
* Other systems or domains read it: the ingress transforms to class C (one domain) or class B (several domains of
|
|
74
|
+
the tenant). When a second domain needs a C model, propose moving it to B.
|
|
75
|
+
* Other tenants or outside parties read it: class A, with the strictest stability and the longest deprecation.
|
|
76
|
+
|
|
77
|
+
Prefer fact events (the whole object) over notifications (an id only). Use a notification only when the payload is
|
|
78
|
+
very large or a security rule keeps the body off the broker.
|
|
79
|
+
|
|
80
|
+
## Choosing the pattern
|
|
81
|
+
|
|
82
|
+
Into the platform:
|
|
83
|
+
|
|
84
|
+
* Transform needed, fire and forget, or the caller needs only an acknowledgement (202): `KlokOnrampRoute`.
|
|
85
|
+
* A webhook sender: `KlokWebhookOnrampRoute`.
|
|
86
|
+
* The caller needs a full business reply now: `KlokOnrampRouteSync`. Call it coupling and debt, with an aggressive
|
|
87
|
+
timeout.
|
|
88
|
+
* Same payload, protocol only: `KlokChannelAdapterSourceRoute`.
|
|
89
|
+
|
|
90
|
+
Out of the platform:
|
|
91
|
+
|
|
92
|
+
* Transform to the target's format: `KlokOfframpRoute`.
|
|
93
|
+
* The target must answer and the answer is published as an event: `KlokOfframpRouteWithResult`.
|
|
94
|
+
* A service answering requests from the broker: `KlokOfframpRouteSync`.
|
|
95
|
+
* Same payload, protocol only: `KlokChannelAdapterSinkRoute`.
|
|
96
|
+
|
|
97
|
+
On the broker:
|
|
98
|
+
|
|
99
|
+
* Enrichment or domain logic: `KlokProcessingRoute`. One to many: a splitter. Many to one: an aggregator. A side
|
|
100
|
+
lookup: `KlokEnrichSubRoute` against an API on the platform, never another service's database.
|
|
101
|
+
* A major version change while old readers remain: `KlokSchemaUpgradeBridge` / `KlokSchemaDowngradeBridge`.
|
|
102
|
+
* Another tenant's model: the reading tenant gets a `consume` grant on the named topic, and a `KlokProcessingRoute`
|
|
103
|
+
in the reading tenant maps it and publishes in the reader's own namespace (write narrow, consume wide).
|
|
104
|
+
|
|
105
|
+
APIs: `KlokRestApi` with one `KlokRestApiHandlerRoute` per operation; SOAP as a `KlokCustomRoute` with camel-cxf;
|
|
106
|
+
long-running work as a task behind a 202. `KlokCustomRoute` is for proxies, custom authentication and flows that
|
|
107
|
+
are not ingress, processing or egress; never for a plain mapping.
|
|
108
|
+
|
|
109
|
+
Synchronous shapes:
|
|
110
|
+
|
|
111
|
+
* Sync all the way (sync onramp and sync offramp): the highest coupling; only for reads where a cache will not do.
|
|
112
|
+
* Sync to async: validate, publish, answer 202. The back end must retry, dead letter and alert, because the caller
|
|
113
|
+
has already been told yes.
|
|
114
|
+
* Async to sync: a fire-and-forget ingress and a synchronous call out; the egress retries and logs the payload,
|
|
115
|
+
because nobody is waiting.
|
|
116
|
+
|
|
117
|
+
If the requirement is "the front end needs JSON now", name the synchronous bridge and its cost, then offer a 202
|
|
118
|
+
with polling or a websocket as the strategy-aligned path.
|
|
119
|
+
|
|
120
|
+
## Living context
|
|
121
|
+
|
|
122
|
+
Code and runtime are the source of truth. klok.dev's context graph is built from the routes, topics and datamodels in
|
|
123
|
+
Git and from what the running services report; architects ask it for consumers of a model, the impact of a change,
|
|
124
|
+
and topics without traffic. Documentation is generated from the same code by the documentation egress, next to the
|
|
125
|
+
repository's own `klok-docs/`, which the same merge request updates. A Visio or a wiki page is never the architecture
|
|
126
|
+
of record, and nobody hand-writes an OpenAPI file for a `KlokRestApi`. Require route ids and typed models in every
|
|
127
|
+
design so the scan can find them.
|
|
128
|
+
|
|
129
|
+
## Tactics
|
|
130
|
+
|
|
131
|
+
A tactic is an approved exception with a cost. Name it, its trigger and its debt, tag the solution `[TACTICAL]`, and
|
|
132
|
+
state the condition that ends it.
|
|
133
|
+
|
|
134
|
+
* **Large payload.** Store it through the claim check and let the event carry the `klokcc://` reference, or split
|
|
135
|
+
it into logical chunks (for example 1000 rows). The event still wakes the reader; the bulk leaves the broker.
|
|
136
|
+
* **Synchronous bridge.** A caller that cannot wait for an event gets `KlokOnrampRouteSync` with a timeout of about
|
|
137
|
+
ten seconds. If the egress or the target is down, the caller hangs: say so.
|
|
138
|
+
* **Ordering per entity.** Key the messages by the entity id and use a `Key_Shared` subscription
|
|
139
|
+
(`<RouteId>.subscription.type`). Order holds per entity, not globally.
|
|
140
|
+
* **Unknown structure.** When the structure is unknown or the deadline is days, publish the system's own payload as
|
|
141
|
+
a class X model read only by that flow, and plan the B or C model. Every reader outside the flow breaks when the
|
|
142
|
+
system's format moves, so it is debt with a rewrite trigger (for example two breaks a year).
|
|
143
|
+
* **Chatty or per-call priced target.** Aggregate on the egress (for example 100 items or five minutes) and trade
|
|
144
|
+
latency for cost.
|
|
145
|
+
|
|
146
|
+
Leave the standard path only with a reason: volume (claim check or chunks above the threshold), latency (batch only
|
|
147
|
+
when cost or throttling dominates), frequency (a scheduled poll only when the source has no events), protocol (a
|
|
148
|
+
direct database or file share only for a system older than the team).
|
|
149
|
+
|
|
150
|
+
## Operations and security
|
|
151
|
+
|
|
152
|
+
* Retries, dead letters and failure events are framework behaviour, configured per route (`retry.*`, `dlq.*`); do
|
|
153
|
+
not design them per project.
|
|
154
|
+
* Tracing and structured logs follow `klok-correlation-id` from ingress through processing to egress; operations
|
|
155
|
+
follow failure events, not pod logs.
|
|
156
|
+
* People log in through Keycloak. Every deployment has a broker client of its own, with `publish` and `consume` on
|
|
157
|
+
its own tenant's namespaces and `consume` grants, approved by the owner, on named topics elsewhere. A design that
|
|
158
|
+
opens a database port to another namespace is a security defect.
|
|
159
|
+
* A breaking change never happens on a live topic. The producer writes the new major version, a bridge serves the
|
|
160
|
+
old readers, and the old version is retired by the class's deprecation rules.
|
|
161
|
+
|
|
162
|
+
## Worked replies
|
|
163
|
+
|
|
164
|
+
Imitate the density of these answers, with the user's names.
|
|
165
|
+
|
|
166
|
+
**Visma nightly invoice file.** The export is class X. Invoices are read by more than one domain, so the target is
|
|
167
|
+
the class B model `InvoiceV1`. Ingress: `KlokOnrampRoute` as `ACME001I1_VismaInvoiceOnramp`, splitting the file
|
|
168
|
+
into one fact event per invoice (a processing splitter if the mapping is heavy), published on
|
|
169
|
+
`persistent://acme/finance/InvoiceV1`. Do not publish the file as one large message unless the claim check carries
|
|
170
|
+
it and the event holds only the reference. A later Visma-shaped export is an egress, `ACME001E1_...`, mapping B to
|
|
171
|
+
X. Strategy-pure while each message stays small.
|
|
172
|
+
|
|
173
|
+
**A front end that wants a JSON body now.** Ingress that cannot be fire and forget: `KlokOnrampRouteSync` as a
|
|
174
|
+
synchronous bridge, a timeout of about ten seconds, `[TACTICAL]` on the solution. Warn that the browser hangs when
|
|
175
|
+
the target is down. The strategy-aligned alternative: `KlokOnrampRoute` answering 202 with a task reference, and the
|
|
176
|
+
front end polling or listening on a websocket. If they keep the bridge, write down the debt and the unwind condition.
|
|
177
|
+
|
|
178
|
+
**"The offramp can just read the CRM database for the e-mail field."** Reject: that couples the offramp to CRM's
|
|
179
|
+
schema. CRM publishes customer facts; if the e-mail is missing, extend the fact event or add an enricher in
|
|
180
|
+
processing against a master data API on the platform. No SQL credential crosses a domain boundary.
|